From patchwork Wed Apr 28 15:11:41 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrey Grodzovsky X-Patchwork-Id: 12229229 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.5 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,MSGID_FROM_MTA_HEADER,SPF_HELO_NONE,SPF_PASS, USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 58AF4C43460 for ; Wed, 28 Apr 2021 15:12:52 +0000 (UTC) Received: from gabe.freedesktop.org (gabe.freedesktop.org [131.252.210.177]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id F2DC661434 for ; Wed, 28 Apr 2021 15:12:51 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org F2DC661434 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=amd.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=dri-devel-bounces@lists.freedesktop.org Received: from gabe.freedesktop.org (localhost [127.0.0.1]) by gabe.freedesktop.org (Postfix) with ESMTP id 83A716E0F2; Wed, 28 Apr 2021 15:12:49 +0000 (UTC) Received: from NAM10-DM6-obe.outbound.protection.outlook.com (mail-dm6nam10on2042.outbound.protection.outlook.com [40.107.93.42]) by gabe.freedesktop.org (Postfix) with ESMTPS id 481ED6E0F5; Wed, 28 Apr 2021 15:12:33 +0000 (UTC) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=Gax63yzIUwXNj82rbF84YS5Q2yE46NM4ZIIZgaFW4J53P02vOUvEQrLtSqkm6vQpyS0T+fsBznjFzKaCk4FKaNyWkEdqixaicqSqFVw8nUJRUi2QVpdryeWF48n2yqGZW6MUpNsEzJi2THLOSfHGJqXrrrSuRpS7WUwvbsO73lT0JbP+KVgVAH5jo5gGOB6GAElP4kFFac3mkYrFSMPGQnwjqwscs0f2cUfvHuQkU51K5/XGOOjgeS0KV8C1PeXKDM3P7Tnf06mhSo9d//HJocMBGd5HVZDSZCytuUtk4UV7t/y/1/45CEoh/SX+pcEqtXeZm67YLoELvNBPDAKgjw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=TlP3Nu2/Ly3UoGYGggWI0PsQaIUgQbOHVqqtihMisbw=; b=j3BllWKmgC1OlrBbzsJFFBA4f/eBz1K8aVVnXWnyoh5bjCjjdiMest2+bA7ubAIzjfOFZZCqS1IcUB7Z4mSty9KMf4hvZYNC/BMcz/NLW82Uk4umCx/WRDCervut5mpB3BzTFxoXWOoeD5mEHehz8BGsDRtzy3Bd5KnToI6SSjw8rD7371IKLw8UIp0BcI+9wP9z2khfRAjpEqFqg+7Ysb2IUaCgNmqh8IPZqQadVVT3w4dPOvyvEWgfyNj8N9biVUUK2JF8UvIWwF61IZ8KXa8fMO4/s9PctVRkJapuPjvXonRg16GKJyfPO7NlZIR0a1dncQAexeT1B9zfek++dQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=amd.com; dmarc=pass action=none header.from=amd.com; dkim=pass header.d=amd.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=TlP3Nu2/Ly3UoGYGggWI0PsQaIUgQbOHVqqtihMisbw=; b=a59HyWdIx1cCYB0nIS8oXzO2GArsUZDvPiO9n6DWXtRxeYUPmowYYGdTs56z7of1VuU9QCI7aJBCLE8Z9N9eqa6pd0pRsCdvWYWnEYM1xtNCm/veXudR9ei9A+zqY75brl2tCL+x10YmuTuldIfbGbwg3wg4wl6ivgxUU4H+hWo= Authentication-Results: lists.freedesktop.org; dkim=none (message not signed) header.d=none; lists.freedesktop.org; dmarc=none action=none header.from=amd.com; Received: from SN6PR12MB4623.namprd12.prod.outlook.com (2603:10b6:805:e9::17) by SN6PR12MB2799.namprd12.prod.outlook.com (2603:10b6:805:77::32) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4065.24; Wed, 28 Apr 2021 15:12:30 +0000 Received: from SN6PR12MB4623.namprd12.prod.outlook.com ([fe80::ad51:8c49:b171:856c]) by SN6PR12MB4623.namprd12.prod.outlook.com ([fe80::ad51:8c49:b171:856c%7]) with mapi id 15.20.4065.026; Wed, 28 Apr 2021 15:12:30 +0000 From: Andrey Grodzovsky To: dri-devel@lists.freedesktop.org, amd-gfx@lists.freedesktop.org, linux-pci@vger.kernel.org, ckoenig.leichtzumerken@gmail.com, daniel.vetter@ffwll.ch, Harry.Wentland@amd.com Subject: [PATCH v5 01/27] drm/ttm: Remap all page faults to per process dummy page. Date: Wed, 28 Apr 2021 11:11:41 -0400 Message-Id: <20210428151207.1212258-2-andrey.grodzovsky@amd.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210428151207.1212258-1-andrey.grodzovsky@amd.com> References: <20210428151207.1212258-1-andrey.grodzovsky@amd.com> X-Originating-IP: [2607:fea8:3edf:49b0:7212:f93a:73b0:8f23] X-ClientProxiedBy: YT1PR01CA0142.CANPRD01.PROD.OUTLOOK.COM (2603:10b6:b01:2f::21) To SN6PR12MB4623.namprd12.prod.outlook.com (2603:10b6:805:e9::17) MIME-Version: 1.0 X-MS-Exchange-MessageSentRepresentingType: 1 Received: from agrodzovsky-All-Series.hitronhub.home (2607:fea8:3edf:49b0:7212:f93a:73b0:8f23) by YT1PR01CA0142.CANPRD01.PROD.OUTLOOK.COM (2603:10b6:b01:2f::21) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4065.25 via Frontend Transport; Wed, 28 Apr 2021 15:12:29 +0000 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 414672ba-0f1c-48ae-4d27-08d90a580add X-MS-TrafficTypeDiagnostic: SN6PR12MB2799: X-MS-Exchange-Transport-Forked: True X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:6108; X-MS-Exchange-SenderADCheck: 1 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: 4ADg1QtPNE3XzaVlhQ924t6MA6L34ecz5daDp7YorUjaSGFtPfT2NjphYv5SW8S+UANI9tqcExVKxuQEGQ0jdTXwlSz2jTStT4DgJed4WWy5pBNlvo/d/2eKZDVPG8EyAQhnx5STp7eratQHCQAkqPZtxEhUITCOG6MkplQNuUwiVmufJGGtPIdiT4uAK77hR9t6paXe1WfBXytgDQYpOBoTRV1Rens+m1kRGWGICt+zK3zqK0ElKSLV2kbvlumluNCkUb4YQrbyFCJqrpl25Ox0SOPIOv9PD6PpSP405OAcRGXiTN032f5EzTQzJlZHnlgKAYOSZSq1/Aj5EofaE5XUuodVzLxHBUdrib9s3XzgRTdd7Dqs/UVgGICEtoTnax5J/oWoMFA3yOB9dyOh6Zc19UdgjQJLryWJA4Pg7dnN5JW37FmQSYmFEthqbOqvdpDnuPRfdjLWkpzClVU7+5R599oFITddt1CcLtbqlHr4w5ovXl8EpDnVkm91BPp/sjqHMhKrP8Ddd4FCsb92KDs5Z1u1Dhf9GXcy9uw0XkefbNwPkOwqx8gKmc5HGayZXg8/2deuNIpch3SKfkMKT2auYh7JAdjkD2OGE2QQUmE= X-Forefront-Antispam-Report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:SN6PR12MB4623.namprd12.prod.outlook.com; PTR:; CAT:NONE; SFS:(4636009)(376002)(39860400002)(366004)(136003)(346002)(396003)(16526019)(8676002)(4326008)(478600001)(66556008)(66476007)(2616005)(66946007)(86362001)(44832011)(186003)(83380400001)(36756003)(8936002)(316002)(6666004)(1076003)(6636002)(6506007)(52116002)(38100700002)(6512007)(5660300002)(2906002)(6486002); DIR:OUT; SFP:1101; X-MS-Exchange-AntiSpam-MessageData: 9eI4x7Io/+A1yXNTjMa/I0SywTfCyRY7tc3XoHBTgeQxChfh42koAwI3hqAICuJz1IqhM3sEWGcsgyiPkQfE4J3hYjhdfR23ZDTPiU81Dha30F1B8UIgQvNpPXeZ7geNfoNkrdDdqAdFsLcioMpEgUCIv74Ug8wpyJSCkjM9wV+LqGz2l/a5kk2T18qzDaNcsrpz5yjD8uGe4lPl+IVxHvdz3Ey15KyNmLnz2rnI/eKiqc0IwGSncpzb6Wadg9BNiggcW2fa14aZA5etkPTFXDoM0EpXTEo82zAB+ZW3kIcAmF+zTdDp/TAZvLLZRTSJM7ijpgL3e2kpnojYqPtLtJzdXko6lmT699wAMFs+4hWfIfgDna1Bk9HcDvo1yoG/PjsztAgKjLgBSx+GTIMWVpqsUN6VFPdvlGThRsfV6/SqRrLjeaalEfDrit6uGUFt/MXWrDDsDK6b32R2LoKG3ZELqfCkx22IsLyRdze6RIjrQaNPtPHhSYUZlqeoswtC9exRuytNUORYkM4DyTfKOYvvygxuNCbpGeOQbsVCr3X9UpYsGOKIew676SBwxRZMxNmUPl7jeYVDfWuuRh6u9jyUHxdLMLICcGi1A9KbP01AmOf7jkrm+y1Vb/i4DIMF4CO21ouIgM0JVD8+LDTOATMIS9lI2d/HKQ2gqRoVoLLAiRkYS3kNbFJIbsr+z/AJYFNZZAsk+ZGsYl1WxiHz3PUqIBWEMKM9ADGRyJdwNy4G1u8vqeIzpUjAAV/fBO9HiWTpvi63ollSwb1kOrsUylGStiq/+pBz7HGDEhTjWGIA3sH9PCxRehXIjqfiyVsgL767OwsFegX16t7ZVwHNw3Drk9BxFT3BTaQJoOtTARDSz0F+JmbzJU14Dv+QEDfbWHJWfoHgI04hqxMIiJpeGIk/TFxEdwRAtUfdzsV1io6ZjzNii8cUud49I+oAaKzLMSMUq4D+L7whCGgr/oBiySb71zT6Ezi6QfcXq+cJoFOwBex52pl1AtpHWQTv1xqzMKjnhESELzxzvRvJZqyWs8ACFR5XM2h97ks7VdEh61KadAzBh/yDaItonAgZCCIFKHPMlWdhkMjvFPpDERiyH3ckBXILOGDDEk6DYI+O5Ja1ZEWzaZ1AkCq7a9odFFNQOfmv/3fkA6MjM8yoXASfn5vR7qZ2GoGodNhDXfA3JgHbS83tbxrMCrtX2bCEA0MpprltyvkAIcYBNSwTMLrmLwf5ho/avQau7/1h8LJy0f4QS3VRm5o1gZGZbHI/zPOQEpRcAEQ8sQxfx0aeX83W2NTNY2Q6CCMRZ7/RG9PsTpprYl/7z+NpGSeJcYe7ZdrpPYEUsMmutZB6QOtkG0FA2qoYupYjH5xlCIR0n0J9KemLw3liwGs7SUVDUdTSSzGf X-OriginatorOrg: amd.com X-MS-Exchange-CrossTenant-Network-Message-Id: 414672ba-0f1c-48ae-4d27-08d90a580add X-MS-Exchange-CrossTenant-AuthSource: SN6PR12MB4623.namprd12.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 28 Apr 2021 15:12:30.6340 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: PMd9UMXpoWBl4rCsAPw1xZLOM1J6Vg0nrI4hh0OLmfUHc8d/fjBsl0QNwz5NsTIqlTie3IgsacFYsN43ByOkgQ== X-MS-Exchange-Transport-CrossTenantHeadersStamped: SN6PR12MB2799 X-BeenThere: dri-devel@lists.freedesktop.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Direct Rendering Infrastructure - Development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: gregkh@linuxfoundation.org, Felix.Kuehling@amd.com, helgaas@kernel.org, Alexander.Deucher@amd.com Errors-To: dri-devel-bounces@lists.freedesktop.org Sender: "dri-devel" On device removal reroute all CPU mappings to dummy page. v3: Remove loop to find DRM file and instead access it by vma->vm_file->private_data. Move dummy page installation into a separate function. v4: Map the entire BOs VA space into on demand allocated dummy page on the first fault for that BO. v5: Remove duplicate return. Signed-off-by: Andrey Grodzovsky --- drivers/gpu/drm/ttm/ttm_bo_vm.c | 79 ++++++++++++++++++++++++++++++++- include/drm/ttm/ttm_bo_api.h | 2 + 2 files changed, 80 insertions(+), 1 deletion(-) diff --git a/drivers/gpu/drm/ttm/ttm_bo_vm.c b/drivers/gpu/drm/ttm/ttm_bo_vm.c index b31b18058965..8b8300551a7f 100644 --- a/drivers/gpu/drm/ttm/ttm_bo_vm.c +++ b/drivers/gpu/drm/ttm/ttm_bo_vm.c @@ -34,6 +34,8 @@ #include #include #include +#include +#include #include #include #include @@ -380,19 +382,94 @@ vm_fault_t ttm_bo_vm_fault_reserved(struct vm_fault *vmf, } EXPORT_SYMBOL(ttm_bo_vm_fault_reserved); +static void ttm_bo_release_dummy_page(struct drm_device *dev, void *res) +{ + struct page *dummy_page = (struct page *)res; + + __free_page(dummy_page); +} + +vm_fault_t ttm_bo_vm_dummy_page(struct vm_fault *vmf, pgprot_t prot) +{ + struct vm_area_struct *vma = vmf->vma; + struct ttm_buffer_object *bo = vma->vm_private_data; + struct drm_device *ddev = bo->base.dev; + vm_fault_t ret = VM_FAULT_NOPAGE; + unsigned long address = vma->vm_start; + unsigned long num_prefault = (vma->vm_end - vma->vm_start) >> PAGE_SHIFT; + unsigned long pfn; + struct page *page; + int i; + + /* + * Wait for buffer data in transit, due to a pipelined + * move. + */ + ret = ttm_bo_vm_fault_idle(bo, vmf); + if (unlikely(ret != 0)) + return ret; + + /* Allocate new dummy page to map all the VA range in this VMA to it*/ + page = alloc_page(GFP_KERNEL | __GFP_ZERO); + if (!page) + return VM_FAULT_OOM; + + pfn = page_to_pfn(page); + + /* + * Prefault the entire VMA range right away to avoid further faults + */ + for (i = 0; i < num_prefault; ++i) { + + if (unlikely(address >= vma->vm_end)) + break; + + if (vma->vm_flags & VM_MIXEDMAP) + ret = vmf_insert_mixed_prot(vma, address, + __pfn_to_pfn_t(pfn, PFN_DEV), + prot); + else + ret = vmf_insert_pfn_prot(vma, address, pfn, prot); + + /* Never error on prefaulted PTEs */ + if (unlikely((ret & VM_FAULT_ERROR))) { + if (i == 0) + return VM_FAULT_NOPAGE; + else + break; + } + + address += PAGE_SIZE; + } + + /* Set the page to be freed using drmm release action */ + if (drmm_add_action_or_reset(ddev, ttm_bo_release_dummy_page, page)) + return VM_FAULT_OOM; + + return ret; +} +EXPORT_SYMBOL(ttm_bo_vm_dummy_page); + vm_fault_t ttm_bo_vm_fault(struct vm_fault *vmf) { struct vm_area_struct *vma = vmf->vma; pgprot_t prot; struct ttm_buffer_object *bo = vma->vm_private_data; + struct drm_device *ddev = bo->base.dev; vm_fault_t ret; + int idx; ret = ttm_bo_vm_reserve(bo, vmf); if (ret) return ret; prot = vma->vm_page_prot; - ret = ttm_bo_vm_fault_reserved(vmf, prot, TTM_BO_VM_NUM_PREFAULT, 1); + if (drm_dev_enter(ddev, &idx)) { + ret = ttm_bo_vm_fault_reserved(vmf, prot, TTM_BO_VM_NUM_PREFAULT, 1); + drm_dev_exit(idx); + } else { + ret = ttm_bo_vm_dummy_page(vmf, prot); + } if (ret == VM_FAULT_RETRY && !(vmf->flags & FAULT_FLAG_RETRY_NOWAIT)) return ret; diff --git a/include/drm/ttm/ttm_bo_api.h b/include/drm/ttm/ttm_bo_api.h index 3587f660e8f4..dbb00e495cb4 100644 --- a/include/drm/ttm/ttm_bo_api.h +++ b/include/drm/ttm/ttm_bo_api.h @@ -635,4 +635,6 @@ int ttm_bo_vm_access(struct vm_area_struct *vma, unsigned long addr, void *buf, int len, int write); bool ttm_bo_delayed_delete(struct ttm_device *bdev, bool remove_all); +vm_fault_t ttm_bo_vm_dummy_page(struct vm_fault *vmf, pgprot_t prot); + #endif From patchwork Wed Apr 28 15:11:42 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrey Grodzovsky X-Patchwork-Id: 12229245 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.5 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,MSGID_FROM_MTA_HEADER,SPF_HELO_NONE,SPF_PASS, USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 0FA3FC433ED for ; Wed, 28 Apr 2021 15:13:20 +0000 (UTC) Received: from gabe.freedesktop.org (gabe.freedesktop.org [131.252.210.177]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id AA99A61434 for ; Wed, 28 Apr 2021 15:13:19 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org AA99A61434 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=amd.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=dri-devel-bounces@lists.freedesktop.org Received: from gabe.freedesktop.org (localhost [127.0.0.1]) by gabe.freedesktop.org (Postfix) with ESMTP id D28416EB70; Wed, 28 Apr 2021 15:13:17 +0000 (UTC) Received: from NAM12-DM6-obe.outbound.protection.outlook.com (mail-dm6nam12on2078.outbound.protection.outlook.com [40.107.243.78]) by gabe.freedesktop.org (Postfix) with ESMTPS id 076B26EB55; Wed, 28 Apr 2021 15:12:36 +0000 (UTC) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=Gsj1pPvRprMSwVizi+GoKLIEv1R4L8ZZ9VCzuYFAE1ojNMOHKiLcG3VcD6ab1BYHKibrp31bSqUFY3feQ1QJ73vbdtisYQNQkFIv2GM7ma6vs3cTZKgDj7gZCUVCumHeBSv4hxC5UcWhAFAn6M94weHaSLW/GgiEteYSP1zE9XzSqkUxFuwnThXT8VDpHXE9AYd73ec7/5RsgmBqC5CYcgB0WPiH/Ra4QDsEiQenCfKim4vyi9HRhMJ4TfXQoZjUlbDCA7m9YthBI8LQHPEfvGZL6cnL1GQDVujxFX1TeuRmkN3IqZlnqzslq1L5fHExz+Zd7qxZffL1ufCRwtOTHw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=39EB68ov2WeP48MIIJQjZulb+ZIqexUKl7RSS3dV+VQ=; b=Y4HX33ynxIgI2oF+xKlwr11dj54xeLgkZVuRlzV8LevKuI/I1SZtaIGHW7HqvFg3KUgC7UqaCQfR2WKNy0Cao3HAgjSy5jE0pfF3Ve0CGYjbZK5Pak0F4E6ujFn1A2OscpUfQCTBCC2fccaFB6oQ+WkAxjggaJQ2fapkkTwflVgYsRwCwMTngG8nLHeZVY4R56C0RDXFqwAp2/aNpCwiOpZFCc53eB3oanMJRciMhQ/Y87zo9C6VSFLaCa/STchL2aknDi3Yi3juPjwRbjKTwjWO3Xzgf0TDpQDWa3VQHjGXO8LiSf8ndwxl6pZwlNhOg26+ZLwLWSK4Gcu2KmK2gg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=amd.com; dmarc=pass action=none header.from=amd.com; dkim=pass header.d=amd.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=39EB68ov2WeP48MIIJQjZulb+ZIqexUKl7RSS3dV+VQ=; b=HVwqscvA5q8BPA0CV+hz4Q6rJ+S5LPLvGykTKfRXdsL1o0k2Y+mb3wqztFf1lbi7hkea/CY9gwIzXRuVDYMMgavx6yj1IQ/ckAYXxgphmFKjh8Mx32EiGvpPJqHSwUFmEcjpECoEYLRFqrc/e/oVDLEMEDu5XJvxmmXlPK0xI3s= Authentication-Results: lists.freedesktop.org; dkim=none (message not signed) header.d=none; lists.freedesktop.org; dmarc=none action=none header.from=amd.com; Received: from SN6PR12MB4623.namprd12.prod.outlook.com (2603:10b6:805:e9::17) by SA0PR12MB4462.namprd12.prod.outlook.com (2603:10b6:806:95::22) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4065.23; Wed, 28 Apr 2021 15:12:35 +0000 Received: from SN6PR12MB4623.namprd12.prod.outlook.com ([fe80::ad51:8c49:b171:856c]) by SN6PR12MB4623.namprd12.prod.outlook.com ([fe80::ad51:8c49:b171:856c%7]) with mapi id 15.20.4065.026; Wed, 28 Apr 2021 15:12:35 +0000 From: Andrey Grodzovsky To: dri-devel@lists.freedesktop.org, amd-gfx@lists.freedesktop.org, linux-pci@vger.kernel.org, ckoenig.leichtzumerken@gmail.com, daniel.vetter@ffwll.ch, Harry.Wentland@amd.com Subject: [PATCH v5 02/27] drm/ttm: Expose ttm_tt_unpopulate for driver use Date: Wed, 28 Apr 2021 11:11:42 -0400 Message-Id: <20210428151207.1212258-3-andrey.grodzovsky@amd.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210428151207.1212258-1-andrey.grodzovsky@amd.com> References: <20210428151207.1212258-1-andrey.grodzovsky@amd.com> X-Originating-IP: [2607:fea8:3edf:49b0:7212:f93a:73b0:8f23] X-ClientProxiedBy: YT1PR01CA0142.CANPRD01.PROD.OUTLOOK.COM (2603:10b6:b01:2f::21) To SN6PR12MB4623.namprd12.prod.outlook.com (2603:10b6:805:e9::17) MIME-Version: 1.0 X-MS-Exchange-MessageSentRepresentingType: 1 Received: from agrodzovsky-All-Series.hitronhub.home (2607:fea8:3edf:49b0:7212:f93a:73b0:8f23) by YT1PR01CA0142.CANPRD01.PROD.OUTLOOK.COM (2603:10b6:b01:2f::21) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4065.25 via Frontend Transport; Wed, 28 Apr 2021 15:12:34 +0000 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 1fdb6af6-e36b-4e19-c60e-08d90a580d9e X-MS-TrafficTypeDiagnostic: SA0PR12MB4462: X-MS-Exchange-Transport-Forked: True X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:489; X-MS-Exchange-SenderADCheck: 1 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: 9eoGq5gV1NGpVSSWWjNBncmwJbQflVQsFk5KlIe0mWtcR3kuyjoQBd17cdIe47r0VCuEBDOlhTuzVHRg+86wpJczjIjO2WjnslXWX8EQ0jgvZDc63Y2BhXFT2hwUJ5AJoRF07RWoqx20lB0Z9WQf0+aWHF7qWa/aQ61fuJxdvx7uare0LUP6C5xrAo4gLvysMpm+KHwm69j+0XNcv/KvBHIElt7Zwa1/R0dTI0adz1JH1rmBGPxMrxOdU4O6tx8K4xTxYKTZT28rPEI2IlT00dZtrqRgF/mMDpX0D7nxnSPqFnnjWNPJb82GwYCpOfaXqZJ/Jl5jw8tU5EE8mda8+0kgZjViCI0/Aco2uEi81u0136Q8Q63uPzV31gVqkm/33ysKi+yx7HN9RmTv8udisgLN79h4f/0EW/0IMkWKWaQX+CjmUV8DWRnwRk4zKHPixkqxHQoGQAx0HKZ8j7pB6DMv8eszNQDHiORdBvppq8S7crR6nYDh7j4MBAZUdjvNBRj5akoOpageF1Pc6Qoz47+rowaSDxfB9MF2lyD1PrMWD6VMwgGODBTErJCeB4qJWzLK0q8nLbgdpOr259FPHeAuq/pyKrWZlcFfpKeNkuo= X-Forefront-Antispam-Report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:SN6PR12MB4623.namprd12.prod.outlook.com; PTR:; CAT:NONE; SFS:(4636009)(346002)(396003)(136003)(39860400002)(376002)(366004)(6512007)(6636002)(66946007)(316002)(186003)(44832011)(2906002)(16526019)(36756003)(83380400001)(6506007)(4326008)(52116002)(8676002)(66476007)(5660300002)(66556008)(1076003)(2616005)(6666004)(478600001)(86362001)(38100700002)(4744005)(8936002)(6486002); DIR:OUT; SFP:1101; X-MS-Exchange-AntiSpam-MessageData: pCddA1drrOZxjMlkM/TEmOWLbi+EvFQE4LVRMcb0/AjBPAdT3COOGLuqHRpLgET64uzgh1A0ci0hcyJGwOXJRwRIdb6UORM/iS8A84wS1XmlR8nzobwyeIzefcuKXwRRPbkjVD85ppfmjH0sNToTAsXS3Ngiz9xTQmk1Oe+JCBiOYAVUH1X0oUr/Zoi2AbCots8kV9qPhxLEg1f2TjNB0fgVKZdPCNFj86kJyJaNEAR3Lhd2Aw7We1kfH/9XsKqAn0pf2LHT69AXVW90u5X0rQoVSOlv1KU5i48V2inwiYVWY3XGAvDu1eSEp4aru+KHq9xXHggTQGxGSXj2k0WSTqYvwITBreCPEF1YJMR5a32+R8Y5W0kFNxWrflldMX8ccooSxiXTnWj7wxCvCiCeu2Hzu8qrPAkAHEPt2zt05w0URXVtXehV6MqMIgicmEd5MHQaAY9MqtekbGTG1GUUiMVr6NkC+/8x8NSkIZfZ5wFrcnMasIvARshCuNSR2coDfROSrWgPANT7nWMprJdq87Q+DdnVswTvvz+LlWl+hd+aJcHO4UK2P4B/aLCmHpUf9PKdnQY0GSZ2+JHfY2zc8Jb9Sr+Gya54Sqb2iFugQpNCPHM40bVOCshSk1Q1Pzr/d2nQ274LoCHuzU+tM1uf26IN3BqfkH3b9t6VOumLxRmvgD8y3Xja2+DBrx867HmECBpRPM8uh1ZzCvVwocG2T/0iywxnoaUiutHP4OBUzE0d2JYIE0vK3iQvU3b7rkWLi0xhnXcCENAFbcdosK46UfLl7QekXFNq+81ELKQ3NZAIJRY+uY3x2JAPARcdC4u96seEMuH3TqqGozw/M62qDD8NgIon1Omi0fpqloiH4k7N/VnVa7t4m8zIBlMt15nMzXThzaU2yfdOHvDzRxgyWPQLUFSkwlr8wnKsmKdKxoEeBrnKONXFi4OjEyiwIvyuY5D33P75UQ2vUVMegRTcYA44dakfb28StCBgIrLS/4MX8nj+4eh3xAMcch13NmikUtraxCWg062EKyAt91yssxx9tsDQ9fyacEspyWV9/UiC4PqELBvYoi+295l1VwtItbSF+7KX/Uwk/E5SE9md91G7wr16l2BvFPr6Oc7nHBiO/BHnz3+/wKPiGyotSvLKFbrOHoGZT1OXKW9rKeyL0MmFtbXUocXjTRXR9SDYXPpC8F3AiCYPgKeIiOy44JjLc7qeIxBQC77B92uy6OGxBLVNCDSyTR+PDz1yJmKR8ur7k5hEGBp1j+vzG97YGnaJLr24lS1IuB6EYGqz3/WH48wXYpN/5e4hYQSaG/fFN/wjHVO7Cpwbkh7WGm4pUBHws1mCYzzQk1kv4kT0icaCuwlqGkAveVqv77WUkeH9WC6zqlFJ+g+gQm77t7mCEfxI X-OriginatorOrg: amd.com X-MS-Exchange-CrossTenant-Network-Message-Id: 1fdb6af6-e36b-4e19-c60e-08d90a580d9e X-MS-Exchange-CrossTenant-AuthSource: SN6PR12MB4623.namprd12.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 28 Apr 2021 15:12:35.3203 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: WI5m1BNmQVVtIVGny6N4ah5Ic6VABG/N9/6dPRf1Luo/0bxBdWlN/NgjahyM8XKSeIgUFAXLjGJQ0lanFqt9Jg== X-MS-Exchange-Transport-CrossTenantHeadersStamped: SA0PR12MB4462 X-BeenThere: dri-devel@lists.freedesktop.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Direct Rendering Infrastructure - Development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: gregkh@linuxfoundation.org, Felix.Kuehling@amd.com, helgaas@kernel.org, Alexander.Deucher@amd.com Errors-To: dri-devel-bounces@lists.freedesktop.org Sender: "dri-devel" It's needed to drop iommu backed pages on device unplug before device's IOMMU group is released. Signed-off-by: Andrey Grodzovsky --- drivers/gpu/drm/ttm/ttm_tt.c | 1 + 1 file changed, 1 insertion(+) diff --git a/drivers/gpu/drm/ttm/ttm_tt.c b/drivers/gpu/drm/ttm/ttm_tt.c index 48c407cff112..f2ce1b372096 100644 --- a/drivers/gpu/drm/ttm/ttm_tt.c +++ b/drivers/gpu/drm/ttm/ttm_tt.c @@ -431,3 +431,4 @@ void ttm_tt_mgr_init(unsigned long num_pages, unsigned long num_dma32_pages) if (!ttm_dma32_pages_limit) ttm_dma32_pages_limit = num_dma32_pages; } +EXPORT_SYMBOL(ttm_tt_unpopulate); From patchwork Wed Apr 28 15:11:43 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Andrey Grodzovsky X-Patchwork-Id: 12229231 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.5 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,MSGID_FROM_MTA_HEADER,SPF_HELO_NONE,SPF_PASS, USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 44035C43462 for ; Wed, 28 Apr 2021 15:12:58 +0000 (UTC) Received: from gabe.freedesktop.org (gabe.freedesktop.org [131.252.210.177]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 9597061434 for ; Wed, 28 Apr 2021 15:12:56 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 9597061434 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=amd.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=dri-devel-bounces@lists.freedesktop.org Received: from gabe.freedesktop.org (localhost [127.0.0.1]) by gabe.freedesktop.org (Postfix) with ESMTP id E3C0A6EB62; Wed, 28 Apr 2021 15:12:55 +0000 (UTC) Received: from NAM12-DM6-obe.outbound.protection.outlook.com (mail-dm6nam12on2054.outbound.protection.outlook.com [40.107.243.54]) by gabe.freedesktop.org (Postfix) with ESMTPS id 75FBA6E0F2; Wed, 28 Apr 2021 15:12:39 +0000 (UTC) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=eQFi/crN7hZPEDYxQUTKaVrtzHvoaQZoKlDgYAEM5Cn5E1guge6mLdrr94Ku2eI5Ged1HZzKs6Nj4ZGUt1KaW4DeQCqBqYDLp945g3Hi5OAqYkZ2VCCukx4DReoD4ZUAVl3bTVvedCCuxaM47cmscMfqLDzthQs68N3Qw0iF7I92YPJSFHcgulOIhgd+Ns4KFOQx9hYKkLgWf+dvg2UQE94iCUwxrtFlTF4VDGM3BSiLd9Iy61eOqrKJ4RL774YSLklNdPvamqEluAzQSP+cOb+7KWbkty8icr7C+bLAOke+9gVePPvFsQyQ3F6zJdGxwfj4K5cx+huNO7L6kp8pRg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=+d3MUsHpF+5lMWV2OlCiCiRjN9rC1uRiwID31o4G1TI=; b=fLsGG4wotWl45U9+kjUeFdP1a86N4chhArVOAoRqDeXMxbwcf1SXpfff31WWtUaS11sHvY5zoNXiCj/cpuBsJZrsh2Lis1Z0OBkjHQSXfc8tS4S/QvRdpwN77Ua8VLDoNK907PfikN4lSgwuhmV+cfJcIfxe8WESt5otUwStqdpq5x3sbpvdOGGARGWtOBwELjMkPCwuUhhHWP/kQ7ysKC/LLEFfwtgZOhLyRvSg6PbevuGemWHD7yo8ZPPHFp5KB2lXqsYecjdskVXScfsgu6YvZRu581VgiXlY/SJngnBr/kmUtHjvFZijHbhanhhAdCfS/zoOY3sNkDZtjGpoqg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=amd.com; dmarc=pass action=none header.from=amd.com; dkim=pass header.d=amd.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=+d3MUsHpF+5lMWV2OlCiCiRjN9rC1uRiwID31o4G1TI=; b=VXFmWbbwGM27V4HQ89Ybte1Id9A7NB5z1kH0OgzSTahGpXqyF6ZW1dxV7Z24b7dNqeOUwI0fz96wYpJ00E0Z8LAtzA03ViY8CsTOkkLYszqjtotGEGkMIn/jMc0TCWbgwMJDr0d4jeP16jyBoLd1E+HkELCrry6YqkArC1DWJ4c= Authentication-Results: lists.freedesktop.org; dkim=none (message not signed) header.d=none; lists.freedesktop.org; dmarc=none action=none header.from=amd.com; Received: from SN6PR12MB4623.namprd12.prod.outlook.com (2603:10b6:805:e9::17) by SA0PR12MB4462.namprd12.prod.outlook.com (2603:10b6:806:95::22) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4065.23; Wed, 28 Apr 2021 15:12:37 +0000 Received: from SN6PR12MB4623.namprd12.prod.outlook.com ([fe80::ad51:8c49:b171:856c]) by SN6PR12MB4623.namprd12.prod.outlook.com ([fe80::ad51:8c49:b171:856c%7]) with mapi id 15.20.4065.026; Wed, 28 Apr 2021 15:12:37 +0000 From: Andrey Grodzovsky To: dri-devel@lists.freedesktop.org, amd-gfx@lists.freedesktop.org, linux-pci@vger.kernel.org, ckoenig.leichtzumerken@gmail.com, daniel.vetter@ffwll.ch, Harry.Wentland@amd.com Subject: [PATCH v5 03/27] drm/amdgpu: Split amdgpu_device_fini into early and late Date: Wed, 28 Apr 2021 11:11:43 -0400 Message-Id: <20210428151207.1212258-4-andrey.grodzovsky@amd.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210428151207.1212258-1-andrey.grodzovsky@amd.com> References: <20210428151207.1212258-1-andrey.grodzovsky@amd.com> X-Originating-IP: [2607:fea8:3edf:49b0:7212:f93a:73b0:8f23] X-ClientProxiedBy: YT1PR01CA0142.CANPRD01.PROD.OUTLOOK.COM (2603:10b6:b01:2f::21) To SN6PR12MB4623.namprd12.prod.outlook.com (2603:10b6:805:e9::17) MIME-Version: 1.0 X-MS-Exchange-MessageSentRepresentingType: 1 Received: from agrodzovsky-All-Series.hitronhub.home (2607:fea8:3edf:49b0:7212:f93a:73b0:8f23) by YT1PR01CA0142.CANPRD01.PROD.OUTLOOK.COM (2603:10b6:b01:2f::21) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4065.25 via Frontend Transport; Wed, 28 Apr 2021 15:12:36 +0000 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 1cd8cd65-a85b-4c08-a13a-08d90a580efa X-MS-TrafficTypeDiagnostic: SA0PR12MB4462: X-MS-Exchange-Transport-Forked: True X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:862; X-MS-Exchange-SenderADCheck: 1 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: oek5Bz5OAPnGhQykAqPI8KlfSyezy5IbQigx6nWAezODAT9xU0uzZTxC0pbVt49eyU+vf30XUX4ZB9Jo3Z5Le1pgw/LFNs27+cWVfPCoytnn/XoIjnL0itceoh5uhqm/CK2v9MiodaC0SkIXiUvezf+219QQ6Evx2cL3HqgwUTCcmR7YvSuOziear3moX+CajbmN2/YMqF/Mit+7NsPwNa91RlS9Xj+gERrCJA3tNdY1xmE3c4Ag20hCw1jYa0ieY3oSo0OEfDg2JaeHEl3orboGZ8oSD9t1/j7KTsSalDdaDkEpDx+32mOCMRUBxEKYFnYC3fPW6YXlp5Dht2mbKvPMGOMhV+fXiocMFiUhQBXjTpWAVOdFpQj8qTiZ9/hwW7L1QDgMLdXQJiDVhrA4+aOqYXHlfNhXTP5clS9+7eiXahkplkABqWM57tOVl5t9Enlok1XVTlxPP5RmvPDPEJhhJnvvN4DhhM61RZsL6JWottOOsWmvqA8YGpeuS6RcsGfVQZDXyyjCzdJXZHZVf/HZRLC/Cy8yfCxBvhE0SGLoYV9+OuoOSTD8mpX4AJcjxkJZDe4F8u8WbvFeu7mZ3BxSeGaC8SkKidGKZnDsImI= X-Forefront-Antispam-Report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:SN6PR12MB4623.namprd12.prod.outlook.com; PTR:; CAT:NONE; SFS:(4636009)(346002)(396003)(136003)(39860400002)(376002)(366004)(6512007)(6636002)(66946007)(316002)(186003)(44832011)(2906002)(16526019)(36756003)(83380400001)(6506007)(4326008)(52116002)(8676002)(66476007)(5660300002)(66556008)(1076003)(2616005)(6666004)(478600001)(30864003)(86362001)(38100700002)(8936002)(6486002); DIR:OUT; SFP:1101; X-MS-Exchange-AntiSpam-MessageData: Ddlo1cWyQ9ZE564eGlyWkW2w0/PoMHaUpEu32BWKDXFXkrsqgGzenFUESX0ABaRdTEWwotmOA1FLMeWG2zU4Iwnfx901yiy8rjNuOQsvL0MMY/voa4MirR+c75hUQmYpjuVBI/B7PIz8836PiaMDija8SWBb5NTw6o7qM/iZk28ut5PqwqPMSrorZVgekVNrlGWzWeIgWT7pIlfIX2bfa83MkKp//mhHwJNTlUaJfm4E83H/GEiIj40+SM4mxLq6lsuWbiskRK/cpDJf3VquLEXwUPLx2G72iPzRg8qGbfYa1CXSuBKj0r19df0UgslC8mmyFbsUY6C0VnrjbCfGDr9zdx18A5Rn9EUKgMpE9tELJl049kbE+yUNpOT/uYitlpkVO211z+JmuW4lSnD8T6zO+uRlU3MzAprlapGemeYUDuXp8uwE9alVB0kskBbP1pdRjt4x8tIq1XAZmZcWvaehm3GjxZ6NNYGvwifX+t/4+GEXwABfHY0A0qd3MiQv+MlLtvn5iiGYY1HE9Yf1HHGoUp1tr4HpssM5gKdcHfoRXqPG4htDCxJCPgsyrsVFO39x+K/Vsc5fEE+SwakPjbXCdAMuBcDnkpaxYcPAQJowHhnMWGh+z2X/5LMFLwz0xjFHFS9SbbkuNqwlnthNMavHXSdDQkqEowcO8jwhGh59n4mgTDhpRK5mxezmzkVTMLONPXeOuI8VcJhzyWj2t5J8uRCk3QAi/jL70wO561H9OEqWwNjgJ08hixLxUFkGm966X9b98qZ8Dw957jdQ6TyeGvTdX37lkAmqsWe4Vm387ZULTjvMhUZn2NZ+XXwf25MzUmfn7rWWIy5Q3JaOgw0Bqp1TDv7zZ/e4wONClF8JUXIPAAOrmwmFoUNalmd8BjB5B0LMraf4PvtHizFHSgCPIkFRJ750aDQU2ermjlODzUGpU1KgPlIwGLhltppqiSVfONksdSo3uL//QC2an/zRGF8+7cb6f1zrgB/FA0v1C44nzrhQFJPSUKweIRlgIjP89LOoqvRoL+Nsu0I0y4MVGLPrC5X4J9EesKxHohLcitF6byO1yl2YdGijYc3+4818kgfcxwK5swnZR7NpToINdcP3qIghxsik1oqQK5vLegC4WjJQyoLTbq6wLQUiFvSysMZDdMDOmbUpKm08ovkvinK7In4fooEXgZ3nmXSeHzzB98/ajflSQeq/HMjzv3mUBpR+Sb26Lm6FY+ZFVXULdv9NaTTKDs96Nvw0aEVocZy+yGqqtoSx12AjSRTX4kIMRzEZOsl6nuDPIIWkuSvAelYdBrgcsM9CrvbsnejhIXFjic6NV4WFZMRE4F+oSnr/X/oNYRhPfhtJaD3m9vfGa7SlySJ002z73A5qe2U7YvIqxB/ARbSnb0kigPmp X-OriginatorOrg: amd.com X-MS-Exchange-CrossTenant-Network-Message-Id: 1cd8cd65-a85b-4c08-a13a-08d90a580efa X-MS-Exchange-CrossTenant-AuthSource: SN6PR12MB4623.namprd12.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 28 Apr 2021 15:12:37.5701 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: G3lylv3bdgSyjGScgAN7h4CECg5i3Qy3R6BG6QxB91T3319nSIWsCqk1sO+ufISxCZI3qhMqYEzPRuSqrAFwFw== X-MS-Exchange-Transport-CrossTenantHeadersStamped: SA0PR12MB4462 X-BeenThere: dri-devel@lists.freedesktop.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Direct Rendering Infrastructure - Development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: gregkh@linuxfoundation.org, Felix.Kuehling@amd.com, helgaas@kernel.org, Alexander.Deucher@amd.com Errors-To: dri-devel-bounces@lists.freedesktop.org Sender: "dri-devel" Some of the stuff in amdgpu_device_fini such as HW interrupts disable and pending fences finilization must be done right away on pci_remove while most of the stuff which relates to finilizing and releasing driver data structures can be kept until drm_driver.release hook is called, i.e. when the last device reference is dropped. v4: Change functions prefix early->hw and late->sw Signed-off-by: Andrey Grodzovsky Acked-by: Christian König Reviewed-by: Alex Deucher --- drivers/gpu/drm/amd/amdgpu/amdgpu.h | 6 ++++- drivers/gpu/drm/amd/amdgpu/amdgpu_device.c | 26 +++++++++++++++------- drivers/gpu/drm/amd/amdgpu/amdgpu_drv.c | 7 ++---- drivers/gpu/drm/amd/amdgpu/amdgpu_fence.c | 15 ++++++++++++- drivers/gpu/drm/amd/amdgpu/amdgpu_irq.c | 26 +++++++++++++--------- drivers/gpu/drm/amd/amdgpu/amdgpu_irq.h | 3 ++- drivers/gpu/drm/amd/amdgpu/amdgpu_kms.c | 12 +++++++++- drivers/gpu/drm/amd/amdgpu/amdgpu_ras.c | 1 + drivers/gpu/drm/amd/amdgpu/amdgpu_ring.h | 3 ++- drivers/gpu/drm/amd/amdgpu/cik_ih.c | 2 +- drivers/gpu/drm/amd/amdgpu/cz_ih.c | 2 +- drivers/gpu/drm/amd/amdgpu/iceland_ih.c | 2 +- drivers/gpu/drm/amd/amdgpu/navi10_ih.c | 2 +- drivers/gpu/drm/amd/amdgpu/si_ih.c | 2 +- drivers/gpu/drm/amd/amdgpu/tonga_ih.c | 2 +- drivers/gpu/drm/amd/amdgpu/vega10_ih.c | 2 +- drivers/gpu/drm/amd/amdgpu/vega20_ih.c | 2 +- 17 files changed, 79 insertions(+), 36 deletions(-) diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu.h b/drivers/gpu/drm/amd/amdgpu/amdgpu.h index 1af2fa1591fd..fddb82897e5d 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu.h +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu.h @@ -1073,7 +1073,9 @@ static inline struct amdgpu_device *amdgpu_ttm_adev(struct ttm_device *bdev) int amdgpu_device_init(struct amdgpu_device *adev, uint32_t flags); -void amdgpu_device_fini(struct amdgpu_device *adev); +void amdgpu_device_fini_hw(struct amdgpu_device *adev); +void amdgpu_device_fini_sw(struct amdgpu_device *adev); + int amdgpu_gpu_wait_for_idle(struct amdgpu_device *adev); void amdgpu_device_vram_access(struct amdgpu_device *adev, loff_t pos, @@ -1289,6 +1291,8 @@ void amdgpu_driver_lastclose_kms(struct drm_device *dev); int amdgpu_driver_open_kms(struct drm_device *dev, struct drm_file *file_priv); void amdgpu_driver_postclose_kms(struct drm_device *dev, struct drm_file *file_priv); +void amdgpu_driver_release_kms(struct drm_device *dev); + int amdgpu_device_ip_suspend(struct amdgpu_device *adev); int amdgpu_device_suspend(struct drm_device *dev, bool fbcon); int amdgpu_device_resume(struct drm_device *dev, bool fbcon); diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c index 6447cd6ca5a8..8d22b79fc1cd 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c @@ -3590,14 +3590,12 @@ int amdgpu_device_init(struct amdgpu_device *adev, * Tear down the driver info (all asics). * Called at driver shutdown. */ -void amdgpu_device_fini(struct amdgpu_device *adev) +void amdgpu_device_fini_hw(struct amdgpu_device *adev) { dev_info(adev->dev, "amdgpu: finishing device.\n"); flush_delayed_work(&adev->delayed_init_work); adev->shutdown = true; - kfree(adev->pci_state); - /* make sure IB test finished before entering exclusive mode * to avoid preemption on IB test * */ @@ -3614,11 +3612,24 @@ void amdgpu_device_fini(struct amdgpu_device *adev) else drm_atomic_helper_shutdown(adev_to_drm(adev)); } - amdgpu_fence_driver_fini(adev); + amdgpu_fence_driver_fini_hw(adev); + if (adev->pm_sysfs_en) amdgpu_pm_sysfs_fini(adev); + if (adev->ucode_sysfs_en) + amdgpu_ucode_sysfs_fini(adev); + sysfs_remove_files(&adev->dev->kobj, amdgpu_dev_attributes); + + amdgpu_fbdev_fini(adev); + + amdgpu_irq_fini_hw(adev); +} + +void amdgpu_device_fini_sw(struct amdgpu_device *adev) +{ amdgpu_device_ip_fini(adev); + amdgpu_fence_driver_fini_sw(adev); release_firmware(adev->firmware.gpu_info_fw); adev->firmware.gpu_info_fw = NULL; adev->accel_working = false; @@ -3647,14 +3658,13 @@ void amdgpu_device_fini(struct amdgpu_device *adev) adev->rmmio = NULL; amdgpu_device_doorbell_fini(adev); - if (adev->ucode_sysfs_en) - amdgpu_ucode_sysfs_fini(adev); - - sysfs_remove_files(&adev->dev->kobj, amdgpu_dev_attributes); if (IS_ENABLED(CONFIG_PERF_EVENTS)) amdgpu_pmu_fini(adev); if (adev->mman.discovery_bin) amdgpu_discovery_fini(adev); + + kfree(adev->pci_state); + } diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_drv.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_drv.c index 671ec1002230..54cb5ee2f563 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_drv.c +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_drv.c @@ -1249,14 +1249,10 @@ amdgpu_pci_remove(struct pci_dev *pdev) { struct drm_device *dev = pci_get_drvdata(pdev); -#ifdef MODULE - if (THIS_MODULE->state != MODULE_STATE_GOING) -#endif - DRM_ERROR("Hotplug removal is not supported\n"); drm_dev_unplug(dev); amdgpu_driver_unload_kms(dev); + pci_disable_device(pdev); - pci_set_drvdata(pdev, NULL); } static void @@ -1587,6 +1583,7 @@ static const struct drm_driver amdgpu_kms_driver = { .dumb_create = amdgpu_mode_dumb_create, .dumb_map_offset = amdgpu_mode_dumb_mmap, .fops = &amdgpu_driver_kms_fops, + .release = &amdgpu_driver_release_kms, .prime_handle_to_fd = drm_gem_prime_handle_to_fd, .prime_fd_to_handle = drm_gem_prime_fd_to_handle, diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_fence.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_fence.c index 8e0a5650d383..34d51e962799 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_fence.c +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_fence.c @@ -523,7 +523,7 @@ int amdgpu_fence_driver_init(struct amdgpu_device *adev) * * Tear down the fence driver for all possible rings (all asics). */ -void amdgpu_fence_driver_fini(struct amdgpu_device *adev) +void amdgpu_fence_driver_fini_hw(struct amdgpu_device *adev) { unsigned i, j; int r; @@ -544,6 +544,19 @@ void amdgpu_fence_driver_fini(struct amdgpu_device *adev) if (!ring->no_scheduler) drm_sched_fini(&ring->sched); del_timer_sync(&ring->fence_drv.fallback_timer); + } +} + +void amdgpu_fence_driver_fini_sw(struct amdgpu_device *adev) +{ + unsigned int i, j; + + for (i = 0; i < AMDGPU_MAX_RINGS; i++) { + struct amdgpu_ring *ring = adev->rings[i]; + + if (!ring || !ring->fence_drv.initialized) + continue; + for (j = 0; j <= ring->fence_drv.num_fences_mask; ++j) dma_fence_put(ring->fence_drv.fences[j]); kfree(ring->fence_drv.fences); diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_irq.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_irq.c index afbbec82a289..63e815c27585 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_irq.c +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_irq.c @@ -49,6 +49,7 @@ #include #include #include +#include #include "amdgpu.h" #include "amdgpu_ih.h" #include "atom.h" @@ -313,6 +314,20 @@ int amdgpu_irq_init(struct amdgpu_device *adev) return 0; } + +void amdgpu_irq_fini_hw(struct amdgpu_device *adev) +{ + if (adev->irq.installed) { + drm_irq_uninstall(&adev->ddev); + adev->irq.installed = false; + if (adev->irq.msi_enabled) + pci_free_irq_vectors(adev->pdev); + + if (!amdgpu_device_has_dc_support(adev)) + flush_work(&adev->hotplug_work); + } +} + /** * amdgpu_irq_fini - shut down interrupt handling * @@ -322,19 +337,10 @@ int amdgpu_irq_init(struct amdgpu_device *adev) * functionality, shuts down vblank, hotplug and reset interrupt handling, * turns off interrupts from all sources (all ASICs). */ -void amdgpu_irq_fini(struct amdgpu_device *adev) +void amdgpu_irq_fini_sw(struct amdgpu_device *adev) { unsigned i, j; - if (adev->irq.installed) { - drm_irq_uninstall(adev_to_drm(adev)); - adev->irq.installed = false; - if (adev->irq.msi_enabled) - pci_free_irq_vectors(adev->pdev); - if (!amdgpu_device_has_dc_support(adev)) - flush_work(&adev->hotplug_work); - } - for (i = 0; i < AMDGPU_IRQ_CLIENTID_MAX; ++i) { if (!adev->irq.client[i].sources) continue; diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_irq.h b/drivers/gpu/drm/amd/amdgpu/amdgpu_irq.h index ac527e5deae6..392a7324e2b1 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_irq.h +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_irq.h @@ -104,7 +104,8 @@ void amdgpu_irq_disable_all(struct amdgpu_device *adev); irqreturn_t amdgpu_irq_handler(int irq, void *arg); int amdgpu_irq_init(struct amdgpu_device *adev); -void amdgpu_irq_fini(struct amdgpu_device *adev); +void amdgpu_irq_fini_sw(struct amdgpu_device *adev); +void amdgpu_irq_fini_hw(struct amdgpu_device *adev); int amdgpu_irq_add_id(struct amdgpu_device *adev, unsigned client_id, unsigned src_id, struct amdgpu_irq_src *source); diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_kms.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_kms.c index 64beb3399604..1af3fba7bfd4 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_kms.c +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_kms.c @@ -29,6 +29,7 @@ #include "amdgpu.h" #include #include +#include #include "amdgpu_uvd.h" #include "amdgpu_vce.h" #include "atom.h" @@ -93,7 +94,7 @@ void amdgpu_driver_unload_kms(struct drm_device *dev) } amdgpu_acpi_fini(adev); - amdgpu_device_fini(adev); + amdgpu_device_fini_hw(adev); } void amdgpu_register_gpu_instance(struct amdgpu_device *adev) @@ -1151,6 +1152,15 @@ void amdgpu_driver_postclose_kms(struct drm_device *dev, pm_runtime_put_autosuspend(dev->dev); } + +void amdgpu_driver_release_kms(struct drm_device *dev) +{ + struct amdgpu_device *adev = drm_to_adev(dev); + + amdgpu_device_fini_sw(adev); + pci_set_drvdata(adev->pdev, NULL); +} + /* * VBlank related functions. */ diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_ras.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_ras.c index 1fb2a91ad30a..c0a16eac4923 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_ras.c +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_ras.c @@ -2142,6 +2142,7 @@ int amdgpu_ras_pre_fini(struct amdgpu_device *adev) if (!con) return 0; + /* Need disable ras on all IPs here before ip [hw/sw]fini */ amdgpu_ras_disable_all_features(adev, 0); amdgpu_ras_recovery_fini(adev); diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_ring.h b/drivers/gpu/drm/amd/amdgpu/amdgpu_ring.h index 56acec1075ac..0f195f7bf797 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_ring.h +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_ring.h @@ -107,7 +107,8 @@ struct amdgpu_fence_driver { }; int amdgpu_fence_driver_init(struct amdgpu_device *adev); -void amdgpu_fence_driver_fini(struct amdgpu_device *adev); +void amdgpu_fence_driver_fini_hw(struct amdgpu_device *adev); +void amdgpu_fence_driver_fini_sw(struct amdgpu_device *adev); void amdgpu_fence_driver_force_completion(struct amdgpu_ring *ring); int amdgpu_fence_driver_init_ring(struct amdgpu_ring *ring, diff --git a/drivers/gpu/drm/amd/amdgpu/cik_ih.c b/drivers/gpu/drm/amd/amdgpu/cik_ih.c index d3745711d55f..183d44a6583c 100644 --- a/drivers/gpu/drm/amd/amdgpu/cik_ih.c +++ b/drivers/gpu/drm/amd/amdgpu/cik_ih.c @@ -309,7 +309,7 @@ static int cik_ih_sw_fini(void *handle) { struct amdgpu_device *adev = (struct amdgpu_device *)handle; - amdgpu_irq_fini(adev); + amdgpu_irq_fini_sw(adev); amdgpu_ih_ring_fini(adev, &adev->irq.ih); amdgpu_irq_remove_domain(adev); diff --git a/drivers/gpu/drm/amd/amdgpu/cz_ih.c b/drivers/gpu/drm/amd/amdgpu/cz_ih.c index 307c01301c87..d32743949003 100644 --- a/drivers/gpu/drm/amd/amdgpu/cz_ih.c +++ b/drivers/gpu/drm/amd/amdgpu/cz_ih.c @@ -301,7 +301,7 @@ static int cz_ih_sw_fini(void *handle) { struct amdgpu_device *adev = (struct amdgpu_device *)handle; - amdgpu_irq_fini(adev); + amdgpu_irq_fini_sw(adev); amdgpu_ih_ring_fini(adev, &adev->irq.ih); amdgpu_irq_remove_domain(adev); diff --git a/drivers/gpu/drm/amd/amdgpu/iceland_ih.c b/drivers/gpu/drm/amd/amdgpu/iceland_ih.c index cc957471f31e..da96c6013477 100644 --- a/drivers/gpu/drm/amd/amdgpu/iceland_ih.c +++ b/drivers/gpu/drm/amd/amdgpu/iceland_ih.c @@ -300,7 +300,7 @@ static int iceland_ih_sw_fini(void *handle) { struct amdgpu_device *adev = (struct amdgpu_device *)handle; - amdgpu_irq_fini(adev); + amdgpu_irq_fini_sw(adev); amdgpu_ih_ring_fini(adev, &adev->irq.ih); amdgpu_irq_remove_domain(adev); diff --git a/drivers/gpu/drm/amd/amdgpu/navi10_ih.c b/drivers/gpu/drm/amd/amdgpu/navi10_ih.c index f4e4040bbd25..5eea4550b856 100644 --- a/drivers/gpu/drm/amd/amdgpu/navi10_ih.c +++ b/drivers/gpu/drm/amd/amdgpu/navi10_ih.c @@ -569,7 +569,7 @@ static int navi10_ih_sw_fini(void *handle) { struct amdgpu_device *adev = (struct amdgpu_device *)handle; - amdgpu_irq_fini(adev); + amdgpu_irq_fini_sw(adev); amdgpu_ih_ring_fini(adev, &adev->irq.ih_soft); amdgpu_ih_ring_fini(adev, &adev->irq.ih2); amdgpu_ih_ring_fini(adev, &adev->irq.ih1); diff --git a/drivers/gpu/drm/amd/amdgpu/si_ih.c b/drivers/gpu/drm/amd/amdgpu/si_ih.c index 51880f6ef634..751307f3252c 100644 --- a/drivers/gpu/drm/amd/amdgpu/si_ih.c +++ b/drivers/gpu/drm/amd/amdgpu/si_ih.c @@ -175,7 +175,7 @@ static int si_ih_sw_fini(void *handle) { struct amdgpu_device *adev = (struct amdgpu_device *)handle; - amdgpu_irq_fini(adev); + amdgpu_irq_fini_sw(adev); amdgpu_ih_ring_fini(adev, &adev->irq.ih); return 0; diff --git a/drivers/gpu/drm/amd/amdgpu/tonga_ih.c b/drivers/gpu/drm/amd/amdgpu/tonga_ih.c index 249fcbee7871..973d80ec7f6c 100644 --- a/drivers/gpu/drm/amd/amdgpu/tonga_ih.c +++ b/drivers/gpu/drm/amd/amdgpu/tonga_ih.c @@ -312,7 +312,7 @@ static int tonga_ih_sw_fini(void *handle) { struct amdgpu_device *adev = (struct amdgpu_device *)handle; - amdgpu_irq_fini(adev); + amdgpu_irq_fini_sw(adev); amdgpu_ih_ring_fini(adev, &adev->irq.ih); amdgpu_irq_remove_domain(adev); diff --git a/drivers/gpu/drm/amd/amdgpu/vega10_ih.c b/drivers/gpu/drm/amd/amdgpu/vega10_ih.c index 88626d83e07b..2d0094c276ca 100644 --- a/drivers/gpu/drm/amd/amdgpu/vega10_ih.c +++ b/drivers/gpu/drm/amd/amdgpu/vega10_ih.c @@ -523,7 +523,7 @@ static int vega10_ih_sw_fini(void *handle) { struct amdgpu_device *adev = (struct amdgpu_device *)handle; - amdgpu_irq_fini(adev); + amdgpu_irq_fini_sw(adev); amdgpu_ih_ring_fini(adev, &adev->irq.ih_soft); amdgpu_ih_ring_fini(adev, &adev->irq.ih2); amdgpu_ih_ring_fini(adev, &adev->irq.ih1); diff --git a/drivers/gpu/drm/amd/amdgpu/vega20_ih.c b/drivers/gpu/drm/amd/amdgpu/vega20_ih.c index 5a3c867d5881..9059b21b079f 100644 --- a/drivers/gpu/drm/amd/amdgpu/vega20_ih.c +++ b/drivers/gpu/drm/amd/amdgpu/vega20_ih.c @@ -558,7 +558,7 @@ static int vega20_ih_sw_fini(void *handle) { struct amdgpu_device *adev = (struct amdgpu_device *)handle; - amdgpu_irq_fini(adev); + amdgpu_irq_fini_sw(adev); amdgpu_ih_ring_fini(adev, &adev->irq.ih_soft); amdgpu_ih_ring_fini(adev, &adev->irq.ih2); amdgpu_ih_ring_fini(adev, &adev->irq.ih1); From patchwork Wed Apr 28 15:11:44 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrey Grodzovsky X-Patchwork-Id: 12229241 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.5 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,MSGID_FROM_MTA_HEADER,SPF_HELO_NONE,SPF_PASS, USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id E25EDC433B4 for ; Wed, 28 Apr 2021 15:13:15 +0000 (UTC) Received: from gabe.freedesktop.org (gabe.freedesktop.org [131.252.210.177]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 8AECB61434 for ; Wed, 28 Apr 2021 15:13:15 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 8AECB61434 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=amd.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=dri-devel-bounces@lists.freedesktop.org Received: from gabe.freedesktop.org (localhost [127.0.0.1]) by gabe.freedesktop.org (Postfix) with ESMTP id 759BD6EB6E; Wed, 28 Apr 2021 15:13:14 +0000 (UTC) Received: from NAM12-DM6-obe.outbound.protection.outlook.com (mail-dm6nam12on2054.outbound.protection.outlook.com [40.107.243.54]) by gabe.freedesktop.org (Postfix) with ESMTPS id EF2246E0F5; Wed, 28 Apr 2021 15:12:44 +0000 (UTC) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=dIa7MFyDbsSgyo6DGs6QxD88NY8lg6yU4j7nU4uroLxixlar09ok51jWyO01+71khvWjDx6yA3Ry5zTO0d+sC+VSMAsSSTUXP0iqzxvkVMwRfV5PN5ELyGpHHTY9DOkwwWfg3GmQKjAYGUelk4oviXdXFJFVK9rDRA5UwbMk8FDOQGZ5Nc8IQJH+VAf4VMQPadPCUqOjMpWA30HkrA/5NxS2LsnpGcXl4KYDkvS1GmTW74ABygjuC8PnQgwalWDzxnnBCyK6sC07xqBlrrs7OrhQK+6l/uo5fpcNlbbZZlG2/+TgEHKsufSMU1ncszT3w42HMlOzTBY7uSDIQH5XXg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=Es/qJ4QTkAKw88gvzPMX0HWHJcL+QYwFsXAz0XffHkI=; b=mb7ClGyap68mkhFMEVdoxP6x5zRNokCB/Cp30gGVRz83FsPBLzobEl+1LBynuf13v+y3P0ZaWE/a9ALkWAou6F+KF7Ggt2NhCX1b3vDGfyCqrZpcgyquM9nkz5iGI7UOTAWVgt9PmIg+FpVsPVsEh6/JHHwE5Ia/LwxbqsOZAW0nRBGS92TLf1CtLAAAsGidl2+ebfQ08zQW7bycyeax0LqSnYJyQmBKzQ1nor5jpHgqPJo7RhEuaJa7a/NFFOarautPGZlVVGY2ioUuT8WZGFgTnqIdND/CVv56b9wCV4FN4ksSO8nH7T2xuTqxrLs/84xr9ZufA1hzuvI4i4aN/g== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=amd.com; dmarc=pass action=none header.from=amd.com; dkim=pass header.d=amd.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=Es/qJ4QTkAKw88gvzPMX0HWHJcL+QYwFsXAz0XffHkI=; b=4FM7Fb4bFiByGmiZ2a2KTiMVbwvjeAmOK5t5muSsHYJvex++QXOLswN11RwKyA2kteyiCMxvJhNNy7IyIXZK8C/iTcLDkKe+ECAgAVgcYKQlUbcU3B2m7UnMaE4lei/iZPNW1jMGlY8gy2eopMFD2UOPeSbUV4QaZLaKjbUnTXw= Authentication-Results: lists.freedesktop.org; dkim=none (message not signed) header.d=none; lists.freedesktop.org; dmarc=none action=none header.from=amd.com; Received: from SN6PR12MB4623.namprd12.prod.outlook.com (2603:10b6:805:e9::17) by SA0PR12MB4462.namprd12.prod.outlook.com (2603:10b6:806:95::22) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4065.23; Wed, 28 Apr 2021 15:12:39 +0000 Received: from SN6PR12MB4623.namprd12.prod.outlook.com ([fe80::ad51:8c49:b171:856c]) by SN6PR12MB4623.namprd12.prod.outlook.com ([fe80::ad51:8c49:b171:856c%7]) with mapi id 15.20.4065.026; Wed, 28 Apr 2021 15:12:39 +0000 From: Andrey Grodzovsky To: dri-devel@lists.freedesktop.org, amd-gfx@lists.freedesktop.org, linux-pci@vger.kernel.org, ckoenig.leichtzumerken@gmail.com, daniel.vetter@ffwll.ch, Harry.Wentland@amd.com Subject: [PATCH v5 04/27] drm/amdkfd: Split kfd suspend from devie exit Date: Wed, 28 Apr 2021 11:11:44 -0400 Message-Id: <20210428151207.1212258-5-andrey.grodzovsky@amd.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210428151207.1212258-1-andrey.grodzovsky@amd.com> References: <20210428151207.1212258-1-andrey.grodzovsky@amd.com> X-Originating-IP: [2607:fea8:3edf:49b0:7212:f93a:73b0:8f23] X-ClientProxiedBy: YT1PR01CA0142.CANPRD01.PROD.OUTLOOK.COM (2603:10b6:b01:2f::21) To SN6PR12MB4623.namprd12.prod.outlook.com (2603:10b6:805:e9::17) MIME-Version: 1.0 X-MS-Exchange-MessageSentRepresentingType: 1 Received: from agrodzovsky-All-Series.hitronhub.home (2607:fea8:3edf:49b0:7212:f93a:73b0:8f23) by YT1PR01CA0142.CANPRD01.PROD.OUTLOOK.COM (2603:10b6:b01:2f::21) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4065.25 via Frontend Transport; Wed, 28 Apr 2021 15:12:38 +0000 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: cb5c4f32-e8b2-44ba-5290-08d90a581000 X-MS-TrafficTypeDiagnostic: SA0PR12MB4462: X-MS-Exchange-Transport-Forked: True X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:510; X-MS-Exchange-SenderADCheck: 1 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: xu1+BjEcWeBvAC/5Vfw4afm0GTz5nOMyXTax+t+QCjNIUTSOESVMA/s20OtUhIiPpZC7FpeeDW20uQJWqh9NIQeaFu6UALW7RBMO5KRulElzt1QcF8wiZgmo87nEz/PIQtiStD4BmAmQWeSzNzo+5VbgXdOihoVWVtMVaDu/J4g2FoY+1pVvogCY6z39aIta8Jykc6lRHJK8/Gh//ezPenleejIiEJWhMgQKhUD3xwIbAiEPwOXi47TV5DQ2rj6nlbU0sJRlqn3FAqSe15gk0GMhcQReSV7jgGnkNKHmU0sxFMyWq3vfLHpVOOS8T5+G+XPbHurBByNE3+PIX4loDJe+p+YNOVf1XIMChy2sLbe/gu4xAxyfjwBMUH0x+Yqd7HGvzZ/IQAobpbhmDIQjxRbEstYa6DacJDubUWp33rAuerxLhyNXe58+QZONq72Z1QoM0qbIIXDCqrd8osJLVnR0ZFraXcq+S9wzX+gtaADHNRAC9P/nfzMEZ2Z9lEe2bwg1cNkiSoIoyiiZYtyua9TbFGVxzxTBjiTjWxGwRuBkWDcVumn2+vDDU6//BMhpMzhvuXLJenhiNYV9VxD4TI2bF+qYfIkeVRsst60Ap0Y= X-Forefront-Antispam-Report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:SN6PR12MB4623.namprd12.prod.outlook.com; PTR:; CAT:NONE; SFS:(4636009)(346002)(396003)(136003)(39860400002)(376002)(366004)(6512007)(6636002)(66946007)(316002)(186003)(44832011)(2906002)(16526019)(36756003)(83380400001)(6506007)(4326008)(52116002)(8676002)(66476007)(5660300002)(66556008)(1076003)(2616005)(6666004)(478600001)(86362001)(38100700002)(8936002)(6486002); DIR:OUT; SFP:1101; X-MS-Exchange-AntiSpam-MessageData: iyxBW4r590bw/ZOWfrckcGCxHnmigetSSink7tq1ZXx0Gev4UrJizRDbtAPS0i2GW8wvIreKCiuEbyAYnTdFmYJF+NOQXOsWzq/hXNTlmMriHROwSQyDi/S6eY24QMQ0hEvZ0K9VpT51fCCCfaAYXlaAORHs1Twqrz/oCr45goz5HZBUOBzzCjTTb1kb5v44XPxgNvCot/Qv3jEznqmEN24imVN1Ip57NM2qqf1yUC1ggfB8iqHUlQux0DSgHN9sy5I40pMbx8tR6NDNF7BmrFLXrOvUrMgM2WMAuWp2rROhGAyhB9zkircT5bXzneLfi3kNe2R+g/OQzQB1dzFyR1SGN0leVJgrB6AyMIQGz0lYcfDxiKRlQAAkvybCDoQB4V9VV/DSd5SLg5RszkadxmefgTd8r85rlnBuRzBnht5MwTshxkwU7p9AxYJfyjKcg6TpuSYN7w0IUnR+P/je6DYVJ3sIe+Am7UR4Jv8bEyro2ChkEq7OGybAS1dz9GXrkSxC1z+h1Pjr+OPfl6CSbYaoUXKlxmZbSxdJ0XmWokxsnj+OZHRnbg0SHfeqA3SLVPG/mF78YiiLqgYBkzXouR7lV6ampHg77heISplPJy2aLTcoYUrhM1AupFe1LmDChhJ+9aRiQkPsH77sVrEZ6zV8abg1GvmWv5CnV/DRSDsVzlD3vcxbdn6XkDSXlG22BbGoLO1/ULe74UhgLCrG/QJFOs40HqmN3f83RPBZh6IVpLmoGtqoBxFeo6sgCA/pgwHnsK+DQ2sOrY3tw2Veg/KgyUpGTXEX1lqU4vLByhB0Quz4j1TLGBpkOEe2+TwLeTcomJIsBDOHpfcwk/F52JMC9Dah4vKDdhevsz5ScU4Hm33TfgunjhTn9yCf976h8TztGowmvUWQh1OCsFG+dg9kbaGHjGWfAWA/Q61VPTuEsamkn5i4K7S5nhUbxib36xJ6A06gu0SD/Cc6oW5/PyFDGXGy2XShYw+ofWEmFWIUiDk3pyifZKKAHqAB9hu/p33jr9tlxt26vHwDoK75KK0DsBVyBQzZRuGO33vyzj1IeMl6R07v5/cMUO6WJZ69ua7aDV59cz5hXi+HLYqUAxAONQNFo+uyWPUXnJkdP3ltOOEZlZdkZ/hLs3Q+C4AYM7p2PzQP2kxJf/2yrisvrpcqwzPYLZMOy8LnP8eZgT1f8yfLx+Lx0BRaOBgdmEjp3/pB6RjdpGCNeF7y7VZmZjgj4F0VODEIjiI1gCMbVLQv655MBQBXwiv+MTxDPZKs9XPO2WXeQe/DLU/MrIRkK26ybKr8zHCEo75GYommddKfz6TooOUhLUSshh3FntTg7O6Q6WMuPltmSwi4KH/aBanvcBj7YcP34apEZVKKgbBqrX+oOeuNCdUjaN16YJbT X-OriginatorOrg: amd.com X-MS-Exchange-CrossTenant-Network-Message-Id: cb5c4f32-e8b2-44ba-5290-08d90a581000 X-MS-Exchange-CrossTenant-AuthSource: SN6PR12MB4623.namprd12.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 28 Apr 2021 15:12:39.2581 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: eTbQfk4jD4YOs3it6LU+0u5P3O8Q4eEHBYUqjg/jXwm7i5k1JZeW0e1ZoOjuT6xPFC58lruClwY+iUPGUNVQYQ== X-MS-Exchange-Transport-CrossTenantHeadersStamped: SA0PR12MB4462 X-BeenThere: dri-devel@lists.freedesktop.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Direct Rendering Infrastructure - Development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: gregkh@linuxfoundation.org, Felix.Kuehling@amd.com, helgaas@kernel.org, Alexander.Deucher@amd.com Errors-To: dri-devel-bounces@lists.freedesktop.org Sender: "dri-devel" Helps to expdite HW related stuff to amdgpu_pci_remove Signed-off-by: Andrey Grodzovsky --- drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd.c | 2 +- drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd.h | 2 +- drivers/gpu/drm/amd/amdkfd/kfd_device.c | 3 ++- 3 files changed, 4 insertions(+), 3 deletions(-) diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd.c index c5343a5eecbe..9edb35ba181b 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd.c +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd.c @@ -169,7 +169,7 @@ void amdgpu_amdkfd_device_init(struct amdgpu_device *adev) } } -void amdgpu_amdkfd_device_fini(struct amdgpu_device *adev) +void amdgpu_amdkfd_device_fini_sw(struct amdgpu_device *adev) { if (adev->kfd.dev) { kgd2kfd_device_exit(adev->kfd.dev); diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd.h b/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd.h index a81d9cacf9b8..c51001602a68 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd.h +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd.h @@ -126,7 +126,7 @@ void amdgpu_amdkfd_interrupt(struct amdgpu_device *adev, const void *ih_ring_entry); void amdgpu_amdkfd_device_probe(struct amdgpu_device *adev); void amdgpu_amdkfd_device_init(struct amdgpu_device *adev); -void amdgpu_amdkfd_device_fini(struct amdgpu_device *adev); +void amdgpu_amdkfd_device_fini_sw(struct amdgpu_device *adev); int amdgpu_amdkfd_submit_ib(struct kgd_dev *kgd, enum kgd_engine_type engine, uint32_t vmid, uint64_t gpu_addr, uint32_t *ib_cmd, uint32_t ib_len); diff --git a/drivers/gpu/drm/amd/amdkfd/kfd_device.c b/drivers/gpu/drm/amd/amdkfd/kfd_device.c index 72c893fff61a..1bb8bc6d85f5 100644 --- a/drivers/gpu/drm/amd/amdkfd/kfd_device.c +++ b/drivers/gpu/drm/amd/amdkfd/kfd_device.c @@ -833,10 +833,11 @@ bool kgd2kfd_device_init(struct kfd_dev *kfd, return kfd->init_complete; } + + void kgd2kfd_device_exit(struct kfd_dev *kfd) { if (kfd->init_complete) { - kgd2kfd_suspend(kfd, false); device_queue_manager_uninit(kfd->dqm); kfd_interrupt_exit(kfd); kfd_topology_remove_device(kfd); From patchwork Wed Apr 28 15:11:45 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrey Grodzovsky X-Patchwork-Id: 12229237 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.5 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,MSGID_FROM_MTA_HEADER,SPF_HELO_NONE,SPF_PASS, USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id EA6E2C433ED for ; Wed, 28 Apr 2021 15:13:04 +0000 (UTC) Received: from gabe.freedesktop.org (gabe.freedesktop.org [131.252.210.177]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 95F3461434 for ; Wed, 28 Apr 2021 15:13:04 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 95F3461434 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=amd.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=dri-devel-bounces@lists.freedesktop.org Received: from gabe.freedesktop.org (localhost [127.0.0.1]) by gabe.freedesktop.org (Postfix) with ESMTP id 4D9076EB57; Wed, 28 Apr 2021 15:13:03 +0000 (UTC) Received: from NAM12-DM6-obe.outbound.protection.outlook.com (mail-dm6nam12on2054.outbound.protection.outlook.com [40.107.243.54]) by gabe.freedesktop.org (Postfix) with ESMTPS id 431516EB56; Wed, 28 Apr 2021 15:12:46 +0000 (UTC) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=cFwRuG5tkViwEfW4DPgxupIF+9EaPw2KvL7exd/wz3z5kWl6b2o3jQ1Q5E7ZgpU93XL2UCKU7TsoO3RiR+1XWp3+sjbXlt9rylXUKsFjspy/YHVMPLeZazgzvNmQWgn0EeE0wbBrB8ksgUQ+to4ndEMRGPa28Y6wLWIynyA7B89qZBM4l3ZOzQAobdyCntDvzdt5IHJ3wqnfb5ZMvxT4Leb5l8MSybV0mcVB99DX/KjQX+u/0Yf3dUHL6e3ZGKBo020KSvwJh+1dqF/UcnRpVjyn+V82er/d1rKzETFPEpNAOZ2Ouz1OGM3sDZtiCbrjE9zgxnxntsgapTQZWBFYow== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=6kl0aLQAmNahoOIDFtRxdbIzMVzZ6VQMzp8rC5XcQuQ=; b=hRkuR2tNeCqShA2L/r+LwoH/jv4RccJvIsnQ9dcPXUgcasor9JjzgWUVD9GtkOaW5uB7l++bhLYA3xdXduj6HoIEwGKTf5SsD+48+Wola9rMvQU5zGvVOabM7vro0TQcpsd0h/OtL/lJrl3UMHUqWhIsPy3ubUk3jwzOKCD75hjvNKgzzKEVDXVFTpdcIuNDD69xQ6dXy0BuGKXXu0pK/WTq0YCo2LCxHg6g4jfmN0cqPjiYPiJT2IaxE5sU9Bm8PKwZliYtv58pS0MVlB9rlgdRYwYQcnlj8bULOcpbLfCurzD1yD0vF4cAin0we6H1g3rjlgZWKuNJOYzbpaCZOA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=amd.com; dmarc=pass action=none header.from=amd.com; dkim=pass header.d=amd.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=6kl0aLQAmNahoOIDFtRxdbIzMVzZ6VQMzp8rC5XcQuQ=; b=i2ucHVk2Epuy1kTta89dpRqxHts+hGnN8fTxneM/wOYUpY5a72rMsgw65WIt6fIGIwkXXUltu1BOEDzSP+pUpKgG7NVzjwyYe66PR3LCZGt7IU9moE35ehsFDEXoqCAoTcvvZKk8MHOywVwSWGS5Roms+QXwRjZihU/abG47GO0= Authentication-Results: lists.freedesktop.org; dkim=none (message not signed) header.d=none; lists.freedesktop.org; dmarc=none action=none header.from=amd.com; Received: from SN6PR12MB4623.namprd12.prod.outlook.com (2603:10b6:805:e9::17) by SA0PR12MB4462.namprd12.prod.outlook.com (2603:10b6:806:95::22) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4065.23; Wed, 28 Apr 2021 15:12:41 +0000 Received: from SN6PR12MB4623.namprd12.prod.outlook.com ([fe80::ad51:8c49:b171:856c]) by SN6PR12MB4623.namprd12.prod.outlook.com ([fe80::ad51:8c49:b171:856c%7]) with mapi id 15.20.4065.026; Wed, 28 Apr 2021 15:12:41 +0000 From: Andrey Grodzovsky To: dri-devel@lists.freedesktop.org, amd-gfx@lists.freedesktop.org, linux-pci@vger.kernel.org, ckoenig.leichtzumerken@gmail.com, daniel.vetter@ffwll.ch, Harry.Wentland@amd.com Subject: [PATCH v5 05/27] drm/amdgpu: Add early fini callback Date: Wed, 28 Apr 2021 11:11:45 -0400 Message-Id: <20210428151207.1212258-6-andrey.grodzovsky@amd.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210428151207.1212258-1-andrey.grodzovsky@amd.com> References: <20210428151207.1212258-1-andrey.grodzovsky@amd.com> X-Originating-IP: [2607:fea8:3edf:49b0:7212:f93a:73b0:8f23] X-ClientProxiedBy: YT1PR01CA0142.CANPRD01.PROD.OUTLOOK.COM (2603:10b6:b01:2f::21) To SN6PR12MB4623.namprd12.prod.outlook.com (2603:10b6:805:e9::17) MIME-Version: 1.0 X-MS-Exchange-MessageSentRepresentingType: 1 Received: from agrodzovsky-All-Series.hitronhub.home (2607:fea8:3edf:49b0:7212:f93a:73b0:8f23) by YT1PR01CA0142.CANPRD01.PROD.OUTLOOK.COM (2603:10b6:b01:2f::21) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4065.25 via Frontend Transport; Wed, 28 Apr 2021 15:12:40 +0000 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: d0fc8a25-58e5-4a7a-95e0-08d90a58111c X-MS-TrafficTypeDiagnostic: SA0PR12MB4462: X-MS-Exchange-Transport-Forked: True X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:5797; X-MS-Exchange-SenderADCheck: 1 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: qL7hocYNB6vrMdupSfiC6WWT25F4GgtMoowe8B57imH2uKW8CFvEcfEMTD2Tm8JlZ+dAcek/prPVjJ8P4wkZEeN5kJPHimS9hjGjuDdnm7IWWGlV2yNC49d3Mpn2uP5Rz2KlVtv3i7C+eD+BizWeCM1HgReQSKzXOz4eV5IR1IOQI0/jWm2TG14bpTEcaZNpMnfJD40OqzO+B/3MVOeLeyrPP88Eyd/RpcTBkxIuujIS1FJAbIHMCMavC2fy4MJzQe9RnkfPA8gMwyF07kwEyZf4xztq3H5Bsc57eyhp5RmSyufMkQg65UENZS1AnH3g0HSyRppMOjlRcShpkryhii404Mp1o1P5zEH6g01Ooa05AdY95rcVJdvRRCI9pxHTJhqryZU74ZsxZ02zDchxxMOUUcBRg6KmeE802B/mJVYhLz2KebWp+1clV8MBUFQFDnOYVilcrZKdGBAGYWTjC8/16mQWgzap0lnqenxiCb0myRxwUlsjgaswUgWbGjK/FEsg+oKRpgDQVo4FRZSt/69HHAJ887lKTRmcgF1nCz8ToegQGFS0HcI+iH4yq4wPuacLvu0UL3mYXdKNz7uJhQ== X-Forefront-Antispam-Report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:SN6PR12MB4623.namprd12.prod.outlook.com; PTR:; CAT:NONE; SFS:(4636009)(346002)(396003)(136003)(39860400002)(376002)(366004)(6512007)(6636002)(66946007)(316002)(186003)(44832011)(2906002)(16526019)(36756003)(83380400001)(6506007)(4326008)(52116002)(8676002)(66476007)(5660300002)(66556008)(1076003)(2616005)(478600001)(86362001)(38100700002)(8936002)(6486002); DIR:OUT; SFP:1101; X-MS-Exchange-AntiSpam-MessageData: JFeIrK+G8d8IYM7zfB0yaUZyh26Uq86Afb1MaPSyEhJQZK607Ew/JNrpxGdbFrYytUIT81vH+cFG7icQIAFfzo5VNbaOVQ6iKjV01WhhOnoa0IBgNXD7YpfNhH+jis5HjPsJhhhx1xfUKUU4vdVTKs6VXTwRsWrLyQjE39afSQnaUUwZ5L5h6Yeh4z8ApUy0/lyokX1zlxRjoK7amkcoxrUOYm7/4V1WxxKLgE2ezp37jVnmA4+YQGfq/s3LR0g5mCAEBML/f8cB2GUsPac3zUYqthCQE+J2dVJB3Bk54K+ubZLNfov5O85Y5XjTn5co/0VWXi4zyMcTysGufJPWrSgK2gJkdqNIp7AYSwiH4hQhGOqu8ou8YHmx3YFWqX4JGl7vtVJf+FaQSGnzLrjM8Rt6BDLFkFHNZQdE5MNzRwPAnO0SD58jEntdJske7wKd849DJt+qzDhnGmeyCVZAqeyDGHdBJsTsJ7++Ge0ARd8l5U3csjYIkRn8eaQf5KB4DgL5LOwGokqXhLRyMH9W/HWBcGRgz307E3ZaOZQYqQmDordilh0VPnQB+0DriQh8vwSAoaOR87Y/lloJKwRQZeEdbJQurvZ0B/hW+3dTfYE8MGqfXWWOM09oBAwXFbGPtRbqvp/Yk4mxzptPvBvrgDP6sUpxGYpqiYVNSEFbJjl6VG1RzlHwywhv0KmwtiOOa5ASqN4WcoqszaYvc5w1iGOMm08SUTOO2m+S/doBriOFmAIeqdEIo0+1ukikVC/rjPUryTh1pyVKypSTftBBNWOD+O7OtiLIbBFnovMYmXWTeIK4pz0xE0JeVlDGohHwmhyc15ng+fpGWhs9F1OUpDS8Y/9uxNFV2VnvS3dEhtkgYnYWoBA/Z2mwABqXz6g6++2dglS3/C8ZCOi/h1QJl2ojCIG3zSAasSfa+uX7bK5qxbJPGpRiFbWLXZ8nK0p9r3Ymn5bKH43fmGu/t/rUiaYjZHzVcecUj9CZiFUqP5Lb1QPEWwNnr32ariIAVbkmctURLG0SxCg75LvZe57x7aopfM9wYn/EWL7MlmnuE0kO8yfayAqGQaczOLI8TQXFFek3mLO0nDoXVIVgPjZ0i3gVLZ/dJpN6SBzLe/tcOALxckS80B4Vn2eWSCbkGIR4w4cuWeiS1ttveSZdd5Z2XzgZaXzq6YT6ynIc6P9P3eobrqv1dKXpS7NQ6DcWvO6sRgQ46Y8x5WTeg8DrqT6etLhYEoRlEihl8tewrCZMVf9qXb1jNXtpjozZhM0pK906DX8PHATtF1B/ACiZ1wUmsKqJ2w6v+ERRq+WzWi1oGkpQq3PlfoUXSEzKNZ4X0B5CduEdvSSvoMW4q9CdT5Zt1I6XTCvm6LRICahwPbpjKoYnMjK/tVUpaVhou90FID8t X-OriginatorOrg: amd.com X-MS-Exchange-CrossTenant-Network-Message-Id: d0fc8a25-58e5-4a7a-95e0-08d90a58111c X-MS-Exchange-CrossTenant-AuthSource: SN6PR12MB4623.namprd12.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 28 Apr 2021 15:12:41.1101 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: kI4PxMbOtOqC2go+6ZkktfQ/M22LZcgUocW4HbRFhdN1EUSvA/7gvl+6tqFoXKgsN2ODP5pfHz2DdyunPi/Ixg== X-MS-Exchange-Transport-CrossTenantHeadersStamped: SA0PR12MB4462 X-BeenThere: dri-devel@lists.freedesktop.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Direct Rendering Infrastructure - Development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: gregkh@linuxfoundation.org, Felix.Kuehling@amd.com, helgaas@kernel.org, Alexander.Deucher@amd.com Errors-To: dri-devel-bounces@lists.freedesktop.org Sender: "dri-devel" Use it to call disply code dependent on device->drv_data before it's set to NULL on device unplug v5: Move HW finilization into this callback to prevent MMIO accesses post cpi remove. Signed-off-by: Andrey Grodzovsky --- drivers/gpu/drm/amd/amdgpu/amdgpu_device.c | 59 +++++++++++++------ .../gpu/drm/amd/display/amdgpu_dm/amdgpu_dm.c | 12 +++- drivers/gpu/drm/amd/include/amd_shared.h | 2 + 3 files changed, 52 insertions(+), 21 deletions(-) diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c index 8d22b79fc1cd..46d646c40338 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c @@ -2536,34 +2536,26 @@ static int amdgpu_device_ip_late_init(struct amdgpu_device *adev) return 0; } -/** - * amdgpu_device_ip_fini - run fini for hardware IPs - * - * @adev: amdgpu_device pointer - * - * Main teardown pass for hardware IPs. The list of all the hardware - * IPs that make up the asic is walked and the hw_fini and sw_fini callbacks - * are run. hw_fini tears down the hardware associated with each IP - * and sw_fini tears down any software state associated with each IP. - * Returns 0 on success, negative error code on failure. - */ -static int amdgpu_device_ip_fini(struct amdgpu_device *adev) +static int amdgpu_device_ip_fini_early(struct amdgpu_device *adev) { int i, r; - if (amdgpu_sriov_vf(adev) && adev->virt.ras_init_done) - amdgpu_virt_release_ras_err_handler_data(adev); + for (i = 0; i < adev->num_ip_blocks; i++) { + if (!adev->ip_blocks[i].version->funcs->early_fini) + continue; - amdgpu_ras_pre_fini(adev); + r = adev->ip_blocks[i].version->funcs->early_fini((void *)adev); + if (r) { + DRM_DEBUG("early_fini of IP block <%s> failed %d\n", + adev->ip_blocks[i].version->funcs->name, r); + } + } - if (adev->gmc.xgmi.num_physical_nodes > 1) - amdgpu_xgmi_remove_device(adev); + amdgpu_amdkfd_suspend(adev, false); amdgpu_device_set_pg_state(adev, AMD_PG_STATE_UNGATE); amdgpu_device_set_cg_state(adev, AMD_CG_STATE_UNGATE); - amdgpu_amdkfd_device_fini(adev); - /* need to disable SMC first */ for (i = 0; i < adev->num_ip_blocks; i++) { if (!adev->ip_blocks[i].status.hw) @@ -2594,6 +2586,33 @@ static int amdgpu_device_ip_fini(struct amdgpu_device *adev) adev->ip_blocks[i].status.hw = false; } + return 0; +} + +/** + * amdgpu_device_ip_fini - run fini for hardware IPs + * + * @adev: amdgpu_device pointer + * + * Main teardown pass for hardware IPs. The list of all the hardware + * IPs that make up the asic is walked and the hw_fini and sw_fini callbacks + * are run. hw_fini tears down the hardware associated with each IP + * and sw_fini tears down any software state associated with each IP. + * Returns 0 on success, negative error code on failure. + */ +static int amdgpu_device_ip_fini(struct amdgpu_device *adev) +{ + int i, r; + + if (amdgpu_sriov_vf(adev) && adev->virt.ras_init_done) + amdgpu_virt_release_ras_err_handler_data(adev); + + amdgpu_ras_pre_fini(adev); + + if (adev->gmc.xgmi.num_physical_nodes > 1) + amdgpu_xgmi_remove_device(adev); + + amdgpu_amdkfd_device_fini_sw(adev); for (i = adev->num_ip_blocks - 1; i >= 0; i--) { if (!adev->ip_blocks[i].status.sw) @@ -3624,6 +3643,8 @@ void amdgpu_device_fini_hw(struct amdgpu_device *adev) amdgpu_fbdev_fini(adev); amdgpu_irq_fini_hw(adev); + + amdgpu_device_ip_fini_early(adev); } void amdgpu_device_fini_sw(struct amdgpu_device *adev) diff --git a/drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm.c b/drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm.c index 55e39b462a5e..c0b9abb773a4 100644 --- a/drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm.c +++ b/drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm.c @@ -1170,6 +1170,15 @@ static int amdgpu_dm_init(struct amdgpu_device *adev) return -EINVAL; } +static int amdgpu_dm_early_fini(void *handle) +{ + struct amdgpu_device *adev = (struct amdgpu_device *)handle; + + amdgpu_dm_audio_fini(adev); + + return 0; +} + static void amdgpu_dm_fini(struct amdgpu_device *adev) { int i; @@ -1178,8 +1187,6 @@ static void amdgpu_dm_fini(struct amdgpu_device *adev) drm_encoder_cleanup(&adev->dm.mst_encoders[i].base); } - amdgpu_dm_audio_fini(adev); - amdgpu_dm_destroy_drm_device(&adev->dm); #ifdef CONFIG_DRM_AMD_DC_HDCP @@ -2194,6 +2201,7 @@ static const struct amd_ip_funcs amdgpu_dm_funcs = { .late_init = dm_late_init, .sw_init = dm_sw_init, .sw_fini = dm_sw_fini, + .early_fini = amdgpu_dm_early_fini, .hw_init = dm_hw_init, .hw_fini = dm_hw_fini, .suspend = dm_suspend, diff --git a/drivers/gpu/drm/amd/include/amd_shared.h b/drivers/gpu/drm/amd/include/amd_shared.h index 43ed6291b2b8..1ad56da486e4 100644 --- a/drivers/gpu/drm/amd/include/amd_shared.h +++ b/drivers/gpu/drm/amd/include/amd_shared.h @@ -240,6 +240,7 @@ enum amd_dpm_forced_level; * @late_init: sets up late driver/hw state (post hw_init) - Optional * @sw_init: sets up driver state, does not configure hw * @sw_fini: tears down driver state, does not configure hw + * @early_fini: tears down stuff before dev detached from driver * @hw_init: sets up the hw state * @hw_fini: tears down the hw state * @late_fini: final cleanup @@ -268,6 +269,7 @@ struct amd_ip_funcs { int (*late_init)(void *handle); int (*sw_init)(void *handle); int (*sw_fini)(void *handle); + int (*early_fini)(void *handle); int (*hw_init)(void *handle); int (*hw_fini)(void *handle); void (*late_fini)(void *handle); From patchwork Wed Apr 28 15:11:46 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrey Grodzovsky X-Patchwork-Id: 12229257 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.5 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,MSGID_FROM_MTA_HEADER,SPF_HELO_NONE,SPF_PASS, USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 869E9C4360C for ; Wed, 28 Apr 2021 15:13:46 +0000 (UTC) Received: from gabe.freedesktop.org (gabe.freedesktop.org [131.252.210.177]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 270F56143A for ; Wed, 28 Apr 2021 15:13:46 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 270F56143A Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=amd.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=dri-devel-bounces@lists.freedesktop.org Received: from gabe.freedesktop.org (localhost [127.0.0.1]) by gabe.freedesktop.org (Postfix) with ESMTP id 044436EB83; Wed, 28 Apr 2021 15:13:45 +0000 (UTC) Received: from NAM12-DM6-obe.outbound.protection.outlook.com (mail-dm6nam12on2054.outbound.protection.outlook.com [40.107.243.54]) by gabe.freedesktop.org (Postfix) with ESMTPS id AE16C6EB57; Wed, 28 Apr 2021 15:12:46 +0000 (UTC) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=XJOre0fKxQ5qErjLk4lUTm6seVSqiqSisuKZy3/JOcJbUU1pH9474Obn/ItMl8ackmlgBxh8bxbdIUtDAM1sQftw7IcLaYDQqJsTCAxdtb+61J91DkWH8KLE7+rU3q+pS/sCJczSRhC/VnxViKn50l0MjAnGHnPzpraRVnMgN7aAZPNbLFxrpAotV70oSu1QHQOQAkGFNHls6Ez1n+ZjJoI9j/eGuLpOnT553GRhi9xLemdB+EpJ8qS2k/jVm/7Hk5AjQQO3rDy/kTk8NCVyT1meWZ6tp3VXUOe64KYXuwYe6LgGF3GOVAZGA0SgftjO1NUaVm37/2nTBlRJqpnz9A== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=Zm9+Re3MVZqxQaS+3rJ45V5zTs3+cQ6nUJPLgZ05SgI=; b=LFCKMRxgkDDzlVOiWnltGAVf4ugirUIFUMOiGAc51biUawYsGGUOm8UtigK6PUmtStNQhIDPGggQFvokgiPfhS1h6356+B2wXSnYhko2DfqzPmdtHpmJSQxhfDI0SwjD6J/Ak2CrTKCMyschemdz8AXGkpn9VyWyeObW9AV5NrLUlXZEo4S12paSbvOX/PFlx3TW8ZOTlFGYVXNNnwTMU9i476dYXvuJVyPblGem6W4vlz2PT+TfUx427pBOO9Dxn/+DJK5rz9oP0AoeA7s1HP2xkgdI+C8viq9tX2dnXv1WAYkxtbKcm0Myrvl71TtF0QrvKxBjNwx59wIse3q3Mg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=amd.com; dmarc=pass action=none header.from=amd.com; dkim=pass header.d=amd.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=Zm9+Re3MVZqxQaS+3rJ45V5zTs3+cQ6nUJPLgZ05SgI=; b=PXRB0idEqdEsFf5iM6cpeeOBO8pX5UAjCelg1ns6XT0z491ES79neEtO5bs5d0XZw5Au9+ltUputpno438ytrRrmAc76qqYfCHbHkvPnibYEr7PixR3sIWdWViaoG18DSGH8/0Xy6+fcQNaNIuANjKSW11m9Q3LUPDJ1C6s278A= Authentication-Results: lists.freedesktop.org; dkim=none (message not signed) header.d=none; lists.freedesktop.org; dmarc=none action=none header.from=amd.com; Received: from SN6PR12MB4623.namprd12.prod.outlook.com (2603:10b6:805:e9::17) by SA0PR12MB4462.namprd12.prod.outlook.com (2603:10b6:806:95::22) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4065.23; Wed, 28 Apr 2021 15:12:42 +0000 Received: from SN6PR12MB4623.namprd12.prod.outlook.com ([fe80::ad51:8c49:b171:856c]) by SN6PR12MB4623.namprd12.prod.outlook.com ([fe80::ad51:8c49:b171:856c%7]) with mapi id 15.20.4065.026; Wed, 28 Apr 2021 15:12:42 +0000 From: Andrey Grodzovsky To: dri-devel@lists.freedesktop.org, amd-gfx@lists.freedesktop.org, linux-pci@vger.kernel.org, ckoenig.leichtzumerken@gmail.com, daniel.vetter@ffwll.ch, Harry.Wentland@amd.com Subject: [PATCH v5 06/27] drm/amdgpu: Handle IOMMU enabled case. Date: Wed, 28 Apr 2021 11:11:46 -0400 Message-Id: <20210428151207.1212258-7-andrey.grodzovsky@amd.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210428151207.1212258-1-andrey.grodzovsky@amd.com> References: <20210428151207.1212258-1-andrey.grodzovsky@amd.com> X-Originating-IP: [2607:fea8:3edf:49b0:7212:f93a:73b0:8f23] X-ClientProxiedBy: YT1PR01CA0142.CANPRD01.PROD.OUTLOOK.COM (2603:10b6:b01:2f::21) To SN6PR12MB4623.namprd12.prod.outlook.com (2603:10b6:805:e9::17) MIME-Version: 1.0 X-MS-Exchange-MessageSentRepresentingType: 1 Received: from agrodzovsky-All-Series.hitronhub.home (2607:fea8:3edf:49b0:7212:f93a:73b0:8f23) by YT1PR01CA0142.CANPRD01.PROD.OUTLOOK.COM (2603:10b6:b01:2f::21) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4065.25 via Frontend Transport; Wed, 28 Apr 2021 15:12:41 +0000 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 741b2d6c-1721-4dc0-2e38-08d90a581213 X-MS-TrafficTypeDiagnostic: SA0PR12MB4462: X-MS-Exchange-Transport-Forked: True X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:3968; X-MS-Exchange-SenderADCheck: 1 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: 0oxsyd3X2WBWWhb85203+iVuphChFTL92+PuRmRCci3O80x60NwvddIuqnhagi8la86dCjcfU0ljXLhDh3ofTJsVlVV4uvixSAiHB+Sk04sGLrgGa34Dj77l7wwmuULxL//Db1d6ZvIW9Lv1AzwOAIibcqoU1yVThy+/cn1XhhtM2ibo0jyEoTSGeW2+YvSF2s6NJHuwiy5XODSGlUAB2O+w3+PcNaIPnE4wLoMMlhylkoDpKS6nQ9u6IV10PMBluPXGYGv18ePuO9KRYII4vz7ckSmTjZgPKpAuFVtLTseKVUJpthRBYilm9cB8mrUOj/0sYLanaphWlzDPVLEszJ4Ss99VjtbFkdaknr+d+0AHivw2lJJDp/ujC1ewmDJJJAZCL4rkP1CDKTo3t0S9oHl6ZUYVyCM2vBQOyabF781dbPS8eCcl36OSppwWbtJTBAG+B/t/93hyknggutB0cP0YgWBzqstaWZsOcu36PPobIzp6q33fLHezPBzDAOknvNgno8SI80b7W+FtgYxVrO0ticnLhghRbkdmgLHEzPl3mYAt0Z92cBjbxiX+E/xEnQQlVSmSb/8znhN6Tx6jz4Galk1u/NPhfjXpKikUWu8= X-Forefront-Antispam-Report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:SN6PR12MB4623.namprd12.prod.outlook.com; PTR:; CAT:NONE; SFS:(4636009)(346002)(396003)(136003)(39860400002)(376002)(366004)(6512007)(6636002)(66946007)(316002)(186003)(44832011)(2906002)(16526019)(36756003)(83380400001)(6506007)(4326008)(52116002)(8676002)(66476007)(5660300002)(66556008)(1076003)(2616005)(478600001)(30864003)(86362001)(38100700002)(8936002)(6486002); DIR:OUT; SFP:1101; X-MS-Exchange-AntiSpam-MessageData: 6oFukTA0+165jaG6WQTqLY8YF+iDPmxTsDp5d4HduRGt5wmUk5MA43KadX3kOdYqxQe0rycg583nQ8p9BYLO9j0WOOfzc9Zj8gjqT1cO/xuGa2x+D4RS3+Bt3U8WCdU4YvJNPjLIsUD955Qxd8f8lyW0kgMtmcxeWDo2ogu7zxw7JtnA5KD4GykWIaptKBBXRjGEvW5+pMhtsGX4yKL+QrC+lvqe6Upe3UjT4GHV1EvohHB2G8MbMUL9toX8SbfdiAKOriZ1WxlE3mjdRLnMaEq3c0rzKk7Ft6wJ/yh1QAIjh1vphTOOgwGw3kir+juqSzD650mGgzYU6mW3VwcpEBtiiRSS66NH9P+ukbACm+g9D7ustxb9xqkI8HohtUe2rOcqQ2BpjxE6HBm2XqlJUb+l56BGbsPOsg5hNeMu+m88G6Ut//lXC6prjLAiN7PsE4CvWFbbXPSRLDBnofghmXcOoA0qHBGEfciYFjmPujFh/i7m+O2JoJ9C/0zLMDv1c8g4C+Whnft/S4fdWDgSiftsWGUA+8czvhX0oCO/z78fuEHFwXoHNa4n+AkA+fOH08cjXMDDWpD3eWckZwdXlUbFpymSc9gkT5PZhT9GUfq/ocisIel7Y17dYexVgDyrcE/XmHamc2ykos1JCbRA+XmlS6j6U/x/vi5Ld4XRg2GxmQC4GalS0yRAAUYQpVZcLHtNkjXX/dNf3viXvg+JsIMj1xbmbBeM/tJIkzlvSd7EoFVAoHduk4LFkUmn9xFlQgbveYvUa8YK1sk90pMSY52Kwm8XUkYeHRkIfey1yVZyXbgPRrEgnD19u8l/VKF6iwONvR4LYAK4DyRrCJN0xDY/FZ194AZ3v9leduQBXxbLgAIcFY7PNarxyFXtgBClSii9WHU0N8zmwY5Sy3zd5PUHwFm4unizpULUiq/SFlfyAcojlDDXKg/gedK8FyCfLa0XJKSLLq9bKrsrOLaAFemYlomVO1ch6YDH69NFYF89dwnCTr/dVmdk77TZfqcfZM9vkq5sF0HHFX5PQl260EZUbJgXVbqdSFeJtQqmgYfLWiIPq9gluXH1GhbA1tD9FdgIbAnSwCj/Mm8gsgkDC+ybl1ycKGqlAJLs+vNWpRCkjF46bHmuZihM66cLO0WzFrhf+BNUsjvpooP/yiDU3jZjJUm1qIJH0STmNFE1TTQQO0z5ttx5sp6fZNL+XRel2tP3h7LWijEt4uEQKYnNa9rV6XLXzJ/51H+yLWeESzmv+YW89JIgsJ/2rKIw/D5NxS2FcbYQFFJWsJPsBCRs310dht6036+8cr7F+JMiyAtfhVhumUj+zT6AQSBE2YboEm09+JKf/ygJ2O6UunwLUPBOa1MPG4UssvcsjEEYRVTi+KCR3gUQgUOsNizoKcfz X-OriginatorOrg: amd.com X-MS-Exchange-CrossTenant-Network-Message-Id: 741b2d6c-1721-4dc0-2e38-08d90a581213 X-MS-Exchange-CrossTenant-AuthSource: SN6PR12MB4623.namprd12.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 28 Apr 2021 15:12:42.7621 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: WusCMnsELmbQTGD0cIxuQQM1S1yZmRWgyezUjXmyEyG+dWrgCPV9t+rNJ1WiJbK7OyrEV89q3OuxwekqRocmOg== X-MS-Exchange-Transport-CrossTenantHeadersStamped: SA0PR12MB4462 X-BeenThere: dri-devel@lists.freedesktop.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Direct Rendering Infrastructure - Development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: gregkh@linuxfoundation.org, Felix.Kuehling@amd.com, helgaas@kernel.org, Alexander.Deucher@amd.com Errors-To: dri-devel-bounces@lists.freedesktop.org Sender: "dri-devel" Handle all DMA IOMMU gropup related dependencies before the group is removed. v5: Drop IOMMU notifier and switch to lockless call to ttm_tt_unpopulate Signed-off-by: Andrey Grodzovsky --- drivers/gpu/drm/amd/amdgpu/amdgpu.h | 2 ++ drivers/gpu/drm/amd/amdgpu/amdgpu_device.c | 31 ++++++++++++++++++++-- drivers/gpu/drm/amd/amdgpu/amdgpu_gart.c | 3 +-- drivers/gpu/drm/amd/amdgpu/amdgpu_gart.h | 1 + drivers/gpu/drm/amd/amdgpu/amdgpu_irq.c | 9 +++++++ drivers/gpu/drm/amd/amdgpu/amdgpu_object.c | 13 ++++++++- drivers/gpu/drm/amd/amdgpu/amdgpu_object.h | 2 ++ drivers/gpu/drm/amd/amdgpu/cik_ih.c | 1 - drivers/gpu/drm/amd/amdgpu/cz_ih.c | 1 - drivers/gpu/drm/amd/amdgpu/iceland_ih.c | 1 - drivers/gpu/drm/amd/amdgpu/navi10_ih.c | 3 --- drivers/gpu/drm/amd/amdgpu/si_ih.c | 1 - drivers/gpu/drm/amd/amdgpu/tonga_ih.c | 1 - drivers/gpu/drm/amd/amdgpu/vega10_ih.c | 3 --- 14 files changed, 56 insertions(+), 16 deletions(-) diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu.h b/drivers/gpu/drm/amd/amdgpu/amdgpu.h index fddb82897e5d..30a24db5f4d1 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu.h +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu.h @@ -1054,6 +1054,8 @@ struct amdgpu_device { bool in_pci_err_recovery; struct pci_saved_state *pci_state; + + struct list_head device_bo_list; }; static inline struct amdgpu_device *drm_to_adev(struct drm_device *ddev) diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c index 46d646c40338..91594ddc2459 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c @@ -70,6 +70,7 @@ #include #include + MODULE_FIRMWARE("amdgpu/vega10_gpu_info.bin"); MODULE_FIRMWARE("amdgpu/vega12_gpu_info.bin"); MODULE_FIRMWARE("amdgpu/raven_gpu_info.bin"); @@ -3211,7 +3212,6 @@ static const struct attribute *amdgpu_dev_attributes[] = { NULL }; - /** * amdgpu_device_init - initialize the driver * @@ -3316,6 +3316,8 @@ int amdgpu_device_init(struct amdgpu_device *adev, INIT_WORK(&adev->xgmi_reset_work, amdgpu_device_xgmi_reset_func); + INIT_LIST_HEAD(&adev->device_bo_list); + adev->gfx.gfx_off_req_count = 1; adev->pm.ac_power = power_supply_is_system_supplied() > 0; @@ -3601,6 +3603,28 @@ int amdgpu_device_init(struct amdgpu_device *adev, return r; } +static void amdgpu_clear_dma_mappings(struct amdgpu_device *adev) +{ + struct amdgpu_bo *bo = NULL; + + /* + * Unmaps all DMA mappings before device will be removed from it's + * IOMMU group otherwise in case of IOMMU enabled system a crash + * will happen. + */ + + spin_lock(&adev->mman.bdev.lru_lock); + while (!list_empty(&adev->device_bo_list)) { + bo = list_first_entry(&adev->device_bo_list, struct amdgpu_bo, bo); + list_del_init(&bo->bo); + spin_unlock(&adev->mman.bdev.lru_lock); + if (bo->tbo.ttm) + ttm_tt_unpopulate(bo->tbo.bdev, bo->tbo.ttm); + spin_lock(&adev->mman.bdev.lru_lock); + } + spin_unlock(&adev->mman.bdev.lru_lock); +} + /** * amdgpu_device_fini - tear down the driver * @@ -3639,12 +3663,15 @@ void amdgpu_device_fini_hw(struct amdgpu_device *adev) amdgpu_ucode_sysfs_fini(adev); sysfs_remove_files(&adev->dev->kobj, amdgpu_dev_attributes); - amdgpu_fbdev_fini(adev); amdgpu_irq_fini_hw(adev); amdgpu_device_ip_fini_early(adev); + + amdgpu_clear_dma_mappings(adev); + + amdgpu_gart_dummy_page_fini(adev); } void amdgpu_device_fini_sw(struct amdgpu_device *adev) diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_gart.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_gart.c index fde2d899b2c4..49cdcaf8512d 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_gart.c +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_gart.c @@ -92,7 +92,7 @@ static int amdgpu_gart_dummy_page_init(struct amdgpu_device *adev) * * Frees the dummy page used by the driver (all asics). */ -static void amdgpu_gart_dummy_page_fini(struct amdgpu_device *adev) +void amdgpu_gart_dummy_page_fini(struct amdgpu_device *adev) { if (!adev->dummy_page_addr) return; @@ -397,5 +397,4 @@ void amdgpu_gart_fini(struct amdgpu_device *adev) vfree(adev->gart.pages); adev->gart.pages = NULL; #endif - amdgpu_gart_dummy_page_fini(adev); } diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_gart.h b/drivers/gpu/drm/amd/amdgpu/amdgpu_gart.h index afa2e2877d87..5678d9c105ab 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_gart.h +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_gart.h @@ -61,6 +61,7 @@ int amdgpu_gart_table_vram_pin(struct amdgpu_device *adev); void amdgpu_gart_table_vram_unpin(struct amdgpu_device *adev); int amdgpu_gart_init(struct amdgpu_device *adev); void amdgpu_gart_fini(struct amdgpu_device *adev); +void amdgpu_gart_dummy_page_fini(struct amdgpu_device *adev); int amdgpu_gart_unbind(struct amdgpu_device *adev, uint64_t offset, int pages); int amdgpu_gart_map(struct amdgpu_device *adev, uint64_t offset, diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_irq.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_irq.c index 63e815c27585..a922154953a7 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_irq.c +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_irq.c @@ -326,6 +326,15 @@ void amdgpu_irq_fini_hw(struct amdgpu_device *adev) if (!amdgpu_device_has_dc_support(adev)) flush_work(&adev->hotplug_work); } + + if (adev->irq.ih_soft.ring) + amdgpu_ih_ring_fini(adev, &adev->irq.ih_soft); + if (adev->irq.ih.ring) + amdgpu_ih_ring_fini(adev, &adev->irq.ih); + if (adev->irq.ih1.ring) + amdgpu_ih_ring_fini(adev, &adev->irq.ih1); + if (adev->irq.ih2.ring) + amdgpu_ih_ring_fini(adev, &adev->irq.ih2); } /** diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_object.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_object.c index 485f249d063a..62d829f5e62c 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_object.c +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_object.c @@ -68,8 +68,13 @@ static void amdgpu_bo_destroy(struct ttm_buffer_object *tbo) list_del_init(&bo->shadow_list); mutex_unlock(&adev->shadow_list_lock); } - amdgpu_bo_unref(&bo->parent); + + spin_lock(&adev->mman.bdev.lru_lock); + list_del(&bo->bo); + spin_unlock(&adev->mman.bdev.lru_lock); + + amdgpu_bo_unref(&bo->parent); kfree(bo->metadata); kfree(bo); } @@ -585,6 +590,12 @@ static int amdgpu_bo_do_create(struct amdgpu_device *adev, if (bp->type == ttm_bo_type_device) bo->flags &= ~AMDGPU_GEM_CREATE_CPU_ACCESS_REQUIRED; + INIT_LIST_HEAD(&bo->bo); + + spin_lock(&adev->mman.bdev.lru_lock); + list_add_tail(&bo->bo, &adev->device_bo_list); + spin_unlock(&adev->mman.bdev.lru_lock); + return 0; fail_unreserve: diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_object.h b/drivers/gpu/drm/amd/amdgpu/amdgpu_object.h index 9ac37569823f..5ae8555ef275 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_object.h +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_object.h @@ -110,6 +110,8 @@ struct amdgpu_bo { struct list_head shadow_list; struct kgd_mem *kfd_bo; + + struct list_head bo; }; static inline struct amdgpu_bo *ttm_to_amdgpu_bo(struct ttm_buffer_object *tbo) diff --git a/drivers/gpu/drm/amd/amdgpu/cik_ih.c b/drivers/gpu/drm/amd/amdgpu/cik_ih.c index 183d44a6583c..df385ffc9768 100644 --- a/drivers/gpu/drm/amd/amdgpu/cik_ih.c +++ b/drivers/gpu/drm/amd/amdgpu/cik_ih.c @@ -310,7 +310,6 @@ static int cik_ih_sw_fini(void *handle) struct amdgpu_device *adev = (struct amdgpu_device *)handle; amdgpu_irq_fini_sw(adev); - amdgpu_ih_ring_fini(adev, &adev->irq.ih); amdgpu_irq_remove_domain(adev); return 0; diff --git a/drivers/gpu/drm/amd/amdgpu/cz_ih.c b/drivers/gpu/drm/amd/amdgpu/cz_ih.c index d32743949003..b8c47e0cf37a 100644 --- a/drivers/gpu/drm/amd/amdgpu/cz_ih.c +++ b/drivers/gpu/drm/amd/amdgpu/cz_ih.c @@ -302,7 +302,6 @@ static int cz_ih_sw_fini(void *handle) struct amdgpu_device *adev = (struct amdgpu_device *)handle; amdgpu_irq_fini_sw(adev); - amdgpu_ih_ring_fini(adev, &adev->irq.ih); amdgpu_irq_remove_domain(adev); return 0; diff --git a/drivers/gpu/drm/amd/amdgpu/iceland_ih.c b/drivers/gpu/drm/amd/amdgpu/iceland_ih.c index da96c6013477..ddfe4eaeea05 100644 --- a/drivers/gpu/drm/amd/amdgpu/iceland_ih.c +++ b/drivers/gpu/drm/amd/amdgpu/iceland_ih.c @@ -301,7 +301,6 @@ static int iceland_ih_sw_fini(void *handle) struct amdgpu_device *adev = (struct amdgpu_device *)handle; amdgpu_irq_fini_sw(adev); - amdgpu_ih_ring_fini(adev, &adev->irq.ih); amdgpu_irq_remove_domain(adev); return 0; diff --git a/drivers/gpu/drm/amd/amdgpu/navi10_ih.c b/drivers/gpu/drm/amd/amdgpu/navi10_ih.c index 5eea4550b856..e171a9e78544 100644 --- a/drivers/gpu/drm/amd/amdgpu/navi10_ih.c +++ b/drivers/gpu/drm/amd/amdgpu/navi10_ih.c @@ -571,9 +571,6 @@ static int navi10_ih_sw_fini(void *handle) amdgpu_irq_fini_sw(adev); amdgpu_ih_ring_fini(adev, &adev->irq.ih_soft); - amdgpu_ih_ring_fini(adev, &adev->irq.ih2); - amdgpu_ih_ring_fini(adev, &adev->irq.ih1); - amdgpu_ih_ring_fini(adev, &adev->irq.ih); return 0; } diff --git a/drivers/gpu/drm/amd/amdgpu/si_ih.c b/drivers/gpu/drm/amd/amdgpu/si_ih.c index 751307f3252c..9a24f17a5750 100644 --- a/drivers/gpu/drm/amd/amdgpu/si_ih.c +++ b/drivers/gpu/drm/amd/amdgpu/si_ih.c @@ -176,7 +176,6 @@ static int si_ih_sw_fini(void *handle) struct amdgpu_device *adev = (struct amdgpu_device *)handle; amdgpu_irq_fini_sw(adev); - amdgpu_ih_ring_fini(adev, &adev->irq.ih); return 0; } diff --git a/drivers/gpu/drm/amd/amdgpu/tonga_ih.c b/drivers/gpu/drm/amd/amdgpu/tonga_ih.c index 973d80ec7f6c..b08905d1c00f 100644 --- a/drivers/gpu/drm/amd/amdgpu/tonga_ih.c +++ b/drivers/gpu/drm/amd/amdgpu/tonga_ih.c @@ -313,7 +313,6 @@ static int tonga_ih_sw_fini(void *handle) struct amdgpu_device *adev = (struct amdgpu_device *)handle; amdgpu_irq_fini_sw(adev); - amdgpu_ih_ring_fini(adev, &adev->irq.ih); amdgpu_irq_remove_domain(adev); return 0; diff --git a/drivers/gpu/drm/amd/amdgpu/vega10_ih.c b/drivers/gpu/drm/amd/amdgpu/vega10_ih.c index 2d0094c276ca..8c8abc00f710 100644 --- a/drivers/gpu/drm/amd/amdgpu/vega10_ih.c +++ b/drivers/gpu/drm/amd/amdgpu/vega10_ih.c @@ -525,9 +525,6 @@ static int vega10_ih_sw_fini(void *handle) amdgpu_irq_fini_sw(adev); amdgpu_ih_ring_fini(adev, &adev->irq.ih_soft); - amdgpu_ih_ring_fini(adev, &adev->irq.ih2); - amdgpu_ih_ring_fini(adev, &adev->irq.ih1); - amdgpu_ih_ring_fini(adev, &adev->irq.ih); return 0; } From patchwork Wed Apr 28 15:11:47 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Andrey Grodzovsky X-Patchwork-Id: 12229233 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.5 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,MSGID_FROM_MTA_HEADER,SPF_HELO_NONE,SPF_PASS, USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 4E490C433ED for ; Wed, 28 Apr 2021 15:13:01 +0000 (UTC) Received: from gabe.freedesktop.org (gabe.freedesktop.org [131.252.210.177]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id E601E6143A for ; Wed, 28 Apr 2021 15:13:00 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org E601E6143A Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=amd.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=dri-devel-bounces@lists.freedesktop.org Received: from gabe.freedesktop.org (localhost [127.0.0.1]) by gabe.freedesktop.org (Postfix) with ESMTP id 3DA976EB63; Wed, 28 Apr 2021 15:12:59 +0000 (UTC) Received: from NAM12-DM6-obe.outbound.protection.outlook.com (mail-dm6nam12on2054.outbound.protection.outlook.com [40.107.243.54]) by gabe.freedesktop.org (Postfix) with ESMTPS id 7AAA96EB5C; Wed, 28 Apr 2021 15:12:47 +0000 (UTC) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=PRylbx4FZsCIMxuXtd4x04jFt+g5IoNmyPHgfzCKkPk/oIzvLyY7eYJ9LhxOWlZjiJTvJHTixdj40MNx4/VgDpoXB/O+y3ANfKZpbAWAcVlPjOAyE1glOw54gfsQ9PzREz6B0VZLDlJZ0PBfYtdxB9drlbm42AvJEy6FAhkJ1+LV0VcfSE3VZFlO6M/RmJrRcjExoTNJAzSTcMaW4xVfJTkkA4BUuXuoM/0xS6P/T3SSR369yxxSsPAh8Ykj/0M5gqS+In7Z2H2KeQt/12cIUk5NLRxbIFhURcJNfNv2GqdvTd/7cpCwoAKX/m2u8SOr2bOk9hnMwnZcajoxuRNlNg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=N3LsJnLfLNgqHUkpZWzjeMUTXKNbI8XQw2ZNffTNCTA=; b=MO60gR7qp9Wi+4/oTsRU7HAzRVdCRA80+IvFx7MifhoNZd/ZNr/0+L/2G3rXPgtqCHfmNGY5FPd/a5IbqjQmwiFCfrBnsMfpyg2McvqqQu5uiLPVV0jNOq+XE7QpZ3GOpbE6HDJMz6rVgUmaw6cr66KmIYUjqh/qslwFcGyxelF2lb8DBHwAFfND0jKblvziT7RblnQLTMV75rxlQo/6bekO6pEo04CmoTsxNDxGNGMqytFjYukgvpIbaTvDh0tQW+JGjvbRdpy97s8YMv6W6C+HVkFHOshURYYriQWm7KSvkYwVPoKGaxxZSPvWxY6dAZ/Hg/X/tLn7wjEqFw1jXA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=amd.com; dmarc=pass action=none header.from=amd.com; dkim=pass header.d=amd.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=N3LsJnLfLNgqHUkpZWzjeMUTXKNbI8XQw2ZNffTNCTA=; b=mKsmiYMIEWNxg4t7ddUGIJsYZSHXXjBm0TQu/Re75gLyxeQgiHt3xGAObu5nCgr/ZGCDnV3iHCsZPtQG/sY4mooezd75oU/r4QzLLxFL08BVA0f5KE3j3NzY5xPDrQ/3eta4koNHYumb6thPigOmdJc741Cpcs1AWTu0pjdm98E= Authentication-Results: lists.freedesktop.org; dkim=none (message not signed) header.d=none; lists.freedesktop.org; dmarc=none action=none header.from=amd.com; Received: from SN6PR12MB4623.namprd12.prod.outlook.com (2603:10b6:805:e9::17) by SA0PR12MB4462.namprd12.prod.outlook.com (2603:10b6:806:95::22) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4065.23; Wed, 28 Apr 2021 15:12:44 +0000 Received: from SN6PR12MB4623.namprd12.prod.outlook.com ([fe80::ad51:8c49:b171:856c]) by SN6PR12MB4623.namprd12.prod.outlook.com ([fe80::ad51:8c49:b171:856c%7]) with mapi id 15.20.4065.026; Wed, 28 Apr 2021 15:12:44 +0000 From: Andrey Grodzovsky To: dri-devel@lists.freedesktop.org, amd-gfx@lists.freedesktop.org, linux-pci@vger.kernel.org, ckoenig.leichtzumerken@gmail.com, daniel.vetter@ffwll.ch, Harry.Wentland@amd.com Subject: [PATCH v5 07/27] drm/amdgpu: Remap all page faults to per process dummy page. Date: Wed, 28 Apr 2021 11:11:47 -0400 Message-Id: <20210428151207.1212258-8-andrey.grodzovsky@amd.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210428151207.1212258-1-andrey.grodzovsky@amd.com> References: <20210428151207.1212258-1-andrey.grodzovsky@amd.com> X-Originating-IP: [2607:fea8:3edf:49b0:7212:f93a:73b0:8f23] X-ClientProxiedBy: YT1PR01CA0142.CANPRD01.PROD.OUTLOOK.COM (2603:10b6:b01:2f::21) To SN6PR12MB4623.namprd12.prod.outlook.com (2603:10b6:805:e9::17) MIME-Version: 1.0 X-MS-Exchange-MessageSentRepresentingType: 1 Received: from agrodzovsky-All-Series.hitronhub.home (2607:fea8:3edf:49b0:7212:f93a:73b0:8f23) by YT1PR01CA0142.CANPRD01.PROD.OUTLOOK.COM (2603:10b6:b01:2f::21) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4065.25 via Frontend Transport; Wed, 28 Apr 2021 15:12:43 +0000 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: e32ad548-e286-41a0-3358-08d90a581305 X-MS-TrafficTypeDiagnostic: SA0PR12MB4462: X-MS-Exchange-Transport-Forked: True X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:2000; X-MS-Exchange-SenderADCheck: 1 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: +xmhb3TqYTTAaFfVm0n7jpoiUJus30VhzEveXyEwB+1Z4vpR1BGjPWoBV53BGJ+YOcROkxhv6aSVafXF0hUlwDVRcsSTJkrcp+eCwHNGBTqPYtUPOrProL191xQE4YXXKKUb1/TDqhtuNwgwVmu99biLZvxwXupYoamZHfQoUnj0mQuZUrjIymLTbJiMNfaO6Hqb86bT8SXvDIFhHgnsYII/LYhv5GuHo+rOrXjbkBuKO9vOyQZj6iIRL9NQ8AYj5CT6kxeWzaJMWTMUENRIQxpRqG9PBP6F8HdHP0sqBcJclnv2LOM4BFw9HSqk1PDud1K4hNFe3J5iqoEftJsMELNDh2NGpz7lANXTcniaL6yfl74/ASWR0w8veQmZkt2PLYKE6EyHCQV9dMfJ1MSTQYvbvFpnnQq3vFXwKtRExj0M+8ICII3IUR3tLwPyl0XvuRpNpSBJvwzP2WS9PYyaKGUN23Qr10pdZErEmBaEwIXo1fGdNgSJc8rOUKq/kPMJaDsQC5jeKpayh9t4HZab/nuIFyoE7xOQHQRZo0LodkRRs1PR5UjY/1AzNrW6ZioVyb+yg99/shxK604iTj7SC0prxls0XY0ZnXtwotGvGZ8= X-Forefront-Antispam-Report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:SN6PR12MB4623.namprd12.prod.outlook.com; PTR:; CAT:NONE; SFS:(4636009)(346002)(396003)(136003)(39860400002)(376002)(366004)(6512007)(6636002)(66946007)(316002)(186003)(44832011)(2906002)(16526019)(36756003)(83380400001)(6506007)(4326008)(52116002)(8676002)(66476007)(5660300002)(66556008)(1076003)(2616005)(478600001)(86362001)(38100700002)(8936002)(6486002); DIR:OUT; SFP:1101; X-MS-Exchange-AntiSpam-MessageData: bhfXW+15GrViQlPo3AXP/TVqzoHGPl0P2iBsTCOvpylA1qH8+aKN/zDr1HCqaR87ZXic2Yp1O4D2l/oVR7BJ4mrbWGn0wTf6sanRW5gvMNPz2IJgp/hildqt65/Y25tbaFJageU30yYd2o8rTDeXJ05PpeXRAW31Kfig+J7jGSTzCy7kYpYkaZc21b5cea9viS9qlulQUrEmsxUlQvB7mB0vdwDFi1NWLkj35yulan1PkQngcbmoYByTg/bQispDQ/xrYL5UeXDkRPbTSw4Z4/YtZa5orRVBGPYTUjyfn3XbdKSQD0sKi/75oos3OJWG14SRyRgHkTB4ioO9BbxxrTWtN7NVGpJWtkuDLv7bSWv41fn9whTGA2/NIZ9a95mBqyAWjcFMf2xnrnZvBc+LMt6NFSoIQU0VZJTj/3DLbR0KKMUxo7ltSlkMzEgWvPl6IPggC1D35AK9YC/pZihoSDVwWwVHJh2fEFYs4M1k4qGHiEtAe6AF3tY/sm/wqiqt4BujC+fa7oBvG4fSN14jeyhJBzq1LQ/zSbHsfGMbk9G21ZS9W1r21FWzrh11l9pKlbXLiFMF1lc0hCAYIxee103fK6Ucqn+t6jWzhWsy/7ppd7BC7lqH0M3nnB+OLaK6P/tpEymXZInuLPdltDzSplDfp5w/K5/VKgf3pHHvxaYEFQPJvay/WHuxm6JGuRQVjbmb7ybGfrEvsOdgAkZPck1zwQ1Asqc4BjTtrpydBUVuOTyDGy9hdkwg66uT7bXrhH7YEmKLreu5Pq4EjpE6UrNB8SGyaotCgfut7e/nLG+MgUzaN0g19uzGC2izjJAgcsn3eCwW0I35pvL0AxHJr6AELTIcVf5J3krseSljExPpK5y5GtPlrOP4lRTeBHtZlhaMJvSga5Xi/cb4+1lIX9ufJoYRiYZw2vrwfCxdx0DzIbQ50rav2IpvciXzoSbPyveV2vLkbFxlMudoVpH2UyXi4psTW2eHA20trs57sr1m5hIsXrc/qaumfmtOAv5dnt57FN1bcXtKy8jYNvzOqb9jRmk0In3ZyMM0tC+zvnH4VoZORptB2IDTpGrLyS6hDdOXg5qyCVhQZOKG7LvHsklT3nwCGjrMmAdNxF49YNAVILtJIOiF3GzCRgBvpv5tqv7atKXy1hNjWmydEZFbrQYfRTI3cG4EMTSKKmfZLvBw4qBG4Y5l5+LnI4OxaXjHkegPLEMfLI5D1LI8+bzW33B4nPQQh5MVVg+/f0cov7MbTLPUWn3JbzSMJghJ+P0k5ryoNo2ndpGGa0MorN/12wtnS7nbDB2mC8QpZofIiaqL6JFCeZiqzaqScqtd2uDoo3No0SJRUoF8ibvND5TypshqToRh5KvH6GcuS9cp8z5IZsHkKsChxdVO1vuGeyUM X-OriginatorOrg: amd.com X-MS-Exchange-CrossTenant-Network-Message-Id: e32ad548-e286-41a0-3358-08d90a581305 X-MS-Exchange-CrossTenant-AuthSource: SN6PR12MB4623.namprd12.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 28 Apr 2021 15:12:44.4072 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: y6r37xdO2rL+BZJkpPhGWiiPupUuNl/csV1uJsnLtuFwob04oJ+V9KDjsshDSVOeJVa1WR/yHU21lfaj3UUcvQ== X-MS-Exchange-Transport-CrossTenantHeadersStamped: SA0PR12MB4462 X-BeenThere: dri-devel@lists.freedesktop.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Direct Rendering Infrastructure - Development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: gregkh@linuxfoundation.org, Felix.Kuehling@amd.com, helgaas@kernel.org, Alexander.Deucher@amd.com Errors-To: dri-devel-bounces@lists.freedesktop.org Sender: "dri-devel" On device removal reroute all CPU mappings to dummy page per drm_file instance or imported GEM object. v4: Update for modified ttm_bo_vm_dummy_page Signed-off-by: Andrey Grodzovsky Reviewed-by: Christian König --- drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.c | 21 ++++++++++++++++----- 1 file changed, 16 insertions(+), 5 deletions(-) diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.c index a785acc09f20..93163b220e46 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.c +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.c @@ -49,6 +49,7 @@ #include #include +#include #include "amdgpu.h" #include "amdgpu_object.h" @@ -1982,18 +1983,28 @@ void amdgpu_ttm_set_buffer_funcs_status(struct amdgpu_device *adev, bool enable) static vm_fault_t amdgpu_ttm_fault(struct vm_fault *vmf) { struct ttm_buffer_object *bo = vmf->vma->vm_private_data; + struct drm_device *ddev = bo->base.dev; vm_fault_t ret; + int idx; ret = ttm_bo_vm_reserve(bo, vmf); if (ret) return ret; - ret = amdgpu_bo_fault_reserve_notify(bo); - if (ret) - goto unlock; + if (drm_dev_enter(ddev, &idx)) { + ret = amdgpu_bo_fault_reserve_notify(bo); + if (ret) { + drm_dev_exit(idx); + goto unlock; + } - ret = ttm_bo_vm_fault_reserved(vmf, vmf->vma->vm_page_prot, - TTM_BO_VM_NUM_PREFAULT, 1); + ret = ttm_bo_vm_fault_reserved(vmf, vmf->vma->vm_page_prot, + TTM_BO_VM_NUM_PREFAULT, 1); + + drm_dev_exit(idx); + } else { + ret = ttm_bo_vm_dummy_page(vmf, vmf->vma->vm_page_prot); + } if (ret == VM_FAULT_RETRY && !(vmf->flags & FAULT_FLAG_RETRY_NOWAIT)) return ret; From patchwork Wed Apr 28 15:11:48 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrey Grodzovsky X-Patchwork-Id: 12229235 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.5 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,MSGID_FROM_MTA_HEADER,SPF_HELO_NONE,SPF_PASS, USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 0F833C43462 for ; Wed, 28 Apr 2021 15:13:03 +0000 (UTC) Received: from gabe.freedesktop.org (gabe.freedesktop.org [131.252.210.177]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id AAD136143C for ; Wed, 28 Apr 2021 15:13:02 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org AAD136143C Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=amd.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=dri-devel-bounces@lists.freedesktop.org Received: from gabe.freedesktop.org (localhost [127.0.0.1]) by gabe.freedesktop.org (Postfix) with ESMTP id 1232E6EB66; Wed, 28 Apr 2021 15:13:01 +0000 (UTC) Received: from NAM12-DM6-obe.outbound.protection.outlook.com (mail-dm6nam12on2054.outbound.protection.outlook.com [40.107.243.54]) by gabe.freedesktop.org (Postfix) with ESMTPS id 15CFB6EB5E; Wed, 28 Apr 2021 15:12:48 +0000 (UTC) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=gnZd8NvAeHjqfxQi1jLz/XIZ9vqe4ApXTFKrPY4w8yKqICCzgEcI6iNGFoJmYBq+63aXqdZaA5tVdHeeMEEYxubRb+7xmpqWM13Wm0m9VuSvQbrh7xWjIq3aQxouaAxSXji/dgvfY3GL+uRH5bRi9CmHaocaX5fNs0vuq5kPLs3t2BW9wGPkdVx3+i68/9GonbGLLcgKuaZ79wh/7SlcO/NgNyHjg8JRcWTiaDmntMTusNE+S8uz1KeZlUPGcRD0J7+YqgSf6hFY7Q8wPago5OQGwKef1V7Eir91WjHA+pq0qj3qsShApWZu6Ym9RvtFS59ASpCKi+gOGj06V98rqw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=fwNW114xwpfVAQ+qP+cK3mFSQ+F24mN+PEGPdbDclLA=; b=H12fKSQJpjgeY00Yc53NQe4TRgGi96PFPSZwmDTmDNVpt97pPUW7EgvfJzpCXS/8AJ642NsAWlh+IURd5KHrMLSvm/Nye5c9PeKN1R4u5bGFy71IgZH2IbMiCoZNIFQ28TKkJgc7yxcnNPYwa1wNwY6kBiGVrJlPdmlf+SWdtfBlTncwwAgDHs4MhLN8smuZPEt49dIEdTqbIakgFIyx28pQZfmnHSkfSZmIXNC37sHrywnQhd908/c4m+a1ByQYIs9bfx9vmO/L8YzW3wkP3BFx4ZYbdkWEc5UVqzEFySSqgrelYOSmP/wKuuPZ3lL6v7YjICAgUrufX+upoZ/urg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=amd.com; dmarc=pass action=none header.from=amd.com; dkim=pass header.d=amd.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=fwNW114xwpfVAQ+qP+cK3mFSQ+F24mN+PEGPdbDclLA=; b=DKrQJTro5QSx+gR/LpNV3O4gZwkAdoCiUXOu0U5b/D17OI9uD26H+iBQJrj4+p8q/n7cZqKa9Omz5KealWP/BtwFJbJFishlz4fAWQI3eIgvMLMG7gNSb1yYnS3NtEw4ywA7wPL9Z1C2QVyW0p3Nhx3xBEQ5YaIOECTq/WF9izk= Authentication-Results: lists.freedesktop.org; dkim=none (message not signed) header.d=none; lists.freedesktop.org; dmarc=none action=none header.from=amd.com; Received: from SN6PR12MB4623.namprd12.prod.outlook.com (2603:10b6:805:e9::17) by SA0PR12MB4462.namprd12.prod.outlook.com (2603:10b6:806:95::22) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4065.23; Wed, 28 Apr 2021 15:12:46 +0000 Received: from SN6PR12MB4623.namprd12.prod.outlook.com ([fe80::ad51:8c49:b171:856c]) by SN6PR12MB4623.namprd12.prod.outlook.com ([fe80::ad51:8c49:b171:856c%7]) with mapi id 15.20.4065.026; Wed, 28 Apr 2021 15:12:46 +0000 From: Andrey Grodzovsky To: dri-devel@lists.freedesktop.org, amd-gfx@lists.freedesktop.org, linux-pci@vger.kernel.org, ckoenig.leichtzumerken@gmail.com, daniel.vetter@ffwll.ch, Harry.Wentland@amd.com Subject: [PATCH v5 08/27] PCI: add support for dev_groups to struct pci_device_driver Date: Wed, 28 Apr 2021 11:11:48 -0400 Message-Id: <20210428151207.1212258-9-andrey.grodzovsky@amd.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210428151207.1212258-1-andrey.grodzovsky@amd.com> References: <20210428151207.1212258-1-andrey.grodzovsky@amd.com> X-Originating-IP: [2607:fea8:3edf:49b0:7212:f93a:73b0:8f23] X-ClientProxiedBy: YT1PR01CA0142.CANPRD01.PROD.OUTLOOK.COM (2603:10b6:b01:2f::21) To SN6PR12MB4623.namprd12.prod.outlook.com (2603:10b6:805:e9::17) MIME-Version: 1.0 X-MS-Exchange-MessageSentRepresentingType: 1 Received: from agrodzovsky-All-Series.hitronhub.home (2607:fea8:3edf:49b0:7212:f93a:73b0:8f23) by YT1PR01CA0142.CANPRD01.PROD.OUTLOOK.COM (2603:10b6:b01:2f::21) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4065.25 via Frontend Transport; Wed, 28 Apr 2021 15:12:45 +0000 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 0e28a1e5-26c4-47b0-8035-08d90a581414 X-MS-TrafficTypeDiagnostic: SA0PR12MB4462: X-MS-Exchange-Transport-Forked: True X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:7219; X-MS-Exchange-SenderADCheck: 1 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: lIZzwCh5j77X9qkgYGQRJpQddfnoVgJCBvAVE90nlwUnTE7rxXO8UIGiORCnO/qUEbxq6KvRthAuObeO+P3X+V8kxS9lP5/c0mXxDXMvXdaaMudAtBywnUGTUQbMMIi+UDZrVjQR3Y2lu+M1mc/xPl+nNXZ7jOMFbA2s26ZYrKvUSS89xiaDXoMFCx9s17teuE99yvYyfZbuaAmdBKsIc0bFOTsV9HF9fcQjUaQiyzlZtdZzHaXrreUh2jPjYmbEK2oGM7Le15HZWhgHayvfjJYe/svNADhulINWKaBH2P3HoQAOaUXmMxvQPBNScHyk+mf/u+ku2GiBQb2he7nZ+ILXx8JNKpJK/8cv4ZqUjCtH2Nab2qnn28nO/4C30Ozb2zu9RU03g+BD7g6dd47qrsr5GX0+MU/ICASZj7NJ361MeudN2MNvypLS1B8LiBX1TzXREqnwxqZzYg7EkKn4KC/Fd+K3z+lY3Ax6GRqVEqjLzaLoi1E461bn7sXcmQzQ5f5RVBvoRS6FPvPID2TY8TpgfQGBiJeuxxi1lCakwg7zDNdx5qS0LE4x1Spa3TDXuix3olbrxZe8/DXAW/0cxMPHCxucnNv2uTlRMVKMU+8= X-Forefront-Antispam-Report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:SN6PR12MB4623.namprd12.prod.outlook.com; PTR:; CAT:NONE; SFS:(4636009)(346002)(396003)(136003)(39860400002)(376002)(366004)(6512007)(6636002)(66946007)(316002)(186003)(44832011)(2906002)(16526019)(36756003)(83380400001)(6506007)(4326008)(52116002)(8676002)(66476007)(5660300002)(66556008)(1076003)(2616005)(478600001)(86362001)(38100700002)(8936002)(6486002); DIR:OUT; SFP:1101; X-MS-Exchange-AntiSpam-MessageData: 6+Qr+CoIlY80f2/WgIGSg4TbHBKBOX8GhU5zzcuXCcsiiUm8k6EAqXSDo49iClD1A605du4JNk6F0H8BcNj5qPk+VVhyuuDY66wF0iGzKBGkdYE/V4Znt0q30sSu2lw8H6lHSh6yVGfhZmgzg6BVDTZK2q7UTx75prMf1il2y6xBUGFeOp7hF1glf1uGEtMzdHYtz0hg1iHQJ+UszLPtP+9ZZP8D5jYeTEpaXOC/UIfJ5GNPnDuKbuAi3r2ykcAVBsOP2RAkLCCKE3HJ6Mz3nCNqQIOcnRxhAo6/xLi3czP1oy769pm95hNHtqFlkjEQu9NkhYm5QwIvXWRpuB49sQgI49mftb8ST7YvEOrX88RQOSn5Yf3AjCSapsCfZOwWD7H6g7l9ZjR/z7086Zi/2StqQW3Q20aSY+0yoFT8P15uprSyMPVjlaD80CWZ1uhaFlv6zMySA4KsjHw7e8NX8srYMgAhfNSO5SKO+ZM1HPaK/dImO0Me2ceyEFAZKRxmZRE30nUBx3paSv4FytGgDQ4SKFnGUEuNyQfHl3Nhpx2+78YCGyPQRBw4oAZUMj6pLwry3Lq/dF431uqp4kLqfBKEw+U54V/1UcaaRM8x0ago9Rb/+fbtlbpHzhurIl7MD4ahXiEoZUvEJfA5IpGj0ouhfJfmtRAnW9gsZe2obDryDD0lEmTfuI5ha+NDLxdpygzHqd8k+/U8nJDqe/LQTAElKyRrGc//wHAQY3U3k82iq9zOp+OTXeC61raotjFjzZPv6czYc1xcmYA2NCswy/iAN2/OwrS/QO5sqjJvJWOmWYq8xrGG2a392tSj26i7Tqt7Kneit2fYU5INxWe8Lle72Fj48SYocRv+KKrqtb+v/MFM2aJ1J7kGxEHxkjTogAMVID1fRz88nUDiTPr916/hHVeqn003YffgG+ahb3EOYl+tUiuCQ1Dwv/mcZRLUBm0jmr8rASWjxVeVwt8Ktp/ozLe4M9/uifFzPAwho3T1DH/IlkvJrVtkMKgJpxysD89M7ugIPmspipU71OxxePkCsUcg11kiKTlGSTYV9NWMu5qXsnEX+LlAZhs1yjxfCQc51H+X9HCeoqAHr27khIxHo3sSy6RUU05cn0Baptuz2OcBu+2G/Q+MpKZwTaAo4Du8P6rPwJgpNMeqnB1WQahdPC+bXDyw1QhSlL7xFQHcpksmVy778Z8BkBtkyia1mHfyhmZ6ZIfurTQ7fU7mqRJSJuSQByo5t7qAT2kDBWm7/gRitPTkV56NNRxtd2iFS673mxYV9hGNNu9QvRWpcKGCdlYleJXXFL+V5u3ADHokU1HmU4DYaTqN2oFdoQKS2YoCo28YEuPF2W/xe1rC5tojK0xHEVeOsnkCbJgpbXXwwuUHUMU7fUYBA0f7XPc0 X-OriginatorOrg: amd.com X-MS-Exchange-CrossTenant-Network-Message-Id: 0e28a1e5-26c4-47b0-8035-08d90a581414 X-MS-Exchange-CrossTenant-AuthSource: SN6PR12MB4623.namprd12.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 28 Apr 2021 15:12:46.1022 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: m2RC56DNr3YZ57u7DGYhK8I4jkbDMhpLwf0UCZPvrnFnbRlp8TZsi5wdUPmTle1sVngD4lejRGEVWMgkCfUmwQ== X-MS-Exchange-Transport-CrossTenantHeadersStamped: SA0PR12MB4462 X-BeenThere: dri-devel@lists.freedesktop.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Direct Rendering Infrastructure - Development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: gregkh@linuxfoundation.org, Felix.Kuehling@amd.com, helgaas@kernel.org, Alexander.Deucher@amd.com Errors-To: dri-devel-bounces@lists.freedesktop.org Sender: "dri-devel" This is exact copy of 'USB: add support for dev_groups to struct usb_device_driver' patch by Greg but just for the PCI case. Signed-off-by: Andrey Grodzovsky Suggested-by: Greg Kroah-Hartman --- drivers/pci/pci-driver.c | 1 + include/linux/pci.h | 3 +++ 2 files changed, 4 insertions(+) diff --git a/drivers/pci/pci-driver.c b/drivers/pci/pci-driver.c index ec44a79e951a..3a72352aa5cf 100644 --- a/drivers/pci/pci-driver.c +++ b/drivers/pci/pci-driver.c @@ -1385,6 +1385,7 @@ int __pci_register_driver(struct pci_driver *drv, struct module *owner, drv->driver.owner = owner; drv->driver.mod_name = mod_name; drv->driver.groups = drv->groups; + drv->driver.dev_groups = drv->dev_groups; spin_lock_init(&drv->dynids.lock); INIT_LIST_HEAD(&drv->dynids.list); diff --git a/include/linux/pci.h b/include/linux/pci.h index 86c799c97b77..b57755b03009 100644 --- a/include/linux/pci.h +++ b/include/linux/pci.h @@ -858,6 +858,8 @@ struct module; * number of VFs to enable via sysfs "sriov_numvfs" file. * @err_handler: See Documentation/PCI/pci-error-recovery.rst * @groups: Sysfs attribute groups. + * @dev_groups: Attributes attached to the device that will be + * created once it is bound to the driver. * @driver: Driver model structure. * @dynids: List of dynamically added device IDs. */ @@ -873,6 +875,7 @@ struct pci_driver { int (*sriov_configure)(struct pci_dev *dev, int num_vfs); /* On PF */ const struct pci_error_handlers *err_handler; const struct attribute_group **groups; + const struct attribute_group **dev_groups; struct device_driver driver; struct pci_dynids dynids; }; From patchwork Wed Apr 28 15:11:49 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Andrey Grodzovsky X-Patchwork-Id: 12229259 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.5 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,MSGID_FROM_MTA_HEADER,SPF_HELO_NONE,SPF_PASS, USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id A4488C433B4 for ; Wed, 28 Apr 2021 15:13:49 +0000 (UTC) Received: from gabe.freedesktop.org (gabe.freedesktop.org [131.252.210.177]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 4BBB8613F1 for ; Wed, 28 Apr 2021 15:13:49 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 4BBB8613F1 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=amd.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=dri-devel-bounces@lists.freedesktop.org Received: from gabe.freedesktop.org (localhost [127.0.0.1]) by gabe.freedesktop.org (Postfix) with ESMTP id 66E996EB85; Wed, 28 Apr 2021 15:13:48 +0000 (UTC) Received: from NAM12-DM6-obe.outbound.protection.outlook.com (mail-dm6nam12on2066.outbound.protection.outlook.com [40.107.243.66]) by gabe.freedesktop.org (Postfix) with ESMTPS id 99FA96E0F5; Wed, 28 Apr 2021 15:12:49 +0000 (UTC) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=iPH8hBYl72Mc6J2ncQgzHXDLu1AVaY6BWab9rxElh3tT3BRk65hNKUFqECRNx82vjntc+O9q4JFy/bEhfxxlQVM/uCtgm6RQbqNhRsJqLviMJ3BMid4d70wMAOFRVn/R2buK1zDuykDM7Ji6L+K6Rf+sBg1Uzkl7PSrO+gQZCgg8jPTp4KWthv6mq1FJA7uHdupj7wC5cuJKiSVbQk+wmIsKMet/jZgRh88IBw/PqyZs13CjCPblY90GXBmoEh1TSc+cYEm9cJHTLhVpFuRKFV6nrirTqT6ZCv6FiqRcIy6NRNQpKcuiUcDd99RwiZXBARvNjRsMGs+/DOe8d3kxrw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=GV16xVnQ4igK/MMYjr6sTVnZY2gMzJTw2X/+xlBmqIc=; b=jhB4sJorS3T0mqHq1oD/V9k9Jd1VZKVeBKMHPlr2f9yExZfz2oi95vQEyCvTt1BgTglnLDS1XhVIYnoPFtWJJ9JxwckggeiD0YJFx8E+0fuPRwESJmolQmszG53xmuwEuvliVzFj7FT6ECIYe0/d6VWPBdsTXlCRH6xy8UaCUn8SjP7+xvuYFRJc7KklC22a0vStYSja/SVhMDKORAoO43ki7R9DQRl6icDudN0h4r87YyVqRCF2aEXtGai4cD+ez8kTApEuaVv2qivqH9XaR1ICsMAILtHWIeqpESXdSy34bchALDA//3GAtwgo8q1ZKpnoYxzEyvTiIhKrs6EwFg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=amd.com; dmarc=pass action=none header.from=amd.com; dkim=pass header.d=amd.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=GV16xVnQ4igK/MMYjr6sTVnZY2gMzJTw2X/+xlBmqIc=; b=akOMQCHIJpZYHvHMUcdS9nkVE+I1OCiXMy/4xf7YX0WPZNY+SoAwaH95qZmXAtCtBwPfD1q0VM7LAJBliGnUEWmEdTsrJUF8Beg82G6E5PI+isBOvi1PruFniyVNTHgPXEbkODS5bHyM6NSGUGhdQYt9vurnOR2Dj3UK01KT0FA= Authentication-Results: lists.freedesktop.org; dkim=none (message not signed) header.d=none; lists.freedesktop.org; dmarc=none action=none header.from=amd.com; Received: from SN6PR12MB4623.namprd12.prod.outlook.com (2603:10b6:805:e9::17) by SA0PR12MB4462.namprd12.prod.outlook.com (2603:10b6:806:95::22) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4065.23; Wed, 28 Apr 2021 15:12:48 +0000 Received: from SN6PR12MB4623.namprd12.prod.outlook.com ([fe80::ad51:8c49:b171:856c]) by SN6PR12MB4623.namprd12.prod.outlook.com ([fe80::ad51:8c49:b171:856c%7]) with mapi id 15.20.4065.026; Wed, 28 Apr 2021 15:12:48 +0000 From: Andrey Grodzovsky To: dri-devel@lists.freedesktop.org, amd-gfx@lists.freedesktop.org, linux-pci@vger.kernel.org, ckoenig.leichtzumerken@gmail.com, daniel.vetter@ffwll.ch, Harry.Wentland@amd.com Subject: [PATCH v5 09/27] dmr/amdgpu: Move some sysfs attrs creation to default_attr Date: Wed, 28 Apr 2021 11:11:49 -0400 Message-Id: <20210428151207.1212258-10-andrey.grodzovsky@amd.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210428151207.1212258-1-andrey.grodzovsky@amd.com> References: <20210428151207.1212258-1-andrey.grodzovsky@amd.com> X-Originating-IP: [2607:fea8:3edf:49b0:7212:f93a:73b0:8f23] X-ClientProxiedBy: YT1PR01CA0142.CANPRD01.PROD.OUTLOOK.COM (2603:10b6:b01:2f::21) To SN6PR12MB4623.namprd12.prod.outlook.com (2603:10b6:805:e9::17) MIME-Version: 1.0 X-MS-Exchange-MessageSentRepresentingType: 1 Received: from agrodzovsky-All-Series.hitronhub.home (2607:fea8:3edf:49b0:7212:f93a:73b0:8f23) by YT1PR01CA0142.CANPRD01.PROD.OUTLOOK.COM (2603:10b6:b01:2f::21) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4065.25 via Frontend Transport; Wed, 28 Apr 2021 15:12:47 +0000 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 3239da2b-0d71-42df-ca9e-08d90a581547 X-MS-TrafficTypeDiagnostic: SA0PR12MB4462: X-MS-Exchange-Transport-Forked: True X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:6108; X-MS-Exchange-SenderADCheck: 1 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: MvAoludBJ2Ya6iXNq6muBS7Gw5PsCnKuq8xT9lXvwK6rkNc/wHD3cUMFSg2XyWOCyCBxdmmC3EYZ/IfUbQk5V/bCoJRsDc6w6MKQD7qVSRjFPID3dYaM2tFgBhCRF74DW3oGtKecOJbBlzZwEOPQYSISNVPHO2uzMZvhKPX1gKT2v7tiXBOWVJfKdfgZVLoN1pXifgMs2B4huCGX0GptFok5/SFHGjNT6zg980xAJy/1P9qlLTHjDDd/aR9vzkMJDmk7GqbgiqgJx4NcDdFdh2m2rKFH2jVBsFlBdUJT+mTn2qZlK1hGlEyIXr4q80qenrphC5yD/qIJUzjpHxe06jt+T1+6du7G26FHsLUC+25YStALUq4bFLf5ocHOYC+Nrt5/LtKTVMZoc+NjOhfIw+70nWUmVw7gyLT5AqBMGS9QBR0Jo26Plp5IPD7WTRC1RtoijPsAIUCHi+VH0KdtN7ThORYf4KH7Hj5dwPDy6bE1UowRcyICscHM6ATgac5lhCb3zSRFCVqjhMLtz54NXeBYzWlicOiReWO3V0XIC8hhSoyq8joM4mgLgVR/mvy/J/5HeWkg8bFE8cXVohsQQJl0wE5i5oox+ptJ/O9ct/g= X-Forefront-Antispam-Report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:SN6PR12MB4623.namprd12.prod.outlook.com; PTR:; CAT:NONE; SFS:(4636009)(346002)(396003)(136003)(39860400002)(376002)(366004)(6512007)(6636002)(66946007)(316002)(186003)(44832011)(2906002)(16526019)(36756003)(83380400001)(6506007)(4326008)(52116002)(8676002)(66476007)(5660300002)(66556008)(1076003)(2616005)(478600001)(86362001)(38100700002)(8936002)(6486002); DIR:OUT; SFP:1101; X-MS-Exchange-AntiSpam-MessageData: 40IOY/rHkn2/XWhDEmMu0EEHk+sx9qU5D8umGdbPgJr+cWsQ3vGeR8Zks5U92SeO/O9q9ACZGyTsYGqpz1s1bzPfgkKCi10xzfYzKfCDjQxB9xA9csetuwHtOlfEZ8NgmkAvg1Qd6RyWv7YoFmVgrZoDHRvXvwVzFxSN20AkUw7lWxgIfd+2CiH2GpNQeDb16M1peOlJVngRpgl8F9W6IGfJejxNv3P/Bk/AGv0OeCxkJhecausETei1NeZ0zzVVXicO/5NBLBWg4SvyYx39196RdS6OA11fjc+fVshwEvjiSyi9MlqZ9S94WPTsVHWRQyOdWLxShIj71wIz8WU3PUlHWrJQCgkIprWJDo38hsaJ9F5UwRWTyyhCv9Cwm0yPcz+utThkAea7TqVvKHBeTv66FI4zmPKZBnk4co28gp06CgFEFOU6gTaDd9f0hLazDMRVKxGOZm5PalNBgiTxjpUNOJrJZaS7wqVYnT8UfNmWQ6SznWHZJYSpEYO01IAgZyId6ykhXqXz/WrRQ5D/IPVeILDFu+fsfwCn9EAheA/cVD3j2eOFcC+W2meQUNwiCYEHFJpfemXPs4CNoZoSWni5Ko/VGI5AdlaDOhdczah1wH44zK/30znaXHd817Khv52xntFH1A1UBkje6FRTQHhlU+1lIP5UxHYVTpNoy2wxGNQdOEvyb5aVzUyYkO0Iko/IMsS/TGcksxoLXk8KnKs26Ej6DZMdeU3zExF33xRjqD3g2qtIRX/fh2F2hkpH1TRKEXiy5LxqgjFmCG6cSLnNIJcnxVEobUXRobDpPZfW5dabSxG6ZZtdqSSdE8IU0hm0ktmSNwif+UjipgBJrYn4rl1QXGCM0e9kU8UhnfukWNrmRV5jvE0Ahg+/mTybMnEhrxjm4082rLv18PA1b6pUkUju2+buUrxWvx/VQbSEorsqrXylJMUKgNBj92wbnF6CANVnTZCQSakR/gOYETgdFZA05Xkbltq0n3zOPIXdqn47XfRoLyGUpN4EnmMILREoT88UquM3H3Ekrplg/le019HekUigPwm4o0TtI4BZlRnbLIwCHFSgBICVEsCv64mj0aHcKSSyeMhZRtxaoSSc8wbi+qKGaIGeS5pRC/hFLvFzZ5X70HV/Z5XzT7ULJhfJvt+f9jkxyuqWItY1OXgJeB4DOimE4yBALtFuDX2mf9mPITCvsEKwg58dt/rE9ApPSks5Ujxj2Do4Gf6Xbs6YIgLryuo3Tburhfa7IeB35b01iACKa+SpIo3+IrkkdbJ+zIscMl6sC9/aT3oBR83GWQ7VfCf+BaAQWr67DVQGQTeeT2UbXxJx09qxDpYcOHJ3fJ5NO5T9ya9ohdW0Vtus9RxjfEgaWvLPIkEVf2MJ0LKBQazsPDEXClHO2Rme X-OriginatorOrg: amd.com X-MS-Exchange-CrossTenant-Network-Message-Id: 3239da2b-0d71-42df-ca9e-08d90a581547 X-MS-Exchange-CrossTenant-AuthSource: SN6PR12MB4623.namprd12.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 28 Apr 2021 15:12:48.0841 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: YbeDRCExMKF9vABJx3RvKX3YPJ2g4TNdSh/2gRwh9LP0qGcDJIXVJkvr4rI6Crq3nUqRqQCIckRjQFtYGtJK7Q== X-MS-Exchange-Transport-CrossTenantHeadersStamped: SA0PR12MB4462 X-BeenThere: dri-devel@lists.freedesktop.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Direct Rendering Infrastructure - Development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: gregkh@linuxfoundation.org, Felix.Kuehling@amd.com, helgaas@kernel.org, Alexander.Deucher@amd.com Errors-To: dri-devel-bounces@lists.freedesktop.org Sender: "dri-devel" This allows to remove explicit creation and destruction of those attrs and by this avoids warnings on device finilizing post physical device extraction. v5: Use newly added pci_driver.dev_groups directly Signed-off-by: Andrey Grodzovsky Acked-by: Christian König --- drivers/gpu/drm/amd/amdgpu/amdgpu_atombios.c | 17 ++++++------- drivers/gpu/drm/amd/amdgpu/amdgpu_drv.c | 13 ++++++++++ drivers/gpu/drm/amd/amdgpu/amdgpu_gtt_mgr.c | 25 ++++++++------------ drivers/gpu/drm/amd/amdgpu/amdgpu_vram_mgr.c | 14 ++++------- 4 files changed, 37 insertions(+), 32 deletions(-) diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_atombios.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_atombios.c index 86add0f4ea4d..0346e124ab8c 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_atombios.c +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_atombios.c @@ -1953,6 +1953,15 @@ static ssize_t amdgpu_atombios_get_vbios_version(struct device *dev, static DEVICE_ATTR(vbios_version, 0444, amdgpu_atombios_get_vbios_version, NULL); +static struct attribute *amdgpu_vbios_version_attrs[] = { + &dev_attr_vbios_version.attr, + NULL +}; + +const struct attribute_group amdgpu_vbios_version_attr_group = { + .attrs = amdgpu_vbios_version_attrs +}; + /** * amdgpu_atombios_fini - free the driver info and callbacks for atombios * @@ -1972,7 +1981,6 @@ void amdgpu_atombios_fini(struct amdgpu_device *adev) adev->mode_info.atom_context = NULL; kfree(adev->mode_info.atom_card_info); adev->mode_info.atom_card_info = NULL; - device_remove_file(adev->dev, &dev_attr_vbios_version); } /** @@ -1989,7 +1997,6 @@ int amdgpu_atombios_init(struct amdgpu_device *adev) { struct card_info *atom_card_info = kzalloc(sizeof(struct card_info), GFP_KERNEL); - int ret; if (!atom_card_info) return -ENOMEM; @@ -2027,12 +2034,6 @@ int amdgpu_atombios_init(struct amdgpu_device *adev) amdgpu_atombios_allocate_fb_scratch(adev); } - ret = device_create_file(adev->dev, &dev_attr_vbios_version); - if (ret) { - DRM_ERROR("Failed to create device file for VBIOS version\n"); - return ret; - } - return 0; } diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_drv.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_drv.c index 54cb5ee2f563..f799c40d7e72 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_drv.c +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_drv.c @@ -1605,6 +1605,18 @@ static struct pci_error_handlers amdgpu_pci_err_handler = { .resume = amdgpu_pci_resume, }; +extern const struct attribute_group amdgpu_vram_mgr_attr_group; +extern const struct attribute_group amdgpu_gtt_mgr_attr_group; +extern const struct attribute_group amdgpu_vbios_version_attr_group; + +static const struct attribute_group *amdgpu_sysfs_groups[] = { + &amdgpu_vram_mgr_attr_group, + &amdgpu_gtt_mgr_attr_group, + &amdgpu_vbios_version_attr_group, + NULL, +}; + + static struct pci_driver amdgpu_kms_pci_driver = { .name = DRIVER_NAME, .id_table = pciidlist, @@ -1613,6 +1625,7 @@ static struct pci_driver amdgpu_kms_pci_driver = { .shutdown = amdgpu_pci_shutdown, .driver.pm = &amdgpu_pm_ops, .err_handler = &amdgpu_pci_err_handler, + .dev_groups = amdgpu_sysfs_groups, }; static int __init amdgpu_init(void) diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_gtt_mgr.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_gtt_mgr.c index 8980329cded0..3b7150e1c5ed 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_gtt_mgr.c +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_gtt_mgr.c @@ -77,6 +77,16 @@ static DEVICE_ATTR(mem_info_gtt_total, S_IRUGO, static DEVICE_ATTR(mem_info_gtt_used, S_IRUGO, amdgpu_mem_info_gtt_used_show, NULL); +static struct attribute *amdgpu_gtt_mgr_attributes[] = { + &dev_attr_mem_info_gtt_total.attr, + &dev_attr_mem_info_gtt_used.attr, + NULL +}; + +const struct attribute_group amdgpu_gtt_mgr_attr_group = { + .attrs = amdgpu_gtt_mgr_attributes +}; + static const struct ttm_resource_manager_func amdgpu_gtt_mgr_func; /** * amdgpu_gtt_mgr_init - init GTT manager and DRM MM @@ -91,7 +101,6 @@ int amdgpu_gtt_mgr_init(struct amdgpu_device *adev, uint64_t gtt_size) struct amdgpu_gtt_mgr *mgr = &adev->mman.gtt_mgr; struct ttm_resource_manager *man = &mgr->manager; uint64_t start, size; - int ret; man->use_tt = true; man->func = &amdgpu_gtt_mgr_func; @@ -104,17 +113,6 @@ int amdgpu_gtt_mgr_init(struct amdgpu_device *adev, uint64_t gtt_size) spin_lock_init(&mgr->lock); atomic64_set(&mgr->available, gtt_size >> PAGE_SHIFT); - ret = device_create_file(adev->dev, &dev_attr_mem_info_gtt_total); - if (ret) { - DRM_ERROR("Failed to create device file mem_info_gtt_total\n"); - return ret; - } - ret = device_create_file(adev->dev, &dev_attr_mem_info_gtt_used); - if (ret) { - DRM_ERROR("Failed to create device file mem_info_gtt_used\n"); - return ret; - } - ttm_set_driver_manager(&adev->mman.bdev, TTM_PL_TT, &mgr->manager); ttm_resource_manager_set_used(man, true); return 0; @@ -144,9 +142,6 @@ void amdgpu_gtt_mgr_fini(struct amdgpu_device *adev) drm_mm_takedown(&mgr->mm); spin_unlock(&mgr->lock); - device_remove_file(adev->dev, &dev_attr_mem_info_gtt_total); - device_remove_file(adev->dev, &dev_attr_mem_info_gtt_used); - ttm_resource_manager_cleanup(man); ttm_set_driver_manager(&adev->mman.bdev, TTM_PL_TT, NULL); } diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_vram_mgr.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_vram_mgr.c index c89b66bb70e2..68369b38aebb 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_vram_mgr.c +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_vram_mgr.c @@ -154,7 +154,7 @@ static DEVICE_ATTR(mem_info_vis_vram_used, S_IRUGO, static DEVICE_ATTR(mem_info_vram_vendor, S_IRUGO, amdgpu_mem_info_vram_vendor, NULL); -static const struct attribute *amdgpu_vram_mgr_attributes[] = { +static struct attribute *amdgpu_vram_mgr_attributes[] = { &dev_attr_mem_info_vram_total.attr, &dev_attr_mem_info_vis_vram_total.attr, &dev_attr_mem_info_vram_used.attr, @@ -163,6 +163,10 @@ static const struct attribute *amdgpu_vram_mgr_attributes[] = { NULL }; +const struct attribute_group amdgpu_vram_mgr_attr_group = { + .attrs = amdgpu_vram_mgr_attributes +}; + static const struct ttm_resource_manager_func amdgpu_vram_mgr_func; /** @@ -176,7 +180,6 @@ int amdgpu_vram_mgr_init(struct amdgpu_device *adev) { struct amdgpu_vram_mgr *mgr = &adev->mman.vram_mgr; struct ttm_resource_manager *man = &mgr->manager; - int ret; ttm_resource_manager_init(man, adev->gmc.real_vram_size >> PAGE_SHIFT); @@ -187,11 +190,6 @@ int amdgpu_vram_mgr_init(struct amdgpu_device *adev) INIT_LIST_HEAD(&mgr->reservations_pending); INIT_LIST_HEAD(&mgr->reserved_pages); - /* Add the two VRAM-related sysfs files */ - ret = sysfs_create_files(&adev->dev->kobj, amdgpu_vram_mgr_attributes); - if (ret) - DRM_ERROR("Failed to register sysfs\n"); - ttm_set_driver_manager(&adev->mman.bdev, TTM_PL_VRAM, &mgr->manager); ttm_resource_manager_set_used(man, true); return 0; @@ -229,8 +227,6 @@ void amdgpu_vram_mgr_fini(struct amdgpu_device *adev) drm_mm_takedown(&mgr->mm); spin_unlock(&mgr->lock); - sysfs_remove_files(&adev->dev->kobj, amdgpu_vram_mgr_attributes); - ttm_resource_manager_cleanup(man); ttm_set_driver_manager(&adev->mman.bdev, TTM_PL_VRAM, NULL); } From patchwork Wed Apr 28 15:11:50 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrey Grodzovsky X-Patchwork-Id: 12229265 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.5 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,MSGID_FROM_MTA_HEADER,SPF_HELO_NONE,SPF_PASS, USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 2331EC433B4 for ; Wed, 28 Apr 2021 15:14:06 +0000 (UTC) Received: from gabe.freedesktop.org (gabe.freedesktop.org [131.252.210.177]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id B4914613F1 for ; Wed, 28 Apr 2021 15:14:05 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org B4914613F1 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=amd.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=dri-devel-bounces@lists.freedesktop.org Received: from gabe.freedesktop.org (localhost [127.0.0.1]) by gabe.freedesktop.org (Postfix) with ESMTP id E33686EB8D; Wed, 28 Apr 2021 15:13:59 +0000 (UTC) Received: from NAM12-MW2-obe.outbound.protection.outlook.com (mail-mw2nam12on2080.outbound.protection.outlook.com [40.107.244.80]) by gabe.freedesktop.org (Postfix) with ESMTPS id EC17D6EB5E; Wed, 28 Apr 2021 15:12:52 +0000 (UTC) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=RqhyYoIs+txLfA7Mi+12XWiFogul4tbH9BdGFwE4CNZcFmZLQlbZy8yJo/ubB+vkAE6CwMMGOQguBqTV41DMqFmfudyN6PDxZpqNsUnSQ8/jJMARsEipRnklhmYSomLriOsVSoh7f3VUub7ACIhscNOkrA6ltEG6dtKCx1Ikn367PMNDGBsVbb13qOsWc0w+k7UBfurE1LoBt/YEbDg2at62ckilmZ+RRNzwqHzQS2M7YQU5V/juZyl9tfpuzGn2ijruzj44ZOzD0GE/qo9GVkAAdiYZNupCRIfnPutItoG1aNVmGoi9V3oyOFASUGYplrkUqFJqdnDnc9NSozOdVw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=E4dGXrWvq2kbAWx3JpjuZ2bslHDf0E+pyKWi1uBy4xk=; b=Cb4Ei5Jqi6Sj11SOGbzGrRGGet3fi7OatZ3rCFtYRQ4JP4XnPaQ5Ldip/BVcSvhNPrjQhQC6qALS7iFXOi5ftzG9kDgYDab1r0ukza7zBu3Jro3CwxowcNvCC/oYgMZq44tuVBjZVyMcbaNXgmiObqoJbVTMu3+M3TvTMQKjefZFR2l/+t7+wSStT3n+/I3saXoO3JH2X6ob9zarkOSu0I01g2YniXfvwmxkk9CC1LzkRcMwOCbWxLKAEi9dvkzMLBqSn7r3tvBxRgHy18UaiU49+ed7+S13zKm3Ck2ZHCJ1R2bsgf3h9PR6jZr8jCcRXMm5KEaLoi8U1aXyNDKJGw== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=amd.com; dmarc=pass action=none header.from=amd.com; dkim=pass header.d=amd.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=E4dGXrWvq2kbAWx3JpjuZ2bslHDf0E+pyKWi1uBy4xk=; b=NVOm/5x8/puVdeOfRUHMdPLyUtt3nVnNSPe1jXaOXuZuGeULkzz40uBqsQTgA9T27j7M++ElXdxLP2nZFb3VZ/TmZRqe6ADURC5rsw3gKpe2KrI1HGImTS/+apo7nW7N34tA+pNb/5pBWD7iTPRtmNNk83k9bn2w5bd7Q3dbXsk= Authentication-Results: lists.freedesktop.org; dkim=none (message not signed) header.d=none; lists.freedesktop.org; dmarc=none action=none header.from=amd.com; Received: from SN6PR12MB4623.namprd12.prod.outlook.com (2603:10b6:805:e9::17) by SA0PR12MB4462.namprd12.prod.outlook.com (2603:10b6:806:95::22) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4065.23; Wed, 28 Apr 2021 15:12:50 +0000 Received: from SN6PR12MB4623.namprd12.prod.outlook.com ([fe80::ad51:8c49:b171:856c]) by SN6PR12MB4623.namprd12.prod.outlook.com ([fe80::ad51:8c49:b171:856c%7]) with mapi id 15.20.4065.026; Wed, 28 Apr 2021 15:12:50 +0000 From: Andrey Grodzovsky To: dri-devel@lists.freedesktop.org, amd-gfx@lists.freedesktop.org, linux-pci@vger.kernel.org, ckoenig.leichtzumerken@gmail.com, daniel.vetter@ffwll.ch, Harry.Wentland@amd.com Subject: [PATCH v5 10/27] drm/amdgpu: Guard against write accesses after device removal Date: Wed, 28 Apr 2021 11:11:50 -0400 Message-Id: <20210428151207.1212258-11-andrey.grodzovsky@amd.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210428151207.1212258-1-andrey.grodzovsky@amd.com> References: <20210428151207.1212258-1-andrey.grodzovsky@amd.com> X-Originating-IP: [2607:fea8:3edf:49b0:7212:f93a:73b0:8f23] X-ClientProxiedBy: YT1PR01CA0142.CANPRD01.PROD.OUTLOOK.COM (2603:10b6:b01:2f::21) To SN6PR12MB4623.namprd12.prod.outlook.com (2603:10b6:805:e9::17) MIME-Version: 1.0 X-MS-Exchange-MessageSentRepresentingType: 1 Received: from agrodzovsky-All-Series.hitronhub.home (2607:fea8:3edf:49b0:7212:f93a:73b0:8f23) by YT1PR01CA0142.CANPRD01.PROD.OUTLOOK.COM (2603:10b6:b01:2f::21) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4065.25 via Frontend Transport; Wed, 28 Apr 2021 15:12:49 +0000 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 78022567-f1a7-4836-fb00-08d90a5816ba X-MS-TrafficTypeDiagnostic: SA0PR12MB4462: X-MS-Exchange-Transport-Forked: True X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:110; X-MS-Exchange-SenderADCheck: 1 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: Dju3R/u2Cx92TeC2xl8Dd3eo/Al/ZPNvKdKA+IW5bKpooN8fdmyfMmJK1PeDdp8Bw60eKR+lzDC2mhZsXUyc/Wav1TEeTviTAghrnUGbVBaPoWZXIdjCo3K1CcyqbJbV9KVrZsxFRiF2m6vzOyY3J5tzRH3xCr8u2tjD42haMRklCxsBLSMwxSvMlxtjVByGXi5QqTpRq85cdnwqIgyjUmw+tlMB9gwejdB/fpqNow6UTTFE13W3e0T0sYMkbcAjBlN8wvEBAdQK0oz12WRojgUbLNKLJvP39fauGlKlmuN0idITxTtv0qSocZMzDVfWeiK7LnMv6y1hHPqBKXlewj9F4V+LeFjWrAZPJQ5Hk0TN3dctim4g+WC0bAtlkeYl5GqqhjSNsSDuqgTS7gEH26Wwm67GjNr8KPYOiHOYBPfmFTN/1alvxl5nox5vwiuaxCKBAIGoUDloAdzciq9nA4U53bxydU1U/B7O13tBNOfZGhFUNUVsHb0jyUqzXShWbaGtISbWl+oJuSmYSGwisO4WpDzM5iCkL/ODLsPqSJFPqZ4I85+6eecus2rercUSuyZMpFg8YHwN6iCjiENNvxefImiamUxvswlYsIbOO2c= X-Forefront-Antispam-Report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:SN6PR12MB4623.namprd12.prod.outlook.com; PTR:; CAT:NONE; SFS:(4636009)(346002)(396003)(136003)(39860400002)(376002)(366004)(6512007)(6636002)(66946007)(316002)(186003)(44832011)(2906002)(16526019)(36756003)(83380400001)(6506007)(4326008)(52116002)(8676002)(66476007)(5660300002)(66556008)(1076003)(2616005)(6666004)(478600001)(30864003)(86362001)(38100700002)(8936002)(6486002)(579004); DIR:OUT; SFP:1101; X-MS-Exchange-AntiSpam-MessageData: QPs3eTk6bpGWDAftTe4DVQz+ceQ9VL30R7o2sIk//xpX8zRFWM2Yj3bMLa4gk0xVxZbBUF5BxnZPfNnHB4cl+LNvqQY3mgZb6QrSCvwQGXQnD8moSeJhY/FHcgf4Gb8/MJXjy8COqr5nohDSVcuAC2CMB6FgRAHa+dcPe/aS7EJCidnvgJsAeuGdoCSiZ7O0JCHo6MBKNZu8WZ5fC3dI0CVFS51Kn0gHLupq7xEX6/L6GegJrvAK2RciTK2WEho+RzbkghrOg2utKoI/YUneOJVhIeamV0Odj0DdeleLOkqmRhaXw/DwAuJLPOAfNt5PzAJLVYK9as45/4VFFInmKvmCtaxUO4S1+G2L5GmlG/YxozIdu0f5hElohmb0PJPZbU6WmOoRdgFNnwpdNNzy4HiXfLeGi5y3OhSv1b1QHqansMGX7mRbWEfNjR/zedBbpXgIPWE1sh61tC4Gn2fdwcGyBqLVXl98QhTRIB7J7KKZh2K0YQ9wC0fenbAwA7kKgJWOSAQR1hX097zhkHAcwgB0xHmruf3ffTtmtxqLB9bqrGJv5e6OCVrqKnjr54nVsB0+nFmcJ5oJ63R0ZJqYSfqmmgHX8Idw1PkrJGINkxOXjYkSeH6GYUahQPQcPiGbCk4TbhEOKKKXWh6w7jP2L0N45PN02U3+Lhx3hfGv2m5/pJz04pR6k61/MoWibMR8UNhgzYDt2yqO6SA9uny9boSiDD09FFem4lKqWc7C4lPPcFSKhrX5tx+Ia/VWiHxEC5sATWOn7ZDAKlKaZE9U9XZ/Yn+nrBYuetGdE0vO3ia0fF4nxT7t+S/7ttgLQqvWNZCyZpccgMcROZ6rzEWQMfckrAjEIa9GjsadsdZvGD+AIe4t43Ji4JccCrV6FMk9fT8jr0mUNiMiwzRnvzxXzkugyfHrrzd9ywDJyziDuY1CZM9XJW8LTAvEC8PIl9Ma8v6EV00suiRMwo9Mvahnj9b+eVDqtleS2KQ65fWxFgEZZWmmlP79CiVH+NCfSkAb/ppxaa1wzmbx7llR13Hx5IlAMsCxMW3C4GYZUcOYRgh1gpBKjPwDXovoCBzmiBFdmL+ehRo+8lYsPcpdLa0MghaoXnKedCZKuhsbD1azXDmI6+NjBCKeWp495dJB0lqw+N/3ktQfBgBzWGaLa4fLo1+lWPsGDUHUyzUdcCehDXpYlaQ4T7Tz2FJTBOrjOi6xrJHSwQoX07bb3ED5Pbp6mWFchY8Sp1gocQRJk7I+B5nxy/GKiJNL0rnwHLMYM+2/HCspaISSbpQ8TDFT14+hjz24hLPtGsMF7Z29+G5dRwb+fvzcb66JZIEPW3npqFWF/h4yLndAI15i1j/h0Gvnc8312zSSuW3TCxMqP5wMLoLMz6jgtK71o+ncXlLwIHcY X-OriginatorOrg: amd.com X-MS-Exchange-CrossTenant-Network-Message-Id: 78022567-f1a7-4836-fb00-08d90a5816ba X-MS-Exchange-CrossTenant-AuthSource: SN6PR12MB4623.namprd12.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 28 Apr 2021 15:12:50.5927 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: drIJUR2wg7puj6OXc71/MwWLtq864DZkOC/DK8rqLSQJiSG1zKB5pThPUUAwgSbVP7CCaqN3aRFqncmIv5ietQ== X-MS-Exchange-Transport-CrossTenantHeadersStamped: SA0PR12MB4462 X-BeenThere: dri-devel@lists.freedesktop.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Direct Rendering Infrastructure - Development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: gregkh@linuxfoundation.org, Felix.Kuehling@amd.com, helgaas@kernel.org, Alexander.Deucher@amd.com Errors-To: dri-devel-bounces@lists.freedesktop.org Sender: "dri-devel" This should prevent writing to memory or IO ranges possibly already allocated for other uses after our device is removed. v5: Protect more places wher memcopy_to/form_io takes place Protect IB submissions Signed-off-by: Andrey Grodzovsky --- drivers/gpu/drm/amd/amdgpu/amdgpu_device.c | 75 +++--- drivers/gpu/drm/amd/amdgpu/amdgpu_gmc.c | 9 + drivers/gpu/drm/amd/amdgpu/amdgpu_ib.c | 228 +++++++++--------- drivers/gpu/drm/amd/amdgpu/amdgpu_psp.c | 115 +++++---- drivers/gpu/drm/amd/amdgpu/amdgpu_psp.h | 3 + drivers/gpu/drm/amd/amdgpu/amdgpu_ring.c | 70 ++++++ drivers/gpu/drm/amd/amdgpu/amdgpu_ring.h | 49 +--- drivers/gpu/drm/amd/amdgpu/amdgpu_uvd.c | 31 ++- drivers/gpu/drm/amd/amdgpu/amdgpu_vce.c | 11 +- drivers/gpu/drm/amd/amdgpu/amdgpu_vcn.c | 22 +- drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c | 7 +- drivers/gpu/drm/amd/amdgpu/psp_v11_0.c | 44 ++-- drivers/gpu/drm/amd/amdgpu/psp_v12_0.c | 8 +- drivers/gpu/drm/amd/amdgpu/psp_v3_1.c | 8 +- drivers/gpu/drm/amd/amdgpu/vce_v4_0.c | 26 +- drivers/gpu/drm/amd/amdgpu/vcn_v3_0.c | 22 +- .../drm/amd/pm/powerplay/smumgr/smu7_smumgr.c | 2 + 17 files changed, 425 insertions(+), 305 deletions(-) diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c index 91594ddc2459..22b09c4db255 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c @@ -71,6 +71,8 @@ #include +#include + MODULE_FIRMWARE("amdgpu/vega10_gpu_info.bin"); MODULE_FIRMWARE("amdgpu/vega12_gpu_info.bin"); MODULE_FIRMWARE("amdgpu/raven_gpu_info.bin"); @@ -279,48 +281,55 @@ void amdgpu_device_vram_access(struct amdgpu_device *adev, loff_t pos, unsigned long flags; uint32_t hi = ~0; uint64_t last; + int idx; + if (drm_dev_enter(&adev->ddev, &idx)) { #ifdef CONFIG_64BIT - last = min(pos + size, adev->gmc.visible_vram_size); - if (last > pos) { - void __iomem *addr = adev->mman.aper_base_kaddr + pos; - size_t count = last - pos; - - if (write) { - memcpy_toio(addr, buf, count); - mb(); - amdgpu_asic_flush_hdp(adev, NULL); - } else { - amdgpu_asic_invalidate_hdp(adev, NULL); - mb(); - memcpy_fromio(buf, addr, count); - } + last = min(pos + size, adev->gmc.visible_vram_size); + if (last > pos) { + void __iomem *addr = adev->mman.aper_base_kaddr + pos; + size_t count = last - pos; + + if (write) { + memcpy_toio(addr, buf, count); + mb(); + amdgpu_asic_flush_hdp(adev, NULL); + } else { + amdgpu_asic_invalidate_hdp(adev, NULL); + mb(); + memcpy_fromio(buf, addr, count); + } - if (count == size) - return; + if (count == size) { + drm_dev_exit(idx); + return; + } - pos += count; - buf += count / 4; - size -= count; - } + pos += count; + buf += count / 4; + size -= count; + } #endif - spin_lock_irqsave(&adev->mmio_idx_lock, flags); - for (last = pos + size; pos < last; pos += 4) { - uint32_t tmp = pos >> 31; + spin_lock_irqsave(&adev->mmio_idx_lock, flags); + for (last = pos + size; pos < last; pos += 4) { + uint32_t tmp = pos >> 31; - WREG32_NO_KIQ(mmMM_INDEX, ((uint32_t)pos) | 0x80000000); - if (tmp != hi) { - WREG32_NO_KIQ(mmMM_INDEX_HI, tmp); - hi = tmp; + WREG32_NO_KIQ(mmMM_INDEX, ((uint32_t)pos) | 0x80000000); + if (tmp != hi) { + WREG32_NO_KIQ(mmMM_INDEX_HI, tmp); + hi = tmp; + } + if (write) + WREG32_NO_KIQ(mmMM_DATA, *buf++); + else + *buf++ = RREG32_NO_KIQ(mmMM_DATA); } - if (write) - WREG32_NO_KIQ(mmMM_DATA, *buf++); - else - *buf++ = RREG32_NO_KIQ(mmMM_DATA); + spin_unlock_irqrestore(&adev->mmio_idx_lock, flags); + + drm_dev_exit(idx); } - spin_unlock_irqrestore(&adev->mmio_idx_lock, flags); } /* @@ -402,6 +411,7 @@ uint8_t amdgpu_mm_rreg8(struct amdgpu_device *adev, uint32_t offset) */ void amdgpu_mm_wreg8(struct amdgpu_device *adev, uint32_t offset, uint8_t value) { + if (adev->in_pci_err_recovery) return; @@ -542,6 +552,7 @@ u32 amdgpu_mm_rdoorbell(struct amdgpu_device *adev, u32 index) */ void amdgpu_mm_wdoorbell(struct amdgpu_device *adev, u32 index, u32 v) { + if (adev->in_pci_err_recovery) return; diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_gmc.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_gmc.c index fe1a39ffda72..1beb4e64b884 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_gmc.c +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_gmc.c @@ -31,6 +31,8 @@ #include "amdgpu_ras.h" #include "amdgpu_xgmi.h" +#include + /** * amdgpu_gmc_get_pde_for_bo - get the PDE for a BO * @@ -98,6 +100,10 @@ int amdgpu_gmc_set_pte_pde(struct amdgpu_device *adev, void *cpu_pt_addr, { void __iomem *ptr = (void *)cpu_pt_addr; uint64_t value; + int idx; + + if (!drm_dev_enter(&adev->ddev, &idx)) + return 0; /* * The following is for PTE only. GART does not have PDEs. @@ -105,6 +111,9 @@ int amdgpu_gmc_set_pte_pde(struct amdgpu_device *adev, void *cpu_pt_addr, value = addr & 0x0000FFFFFFFFF000ULL; value |= flags; writeq(value, ptr + (gpu_page_idx * 8)); + + drm_dev_exit(idx); + return 0; } diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_ib.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_ib.c index 7645223ea0ef..b3a1ff04c10f 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_ib.c +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_ib.c @@ -31,6 +31,7 @@ #include #include +#include #include "amdgpu.h" #include "atom.h" @@ -138,7 +139,7 @@ int amdgpu_ib_schedule(struct amdgpu_ring *ring, unsigned num_ibs, bool secure; unsigned i; - int r = 0; + int idx, r = 0; bool need_pipe_sync = false; if (num_ibs == 0) @@ -170,142 +171,151 @@ int amdgpu_ib_schedule(struct amdgpu_ring *ring, unsigned num_ibs, return -EINVAL; } - alloc_size = ring->funcs->emit_frame_size + num_ibs * - ring->funcs->emit_ib_size; + if (drm_dev_enter(&adev->ddev, &idx)) { - r = amdgpu_ring_alloc(ring, alloc_size); - if (r) { - dev_err(adev->dev, "scheduling IB failed (%d).\n", r); - return r; - } - - need_ctx_switch = ring->current_ctx != fence_ctx; - if (ring->funcs->emit_pipeline_sync && job && - ((tmp = amdgpu_sync_get_fence(&job->sched_sync)) || - (amdgpu_sriov_vf(adev) && need_ctx_switch) || - amdgpu_vm_need_pipeline_sync(ring, job))) { - need_pipe_sync = true; - - if (tmp) - trace_amdgpu_ib_pipe_sync(job, tmp); - - dma_fence_put(tmp); - } + alloc_size = ring->funcs->emit_frame_size + num_ibs * + ring->funcs->emit_ib_size; - if ((ib->flags & AMDGPU_IB_FLAG_EMIT_MEM_SYNC) && ring->funcs->emit_mem_sync) - ring->funcs->emit_mem_sync(ring); + r = amdgpu_ring_alloc(ring, alloc_size); + if (r) { + dev_err(adev->dev, "scheduling IB failed (%d).\n", r); + goto exit; + } - if (ring->funcs->emit_wave_limit && - ring->hw_prio == AMDGPU_GFX_PIPE_PRIO_HIGH) - ring->funcs->emit_wave_limit(ring, true); + need_ctx_switch = ring->current_ctx != fence_ctx; + if (ring->funcs->emit_pipeline_sync && job && + ((tmp = amdgpu_sync_get_fence(&job->sched_sync)) || + (amdgpu_sriov_vf(adev) && need_ctx_switch) || + amdgpu_vm_need_pipeline_sync(ring, job))) { + need_pipe_sync = true; - if (ring->funcs->insert_start) - ring->funcs->insert_start(ring); + if (tmp) + trace_amdgpu_ib_pipe_sync(job, tmp); - if (job) { - r = amdgpu_vm_flush(ring, job, need_pipe_sync); - if (r) { - amdgpu_ring_undo(ring); - return r; + dma_fence_put(tmp); } - } - if (job && ring->funcs->init_cond_exec) - patch_offset = amdgpu_ring_init_cond_exec(ring); + if ((ib->flags & AMDGPU_IB_FLAG_EMIT_MEM_SYNC) && ring->funcs->emit_mem_sync) + ring->funcs->emit_mem_sync(ring); -#ifdef CONFIG_X86_64 - if (!(adev->flags & AMD_IS_APU)) -#endif - { - if (ring->funcs->emit_hdp_flush) - amdgpu_ring_emit_hdp_flush(ring); - else - amdgpu_asic_flush_hdp(adev, ring); - } + if (ring->funcs->emit_wave_limit && + ring->hw_prio == AMDGPU_GFX_PIPE_PRIO_HIGH) + ring->funcs->emit_wave_limit(ring, true); - if (need_ctx_switch) - status |= AMDGPU_HAVE_CTX_SWITCH; + if (ring->funcs->insert_start) + ring->funcs->insert_start(ring); - skip_preamble = ring->current_ctx == fence_ctx; - if (job && ring->funcs->emit_cntxcntl) { - status |= job->preamble_status; - status |= job->preemption_status; - amdgpu_ring_emit_cntxcntl(ring, status); - } + if (job) { + r = amdgpu_vm_flush(ring, job, need_pipe_sync); + if (r) { + amdgpu_ring_undo(ring); + goto exit; + } + } - /* Setup initial TMZiness and send it off. - */ - secure = false; - if (job && ring->funcs->emit_frame_cntl) { - secure = ib->flags & AMDGPU_IB_FLAGS_SECURE; - amdgpu_ring_emit_frame_cntl(ring, true, secure); - } + if (job && ring->funcs->init_cond_exec) + patch_offset = amdgpu_ring_init_cond_exec(ring); + + #ifdef CONFIG_X86_64 + if (!(adev->flags & AMD_IS_APU)) + #endif + { + if (ring->funcs->emit_hdp_flush) + amdgpu_ring_emit_hdp_flush(ring); + else + amdgpu_asic_flush_hdp(adev, ring); + } - for (i = 0; i < num_ibs; ++i) { - ib = &ibs[i]; + if (need_ctx_switch) + status |= AMDGPU_HAVE_CTX_SWITCH; - /* drop preamble IBs if we don't have a context switch */ - if ((ib->flags & AMDGPU_IB_FLAG_PREAMBLE) && - skip_preamble && - !(status & AMDGPU_PREAMBLE_IB_PRESENT_FIRST) && - !amdgpu_mcbp && - !amdgpu_sriov_vf(adev)) /* for SRIOV preemption, Preamble CE ib must be inserted anyway */ - continue; + skip_preamble = ring->current_ctx == fence_ctx; + if (job && ring->funcs->emit_cntxcntl) { + status |= job->preamble_status; + status |= job->preemption_status; + amdgpu_ring_emit_cntxcntl(ring, status); + } + /* Setup initial TMZiness and send it off. + */ + secure = false; if (job && ring->funcs->emit_frame_cntl) { - if (secure != !!(ib->flags & AMDGPU_IB_FLAGS_SECURE)) { - amdgpu_ring_emit_frame_cntl(ring, false, secure); - secure = !secure; - amdgpu_ring_emit_frame_cntl(ring, true, secure); + secure = ib->flags & AMDGPU_IB_FLAGS_SECURE; + amdgpu_ring_emit_frame_cntl(ring, true, secure); + } + + for (i = 0; i < num_ibs; ++i) { + ib = &ibs[i]; + + /* drop preamble IBs if we don't have a context switch */ + if ((ib->flags & AMDGPU_IB_FLAG_PREAMBLE) && + skip_preamble && + !(status & AMDGPU_PREAMBLE_IB_PRESENT_FIRST) && + !amdgpu_mcbp && + !amdgpu_sriov_vf(adev)) /* for SRIOV preemption, Preamble CE ib must be inserted anyway */ + continue; + + if (job && ring->funcs->emit_frame_cntl) { + if (secure != !!(ib->flags & AMDGPU_IB_FLAGS_SECURE)) { + amdgpu_ring_emit_frame_cntl(ring, false, secure); + secure = !secure; + amdgpu_ring_emit_frame_cntl(ring, true, secure); + } } + + amdgpu_ring_emit_ib(ring, job, ib, status); + status &= ~AMDGPU_HAVE_CTX_SWITCH; } - amdgpu_ring_emit_ib(ring, job, ib, status); - status &= ~AMDGPU_HAVE_CTX_SWITCH; - } + if (job && ring->funcs->emit_frame_cntl) + amdgpu_ring_emit_frame_cntl(ring, false, secure); - if (job && ring->funcs->emit_frame_cntl) - amdgpu_ring_emit_frame_cntl(ring, false, secure); + #ifdef CONFIG_X86_64 + if (!(adev->flags & AMD_IS_APU)) + #endif + amdgpu_asic_invalidate_hdp(adev, ring); -#ifdef CONFIG_X86_64 - if (!(adev->flags & AMD_IS_APU)) -#endif - amdgpu_asic_invalidate_hdp(adev, ring); + if (ib->flags & AMDGPU_IB_FLAG_TC_WB_NOT_INVALIDATE) + fence_flags |= AMDGPU_FENCE_FLAG_TC_WB_ONLY; - if (ib->flags & AMDGPU_IB_FLAG_TC_WB_NOT_INVALIDATE) - fence_flags |= AMDGPU_FENCE_FLAG_TC_WB_ONLY; + /* wrap the last IB with fence */ + if (job && job->uf_addr) { + amdgpu_ring_emit_fence(ring, job->uf_addr, job->uf_sequence, + fence_flags | AMDGPU_FENCE_FLAG_64BIT); + } - /* wrap the last IB with fence */ - if (job && job->uf_addr) { - amdgpu_ring_emit_fence(ring, job->uf_addr, job->uf_sequence, - fence_flags | AMDGPU_FENCE_FLAG_64BIT); - } + r = amdgpu_fence_emit(ring, f, fence_flags); + if (r) { + dev_err(adev->dev, "failed to emit fence (%d)\n", r); + if (job && job->vmid) + amdgpu_vmid_reset(adev, ring->funcs->vmhub, job->vmid); + amdgpu_ring_undo(ring); + goto exit; + } - r = amdgpu_fence_emit(ring, f, fence_flags); - if (r) { - dev_err(adev->dev, "failed to emit fence (%d)\n", r); - if (job && job->vmid) - amdgpu_vmid_reset(adev, ring->funcs->vmhub, job->vmid); - amdgpu_ring_undo(ring); - return r; - } + if (ring->funcs->insert_end) + ring->funcs->insert_end(ring); - if (ring->funcs->insert_end) - ring->funcs->insert_end(ring); + if (patch_offset != ~0 && ring->funcs->patch_cond_exec) + amdgpu_ring_patch_cond_exec(ring, patch_offset); - if (patch_offset != ~0 && ring->funcs->patch_cond_exec) - amdgpu_ring_patch_cond_exec(ring, patch_offset); + ring->current_ctx = fence_ctx; + if (vm && ring->funcs->emit_switch_buffer) + amdgpu_ring_emit_switch_buffer(ring); - ring->current_ctx = fence_ctx; - if (vm && ring->funcs->emit_switch_buffer) - amdgpu_ring_emit_switch_buffer(ring); + if (ring->funcs->emit_wave_limit && + ring->hw_prio == AMDGPU_GFX_PIPE_PRIO_HIGH) + ring->funcs->emit_wave_limit(ring, false); - if (ring->funcs->emit_wave_limit && - ring->hw_prio == AMDGPU_GFX_PIPE_PRIO_HIGH) - ring->funcs->emit_wave_limit(ring, false); + amdgpu_ring_commit(ring); - amdgpu_ring_commit(ring); - return 0; + } else { + return -ENODEV; + } + +exit: + drm_dev_exit(idx); + return r; } /** diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_psp.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_psp.c index 839917eb7bc3..638b7fd1857c 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_psp.c +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_psp.c @@ -25,6 +25,7 @@ #include #include +#include #include "amdgpu.h" #include "amdgpu_psp.h" @@ -38,6 +39,8 @@ #include "amdgpu_ras.h" #include "amdgpu_securedisplay.h" +#include + static int psp_sysfs_init(struct amdgpu_device *adev); static void psp_sysfs_fini(struct amdgpu_device *adev); @@ -249,7 +252,7 @@ psp_cmd_submit_buf(struct psp_context *psp, struct psp_gfx_cmd_resp *cmd, uint64_t fence_mc_addr) { int ret; - int index; + int index, idx; int timeout = 20000; bool ras_intr = false; bool skip_unsupport = false; @@ -257,6 +260,9 @@ psp_cmd_submit_buf(struct psp_context *psp, if (psp->adev->in_pci_err_recovery) return 0; + if (!drm_dev_enter(&psp->adev->ddev, &idx)) + return 0; + mutex_lock(&psp->mutex); memset(psp->cmd_buf_mem, 0, PSP_CMD_BUFFER_SIZE); @@ -267,8 +273,7 @@ psp_cmd_submit_buf(struct psp_context *psp, ret = psp_ring_cmd_submit(psp, psp->cmd_buf_mc_addr, fence_mc_addr, index); if (ret) { atomic_dec(&psp->fence_value); - mutex_unlock(&psp->mutex); - return ret; + goto exit; } amdgpu_asic_invalidate_hdp(psp->adev, NULL); @@ -308,8 +313,8 @@ psp_cmd_submit_buf(struct psp_context *psp, psp->cmd_buf_mem->cmd_id, psp->cmd_buf_mem->resp.status); if (!timeout) { - mutex_unlock(&psp->mutex); - return -EINVAL; + ret = -EINVAL; + goto exit; } } @@ -317,8 +322,10 @@ psp_cmd_submit_buf(struct psp_context *psp, ucode->tmr_mc_addr_lo = psp->cmd_buf_mem->resp.fw_addr_lo; ucode->tmr_mc_addr_hi = psp->cmd_buf_mem->resp.fw_addr_hi; } - mutex_unlock(&psp->mutex); +exit: + mutex_unlock(&psp->mutex); + drm_dev_exit(idx); return ret; } @@ -355,8 +362,7 @@ static int psp_load_toc(struct psp_context *psp, if (!cmd) return -ENOMEM; /* Copy toc to psp firmware private buffer */ - memset(psp->fw_pri_buf, 0, PSP_1_MEG); - memcpy(psp->fw_pri_buf, psp->toc_start_addr, psp->toc_bin_size); + psp_copy_fw(psp, psp->toc_start_addr, psp->toc_bin_size); psp_prep_load_toc_cmd_buf(cmd, psp->fw_pri_mc_addr, psp->toc_bin_size); @@ -571,8 +577,7 @@ static int psp_asd_load(struct psp_context *psp) if (!cmd) return -ENOMEM; - memset(psp->fw_pri_buf, 0, PSP_1_MEG); - memcpy(psp->fw_pri_buf, psp->asd_start_addr, psp->asd_ucode_size); + psp_copy_fw(psp, psp->asd_start_addr, psp->asd_ucode_size); psp_prep_asd_load_cmd_buf(cmd, psp->fw_pri_mc_addr, psp->asd_ucode_size); @@ -727,8 +732,7 @@ static int psp_xgmi_load(struct psp_context *psp) if (!cmd) return -ENOMEM; - memset(psp->fw_pri_buf, 0, PSP_1_MEG); - memcpy(psp->fw_pri_buf, psp->ta_xgmi_start_addr, psp->ta_xgmi_ucode_size); + psp_copy_fw(psp, psp->ta_xgmi_start_addr, psp->ta_xgmi_ucode_size); psp_prep_ta_load_cmd_buf(cmd, psp->fw_pri_mc_addr, @@ -983,8 +987,7 @@ static int psp_ras_load(struct psp_context *psp) if (!cmd) return -ENOMEM; - memset(psp->fw_pri_buf, 0, PSP_1_MEG); - memcpy(psp->fw_pri_buf, psp->ta_ras_start_addr, psp->ta_ras_ucode_size); + psp_copy_fw(psp, psp->ta_ras_start_addr, psp->ta_ras_ucode_size); psp_prep_ta_load_cmd_buf(cmd, psp->fw_pri_mc_addr, @@ -1220,8 +1223,7 @@ static int psp_hdcp_load(struct psp_context *psp) if (!cmd) return -ENOMEM; - memset(psp->fw_pri_buf, 0, PSP_1_MEG); - memcpy(psp->fw_pri_buf, psp->ta_hdcp_start_addr, + psp_copy_fw(psp, psp->ta_hdcp_start_addr, psp->ta_hdcp_ucode_size); psp_prep_ta_load_cmd_buf(cmd, @@ -1372,8 +1374,7 @@ static int psp_dtm_load(struct psp_context *psp) if (!cmd) return -ENOMEM; - memset(psp->fw_pri_buf, 0, PSP_1_MEG); - memcpy(psp->fw_pri_buf, psp->ta_dtm_start_addr, psp->ta_dtm_ucode_size); + psp_copy_fw(psp, psp->ta_dtm_start_addr, psp->ta_dtm_ucode_size); psp_prep_ta_load_cmd_buf(cmd, psp->fw_pri_mc_addr, @@ -1518,8 +1519,7 @@ static int psp_rap_load(struct psp_context *psp) if (!cmd) return -ENOMEM; - memset(psp->fw_pri_buf, 0, PSP_1_MEG); - memcpy(psp->fw_pri_buf, psp->ta_rap_start_addr, psp->ta_rap_ucode_size); + psp_copy_fw(psp, psp->ta_rap_start_addr, psp->ta_rap_ucode_size); psp_prep_ta_load_cmd_buf(cmd, psp->fw_pri_mc_addr, @@ -2928,7 +2928,7 @@ static ssize_t psp_usbc_pd_fw_sysfs_write(struct device *dev, struct amdgpu_device *adev = drm_to_adev(ddev); void *cpu_addr; dma_addr_t dma_addr; - int ret; + int ret, idx; char fw_name[100]; const struct firmware *usbc_pd_fw; @@ -2937,47 +2937,66 @@ static ssize_t psp_usbc_pd_fw_sysfs_write(struct device *dev, return -EBUSY; } - snprintf(fw_name, sizeof(fw_name), "amdgpu/%s", buf); - ret = request_firmware(&usbc_pd_fw, fw_name, adev->dev); - if (ret) - goto fail; + if (drm_dev_enter(ddev, &idx)) { - /* We need contiguous physical mem to place the FW for psp to access */ - cpu_addr = dma_alloc_coherent(adev->dev, usbc_pd_fw->size, &dma_addr, GFP_KERNEL); + snprintf(fw_name, sizeof(fw_name), "amdgpu/%s", buf); + ret = request_firmware(&usbc_pd_fw, fw_name, adev->dev); + if (ret) + goto fail; - ret = dma_mapping_error(adev->dev, dma_addr); - if (ret) - goto rel_buf; + /* We need contiguous physical mem to place the FW for psp to access */ + cpu_addr = dma_alloc_coherent(adev->dev, usbc_pd_fw->size, &dma_addr, GFP_KERNEL); - memcpy_toio(cpu_addr, usbc_pd_fw->data, usbc_pd_fw->size); + ret = dma_mapping_error(adev->dev, dma_addr); + if (ret) + goto rel_buf; - /* - * x86 specific workaround. - * Without it the buffer is invisible in PSP. - * - * TODO Remove once PSP starts snooping CPU cache - */ + memcpy_toio(cpu_addr, usbc_pd_fw->data, usbc_pd_fw->size); + + /* + * x86 specific workaround. + * Without it the buffer is invisible in PSP. + * + * TODO Remove once PSP starts snooping CPU cache + */ #ifdef CONFIG_X86 - clflush_cache_range(cpu_addr, (usbc_pd_fw->size & ~(L1_CACHE_BYTES - 1))); + clflush_cache_range(cpu_addr, (usbc_pd_fw->size & ~(L1_CACHE_BYTES - 1))); #endif - mutex_lock(&adev->psp.mutex); - ret = psp_load_usbc_pd_fw(&adev->psp, dma_addr); - mutex_unlock(&adev->psp.mutex); + mutex_lock(&adev->psp.mutex); + ret = psp_load_usbc_pd_fw(&adev->psp, dma_addr); + mutex_unlock(&adev->psp.mutex); rel_buf: - dma_free_coherent(adev->dev, usbc_pd_fw->size, cpu_addr, dma_addr); - release_firmware(usbc_pd_fw); - + dma_free_coherent(adev->dev, usbc_pd_fw->size, cpu_addr, dma_addr); + release_firmware(usbc_pd_fw); fail: - if (ret) { - DRM_ERROR("Failed to load USBC PD FW, err = %d", ret); - return ret; + if (ret) { + DRM_ERROR("Failed to load USBC PD FW, err = %d", ret); + return ret; + } + + drm_dev_exit(idx); + return count; + } else { + return -ENODEV; } +} + +void psp_copy_fw(struct psp_context *psp, uint8_t *start_addr, uint32_t bin_size) +{ + int idx; - return count; + if (!drm_dev_enter(&psp->adev->ddev, &idx)) + return; + + memset(psp->fw_pri_buf, 0, PSP_1_MEG); + memcpy(psp->fw_pri_buf, start_addr, bin_size); + + drm_dev_exit(idx); } + static DEVICE_ATTR(usbc_pd_fw, S_IRUGO | S_IWUSR, psp_usbc_pd_fw_sysfs_read, psp_usbc_pd_fw_sysfs_write); diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_psp.h b/drivers/gpu/drm/amd/amdgpu/amdgpu_psp.h index cb50ba445f8c..0dbbedb84c84 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_psp.h +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_psp.h @@ -417,4 +417,7 @@ int psp_init_ta_microcode(struct psp_context *psp, const char *chip_name); int psp_get_fw_attestation_records_addr(struct psp_context *psp, uint64_t *output_ptr); + +void psp_copy_fw(struct psp_context *psp, uint8_t *start_addr, uint32_t bin_size); + #endif diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_ring.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_ring.c index b644c78475fd..f9f71008f454 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_ring.c +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_ring.c @@ -35,6 +35,8 @@ #include "amdgpu.h" #include "atom.h" +#include + /* * Rings * Most engines on the GPU are fed via ring buffers. Ring @@ -459,3 +461,71 @@ int amdgpu_ring_test_helper(struct amdgpu_ring *ring) ring->sched.ready = !r; return r; } + +void amdgpu_ring_clear_ring(struct amdgpu_ring *ring) +{ + int idx; + int i = 0; + + if (!drm_dev_enter(&ring->adev->ddev, &idx)) + return; + + while (i <= ring->buf_mask) + ring->ring[i++] = ring->funcs->nop; + + drm_dev_exit(idx); + +} + +void amdgpu_ring_write(struct amdgpu_ring *ring, uint32_t v) +{ + int idx; + + if (!drm_dev_enter(&ring->adev->ddev, &idx)) + return; + + if (ring->count_dw <= 0) + DRM_ERROR("amdgpu: writing more dwords to the ring than expected!\n"); + ring->ring[ring->wptr++ & ring->buf_mask] = v; + ring->wptr &= ring->ptr_mask; + ring->count_dw--; + + drm_dev_exit(idx); +} + +void amdgpu_ring_write_multiple(struct amdgpu_ring *ring, + void *src, int count_dw) +{ + unsigned occupied, chunk1, chunk2; + void *dst; + int idx; + + if (!drm_dev_enter(&ring->adev->ddev, &idx)) + return; + + if (unlikely(ring->count_dw < count_dw)) + DRM_ERROR("amdgpu: writing more dwords to the ring than expected!\n"); + + occupied = ring->wptr & ring->buf_mask; + dst = (void *)&ring->ring[occupied]; + chunk1 = ring->buf_mask + 1 - occupied; + chunk1 = (chunk1 >= count_dw) ? count_dw: chunk1; + chunk2 = count_dw - chunk1; + chunk1 <<= 2; + chunk2 <<= 2; + + if (chunk1) + memcpy(dst, src, chunk1); + + if (chunk2) { + src += chunk1; + dst = (void *)ring->ring; + memcpy(dst, src, chunk2); + } + + ring->wptr += count_dw; + ring->wptr &= ring->ptr_mask; + ring->count_dw -= count_dw; + + drm_dev_exit(idx); +} diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_ring.h b/drivers/gpu/drm/amd/amdgpu/amdgpu_ring.h index 0f195f7bf797..c2b83f48e6d6 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_ring.h +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_ring.h @@ -297,53 +297,12 @@ static inline void amdgpu_ring_set_preempt_cond_exec(struct amdgpu_ring *ring, *ring->cond_exe_cpu_addr = cond_exec; } -static inline void amdgpu_ring_clear_ring(struct amdgpu_ring *ring) -{ - int i = 0; - while (i <= ring->buf_mask) - ring->ring[i++] = ring->funcs->nop; - -} - -static inline void amdgpu_ring_write(struct amdgpu_ring *ring, uint32_t v) -{ - if (ring->count_dw <= 0) - DRM_ERROR("amdgpu: writing more dwords to the ring than expected!\n"); - ring->ring[ring->wptr++ & ring->buf_mask] = v; - ring->wptr &= ring->ptr_mask; - ring->count_dw--; -} +void amdgpu_ring_clear_ring(struct amdgpu_ring *ring); -static inline void amdgpu_ring_write_multiple(struct amdgpu_ring *ring, - void *src, int count_dw) -{ - unsigned occupied, chunk1, chunk2; - void *dst; - - if (unlikely(ring->count_dw < count_dw)) - DRM_ERROR("amdgpu: writing more dwords to the ring than expected!\n"); - - occupied = ring->wptr & ring->buf_mask; - dst = (void *)&ring->ring[occupied]; - chunk1 = ring->buf_mask + 1 - occupied; - chunk1 = (chunk1 >= count_dw) ? count_dw: chunk1; - chunk2 = count_dw - chunk1; - chunk1 <<= 2; - chunk2 <<= 2; +void amdgpu_ring_write(struct amdgpu_ring *ring, uint32_t v); - if (chunk1) - memcpy(dst, src, chunk1); - - if (chunk2) { - src += chunk1; - dst = (void *)ring->ring; - memcpy(dst, src, chunk2); - } - - ring->wptr += count_dw; - ring->wptr &= ring->ptr_mask; - ring->count_dw -= count_dw; -} +void amdgpu_ring_write_multiple(struct amdgpu_ring *ring, + void *src, int count_dw); int amdgpu_ring_test_helper(struct amdgpu_ring *ring); diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_uvd.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_uvd.c index e2ed4689118a..df47f5ffa08f 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_uvd.c +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_uvd.c @@ -32,6 +32,7 @@ #include #include +#include #include "amdgpu.h" #include "amdgpu_pm.h" @@ -375,7 +376,7 @@ int amdgpu_uvd_suspend(struct amdgpu_device *adev) { unsigned size; void *ptr; - int i, j; + int i, j, idx; bool in_ras_intr = amdgpu_ras_intr_triggered(); cancel_delayed_work_sync(&adev->uvd.idle_work); @@ -403,11 +404,15 @@ int amdgpu_uvd_suspend(struct amdgpu_device *adev) if (!adev->uvd.inst[j].saved_bo) return -ENOMEM; - /* re-write 0 since err_event_athub will corrupt VCPU buffer */ - if (in_ras_intr) - memset(adev->uvd.inst[j].saved_bo, 0, size); - else - memcpy_fromio(adev->uvd.inst[j].saved_bo, ptr, size); + if (drm_dev_enter(&adev->ddev, &idx)) { + /* re-write 0 since err_event_athub will corrupt VCPU buffer */ + if (in_ras_intr) + memset(adev->uvd.inst[j].saved_bo, 0, size); + else + memcpy_fromio(adev->uvd.inst[j].saved_bo, ptr, size); + + drm_dev_exit(idx); + } } if (in_ras_intr) @@ -420,7 +425,7 @@ int amdgpu_uvd_resume(struct amdgpu_device *adev) { unsigned size; void *ptr; - int i; + int i, idx; for (i = 0; i < adev->uvd.num_uvd_inst; i++) { if (adev->uvd.harvest_config & (1 << i)) @@ -432,7 +437,10 @@ int amdgpu_uvd_resume(struct amdgpu_device *adev) ptr = adev->uvd.inst[i].cpu_addr; if (adev->uvd.inst[i].saved_bo != NULL) { - memcpy_toio(ptr, adev->uvd.inst[i].saved_bo, size); + if (drm_dev_enter(&adev->ddev, &idx)) { + memcpy_toio(ptr, adev->uvd.inst[i].saved_bo, size); + drm_dev_exit(idx); + } kvfree(adev->uvd.inst[i].saved_bo); adev->uvd.inst[i].saved_bo = NULL; } else { @@ -442,8 +450,11 @@ int amdgpu_uvd_resume(struct amdgpu_device *adev) hdr = (const struct common_firmware_header *)adev->uvd.fw->data; if (adev->firmware.load_type != AMDGPU_FW_LOAD_PSP) { offset = le32_to_cpu(hdr->ucode_array_offset_bytes); - memcpy_toio(adev->uvd.inst[i].cpu_addr, adev->uvd.fw->data + offset, - le32_to_cpu(hdr->ucode_size_bytes)); + if (drm_dev_enter(&adev->ddev, &idx)) { + memcpy_toio(adev->uvd.inst[i].cpu_addr, adev->uvd.fw->data + offset, + le32_to_cpu(hdr->ucode_size_bytes)); + drm_dev_exit(idx); + } size -= le32_to_cpu(hdr->ucode_size_bytes); ptr += le32_to_cpu(hdr->ucode_size_bytes); } diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_vce.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_vce.c index ea6a62f67e38..833203401ef4 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_vce.c +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_vce.c @@ -29,6 +29,7 @@ #include #include +#include #include "amdgpu.h" #include "amdgpu_pm.h" @@ -293,7 +294,7 @@ int amdgpu_vce_resume(struct amdgpu_device *adev) void *cpu_addr; const struct common_firmware_header *hdr; unsigned offset; - int r; + int r, idx; if (adev->vce.vcpu_bo == NULL) return -EINVAL; @@ -313,8 +314,12 @@ int amdgpu_vce_resume(struct amdgpu_device *adev) hdr = (const struct common_firmware_header *)adev->vce.fw->data; offset = le32_to_cpu(hdr->ucode_array_offset_bytes); - memcpy_toio(cpu_addr, adev->vce.fw->data + offset, - adev->vce.fw->size - offset); + + if (drm_dev_enter(&adev->ddev, &idx)) { + memcpy_toio(cpu_addr, adev->vce.fw->data + offset, + adev->vce.fw->size - offset); + drm_dev_exit(idx); + } amdgpu_bo_kunmap(adev->vce.vcpu_bo); diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_vcn.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_vcn.c index 99b82f3c2617..b42db22761b8 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_vcn.c +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_vcn.c @@ -27,6 +27,7 @@ #include #include #include +#include #include "amdgpu.h" #include "amdgpu_pm.h" @@ -267,7 +268,7 @@ int amdgpu_vcn_suspend(struct amdgpu_device *adev) { unsigned size; void *ptr; - int i; + int i, idx; cancel_delayed_work_sync(&adev->vcn.idle_work); @@ -284,7 +285,10 @@ int amdgpu_vcn_suspend(struct amdgpu_device *adev) if (!adev->vcn.inst[i].saved_bo) return -ENOMEM; - memcpy_fromio(adev->vcn.inst[i].saved_bo, ptr, size); + if (drm_dev_enter(&adev->ddev, &idx)) { + memcpy_fromio(adev->vcn.inst[i].saved_bo, ptr, size); + drm_dev_exit(idx); + } } return 0; } @@ -293,7 +297,7 @@ int amdgpu_vcn_resume(struct amdgpu_device *adev) { unsigned size; void *ptr; - int i; + int i, idx; for (i = 0; i < adev->vcn.num_vcn_inst; ++i) { if (adev->vcn.harvest_config & (1 << i)) @@ -305,7 +309,10 @@ int amdgpu_vcn_resume(struct amdgpu_device *adev) ptr = adev->vcn.inst[i].cpu_addr; if (adev->vcn.inst[i].saved_bo != NULL) { - memcpy_toio(ptr, adev->vcn.inst[i].saved_bo, size); + if (drm_dev_enter(&adev->ddev, &idx)) { + memcpy_toio(ptr, adev->vcn.inst[i].saved_bo, size); + drm_dev_exit(idx); + } kvfree(adev->vcn.inst[i].saved_bo); adev->vcn.inst[i].saved_bo = NULL; } else { @@ -315,8 +322,11 @@ int amdgpu_vcn_resume(struct amdgpu_device *adev) hdr = (const struct common_firmware_header *)adev->vcn.fw->data; if (adev->firmware.load_type != AMDGPU_FW_LOAD_PSP) { offset = le32_to_cpu(hdr->ucode_array_offset_bytes); - memcpy_toio(adev->vcn.inst[i].cpu_addr, adev->vcn.fw->data + offset, - le32_to_cpu(hdr->ucode_size_bytes)); + if (drm_dev_enter(&adev->ddev, &idx)) { + memcpy_toio(adev->vcn.inst[i].cpu_addr, adev->vcn.fw->data + offset, + le32_to_cpu(hdr->ucode_size_bytes)); + drm_dev_exit(idx); + } size -= le32_to_cpu(hdr->ucode_size_bytes); ptr += le32_to_cpu(hdr->ucode_size_bytes); } diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c index ae18c0e32347..7b622056df58 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c @@ -31,6 +31,7 @@ #include #include +#include #include "amdgpu.h" #include "amdgpu_trace.h" #include "amdgpu_amdkfd.h" @@ -1604,7 +1605,10 @@ static int amdgpu_vm_bo_update_mapping(struct amdgpu_device *adev, struct amdgpu_vm_update_params params; enum amdgpu_sync_mode sync_mode; uint64_t pfn; - int r; + int r, idx; + + if (!drm_dev_enter(&adev->ddev, &idx)) + return -ENODEV; memset(¶ms, 0, sizeof(params)); params.adev = adev; @@ -1713,6 +1717,7 @@ static int amdgpu_vm_bo_update_mapping(struct amdgpu_device *adev, error_unlock: amdgpu_vm_eviction_unlock(vm); + drm_dev_exit(idx); return r; } diff --git a/drivers/gpu/drm/amd/amdgpu/psp_v11_0.c b/drivers/gpu/drm/amd/amdgpu/psp_v11_0.c index c325d6f53a71..94cce172b98e 100644 --- a/drivers/gpu/drm/amd/amdgpu/psp_v11_0.c +++ b/drivers/gpu/drm/amd/amdgpu/psp_v11_0.c @@ -23,6 +23,7 @@ #include #include #include +#include #include "amdgpu.h" #include "amdgpu_psp.h" @@ -269,10 +270,8 @@ static int psp_v11_0_bootloader_load_kdb(struct psp_context *psp) if (ret) return ret; - memset(psp->fw_pri_buf, 0, PSP_1_MEG); - /* Copy PSP KDB binary to memory */ - memcpy(psp->fw_pri_buf, psp->kdb_start_addr, psp->kdb_bin_size); + psp_copy_fw(psp, psp->kdb_start_addr, psp->kdb_bin_size); /* Provide the PSP KDB to bootloader */ WREG32_SOC15(MP0, 0, mmMP0_SMN_C2PMSG_36, @@ -302,10 +301,8 @@ static int psp_v11_0_bootloader_load_spl(struct psp_context *psp) if (ret) return ret; - memset(psp->fw_pri_buf, 0, PSP_1_MEG); - /* Copy PSP SPL binary to memory */ - memcpy(psp->fw_pri_buf, psp->spl_start_addr, psp->spl_bin_size); + psp_copy_fw(psp, psp->spl_start_addr, psp->spl_bin_size); /* Provide the PSP SPL to bootloader */ WREG32_SOC15(MP0, 0, mmMP0_SMN_C2PMSG_36, @@ -335,10 +332,8 @@ static int psp_v11_0_bootloader_load_sysdrv(struct psp_context *psp) if (ret) return ret; - memset(psp->fw_pri_buf, 0, PSP_1_MEG); - /* Copy PSP System Driver binary to memory */ - memcpy(psp->fw_pri_buf, psp->sys_start_addr, psp->sys_bin_size); + psp_copy_fw(psp, psp->sys_start_addr, psp->sys_bin_size); /* Provide the sys driver to bootloader */ WREG32_SOC15(MP0, 0, mmMP0_SMN_C2PMSG_36, @@ -371,10 +366,8 @@ static int psp_v11_0_bootloader_load_sos(struct psp_context *psp) if (ret) return ret; - memset(psp->fw_pri_buf, 0, PSP_1_MEG); - /* Copy Secure OS binary to PSP memory */ - memcpy(psp->fw_pri_buf, psp->sos_start_addr, psp->sos_bin_size); + psp_copy_fw(psp, psp->sos_start_addr, psp->sos_bin_size); /* Provide the PSP secure OS to bootloader */ WREG32_SOC15(MP0, 0, mmMP0_SMN_C2PMSG_36, @@ -608,7 +601,7 @@ static int psp_v11_0_memory_training(struct psp_context *psp, uint32_t ops) uint32_t p2c_header[4]; uint32_t sz; void *buf; - int ret; + int ret, idx; if (ctx->init == PSP_MEM_TRAIN_NOT_SUPPORT) { DRM_DEBUG("Memory training is not supported.\n"); @@ -681,17 +674,24 @@ static int psp_v11_0_memory_training(struct psp_context *psp, uint32_t ops) return -ENOMEM; } - memcpy_fromio(buf, adev->mman.aper_base_kaddr, sz); - ret = psp_v11_0_memory_training_send_msg(psp, PSP_BL__DRAM_LONG_TRAIN); - if (ret) { - DRM_ERROR("Send long training msg failed.\n"); + if (drm_dev_enter(&adev->ddev, &idx)) { + memcpy_fromio(buf, adev->mman.aper_base_kaddr, sz); + ret = psp_v11_0_memory_training_send_msg(psp, PSP_BL__DRAM_LONG_TRAIN); + if (ret) { + DRM_ERROR("Send long training msg failed.\n"); + vfree(buf); + drm_dev_exit(idx); + return ret; + } + + memcpy_toio(adev->mman.aper_base_kaddr, buf, sz); + adev->hdp.funcs->flush_hdp(adev, NULL); vfree(buf); - return ret; + drm_dev_exit(idx); + } else { + vfree(buf); + return -ENODEV; } - - memcpy_toio(adev->mman.aper_base_kaddr, buf, sz); - adev->hdp.funcs->flush_hdp(adev, NULL); - vfree(buf); } if (ops & PSP_MEM_TRAIN_SAVE) { diff --git a/drivers/gpu/drm/amd/amdgpu/psp_v12_0.c b/drivers/gpu/drm/amd/amdgpu/psp_v12_0.c index c4828bd3264b..618e5b6b85d9 100644 --- a/drivers/gpu/drm/amd/amdgpu/psp_v12_0.c +++ b/drivers/gpu/drm/amd/amdgpu/psp_v12_0.c @@ -138,10 +138,8 @@ static int psp_v12_0_bootloader_load_sysdrv(struct psp_context *psp) if (ret) return ret; - memset(psp->fw_pri_buf, 0, PSP_1_MEG); - /* Copy PSP System Driver binary to memory */ - memcpy(psp->fw_pri_buf, psp->sys_start_addr, psp->sys_bin_size); + psp_copy_fw(psp, psp->sys_start_addr, psp->sys_bin_size); /* Provide the sys driver to bootloader */ WREG32_SOC15(MP0, 0, mmMP0_SMN_C2PMSG_36, @@ -179,10 +177,8 @@ static int psp_v12_0_bootloader_load_sos(struct psp_context *psp) if (ret) return ret; - memset(psp->fw_pri_buf, 0, PSP_1_MEG); - /* Copy Secure OS binary to PSP memory */ - memcpy(psp->fw_pri_buf, psp->sos_start_addr, psp->sos_bin_size); + psp_copy_fw(psp, psp->sos_start_addr, psp->sos_bin_size); /* Provide the PSP secure OS to bootloader */ WREG32_SOC15(MP0, 0, mmMP0_SMN_C2PMSG_36, diff --git a/drivers/gpu/drm/amd/amdgpu/psp_v3_1.c b/drivers/gpu/drm/amd/amdgpu/psp_v3_1.c index f2e725f72d2f..d0a6cccd0897 100644 --- a/drivers/gpu/drm/amd/amdgpu/psp_v3_1.c +++ b/drivers/gpu/drm/amd/amdgpu/psp_v3_1.c @@ -102,10 +102,8 @@ static int psp_v3_1_bootloader_load_sysdrv(struct psp_context *psp) if (ret) return ret; - memset(psp->fw_pri_buf, 0, PSP_1_MEG); - /* Copy PSP System Driver binary to memory */ - memcpy(psp->fw_pri_buf, psp->sys_start_addr, psp->sys_bin_size); + psp_copy_fw(psp, psp->sys_start_addr, psp->sys_bin_size); /* Provide the sys driver to bootloader */ WREG32_SOC15(MP0, 0, mmMP0_SMN_C2PMSG_36, @@ -143,10 +141,8 @@ static int psp_v3_1_bootloader_load_sos(struct psp_context *psp) if (ret) return ret; - memset(psp->fw_pri_buf, 0, PSP_1_MEG); - /* Copy Secure OS binary to PSP memory */ - memcpy(psp->fw_pri_buf, psp->sos_start_addr, psp->sos_bin_size); + psp_copy_fw(psp, psp->sos_start_addr, psp->sos_bin_size); /* Provide the PSP secure OS to bootloader */ WREG32_SOC15(MP0, 0, mmMP0_SMN_C2PMSG_36, diff --git a/drivers/gpu/drm/amd/amdgpu/vce_v4_0.c b/drivers/gpu/drm/amd/amdgpu/vce_v4_0.c index 37fa163393fd..fd859c778df0 100644 --- a/drivers/gpu/drm/amd/amdgpu/vce_v4_0.c +++ b/drivers/gpu/drm/amd/amdgpu/vce_v4_0.c @@ -25,6 +25,7 @@ */ #include +#include #include "amdgpu.h" #include "amdgpu_vce.h" @@ -555,16 +556,19 @@ static int vce_v4_0_hw_fini(void *handle) static int vce_v4_0_suspend(void *handle) { struct amdgpu_device *adev = (struct amdgpu_device *)handle; - int r; + int r, idx; if (adev->vce.vcpu_bo == NULL) return 0; - if (adev->firmware.load_type == AMDGPU_FW_LOAD_PSP) { - unsigned size = amdgpu_bo_size(adev->vce.vcpu_bo); - void *ptr = adev->vce.cpu_addr; + if (drm_dev_enter(&adev->ddev, &idx)) { + if (adev->firmware.load_type == AMDGPU_FW_LOAD_PSP) { + unsigned size = amdgpu_bo_size(adev->vce.vcpu_bo); + void *ptr = adev->vce.cpu_addr; - memcpy_fromio(adev->vce.saved_bo, ptr, size); + memcpy_fromio(adev->vce.saved_bo, ptr, size); + } + drm_dev_exit(idx); } r = vce_v4_0_hw_fini(adev); @@ -577,16 +581,20 @@ static int vce_v4_0_suspend(void *handle) static int vce_v4_0_resume(void *handle) { struct amdgpu_device *adev = (struct amdgpu_device *)handle; - int r; + int r, idx; if (adev->vce.vcpu_bo == NULL) return -EINVAL; if (adev->firmware.load_type == AMDGPU_FW_LOAD_PSP) { - unsigned size = amdgpu_bo_size(adev->vce.vcpu_bo); - void *ptr = adev->vce.cpu_addr; - memcpy_toio(ptr, adev->vce.saved_bo, size); + if (drm_dev_enter(&adev->ddev, &idx)) { + unsigned size = amdgpu_bo_size(adev->vce.vcpu_bo); + void *ptr = adev->vce.cpu_addr; + + memcpy_toio(ptr, adev->vce.saved_bo, size); + drm_dev_exit(idx); + } } else { r = amdgpu_vce_resume(adev); if (r) diff --git a/drivers/gpu/drm/amd/amdgpu/vcn_v3_0.c b/drivers/gpu/drm/amd/amdgpu/vcn_v3_0.c index def583916294..5ac8ffe197aa 100644 --- a/drivers/gpu/drm/amd/amdgpu/vcn_v3_0.c +++ b/drivers/gpu/drm/amd/amdgpu/vcn_v3_0.c @@ -34,6 +34,8 @@ #include "vcn/vcn_3_0_0_sh_mask.h" #include "ivsrcid/vcn/irqsrcs_vcn_2_0.h" +#include + #define mmUVD_CONTEXT_ID_INTERNAL_OFFSET 0x27 #define mmUVD_GPCOM_VCPU_CMD_INTERNAL_OFFSET 0x0f #define mmUVD_GPCOM_VCPU_DATA0_INTERNAL_OFFSET 0x10 @@ -263,16 +265,20 @@ static int vcn_v3_0_sw_init(void *handle) static int vcn_v3_0_sw_fini(void *handle) { struct amdgpu_device *adev = (struct amdgpu_device *)handle; - int i, r; + int i, r, idx; - for (i = 0; i < adev->vcn.num_vcn_inst; i++) { - volatile struct amdgpu_fw_shared *fw_shared; + if (drm_dev_enter(&adev->ddev, &idx)) { + for (i = 0; i < adev->vcn.num_vcn_inst; i++) { + volatile struct amdgpu_fw_shared *fw_shared; - if (adev->vcn.harvest_config & (1 << i)) - continue; - fw_shared = adev->vcn.inst[i].fw_shared_cpu_addr; - fw_shared->present_flag_0 = 0; - fw_shared->sw_ring.is_enabled = false; + if (adev->vcn.harvest_config & (1 << i)) + continue; + fw_shared = adev->vcn.inst[i].fw_shared_cpu_addr; + fw_shared->present_flag_0 = 0; + fw_shared->sw_ring.is_enabled = false; + } + + drm_dev_exit(idx); } if (amdgpu_sriov_vf(adev)) diff --git a/drivers/gpu/drm/amd/pm/powerplay/smumgr/smu7_smumgr.c b/drivers/gpu/drm/amd/pm/powerplay/smumgr/smu7_smumgr.c index aae25243eb10..d628b91846c9 100644 --- a/drivers/gpu/drm/amd/pm/powerplay/smumgr/smu7_smumgr.c +++ b/drivers/gpu/drm/amd/pm/powerplay/smumgr/smu7_smumgr.c @@ -405,6 +405,8 @@ int smu7_request_smu_load_fw(struct pp_hwmgr *hwmgr) UCODE_ID_MEC_STORAGE, &toc->entry[toc->num_entries++]), "Failed to Get Firmware Entry.", r = -EINVAL; goto failed); } + + /* AG TODO Can't call drm_dev_enter/exit because access adev->ddev here ... */ memcpy_toio(smu_data->header_buffer.kaddr, smu_data->toc, sizeof(struct SMU_DRAMData_TOC)); smum_send_msg_to_smc_with_parameter(hwmgr, From patchwork Wed Apr 28 15:11:51 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrey Grodzovsky X-Patchwork-Id: 12229239 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.5 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,MSGID_FROM_MTA_HEADER,SPF_HELO_NONE,SPF_PASS, USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 8BF68C43461 for ; Wed, 28 Apr 2021 15:13:12 +0000 (UTC) Received: from gabe.freedesktop.org (gabe.freedesktop.org [131.252.210.177]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 385EF61434 for ; Wed, 28 Apr 2021 15:13:12 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 385EF61434 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=amd.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=dri-devel-bounces@lists.freedesktop.org Received: from gabe.freedesktop.org (localhost [127.0.0.1]) by gabe.freedesktop.org (Postfix) with ESMTP id 1072A6EB6D; Wed, 28 Apr 2021 15:13:11 +0000 (UTC) Received: from NAM12-MW2-obe.outbound.protection.outlook.com (mail-mw2nam12on2080.outbound.protection.outlook.com [40.107.244.80]) by gabe.freedesktop.org (Postfix) with ESMTPS id 1FC3E6EB61; Wed, 28 Apr 2021 15:12:55 +0000 (UTC) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=YENy3VyRcwy/D73IUjPpPyzcis+9CtCSy+7I71QYp0SPndqud7sFmXbHj5yEwDOUr9+SYZWzBk0Rlg8trCPPBwepSKR+8e9Dop2P2yrdiQMmAp4r+MZ5a/mesc1Vh8Y3Ey5ZmrDUsYr6RvdpXYbFGslDYxMWL2lQrPrO/1SZYaDomUAgT4GH3hWhFXCEf8PV55Xdnh9j/KMf3RvJxW7tGbP4nUYLoFkD8D2gAfVWZhld8xq/t3sITVs1+k5J2ZTXnMS2Ue73OsHAJDunV8CWpVFhM9oT+3m60asafhhxdXEfXJsgsyASVgS5321HPuwfxv29XBUosZ9hZPDDLsRdpw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=1ylkxX6mZ6oWnt6g0gOSaNMKTRfvHu/TDcdKALZjVVA=; b=i7NvYa+waJ/hAro3h4kgjiZ/a6N6lQ28bt7YsYNrYex1hgPQu1Xb+pPJnNzutb0LFx+exLAH9QwrpO9UgTZ3QEDQ2avwDaosgAYKW7cHC1+HpCpEZ0FMyDKVqJnxhJrWzALoa77uoQKqVl7V5jjljMKMPhjAD4EREa8PFvOXuEwpWF1a+NRUzbPis0JO0+OguEHTcNrXjxy03JKwAn98cJq1nrOgQJXBrgciRevjYShAjyNa6NxAYCp2fD7nKK2jyLT6eRL9V/8AZel1q0Md4sdHRujLMCav360AuUm0WvWa4F+c0Iw5CYI79qQG9UwPTLxsgXxisuJfXobxgxyjew== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=amd.com; dmarc=pass action=none header.from=amd.com; dkim=pass header.d=amd.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=1ylkxX6mZ6oWnt6g0gOSaNMKTRfvHu/TDcdKALZjVVA=; b=mPUiojerRpgI0zaDLLb+F6jEklxt3+7/qWJUa2lHDbYUjQS2lTGbSTLdobpUrZYUQQTTErmEeOoV6N0npOQ6gms0H7ho2+2z3xcYNGGXGjGwvkg/BYWPkGtLaV68hHbkNyECT/Bg2Bd+B7nvQ4fWdGzQIPgY8312/BiAjts3E74= Authentication-Results: lists.freedesktop.org; dkim=none (message not signed) header.d=none; lists.freedesktop.org; dmarc=none action=none header.from=amd.com; Received: from SN6PR12MB4623.namprd12.prod.outlook.com (2603:10b6:805:e9::17) by SA0PR12MB4462.namprd12.prod.outlook.com (2603:10b6:806:95::22) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4065.23; Wed, 28 Apr 2021 15:12:52 +0000 Received: from SN6PR12MB4623.namprd12.prod.outlook.com ([fe80::ad51:8c49:b171:856c]) by SN6PR12MB4623.namprd12.prod.outlook.com ([fe80::ad51:8c49:b171:856c%7]) with mapi id 15.20.4065.026; Wed, 28 Apr 2021 15:12:52 +0000 From: Andrey Grodzovsky To: dri-devel@lists.freedesktop.org, amd-gfx@lists.freedesktop.org, linux-pci@vger.kernel.org, ckoenig.leichtzumerken@gmail.com, daniel.vetter@ffwll.ch, Harry.Wentland@amd.com Subject: [PATCH v5 11/27] drm/sched: Make timeout timer rearm conditional. Date: Wed, 28 Apr 2021 11:11:51 -0400 Message-Id: <20210428151207.1212258-12-andrey.grodzovsky@amd.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210428151207.1212258-1-andrey.grodzovsky@amd.com> References: <20210428151207.1212258-1-andrey.grodzovsky@amd.com> X-Originating-IP: [2607:fea8:3edf:49b0:7212:f93a:73b0:8f23] X-ClientProxiedBy: YT1PR01CA0142.CANPRD01.PROD.OUTLOOK.COM (2603:10b6:b01:2f::21) To SN6PR12MB4623.namprd12.prod.outlook.com (2603:10b6:805:e9::17) MIME-Version: 1.0 X-MS-Exchange-MessageSentRepresentingType: 1 Received: from agrodzovsky-All-Series.hitronhub.home (2607:fea8:3edf:49b0:7212:f93a:73b0:8f23) by YT1PR01CA0142.CANPRD01.PROD.OUTLOOK.COM (2603:10b6:b01:2f::21) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4065.25 via Frontend Transport; Wed, 28 Apr 2021 15:12:51 +0000 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 866bd927-0547-44ca-988c-08d90a5817e3 X-MS-TrafficTypeDiagnostic: SA0PR12MB4462: X-MS-Exchange-Transport-Forked: True X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:6108; X-MS-Exchange-SenderADCheck: 1 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: 76bFNYBtnLjIx5GRtCRicTgOazg2aYyCr5wkNtHNop2IAAUu0uHOYK1bnY+nLgsi5eq+gD2Pfp5VxA/O697VTDwM2CojwTQGJTzabnnXzrKx0AWjtkJynyK2dF7Szron1HS+wkjmUXWMpl+LZbmP//vOQLtFVw7K0CfiDQ3fcryOr9f1jo/aFJ25pV9VTZYifHGwJoPelvDJcInnD+XT+FLq+kzS7J2rA7LE8KI8StGho7TRMu7bPH/HblkTBVQuz9d48ybnckr4rEaYSdV5o69/tTpczgK9Z0D0WVgY8pmEwkxeAuz5BEblIei32MsBPxkw+Zn1ViaGG5xb2rzBlbAwBxgFASD8RaLsw8Bk4zDVla2HNqn7YquN/7pTBQHKH/zeaCY2+57qgPDUJiAekm3zH+AV0R8qn4UtTPQGSmO9b9iyt69HlmAfDWJ7PpJeWpT8ykc1jWGB5TnyGzpN3WP9KlDmCrxsc98KOmlqJOEV/CbMSNkfcqowllqf+lwr8R/VExHe7A9QSVoyHpQ9ynZi3gSS/+MB4PlNKIi35yABiS5/kCEcv+tt7DSb6GdH27dSD9PMmVKYCpZ2CC1Z/frnkHLO7UZGeY7uNBcSTzs= X-Forefront-Antispam-Report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:SN6PR12MB4623.namprd12.prod.outlook.com; PTR:; CAT:NONE; SFS:(4636009)(346002)(396003)(136003)(39860400002)(376002)(366004)(6512007)(6636002)(66946007)(316002)(186003)(44832011)(2906002)(16526019)(36756003)(83380400001)(6506007)(4326008)(52116002)(8676002)(66476007)(5660300002)(66556008)(1076003)(2616005)(478600001)(86362001)(38100700002)(8936002)(6486002); DIR:OUT; SFP:1101; X-MS-Exchange-AntiSpam-MessageData: 0OQ+DfrlsaUZoeEZ+G4hz1TYQGval0CAUPDVWcUU2TKxMyn75HsGnjI5GnZASdzvg68UpygPJXSWz126ubZ2sryM6kcIvCl2BPWo9em2oseWC2DqXDeTBJDQNOq311mO2kfGlo54ZoyqEIiBAvJNQtI8zI2wXOs+aNOM/5phF/p1+4N0g4ZfIFBEap6vcwhoJuGCpPNJ0mncfWaO9AfGHboaEIHt3yKNH7VYHXxWkJtM/eOnG1N2BBXUMmyyadP2b6x8QCuzMw05vNX/qtTfVjbFXLdHXhGh+OJRnhS4mbFMV5kY1L3gt8DzhSuPCCG7N2HqWbVYy9K99Qx2WTsdr0AkQujWOInERPGVTp2uxIJbTWFKZl4h9DMfWpBd/7otC2902a+CBx2pbz/H+vWwDP36tcCfGVvEMFz71fFurgN4E+1XCuHaaQoipw1Aw/Bwh7s5xCv/rYcnMPyx+0OBx60hcWI4zIbFeVUzRFLEeL8G0++m2d4K6fLCyqEU6YRERBLjM5h1qegoZ2DH0EaGB+OX3iC2NnRtm0Pk7hXHI9MOzWJOy0SAP+17p9bdtNOfA6GBWxDjtAq4lloFYot21YDtZWIvCX+ll5LeR/MukqttX9eim9SrP8EYCYyobYbPQPQZ2gg5KWWckA/clctViiymbEEY9TSmLch39tDRzI8fxNnccEB9jkIeievcWZhiBMWCiuJuKeVwciKZH6LXxDCyNUWkQI0sjsGKZsQQhHtQVkAJ/bXU93pP34uH+ZSUtAWB7iBvAGxhWqU5tDJE2MC/k/H4ECXidp4sif/f05/JP5w59b6qN6lgI6Bd9VdcL5kGp7WY0eXciVbl1Y+qf9arXa1xTZZkulucBiU8WwlifJjgUoy7To/78I+Cmib9dh/oERfMmHjMEogxRbpULUbsrv5ez4T7lc/tiC/5d6XWzckReENjIZoiKt0qBCGFmaJN0727vxETInnTznWobnJUe9mj3FknmU9xS9iUBgyRn5nynKnhRpby9dvFCp/s6eZBRkV9PSvhubjCKmvjUFfQ891L9p3Di0AqgArmIrwAHjYML3dpOiPM3X2q9dYAK19m+1+3LOTWUXbyGNBqFgksj+pu3hKIjmnWG3nvGP+N81pwjoaVBAE8TbLJW/NrzyAWQmKHzzGqGk8UZBeEZK7eCn6qoPSsD2tUwwpENloDX8CaQR8unZL8ohEkhQjabbgKoKmMINhJiOBMR1BfWOJEhw/L1MszhHduoZJGNOz7JCGIfEcfkM5nrV1o+CXHxTWbjfZgRgVON0Xuq8FLk1LqOjFK2ysxWg8Ye5WPu/rIiTQxhtTyfEa2mfUI3Fpi/OurdEBnCTSgKJjF7c4WQ2HRjY4ZEP9iNzMOPJnEKii5inV77cleA2qyQ5duVlCB X-OriginatorOrg: amd.com X-MS-Exchange-CrossTenant-Network-Message-Id: 866bd927-0547-44ca-988c-08d90a5817e3 X-MS-Exchange-CrossTenant-AuthSource: SN6PR12MB4623.namprd12.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 28 Apr 2021 15:12:52.4856 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: dasEwWrwzh0D6SlaNehT+QZwEZDUA1PWrN9jKG1UVwczlyAPMX/VvncpKZL95da/m/GqferNpdIkpXpuI+ZrgQ== X-MS-Exchange-Transport-CrossTenantHeadersStamped: SA0PR12MB4462 X-BeenThere: dri-devel@lists.freedesktop.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Direct Rendering Infrastructure - Development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: gregkh@linuxfoundation.org, Felix.Kuehling@amd.com, helgaas@kernel.org, Alexander.Deucher@amd.com Errors-To: dri-devel-bounces@lists.freedesktop.org Sender: "dri-devel" We don't want to rearm the timer if driver hook reports that the device is gone. v5: Update drm_gpu_sched_stat values in code. Signed-off-by: Andrey Grodzovsky --- drivers/gpu/drm/scheduler/sched_main.c | 11 +++++++---- 1 file changed, 7 insertions(+), 4 deletions(-) diff --git a/drivers/gpu/drm/scheduler/sched_main.c b/drivers/gpu/drm/scheduler/sched_main.c index d82a7ebf6099..908b0b56032d 100644 --- a/drivers/gpu/drm/scheduler/sched_main.c +++ b/drivers/gpu/drm/scheduler/sched_main.c @@ -314,6 +314,7 @@ static void drm_sched_job_timedout(struct work_struct *work) { struct drm_gpu_scheduler *sched; struct drm_sched_job *job; + enum drm_gpu_sched_stat status = DRM_GPU_SCHED_STAT_NOMINAL; sched = container_of(work, struct drm_gpu_scheduler, work_tdr.work); @@ -331,7 +332,7 @@ static void drm_sched_job_timedout(struct work_struct *work) list_del_init(&job->list); spin_unlock(&sched->job_list_lock); - job->sched->ops->timedout_job(job); + status = job->sched->ops->timedout_job(job); /* * Guilty job did complete and hence needs to be manually removed @@ -345,9 +346,11 @@ static void drm_sched_job_timedout(struct work_struct *work) spin_unlock(&sched->job_list_lock); } - spin_lock(&sched->job_list_lock); - drm_sched_start_timeout(sched); - spin_unlock(&sched->job_list_lock); + if (status != DRM_GPU_SCHED_STAT_ENODEV) { + spin_lock(&sched->job_list_lock); + drm_sched_start_timeout(sched); + spin_unlock(&sched->job_list_lock); + } } /** From patchwork Wed Apr 28 15:11:52 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrey Grodzovsky X-Patchwork-Id: 12229277 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.5 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,MSGID_FROM_MTA_HEADER,SPF_HELO_NONE,SPF_PASS, USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id B12BCC43470 for ; Wed, 28 Apr 2021 15:14:21 +0000 (UTC) Received: from gabe.freedesktop.org (gabe.freedesktop.org [131.252.210.177]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 591526143A for ; Wed, 28 Apr 2021 15:14:21 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 591526143A Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=amd.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=dri-devel-bounces@lists.freedesktop.org Received: from gabe.freedesktop.org (localhost [127.0.0.1]) by gabe.freedesktop.org (Postfix) with ESMTP id C59566EB94; Wed, 28 Apr 2021 15:14:19 +0000 (UTC) Received: from NAM12-MW2-obe.outbound.protection.outlook.com (mail-mw2nam12on2080.outbound.protection.outlook.com [40.107.244.80]) by gabe.freedesktop.org (Postfix) with ESMTPS id 57C806EB57; Wed, 28 Apr 2021 15:12:56 +0000 (UTC) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=eRuygGgmKasg9Kjv8OyOghjDg1r42c7+aplNfOI3jYKFmp7V+xmDLdXsXwIUqsL43xSZUR0CcMMcqiv0dFckaRV4Bhd+MlFgDOzSLF77LSFvXTNDXajiArQTrBINZNGTdSnmDYjjQfq34e4vMefqGKKr+p2fK4dtu8J5VyK/bM3ICiH/OKgga9MZG55gfbX45AWm2F54M6N3EY8IbZ8fFkuKDWQZ69e1l7EhLcELKG7LRxPG7F0Djh9pVJgQ30szH/um4H2ZJHEoqPhPrJAquSKA0+1BtGNRFiZ2duzP48vx2494LrYUZmZmlA38H10cBQCEHvp+Q8gvjm/SlPrK0Q== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=Yq2Nfd7RRClQQ37WYASWsolamzag+TDEAOgKHpaDDxg=; b=Cor7GaEHsxioXb1sMWCDKmgUpLUQN3W92vL72JYuolvZi+3xlAi6+dR9sFmcO6e04oFtGHWfMx/hpFpK53ChqpeEI/OCwIbfHLkzF7aQF/HUGgbIZycrNyqLdctW0IRWNZHSIAdpukxmxmMRsBkFcQuougWHaUD4wRVFNn8W4BQiD3JY0i+fQN5dp8qo56hccJNqfdzfQiEs/Ih74Qu66c5v57Vr/m+MEiShhdc//SRqHYbDBhnF1VhzfGq5klsUANCBrGhEjQVUyGBf7/3ZH9hugWesvJm6Jg2pOpHIccDE6QOJcvUJfOoljaZGEdbS7XH3qO7cRN+ADbXMlokzfQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=amd.com; dmarc=pass action=none header.from=amd.com; dkim=pass header.d=amd.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=Yq2Nfd7RRClQQ37WYASWsolamzag+TDEAOgKHpaDDxg=; b=2aTvsDnRUjR3EOUdz9sxg75QN2olI0AGSm1F1jrt1yZCfVazUdFx+tMML30gs1oYbGv0c4iPmCNb+ZmQiUzROyi2VD35t0oy6TzvnSBL/BZ+2FLbJCVMlAzyFYQWUwsVAAiY9H9J6NnFpigIj5gVA/cVHaUNeM0u1woUofyGjEQ= Authentication-Results: lists.freedesktop.org; dkim=none (message not signed) header.d=none; lists.freedesktop.org; dmarc=none action=none header.from=amd.com; Received: from SN6PR12MB4623.namprd12.prod.outlook.com (2603:10b6:805:e9::17) by SA0PR12MB4462.namprd12.prod.outlook.com (2603:10b6:806:95::22) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4065.23; Wed, 28 Apr 2021 15:12:54 +0000 Received: from SN6PR12MB4623.namprd12.prod.outlook.com ([fe80::ad51:8c49:b171:856c]) by SN6PR12MB4623.namprd12.prod.outlook.com ([fe80::ad51:8c49:b171:856c%7]) with mapi id 15.20.4065.026; Wed, 28 Apr 2021 15:12:54 +0000 From: Andrey Grodzovsky To: dri-devel@lists.freedesktop.org, amd-gfx@lists.freedesktop.org, linux-pci@vger.kernel.org, ckoenig.leichtzumerken@gmail.com, daniel.vetter@ffwll.ch, Harry.Wentland@amd.com Subject: [PATCH v5 12/27] drm/amdgpu: Prevent any job recoveries after device is unplugged. Date: Wed, 28 Apr 2021 11:11:52 -0400 Message-Id: <20210428151207.1212258-13-andrey.grodzovsky@amd.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210428151207.1212258-1-andrey.grodzovsky@amd.com> References: <20210428151207.1212258-1-andrey.grodzovsky@amd.com> X-Originating-IP: [2607:fea8:3edf:49b0:7212:f93a:73b0:8f23] X-ClientProxiedBy: YT1PR01CA0142.CANPRD01.PROD.OUTLOOK.COM (2603:10b6:b01:2f::21) To SN6PR12MB4623.namprd12.prod.outlook.com (2603:10b6:805:e9::17) MIME-Version: 1.0 X-MS-Exchange-MessageSentRepresentingType: 1 Received: from agrodzovsky-All-Series.hitronhub.home (2607:fea8:3edf:49b0:7212:f93a:73b0:8f23) by YT1PR01CA0142.CANPRD01.PROD.OUTLOOK.COM (2603:10b6:b01:2f::21) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4065.25 via Frontend Transport; Wed, 28 Apr 2021 15:12:53 +0000 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 4caeca8b-64ce-47d0-d185-08d90a5818d8 X-MS-TrafficTypeDiagnostic: SA0PR12MB4462: X-MS-Exchange-Transport-Forked: True X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:5797; X-MS-Exchange-SenderADCheck: 1 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: OxorgGlpB5BJhIgKQW0O8o8MsICkkUr7/QLZDXTvAmq5RdlyuZuBNS4ydPEB49+zggycUhvomD6IFdZNzYpYgZ5lX3fBxohBaWeLvJ+s4dgehSmoKL18qItgJQnkAU4jUqUsyjIPuDLYeeeM0WTnYfeXzSf2RlxQIzXFL5xn/M5KWlcuXeWwgdWam3hb800hJdX+rGWXpSE0WhZbuC3zY843weuQuFl9YaYJbpKq8o3Efjiz3uuW5a40Z9cCMruUMipqQ+U9ocK1Y7PduEBHSdmKHbTaTn3nWM17mPjCrKnDlhju0uPoxrdY5G5leRDYZEyDl33eJ/S3AJqKRL5Z0dOGIu94GornXKYY8m7t6LcjGI6Gy+6udK2gKge4C5GJ8Nnh+iBs0xiiQTpeTelGFQFdT3dlPh6At5uLCtacqm4MjswQfSec5tk28ZiQmhjgNBUQKcKJXOvLMIOSddCUZKO/MhkAi1LtJcFVLYLvkjORJGZ258blSZTXuoa3EmLh1Ol+DJ4glAKS/83nq5wjD7oCtKzl2dvDivw5J1O8MVUMTuvftpgxYe2x4DmwNFvM8KM8T4SqT25uQ6IzDGEne2KS509jvQsUzu5756DuaVI= X-Forefront-Antispam-Report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:SN6PR12MB4623.namprd12.prod.outlook.com; PTR:; CAT:NONE; SFS:(4636009)(346002)(396003)(136003)(39860400002)(376002)(366004)(6512007)(6636002)(66946007)(316002)(186003)(44832011)(2906002)(16526019)(36756003)(83380400001)(6506007)(4326008)(52116002)(8676002)(66476007)(5660300002)(66556008)(1076003)(2616005)(478600001)(86362001)(38100700002)(8936002)(6486002); DIR:OUT; SFP:1101; X-MS-Exchange-AntiSpam-MessageData: 6FtouEJT7cyLFKUNDNNItX55DGroqe3icwqePAcQSf182nX36a0h6t+Hbzjk7/65TquuMP3yzbJ0u+02EpGE8CFsSmt8//37XH2gPn4c8YeGAe7TFAMYVtK+gO32SUcuyePc7TLpBOjsdZcdROBdeuwEHkLgFOPIChEIdfJIP9nrt49kG2g4IUfMlyLWj/2m7SB+IBpE+G0iVwp/XpeQuRNb56Lu5oEvhrTpzHzC/n7Vy6HTqdXpm+gnWqVDUbB/FIFlHpk34lIlztIXToq2bJSCKofJWAfVuJP1pTgxO8m81zvd9YLdWTGrqiQecfp4aWw/uyJuCV5QBM53/pO+xTByj0rvzzrGXv6pfdBDzcVOkFBH0Lx9O3I24pn3EX5RDaI++gU66DX78RzQJg04X4f+4qzGlfa5IUAPxsqspZ4y1Sza455mUsVvw2H8uUkxyHIkvSSTijUZ5UStbV+YJ6MOi1QgKGjbA9rHG+8ag7skjBOvjEzTajX76kI68Z1hoAemUWR3T2iXpTs+rTB93a5qGCawfkx5XhqLbvNs1/m4lVpKZ/ShKqL3G5bhMScl2qudFM/lNxEJgLrkEkj2oR41kDjPf5/NLkq4tUqsCvH9oWrE17rc5i93hWbEDp6XDun6NiMoSBEpFjLzugkGOu7FNiqGKYi60hRkQha8XxX3bgksTmnll+7w5lVtP2oC/fCQEKlCNDBMVBsZJ3dmSullpFtDm6hIdbINBCxlsp76CmJov1+Aly9Vcg72mXKivd7/ME/hTFXJOLfKb7RLZ1tsGxuRhGOIszJZMpOFWOny5TvoF7tNcc1lrPHp6as5n5IJh+oVHN8QlfDzyDq+lKTiaRySG6Ds22i5xKOLQniofIHB1ppohlcf3/d+j32Q11fl7ls0rwXFffkMAqnIDhUumeFr/q5n5wkg3hcnAJ4xrvSH5ROpH3mnKq0HUffNHF+3XFqCfDL3YZk/snK9K/dooeQpvAHpq6FTGr9gt1LtWnx1reXuR2v9NItW2qw307j8gJpKJ9YUUVTGZZMwF19fQUDOIKO0419WdzI9gZUQsgJxJ2klUS8yXo3vViWQdMGRKKk5BfQzm+PAnej53EsrCDaxO805e9L7IoZWmHFP83ctpTGALfAPka7DePG5p+ivQNx/AI/5nLJ7sjIJFzLkqdI1prii7dfcthc4d4df4S/CKhuwlas43Fky7KZt1QGSlcqE+t8oYKnHkRNhwiQwzLrEPNfl+l3oIHFMFji0lRJhxz8PuwAvFEHMh74tkdtpEGU+D+5ffhvkgryhpujc+0a9eaus7etMta2y5/KxHYV2EyrimCqYOBKhUGWMX4cyM0ZHaaMNSU3hjasujyV1S/Q/uNdWkKfUMHmAUrRliuyzXHbVdDxUAKQw0+a4 X-OriginatorOrg: amd.com X-MS-Exchange-CrossTenant-Network-Message-Id: 4caeca8b-64ce-47d0-d185-08d90a5818d8 X-MS-Exchange-CrossTenant-AuthSource: SN6PR12MB4623.namprd12.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 28 Apr 2021 15:12:54.0887 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: 133BX9YHX00l1RRiDze3iD77DRJBA8gQAvTsHQ5umBH83n/kqThEAwXmEOCQ9NQEO37pOLI5NZLZWzcMRNxjJA== X-MS-Exchange-Transport-CrossTenantHeadersStamped: SA0PR12MB4462 X-BeenThere: dri-devel@lists.freedesktop.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Direct Rendering Infrastructure - Development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: gregkh@linuxfoundation.org, Felix.Kuehling@amd.com, helgaas@kernel.org, Alexander.Deucher@amd.com Errors-To: dri-devel-bounces@lists.freedesktop.org Sender: "dri-devel" Return DRM_TASK_STATUS_ENODEV back to the scheduler when device is not present so they timeout timer will not be rearmed. v5: Update to match updated return values in enum drm_gpu_sched_stat Signed-off-by: Andrey Grodzovsky --- drivers/gpu/drm/amd/amdgpu/amdgpu_job.c | 19 ++++++++++++++++--- 1 file changed, 16 insertions(+), 3 deletions(-) diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_job.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_job.c index 759b34799221..d33e6d97cc89 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_job.c +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_job.c @@ -25,6 +25,8 @@ #include #include +#include + #include "amdgpu.h" #include "amdgpu_trace.h" @@ -34,6 +36,15 @@ static enum drm_gpu_sched_stat amdgpu_job_timedout(struct drm_sched_job *s_job) struct amdgpu_job *job = to_amdgpu_job(s_job); struct amdgpu_task_info ti; struct amdgpu_device *adev = ring->adev; + int idx; + + if (!drm_dev_enter(&adev->ddev, &idx)) { + DRM_INFO("%s - device unplugged skipping recovery on scheduler:%s", + __func__, s_job->sched->name); + + /* Effectively the job is aborted as the device is gone */ + return DRM_GPU_SCHED_STAT_ENODEV; + } memset(&ti, 0, sizeof(struct amdgpu_task_info)); @@ -41,7 +52,7 @@ static enum drm_gpu_sched_stat amdgpu_job_timedout(struct drm_sched_job *s_job) amdgpu_ring_soft_recovery(ring, job->vmid, s_job->s_fence->parent)) { DRM_ERROR("ring %s timeout, but soft recovered\n", s_job->sched->name); - return DRM_GPU_SCHED_STAT_NOMINAL; + goto exit; } amdgpu_vm_get_task_info(ring->adev, job->pasid, &ti); @@ -53,13 +64,15 @@ static enum drm_gpu_sched_stat amdgpu_job_timedout(struct drm_sched_job *s_job) if (amdgpu_device_should_recover_gpu(ring->adev)) { amdgpu_device_gpu_recover(ring->adev, job); - return DRM_GPU_SCHED_STAT_NOMINAL; } else { drm_sched_suspend_timeout(&ring->sched); if (amdgpu_sriov_vf(adev)) adev->virt.tdr_debug = true; - return DRM_GPU_SCHED_STAT_NOMINAL; } + +exit: + drm_dev_exit(idx); + return DRM_GPU_SCHED_STAT_NOMINAL; } int amdgpu_job_alloc(struct amdgpu_device *adev, unsigned num_ibs, From patchwork Wed Apr 28 15:11:53 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrey Grodzovsky X-Patchwork-Id: 12229263 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.5 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,MSGID_FROM_MTA_HEADER,SPF_HELO_NONE,SPF_PASS, USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id BA4B1C43460 for ; Wed, 28 Apr 2021 15:14:00 +0000 (UTC) Received: from gabe.freedesktop.org (gabe.freedesktop.org [131.252.210.177]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id C7AC161434 for ; Wed, 28 Apr 2021 15:13:58 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org C7AC161434 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=amd.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=dri-devel-bounces@lists.freedesktop.org Received: from gabe.freedesktop.org (localhost [127.0.0.1]) by gabe.freedesktop.org (Postfix) with ESMTP id 1CFC86EB87; Wed, 28 Apr 2021 15:13:56 +0000 (UTC) Received: from NAM12-MW2-obe.outbound.protection.outlook.com (mail-mw2nam12on2080.outbound.protection.outlook.com [40.107.244.80]) by gabe.freedesktop.org (Postfix) with ESMTPS id 44F436EB63; Wed, 28 Apr 2021 15:12:57 +0000 (UTC) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=hapSiOwxzz+LtNiTu8ItVX0d5xPdtMc3sX8fqoTNlSUPNssMoAvi5iDkdVW3qh5ryWZ+/XccC93zSYKky+qEZgHE7K83DTFrm93MNJmHLqaPfoL3JLzbSrzPzT9CIBbJPLQJbS4qATuUGkm12p5ylpHHlbIPuwMtfWBBDEtiBgWlfLwfzRI0mjK3x3v2VXp42GVqXkHQ5e+JJAK7X97wm1x3Hx7CyysE3mY9EwTjGKZceJBzUcQFEUker1qERPbBxEvdOIj5Tkgq/Ckn5JteasPfMRfinJYPb5AtUIUxz1uCjxuyau81SAVGOdW/FbU16xzEfJXmFbfOl1P4JcNzRA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=YE5M3YUtiK6EqFdlfOjelVGMYBPtUfHpk9ujawMofIk=; b=BDR8PqOMI07BEZoNnGKa/SVvv1szQkjgUrjnf+A7ZjycpOuZ8vqNq+6D7QSQCm0RrVbdMTAiEHiiAEzn6vPD1C3CL6tqVi5VIzAJ35P/RKO0GU/iUl+pI1ay99zyDWRF6f85fAibKrrqFQV9plTDDecJSq0wpMNMA0h1c6gg9jQjstgYRsLwuKvN3UE3vhhLMr814KzD2zy5me2LRzF0vyY+GZFLAf/gEtbg+h52e9lNsYcS1bibMzKhXzxXwyAQnr2P06mN7qA+3KQxiJWHENYG667Dob2nkBFj9DJf9X641IgCQ0p0k4X+yW/0A/ogvbOTBnW0mo+h4OLEITy5/g== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=amd.com; dmarc=pass action=none header.from=amd.com; dkim=pass header.d=amd.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=YE5M3YUtiK6EqFdlfOjelVGMYBPtUfHpk9ujawMofIk=; b=Sdt/0rkPxS+BN19QRGP2XprIY2rDCeFORmfmSy/0ZBemnsgrfOEK8mSyNjoSS+nLFEoRK/We3xNMBuPB/QEF/w/YoxK03ztyFAvH98aq0+9f+OIh9a4fJpPu3w+zxQBS/yVKop24gDvmcuzQEgl/i5g9fjYJhNYAohZ0Rom7Pfc= Authentication-Results: lists.freedesktop.org; dkim=none (message not signed) header.d=none; lists.freedesktop.org; dmarc=none action=none header.from=amd.com; Received: from SN6PR12MB4623.namprd12.prod.outlook.com (2603:10b6:805:e9::17) by SA0PR12MB4462.namprd12.prod.outlook.com (2603:10b6:806:95::22) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4065.23; Wed, 28 Apr 2021 15:12:56 +0000 Received: from SN6PR12MB4623.namprd12.prod.outlook.com ([fe80::ad51:8c49:b171:856c]) by SN6PR12MB4623.namprd12.prod.outlook.com ([fe80::ad51:8c49:b171:856c%7]) with mapi id 15.20.4065.026; Wed, 28 Apr 2021 15:12:56 +0000 From: Andrey Grodzovsky To: dri-devel@lists.freedesktop.org, amd-gfx@lists.freedesktop.org, linux-pci@vger.kernel.org, ckoenig.leichtzumerken@gmail.com, daniel.vetter@ffwll.ch, Harry.Wentland@amd.com Subject: [PATCH v5 13/27] drm/amdgpu: When filizing the fence driver. stop scheduler first. Date: Wed, 28 Apr 2021 11:11:53 -0400 Message-Id: <20210428151207.1212258-14-andrey.grodzovsky@amd.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210428151207.1212258-1-andrey.grodzovsky@amd.com> References: <20210428151207.1212258-1-andrey.grodzovsky@amd.com> X-Originating-IP: [2607:fea8:3edf:49b0:7212:f93a:73b0:8f23] X-ClientProxiedBy: YT1PR01CA0142.CANPRD01.PROD.OUTLOOK.COM (2603:10b6:b01:2f::21) To SN6PR12MB4623.namprd12.prod.outlook.com (2603:10b6:805:e9::17) MIME-Version: 1.0 X-MS-Exchange-MessageSentRepresentingType: 1 Received: from agrodzovsky-All-Series.hitronhub.home (2607:fea8:3edf:49b0:7212:f93a:73b0:8f23) by YT1PR01CA0142.CANPRD01.PROD.OUTLOOK.COM (2603:10b6:b01:2f::21) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4065.25 via Frontend Transport; Wed, 28 Apr 2021 15:12:54 +0000 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 30d0a7fa-5760-4642-efa3-08d90a5819d9 X-MS-TrafficTypeDiagnostic: SA0PR12MB4462: X-MS-Exchange-Transport-Forked: True X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:7219; X-MS-Exchange-SenderADCheck: 1 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: 24AAkvZYn+yUSmlHjwzA/j8QtAIkDY5ZOKO8ZRfJ+2hZZA/p2nut7tLU5HTyrq3sUfGfmDdReQylSgCEaPL3i92AxyacYKevOH07MQnCufPAmI+lrzwOa9aIXVTxIy68lQyHhhnAU8sO/Jm20UE0hx8I+pDKkw8aDkjmnLupgfM3T6WavzZeg+IcrKKxYA4cRK2E4kNyy1YjpKuvKcJkIhXj5HDY+maI9qbt0LIF6XYU7KwSF79iyzzSWi0F0pShRuePQnUdqM7Um65bkFTapUtgx92wk2C/7/okgNPz8bgYemVVxfwywLmIYgk5xPzwrXeWZxH3YgGQzfZfWLU90OXPZlnO0VstI5Kznl+BK05ONbfRUpNNwNJEB//q/yEw893hk5oxY2RjxBk//pPB9E3XTFKheQdmMRDHUu15imc1LpSQi7TfjAZOaFEEXZDq4zzWi/KoHxokLqa5yNxS6nQNP4X3UAA6rKdHV4FSJAZaHwzJIEuzoogSkSyRrm5SdYi977+JCbdjcU/Sv4oRj+3DbxOdEhu3JpAN4D+cChP0Jv7ZPre85m+PITXiz3YIxIxBRz5Ta2/HJNDjbaicuK5N1u99XiyOMahVtS1UZbg= X-Forefront-Antispam-Report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:SN6PR12MB4623.namprd12.prod.outlook.com; PTR:; CAT:NONE; SFS:(4636009)(346002)(396003)(136003)(39860400002)(376002)(366004)(6512007)(6636002)(66946007)(316002)(186003)(44832011)(2906002)(16526019)(36756003)(83380400001)(6506007)(4326008)(52116002)(8676002)(66476007)(5660300002)(66556008)(1076003)(2616005)(478600001)(86362001)(38100700002)(8936002)(6486002); DIR:OUT; SFP:1101; X-MS-Exchange-AntiSpam-MessageData: xmb0RIl8Hi1CPG1jVs2sX1LlbS3EVKWfOiJ1CuA4wI5kymeGYMUmxjnhwBZZn+rSI3lZNXzYaiJ6avhKlELRcBMJIqQ0I2OWGtw8EBcavbaDShiM/aCuywUpQieRh4XmbfuqsUlRIioUIuj85OAbiVP4pMNzBc5s4/v45+SskOcYFFZilAgiGiMvOUVcXv2U23ouzkNmoedZ0w4U+3LEe+Oir8lUYLU5BL4N2StjUF7ILujzoph+F16DIJhpLnlvnwew6DyBC+Szg9FqkVSw2Fk2KTl75TtaxdDraD4r7x+VV54KHSi8qdP6z3ExJZ/Uyb0U2qJTiwG4I2EhQ16RUqXON1fnJJUSa8CS9dH3h1qR77OCH0fg8IbY6QkUzEoMKCODvlW6EcgIic9toMegVK5kT7+7sUFfMseLn5KotzoeWZNd+sgRp58SEaZC8PeTQyEANhxzWS6G4r+fbfEIyz+a/Agq+E2J0046s/VtYOArUe0PZjOl40EskPwTyjdKNDgo/q6VRlshhRshObMQn/Ib/C76sJGfgpe/8b1JRPxcFqXTMtrALK04JzZWLy3zffnhHSOh2XRlmp0i/waLnbGLXYuCXU3JfDbCnX18jgfq8yMq22J/fcRSEB/xd+k9YQwCtkNVL3hRnGca7xBknJR99K/vyVbe8XL4I1yp7QgjZFRLVJrnF8mmwZsQpMG9DLuQztjZ6cwbjUZ7w1P6zO+NfCEyzSIKm2psZeEnXJqjbLscpCMmarpVf0CU1+AmmO1mWMs8B3/aC4NX3FAURuD3ou6yW5QWXY03Ec3YZRbbFcThMP93wdvWZi7ukOrUlc+/dMWdPAkHhT0xPGW1V9XuMfDM/KJg0mXJH27gMPHPbEIO9wSeUki4i9k4oFC3tTyDVCb/UcAgp4akxFVKp145dwKd48rVA9rsBMtAZ6EYSbthmvnrMDpmnby3sXyDBBoAEKR4iLkk/EJ68xXwOCpplrmHiL/0til5UwBo9eghlP0q1o8Qpk5guFGScAbUjnChEJiHIFWZAjWof0ohylLN8sFBzkj9OOxgXYL6Sp8TJD10CSkR2TkRpnKL9pqZxGfJiNxMsdIxqB8lVXboIJNT8rKvR+vpHWUfBETPdoW8NyifC/a967F2nSTerFOPOpfm/1+fFYB3lgAIwe+N6MSc0kZChIDt4RXAiykhbOUSaImm1WkOr+W13xup+mvsJUMk1mdEHVE/HFkgjKMesUj5jeiEI6Yi14Y/TByUVuXJibJL+sCJ4uBL0UeeH2nB0gDCXKu/cVO0abOVGGqivChUktVfnbSpzRaA4SGs9uchyzcGOMEIAl6nf5AxFUy8ub8wu7ylFABfYIY9a8KA3ZT45r1F5BxA2Wsl4RBSguVGRlYSj7lUnYdqx9ti71Xz X-OriginatorOrg: amd.com X-MS-Exchange-CrossTenant-Network-Message-Id: 30d0a7fa-5760-4642-efa3-08d90a5819d9 X-MS-Exchange-CrossTenant-AuthSource: SN6PR12MB4623.namprd12.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 28 Apr 2021 15:12:55.8937 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: PRQhDbn70CGOlUOUVgxhvkNuN/LqQeBzRAe6uBPyol306BaBH95PxlVtX1v2KHtAgOJE1Oi2Ju4B66sAZtfLxg== X-MS-Exchange-Transport-CrossTenantHeadersStamped: SA0PR12MB4462 X-BeenThere: dri-devel@lists.freedesktop.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Direct Rendering Infrastructure - Development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: gregkh@linuxfoundation.org, Felix.Kuehling@amd.com, helgaas@kernel.org, Alexander.Deucher@amd.com Errors-To: dri-devel-bounces@lists.freedesktop.org Sender: "dri-devel" No point calling amdgpu_fence_wait_empty before stopping the SW scheduler otherwise there is always a chance another job sneaked in after the wait. Signed-off-by: Andrey Grodzovsky --- drivers/gpu/drm/amd/amdgpu/amdgpu_fence.c | 9 +++++++-- 1 file changed, 7 insertions(+), 2 deletions(-) diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_fence.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_fence.c index 34d51e962799..fd9282637549 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_fence.c +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_fence.c @@ -533,6 +533,12 @@ void amdgpu_fence_driver_fini_hw(struct amdgpu_device *adev) if (!ring || !ring->fence_drv.initialized) continue; + + /* Stop any new job submissions from sched before flushing the ring */ + /* TODO Handle amdgpu_job_submit_direct and amdgpu_amdkfd_submit_ib */ + if (!ring->no_scheduler) + drm_sched_fini(&ring->sched); + r = amdgpu_fence_wait_empty(ring); if (r) { /* no need to trigger GPU reset as we are unloading */ @@ -541,8 +547,7 @@ void amdgpu_fence_driver_fini_hw(struct amdgpu_device *adev) if (ring->fence_drv.irq_src) amdgpu_irq_put(adev, ring->fence_drv.irq_src, ring->fence_drv.irq_type); - if (!ring->no_scheduler) - drm_sched_fini(&ring->sched); + del_timer_sync(&ring->fence_drv.fallback_timer); } } From patchwork Wed Apr 28 15:11:54 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrey Grodzovsky X-Patchwork-Id: 12229243 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.5 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,MSGID_FROM_MTA_HEADER,SPF_HELO_NONE,SPF_PASS, USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 6354DC43460 for ; Wed, 28 Apr 2021 15:13:17 +0000 (UTC) Received: from gabe.freedesktop.org (gabe.freedesktop.org [131.252.210.177]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 086ED613F1 for ; Wed, 28 Apr 2021 15:13:16 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 086ED613F1 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=amd.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=dri-devel-bounces@lists.freedesktop.org Received: from gabe.freedesktop.org (localhost [127.0.0.1]) by gabe.freedesktop.org (Postfix) with ESMTP id BDDC26EB6F; Wed, 28 Apr 2021 15:13:15 +0000 (UTC) Received: from NAM12-MW2-obe.outbound.protection.outlook.com (mail-mw2nam12on2080.outbound.protection.outlook.com [40.107.244.80]) by gabe.freedesktop.org (Postfix) with ESMTPS id 306466EB5C; Wed, 28 Apr 2021 15:12:59 +0000 (UTC) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=jESc0j2bvIBWccAOa8/9GT7yJwBcjBe+ikhvSX6TOcLEWyGVBtHGeL9rwp+/6PnUDxIZmDZmg9TaQoZdNoRL3H60jDoDYfHaqHu1E9/baAcyymTy8Ui3dkVBh3EEMjebUF1DJe2KFNTIBmtcb3rQvPfAhZD6KvrFlBw2DAbZygCoRMWXSPMJ6CDnQpidT9HepDOxF2A45vSpOnK2JNPs1tOgC0LSFyNhryFGehulfo3+zt0mgnFLP8tHRay+8nbblLOi39icoh8lZb5to1KqeF/Z0LkbGN055HnQYtfxcJHgVAbUgwkjHaMUDhwqklkIKSBxUm6lDPx1CqKdlo/1RA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=aY/Bg/klW6ajdNMlph32TuucSwu+zbFtbneLFpZ8Rpc=; b=XPuWTD9sgk0iy/TCm6klCznam4IdMo+IxX3CvUE1WVQYu7EkWZoNJ2VFbSyPuMY/gRBThd3kGa+oWiqZzhMYZsYk+PlP+ir2c7+HRMV/WMGPMa1+242A5L6qzgy56veJZS7ZHGrhBIXVpc6QZ2mXLCh4MJkYbUA53Zz+8NcexWgf22/DSU1bwLNxM0P1ukkaZhqnvfgt6od43ThZqtQeDqROzwi3exxoDQxb7HrV9jNyXQZB0JvD3tm0wLyrtxtXL3WLJ1lye84AxMgglohfhSWaiEK33IdniAkRB9CQ7Zz91WL4S9xad+FWzZHnG/WsUri7K6cpyFt9iWu151ci+g== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=amd.com; dmarc=pass action=none header.from=amd.com; dkim=pass header.d=amd.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=aY/Bg/klW6ajdNMlph32TuucSwu+zbFtbneLFpZ8Rpc=; b=eHbam1YOy2r/7KgIG72od9n2cTzYPbq5AmWvNWrMIoCdMtarh91uNBIIp4nJZzvbJkbf+oxjIcZk55pwcRjNxJia5Nz7ZyXPnfwOg6989jf/tjWmpFF5w5cG8fQ/mS/Hhmbq6IsoATTxvUSce0NCub9kK4uwgnWEoA++It88gY4= Authentication-Results: lists.freedesktop.org; dkim=none (message not signed) header.d=none; lists.freedesktop.org; dmarc=none action=none header.from=amd.com; Received: from SN6PR12MB4623.namprd12.prod.outlook.com (2603:10b6:805:e9::17) by SA0PR12MB4462.namprd12.prod.outlook.com (2603:10b6:806:95::22) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4065.23; Wed, 28 Apr 2021 15:12:58 +0000 Received: from SN6PR12MB4623.namprd12.prod.outlook.com ([fe80::ad51:8c49:b171:856c]) by SN6PR12MB4623.namprd12.prod.outlook.com ([fe80::ad51:8c49:b171:856c%7]) with mapi id 15.20.4065.026; Wed, 28 Apr 2021 15:12:58 +0000 From: Andrey Grodzovsky To: dri-devel@lists.freedesktop.org, amd-gfx@lists.freedesktop.org, linux-pci@vger.kernel.org, ckoenig.leichtzumerken@gmail.com, daniel.vetter@ffwll.ch, Harry.Wentland@amd.com Subject: [PATCH v5 14/27] drm/amdgpu: Fix hang on device removal. Date: Wed, 28 Apr 2021 11:11:54 -0400 Message-Id: <20210428151207.1212258-15-andrey.grodzovsky@amd.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210428151207.1212258-1-andrey.grodzovsky@amd.com> References: <20210428151207.1212258-1-andrey.grodzovsky@amd.com> X-Originating-IP: [2607:fea8:3edf:49b0:7212:f93a:73b0:8f23] X-ClientProxiedBy: YT1PR01CA0142.CANPRD01.PROD.OUTLOOK.COM (2603:10b6:b01:2f::21) To SN6PR12MB4623.namprd12.prod.outlook.com (2603:10b6:805:e9::17) MIME-Version: 1.0 X-MS-Exchange-MessageSentRepresentingType: 1 Received: from agrodzovsky-All-Series.hitronhub.home (2607:fea8:3edf:49b0:7212:f93a:73b0:8f23) by YT1PR01CA0142.CANPRD01.PROD.OUTLOOK.COM (2603:10b6:b01:2f::21) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4065.25 via Frontend Transport; Wed, 28 Apr 2021 15:12:56 +0000 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 57ebeb21-f18c-42e2-b645-08d90a581b24 X-MS-TrafficTypeDiagnostic: SA0PR12MB4462: X-MS-Exchange-Transport-Forked: True X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:3383; X-MS-Exchange-SenderADCheck: 1 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: VVxJ+qGOJB6vPA1w+p9+wvYechNnrVfvySsTrYJhZ9ftclQ+jv8KiNYUQlJw5/etYw7NHwCRF1srCyH/rGqSSsnvEnaPF0zP88G85xHHOvw3hRIf9zKhMbe393WgUfI9IO9GqjeJ9tJ8VO6agAc9X8UqSlFVXDrjIZDvR9Mt5NuGypmujV4Jc8fO5mRnHrFeXPUj/kItDtes+LuybBFFrxLZ+omo2cFz4eX7aiJix7Q5Yge3uxXdoPd9d8l+JTayGkUXOJbm0mMFsWRroC2kl+z4k0sup3FnWLSjtu7NB/tFo94YOpL0v5i76LBnz8cWtYTIpD71PRMYtgNPip8mbDtibYg5AKdKzzmGvWGcwYkAgYQRvq/wG6zcJPwJVsNecqfqQNwMzWKrt0AH3Mua0YrRainYdwRDLw6SluwdinY0loNXh5DJUejvqorASrXWzbwGHPFtxZfJi3N8CAX4L4JJivKHjVf2iq+jJ7VLikpfcU+pFiFeEhz4eP62e2qAMJ80V2OPeAQUqvGeZganTLfBmLcdvUakVHh17RKOlaGGUMMCiyBi+yUV4vKSH6tAFJLafW6YrtWjJgtOnyzzbq8xrqDoqINB2qzidt09meo= X-Forefront-Antispam-Report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:SN6PR12MB4623.namprd12.prod.outlook.com; PTR:; CAT:NONE; SFS:(4636009)(346002)(396003)(136003)(39860400002)(376002)(366004)(6512007)(6636002)(66946007)(316002)(186003)(44832011)(2906002)(16526019)(36756003)(83380400001)(6506007)(4326008)(52116002)(8676002)(66476007)(5660300002)(66556008)(1076003)(2616005)(478600001)(86362001)(38100700002)(8936002)(6486002); DIR:OUT; SFP:1101; X-MS-Exchange-AntiSpam-MessageData: A1sF5VFQRZwc5usIiwiraRcnS++XDKQ0LlkJsAj7+/HQJAWzhBE+n+h/i/LUhSiKvrmTx1XE/Z7Jafj0lBrTy0A0s42DiLru+W1y1+yov5jLyOoKfZLeefBzuO4MuD2O8d4JuaRsPHt+01QwevkX6mgMceK8nrLQbzb0gZ74Nrk+uOlK8r4Y3e4QbNmQHlyHcLmvkVUyIaFJkL1VeZ/aHFsXHaOBLUcUlTu+bl4izoMoNMePllKbZY9/2sicblHNxzvA0JGBfhminY7Pox720f9paIOPts3/j+ZVz+efSZpAbP1eg4SsPDQmv29K/NwxK28UuH+wK6DVRm8xn2G5bHaAiXejiQd9JTWTTNFCox9inTnvg/tcA18xjmcsVFK5KWcGSy9Bt2kOfsD6vQmr2ezdsAA8onRsPtAHknOew/zzPHCyw2S9IwHjvWOzEvmRO0rvRYPxHB5SG+UK4zH9W5aW/6vh636q1acat2iFcMIss2eMZAwplIqVAvwsuX59kyKWDqKNOjYbj6kjHlSVpgKiBdMYCq52ckeKo28YyOit/BL27ZEnr637KW2WVQgB+t7lZaNRlpwbHIwXjoqUujfYXJ3efrSzR4W00wkTfHMG5pdtBF+7qRUWbHPWcQpcYfZnbcBzGA+77AV7hwEXsCbaHS7gPYzcHTtfIFChPFtyZlT9TpfNm2ET9CsP2z4Fa3eUGOMj42W74OyLwcrVFU3rqmuQzajRRokzzG+ZXf/sXIdvtxHuOvJ2ONqVWSt+ydbcjuE0JcFAKv6SeHZhcSKpROee2934c7YHh8f0NPa1shfjmNMZnWD0qiCodSpZRGMzhDBC69fl1p1NODCOVzIh1jpYvT7UmDpI2VibH1z/t30VBbeaRjUneedcDX2ty7keZZyrlqoSusg7PuQkL9AxC4Y6F9j/snGk5JEeOnEdonbqC4Cpdnlhv81ES0CE2/Z7xA+lj6rIMiH3gDCE1zRGYSVXLhFdrRCLkE4SzP2liuYlkIp2+JkdhHr2X/x6WEtfY5loz+3iEflBz2Udg/8znotAccvBaC9udHZrT8XQ9So21h03ppwj3gJP78e7iOxgupuSmXILxMUIwsG8pvRX3cB3OGWVJosYbKDRNLwKp+mT2blsymTociSK+wn/PTyuXq8rZyFqVW+l8uzzyKy4AY5idsPzdZHMOXYnINruPOh4IXI5PdOA4n4EwN6GK8KCsH9DIeAbs1ryWY6X+sOL+t9z7WoY6no1jtIJ8Xwn92qaMDKwqysj9qsBS2GafkDmg0JrcDnR0tmzUVtRfAvgNgd1aA31uLp47YsDmaDIhvSNi5m7QzflBoca+OvctGgapAWQnA25VifKqtWqJsbia8JbXafDr2t/yKw7VbyS5tZhbcuonWHwe++uYSRF X-OriginatorOrg: amd.com X-MS-Exchange-CrossTenant-Network-Message-Id: 57ebeb21-f18c-42e2-b645-08d90a581b24 X-MS-Exchange-CrossTenant-AuthSource: SN6PR12MB4623.namprd12.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 28 Apr 2021 15:12:57.9565 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: jJglMBp7i02MvmXmvjIsorQPWpMSQEni2Gk68rQuUP2dGNRvlkT0BUEleiZXYKtzuiAqF8Id4/SR+HmPBjCO+g== X-MS-Exchange-Transport-CrossTenantHeadersStamped: SA0PR12MB4462 X-BeenThere: dri-devel@lists.freedesktop.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Direct Rendering Infrastructure - Development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: gregkh@linuxfoundation.org, Felix.Kuehling@amd.com, helgaas@kernel.org, Alexander.Deucher@amd.com Errors-To: dri-devel-bounces@lists.freedesktop.org Sender: "dri-devel" If removing while commands in flight you cannot wait to flush the HW fences on a ring since the device is gone. Signed-off-by: Andrey Grodzovsky --- drivers/gpu/drm/amd/amdgpu/amdgpu_fence.c | 16 ++++++++++------ 1 file changed, 10 insertions(+), 6 deletions(-) diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_fence.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_fence.c index fd9282637549..2670201e78d3 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_fence.c +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_fence.c @@ -37,6 +37,7 @@ #include #include +#include #include "amdgpu.h" #include "amdgpu_trace.h" @@ -525,8 +526,7 @@ int amdgpu_fence_driver_init(struct amdgpu_device *adev) */ void amdgpu_fence_driver_fini_hw(struct amdgpu_device *adev) { - unsigned i, j; - int r; + int i, r; for (i = 0; i < AMDGPU_MAX_RINGS; i++) { struct amdgpu_ring *ring = adev->rings[i]; @@ -539,11 +539,15 @@ void amdgpu_fence_driver_fini_hw(struct amdgpu_device *adev) if (!ring->no_scheduler) drm_sched_fini(&ring->sched); - r = amdgpu_fence_wait_empty(ring); - if (r) { - /* no need to trigger GPU reset as we are unloading */ + /* You can't wait for HW to signal if it's gone */ + if (!drm_dev_is_unplugged(&adev->ddev)) + r = amdgpu_fence_wait_empty(ring); + else + r = -ENODEV; + /* no need to trigger GPU reset as we are unloading */ + if (r) amdgpu_fence_driver_force_completion(ring); - } + if (ring->fence_drv.irq_src) amdgpu_irq_put(adev, ring->fence_drv.irq_src, ring->fence_drv.irq_type); From patchwork Wed Apr 28 15:11:55 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Andrey Grodzovsky X-Patchwork-Id: 12229251 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.5 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,MSGID_FROM_MTA_HEADER,SPF_HELO_NONE,SPF_PASS, USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 36E97C433ED for ; Wed, 28 Apr 2021 15:13:36 +0000 (UTC) Received: from gabe.freedesktop.org (gabe.freedesktop.org [131.252.210.177]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id C494E613F1 for ; Wed, 28 Apr 2021 15:13:35 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org C494E613F1 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=amd.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=dri-devel-bounces@lists.freedesktop.org Received: from gabe.freedesktop.org (localhost [127.0.0.1]) by gabe.freedesktop.org (Postfix) with ESMTP id 835246EB7E; Wed, 28 Apr 2021 15:13:32 +0000 (UTC) Received: from NAM11-CO1-obe.outbound.protection.outlook.com (mail-co1nam11on2060.outbound.protection.outlook.com [40.107.220.60]) by gabe.freedesktop.org (Postfix) with ESMTPS id 5F6F36EB68; Wed, 28 Apr 2021 15:13:01 +0000 (UTC) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=ZlUabXx78O+VdqPt94BtQGyyT9HYPpo0/aDZgJr6/x2io51Lj9EkLQDPplWpfJuEzGHJ0G/U4vS+QBy0NW47f78DggqhwXrfOf614x8pktp+MVWHS7t8cj5Dt1Mo5a7IiyPo+GI3BjCksNwkA0lX2PwT5PIu7WWv1nGBsp/mF3GTQIqI/PQ+3l0So/s+dYL4Zta3p1tr8dOnEAlWZy2WHjtVYMBXCPEIYm/3boDP3shdIMHLfN69oI0PQR7CnwClCWOYxb6n3jg8qgMwpEcrfLM3ZSMWpIuFUDh8H4pG+1I9pmcUMc9bINPXlaEbtIiaIZYpAWa9gZo7Z++4KgVbUA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=nl/O/shlSz/P5DjdJh1C7b4UObg5BRm5//EqkEZWN84=; b=mm2To12TQrYrtHDTzQMgQIJ4asJvvjqFBkbyFYNQR8c7ST68MA6wd7eyvTjcxq73gHdVX/ovgSo3ha1165KPJw8osGFbdA4KHPQJPMe/0R+u1xlD9n0Z/DxPiDawFCJsChHJ1nZYIaPwAXMUDztDGS26QC1hZjx7D8nzX0ky1GawA279NlAKLvb6IQw4lcdFWkzGb3537vIecHWB8K5qHSyKKKlx6Di/ey4Rh8duKZ/7Gw8pzU+42rnsPD6PB4YDcUwScm5EwZR4HYnWy7AZdtIPZvesgRCXrh4g79fCa+U1Ayo9A1EoWcustF9M39/rv1Z84IpURkIhTAY4Sr+A9w== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=amd.com; dmarc=pass action=none header.from=amd.com; dkim=pass header.d=amd.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=nl/O/shlSz/P5DjdJh1C7b4UObg5BRm5//EqkEZWN84=; b=PfHgEYUHdlvBy6D0pqOhh5WDv2yEwM32zAQ3AcUMq/mTx7bSV+wdGfRq7c1mOjYZ6XX/lDrwAeMoihbNkkhkC5hxLD5Pe67lVyg1e+8scWb4CYYZRwi/Ai+HTRHFlqUlTbC0ILN6gijhzw637uFvmIaNjbWVY14leIR17h5cwV0= Authentication-Results: lists.freedesktop.org; dkim=none (message not signed) header.d=none; lists.freedesktop.org; dmarc=none action=none header.from=amd.com; Received: from SN6PR12MB4623.namprd12.prod.outlook.com (2603:10b6:805:e9::17) by SN6PR12MB2749.namprd12.prod.outlook.com (2603:10b6:805:6d::18) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4065.22; Wed, 28 Apr 2021 15:13:00 +0000 Received: from SN6PR12MB4623.namprd12.prod.outlook.com ([fe80::ad51:8c49:b171:856c]) by SN6PR12MB4623.namprd12.prod.outlook.com ([fe80::ad51:8c49:b171:856c%7]) with mapi id 15.20.4065.026; Wed, 28 Apr 2021 15:12:59 +0000 From: Andrey Grodzovsky To: dri-devel@lists.freedesktop.org, amd-gfx@lists.freedesktop.org, linux-pci@vger.kernel.org, ckoenig.leichtzumerken@gmail.com, daniel.vetter@ffwll.ch, Harry.Wentland@amd.com Subject: [PATCH v5 15/27] drm/scheduler: Fix hang when sched_entity released Date: Wed, 28 Apr 2021 11:11:55 -0400 Message-Id: <20210428151207.1212258-16-andrey.grodzovsky@amd.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210428151207.1212258-1-andrey.grodzovsky@amd.com> References: <20210428151207.1212258-1-andrey.grodzovsky@amd.com> X-Originating-IP: [2607:fea8:3edf:49b0:7212:f93a:73b0:8f23] X-ClientProxiedBy: YT1PR01CA0142.CANPRD01.PROD.OUTLOOK.COM (2603:10b6:b01:2f::21) To SN6PR12MB4623.namprd12.prod.outlook.com (2603:10b6:805:e9::17) MIME-Version: 1.0 X-MS-Exchange-MessageSentRepresentingType: 1 Received: from agrodzovsky-All-Series.hitronhub.home (2607:fea8:3edf:49b0:7212:f93a:73b0:8f23) by YT1PR01CA0142.CANPRD01.PROD.OUTLOOK.COM (2603:10b6:b01:2f::21) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4065.25 via Frontend Transport; Wed, 28 Apr 2021 15:12:58 +0000 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: ce484d86-8da4-4163-f65e-08d90a581c30 X-MS-TrafficTypeDiagnostic: SN6PR12MB2749: X-MS-Exchange-Transport-Forked: True X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:663; X-MS-Exchange-SenderADCheck: 1 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: Lulc5oekbOtXj1MaqcGN1BTIc6Cp4R8OS7NZVrhvetkmTF2eGi3vjmJg68sjSit8fOIH3NvALxllkZNEjI2sLLwiqlNoGLfGpzKweMHlFQp5JTVn6udkfb2YkSejPNbl3KQnTMKOgHf1MSJdXLncV+lke3L9o4g7ATTmg7b5RsLU92e5uzTi6Qu41qWWF8Q1/t12+OMeySx8+gCKFK+prSnSeE82w643Dt74J8m73mTt7AD+bFntua6xXB1ljDTR0SytT07eLBzOJ5dhg39K7dKRLOG983KMX5E0iqcbfxzDxY7QqIQK/mlO3lBAvWNHTAldG7FxyOK6KbF4DgTrWKqS/MmoSyY8knNnujNPzdQ8bRfo/8mMAB1XvZqi+tUzuvqXxJglKqQv207MrzcDhxazB32AbJZnhAiPJz1YnOF0DpCX9GVFREjkiW1Nc1lbMh4IUa39NOyxeqk+Qz+LclHDn6SRFwow4PQ93Bcg8bYN35FOoxH6Minb/hi5RYMCbFIPSzBSo76ib+Wh/jrGwXotA7BeVoEn99jNB5MsiJ+Ft5/ijvJJTZoYnic2MpOPVyCz5QydCbjYsCAql4LHWy8F+No8z2buxd4W6k46sA8= X-Forefront-Antispam-Report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:SN6PR12MB4623.namprd12.prod.outlook.com; PTR:; CAT:NONE; SFS:(4636009)(376002)(346002)(39860400002)(136003)(396003)(366004)(16526019)(6666004)(66574015)(6512007)(2616005)(1076003)(2906002)(6636002)(66556008)(186003)(86362001)(5660300002)(8936002)(83380400001)(6486002)(478600001)(44832011)(38100700002)(54906003)(66946007)(52116002)(6506007)(66476007)(8676002)(4326008)(36756003)(316002); DIR:OUT; SFP:1101; X-MS-Exchange-AntiSpam-MessageData: =?utf-8?q?hHXQAZNqngzXO02+lhSBdeMfGMC3ct?= =?utf-8?q?VK7y/rVxwogi8PvqCi8pFbfVcfkUdaiGZlvZ2uYyqviyhtODcqtSc5AyEpA4G3cMk?= =?utf-8?q?LQuPsixkvvSAQcka5Pjp8aXLCvvyhHSMk6GhJY2EaeMKSzaoO5cZSMc50ETHXxXZV?= =?utf-8?q?gxo3yE2h2jTZz5gLpJwXyZHCHQczH24/amYRcBzByc6acjHp/f23EydWILsjqO/Yq?= =?utf-8?q?N0QncJshmNx+w90lCC1S3Zarq2yPAnBdHHK+b3OO8ez1owLC1C/68jCUZxDnRlaoU?= =?utf-8?q?KMzme0ymxeo6q56Fgba5CUnhO2vD/Chhz8S1rXiX4D9H4iRHCUNGNYsZDIpMvV64N?= =?utf-8?q?ZvUeYVdmTGIVpxBOGLxl020K2rC5dy61IZ5/WSD/rLzT5rvSmFGgqJ9pFU9Rvf+1g?= =?utf-8?q?4491IeDOq7Ifux4Kc/t4FXE/YlYH/IVjD+q8AZyDsw7vVBIKDbi2a8JSAT6WixbEX?= =?utf-8?q?sqxHKD4RrABaBiCom2v7I+0jmz2AFq9Ce0jvu+sMSPDVWVhAij20qTYt94P0/4zAf?= =?utf-8?q?cmePw0M8gx65k3KNmbbGnbMEB9Ct+VfaWJ5+Udu0Rg4h4ao9ZmQCCNqZXBOmHZ1yG?= =?utf-8?q?AT4iV8KCsxjbcKdIZY/udfGZ81JSnqE/A49yYG6aBlykmBD4DHPtzSIFJXRjJz1jA?= =?utf-8?q?8wdV+y6yOwDJA01n5MYdW7SGuaCKERML717uOgTTWkWg3CwptsJ15UTKpccCfTbqO?= =?utf-8?q?VdoPD2inEG8ZTnUYhQwR0JOWdU3NWs25KFfiOyG7YATrNDAAUqWYF3pDdMUckoQ5m?= =?utf-8?q?7Dxc8NyE+mr55Sf/VysBTOBE9vkcWg57zncnuSXh1CxOpzsjYPB8cAD+TSNtL4DWG?= =?utf-8?q?syP7ahWsElZBYAwQwj3kpnr/9Gh5SFzs9K3GT5KHek2qShI4NsIXiFWMEMBxtXMjc?= =?utf-8?q?jRZhywl7iERpBz/0XgUdda/9UFiQfDLipTEk5sVCfl9t2evQOouqx7v7baC6tdBVl?= =?utf-8?q?//oyyJdPAtnrKp2h6K3vhqATExdY0fkwAoeQMiOBaLu0YrM1mjpGbDyZOJxRXYpHf?= =?utf-8?q?RV3icstkw9qLQDaWiNxdMj6ae0CU4Mr0Z/AS1upaf0ipvOfZMpoLQEPJMGnf1+rjL?= =?utf-8?q?1qgugeMjFiNhHQsalK9dxZNxy8Da/GrnrFxkujXf4F6PoDXAGgTjUtJnfJEU1hWFL?= =?utf-8?q?v3Rg0uP6HX+WWJdOEteGqwtF/H7m52Eocvhu30jKMYHVu+MtUzaVlR2ZvfbgVIet6?= =?utf-8?q?jO2t53HIxcsUam2Nw7r+flgxFfXThHUZZz7LGPOEQGy/DWu0iSJ5GhwLlsvCJkiRh?= =?utf-8?q?Z0YP+Ae5/8h8aupoRTKOlUHnuL+qkXkkxnpiijiam5I6ZFumnppXmSey1WaEjDZr4?= =?utf-8?q?gUg4MOFWqRYRr?= X-OriginatorOrg: amd.com X-MS-Exchange-CrossTenant-Network-Message-Id: ce484d86-8da4-4163-f65e-08d90a581c30 X-MS-Exchange-CrossTenant-AuthSource: SN6PR12MB4623.namprd12.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 28 Apr 2021 15:12:59.7125 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: tidZahAWe63VbKeEMpdBSQLgXVe1Gr1otsTGygf0HvRQLXgyrWPJ2pI+vIhfyuA3b+uMlbuqO7bunDgzpq3QKQ== X-MS-Exchange-Transport-CrossTenantHeadersStamped: SN6PR12MB2749 X-BeenThere: dri-devel@lists.freedesktop.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Direct Rendering Infrastructure - Development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: gregkh@linuxfoundation.org, Felix.Kuehling@amd.com, helgaas@kernel.org, Alexander.Deucher@amd.com, =?utf-8?q?Christian_K=C3=B6nig?= Errors-To: dri-devel-bounces@lists.freedesktop.org Sender: "dri-devel" Problem: If scheduler is already stopped by the time sched_entity is released and entity's job_queue not empty I encountred a hang in drm_sched_entity_flush. This is because drm_sched_entity_is_idle never becomes false. Fix: In drm_sched_fini detach all sched_entities from the scheduler's run queues. This will satisfy drm_sched_entity_is_idle. Also wakeup all those processes stuck in sched_entity flushing as the scheduler main thread which wakes them up is stopped by now. v2: Reverse order of drm_sched_rq_remove_entity and marking s_entity as stopped to prevent reinserion back to rq due to race. v3: Drop drm_sched_rq_remove_entity, only modify entity->stopped and check for it in drm_sched_entity_is_idle Signed-off-by: Andrey Grodzovsky Reviewed-by: Christian König --- drivers/gpu/drm/scheduler/sched_entity.c | 3 ++- drivers/gpu/drm/scheduler/sched_main.c | 24 ++++++++++++++++++++++++ 2 files changed, 26 insertions(+), 1 deletion(-) diff --git a/drivers/gpu/drm/scheduler/sched_entity.c b/drivers/gpu/drm/scheduler/sched_entity.c index f0790e9471d1..cb58f692dad9 100644 --- a/drivers/gpu/drm/scheduler/sched_entity.c +++ b/drivers/gpu/drm/scheduler/sched_entity.c @@ -116,7 +116,8 @@ static bool drm_sched_entity_is_idle(struct drm_sched_entity *entity) rmb(); /* for list_empty to work without lock */ if (list_empty(&entity->list) || - spsc_queue_count(&entity->job_queue) == 0) + spsc_queue_count(&entity->job_queue) == 0 || + entity->stopped) return true; return false; diff --git a/drivers/gpu/drm/scheduler/sched_main.c b/drivers/gpu/drm/scheduler/sched_main.c index 908b0b56032d..ba087354d0a8 100644 --- a/drivers/gpu/drm/scheduler/sched_main.c +++ b/drivers/gpu/drm/scheduler/sched_main.c @@ -897,9 +897,33 @@ EXPORT_SYMBOL(drm_sched_init); */ void drm_sched_fini(struct drm_gpu_scheduler *sched) { + struct drm_sched_entity *s_entity; + int i; + if (sched->thread) kthread_stop(sched->thread); + for (i = DRM_SCHED_PRIORITY_COUNT - 1; i >= DRM_SCHED_PRIORITY_MIN; i--) { + struct drm_sched_rq *rq = &sched->sched_rq[i]; + + if (!rq) + continue; + + spin_lock(&rq->lock); + list_for_each_entry(s_entity, &rq->entities, list) + /* + * Prevents reinsertion and marks job_queue as idle, + * it will removed from rq in drm_sched_entity_fini + * eventually + */ + s_entity->stopped = true; + spin_unlock(&rq->lock); + + } + + /* Wakeup everyone stuck in drm_sched_entity_flush for this scheduler */ + wake_up_all(&sched->job_scheduled); + /* Confirm no work left behind accessing device structures */ cancel_delayed_work_sync(&sched->work_tdr); From patchwork Wed Apr 28 15:11:56 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrey Grodzovsky X-Patchwork-Id: 12229261 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.5 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,MSGID_FROM_MTA_HEADER,SPF_HELO_NONE,SPF_PASS, USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 7C9ADC433B4 for ; Wed, 28 Apr 2021 15:13:58 +0000 (UTC) Received: from gabe.freedesktop.org (gabe.freedesktop.org [131.252.210.177]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 231EA61434 for ; Wed, 28 Apr 2021 15:13:58 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 231EA61434 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=amd.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=dri-devel-bounces@lists.freedesktop.org Received: from gabe.freedesktop.org (localhost [127.0.0.1]) by gabe.freedesktop.org (Postfix) with ESMTP id B40226EB89; Wed, 28 Apr 2021 15:13:56 +0000 (UTC) Received: from NAM11-CO1-obe.outbound.protection.outlook.com (mail-co1nam11on2049.outbound.protection.outlook.com [40.107.220.49]) by gabe.freedesktop.org (Postfix) with ESMTPS id 0991C6EB68; Wed, 28 Apr 2021 15:13:04 +0000 (UTC) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=PtXV+4Sf0wuyhy2XhgsI1tsM2UVjOXkczRpM2zbWOtpITJt14Kzuuyyhyfa3lRg7I4FQdC3Dqpvjq4JzXq/GyN8z0bE+MUZkNeJa+DN8mZMhET1jn60cT4QdFBqgHUfPkWbA8u9houRfZspzeW39RGII6cSRM7mnfMNcGanaG+2XOkvU3kzhP9ZMdyj3Wbn9O2+zRTF2EWNJ5EFZJJuXK15zEcAeBhWQCAm6uGQUkItRMhoTIUPFCVpRbHGy3UYcOR2L2Ld7qkTomJ88Xc+tvfwnNepGBWnHS92ywDNHF7/hp0SBROb1WlvamUFamWww87UmbwnYiafU4Ps3MkaTfA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=JjhglDgmtQ+my03Pcogfas8bxrlRmGAUdzR53xIZWls=; b=O2C5TJjGQfjGNQABw2GFxNvnQYSu6IpzMVTGmOnaBzxJHyb6lPvDRSt+w0OEOGvZVAMbzNye3gY9MuL5Lw6hWg7EtQtQq34JPA8ZCZbEq6qG/yTdoQeog6hko+Q/GWiXhpbY6dcq7DvRYX6Rkk8dcMKxNp5lOW4+FPMOZtd+NL94zKJ7HlLKngl3/E57CyJypMKmsqzOF6nnHWpNupF0jkWcCBKhq6RBHZJ8bu3h6f2iiwTExxYJxg5IcaOclVevfEc6npGUYlI5BirQV22i6yQ0Z+wcanUEboA+7aIHtcCAwpxf93nw7btFrs0wsQsJ/W8GsfFldch+EUDQv/gXyg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=amd.com; dmarc=pass action=none header.from=amd.com; dkim=pass header.d=amd.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=JjhglDgmtQ+my03Pcogfas8bxrlRmGAUdzR53xIZWls=; b=2yyoJi52cjDqE8v4qPKQoGqGYcA52ZePCVWt6V1Vp+s8HPsQ73T2Ci5b6cJTrUlzM+BO4yC5AqWYrpWE20qH0jraSbx73rxT4l6d1jY6R2LRL/C7FDe9QiwbDG31NkzL/3OdvDiSjNWcjzYhulOzcstCK8qLh9KeuxTYWmU6B/g= Authentication-Results: lists.freedesktop.org; dkim=none (message not signed) header.d=none; lists.freedesktop.org; dmarc=none action=none header.from=amd.com; Received: from SN6PR12MB4623.namprd12.prod.outlook.com (2603:10b6:805:e9::17) by SN6PR12MB2749.namprd12.prod.outlook.com (2603:10b6:805:6d::18) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4065.22; Wed, 28 Apr 2021 15:13:01 +0000 Received: from SN6PR12MB4623.namprd12.prod.outlook.com ([fe80::ad51:8c49:b171:856c]) by SN6PR12MB4623.namprd12.prod.outlook.com ([fe80::ad51:8c49:b171:856c%7]) with mapi id 15.20.4065.026; Wed, 28 Apr 2021 15:13:01 +0000 From: Andrey Grodzovsky To: dri-devel@lists.freedesktop.org, amd-gfx@lists.freedesktop.org, linux-pci@vger.kernel.org, ckoenig.leichtzumerken@gmail.com, daniel.vetter@ffwll.ch, Harry.Wentland@amd.com Subject: [PATCH v5 16/27] drm/amdgpu: Unmap all MMIO mappings Date: Wed, 28 Apr 2021 11:11:56 -0400 Message-Id: <20210428151207.1212258-17-andrey.grodzovsky@amd.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210428151207.1212258-1-andrey.grodzovsky@amd.com> References: <20210428151207.1212258-1-andrey.grodzovsky@amd.com> X-Originating-IP: [2607:fea8:3edf:49b0:7212:f93a:73b0:8f23] X-ClientProxiedBy: YT1PR01CA0142.CANPRD01.PROD.OUTLOOK.COM (2603:10b6:b01:2f::21) To SN6PR12MB4623.namprd12.prod.outlook.com (2603:10b6:805:e9::17) MIME-Version: 1.0 X-MS-Exchange-MessageSentRepresentingType: 1 Received: from agrodzovsky-All-Series.hitronhub.home (2607:fea8:3edf:49b0:7212:f93a:73b0:8f23) by YT1PR01CA0142.CANPRD01.PROD.OUTLOOK.COM (2603:10b6:b01:2f::21) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4065.25 via Frontend Transport; Wed, 28 Apr 2021 15:13:00 +0000 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: d62ad35b-6987-48a5-f5cb-08d90a581d36 X-MS-TrafficTypeDiagnostic: SN6PR12MB2749: X-MS-Exchange-Transport-Forked: True X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:1303; X-MS-Exchange-SenderADCheck: 1 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: b9ibYGomiqBI52axkuhjNcBXz1DdkJlZiw/ZfXg2vzqeAqzcmwfVpekx9Fu6Dp/6UgxiZLl51aODmmphSzlEXz3K7lh/lthLZo9M3SrHW5uH9oXwXWonOD15GRVwDRCahA/ks2d5HHMZyDEtDQDrkSePmVne2XEv2K8A0KW4aZYcgqRpKMNjnJwPhwsDLVQDIBP7DLm0UWOdgOf/Vm82jqhm5Up5BnGNaDzoUNVEER9bYybJoNWBjlX1Glw59KjvfkUnuCBa4mVRsu6SxwBT9MBvZKoHiojlR8VOzr5SkV5voWYhzeBjA1y8mbGdVZQi3XINoGgP4X2ynlcA7Drblj3dJ/9ibJe7+cq6uWzjSRttWyZZ6kqShl/ZaZc2hoNyv03on5as5eY7VbCrH8GyVINqX36i4HWSMBXTNRjS62xaayRrGV7EU/wCF+Kdx6po9p+mBz+Ipy7xBYwvqCpB/P8EsQxojc360UzShnbJXGyyoJqP/qI7V5jbgz+aglIk4u6+GZcOoveA+/O/slWC9ZM/cRG5yulM9yK/7I6XMX1VtsMNHysjWXom7U36eWIhuwwphLpyXMIZHSM7ebbpV+qafA6st8Hr3sIMvcpc1Mk= X-Forefront-Antispam-Report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:SN6PR12MB4623.namprd12.prod.outlook.com; PTR:; CAT:NONE; SFS:(4636009)(376002)(346002)(39860400002)(136003)(396003)(366004)(16526019)(6666004)(6512007)(2616005)(1076003)(2906002)(6636002)(66556008)(186003)(86362001)(5660300002)(8936002)(83380400001)(6486002)(478600001)(44832011)(38100700002)(66946007)(52116002)(6506007)(66476007)(8676002)(4326008)(36756003)(316002); DIR:OUT; SFP:1101; X-MS-Exchange-AntiSpam-MessageData: pzsxGMqynz4O6kpnz0PofWMFzDvDUTmLgL/4hglX9Tp7NqDFYL/ywrPji+d6a7XtbkVQnie6Nq6fqSGGpCFqvjRddawRmfrQXcqzF/NuK/oHY6iU4wd88oVLKaovfaBvAuAB8BOeR2cJOa1jVtYUs6YrGc75CgSB9do+0mTgniJaUfCEOy6oqTk5vy1CrRGnhAvmqOYb0HE4HPQU0MKP1YdS56e4dVgD9LugdOXCSJMDYtJPUe6Z0dh1uDCyTmKK+DfF8CcslV0o5+K4e3p+IoOHseQyVfQwVnYRpmV0ODFaKYB9LG5GogZ9doC6BvE+phZeihkMANbZkB42+7cF73iXNL2K1t0ZNQ+O/hOtoxVKPPdZPc7pWXBk5qbJ4hXkKFiRxEqt4lMKHSZg9Dj0MC6nsGx29uO5MdP++2pFOIha5321pPAyuMgRVorrmlGAAp81rxIVePouS4Whiv7mlNRzftkLFrzXD5I9sNwhVFMaz6GgxEqvSp3Bx0OaH6MwkwIcd3WAxvtjO/7gkq7A9et3shw8TcwVQta7rO0V896MYRZHnsFvXNL6TmSutBH4F7vM/tbEYa2QnzWOwdz1LeOmqM0UEGQxBJzIbh9z+/SQYI3/yGZjLue3zIDY37e/HI4nab1BzgrStFcuJ00axhrHHFjAihxe4S3Zv1w2pWNlqtgO4atSAvRiO/ZOqtlZt2o5NAHdyfIyK8fF5TAAF0CO3tvVby4LAo/BGHNaFC7ulyXZ9JowvjALR9IY0aI3gih/tWl30agpAKuidA4TtZaK9NSTMFgPzCGxPmkO1IFRtMrrYKQtDPhDb4Vv2ylwMbG9i3d9ycw1mrQvOLAC6QDE4mcaHEIIlAS5jvjIGfTZTcf7dgKDRlehZKXYm+DgvXn1YmQN1mH5xJOP2Ypj3nAhDDMM9rwGuNQO+D3t+v93rpReipjKILwRd6ZGftLBVZw6qGAvq75AoqeWNKLHJYCjeJxTQRtL+L5s7NHUgt0grxMy0U6jjZqB1DRRarzrRtFHWIoLrsZ8aKtm9dEFnYAxZGa/oJ3WvJ3qWU3mNAreXmLb3Iy9s9CN+LhmBcqCc8DVBWaDV8A+cuVZ+orA0/s2eYtvJopyPqoCJYAT3cnK8BKu97K5qZBzopAVDBDTHezVIo5E6aCB8kriNC50Rz8ZUibLDDOEkMoHCE8fqIrwT/VP0ltSyQQAv8JcLtdbtIc7kx49GXfuPZgMrYTdChqqOyKGTzjJY17Iqs+BZJXeQ/1kKyUkneRXpYVmv/1+UQYB51mHsQcpf5n0o0TLgJQF9TakcuEX432FYUlkEb6GlyjruQ7xHjIfCvrkU6bZdw+/uF5kH/q/62Yny1bdiPNwp2QJZ5gIsZYuNGlqpVOIvziA9iiCcbEgEuKxvynn X-OriginatorOrg: amd.com X-MS-Exchange-CrossTenant-Network-Message-Id: d62ad35b-6987-48a5-f5cb-08d90a581d36 X-MS-Exchange-CrossTenant-AuthSource: SN6PR12MB4623.namprd12.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 28 Apr 2021 15:13:01.4256 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: gdYEHD4UvdFRqLzFlslahYehOXgVm9XBdHQeePEc8uvrgJu6wKNVcaxb1BVs9scSodU2XEZfGR/GRq5DwLiSFA== X-MS-Exchange-Transport-CrossTenantHeadersStamped: SN6PR12MB2749 X-BeenThere: dri-devel@lists.freedesktop.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Direct Rendering Infrastructure - Development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: gregkh@linuxfoundation.org, Felix.Kuehling@amd.com, helgaas@kernel.org, Alexander.Deucher@amd.com Errors-To: dri-devel-bounces@lists.freedesktop.org Sender: "dri-devel" Access to those must be prevented post pci_remove Signed-off-by: Andrey Grodzovsky --- drivers/gpu/drm/amd/amdgpu/amdgpu.h | 5 +++ drivers/gpu/drm/amd/amdgpu/amdgpu_device.c | 38 ++++++++++++++++++++-- drivers/gpu/drm/amd/amdgpu/amdgpu_object.c | 28 ++++++++++++++-- drivers/gpu/drm/amd/amdgpu/amdgpu_object.h | 5 +++ 4 files changed, 71 insertions(+), 5 deletions(-) diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu.h b/drivers/gpu/drm/amd/amdgpu/amdgpu.h index 30a24db5f4d1..3e4755fc10c8 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu.h +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu.h @@ -1056,6 +1056,11 @@ struct amdgpu_device { struct pci_saved_state *pci_state; struct list_head device_bo_list; + + /* List of all MMIO BOs */ + struct list_head mmio_list; + struct mutex mmio_list_lock; + }; static inline struct amdgpu_device *drm_to_adev(struct drm_device *ddev) diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c index 22b09c4db255..3ddad6cba62d 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c @@ -3320,6 +3320,9 @@ int amdgpu_device_init(struct amdgpu_device *adev, INIT_LIST_HEAD(&adev->shadow_list); mutex_init(&adev->shadow_list_lock); + INIT_LIST_HEAD(&adev->mmio_list); + mutex_init(&adev->mmio_list_lock); + INIT_DELAYED_WORK(&adev->delayed_init_work, amdgpu_device_delayed_init_work_handler); INIT_DELAYED_WORK(&adev->gfx.gfx_off_delay_work, @@ -3636,6 +3639,36 @@ static void amdgpu_clear_dma_mappings(struct amdgpu_device *adev) spin_unlock(&adev->mman.bdev.lru_lock); } +static void amdgpu_device_unmap_mmio(struct amdgpu_device *adev) +{ + struct amdgpu_bo *bo; + + /* Clear all CPU mappings pointing to this device */ + unmap_mapping_range(adev->ddev.anon_inode->i_mapping, 0, 0, 1); + + /* Unmap all MMIO mapped kernel BOs */ + mutex_lock(&adev->mmio_list_lock); + list_for_each_entry(bo, &adev->mmio_list, mmio_list) { + amdgpu_bo_kunmap(bo); + if (*bo->kmap_ptr) + *bo->kmap_ptr = NULL; + } + mutex_unlock(&adev->mmio_list_lock); + + /* Unmap all mapped bars - Doorbell, registers and VRAM */ + amdgpu_device_doorbell_fini(adev); + + iounmap(adev->rmmio); + adev->rmmio = NULL; + if (adev->mman.aper_base_kaddr) + iounmap(adev->mman.aper_base_kaddr); + adev->mman.aper_base_kaddr = NULL; + + /* Memory manager related */ + arch_phys_wc_del(adev->gmc.vram_mtrr); + arch_io_free_memtype_wc(adev->gmc.aper_base, adev->gmc.aper_size); +} + /** * amdgpu_device_fini - tear down the driver * @@ -3683,6 +3716,8 @@ void amdgpu_device_fini_hw(struct amdgpu_device *adev) amdgpu_clear_dma_mappings(adev); amdgpu_gart_dummy_page_fini(adev); + + amdgpu_device_unmap_mmio(adev); } void amdgpu_device_fini_sw(struct amdgpu_device *adev) @@ -3713,9 +3748,6 @@ void amdgpu_device_fini_sw(struct amdgpu_device *adev) if (adev->rio_mem) pci_iounmap(adev->pdev, adev->rio_mem); adev->rio_mem = NULL; - iounmap(adev->rmmio); - adev->rmmio = NULL; - amdgpu_device_doorbell_fini(adev); if (IS_ENABLED(CONFIG_PERF_EVENTS)) amdgpu_pmu_fini(adev); diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_object.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_object.c index 62d829f5e62c..9b05e3b96fa0 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_object.c +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_object.c @@ -531,6 +531,9 @@ static int amdgpu_bo_do_create(struct amdgpu_device *adev, return -ENOMEM; drm_gem_private_object_init(adev_to_drm(adev), &bo->tbo.base, size); INIT_LIST_HEAD(&bo->shadow_list); + + INIT_LIST_HEAD(&bo->mmio_list); + bo->vm_bo = NULL; bo->preferred_domains = bp->preferred_domain ? bp->preferred_domain : bp->domain; @@ -774,9 +777,21 @@ int amdgpu_bo_kmap(struct amdgpu_bo *bo, void **ptr) if (r) return r; - if (ptr) + if (bo->kmap.bo_kmap_type == ttm_bo_map_iomap) { + struct amdgpu_device *adev = amdgpu_ttm_adev(bo->tbo.bdev); + + mutex_lock(&adev->mmio_list_lock); + list_add_tail(&bo->mmio_list, &adev->mmio_list); + mutex_unlock(&adev->mmio_list_lock); + } + + if (ptr) { *ptr = amdgpu_bo_kptr(bo); + if (bo->kmap.bo_kmap_type == ttm_bo_map_iomap) + bo->kmap_ptr = ptr; + } + return 0; } @@ -804,8 +819,17 @@ void *amdgpu_bo_kptr(struct amdgpu_bo *bo) */ void amdgpu_bo_kunmap(struct amdgpu_bo *bo) { - if (bo->kmap.bo) + struct amdgpu_device *adev = amdgpu_ttm_adev(bo->tbo.bdev); + + if (bo->kmap.bo) { + if (bo->kmap.bo_kmap_type == ttm_bo_map_iomap) { + mutex_lock(&adev->mmio_list_lock); + list_del_init(&bo->mmio_list); + mutex_unlock(&adev->mmio_list_lock); + } + ttm_bo_kunmap(&bo->kmap); + } } /** diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_object.h b/drivers/gpu/drm/amd/amdgpu/amdgpu_object.h index 5ae8555ef275..3129d9bbfa22 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_object.h +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_object.h @@ -112,6 +112,11 @@ struct amdgpu_bo { struct kgd_mem *kfd_bo; struct list_head bo; + + struct list_head mmio_list; + /* Address of kernel VA pointer to MMIO so they can be updated post remap */ + void **kmap_ptr; + }; static inline struct amdgpu_bo *ttm_to_amdgpu_bo(struct ttm_buffer_object *tbo) From patchwork Wed Apr 28 15:11:57 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrey Grodzovsky X-Patchwork-Id: 12229255 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.5 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,MSGID_FROM_MTA_HEADER,SPF_HELO_NONE,SPF_PASS, USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 61A93C43618 for ; Wed, 28 Apr 2021 15:13:43 +0000 (UTC) Received: from gabe.freedesktop.org (gabe.freedesktop.org [131.252.210.177]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 03F256143A for ; Wed, 28 Apr 2021 15:13:42 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 03F256143A Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=amd.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=dri-devel-bounces@lists.freedesktop.org Received: from gabe.freedesktop.org (localhost [127.0.0.1]) by gabe.freedesktop.org (Postfix) with ESMTP id 8038F6EB81; Wed, 28 Apr 2021 15:13:39 +0000 (UTC) Received: from NAM11-CO1-obe.outbound.protection.outlook.com (mail-co1nam11on2049.outbound.protection.outlook.com [40.107.220.49]) by gabe.freedesktop.org (Postfix) with ESMTPS id 7B45D6EB6A; Wed, 28 Apr 2021 15:13:04 +0000 (UTC) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=f5Z0uMQbTRtePddYkO6WMB86uVFbgOhBP2FDITkxdufktCnHeFwfZAXqfrinV1aay99SRvQY9jZJkSzLCUtQaUo1CiwgNzpZWZ5HPSMEtQulrIBcyOi1vY2+UjXaphR8RNIAEd7RtOd2WHjTroh4Harml0UQqNZY7fhyX3Ii2MOFci2Iao3qBcADJzZng+z65c6BNMTU1/o94KVy89SHTBi8yPV/ydYwWm+e4pP+XvF99IuG0v7UeQmm2tiSJbbVY//HHfg/I/zeWMus9EDymfAAOi9qGV1FIqmjirpBMVjqaDRgNxy6zu3kjqrJxGJ39Ggtifz5VdOsWKAeQvG+kw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=tn3AGqPGds9oe9uOfZ6FOO+wQivsFIH6NDom/ibc5IE=; b=mWsfPxoQoWr8qdmmVWk3/Gfhev0WLhMbPT5ITNKylRJPOsAgrWaxspTA7PDJyg6YmzyOp7/Y4Z5jDzdC0beXKtx/d0G+6lZRKUBZ9mZaMUpxFOeEFeLI0WUb90rDI9Aeg0IMYA7gAaRvA5Ayx+g77C/mSVH5Oxl2JtUJy/nl4YH0i2+Eeeeey42RIkOdTO2JXVDKEZFBOeL/HcmuIxBGyhxDZnPCvsGGm1vs00X/XQ8jpZqfztN4InTKT3BaDswf/o0N0FNs1L7mXcyaqZN7XiLDQ2epiAPchBpF1Wc6IFAt6Y2Ym095fk9clowor0nD5fNnZN/K9gZ7hCDM/0PKNQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=amd.com; dmarc=pass action=none header.from=amd.com; dkim=pass header.d=amd.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=tn3AGqPGds9oe9uOfZ6FOO+wQivsFIH6NDom/ibc5IE=; b=0dAgA8TGJuDmsmq6kyA7hcrDzjM0aCAJufcQn0PXhBryu3WaPefFLy93bK4TKsywD30wA/h5nYFAjsnxlRu51beXoHq+WQc3R8xv2n4Dg9Bxpse2fQBSVkCW5+5iQmRLv8YxHHyXcJpWY9+M9TGE4YBtp/80pYn/GevGySF1cLA= Authentication-Results: lists.freedesktop.org; dkim=none (message not signed) header.d=none; lists.freedesktop.org; dmarc=none action=none header.from=amd.com; Received: from SN6PR12MB4623.namprd12.prod.outlook.com (2603:10b6:805:e9::17) by SN6PR12MB2749.namprd12.prod.outlook.com (2603:10b6:805:6d::18) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4065.22; Wed, 28 Apr 2021 15:13:03 +0000 Received: from SN6PR12MB4623.namprd12.prod.outlook.com ([fe80::ad51:8c49:b171:856c]) by SN6PR12MB4623.namprd12.prod.outlook.com ([fe80::ad51:8c49:b171:856c%7]) with mapi id 15.20.4065.026; Wed, 28 Apr 2021 15:13:03 +0000 From: Andrey Grodzovsky To: dri-devel@lists.freedesktop.org, amd-gfx@lists.freedesktop.org, linux-pci@vger.kernel.org, ckoenig.leichtzumerken@gmail.com, daniel.vetter@ffwll.ch, Harry.Wentland@amd.com Subject: [PATCH v5 17/27] drm/amdgpu: Add rw_sem to pushing job into sched queue Date: Wed, 28 Apr 2021 11:11:57 -0400 Message-Id: <20210428151207.1212258-18-andrey.grodzovsky@amd.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210428151207.1212258-1-andrey.grodzovsky@amd.com> References: <20210428151207.1212258-1-andrey.grodzovsky@amd.com> X-Originating-IP: [2607:fea8:3edf:49b0:7212:f93a:73b0:8f23] X-ClientProxiedBy: YT1PR01CA0142.CANPRD01.PROD.OUTLOOK.COM (2603:10b6:b01:2f::21) To SN6PR12MB4623.namprd12.prod.outlook.com (2603:10b6:805:e9::17) MIME-Version: 1.0 X-MS-Exchange-MessageSentRepresentingType: 1 Received: from agrodzovsky-All-Series.hitronhub.home (2607:fea8:3edf:49b0:7212:f93a:73b0:8f23) by YT1PR01CA0142.CANPRD01.PROD.OUTLOOK.COM (2603:10b6:b01:2f::21) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4065.25 via Frontend Transport; Wed, 28 Apr 2021 15:13:02 +0000 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: a63f8626-3a48-4742-3ec6-08d90a581e3f X-MS-TrafficTypeDiagnostic: SN6PR12MB2749: X-MS-Exchange-Transport-Forked: True X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:49; X-MS-Exchange-SenderADCheck: 1 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: 7l5YAYjm5kmhrtW+IVtyoaOR4risks/vJtYoGpjqMh6Zlow0VSg/+PgdTUbBKR29kJowSIkOkEMct7QOLzlPhs1oZK7WllU/FPTYcDshRPZA/n+wLPybyBeowtJmKIcY8pIb+ilW6F35B71w4f9UiQSHBFzAaCOCEmdcFMRrrZnKMrA9oMuUnUZmU7sUAF5EMWTAIpQpFlFuxOFfe6QsDZ2FjReO2IddoEayCtv5kpXPLPeYxtSIRHbyF3zjx+F5pf5Ed3/wtdrsES9jd04SnkVbXOyZr525pTSuHGoVObVmuvvpKTe3szzrjvrPw3SgMguNj1OdQlNl6HWQD2c8gxN1Mky931Gbyj1UeLY7r17j7IiWdt7zq/DvxOHHIRsVSOrdwNvaYR9oS3mH8jCAT9R1+Hro2n0ZqCUvG2Bidjhc57vw1F8Fk9QyOIDic78gal1xuWvQH7QI172Ge6eufBfrtVit+LC+zRwx9HjZV1fc0hWKtjrbOxqjvWHwno6DJklKqNpFB4vHNuV+3glmKDXZCcGORL+RR9dXEkpDFFAph/grshtrwqRx1gT6cpmCcc46PHgxv4UTjLUlgfJj7MzF7adttTye+VDRLfaFsuI= X-Forefront-Antispam-Report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:SN6PR12MB4623.namprd12.prod.outlook.com; PTR:; CAT:NONE; SFS:(4636009)(376002)(346002)(39860400002)(136003)(396003)(366004)(16526019)(6666004)(6512007)(2616005)(1076003)(2906002)(6636002)(66556008)(186003)(86362001)(5660300002)(8936002)(83380400001)(6486002)(478600001)(44832011)(38100700002)(66946007)(52116002)(6506007)(66476007)(8676002)(4326008)(36756003)(316002); DIR:OUT; SFP:1101; X-MS-Exchange-AntiSpam-MessageData: 8CYk9gjDU+kSOAaaQFY8AZVzbe+YXuysFSO9khckNLe6Q9qBQvn97qt7Dgb73WRocjccZu59KWIhq82cbRHlMntJQ4xtTMhmqI7+qs9JNRUJhhA6K8Yp7K1UYODUFayWoVw6Z24uqc4x7GQBgHFuue1r3vxq46V4GTiahRU/zUtmVKxX0ecR18rPCiwRtPejbyIRtJ7AMdCoLROSGh+SoHMZ1a0DS/IFlwlNDqQa4RYRaUzUKfXs65suy8KkhsQBUoWONMZM5IFBjCtEkdQ2IpiENolzH2z54RkjjB5YFt539Y3FMTbFQUA7RpQLTj43l1kHgNsAomtQnDTjq/HAj08SKGPRmi2CA9TLF28CB3KKPGeoOuFDwhma/8H6AsNBnX418Mq2NYER0zY937Y4pMec68GGYpcJGOji/Pz/HjEhJTUeT4gosSfUnsdQ9i2iY49uKUr84XS4xjT2uiK/UFLVgRx/ASuwXmKTY6UHj6LNDAVtlZe55cJRmr8qxxSTygdcCmmv5vbE8X/StEE1p3wbX0GLgVQ2MGtX2rvXGQz/e76R041IzAt2LJIiyCZhTJ2xZyzqEEMNy3iSvuonYrB0cqfAhWEU6CoxtG/8uGG6fSGiFxUBCF4Jm5SC9ota9qvKrzuqdXOuUbUQyOQyUUbI4TxoAGzEbHftPtoEqOftRuXUePMieh22etRNL1GYNKXjuawCNNK+w46XOcy8E+eJqIoQ4CplB3b+GkIilye4kmhjo09GNEk28rC43qJZ6oupcq/s8ElImTh8JeahD6MkmaVKXOPt92PNxfPaazi7e6ac6DF05enUCbWzlXGueyuzqE79jtflZUSN1cVApO/r94BMA70LO2qWyq9rZRXLt88KvyCYrBSGAZIfY4QnMmP+lIXV9iyXKSmIWUL+GZwfHtEjAzg6sdszQB1S90qevgERXfeBgjCRi5i2cq1SuMPGcRBKGcavOVY5/Xd02ss451mQp0Byt0aQszlL0xd+vMzP/LuRfjbEroEsIe/8NDaFhl/XrhVf1EQuZB/KHJq1mtnYlHULffpR72LEGp446qctBJSWMKTGg70z/cWkwq0ZAnsrKhV8OenMNGRElFn832sPCBsewA+yiHwF+zDvrKdu32V7qcuGfL4+kaLc69Ong/3rXU441HMqlk7FG86Twkl0vnWTR0RKppIINeEpybbj0vKdKHQ/gYNNByGiOgNbrkKoZ52UpV+UZH31qYEVrVQQERmbaxO2W+rQ0rs1aeCHnxJ4V6QRDQxE4SQUvQPhwsgBKMZ7mrZgbNPVlpmSvPcmoDZgf9KwS+nkgWT4TIPSGMQa34M7tquuV7fKCukJCFkqj9U0HpS2QEKBzPb/ZCev9obl+liqyfieytqjK3F2pJfUWzjcjskyca+f X-OriginatorOrg: amd.com X-MS-Exchange-CrossTenant-Network-Message-Id: a63f8626-3a48-4742-3ec6-08d90a581e3f X-MS-Exchange-CrossTenant-AuthSource: SN6PR12MB4623.namprd12.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 28 Apr 2021 15:13:03.1496 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: LKKA31mltqhi5ipxv2/6dFBy9acbHfbaBH3v6ck2xqxsoVJvwctthCey66N/Ec63GXkUIrXnS7vQk54TbXW2Tw== X-MS-Exchange-Transport-CrossTenantHeadersStamped: SN6PR12MB2749 X-BeenThere: dri-devel@lists.freedesktop.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Direct Rendering Infrastructure - Development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: gregkh@linuxfoundation.org, Felix.Kuehling@amd.com, helgaas@kernel.org, Alexander.Deucher@amd.com Errors-To: dri-devel-bounces@lists.freedesktop.org Sender: "dri-devel" Will be later used block further submissions once device is removed. Also complete schedule fence if scheduling failed due to submission blocking. Signed-off-by: Andrey Grodzovsky --- drivers/gpu/drm/amd/amdgpu/amdgpu.h | 3 +++ drivers/gpu/drm/amd/amdgpu/amdgpu_cs.c | 13 ++++++++++++- drivers/gpu/drm/amd/amdgpu/amdgpu_device.c | 1 + drivers/gpu/drm/amd/amdgpu/amdgpu_job.c | 14 +++++++++++++- 4 files changed, 29 insertions(+), 2 deletions(-) diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu.h b/drivers/gpu/drm/amd/amdgpu/amdgpu.h index 3e4755fc10c8..0db0ba4fba89 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu.h +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu.h @@ -1057,6 +1057,9 @@ struct amdgpu_device { struct list_head device_bo_list; + bool stop_job_submissions; + struct rw_semaphore sched_fence_completion_sem; + /* List of all MMIO BOs */ struct list_head mmio_list; struct mutex mmio_list_lock; diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_cs.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_cs.c index 3e240b952e79..ac092a5eb4e7 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_cs.c +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_cs.c @@ -1256,7 +1256,18 @@ static int amdgpu_cs_submit(struct amdgpu_cs_parser *p, trace_amdgpu_cs_ioctl(job); amdgpu_vm_bo_trace_cs(&fpriv->vm, &p->ticket); - drm_sched_entity_push_job(&job->base, entity); + + down_read(&p->adev->sched_fence_completion_sem); + if (!p->adev->stop_job_submissions) { + drm_sched_entity_push_job(&job->base, entity); + } else { + dma_fence_set_error(&job->base.s_fence->scheduled, -ENODEV); + dma_fence_set_error(&job->base.s_fence->finished, -ENODEV); + dma_fence_signal(&job->base.s_fence->scheduled); + dma_fence_signal(&job->base.s_fence->finished); + } + + up_read(&p->adev->sched_fence_completion_sem); amdgpu_vm_move_to_lru_tail(p->adev, &fpriv->vm); diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c index 3ddad6cba62d..33e8e9e1d1fe 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c @@ -3302,6 +3302,7 @@ int amdgpu_device_init(struct amdgpu_device *adev, init_rwsem(&adev->reset_sem); mutex_init(&adev->psp.mutex); mutex_init(&adev->notifier_lock); + init_rwsem(&adev->sched_fence_completion_sem); r = amdgpu_device_check_arguments(adev); if (r) diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_job.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_job.c index d33e6d97cc89..26d8b79ea165 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_job.c +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_job.c @@ -162,6 +162,7 @@ int amdgpu_job_submit(struct amdgpu_job *job, struct drm_sched_entity *entity, void *owner, struct dma_fence **f) { int r; + struct amdgpu_ring *ring = to_amdgpu_ring(job->base.sched); if (!f) return -EINVAL; @@ -172,7 +173,18 @@ int amdgpu_job_submit(struct amdgpu_job *job, struct drm_sched_entity *entity, *f = dma_fence_get(&job->base.s_fence->finished); amdgpu_job_free_resources(job); - drm_sched_entity_push_job(&job->base, entity); + + down_read(&ring->adev->sched_fence_completion_sem); + if (!ring->adev->stop_job_submissions) { + drm_sched_entity_push_job(&job->base, entity); + } else { + dma_fence_set_error(&job->base.s_fence->scheduled, -ENODEV); + dma_fence_set_error(&job->base.s_fence->finished, -ENODEV); + dma_fence_signal(&job->base.s_fence->scheduled); + dma_fence_signal(&job->base.s_fence->finished); + + } + up_read(&ring->adev->sched_fence_completion_sem); return 0; } From patchwork Wed Apr 28 15:11:58 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrey Grodzovsky X-Patchwork-Id: 12229253 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.5 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,MSGID_FROM_MTA_HEADER,SPF_HELO_NONE,SPF_PASS, USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id A5CEBC43461 for ; Wed, 28 Apr 2021 15:13:36 +0000 (UTC) Received: from gabe.freedesktop.org (gabe.freedesktop.org [131.252.210.177]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 4E34E613F1 for ; Wed, 28 Apr 2021 15:13:36 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 4E34E613F1 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=amd.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=dri-devel-bounces@lists.freedesktop.org Received: from gabe.freedesktop.org (localhost [127.0.0.1]) by gabe.freedesktop.org (Postfix) with ESMTP id EBEE26EB7C; Wed, 28 Apr 2021 15:13:33 +0000 (UTC) Received: from NAM11-CO1-obe.outbound.protection.outlook.com (mail-co1nam11on2042.outbound.protection.outlook.com [40.107.220.42]) by gabe.freedesktop.org (Postfix) with ESMTPS id 5BBB96EB68; Wed, 28 Apr 2021 15:13:06 +0000 (UTC) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=XE+cHfmXKX7vww9G+py3rHn0Ib/DeeA1yQwkREsSWMqbgLxsS+C0tphZzBZ6lmfaZ/X5Tn4mud7uWWPAN16Lyu5dwAAnyie7w9qJTjYmKSNZbotP28nQDwm+WcrQYWXveLxEepZJDNm/0ihOsy0z/ptva0cFKqYrsneRBLl8Cpsl4B7byQt0AqsxQAOZfNxABKnHRBxPz6WNHBX2h8Ca38KgcIRkqZgCQZEF+IsfMcafSfc7p9BIfCuIvTZy6wnDIV+1M3PUlHHDwUjvdtwNCFSFlQD9YRvNvRJN774wJ+TtnOufuo+W7QNsyX8mv1m7uMjaorNSIcgsghM5K/6NEA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=N3YplCOTsOWVEPBM0hbZ9GublLwFAwkekCPXORZYxoI=; b=fIE3XsP01OG3w/ikvyV7Ph1rps9OgVPWulcPr8KbHLwFDKrfGQ7o3GwFJSL2Bh77imBhCUwBPX6XUyu1jsBSG5i4A3GJ98bxNfHHL2hIibU8tfOFinlfZGjKGOAaqr3OiVLtSAu6Hpnzaa8XMJeFM+fLmklRLw5GLM1pfzb1jvt4FVqP+LVqpVz7mmOro+SRvD/CSS6+O0KATSy6uD0dPyNLIGCvfcQJCkQvtFPdH0c3gIQBuCBdB9i6mLZ9/1IfQtrl7nkm5kC3DVHZXCk0vT0bFFg+kxs0NnJ4LZcjKBMjl6egXBfsPiBy8FSD9Xt+7ZjZWynvYs32NePlrY/7Fg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=amd.com; dmarc=pass action=none header.from=amd.com; dkim=pass header.d=amd.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=N3YplCOTsOWVEPBM0hbZ9GublLwFAwkekCPXORZYxoI=; b=NwG0H3rEz/3kUb21CT+AtPRE5p8ALskS/ca1f3ueE8kmyyMdNyfhziumVzTaG0PtjKBK3Sg2VFCp4aUY1yFow0eF04dNk8dC6jSZ6H+JkStUaHAC4WsSq+Afpe8W8Bc7aGHUfXvaU8qn0PLGB6j3qCb4CJV6ciTqL8gMx0aUzec= Authentication-Results: lists.freedesktop.org; dkim=none (message not signed) header.d=none; lists.freedesktop.org; dmarc=none action=none header.from=amd.com; Received: from SN6PR12MB4623.namprd12.prod.outlook.com (2603:10b6:805:e9::17) by SN6PR12MB2749.namprd12.prod.outlook.com (2603:10b6:805:6d::18) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4065.22; Wed, 28 Apr 2021 15:13:04 +0000 Received: from SN6PR12MB4623.namprd12.prod.outlook.com ([fe80::ad51:8c49:b171:856c]) by SN6PR12MB4623.namprd12.prod.outlook.com ([fe80::ad51:8c49:b171:856c%7]) with mapi id 15.20.4065.026; Wed, 28 Apr 2021 15:13:04 +0000 From: Andrey Grodzovsky To: dri-devel@lists.freedesktop.org, amd-gfx@lists.freedesktop.org, linux-pci@vger.kernel.org, ckoenig.leichtzumerken@gmail.com, daniel.vetter@ffwll.ch, Harry.Wentland@amd.com Subject: [PATCH v5 18/27] drm/sched: Expose drm_sched_entity_kill_jobs Date: Wed, 28 Apr 2021 11:11:58 -0400 Message-Id: <20210428151207.1212258-19-andrey.grodzovsky@amd.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210428151207.1212258-1-andrey.grodzovsky@amd.com> References: <20210428151207.1212258-1-andrey.grodzovsky@amd.com> X-Originating-IP: [2607:fea8:3edf:49b0:7212:f93a:73b0:8f23] X-ClientProxiedBy: YT1PR01CA0142.CANPRD01.PROD.OUTLOOK.COM (2603:10b6:b01:2f::21) To SN6PR12MB4623.namprd12.prod.outlook.com (2603:10b6:805:e9::17) MIME-Version: 1.0 X-MS-Exchange-MessageSentRepresentingType: 1 Received: from agrodzovsky-All-Series.hitronhub.home (2607:fea8:3edf:49b0:7212:f93a:73b0:8f23) by YT1PR01CA0142.CANPRD01.PROD.OUTLOOK.COM (2603:10b6:b01:2f::21) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4065.25 via Frontend Transport; Wed, 28 Apr 2021 15:13:03 +0000 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 74a03a83-73e7-4550-6a4d-08d90a581f3a X-MS-TrafficTypeDiagnostic: SN6PR12MB2749: X-MS-Exchange-Transport-Forked: True X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:83; X-MS-Exchange-SenderADCheck: 1 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: oyeokwACEEb6kYJgJffPwFT0HVC7SzZV/XFCJZXXs66KcELWOq4z+PYmoBhAhTu81vOXSqsCV9Y21rcGs3iw0cYiQvMgoM8167BkMmpffmhfbO/0vahae4wdJBKOE7oWYonUeE6JcCVYlRDJO6HYp9fYnAiTwdtVRnOaZSAKU1kmdACM+mGFnV0Q75jw6ct8LyBV88K8Jsi9s7ur0cmv41JsYzNh2yrfZ8wWy11lEhizmlRb8iPQ8ZOhv9MAlo3MPSJix/xo0PGJNTIIUdlMvQx+d2yTuC85hxFKBrYLN1rowhOjAYJVxgrjvFQ4TAR4b0nStEKWl3zNKwqIpSmlBqTv1ONLIh2rQBzgRhVvQValSQHUchIV55W3MS80UDK8AHl3AFlSsxsL0DRGtOwpn1f38oexFDHU3A5he2XNBJ6H+6+QXVw7AovxCMJMhm58iwI+ad+dDz7i6hroGkWPjQ34cEwjG1r6a7OvgvcOFjtmxc3cOB+rVDu4uBUziCekkSd+CMIE4x6CkQagywC3v8+agDHVKnF4crLJjk3rVLnuIC5VWGWNv/XrvPXOf0MxZkiTFLcVyvxGPYK5j2I/MOWKhlvIQTf7TLWTx118Wn0= X-Forefront-Antispam-Report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:SN6PR12MB4623.namprd12.prod.outlook.com; PTR:; CAT:NONE; SFS:(4636009)(376002)(346002)(39860400002)(136003)(396003)(366004)(16526019)(6666004)(6512007)(2616005)(1076003)(2906002)(6636002)(66556008)(186003)(86362001)(5660300002)(8936002)(83380400001)(6486002)(478600001)(44832011)(38100700002)(66946007)(52116002)(6506007)(66476007)(8676002)(4326008)(36756003)(316002); DIR:OUT; SFP:1101; X-MS-Exchange-AntiSpam-MessageData: vR3MRUO9FSYjh55ZIaebNLDKLjRUdudpps6nyfy3N+BO0SidzKShVPOAT4qsA6gULmZBb9cfGIrGDiv6IvTXbgW45vbXzNK2rr24sk3tgSc7kKpNY6yGF+2pMHVOSqmENvOMb4XpA+m9kgby9D0XHim65QTwD1gCtFwdee6oCrDyk2HtyaQi7uE57FgK4MkxkuO6z51yu2mnFPNH4XZyOM59RrNAMbeoFrAg0TnyPtGKtQzXrMLHRlZUJsdx/XN37IrZqr9ybnoUHsruF+vx0gOCSIP/OBXuZ/vcQijEVL/GB1pQRvzK0B0FiK0zAG5vyIRgLR5olvOqSt92P9ARLGJTmwCd8KlNg/J0G8m3pz+D9h2mPFeZhEkcCanivj//NdkRhCLNJkfgdX0Npj4iRneYhx2etR35z15oGZosjxhoQ2+Yrm9UiIW5C9HcDIHVtkJM+9GndjuBBiiio6emkRjlWo7pCxVSRgYEuwqUNwBFYKd+RqRg2l0korBxGxpxpry7h96WBRjSaM1xq1lol3vLhn4VzbxHIYo5yWgmH9P7VLgTG+l/SKzw+fuV8cjBNGendzIJJkai3lvN4xumjDKBGWwqfBSCLjKB0aybWWnqHR6XQ/D3ObYTlbB15ILAorM5JD5saA/2Qh/QqL67+vETC3vLrRLk+krdWoyXUexnIHYp//BkNbxKGHHkDjKXsecGQCBIR5Pl1HnnV0YOK5MiWNvm9TevqmsTmTvJECdYmINvwrmVzIrQYPtvUQ92jY1NB5a9Ph18dDD34a7a/OFgpiHKy5C4M04j3Qd0OzzPJkjSXSLS7whizCDXF4uRLGIYoFh9bS40K8d/GAcqcAN7ifg/xiZNUkjcDYtVphi+KY0RQXqSJsDc10XZwuzL1QZhihoDRcyiNPovUo3bTJ4jBMBOya8EXL9TuTMuo9ZUf/mF4UXkQY1IGKnIdgQehZ5HS8wmsfXCMkUb3O6iI4mdx+qA59ouVNoT5DYcLhC/L8qTEECDaOI1ABb2WQ+0c2Vy2vPqrEPby/h0T6Nf89hLXZlczjytrwGNoa4UMakoMQUCdYPErnOpmywUDXar0EeFn8rnPwJpDd3NIN+A2vXEfI9tG+/9SQt3rE0bztGzC/rBX88VUSt18CJVthYAAiBMl27Gyco7corWRwUsOIbuJWtGRRLZZVJpSxm2ns3e0iGv/PfelFsoeNl9qVsNw5ZrxrEIDPFxgD0c6U3f0J0VzUL0HTC8MKuF/sDEIGTzuQBVqzuXqeRErEbrvObwA+hxWrCvANKrC1phgZUkRxGOxicc7+p826xh+ZanBdXOc+RpaGmR75/UqJeDSAIvTgld9IcPaZ1F5AxezeAy0YNryncdEWejLCx3aR7/E1zGxgDph52T0vO/frALQoAX X-OriginatorOrg: amd.com X-MS-Exchange-CrossTenant-Network-Message-Id: 74a03a83-73e7-4550-6a4d-08d90a581f3a X-MS-Exchange-CrossTenant-AuthSource: SN6PR12MB4623.namprd12.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 28 Apr 2021 15:13:04.7997 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: IoLXhihCyaR7tl77E5y300I92pfiL3P6qjPrsSC508aX/Ul616d6Vyn0QoyGWXE4zh+xJFbO6RPTgG5j7AGSNQ== X-MS-Exchange-Transport-CrossTenantHeadersStamped: SN6PR12MB2749 X-BeenThere: dri-devel@lists.freedesktop.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Direct Rendering Infrastructure - Development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: gregkh@linuxfoundation.org, Felix.Kuehling@amd.com, helgaas@kernel.org, Alexander.Deucher@amd.com Errors-To: dri-devel-bounces@lists.freedesktop.org Sender: "dri-devel" Will be used to complete all schedulte fences on device remove Signed-off-by: Andrey Grodzovsky --- drivers/gpu/drm/scheduler/sched_entity.c | 3 ++- include/drm/gpu_scheduler.h | 1 + 2 files changed, 3 insertions(+), 1 deletion(-) diff --git a/drivers/gpu/drm/scheduler/sched_entity.c b/drivers/gpu/drm/scheduler/sched_entity.c index cb58f692dad9..9ff4bfd8f548 100644 --- a/drivers/gpu/drm/scheduler/sched_entity.c +++ b/drivers/gpu/drm/scheduler/sched_entity.c @@ -219,7 +219,7 @@ static void drm_sched_entity_kill_jobs_cb(struct dma_fence *f, * Makes sure that all remaining jobs in an entity are killed before it is * destroyed. */ -static void drm_sched_entity_kill_jobs(struct drm_sched_entity *entity) +void drm_sched_entity_kill_jobs(struct drm_sched_entity *entity) { struct drm_sched_job *job; int r; @@ -249,6 +249,7 @@ static void drm_sched_entity_kill_jobs(struct drm_sched_entity *entity) DRM_ERROR("fence add callback failed (%d)\n", r); } } +EXPORT_SYMBOL(drm_sched_entity_kill_jobs); /** * drm_sched_entity_cleanup - Destroy a context entity diff --git a/include/drm/gpu_scheduler.h b/include/drm/gpu_scheduler.h index f888b5e9583a..9601d5b966ba 100644 --- a/include/drm/gpu_scheduler.h +++ b/include/drm/gpu_scheduler.h @@ -339,6 +339,7 @@ int drm_sched_entity_init(struct drm_sched_entity *entity, unsigned int num_sched_list, atomic_t *guilty); long drm_sched_entity_flush(struct drm_sched_entity *entity, long timeout); +void drm_sched_entity_kill_jobs(struct drm_sched_entity *entity); void drm_sched_entity_fini(struct drm_sched_entity *entity); void drm_sched_entity_destroy(struct drm_sched_entity *entity); void drm_sched_entity_select_rq(struct drm_sched_entity *entity); From patchwork Wed Apr 28 15:11:59 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrey Grodzovsky X-Patchwork-Id: 12229275 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.5 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,MSGID_FROM_MTA_HEADER,SPF_HELO_NONE,SPF_PASS, USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 798A8C43461 for ; Wed, 28 Apr 2021 15:14:21 +0000 (UTC) Received: from gabe.freedesktop.org (gabe.freedesktop.org [131.252.210.177]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 0F73F613F1 for ; Wed, 28 Apr 2021 15:14:20 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 0F73F613F1 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=amd.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=dri-devel-bounces@lists.freedesktop.org Received: from gabe.freedesktop.org (localhost [127.0.0.1]) by gabe.freedesktop.org (Postfix) with ESMTP id 6D8D86EB92; Wed, 28 Apr 2021 15:14:18 +0000 (UTC) Received: from NAM11-CO1-obe.outbound.protection.outlook.com (mail-co1nam11on2042.outbound.protection.outlook.com [40.107.220.42]) by gabe.freedesktop.org (Postfix) with ESMTPS id F01486EB69; Wed, 28 Apr 2021 15:13:08 +0000 (UTC) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=ELMxuyldqXmLpEaQzn5CmeMnESTpBpxRecHpOurLecQnAZV13yMA093UlSR19Ko/KIiy2OuvV2PX4JCCRi1Ltm+JBV0Qfnc19BioE2mjhx4FIAWkJa8ywKS05qdhXCjDCRcGsk1M3mIsaZgwTJ0yCtiIpWQaj3jX60PnwOMMy5Znw5iJxg/Fgj6C1P7QiD1hXcz3XHr84UxEiWjanWZ575zgKH0VWjrMVGZ3gWlW5fNx3CcWly98unM7e25sO1QXznhZ9jHCLYcpfjm+deEfFGeCChtFYGs/hKf4giTJfRj6Y00ceeZloHMgpZkwIFtO15V1nVWivy3V1EAuI1u5OQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=AGvjbVIrpRahh6cQMEYqaY7OXbk4P1Q0/vRMuCFVhNM=; b=PfsAunDy2OPDyMDsWv6GeIukGL2XMPHB1tlIinwdGe79Rz7tts9Is5uCyO9WPy074zMoLXW2JHXl21tncdUOsFB5rp2fGMJmsY1wmVht0fYweD0DXs+Tkk5UX4DDjbBjajuyVUokiv1mIy8Smh4rZYTr70cSbB6TjoTBJBtg9LQwGNLP4CC2EvruTp9vBa6gFNGkVVm+Vz1mGQoYrGOds33KOVCiwMdN4+eRI1ZKh4ZtYzKw14rOFtY+E20rcr74THHK0OeA0hzhidmhXhZekbnuZmLpLzk0prGCKjJW6BU9nAvsmKFXRlXJPy/ZPa7Oc6QAAqKVOYTiBXJsejPYRg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=amd.com; dmarc=pass action=none header.from=amd.com; dkim=pass header.d=amd.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=AGvjbVIrpRahh6cQMEYqaY7OXbk4P1Q0/vRMuCFVhNM=; b=3Ao7XekyC9Uyyntktt4bTgH+wyMWFy08CDA8eIQM+WVA8cP3zOBwdmkY5ZKT52Sy8L0yGtvanUbr/ZAh5aaoY1GJHpLXIeb29AgpSE6/VJJGjP798Wjd1ysNEnb6kabWZtaONgDW3I/LqsgVJGIy0LfTGG/bvOlIyLI9Su5Ozms= Authentication-Results: lists.freedesktop.org; dkim=none (message not signed) header.d=none; lists.freedesktop.org; dmarc=none action=none header.from=amd.com; Received: from SN6PR12MB4623.namprd12.prod.outlook.com (2603:10b6:805:e9::17) by SN6PR12MB2749.namprd12.prod.outlook.com (2603:10b6:805:6d::18) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4065.22; Wed, 28 Apr 2021 15:13:06 +0000 Received: from SN6PR12MB4623.namprd12.prod.outlook.com ([fe80::ad51:8c49:b171:856c]) by SN6PR12MB4623.namprd12.prod.outlook.com ([fe80::ad51:8c49:b171:856c%7]) with mapi id 15.20.4065.026; Wed, 28 Apr 2021 15:13:06 +0000 From: Andrey Grodzovsky To: dri-devel@lists.freedesktop.org, amd-gfx@lists.freedesktop.org, linux-pci@vger.kernel.org, ckoenig.leichtzumerken@gmail.com, daniel.vetter@ffwll.ch, Harry.Wentland@amd.com Subject: [PATCH v5 19/27] drm/amdgpu: Finilise device fences on device remove. Date: Wed, 28 Apr 2021 11:11:59 -0400 Message-Id: <20210428151207.1212258-20-andrey.grodzovsky@amd.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210428151207.1212258-1-andrey.grodzovsky@amd.com> References: <20210428151207.1212258-1-andrey.grodzovsky@amd.com> X-Originating-IP: [2607:fea8:3edf:49b0:7212:f93a:73b0:8f23] X-ClientProxiedBy: YT1PR01CA0142.CANPRD01.PROD.OUTLOOK.COM (2603:10b6:b01:2f::21) To SN6PR12MB4623.namprd12.prod.outlook.com (2603:10b6:805:e9::17) MIME-Version: 1.0 X-MS-Exchange-MessageSentRepresentingType: 1 Received: from agrodzovsky-All-Series.hitronhub.home (2607:fea8:3edf:49b0:7212:f93a:73b0:8f23) by YT1PR01CA0142.CANPRD01.PROD.OUTLOOK.COM (2603:10b6:b01:2f::21) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4065.25 via Frontend Transport; Wed, 28 Apr 2021 15:13:05 +0000 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 1582cf64-1891-42d5-549c-08d90a582038 X-MS-TrafficTypeDiagnostic: SN6PR12MB2749: X-MS-Exchange-Transport-Forked: True X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:9508; X-MS-Exchange-SenderADCheck: 1 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: zBz1a9VD2XnAxFSeD0LOOFbFfEkN387mnxOX6pAEG0369FJt6fT4TW2SaVfSdS15uSFy3fTbccQY0ubVwvN0MSi0EoEKlHW6FXnfysMjoeUNI6MTrD0faL0TDKC325buNN//JqVCmFKloxq9jZX2tGJKjgeQuPJXVxWMT2uXGDPCRe/pypZqvosaiBXNAUGoZmnaFqAB5xWnIxyIG/Ca/IdUxTVoVU8oY9D/C7/7uEgBXQ9DBDTEMBPuCAlZ/gHl8sUL8TKiVQSgsYzGd6A0p/ysNvgFlibglnlrcPMOB4XBEBHwRyYpKVUxATlrWjsHrSHOyonIeUd46My/iD2U5ERcj94VLZ6/S554R/+hMUu0cZp4DsVXS7iijHhZIgxfwwGezdL2+RLlQ85vN8UGf19vM2MsQLauoVkQWruGe9ytWJ8FianFKOBPkpgNIBCAGfCbpZZ1H+L39eA2DQznO+Egd1rwLOS+IKKVTIeUn4eSdBmHu34ZLyuND8rCD3EGvlLHJumaLLIOKCK5Sja5eZmw4vTYKIzPMWbId3b+sFIwJCm8pWfMiqMebF3nBWcYBT9WTNt9LNKtPFo8t83IeQ== X-Forefront-Antispam-Report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:SN6PR12MB4623.namprd12.prod.outlook.com; PTR:; CAT:NONE; SFS:(4636009)(376002)(346002)(39860400002)(136003)(396003)(366004)(16526019)(6666004)(6512007)(2616005)(1076003)(2906002)(6636002)(66556008)(186003)(86362001)(5660300002)(8936002)(83380400001)(6486002)(478600001)(44832011)(38100700002)(66946007)(52116002)(6506007)(66476007)(8676002)(4326008)(36756003)(316002); DIR:OUT; SFP:1101; X-MS-Exchange-AntiSpam-MessageData: QCtLlIkVtH45As2v4d4JwIe8sxP3HCuzg+Fc5oJMCuhp/us+f9Z9zVU6abaIRJTJIOiyq7XjtHZXhH8tj+BSnazvrhjcaafHtaXJPtb4cdQK60S43g4PvuLRld6JXaIfLX7IDzNtY8nODoIPq7WlfAe9h6HCslUXwlJfUjw9GUv12KkwNAh+oFqz2bAOQNQ5x8fI9k9+3VmqwYdJJ/6Tm1kaNE75amQvb06AEwg8YySWFoHySr4evvZXfPwKz8K51BDN62RzjazzqeDJERvzpscYa0la70uIZocbejuYE57eebrCP6sf9GkzESnuVSmVam1R1ct8BkrDGCKQ5kallAwqIkTi55UHNHBz/Qgioi0/TmU81dEMp2WDJ+xbrROYksS61npDRzobycae0JcnrrUEV914NOaF0trgOap3EgFZgChibOsWjeso3O8kRXGYUOyE42IacMmnSVdWiqsUcDPb4g+i2JexGCc1Phf7BEOKKKtupuwjWpIdsLhOy1WRosaMY2Z5sQn4GaCl8P1yYv5XwYymyUEnGdoQs5uQyPp/5zi7wp91U2mKnz/Vgs92icBaa/qUVVjezDTHqNXSagSYig6xgvFjc88DXzKYRZards5uYaDMnVSPVgv6mLPJkZrENTEmuvun6vmCjHifXvKDR1z5yAeRr8ewHgNgYgvcqQvT6Z/pkJNKMoJ/yDAo4qQ4CWYEdA+aGpT+7iIYpkxNtRscxMGHemEkEaHy7nSRZ58/GMT2iYlz/tOh5NGhXPOx2bxdz9lugcxlivSf3Gj9Ryi0w3+mqLfspkdskEfSHzus9632QfvBlct/XgI+WKBKZXsvmoZm13xXN6h3hqca+jGUfD7xBwojCaNhA8oROl22zcxvRB6HC922Mmoj+ijfyHzHJyuOwt34wsp1+wdoywvwxdyWqlrh/u12LtAAXkCv8b+nJQL2zKMdVKr5ytNAMzP5MK27OOL5L4fR1djJmf6PGsWEVvsqwbv//XE9Bzjk8P6owxhikQSotMkNWBWYPdkpv7huCIVuSDj8iPw7g9Ult60E4FIjJGOoQzWM0H/PLqTyjXuIavH9B6iuLwyHeGXp+vxY5voRril4omsekG9JbIocNhaMed5ZYiE7U341unK38VUx2bqKWAZR3HQXnNCWcoyE05cyihQBGOnRoLpEUK5v5Tnwrh1lzPAzuLmc+Ctx4QdfWtc6/sGooUrropbD0bBJcJgwvb9QjqwPt2nkRyo4DLZ3aT0h8MO1i3mHi5kC1imI+7/zD+xV9ubPJl7mzRX7tgzGzqNNa793x8/u668vaavelJ8xmgl7dzA5MvSMyKWc4joQdsGqHYBTCvWUD9/UDqyimwjQlb6xBLz5clLDFldjH4Efy01SHVafSbObt2xnhYu18k+O X-OriginatorOrg: amd.com X-MS-Exchange-CrossTenant-Network-Message-Id: 1582cf64-1891-42d5-549c-08d90a582038 X-MS-Exchange-CrossTenant-AuthSource: SN6PR12MB4623.namprd12.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 28 Apr 2021 15:13:06.4757 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: 0ZYhqjxWnwR8zvB9JI7dnQuAgZ8QSY1Vbj5N50jzcjMBlMELmEHRTK9vczHWRNUgMXh2P4QckgUxmfXmHeYCiA== X-MS-Exchange-Transport-CrossTenantHeadersStamped: SN6PR12MB2749 X-BeenThere: dri-devel@lists.freedesktop.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Direct Rendering Infrastructure - Development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: gregkh@linuxfoundation.org, Felix.Kuehling@amd.com, helgaas@kernel.org, Alexander.Deucher@amd.com Errors-To: dri-devel-bounces@lists.freedesktop.org Sender: "dri-devel" Make sure all fecens dependent on HW present are force signaled when handling device removal. This helpes later to scope all HW accesing code such as IOCTLs in drm_dev_enter/exit and use drm_dev_unplug as synchronization point past which we know HW will not be accessed anymore outside of pci remove driver callback. Signed-off-by: Andrey Grodzovsky --- drivers/gpu/drm/amd/amdgpu/amdgpu.h | 2 + drivers/gpu/drm/amd/amdgpu/amdgpu_device.c | 98 ++++++++++++++++++++-- drivers/gpu/drm/amd/amdgpu/amdgpu_drv.c | 6 ++ drivers/gpu/drm/amd/amdgpu/amdgpu_fence.c | 12 +-- 4 files changed, 103 insertions(+), 15 deletions(-) diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu.h b/drivers/gpu/drm/amd/amdgpu/amdgpu.h index 0db0ba4fba89..df6c5ed676b1 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu.h +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu.h @@ -1374,6 +1374,8 @@ void amdgpu_pci_resume(struct pci_dev *pdev); bool amdgpu_device_cache_pci_state(struct pci_dev *pdev); bool amdgpu_device_load_pci_state(struct pci_dev *pdev); +void amdgpu_finilize_device_fences(struct drm_device *dev); + #include "amdgpu_object.h" static inline bool amdgpu_is_tmz(struct amdgpu_device *adev) diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c index 33e8e9e1d1fe..55afc11c17e6 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c @@ -3692,15 +3692,12 @@ void amdgpu_device_fini_hw(struct amdgpu_device *adev) amdgpu_virt_fini_data_exchange(adev); } - /* disable all interrupts */ - amdgpu_irq_disable_all(adev); if (adev->mode_info.mode_config_initialized){ if (!amdgpu_device_has_dc_support(adev)) drm_helper_force_disable_all(adev_to_drm(adev)); else drm_atomic_helper_shutdown(adev_to_drm(adev)); } - amdgpu_fence_driver_fini_hw(adev); if (adev->pm_sysfs_en) amdgpu_pm_sysfs_fini(adev); @@ -4567,14 +4564,19 @@ static bool amdgpu_device_lock_adev(struct amdgpu_device *adev, return true; } -static void amdgpu_device_unlock_adev(struct amdgpu_device *adev) +static void amdgpu_device_unlock_adev_imp(struct amdgpu_device *adev, bool skip_in_gpu_reset) { amdgpu_vf_error_trans_all(adev); adev->mp1_state = PP_MP1_STATE_NONE; - atomic_set(&adev->in_gpu_reset, 0); + !skip_in_gpu_reset ? atomic_set(&adev->in_gpu_reset, 0) : 0; up_write(&adev->reset_sem); } +static void amdgpu_device_unlock_adev(struct amdgpu_device *adev) +{ + amdgpu_device_unlock_adev_imp(adev, false); +} + /* * to lockup a list of amdgpu devices in a hive safely, if not a hive * with multiple nodes, it will be similar as amdgpu_device_lock_adev. @@ -5321,3 +5323,89 @@ bool amdgpu_device_load_pci_state(struct pci_dev *pdev) } +static void amdgpu_finilize_schedulded_fences(struct amdgpu_ctx_mgr *mgr) +{ + struct amdgpu_ctx *ctx; + struct idr *idp; + uint32_t id, i, j; + + idp = &mgr->ctx_handles; + + idr_for_each_entry(idp, ctx, id) { + for (i = 0; i < AMDGPU_HW_IP_NUM; ++i) { + for (j = 0; j < amdgpu_ctx_num_entities[i]; ++j) { + struct drm_sched_entity *entity; + + if (!ctx->entities[i][j]) + continue; + + entity = &ctx->entities[i][j]->entity; + drm_sched_entity_kill_jobs(entity); + } + } + } +} + +/** + * amdgpu_finilize_device_fences() - Finilize all device fences + * @pdev: pointer to PCI device + * + * Will disable and finilise ISRs and will signal all fences + * that might hang if HW is gone + */ +void amdgpu_finilize_device_fences(struct drm_device *dev) +{ + struct amdgpu_device *adev = drm_to_adev(dev); + struct drm_file *file; + + /* + * Block TDRs from further execution by setting adev->in_gpu_reset + * instead of holding full reset lock in order to not deadlock + * further ahead against any thread locking the reset lock when we + * wait for it's completion + */ + while (!amdgpu_device_lock_adev(adev, NULL)) + amdgpu_cancel_all_tdr(adev); + amdgpu_device_unlock_adev_imp(adev, true); + + + /* disable all HW interrupts */ + amdgpu_irq_disable_all(adev); + + /* stop and flush all in flight HW interrupts handlers */ + disable_irq(pci_irq_vector(adev->pdev, 0)); + + /* + * Stop SW GPU schedulers and force completion on all HW fences. Since + * in the prev. step all ISRs were disabled and completed the + * HW fence array is idle (no insertions or extractions) and so it's + * safe to iterate it bellow. + * After this step all HW fences in the system are signaled. As a result + * also all the scheduler 'finished' fences are also signaled. + */ + amdgpu_fence_driver_fini_hw(adev); + + /* + * Reject any further jobs to any scheduler entity queue. After this + * step no new insertions and because schedulers are stopped also no + * new extractions. + */ + down_read(&adev->sched_fence_completion_sem); + adev->stop_job_submissions = true; + up_read(&adev->sched_fence_completion_sem); + + /* + * Complete all scheduler 'scheduled' fences currently pending. + * It's OK if new contexts and sched entities are concurrently + * still created as they will fail in pushing jobs to SW queues + * and their schedule fences will be signaled with error + */ + mutex_lock(&adev->ddev.filelist_mutex); + list_for_each_entry(file, &adev->ddev.filelist, lhead) { + struct amdgpu_fpriv *fpriv = file->driver_priv; + amdgpu_finilize_schedulded_fences(&fpriv->ctx_mgr); + } + mutex_unlock(&adev->ddev.filelist_mutex); +} + + diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_drv.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_drv.c index f799c40d7e72..8a19b8dd02ee 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_drv.c +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_drv.c @@ -1249,6 +1249,12 @@ amdgpu_pci_remove(struct pci_dev *pdev) { struct drm_device *dev = pci_get_drvdata(pdev); + /* + * Force completion of all device related fences that might hang us when + * synchronizing SRCU in the following step. + */ + amdgpu_finilize_device_fences(dev); + drm_dev_unplug(dev); amdgpu_driver_unload_kms(dev); diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_fence.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_fence.c index 2670201e78d3..af592b28cd35 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_fence.c +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_fence.c @@ -526,7 +526,7 @@ int amdgpu_fence_driver_init(struct amdgpu_device *adev) */ void amdgpu_fence_driver_fini_hw(struct amdgpu_device *adev) { - int i, r; + int i; for (i = 0; i < AMDGPU_MAX_RINGS; i++) { struct amdgpu_ring *ring = adev->rings[i]; @@ -535,18 +535,10 @@ void amdgpu_fence_driver_fini_hw(struct amdgpu_device *adev) continue; /* Stop any new job submissions from sched before flushing the ring */ - /* TODO Handle amdgpu_job_submit_direct and amdgpu_amdkfd_submit_ib */ if (!ring->no_scheduler) drm_sched_fini(&ring->sched); - /* You can't wait for HW to signal if it's gone */ - if (!drm_dev_is_unplugged(&adev->ddev)) - r = amdgpu_fence_wait_empty(ring); - else - r = -ENODEV; - /* no need to trigger GPU reset as we are unloading */ - if (r) - amdgpu_fence_driver_force_completion(ring); + amdgpu_fence_driver_force_completion(ring); if (ring->fence_drv.irq_src) amdgpu_irq_put(adev, ring->fence_drv.irq_src, From patchwork Wed Apr 28 15:12:00 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrey Grodzovsky X-Patchwork-Id: 12229247 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.5 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,MSGID_FROM_MTA_HEADER,SPF_HELO_NONE,SPF_PASS, USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 2B064C433B4 for ; Wed, 28 Apr 2021 15:13:21 +0000 (UTC) Received: from gabe.freedesktop.org (gabe.freedesktop.org [131.252.210.177]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id CA1EF61434 for ; Wed, 28 Apr 2021 15:13:20 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org CA1EF61434 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=amd.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=dri-devel-bounces@lists.freedesktop.org Received: from gabe.freedesktop.org (localhost [127.0.0.1]) by gabe.freedesktop.org (Postfix) with ESMTP id 6FA696EB72; Wed, 28 Apr 2021 15:13:18 +0000 (UTC) Received: from NAM11-CO1-obe.outbound.protection.outlook.com (mail-co1nam11on2042.outbound.protection.outlook.com [40.107.220.42]) by gabe.freedesktop.org (Postfix) with ESMTPS id DD6F36EB6C; Wed, 28 Apr 2021 15:13:10 +0000 (UTC) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=IlspMSbwS2uOwU6cUkfk6sakE08kRJhWjyGaKbYrOGYoe58ZfE/J2pSq6ZoR2SJfnr1Rq2sdoPJlQhWADTbNP9HlGmjyG+j0V3mnF6bpb3I3NNzRAMxskYCqED2DBTRjP/7L6jwDJCB5boxbb5iynkGLvtioBJ4e2NP+a6LRMnkeJ3Yybp3c+7h3wBPMctnnn0FJ+wwPCo3G7/yS9p4rJ46SqmCjA4Z/25F9JXDPkNSxEaZK8RWNH3Vym0oYUVAkJTWVJHugPOKZHpSzaH5dH18E6xmSGaGJGwrXp4kmHSClBWXNh+hS2zRDViv8q789euneQjgjDPhz8pXPrDQxow== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=oi6EUX0FKfCptExWIFjDRDfgt19i4KeMWV08PIEKsVE=; b=XZojVhrC4j+hGWiaRPcVYoUNOLP/4AqSQD7gDpuQjHViwmjj4XqCoHc765ZYtTNdOPCpBezEsRiv2jkQfhcyKd1LfocA0k+i9LB9aSGYilUQOtwm7MM3m3QaRsIGrN+N3HQa5IjVJD8j2dp7UbxDFvcbya5CocoHRL5zaY9F+SW2aJFTVVGmPhSfeAawUPiu80VaVbZHaum4HN0b0L/SORsLF/xFdL+ijxIDlbbLu31pbKdzrDn65qcS8otMjUpgK1wSScPMr7/jPnUgDFu1HewZfhZCKLBQyIx3LSbKtkgT+STYXby8ILhP7orhJ6PXwqOXS7fkfspCa+Ck7o224w== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=amd.com; dmarc=pass action=none header.from=amd.com; dkim=pass header.d=amd.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=oi6EUX0FKfCptExWIFjDRDfgt19i4KeMWV08PIEKsVE=; b=EhH6/RJDfZIIqb5LHQ9obMHatgtkhEN0Ph7uhfdyw0C5QTSNX1Hu6WUqvfZ34Fk9Q9prDETaLp8/e/2aJO14IZxnizsvnJEKY2CB8zu4IMIBtgbrcxJRMDp9rdbWdwxzuZkI6Z3P9RiZuqIcvS/Znxj9b9nKGBg26vTTQ4/+jgA= Authentication-Results: lists.freedesktop.org; dkim=none (message not signed) header.d=none; lists.freedesktop.org; dmarc=none action=none header.from=amd.com; Received: from SN6PR12MB4623.namprd12.prod.outlook.com (2603:10b6:805:e9::17) by SN6PR12MB2749.namprd12.prod.outlook.com (2603:10b6:805:6d::18) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4065.22; Wed, 28 Apr 2021 15:13:08 +0000 Received: from SN6PR12MB4623.namprd12.prod.outlook.com ([fe80::ad51:8c49:b171:856c]) by SN6PR12MB4623.namprd12.prod.outlook.com ([fe80::ad51:8c49:b171:856c%7]) with mapi id 15.20.4065.026; Wed, 28 Apr 2021 15:13:08 +0000 From: Andrey Grodzovsky To: dri-devel@lists.freedesktop.org, amd-gfx@lists.freedesktop.org, linux-pci@vger.kernel.org, ckoenig.leichtzumerken@gmail.com, daniel.vetter@ffwll.ch, Harry.Wentland@amd.com Subject: [PATCH v5 20/27] drm: Scope all DRM IOCTLs with drm_dev_enter/exit Date: Wed, 28 Apr 2021 11:12:00 -0400 Message-Id: <20210428151207.1212258-21-andrey.grodzovsky@amd.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210428151207.1212258-1-andrey.grodzovsky@amd.com> References: <20210428151207.1212258-1-andrey.grodzovsky@amd.com> X-Originating-IP: [2607:fea8:3edf:49b0:7212:f93a:73b0:8f23] X-ClientProxiedBy: YT1PR01CA0142.CANPRD01.PROD.OUTLOOK.COM (2603:10b6:b01:2f::21) To SN6PR12MB4623.namprd12.prod.outlook.com (2603:10b6:805:e9::17) MIME-Version: 1.0 X-MS-Exchange-MessageSentRepresentingType: 1 Received: from agrodzovsky-All-Series.hitronhub.home (2607:fea8:3edf:49b0:7212:f93a:73b0:8f23) by YT1PR01CA0142.CANPRD01.PROD.OUTLOOK.COM (2603:10b6:b01:2f::21) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4065.25 via Frontend Transport; Wed, 28 Apr 2021 15:13:07 +0000 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: c08a66c7-c932-4bb1-7c4f-08d90a58212c X-MS-TrafficTypeDiagnostic: SN6PR12MB2749: X-MS-Exchange-Transport-Forked: True X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:3826; X-MS-Exchange-SenderADCheck: 1 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: /2gAl/SYK//TcPDdgy/D24UqABDaWrXdfcsL2upOip+l7G0j/j/KjJ0+TdRYqYbnB74O7PWQg5jYutpz9GhTtFvxUZQuVQedqrHBtY3ztPVl+80jbbeL2sBQ8+PaNE5TJYbNVTwdrd/uRHxKCly9p0bX4OIVTH1DyKd0PI+okPlKUr0T9+7mbUWSxtDlLnctl9W2whhlNhIzQ4UnSOYt7i5mApQhTpRuv/EdjsTKxzK04Wp1IWRBA8h+6Jt0Hu85JpI5hK0Idp4PxdKZP5KHshE1qi55TL+3lDvI0/IndI7xHMmHQyKlpaLU0bPrk6ItiX7eMB1xJ/mmEnGfM06G9VZ2KRoo2uZR83WGV4vJIP46hlj9eNpxKUuhz7vG/RCNHcw9ec7/hku+JlHJs8V6Mro8+LxBn23X7fvqlh5/86BvQu5fXePMswGUXqI2uoUZfhdM6aaO2HEz/lUcna2Pegp9wNFcCFN7YcWBjeKRURKxQnp402ok+LhtDMCY8xAQXfU+outz8tL2Vn5PzTiG8guMNXzWrp064lUmYY6mG8g1Ao8L2iduF+0Fij1DWVYClzDx1yvs+k5dbaRdbld8wE4wnK2bf4waEcRWMguWU8c= X-Forefront-Antispam-Report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:SN6PR12MB4623.namprd12.prod.outlook.com; PTR:; CAT:NONE; SFS:(4636009)(376002)(346002)(39860400002)(136003)(396003)(366004)(16526019)(6666004)(6512007)(2616005)(1076003)(2906002)(6636002)(66556008)(186003)(86362001)(5660300002)(8936002)(83380400001)(6486002)(478600001)(44832011)(38100700002)(66946007)(52116002)(6506007)(66476007)(8676002)(4326008)(36756003)(316002); DIR:OUT; SFP:1101; X-MS-Exchange-AntiSpam-MessageData: 4YmPK/ATM/9/Pfbn/eouuYah88GuWGZjherXf7imh7mT6M/udxzyImSc9JF1KOLY6vf8uvI1pdT4O5ylWz4z4PHRXPFwGcnyJZDxfQZ8JWi00zRSFfG60Exy/aGN3bnuyzG6lNL3+3MIO8iaj1IEyPst2AAXdWcn4W65xRgqltQIDBHvXWkzQtfHTrjRuLs1+5pEgqxR7gj0KEbemZEBqv6fTsqFifFc1gs5HbteMEA5s2oVvgbvIjP5azw4HcP6yP9klUOFzXxaNnmO8rpHfuctxemmB1zthKStFt/Ivcr90M//+YhJCSWB/YTnWn31P6do14o5/dHROOgNPsVsyKbPRLRWNa4V95ViRP1fE3OPDtLVuPXUAUqe8QwFbqW1pQhU155qAzqzuBKVdZBbNhhdWuve+7mBaTYD21ryLrgx5sjo49ykEF+G1MuxqhRQQ2kMeJcj+8CijEOAQzlLaDCmRYY3QYCDdKVmAqZRftkFoWWBPsjOkTDaoqJWifYaN4ju1Izflz3cKhTGusn8TjOuxcCtX/zbkaP9gXN49E/u6Jj+qlhgjx3wsHfHUk2tcgxgF63nOYpfl43OOYSQsnQgq5q1d3TVt31cxrUEAHXh7kKepwg/EuOupQsQT/L/k+Weo6OC1OV8OYcuV3LZQwRM20xU+hiyeXe2Sy9r4uwPkqWvimwR+cKc2R154Q58mBqr56edMhVlyLs3CThBSMZWwJj6/IAT7JbIpKmXdPrWRh19dVV9YhB6Pduif2Rg9ze+yKA7mKp6EbJ4BrjRQa9MhEePIe5laLk3qhIlBRCBv5hSACsu3xyymwPdkkIk1Sx9XhxfZUVJiUd3oHRZNlJZgpxt3SsjTSO/Ba3s5YL1VqgV1LKN9F4wxZEjfB721gfQvA4oVAY9z8kMcoMqRJUAWXgcUKi6QPH1f10dUMvBE6SHmLo7w2cZwllAW8JasZba5xDqUZKRiMhjeGAtVodeAqG99uLrAfWHGgJ/oREF54ctPhr78GpVUlQwLgD7d3HZrPovn9JAv/v1WJ0/hTapcd19U/RCT1VzWLRVfj/t9cWyUsFkYyfAv3mw7zs0sFqIPA2PqePajFwti8yrFlsIE6p3r+m460/5hAX6foUYshyREqtYG625PF+qbPbxNf2N8YBF7lKs72UYaJ0BN4Gh8l9hXwyRCOfahWJr9KPQ/t13ONmzfXd3rfMKxNvq2efCbqmG/GGQC4MSToysZ/aQ9S9kuE9rcUKMB5TGXykq/Msjg+5mUfFYYkof1WzaYes2KCCOUj/LNMbARNvybylcYSQghqw3WKyp/SojdcGEdkR0+3OYUitqxfUUNcfXqF7LaMcK/6hnSNRrj7runIjGC4OzyXJhWYglnEzBNla35sZicSV+csuo7c8ZbGzf X-OriginatorOrg: amd.com X-MS-Exchange-CrossTenant-Network-Message-Id: c08a66c7-c932-4bb1-7c4f-08d90a58212c X-MS-Exchange-CrossTenant-AuthSource: SN6PR12MB4623.namprd12.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 28 Apr 2021 15:13:08.0658 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: Mgm8L703L8P9+PdXrScBVak30Rg/eX/iMS7ywBJHzl8rkHMM35gLPmn8L3nKngrAXl1NDlAnyrTugXwOWzM2OQ== X-MS-Exchange-Transport-CrossTenantHeadersStamped: SN6PR12MB2749 X-BeenThere: dri-devel@lists.freedesktop.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Direct Rendering Infrastructure - Development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: gregkh@linuxfoundation.org, Felix.Kuehling@amd.com, helgaas@kernel.org, Alexander.Deucher@amd.com Errors-To: dri-devel-bounces@lists.freedesktop.org Sender: "dri-devel" With this calling drm_dev_unplug will flush and block all in flight IOCTLs Also, add feature such that if device supports graceful unplug we enclose entire IOCTL in SRCU critical section. Signed-off-by: Andrey Grodzovsky --- drivers/gpu/drm/drm_ioctl.c | 15 +++++++++++++-- include/drm/drm_drv.h | 6 ++++++ 2 files changed, 19 insertions(+), 2 deletions(-) diff --git a/drivers/gpu/drm/drm_ioctl.c b/drivers/gpu/drm/drm_ioctl.c index d273d1a8603a..5882ef2183bb 100644 --- a/drivers/gpu/drm/drm_ioctl.c +++ b/drivers/gpu/drm/drm_ioctl.c @@ -815,7 +815,7 @@ long drm_ioctl(struct file *filp, const struct drm_ioctl_desc *ioctl = NULL; drm_ioctl_t *func; unsigned int nr = DRM_IOCTL_NR(cmd); - int retcode = -EINVAL; + int idx, retcode = -EINVAL; char stack_kdata[128]; char *kdata = NULL; unsigned int in_size, out_size, drv_size, ksize; @@ -884,7 +884,18 @@ long drm_ioctl(struct file *filp, if (ksize > in_size) memset(kdata + in_size, 0, ksize - in_size); - retcode = drm_ioctl_kernel(filp, func, kdata, ioctl->flags); + if (drm_core_check_feature(dev, DRIVER_HOTUNPLUG_SUPPORT)) { + if (drm_dev_enter(dev, &idx)) { + retcode = drm_ioctl_kernel(filp, func, kdata, ioctl->flags); + drm_dev_exit(idx); + } else { + retcode = -ENODEV; + goto err_i1; + } + } else { + retcode = drm_ioctl_kernel(filp, func, kdata, ioctl->flags); + } + if (copy_to_user((void __user *)arg, kdata, out_size) != 0) retcode = -EFAULT; diff --git a/include/drm/drm_drv.h b/include/drm/drm_drv.h index b439ae1921b8..63e05cec46c1 100644 --- a/include/drm/drm_drv.h +++ b/include/drm/drm_drv.h @@ -94,6 +94,12 @@ enum drm_driver_feature { * synchronization of command submission. */ DRIVER_SYNCOBJ_TIMELINE = BIT(6), + /** + * @DRIVER_NO_HOTUNPLUG_SUPPORT: + * + * Driver support gracefull remove. + */ + DRIVER_HOTUNPLUG_SUPPORT = BIT(7), /* IMPORTANT: Below are all the legacy flags, add new ones above. */ From patchwork Wed Apr 28 15:12:01 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrey Grodzovsky X-Patchwork-Id: 12229267 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.5 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,MSGID_FROM_MTA_HEADER,SPF_HELO_NONE,SPF_PASS, USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 453B7C43462 for ; Wed, 28 Apr 2021 15:14:12 +0000 (UTC) Received: from gabe.freedesktop.org (gabe.freedesktop.org [131.252.210.177]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id E920D6143D for ; Wed, 28 Apr 2021 15:14:11 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org E920D6143D Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=amd.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=dri-devel-bounces@lists.freedesktop.org Received: from gabe.freedesktop.org (localhost [127.0.0.1]) by gabe.freedesktop.org (Postfix) with ESMTP id 7DDC56EB8F; Wed, 28 Apr 2021 15:14:10 +0000 (UTC) Received: from NAM11-CO1-obe.outbound.protection.outlook.com (mail-co1nam11on2042.outbound.protection.outlook.com [40.107.220.42]) by gabe.freedesktop.org (Postfix) with ESMTPS id EDD1B6EB6E; Wed, 28 Apr 2021 15:13:11 +0000 (UTC) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=XgksMQ07JI2RbM4qQEes06vQRQw27WPwMW3eb0Bj6W3WD2eT/VvZdlVMhcZbdJrfcYDyMtDi5dVWO9aPsQlw8bx1QVI+nAuHzMjX2mdtRMTDaQkdzN6JSBz8GfcYzU6ApEGpgxFaKr4lriwpYVGzN1ycrL9I34K7tZJyfI0FoXGL4Y98fqGzAi79tJPA/AM7gwJMoftoigT2uH7nT9e4BQ7h88cekkZMMf3izFeosuUSbZajFLcXzyoPFyCMAcKh9ifx6tWYT4ByOAOZUTghmz+r7JB1kokTS/YIbNiCrRL8EOstq4dAX6VdLSp948mgmZdjp6C1b7q3L6EskgaPWQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=meHqu+jOFx/NRlvPU7mQmRV3l7XcXGwpzu52oFSr+mw=; b=L+zTszkrwlbYSlBFEP9ZPWyu9pBvuYyvkvZhlpPXM0sL8SDqGV7yozie+cEeHbbpjNWdxzCSJbwfEtbzZxO8f4+tdH4dOYu2aK0xECqhgcLXzq5UPUCWDuryB8HwbFsHSc0QUJkl2G8sDm0z6WHUPz4wW5HcJPqXoRL4PSY6RzoTFV0LOeHgc8ifkPWa6Co8NZccT6+WUFhkqkCnnS6cXdUPlSO4HxfHwvaF9gLqv+1ncp6x0nubtFr+Q+LweKYpM7tsPv+9fIFRnjj6RJ475wzCdD/PGUAC4actaJ2Gis0Lgv8ZRlvaS/DxvDrKoDuvYRf7z3xWOr+x3HT2Es0AKA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=amd.com; dmarc=pass action=none header.from=amd.com; dkim=pass header.d=amd.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=meHqu+jOFx/NRlvPU7mQmRV3l7XcXGwpzu52oFSr+mw=; b=26zbk4XIzqTVs86507SBqq7G6chmCbVULo5AtxVvVsEF5ftrvq+v9QeoudEIq0k1mKIhXispGTiEbNFFiI96rGPvZYrFgpxaGXjijpMewHtAGxJY0fIp/WAe+CRxtcQwOjcImTPzYhP7sb8OpoeuNjqt8qIC3DnmrdGYuW9PQG0= Authentication-Results: lists.freedesktop.org; dkim=none (message not signed) header.d=none; lists.freedesktop.org; dmarc=none action=none header.from=amd.com; Received: from SN6PR12MB4623.namprd12.prod.outlook.com (2603:10b6:805:e9::17) by SN6PR12MB2749.namprd12.prod.outlook.com (2603:10b6:805:6d::18) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4065.22; Wed, 28 Apr 2021 15:13:10 +0000 Received: from SN6PR12MB4623.namprd12.prod.outlook.com ([fe80::ad51:8c49:b171:856c]) by SN6PR12MB4623.namprd12.prod.outlook.com ([fe80::ad51:8c49:b171:856c%7]) with mapi id 15.20.4065.026; Wed, 28 Apr 2021 15:13:09 +0000 From: Andrey Grodzovsky To: dri-devel@lists.freedesktop.org, amd-gfx@lists.freedesktop.org, linux-pci@vger.kernel.org, ckoenig.leichtzumerken@gmail.com, daniel.vetter@ffwll.ch, Harry.Wentland@amd.com Subject: [PATCH v5 21/27] drm/amdgpu: Add support for hot-unplug feature at DRM level. Date: Wed, 28 Apr 2021 11:12:01 -0400 Message-Id: <20210428151207.1212258-22-andrey.grodzovsky@amd.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210428151207.1212258-1-andrey.grodzovsky@amd.com> References: <20210428151207.1212258-1-andrey.grodzovsky@amd.com> X-Originating-IP: [2607:fea8:3edf:49b0:7212:f93a:73b0:8f23] X-ClientProxiedBy: YT1PR01CA0142.CANPRD01.PROD.OUTLOOK.COM (2603:10b6:b01:2f::21) To SN6PR12MB4623.namprd12.prod.outlook.com (2603:10b6:805:e9::17) MIME-Version: 1.0 X-MS-Exchange-MessageSentRepresentingType: 1 Received: from agrodzovsky-All-Series.hitronhub.home (2607:fea8:3edf:49b0:7212:f93a:73b0:8f23) by YT1PR01CA0142.CANPRD01.PROD.OUTLOOK.COM (2603:10b6:b01:2f::21) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4065.25 via Frontend Transport; Wed, 28 Apr 2021 15:13:08 +0000 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: b3c4d4a1-8837-48fa-e49b-08d90a582226 X-MS-TrafficTypeDiagnostic: SN6PR12MB2749: X-MS-Exchange-Transport-Forked: True X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:296; X-MS-Exchange-SenderADCheck: 1 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: tle/rZQEq3prdKuWJjJZ4LBP8qlTX98sIBzJkr6lnEuWlK8x+TEDTnvBjJrcDkeHJEG+WCG9oW4BKsAq8hU1NhVkeImP7cp0wysfyf49zfBs2UmoY+yQM0x9+agb3loVFKqkkb65jv86yKe2BhNc1D6ZKYzN2IbfCf3VXLQEJ9dVm8iN1CTV9VFATSkyzpmZSYD11WiVp66cDc91GDmxZ5l85n1J2nNgPJTEVAUP7mBsYMZ2KwUYi6UpGqESdwXaA3Y9EAWlo4Mak1o2x7zu42HtVlJzLQn0b9DqqxkpBC5JSIywr8JastaiTLssrQjWgRlD4arpugqiLksoMSIdSY/KvvCaXvDwR9SM9t/69fffxmky6ePzh0u+0DW3Kg6GvbSJv8fn0qowQuLxLu7t0w60crty0ZDddPbCAwJ+C0LJB5wa0JeerEOi8QPvYDBLci9wUwe/xGtBkFClDVV29bGTZhDc43f3g4VyTpoZNv9e0v3LPMc5sWwUDHr1YZVdi0FB3nRpWm9Asr7l4w18GvKLOWZXgOY/4gY9IHE0EmVAl5+p4H7A9F8PSEMDGCE25Kc9a0ZRc91S8yl80hZgLalIWNu8Mdd9Rb26t6dcP/4= X-Forefront-Antispam-Report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:SN6PR12MB4623.namprd12.prod.outlook.com; PTR:; CAT:NONE; SFS:(4636009)(376002)(346002)(39860400002)(136003)(396003)(366004)(16526019)(4744005)(6666004)(6512007)(2616005)(1076003)(2906002)(6636002)(66556008)(186003)(86362001)(5660300002)(8936002)(83380400001)(6486002)(478600001)(44832011)(38100700002)(66946007)(52116002)(6506007)(66476007)(8676002)(4326008)(36756003)(316002); DIR:OUT; SFP:1101; X-MS-Exchange-AntiSpam-MessageData: bre9UeMHsFArHqmsYkQ2V2xj6mWir6VV+c8WOk5iwtdxORLthg3oN+EbJyoVjJtQB9xbErtJcoPJqA9MEaAz4jcVg5a0feqTqdb732Q5tzEWe/5zPkFd7PAPlu893pzvOTeLrNekdyfNGAo9GM2HIrISve1+VZgRROyKAqQ64XdYrP04uYMTf9mFGgpcZOt3m7vgVodqGley8nGlvTmAKNFsVCJXVrQiuB9BAzKvgxLWFrBBEDg54eRDjbJeMOVqwac6i+knVT6IQ53jqJsYUCCXPyS0S/6DYo0wZ0iKMQ24pa+LaQdYsfqVUr/xs+SGi5MwieXd0cB/dhBuGH8acG4TKyp7Sej1u17XFWCAmgzdbxde+lxv2XEp0Ad9+lWMmAtfSKWtRJEqBMMRZdBVq3T7rl0mGN89q/xY9qV9pcpXr6TPNxlFl2i9c+7Fwh9HGlFCy0r8lGTkpSQSlrUocuwHEH/0v+ITmJ0IhioKzE5ILB5xR760piDOBRQSTFQmiz0M1oLhtTKBJYbNL7VKN6hsLE2JDZ3OHbgvj9wXvYxIsbUn6o4s1bv/k+sfx4JQz1zEilxd64Yp1++ZhkK/8d9u7bsR8Rxp/SZ5/UzTUlwfnKCyf8z50rpIkgEBWX8qP/f5P9DkMEBhKjrTr0kLr09ADuuLlzpxmFqI338FdNQIhkau/RV27tKfmAW1SZSGV4X0QioEjXTnWGwbhQdlb3LIYd10cgLQw3CiKrqdxfoNfUwOktDbhx3Js2jeKT0zkUnr6S4T6xupFQnxRNPm20ddSqITOkf65iDfqKiz469uJePg3RhtvE07hIYQFPzzr2XcsIT+LggXRsNQwQCMTqfux2qPMEMV3Vr6RDoC9CvYQ55yWZ+sOSB89sR8ylbcc14nse/3+BD7bk1fPEFrzuBfHoLh9RV/90dffjW8vAHN79so7f1Fy5khHlKfSaaIInBGJuEJFZ1NFCFF9NFXXZqf20LJ0iUOd2E0te/Jj0Esc78wak8xPufZWX/ZOKMQHB2bsdG/TRVpa5BcL4vbAjuT9qQzEvTyYDGkUJEAlXRKK7W8mz8xTxauwQklpyc2lXrC4jFFrcBn2orAjl3LbaukEBkzlgPQHbc5Fx6YR0aQxncRNwzdvjjjtP6oh9nP+1T6MswxXEOiQMQQ5dnZnhESIFSl9jJwAbSBdkY5Gt8VK8V/L3V+oxL+Ufz+sEk8Lx75LZnxqvYgunKq0ALL9rJ3aThB8JaOBItSmIJNZ3qtKAByq3VUcIZfBYdYH945JK+lRTejQlVdLAIvmeQSYUxHmFMcPcPmULzYsDvzHh6IEA1sVSupeiyfdZ3I5jib2vYdcfeOQPJO+NGzH81otz3KBXwnkdg/f3AgP2degxGJpZTe8JZcUfi28hJV7Ckv X-OriginatorOrg: amd.com X-MS-Exchange-CrossTenant-Network-Message-Id: b3c4d4a1-8837-48fa-e49b-08d90a582226 X-MS-Exchange-CrossTenant-AuthSource: SN6PR12MB4623.namprd12.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 28 Apr 2021 15:13:09.7878 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: 8GmPxwRIL5XN+T/ADHVF2TfHK6SM+OggFj5Y1xyVkeJ43ZjaRpJOChzpSRK/3J48qEtuR0qZYhZQogpryHxPEA== X-MS-Exchange-Transport-CrossTenantHeadersStamped: SN6PR12MB2749 X-BeenThere: dri-devel@lists.freedesktop.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Direct Rendering Infrastructure - Development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: gregkh@linuxfoundation.org, Felix.Kuehling@amd.com, helgaas@kernel.org, Alexander.Deucher@amd.com Errors-To: dri-devel-bounces@lists.freedesktop.org Sender: "dri-devel" To allow scoping DRM IOCTLs with drm_dev_enter/exit. Signed-off-by: Andrey Grodzovsky --- drivers/gpu/drm/amd/amdgpu/amdgpu_drv.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_drv.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_drv.c index 8a19b8dd02ee..d0f34f230ef3 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_drv.c +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_drv.c @@ -1579,7 +1579,7 @@ static const struct drm_driver amdgpu_kms_driver = { DRIVER_ATOMIC | DRIVER_GEM | DRIVER_RENDER | DRIVER_MODESET | DRIVER_SYNCOBJ | - DRIVER_SYNCOBJ_TIMELINE, + DRIVER_SYNCOBJ_TIMELINE | DRIVER_HOTUNPLUG_SUPPORT, .open = amdgpu_driver_open_kms, .postclose = amdgpu_driver_postclose_kms, .lastclose = amdgpu_driver_lastclose_kms, From patchwork Wed Apr 28 15:12:02 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrey Grodzovsky X-Patchwork-Id: 12229279 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.5 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,MSGID_FROM_MTA_HEADER,SPF_HELO_NONE,SPF_PASS, USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id A9327C43462 for ; Wed, 28 Apr 2021 15:14:25 +0000 (UTC) Received: from gabe.freedesktop.org (gabe.freedesktop.org [131.252.210.177]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 4D3D961434 for ; Wed, 28 Apr 2021 15:14:23 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 4D3D961434 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=amd.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=dri-devel-bounces@lists.freedesktop.org Received: from gabe.freedesktop.org (localhost [127.0.0.1]) by gabe.freedesktop.org (Postfix) with ESMTP id BE7626EB97; Wed, 28 Apr 2021 15:14:21 +0000 (UTC) Received: from NAM11-CO1-obe.outbound.protection.outlook.com (mail-co1nam11on2045.outbound.protection.outlook.com [40.107.220.45]) by gabe.freedesktop.org (Postfix) with ESMTPS id 311686EB68; Wed, 28 Apr 2021 15:13:13 +0000 (UTC) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=XcazOf2KAz6f3OR5paLcKXq0To1s5nAWLaHPHQXzJteFYnUoh2CQjefoTyRfJqSg29Zpx3fqGcEUsVPCyDA03J4HuSyfL+Oz8ecUZRHBhmgK2wIYprDRA4O3oWJ+8TH/BYH3kRlmurmUM/fVZSEOJVA0c27Na7PkbXGhDFfv4R1f77nDeDwbvFby1Q2esaMI7yV2Dsw+yM40OHv3s5jCuAEo1uJCnpWpvj3Xad6mAezleD4BIy8Oe0jGKaeYyIuDXx8zeeSzXfRDXfDTgSVxnAVL7cAUKNyNbRwFwfo+uHBtYEgC0eJVkL2VzNj7I7zGZI6iW+8+LMIwVzyjRQPJSQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=SzivJfSoUBAQWvGZdfqbypvxwjVVB7vUjRR8QnYNApE=; b=mPP1umtYOLRA7z/WXFYX8S5esESnhSAvFK3hkEcfTlK3sSlg9j0hafrnHYvDXTB4hP00ohsvql9ttxO1nwEquYCQDM+Dv2hLZ4xKIu3sGyUbfVFoyWNwLWEJZqk3bzs/Tpu22AaFQmnWrDa8KPCdexJkDALwnSoVNdGZE1N8HOnagx5bxovLtUtq+O2c/p/fW49EV/kEiy8uMNB2JFBAF3qWuqJ1/SizV/2gRyZaSByC7YmARUwrnKGi1oT69dWGkGjPPmTm8yQT0+1D0ngJa3WaaVv+sMPAsDIPckXZuB69czaYHaERfEK7+hDagLjSBT9+tH5ivHBx3mpNSDIdBA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=amd.com; dmarc=pass action=none header.from=amd.com; dkim=pass header.d=amd.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=SzivJfSoUBAQWvGZdfqbypvxwjVVB7vUjRR8QnYNApE=; b=aAL47qo7B/r7tjOl/7Xt5c66QfjwcI8pgEpL0ygdSCtRxShJq0mo0tozWeUr53WZuelJC12QUZSWH8i0MHj1x9Qb/UrNqJC11+0tpdyM7wQR011KJUbQXiK9sAtenA10JWHT1E74xbpzZjfg86jQ7YymSjClkjg0/uQVngPzZfA= Authentication-Results: lists.freedesktop.org; dkim=none (message not signed) header.d=none; lists.freedesktop.org; dmarc=none action=none header.from=amd.com; Received: from SN6PR12MB4623.namprd12.prod.outlook.com (2603:10b6:805:e9::17) by SN6PR12MB2749.namprd12.prod.outlook.com (2603:10b6:805:6d::18) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4065.22; Wed, 28 Apr 2021 15:13:11 +0000 Received: from SN6PR12MB4623.namprd12.prod.outlook.com ([fe80::ad51:8c49:b171:856c]) by SN6PR12MB4623.namprd12.prod.outlook.com ([fe80::ad51:8c49:b171:856c%7]) with mapi id 15.20.4065.026; Wed, 28 Apr 2021 15:13:11 +0000 From: Andrey Grodzovsky To: dri-devel@lists.freedesktop.org, amd-gfx@lists.freedesktop.org, linux-pci@vger.kernel.org, ckoenig.leichtzumerken@gmail.com, daniel.vetter@ffwll.ch, Harry.Wentland@amd.com Subject: [PATCH v5 22/27] drm/amd/display: Scope all DM queued work with drm_dev_enter/exit Date: Wed, 28 Apr 2021 11:12:02 -0400 Message-Id: <20210428151207.1212258-23-andrey.grodzovsky@amd.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210428151207.1212258-1-andrey.grodzovsky@amd.com> References: <20210428151207.1212258-1-andrey.grodzovsky@amd.com> X-Originating-IP: [2607:fea8:3edf:49b0:7212:f93a:73b0:8f23] X-ClientProxiedBy: YT1PR01CA0142.CANPRD01.PROD.OUTLOOK.COM (2603:10b6:b01:2f::21) To SN6PR12MB4623.namprd12.prod.outlook.com (2603:10b6:805:e9::17) MIME-Version: 1.0 X-MS-Exchange-MessageSentRepresentingType: 1 Received: from agrodzovsky-All-Series.hitronhub.home (2607:fea8:3edf:49b0:7212:f93a:73b0:8f23) by YT1PR01CA0142.CANPRD01.PROD.OUTLOOK.COM (2603:10b6:b01:2f::21) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4065.25 via Frontend Transport; Wed, 28 Apr 2021 15:13:10 +0000 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 5b85e3b6-4552-4279-9c70-08d90a58234c X-MS-TrafficTypeDiagnostic: SN6PR12MB2749: X-MS-Exchange-Transport-Forked: True X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:68; X-MS-Exchange-SenderADCheck: 1 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: x1RKhK11Dr4WlnQRCmPDLr4VQufiF6dB+8G0kX4G2if+1tYdDihzDYaD6sIrzizs5CuTClgupHhMt8mxKgYMd+x9/M4Xp4N1ky4tQhLWnu2EFR0hNUP/GfmZdibjCkqCeMtAaujU7c2sqvzb4afXhCX9OWlDPkZau/qDd4sizZ7wlJe+6iUXomAEOumXY98hqt9kfv9PkOdGVty6JWYe14bx3+HD9OkVPOUxORpdIOTWZEfEnoS5ODdbi7V8nrd247gA/4i9vgyhxp1ETsSkhoIfYrn/ftw4j1C6Vw3eVq8ifSm0IQ1AUaF+f4KwMGZc+osMQzDGtr6A8DCsGyPSWlUo9kUXQnq6hz8DxDStPZ2bantXUIFX7BO31eaiuRv1x6u2JoqBR/tGixUv6oy6suGAFW1Cv4lKp24uo0NXaCAj2q+42SH/WMk64DH2PdiQTo41pulQQJC3IJGg4KaN3tI7/72q4IYEKXvKBKGvJuMJT53BfJV0S99Jsg/23Ujtx+6tBDaEq9XGzfxO6LkSmRjUxdzR3KnH2B/73X0jj3VqjDA4BqnqddnqCaIhF21YNcIJZ97QldnFNQkS/u53Hzi2e4s0aZoclzljeT97PY0= X-Forefront-Antispam-Report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:SN6PR12MB4623.namprd12.prod.outlook.com; PTR:; CAT:NONE; SFS:(4636009)(376002)(346002)(39860400002)(136003)(396003)(366004)(16526019)(6666004)(6512007)(2616005)(1076003)(2906002)(6636002)(66556008)(186003)(86362001)(5660300002)(8936002)(83380400001)(6486002)(478600001)(44832011)(38100700002)(66946007)(52116002)(6506007)(66476007)(8676002)(4326008)(36756003)(316002); DIR:OUT; SFP:1101; X-MS-Exchange-AntiSpam-MessageData: 2ACJ5KeH3uyXWQ4R1ud9fJbhGzafVSlyVvFWa4EnKV1zMKlFCW9cxicz45oaf8SvpMsJHtRFdwln3dFFovoAFopiLgt3j0crshQw/ffTs1Mt7lwAhGNRquQyE4Ql472DYqSa9VyZnDFZO4AJLY/m1HlDYfAn67gtzLSRkz6ABee1llXJIK/5UBaUIz37Y6Ox9sZNv9DZqV2DBZ8mN3FGRhFS0mORuUTSTZi8o9G6rAXxwzckGsEVBaAlGzqWE7yLeGAvFvDWHJ8pWXlEe5mHRpQTgf/BppTu3wOgX4LZ4y2UP4Pa1YsLK4hXRTfNKO5hSXI1C4DrU/ZidgTG1JkB4E12zGpZIiqtiWSOPvl7+C/mwnklnU+buQPtw4RWtptIsp0yN++HOSxEOIfq89TyvwG50zoB175l6CpROtxomgsPUwy7P0y9olC/AP1GyGfHy4atn78e8DUdnQwRzXR+1vQtd/DdedOxfZRIiMS6efFCDLubLgSfRnCnqgLnvOYoARfES4UHzuoynf8oP07QIxb+m2YetDwhUxeh93ijClnFZshFvLnb4Id0e3qSOsYeDZb27Z9pEcFuVykZryh+F0j2g4UsUSUc6DQGCLYN8g/AtbyVOYV/IwSp6KwuaiJXDh9BOC/oRVhawd5gIVQVjzcDwQhjm55zV8DG232iAVdWbj+MCpsk/vluf0GqBxYtsij/FsUMxt50HE+BP4QYJ1XHp6kcpxImebPnXKrHCchYXbKBMoh3H0B2QfIgMZhnF6LaWS4tKvmdFJ2yHdSRvEsPV/JnV2exOf9HNt4guFN/P/ImRqieQCb47OLug+ZbdD2bCZRl+oL0oNF45hpjJ4dITlpZ2oQXPTxFkxGvK043v+u0otSK0IZU4H/ZIBOxunPiD/jzGB5XzWwGC7+K7r9NoucM6+fjcu4oCZ3frUe4FPInGBg0n+oEIyJ5q93/TfUfC7CE79iHNFWIa9uPU15+jHp5luWz55Nep39hF5QAcz+MocQDX42k/lsO/LSA227FpSHnFZFKO+NPd7OjmhpLmLqn2Kc41V7Q1oMAoUoYxBPb1o9GZg2sGRFJcIFFG8+mXmhQDc+VrJ5SwYcFyRGccuZjhOpVFC3Ouf/D97uX5YuM0gAUpZesVBycSLIZmqDpXZ70elEd9rhoJVERmJ3yo/YwvF0La8oCKLHEVHTpJHZZjThMNetBOA36mCdl95XZpZeKAd4kHVbgTvbKbk2+C+UkMuLytJWsllPS3T7cLgijGt53wWXSJCG4hE7CrBmxju6SK0U+TYJC3HOZb9QfcpWmBGs8YF+X9Epy6ihIhrdBkeqasauFSxiBqYyNtN/QEf+kG+lOwpJozC/xkYs87N3InnnCc34YZfWD/MyeFWzGMe5lDMJJ6PGymlDv X-OriginatorOrg: amd.com X-MS-Exchange-CrossTenant-Network-Message-Id: 5b85e3b6-4552-4279-9c70-08d90a58234c X-MS-Exchange-CrossTenant-AuthSource: SN6PR12MB4623.namprd12.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 28 Apr 2021 15:13:11.6638 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: PSTeXZTCwCPDbHYCOt9686yHGB60m7hCxTbxPgAcLinNusjfJHXFati8aP7dX0jBirwQrzv5QhiIi8RxYaPSnQ== X-MS-Exchange-Transport-CrossTenantHeadersStamped: SN6PR12MB2749 X-BeenThere: dri-devel@lists.freedesktop.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Direct Rendering Infrastructure - Development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: gregkh@linuxfoundation.org, Felix.Kuehling@amd.com, helgaas@kernel.org, Alexander.Deucher@amd.com Errors-To: dri-devel-bounces@lists.freedesktop.org Sender: "dri-devel" To allow completion and further block of HW accesses post device PCI remove. Signed-off-by: Andrey Grodzovsky --- .../amd/display/amdgpu_dm/amdgpu_dm_hdcp.c | 124 +++++++++++------- .../drm/amd/display/amdgpu_dm/amdgpu_dm_irq.c | 24 +++- 2 files changed, 98 insertions(+), 50 deletions(-) diff --git a/drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm_hdcp.c b/drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm_hdcp.c index 0cdbfcd475ec..81ea5a1ea46b 100644 --- a/drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm_hdcp.c +++ b/drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm_hdcp.c @@ -28,6 +28,7 @@ #include "amdgpu_dm.h" #include "dm_helpers.h" #include +#include #include "hdcp_psp.h" /* @@ -260,20 +261,27 @@ void hdcp_handle_cpirq(struct hdcp_workqueue *hdcp_work, unsigned int link_index static void event_callback(struct work_struct *work) { struct hdcp_workqueue *hdcp_work; + int idx; hdcp_work = container_of(to_delayed_work(work), struct hdcp_workqueue, callback_dwork); - mutex_lock(&hdcp_work->mutex); + if (drm_dev_enter(hdcp_work->aconnector->base.dev, &idx)) { - cancel_delayed_work(&hdcp_work->callback_dwork); + mutex_lock(&hdcp_work->mutex); - mod_hdcp_process_event(&hdcp_work->hdcp, MOD_HDCP_EVENT_CALLBACK, - &hdcp_work->output); + cancel_delayed_work(&hdcp_work->callback_dwork); + + mod_hdcp_process_event(&hdcp_work->hdcp, MOD_HDCP_EVENT_CALLBACK, + &hdcp_work->output); - process_output(hdcp_work); + process_output(hdcp_work); - mutex_unlock(&hdcp_work->mutex); + mutex_unlock(&hdcp_work->mutex); + + drm_dev_exit(idx); + + } } @@ -284,34 +292,41 @@ static void event_property_update(struct work_struct *work) struct amdgpu_dm_connector *aconnector = hdcp_work->aconnector; struct drm_device *dev = hdcp_work->aconnector->base.dev; long ret; + int idx; + + if (drm_dev_enter(dev, &idx)) { + + drm_modeset_lock(&dev->mode_config.connection_mutex, NULL); + mutex_lock(&hdcp_work->mutex); - drm_modeset_lock(&dev->mode_config.connection_mutex, NULL); - mutex_lock(&hdcp_work->mutex); + if (aconnector->base.state->commit) { + ret = wait_for_completion_interruptible_timeout(&aconnector->base.state->commit->hw_done, 10 * HZ); - if (aconnector->base.state->commit) { - ret = wait_for_completion_interruptible_timeout(&aconnector->base.state->commit->hw_done, 10 * HZ); + if (ret == 0) { + DRM_ERROR("HDCP state unknown! Setting it to DESIRED"); + hdcp_work->encryption_status = MOD_HDCP_ENCRYPTION_STATUS_HDCP_OFF; + } + } - if (ret == 0) { - DRM_ERROR("HDCP state unknown! Setting it to DESIRED"); - hdcp_work->encryption_status = MOD_HDCP_ENCRYPTION_STATUS_HDCP_OFF; + if (hdcp_work->encryption_status != MOD_HDCP_ENCRYPTION_STATUS_HDCP_OFF) { + if (aconnector->base.state->hdcp_content_type == DRM_MODE_HDCP_CONTENT_TYPE0 && + hdcp_work->encryption_status <= MOD_HDCP_ENCRYPTION_STATUS_HDCP2_TYPE0_ON) + drm_hdcp_update_content_protection(&aconnector->base, DRM_MODE_CONTENT_PROTECTION_ENABLED); + else if (aconnector->base.state->hdcp_content_type == DRM_MODE_HDCP_CONTENT_TYPE1 && + hdcp_work->encryption_status == MOD_HDCP_ENCRYPTION_STATUS_HDCP2_TYPE1_ON) + drm_hdcp_update_content_protection(&aconnector->base, DRM_MODE_CONTENT_PROTECTION_ENABLED); + } else { + drm_hdcp_update_content_protection(&aconnector->base, DRM_MODE_CONTENT_PROTECTION_DESIRED); } - } - if (hdcp_work->encryption_status != MOD_HDCP_ENCRYPTION_STATUS_HDCP_OFF) { - if (aconnector->base.state->hdcp_content_type == DRM_MODE_HDCP_CONTENT_TYPE0 && - hdcp_work->encryption_status <= MOD_HDCP_ENCRYPTION_STATUS_HDCP2_TYPE0_ON) - drm_hdcp_update_content_protection(&aconnector->base, DRM_MODE_CONTENT_PROTECTION_ENABLED); - else if (aconnector->base.state->hdcp_content_type == DRM_MODE_HDCP_CONTENT_TYPE1 && - hdcp_work->encryption_status == MOD_HDCP_ENCRYPTION_STATUS_HDCP2_TYPE1_ON) - drm_hdcp_update_content_protection(&aconnector->base, DRM_MODE_CONTENT_PROTECTION_ENABLED); - } else { - drm_hdcp_update_content_protection(&aconnector->base, DRM_MODE_CONTENT_PROTECTION_DESIRED); - } + mutex_unlock(&hdcp_work->mutex); + drm_modeset_unlock(&dev->mode_config.connection_mutex); + + drm_dev_exit(idx); - mutex_unlock(&hdcp_work->mutex); - drm_modeset_unlock(&dev->mode_config.connection_mutex); + } } static void event_property_validate(struct work_struct *work) @@ -320,58 +335,77 @@ static void event_property_validate(struct work_struct *work) container_of(to_delayed_work(work), struct hdcp_workqueue, property_validate_dwork); struct mod_hdcp_display_query query; struct amdgpu_dm_connector *aconnector = hdcp_work->aconnector; + int idx; if (!aconnector) return; - mutex_lock(&hdcp_work->mutex); + if (drm_dev_enter(hdcp_work->aconnector->base.dev, &idx)) { - query.encryption_status = MOD_HDCP_ENCRYPTION_STATUS_HDCP_OFF; - mod_hdcp_query_display(&hdcp_work->hdcp, aconnector->base.index, &query); + mutex_lock(&hdcp_work->mutex); - if (query.encryption_status != hdcp_work->encryption_status) { - hdcp_work->encryption_status = query.encryption_status; - schedule_work(&hdcp_work->property_update_work); - } + query.encryption_status = MOD_HDCP_ENCRYPTION_STATUS_HDCP_OFF; + mod_hdcp_query_display(&hdcp_work->hdcp, aconnector->base.index, &query); + + if (query.encryption_status != hdcp_work->encryption_status) { + hdcp_work->encryption_status = query.encryption_status; + schedule_work(&hdcp_work->property_update_work); + } + + mutex_unlock(&hdcp_work->mutex); - mutex_unlock(&hdcp_work->mutex); + drm_dev_exit(idx); + + } } static void event_watchdog_timer(struct work_struct *work) { struct hdcp_workqueue *hdcp_work; + int idx; hdcp_work = container_of(to_delayed_work(work), struct hdcp_workqueue, watchdog_timer_dwork); - mutex_lock(&hdcp_work->mutex); + if (drm_dev_enter(hdcp_work->aconnector->base.dev, &idx)) { - cancel_delayed_work(&hdcp_work->watchdog_timer_dwork); + mutex_lock(&hdcp_work->mutex); - mod_hdcp_process_event(&hdcp_work->hdcp, - MOD_HDCP_EVENT_WATCHDOG_TIMEOUT, - &hdcp_work->output); + cancel_delayed_work(&hdcp_work->watchdog_timer_dwork); - process_output(hdcp_work); + mod_hdcp_process_event(&hdcp_work->hdcp, + MOD_HDCP_EVENT_WATCHDOG_TIMEOUT, + &hdcp_work->output); - mutex_unlock(&hdcp_work->mutex); + process_output(hdcp_work); + + mutex_unlock(&hdcp_work->mutex); + + drm_dev_exit(idx); + } } static void event_cpirq(struct work_struct *work) { struct hdcp_workqueue *hdcp_work; + int idx; hdcp_work = container_of(work, struct hdcp_workqueue, cpirq_work); - mutex_lock(&hdcp_work->mutex); + if (drm_dev_enter(hdcp_work->aconnector->base.dev, &idx)) { - mod_hdcp_process_event(&hdcp_work->hdcp, MOD_HDCP_EVENT_CPIRQ, &hdcp_work->output); + mutex_lock(&hdcp_work->mutex); - process_output(hdcp_work); + mod_hdcp_process_event(&hdcp_work->hdcp, MOD_HDCP_EVENT_CPIRQ, &hdcp_work->output); - mutex_unlock(&hdcp_work->mutex); + process_output(hdcp_work); + + mutex_unlock(&hdcp_work->mutex); + + drm_dev_exit(idx); + } } diff --git a/drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm_irq.c b/drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm_irq.c index e0000c180ed1..d8ee552d373e 100644 --- a/drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm_irq.c +++ b/drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm_irq.c @@ -30,6 +30,8 @@ #include "amdgpu_dm.h" #include "amdgpu_dm_irq.h" +#include + /** * DOC: overview * @@ -115,15 +117,27 @@ static void dm_irq_work_func(struct work_struct *work) container_of(work, struct irq_list_head, work); struct list_head *handler_list = &irq_list_head->head; struct amdgpu_dm_irq_handler_data *handler_data; + int idx; + + + handler_data = list_first_entry_or_null(handler_list, struct amdgpu_dm_irq_handler_data, list); + + if (!handler_data) + return; - list_for_each_entry(handler_data, handler_list, list) { - DRM_DEBUG_KMS("DM_IRQ: work_func: for dal_src=%d\n", + if (drm_dev_enter(handler_data->dm->ddev, &idx)) { + + list_for_each_entry(handler_data, handler_list, list) { + DRM_DEBUG_KMS("DM_IRQ: work_func: for dal_src=%d\n", + handler_data->irq_source); + + DRM_DEBUG_KMS("DM_IRQ: schedule_work: for dal_src=%d\n", handler_data->irq_source); - DRM_DEBUG_KMS("DM_IRQ: schedule_work: for dal_src=%d\n", - handler_data->irq_source); + handler_data->handler(handler_data->handler_arg); + } - handler_data->handler(handler_data->handler_arg); + drm_dev_exit(idx); } /* Call a DAL subcomponent which registered for interrupt notification From patchwork Wed Apr 28 15:12:03 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrey Grodzovsky X-Patchwork-Id: 12229269 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.5 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,MSGID_FROM_MTA_HEADER,SPF_HELO_NONE,SPF_PASS, USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id A87FFC43461 for ; Wed, 28 Apr 2021 15:14:14 +0000 (UTC) Received: from gabe.freedesktop.org (gabe.freedesktop.org [131.252.210.177]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 4DF1C61434 for ; Wed, 28 Apr 2021 15:14:14 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 4DF1C61434 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=amd.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=dri-devel-bounces@lists.freedesktop.org Received: from gabe.freedesktop.org (localhost [127.0.0.1]) by gabe.freedesktop.org (Postfix) with ESMTP id 7DA446EB90; Wed, 28 Apr 2021 15:14:12 +0000 (UTC) Received: from NAM11-CO1-obe.outbound.protection.outlook.com (mail-co1nam11on2043.outbound.protection.outlook.com [40.107.220.43]) by gabe.freedesktop.org (Postfix) with ESMTPS id 103E16EB71; Wed, 28 Apr 2021 15:13:18 +0000 (UTC) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=KVceJBU1pJr0BWbj8II2dVPbpVEkJJNF3gDvSPUg6b2ZL6BMX+MBuzpxezeEz0OJXc520n/PMsKnbhi6eEZTxyFmJcUSZTZFvO6H/vuJzMXNaVFjCBgccDBf6mmu/fOl946ko9QyspaoOJmRul+FHJlaDGht7AHtRgrfHK3INYwuZEHe+fQIj9buugc1fXGhwPO8UKYcRb09Dv0P10EauZDh2Fmth+HoXhsvPo5HhHXB4a0q6dpwZT9WudFKnhc6c5Vrck2LZdIl/j32U6KdXI8g15E+tkLsVx8P96YW5+gsSkfJw0YH/li0yYfRVmP9I/b6122i1YfZqsuZ7EXVug== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=J0zppKyDwSYEFkbc1PJUglI0RjqZZnmm5vJgjM8KX04=; b=HFMd43HiEaV43zCb2783lAgfTvbhAT4jEvi9enrn0UyOViADcaicshy5jl7cyp1HV56f3iWrawWMCGrIVNVyHlXgfyNpCS7Fhx7nTTeP2XpWgIw9SVdNTJf+74UksoRjXlA+O2JvYFVDG1XViszK6EffLADgHnJLyFgcAyyhdYj0hcC+95WU3YXLfx55YT4mC37zYSe9tihh0AFL5rM0w9QCT2QIx2wapQ2EHLYpXfuWymAXWhgB58g6YnS+B2feUJuDf3NF4Ulzvtzyj77UbfTw1D1HNQPvpFYXgTPX46uyMx9SG8Uf4XLb7O4Qhja/wuaLlvSuqVN3GEJ0EJLbFA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=amd.com; dmarc=pass action=none header.from=amd.com; dkim=pass header.d=amd.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=J0zppKyDwSYEFkbc1PJUglI0RjqZZnmm5vJgjM8KX04=; b=zIM56gocFvrRNYy/uxsodsyjVJTpKh3xa69ISL8aGXTmaAayPaBwFIaByzrj5iOMANnINRrBN3RpBFq5C6jyQbze+AKuezvqLV/DXYP4GLbXRpEvlVYsmiYFUYidvt6sA4gWMgf0BWwTXogvqAm7AMMFgVECk9NgfSFVf7rjdu0= Authentication-Results: lists.freedesktop.org; dkim=none (message not signed) header.d=none; lists.freedesktop.org; dmarc=none action=none header.from=amd.com; Received: from SN6PR12MB4623.namprd12.prod.outlook.com (2603:10b6:805:e9::17) by SN6PR12MB2749.namprd12.prod.outlook.com (2603:10b6:805:6d::18) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4065.22; Wed, 28 Apr 2021 15:13:13 +0000 Received: from SN6PR12MB4623.namprd12.prod.outlook.com ([fe80::ad51:8c49:b171:856c]) by SN6PR12MB4623.namprd12.prod.outlook.com ([fe80::ad51:8c49:b171:856c%7]) with mapi id 15.20.4065.026; Wed, 28 Apr 2021 15:13:13 +0000 From: Andrey Grodzovsky To: dri-devel@lists.freedesktop.org, amd-gfx@lists.freedesktop.org, linux-pci@vger.kernel.org, ckoenig.leichtzumerken@gmail.com, daniel.vetter@ffwll.ch, Harry.Wentland@amd.com Subject: [PATCH v5 23/27] drm/amd/powerplay: Scope all PM queued work with drm_dev_enter/exit Date: Wed, 28 Apr 2021 11:12:03 -0400 Message-Id: <20210428151207.1212258-24-andrey.grodzovsky@amd.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210428151207.1212258-1-andrey.grodzovsky@amd.com> References: <20210428151207.1212258-1-andrey.grodzovsky@amd.com> X-Originating-IP: [2607:fea8:3edf:49b0:7212:f93a:73b0:8f23] X-ClientProxiedBy: YT1PR01CA0142.CANPRD01.PROD.OUTLOOK.COM (2603:10b6:b01:2f::21) To SN6PR12MB4623.namprd12.prod.outlook.com (2603:10b6:805:e9::17) MIME-Version: 1.0 X-MS-Exchange-MessageSentRepresentingType: 1 Received: from agrodzovsky-All-Series.hitronhub.home (2607:fea8:3edf:49b0:7212:f93a:73b0:8f23) by YT1PR01CA0142.CANPRD01.PROD.OUTLOOK.COM (2603:10b6:b01:2f::21) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4065.25 via Frontend Transport; Wed, 28 Apr 2021 15:13:12 +0000 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 625ebaae-f470-477a-a437-08d90a582457 X-MS-TrafficTypeDiagnostic: SN6PR12MB2749: X-MS-Exchange-Transport-Forked: True X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:1265; X-MS-Exchange-SenderADCheck: 1 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: h3OTczl7HeKmjtk3gdPLijTFYJGS7KM5qZovNp7hdgXvxYGECOi474/WNGbQidyhY3OWRqTxZK00REOtrcuDP8ccD6k9FK4rMS3FgrxU297zMAQ0huSKoNEL1oLoRCVE48ybe5n6SWyVMzBMF+Wsv1sdFGOwoYcptX+UFZxyIdYyRcjMG0ENFUkhR0UoZ24wYfGaTqhsJKg9NQpoMoD2P7uGd56zsh2aJcHAjoqFt+adNoiNNiqGgbgEDMwHNSUTo6jpynWyl3q4o4XKib1yi92r5U4oZ5mZc34KWHEvcGFBewju4Zpf0Qdf4gv5yzkvNyypPcq2SflKhxZT5SBtlGWsIcuKsXZaCa7i4lUajT0gKupIMu4tjvv/WKyPUXw/2zW9kznwBIJpX09nEx/PGHBldFlxdqasJ8l4EHBOCEC5w1edh7RfgxqinK/5KjX8xEbx4+8se6ea+dIY7PL0cIzABTL4LnK4zWDt2xb3/sFN+mkowJcAtiTBOWU3lMoaFUbA/+lNkaba4B3k4MEjfiaLzmSTcfUydwVbNbz9o5SmD1f6cXKV7EKxzFuQ6HTfABHLZ1vWoJtI7cm0CjGFblkIySdpPGk2S+lCcE+Lw2E= X-Forefront-Antispam-Report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:SN6PR12MB4623.namprd12.prod.outlook.com; PTR:; CAT:NONE; SFS:(4636009)(376002)(346002)(39860400002)(136003)(396003)(366004)(16526019)(6666004)(6512007)(2616005)(1076003)(2906002)(6636002)(66556008)(186003)(86362001)(5660300002)(8936002)(83380400001)(6486002)(478600001)(44832011)(38100700002)(66946007)(52116002)(6506007)(66476007)(8676002)(4326008)(36756003)(316002); DIR:OUT; SFP:1101; X-MS-Exchange-AntiSpam-MessageData: wxa3pv5Ogfbz29sWVuplnuwi88/2VCtgjXAD0evExDiDATlZDHYokxbewOKXPYw8CVihWHUE+accSebIY8m89h0zKPdYo2JsQZnBz+vvb/LrwJuw3v8BUobGLfGukKf5RzppRGAIMhNCKt6TJrmZks//+IdTkN3uN6pjpO2nuNB/qWC8i7t2obGRg1Br7o6qVadbiAt0aX5rp4YJK7AIiMSaiHYLAV47iGdTHwdD94myJ7PtJxZpPnkwTJvzerKFiJrKEf/gm9MTfId8id9bnK8QRiMt1BMY2bxV5PVBYTBcwtUjlADqrmZnDZ+NlEEn2MA0c9sq+0NeiJN/VnMTAV7kEzx4hg9ccxYf4nWM/YPyOPZTjLulocG6CADTz1Nbs0HXPkioJiToDoAH71Ynw2ViaabRfEVIRdOvC99RRqIW3c6I49cmwp08Rpmmk+xbwQlEDjgSHcKQzbEK8aBz8kM4FfkLnLZeZSWlofJB8VtqsqapQ/GY+pMv1dwQqNY8wbdr7f/0h5Z6hI9Y9kFtdycaheHlGZFlXe+fqp6ompIIfOJwOby2+WpLz47hq7pSRV3ayrR/0tRhaGSpefBRnYqipssuHZRWlx7EY4Orb5K/32SRaa2laSDPx+uoPEzyL5/flSask6/XZTPXSO0qo+sqaGXzd51QC5dsPKc2KPfRcJYnOinFBoavPzLni4FBZHXBMJYW26WNQR62lpCfo42jVK3EMkBSLFYWUYyrLcQSGycBKe/cWxdcGqQyZ8IPP5zuK0UWpJ9AsTjS3X/kDjrHKv2/TYeovzz2nUQl5hPiFOPHSHFnMNHAgf39njIvPK6l0uzUZ9qMBFvhL5K0yjrn7VRngX4Nf29nCfQI3dYZLqtIqgiw1H1WWKkXhe35zJ5l8tSVaaBUmmvpDujMDaAxqxzTYywkg4phC3bX3utpdyU2wd0psi5JTWudgrvZ7BqleFpVxtFqaSZxXADd323X60A5TD2cB+YQafq8/FfdzFWegfvtMWznFVNeH1tk3K7GnJYPRmA8cYkfRdvT9mej64vN4FDSJfcDkLpm3Wtcm8Kp1P8woASzpfOiGIhTys2hYec5ICQ1YHAaYQ8UQydSqGpsjO9XB6BZzPIFJq3wBNYmZoLczCa7O2CRMrO1dVFycT1X8ME1dt3Yx5kEp1wzqHLr2stkycM6936572eiYHRmUkGWVc72MbRluJNPI4i8SbOeaYZA+5Sc0A8P7kExlYvP2YcF9o2ywKPA13Lw9wuIMMnWTyEaHWOFKWrWViaS4qpJ6q3Mbkxr7RM7C4JhF2g6On0jfQK15LQqvkz060LvM6TsSLzsMA0jJMzTEK2XltCjxK57MPp9fP2UPAWkodVc2cBTRM4EbY6gn73GQBUXLwfKughKSxwL53PM X-OriginatorOrg: amd.com X-MS-Exchange-CrossTenant-Network-Message-Id: 625ebaae-f470-477a-a437-08d90a582457 X-MS-Exchange-CrossTenant-AuthSource: SN6PR12MB4623.namprd12.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 28 Apr 2021 15:13:13.3778 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: J52UHNr5QWR1FHoWO7zP32aInnUr17tBEDAUAItPgNWKsisIoWbJ5zoEBYc4xFFm1GnI7TMyrcYOLKl6Vh/uxw== X-MS-Exchange-Transport-CrossTenantHeadersStamped: SN6PR12MB2749 X-BeenThere: dri-devel@lists.freedesktop.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Direct Rendering Infrastructure - Development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: gregkh@linuxfoundation.org, Felix.Kuehling@amd.com, helgaas@kernel.org, Alexander.Deucher@amd.com Errors-To: dri-devel-bounces@lists.freedesktop.org Sender: "dri-devel" To allow completion and further block of HW accesses post device PCI remove. Signed-off-by: Andrey Grodzovsky --- drivers/gpu/drm/amd/pm/amdgpu_dpm.c | 44 +++++++++++++---------- drivers/gpu/drm/amd/pm/swsmu/amdgpu_smu.c | 26 +++++++++++--- 2 files changed, 47 insertions(+), 23 deletions(-) diff --git a/drivers/gpu/drm/amd/pm/amdgpu_dpm.c b/drivers/gpu/drm/amd/pm/amdgpu_dpm.c index 8fb12afe3c96..649e10d52d17 100644 --- a/drivers/gpu/drm/amd/pm/amdgpu_dpm.c +++ b/drivers/gpu/drm/amd/pm/amdgpu_dpm.c @@ -31,6 +31,7 @@ #include "amdgpu_display.h" #include "hwmgr.h" #include +#include #define WIDTH_4K 3840 @@ -1316,29 +1317,36 @@ void amdgpu_dpm_thermal_work_handler(struct work_struct *work) /* switch to the thermal state */ enum amd_pm_state_type dpm_state = POWER_STATE_TYPE_INTERNAL_THERMAL; int temp, size = sizeof(temp); + int idx; if (!adev->pm.dpm_enabled) return; - if (!amdgpu_dpm_read_sensor(adev, AMDGPU_PP_SENSOR_GPU_TEMP, - (void *)&temp, &size)) { - if (temp < adev->pm.dpm.thermal.min_temp) - /* switch back the user state */ - dpm_state = adev->pm.dpm.user_state; - } else { - if (adev->pm.dpm.thermal.high_to_low) - /* switch back the user state */ - dpm_state = adev->pm.dpm.user_state; - } - mutex_lock(&adev->pm.mutex); - if (dpm_state == POWER_STATE_TYPE_INTERNAL_THERMAL) - adev->pm.dpm.thermal_active = true; - else - adev->pm.dpm.thermal_active = false; - adev->pm.dpm.state = dpm_state; - mutex_unlock(&adev->pm.mutex); + if (drm_dev_enter(&adev->ddev, &idx)) { - amdgpu_pm_compute_clocks(adev); + if (!amdgpu_dpm_read_sensor(adev, AMDGPU_PP_SENSOR_GPU_TEMP, + (void *)&temp, &size)) { + if (temp < adev->pm.dpm.thermal.min_temp) + /* switch back the user state */ + dpm_state = adev->pm.dpm.user_state; + } else { + if (adev->pm.dpm.thermal.high_to_low) + /* switch back the user state */ + dpm_state = adev->pm.dpm.user_state; + } + mutex_lock(&adev->pm.mutex); + if (dpm_state == POWER_STATE_TYPE_INTERNAL_THERMAL) + adev->pm.dpm.thermal_active = true; + else + adev->pm.dpm.thermal_active = false; + adev->pm.dpm.state = dpm_state; + mutex_unlock(&adev->pm.mutex); + + amdgpu_pm_compute_clocks(adev); + + drm_dev_exit(idx); + + } } static struct amdgpu_ps *amdgpu_dpm_pick_power_state(struct amdgpu_device *adev, diff --git a/drivers/gpu/drm/amd/pm/swsmu/amdgpu_smu.c b/drivers/gpu/drm/amd/pm/swsmu/amdgpu_smu.c index d143ef1b460b..f034c8a5eb44 100644 --- a/drivers/gpu/drm/amd/pm/swsmu/amdgpu_smu.c +++ b/drivers/gpu/drm/amd/pm/swsmu/amdgpu_smu.c @@ -25,6 +25,8 @@ #include #include +#include + #include "amdgpu.h" #include "amdgpu_smu.h" #include "smu_internal.h" @@ -904,21 +906,35 @@ static void smu_throttling_logging_work_fn(struct work_struct *work) { struct smu_context *smu = container_of(work, struct smu_context, throttling_logging_work); + int idx; + + + if (drm_dev_enter(&smu->adev->ddev, &idx)) { + + smu_log_thermal_throttling(smu); - smu_log_thermal_throttling(smu); + drm_dev_exit(idx); + } } static void smu_interrupt_work_fn(struct work_struct *work) { struct smu_context *smu = container_of(work, struct smu_context, interrupt_work); + int idx; - mutex_lock(&smu->mutex); + if (drm_dev_enter(&smu->adev->ddev, &idx)) { - if (smu->ppt_funcs && smu->ppt_funcs->interrupt_work) - smu->ppt_funcs->interrupt_work(smu); + mutex_lock(&smu->mutex); - mutex_unlock(&smu->mutex); + if (smu->ppt_funcs && smu->ppt_funcs->interrupt_work) + smu->ppt_funcs->interrupt_work(smu); + + mutex_unlock(&smu->mutex); + + drm_dev_exit(idx); + + } } static int smu_sw_init(void *handle) From patchwork Wed Apr 28 15:12:04 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrey Grodzovsky X-Patchwork-Id: 12229271 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.5 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,MSGID_FROM_MTA_HEADER,SPF_HELO_NONE,SPF_PASS, USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id BC893C43460 for ; Wed, 28 Apr 2021 15:14:17 +0000 (UTC) Received: from gabe.freedesktop.org (gabe.freedesktop.org [131.252.210.177]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 6345A613F1 for ; Wed, 28 Apr 2021 15:14:17 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 6345A613F1 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=amd.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=dri-devel-bounces@lists.freedesktop.org Received: from gabe.freedesktop.org (localhost [127.0.0.1]) by gabe.freedesktop.org (Postfix) with ESMTP id C2F786EB8C; Wed, 28 Apr 2021 15:14:13 +0000 (UTC) Received: from NAM11-CO1-obe.outbound.protection.outlook.com (mail-co1nam11on2043.outbound.protection.outlook.com [40.107.220.43]) by gabe.freedesktop.org (Postfix) with ESMTPS id 97AF66EB73; Wed, 28 Apr 2021 15:13:18 +0000 (UTC) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=UrYwzIsa9fe1S/5HR6lZH+woSVOZYs9LrWryuc/02ONPYXbOUYzJ23geaZmiG8k/ECNxPteTqzNycvNKVK9TKdnSlnUrSaWXRftZhCmBGRg93BimbkMHqCQRks+8tNCWSLrLBuJLOVFDmcZT+HXFSMURAPvZpa/vUONeM0u4Ap5VFXtrm4D194YnhURh394v7SAQd8DOEa+iE1+Bug+u+CNl8UHWzhC5SE//i/GJUNxEqCbQvQjW6F08NLYrsPpVGyhvu4EqOm9vxdVkKjUfWjFdbVxZyrQaJqUlCIqEk8HR1piu9Duw7V9JanMYo75umgMh8siGN1lw7kEbp96S+g== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=xSHb3NhRY8x9Y0enrqhijZobX8cCyOSOvM4zW8iMOzI=; b=dRLXGZjheGqPYRDuE/Sr6er39oa/P1LGdOaYlt6evDdIVAkKoB8cfyn/0bKhPA3RH469Bi4JCMFU4Kq1EU3btnUq/DLzrLszVMVFo1/EpjRn/K0go+eA0SLSHJQEZ57H5YwOVGHaYovtXoQEBv4bc27CPkpxK7UQV/JqEWgzMMa216FnZyjHqThbCjEPun6aWwAQHNVcvUvl8VDN0477dqYa0G9rXbse8AFykRurXecpDlfGmsQ3AzBDTmduvDEKYExlsoJxwsnzQH6aIlW52pdYszMVvS3r1XY/o0QbTZ3A4Oq+ilGJJSx9elZONlQ96wbEtgs/2sjB+xvIK5adPA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=amd.com; dmarc=pass action=none header.from=amd.com; dkim=pass header.d=amd.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=xSHb3NhRY8x9Y0enrqhijZobX8cCyOSOvM4zW8iMOzI=; b=mWd+OsA6BGPPDwC3SUv9XPjKRsdfwKXa0C+RHRIEKXC+OInjQ8SqzP58KLlazgD3DscQ8ef3DXabdS0HYrEdIqJBICZK8xysQIH2FvtfrFdIvBtNU5Zv46DhDZryrfNgjYTHoHrX0SbvN8TGnFgdjG+9ZoTdd7gSpty+/4cu72I= Authentication-Results: lists.freedesktop.org; dkim=none (message not signed) header.d=none; lists.freedesktop.org; dmarc=none action=none header.from=amd.com; Received: from SN6PR12MB4623.namprd12.prod.outlook.com (2603:10b6:805:e9::17) by SN6PR12MB2749.namprd12.prod.outlook.com (2603:10b6:805:6d::18) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4065.22; Wed, 28 Apr 2021 15:13:17 +0000 Received: from SN6PR12MB4623.namprd12.prod.outlook.com ([fe80::ad51:8c49:b171:856c]) by SN6PR12MB4623.namprd12.prod.outlook.com ([fe80::ad51:8c49:b171:856c%7]) with mapi id 15.20.4065.026; Wed, 28 Apr 2021 15:13:17 +0000 From: Andrey Grodzovsky To: dri-devel@lists.freedesktop.org, amd-gfx@lists.freedesktop.org, linux-pci@vger.kernel.org, ckoenig.leichtzumerken@gmail.com, daniel.vetter@ffwll.ch, Harry.Wentland@amd.com Subject: [PATCH v5 24/27] drm/amdkfd: Scope all KFD queued work with drm_dev_enter/exit Date: Wed, 28 Apr 2021 11:12:04 -0400 Message-Id: <20210428151207.1212258-25-andrey.grodzovsky@amd.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210428151207.1212258-1-andrey.grodzovsky@amd.com> References: <20210428151207.1212258-1-andrey.grodzovsky@amd.com> X-Originating-IP: [2607:fea8:3edf:49b0:7212:f93a:73b0:8f23] X-ClientProxiedBy: YT1PR01CA0142.CANPRD01.PROD.OUTLOOK.COM (2603:10b6:b01:2f::21) To SN6PR12MB4623.namprd12.prod.outlook.com (2603:10b6:805:e9::17) MIME-Version: 1.0 X-MS-Exchange-MessageSentRepresentingType: 1 Received: from agrodzovsky-All-Series.hitronhub.home (2607:fea8:3edf:49b0:7212:f93a:73b0:8f23) by YT1PR01CA0142.CANPRD01.PROD.OUTLOOK.COM (2603:10b6:b01:2f::21) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4065.25 via Frontend Transport; Wed, 28 Apr 2021 15:13:14 +0000 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 4cbaced3-a07a-42de-51ae-08d90a582556 X-MS-TrafficTypeDiagnostic: SN6PR12MB2749: X-MS-Exchange-Transport-Forked: True X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:289; X-MS-Exchange-SenderADCheck: 1 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: qRfwkMQB883fKIXDAI9MYAsjQIFsTZVuwIERddFU65DHtIaCUJoHaX3oKsmL+glK7jo4te0YyDaGZouSHSdGx19EWBGhNaJS2yGugnyaSS60wa8DIjgKRcZhDzMEPcBZeSwX51rcLFuoMdawgCdNWIFl+Z3uZPqow+WVR6MuNIhnosXdQQWU5EIUp3eBW3aSTVNnboFhqc9PM4YPOzCCLdkVPQpApzdde6eoCS4G41K4hm4ve9CndX7yvsBv1eHUp69eMOerHzBgPIZzzBmWoZxh+Dkt+T1jXd5Lxn612Z+8GiAeNXmUnysmQ79nsEnrenk3kNLXk5WH2XbzOHjy1faVLwWRAF7MVPhw+Eb0cskDiUuU+iPLr7WQ4VPviAU7Za4GK5dIMmwSt1CdptFl7TMv9vQUo7hgIWeFJY5CZCoGt5r4lA8UVUuh+xQCqLmgxBkT3qQhyBV1M4+ef1wLndmTB5KRfEsmlMd88NWV4ZSHnHyLL3DOI967S9SudZW/eFcZvHZrQ5XWvUuQwy2v1wCpvzssnpqFwC/qCTBP1RmTdb807sptOGDV3K/lCOPosRyKrhKEnEEHiB1iCum+LYvMuewvOQtYfv9VqYthj28= X-Forefront-Antispam-Report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:SN6PR12MB4623.namprd12.prod.outlook.com; PTR:; CAT:NONE; SFS:(4636009)(376002)(346002)(39860400002)(136003)(396003)(366004)(16526019)(6666004)(6512007)(2616005)(1076003)(2906002)(6636002)(66556008)(186003)(86362001)(5660300002)(8936002)(83380400001)(6486002)(478600001)(44832011)(38100700002)(66946007)(52116002)(6506007)(66476007)(8676002)(4326008)(36756003)(316002); DIR:OUT; SFP:1101; X-MS-Exchange-AntiSpam-MessageData: LN7O+az4sVNRJyVOtCYGy4P5pD5BZpahPma9Mds3LDKX3SHH2tLpzkLpta0pZLtrRVtTCmYzSAoMNuTX6Z8oQRYo0hNTNJOCc806PaA6Q3PZJe/RNBj0sOS6ofK2u44oSwaFEBB+zHawTNIpa21c6dbL6OJsGctN0jCs1ab3G31VHrchU1rnVL6gz7a3Vba5ryxReiD62NEwojdBqiOvzoeHT2+eqSz7Xf48yafmEZ7c4lhA3VNqKirbCOFTzXpg/qjFGw2GOgXr/inmMu+aWEc2CRpiCQ7TDozv51Kiy0YLKihKUN1O9yLXX9HNLERRXBxn/+Xn4A6RHpuZMesX33G3+41nt1uTVXqY49I4A5SZnVFi1tfBl3JYufqSDYZF7W0WkpxPbr5X5C8kZBEwJXBZJnbjdG7XUgl3voHtR5ngRB5iSMIqUnSSqPulB8TAjs+NF/pWO6743BC469Mym09XdYmC4ljcmq6Y4qJECdFcVtn47imUuWKF3YwjFA16OSr8kHkX5zWzq1V6dxaAoL1ehQ5VXE/fhqeNI1/crUt4vFWrUxt20O8tplSEy+Rvp0yu7vjsNbcGQ+yzXeTMq8j3DburjI3VxRTMZFjdf9jat3qv5YEFFTCIpSOsD5iDQieyDikGUisrDNL/qWHYTxiqGNn9Hvy6t7uPwqSGNULPTHgP16EsVbsBwuyQ6gN42rzlHUxh+JbUqwX2D6uGGeWamRDQ+yf9hG8xcxwpxX+p6QCLGhrN1k+jFgpiav50IL0kJJlMKCub2dEifZSkZtc8ZXv7U1RJPpsXuPSml7ndsziKcYwRok8LgAYhPoEE0PEm4UQ0oipmk7WcopxxjMxQohAXSKrUGPucuBs7VRIySR7U224TFg5vM6KOO8bisTkqLE5lIUQZoD0kRTqr0YCQ22fyfirV1MM7jckrPfCpG6BN0AFse5Qgj1+yBqoEjk9QaCJ+a/7O1hpjcxuqW9oBXJl7SxieKhnNZ6GPc+Efho93zZMlbi8J2+xOyH/cclxiWvTDK4PImeEWCukL1BuPCjalb+s7s0yFd8Q1cBlJTwADMTIU6h/FzWQPnyiDDiWIXhC8aL2lqMQ8sYo3o267t1xOoOVjT4rr1BjZoQiVlc5zTgf8WCDx6O5hbmoyMa+ae8WPuEmIlkxqrW7tvkpQcTG3yZy9D7kpG01x5CuKPmqOEDCLj7AAYk5FT2GVEXyoXcETYVj11HfH5fJXwCNlpHopn71s45XU9wDNg7gUdiSDDP3KtfLOs15RsBnEqRxPsfn38wNHEkU+f6uXMpq9TKA/dDV2Nz+VwIrJsXlrfmIxcq40OVNP1PLxURzreCZ3TwozI4kyBRQ0QrhFCSde7FkJh6h7bUZaWdblgLcmbmBAB2lgoI1MTzCPt/gk X-OriginatorOrg: amd.com X-MS-Exchange-CrossTenant-Network-Message-Id: 4cbaced3-a07a-42de-51ae-08d90a582556 X-MS-Exchange-CrossTenant-AuthSource: SN6PR12MB4623.namprd12.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 28 Apr 2021 15:13:15.1058 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: 2qTrlNKQWAZwSM5eVslSpxznnUVmsVuGK6Yowjw5pHHxG2MDufOf9JG7YsrjbW9LypHVY5etwqzkpqfyHaryZQ== X-MS-Exchange-Transport-CrossTenantHeadersStamped: SN6PR12MB2749 X-BeenThere: dri-devel@lists.freedesktop.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Direct Rendering Infrastructure - Development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: gregkh@linuxfoundation.org, Felix.Kuehling@amd.com, helgaas@kernel.org, Alexander.Deucher@amd.com Errors-To: dri-devel-bounces@lists.freedesktop.org Sender: "dri-devel" To allow completion and further block of HW accesses post device PCI remove. Signed-off-by: Andrey Grodzovsky --- drivers/gpu/drm/amd/amdkfd/kfd_interrupt.c | 14 +++++++++++--- 1 file changed, 11 insertions(+), 3 deletions(-) diff --git a/drivers/gpu/drm/amd/amdkfd/kfd_interrupt.c b/drivers/gpu/drm/amd/amdkfd/kfd_interrupt.c index bc47f6a44456..563f02ab5b95 100644 --- a/drivers/gpu/drm/amd/amdkfd/kfd_interrupt.c +++ b/drivers/gpu/drm/amd/amdkfd/kfd_interrupt.c @@ -43,8 +43,10 @@ #include #include #include +#include #include "kfd_priv.h" + #define KFD_IH_NUM_ENTRIES 8192 static void interrupt_wq(struct work_struct *); @@ -145,15 +147,21 @@ static void interrupt_wq(struct work_struct *work) struct kfd_dev *dev = container_of(work, struct kfd_dev, interrupt_work); uint32_t ih_ring_entry[KFD_MAX_RING_ENTRY_SIZE]; + int idx; if (dev->device_info->ih_ring_entry_size > sizeof(ih_ring_entry)) { dev_err_once(kfd_chardev(), "Ring entry too small\n"); return; } - while (dequeue_ih_ring_entry(dev, ih_ring_entry)) - dev->device_info->event_interrupt_class->interrupt_wq(dev, - ih_ring_entry); + if (drm_dev_enter(dev->ddev, &idx)) { + + while (dequeue_ih_ring_entry(dev, ih_ring_entry)) + dev->device_info->event_interrupt_class->interrupt_wq(dev, + ih_ring_entry); + + drm_dev_exit(idx); + } } bool interrupt_is_wanted(struct kfd_dev *dev, From patchwork Wed Apr 28 15:12:05 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrey Grodzovsky X-Patchwork-Id: 12229273 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.5 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,MSGID_FROM_MTA_HEADER,SPF_HELO_NONE,SPF_PASS, USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 1D6EDC43600 for ; Wed, 28 Apr 2021 15:14:18 +0000 (UTC) Received: from gabe.freedesktop.org (gabe.freedesktop.org [131.252.210.177]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id B591661434 for ; Wed, 28 Apr 2021 15:14:17 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org B591661434 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=amd.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=dri-devel-bounces@lists.freedesktop.org Received: from gabe.freedesktop.org (localhost [127.0.0.1]) by gabe.freedesktop.org (Postfix) with ESMTP id 3C8FE6EB93; Wed, 28 Apr 2021 15:14:16 +0000 (UTC) Received: from NAM11-CO1-obe.outbound.protection.outlook.com (mail-co1nam11on2043.outbound.protection.outlook.com [40.107.220.43]) by gabe.freedesktop.org (Postfix) with ESMTPS id 5252B6EB7D; Wed, 28 Apr 2021 15:13:19 +0000 (UTC) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=Ciiqqan7NJLs4EJH1m9DvSun83r5Fs/zysZEpq1dFi/XtlO1lLHJ/y4bsEaJh8hJpnNBqOJ9k3RvoZmWpU08qaGcRgyzD278/giPZILCTTU+lYsVockb/FdFWc/zT3Pyz+t/Nlz6k0hDCavLRSi4sdFdKugrvzx7M/C234u1SfyxvvPxoRc+KjXMsoRzDaqq4HDiLhsNOy8PriXr8qA3AbUwYYsJ2Wa2b9xdydL29oZHGebugXdZQwklsEbkAbl4+V+f8dUrPFU2lUv2TW2uT2IGwjxEbyl7LJlH79yglOTxBcnnmpEGS94ufoyn/Nl4990QJKGAghfIgSkKLvjxxw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=s1bAhGv2kWYW8LlbDT5BH+iVENuohJgwHeQDo0+fR6E=; b=UUBSPwZzn6F18Cut9KjLK6kmTib4KpnKY7mrV2UArJm7tjE4B4S/Yk+4d1paZOtqWe2bAB8tqx4JXVf6P12JWtvB3nn4+tegbDWwoh834sc88XJHHoSghm/SC/lfZKbkvvJhi2VRrQqcwEW8zmWbITJCCEm/lDNGXhbnFveM3gXXu1fdBWj2hAoAmy1e04/nt/Idhcwf34q1X6sc5CeGNUXoNSAV3jfr4l6CsDZz8za7DmDkM2eoo9HnAERS8qfTj3+XcYkDuL17ZoDf2HAu7HBINpZA18SibXzrQWG3ccPU213+gO6mqTlQqKXErbhgb7gPhMyF0gZqOcC8INdM3g== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=amd.com; dmarc=pass action=none header.from=amd.com; dkim=pass header.d=amd.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=s1bAhGv2kWYW8LlbDT5BH+iVENuohJgwHeQDo0+fR6E=; b=5GiMn0bgMG0jJCXXXMd1pfobgUaoRYJjp4RbiPn6p/ETCf8EDawYkKBR7X2kRY3Mcd7HZLhQRPodktnUZrP8VEyQoX1etfwQrw0EqLqJDcj3ffHFreIBcAq+tESaqzG/girz4Sv0Z2II+adf1uXKqAeh9I5YawaZQ/VgMmxYmAg= Authentication-Results: lists.freedesktop.org; dkim=none (message not signed) header.d=none; lists.freedesktop.org; dmarc=none action=none header.from=amd.com; Received: from SN6PR12MB4623.namprd12.prod.outlook.com (2603:10b6:805:e9::17) by SN6PR12MB2749.namprd12.prod.outlook.com (2603:10b6:805:6d::18) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4065.22; Wed, 28 Apr 2021 15:13:17 +0000 Received: from SN6PR12MB4623.namprd12.prod.outlook.com ([fe80::ad51:8c49:b171:856c]) by SN6PR12MB4623.namprd12.prod.outlook.com ([fe80::ad51:8c49:b171:856c%7]) with mapi id 15.20.4065.026; Wed, 28 Apr 2021 15:13:17 +0000 From: Andrey Grodzovsky To: dri-devel@lists.freedesktop.org, amd-gfx@lists.freedesktop.org, linux-pci@vger.kernel.org, ckoenig.leichtzumerken@gmail.com, daniel.vetter@ffwll.ch, Harry.Wentland@amd.com Subject: [PATCH v5 25/27] drm/amdgpu: Scope all amdgpu queued work with drm_dev_enter/exit Date: Wed, 28 Apr 2021 11:12:05 -0400 Message-Id: <20210428151207.1212258-26-andrey.grodzovsky@amd.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210428151207.1212258-1-andrey.grodzovsky@amd.com> References: <20210428151207.1212258-1-andrey.grodzovsky@amd.com> X-Originating-IP: [2607:fea8:3edf:49b0:7212:f93a:73b0:8f23] X-ClientProxiedBy: YT1PR01CA0142.CANPRD01.PROD.OUTLOOK.COM (2603:10b6:b01:2f::21) To SN6PR12MB4623.namprd12.prod.outlook.com (2603:10b6:805:e9::17) MIME-Version: 1.0 X-MS-Exchange-MessageSentRepresentingType: 1 Received: from agrodzovsky-All-Series.hitronhub.home (2607:fea8:3edf:49b0:7212:f93a:73b0:8f23) by YT1PR01CA0142.CANPRD01.PROD.OUTLOOK.COM (2603:10b6:b01:2f::21) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4065.25 via Frontend Transport; Wed, 28 Apr 2021 15:13:15 +0000 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 63cad74e-bf15-4fc6-5073-08d90a582665 X-MS-TrafficTypeDiagnostic: SN6PR12MB2749: X-MS-Exchange-Transport-Forked: True X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:2089; X-MS-Exchange-SenderADCheck: 1 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: mtAPMZ28dSDSvU8fs+OHMK0RdHsg4e3ixwPEnT6ctujX9K33ybH7oViJj7SzfpsmNA3OyE1bKKPI2Of7W7e+VI0lQPl80qhERan/xwBiMdagIzvP5aHt+XNTK5oYFPvpZipDLIUcY2estqVcwIfONuu/I/qQtfJM0w2bBB5avRKPmmvOaUYl6C15WgGxwy6N/2tfI7h6hTLEf2UUzM/tO/2QKIXYo45qQQZSwhxG55ZcYl1njlsD+wW6BxWDN2PGyM4a1C+K60Hf0C7qKd92lq9yTesfHvMD1DSN2wBaRBd0pdXVCK4VKQnVV4WyGxVB+xmz2dJ1vi/2C+PK5ltU2AndsO9CmN7LTnCQDBScM4c29sKXPzQWzBmNyvHopXsI9d2YRfETaDc9VG32hSO7jpN3BXuL38v/PWGkX3ewNtjXCw2JJxp672oTrJ5hFRzi9RjIJ9oHKl29NOAXfwJRr5pdmfZHc6X3AEBu0J2Y1O8Y3zaA1u/D0K4k05IzRFcTgwmngtzxMKRvx0rGZ24xwEdQZr/UeVber80cIPuKkwADa9kr4HaeijOR3rdfiiEIJglmaBz6osLWpxnRjt9gwQ== X-Forefront-Antispam-Report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:SN6PR12MB4623.namprd12.prod.outlook.com; PTR:; CAT:NONE; SFS:(4636009)(376002)(346002)(39860400002)(136003)(396003)(366004)(16526019)(6666004)(6512007)(2616005)(1076003)(2906002)(30864003)(6636002)(66556008)(186003)(86362001)(5660300002)(8936002)(83380400001)(6486002)(478600001)(44832011)(38100700002)(66946007)(52116002)(6506007)(66476007)(8676002)(4326008)(36756003)(316002); DIR:OUT; SFP:1101; X-MS-Exchange-AntiSpam-MessageData: px8v/VbXivvcQlXeFlAcA46sthe5KYjJbgCFvyzFwQi2oblc+K9LQGss6fwl8vHvrpJYR1/Kg0T94LAkSopkaZlEKtpHdcu9nR1hqc2ER6PFjQLS9DmJDKOoOSQjJtOTZh0xfTjzmQ+Wlz4GuODZJTRHHv4aQraN2MKMjcRXRQ8yHHmPDIiqK3PMvLC6pBQyvWbOBWhidS59w19/VIz+xl71rQrdWSCP4Ijv/g/lR/+FLuSn0VsYS4fabj1Sj6Rvm35TVQPHU3Pfk1w2Fbm2oT23kAxs523eFfE4wHxyNuNkCUNReA4LE+CpAItcFu1wLR98TqQDJP80um9KWghjWHgOS2ofzXM0rEUCxGmRQisDtwo7Q/Q1xb8Sj20fjmYVSF0AO1lENXq+EQrFKhJUwQddeem9Of/NZYWMckWO7a9900pTJL7tRVvY+uAg9tpT19hVNzo+n+/EvWuMdQvi5iU9QZ/YfAaLIJ0q2jOodXaobP19NpaI6H7FpjC0o8VbKIANHAT085+tsTmeaMemX5540V0hMc/UCvsTPo5x35R4pyFkcmAvXShokhCS1I/pNs+WoMFScmfo5iTxPtv7PFXbRVjKhWJatfR3QOqLcEphgsP5fV922HZ6kOYlS0aRnHOEU2Hs1oMxOfDqb5b3MNWXMSOrCkTSjoFhRW1FGhpPo0eGyrI9Q4rPu6jay7a0JQnCImq7GuiVF30gAF9Co+4XEGnhgqGstbRbpVNuAoCrmMZSr414U/PMkxBdVFref5lvBnr8AHiQpkFQV07rCL6NPM2T+BaeQZP0Mw7gFVU0uRRZhvVhmiXapz1zgARm/Hf05cey4m9BoIVE5njim528ojMcHnhFOzTHc9SXs2CJWy19ImnfZDmtLiuHV/AWWvsgrrEhH/abi7SCj9vX09r5WWud3JCp7yCDXr1v8A2tAkJECVsbulxB/5YoMNo7A2so+aBjVYWnRJ/tXxIuOioL8ysHXlnxKwhqxqxlXzKflO/PwRCyTrk2TUUW8blRqpHRMfF90wOPhKAAarLkaANL0fzMmgXvObqj5mbz1j1oD8DtqrcBmcxSJ86WlpRXfYLX0i6Tp13Wk5SpK8jNCkPcTyX6jEHmNwGAYN/xLqNuT2S/y5FZu7145U0hmXzG87gHfDuZf/w4qh4anqCl6GiNO1n0FPQrWjUQbEdG8ASv5gtkrwJ2TkcgZAbd1+cdL3fxCsX0PRdOvrezGGnRVSo5iBKBT83xuKQscbshRLJ0ukpKdeoFQhkY1twFa9vKXnOU/OCzbrWZ6hx1FHCJ8oUitVcC/p0vEGBoiEsdKjbPssHihFCps4SymwMpegdaGHZOBOYaWn9vOT/Q/xRft+3bWRmbC94jU4u4td4WLlk+2010moN5xt484cxl564r X-OriginatorOrg: amd.com X-MS-Exchange-CrossTenant-Network-Message-Id: 63cad74e-bf15-4fc6-5073-08d90a582665 X-MS-Exchange-CrossTenant-AuthSource: SN6PR12MB4623.namprd12.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 28 Apr 2021 15:13:16.8838 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: Ceq+4tNQ8FWDgtIMolQZqjUO9VQGDRiK04kAhy/Mavc0ivZmXTQZWUc6FuPw1oxBry7JCOuIREc5Po/KGkyb0Q== X-MS-Exchange-Transport-CrossTenantHeadersStamped: SN6PR12MB2749 X-BeenThere: dri-devel@lists.freedesktop.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Direct Rendering Infrastructure - Development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: gregkh@linuxfoundation.org, Felix.Kuehling@amd.com, helgaas@kernel.org, Alexander.Deucher@amd.com Errors-To: dri-devel-bounces@lists.freedesktop.org Sender: "dri-devel" To allow completion and further block of HW accesses post device PCI remove. Signed-off-by: Andrey Grodzovsky --- drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd.c | 11 +++- drivers/gpu/drm/amd/amdgpu/amdgpu_device.c | 29 ++++++---- drivers/gpu/drm/amd/amdgpu/amdgpu_irq.c | 26 +++++++-- drivers/gpu/drm/amd/amdgpu/amdgpu_jpeg.c | 28 ++++++---- drivers/gpu/drm/amd/amdgpu/amdgpu_ras.c | 55 ++++++++++++------- drivers/gpu/drm/amd/amdgpu/amdgpu_uvd.c | 43 ++++++++------- drivers/gpu/drm/amd/amdgpu/amdgpu_vce.c | 30 ++++++----- drivers/gpu/drm/amd/amdgpu/amdgpu_vcn.c | 61 ++++++++++++---------- drivers/gpu/drm/amd/amdgpu/gfx_v8_0.c | 10 +++- 9 files changed, 189 insertions(+), 104 deletions(-) diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd.c index 9edb35ba181b..f942496c2b35 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd.c +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd.c @@ -30,6 +30,7 @@ #include #include "amdgpu_xgmi.h" #include +#include /* Total memory size in system memory and all GPU VRAM. Used to * estimate worst case amount of memory to reserve for page tables @@ -223,9 +224,15 @@ int amdgpu_amdkfd_post_reset(struct amdgpu_device *adev) void amdgpu_amdkfd_gpu_reset(struct kgd_dev *kgd) { struct amdgpu_device *adev = (struct amdgpu_device *)kgd; + int idx; - if (amdgpu_device_should_recover_gpu(adev)) - amdgpu_device_gpu_recover(adev, NULL); + if (drm_dev_enter(&adev->ddev, &idx)) { + + if (amdgpu_device_should_recover_gpu(adev)) + amdgpu_device_gpu_recover(adev, NULL); + + drm_dev_exit(idx); + } } int amdgpu_amdkfd_alloc_gtt_mem(struct kgd_dev *kgd, size_t size, diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c index 55afc11c17e6..c30e0b0596a5 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c @@ -2674,24 +2674,35 @@ static void amdgpu_device_delayed_init_work_handler(struct work_struct *work) { struct amdgpu_device *adev = container_of(work, struct amdgpu_device, delayed_init_work.work); - int r; + int r, idx; - r = amdgpu_ib_ring_tests(adev); - if (r) - DRM_ERROR("ib ring test failed (%d).\n", r); + if (drm_dev_enter(&adev->ddev, &idx)) { + r = amdgpu_ib_ring_tests(adev); + if (r) + DRM_ERROR("ib ring test failed (%d).\n", r); + + drm_dev_exit(idx); + } } static void amdgpu_device_delay_enable_gfx_off(struct work_struct *work) { struct amdgpu_device *adev = container_of(work, struct amdgpu_device, gfx.gfx_off_delay_work.work); + int idx; + + if (drm_dev_enter(&adev->ddev, &idx)) { + + mutex_lock(&adev->gfx.gfx_off_mutex); + if (!adev->gfx.gfx_off_state && !adev->gfx.gfx_off_req_count) { + if (!amdgpu_dpm_set_powergating_by_smu(adev, AMD_IP_BLOCK_TYPE_GFX, true)) + adev->gfx.gfx_off_state = true; + } + mutex_unlock(&adev->gfx.gfx_off_mutex); + + drm_dev_exit(idx); - mutex_lock(&adev->gfx.gfx_off_mutex); - if (!adev->gfx.gfx_off_state && !adev->gfx.gfx_off_req_count) { - if (!amdgpu_dpm_set_powergating_by_smu(adev, AMD_IP_BLOCK_TYPE_GFX, true)) - adev->gfx.gfx_off_state = true; } - mutex_unlock(&adev->gfx.gfx_off_mutex); } /** diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_irq.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_irq.c index a922154953a7..5eda0d0fc974 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_irq.c +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_irq.c @@ -188,8 +188,15 @@ static void amdgpu_irq_handle_ih1(struct work_struct *work) { struct amdgpu_device *adev = container_of(work, struct amdgpu_device, irq.ih1_work); + int idx; - amdgpu_ih_process(adev, &adev->irq.ih1); + if (drm_dev_enter(&adev->ddev, &idx)) { + + amdgpu_ih_process(adev, &adev->irq.ih1); + + drm_dev_exit(idx); + + } } /** @@ -203,8 +210,14 @@ static void amdgpu_irq_handle_ih2(struct work_struct *work) { struct amdgpu_device *adev = container_of(work, struct amdgpu_device, irq.ih2_work); + int idx; + + if (drm_dev_enter(&adev->ddev, &idx)) { + + amdgpu_ih_process(adev, &adev->irq.ih2); - amdgpu_ih_process(adev, &adev->irq.ih2); + drm_dev_exit(idx); + } } /** @@ -218,8 +231,15 @@ static void amdgpu_irq_handle_ih_soft(struct work_struct *work) { struct amdgpu_device *adev = container_of(work, struct amdgpu_device, irq.ih_soft_work); + int idx; + + if (drm_dev_enter(&adev->ddev, &idx)) { + + amdgpu_ih_process(adev, &adev->irq.ih_soft); - amdgpu_ih_process(adev, &adev->irq.ih_soft); + drm_dev_exit(idx); + + } } /** diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_jpeg.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_jpeg.c index 8996cb4ed57a..1e8fd66c1e43 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_jpeg.c +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_jpeg.c @@ -29,6 +29,7 @@ #include "amdgpu_pm.h" #include "soc15d.h" #include "soc15_common.h" +#include #define JPEG_IDLE_TIMEOUT msecs_to_jiffies(1000) @@ -78,20 +79,25 @@ static void amdgpu_jpeg_idle_work_handler(struct work_struct *work) struct amdgpu_device *adev = container_of(work, struct amdgpu_device, jpeg.idle_work.work); unsigned int fences = 0; - unsigned int i; + unsigned int i, idx; - for (i = 0; i < adev->jpeg.num_jpeg_inst; ++i) { - if (adev->jpeg.harvest_config & (1 << i)) - continue; + if (drm_dev_enter(&adev->ddev, &idx)) { - fences += amdgpu_fence_count_emitted(&adev->jpeg.inst[i].ring_dec); - } + for (i = 0; i < adev->jpeg.num_jpeg_inst; ++i) { + if (adev->jpeg.harvest_config & (1 << i)) + continue; - if (!fences && !atomic_read(&adev->jpeg.total_submission_cnt)) - amdgpu_device_ip_set_powergating_state(adev, AMD_IP_BLOCK_TYPE_JPEG, - AMD_PG_STATE_GATE); - else - schedule_delayed_work(&adev->jpeg.idle_work, JPEG_IDLE_TIMEOUT); + fences += amdgpu_fence_count_emitted(&adev->jpeg.inst[i].ring_dec); + } + + if (!fences && !atomic_read(&adev->jpeg.total_submission_cnt)) + amdgpu_device_ip_set_powergating_state(adev, AMD_IP_BLOCK_TYPE_JPEG, + AMD_PG_STATE_GATE); + else + schedule_delayed_work(&adev->jpeg.idle_work, JPEG_IDLE_TIMEOUT); + + drm_dev_exit(idx); + } } void amdgpu_jpeg_ring_begin_use(struct amdgpu_ring *ring) diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_ras.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_ras.c index c0a16eac4923..97a6c028ac74 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_ras.c +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_ras.c @@ -27,6 +27,7 @@ #include #include #include +#include #include "amdgpu.h" #include "amdgpu_ras.h" @@ -1334,7 +1335,15 @@ static void amdgpu_ras_interrupt_process_handler(struct work_struct *work) struct ras_manager *obj = container_of(data, struct ras_manager, ih_data); - amdgpu_ras_interrupt_handler(obj); + int idx; + + if (drm_dev_enter(&obj->adev->ddev, &idx)) { + + amdgpu_ras_interrupt_handler(obj); + + drm_dev_exit(idx); + + } } int amdgpu_ras_interrupt_dispatch(struct amdgpu_device *adev, @@ -1565,31 +1574,37 @@ static void amdgpu_ras_do_recovery(struct work_struct *work) struct amdgpu_device *remote_adev = NULL; struct amdgpu_device *adev = ras->adev; struct list_head device_list, *device_list_handle = NULL; + int idx; - if (!ras->disable_ras_err_cnt_harvest) { - struct amdgpu_hive_info *hive = amdgpu_get_xgmi_hive(adev); + if (drm_dev_enter(&adev->ddev, &idx)) { - /* Build list of devices to query RAS related errors */ - if (hive && adev->gmc.xgmi.num_physical_nodes > 1) { - device_list_handle = &hive->device_list; - } else { - INIT_LIST_HEAD(&device_list); - list_add_tail(&adev->gmc.xgmi.head, &device_list); - device_list_handle = &device_list; - } + if (!ras->disable_ras_err_cnt_harvest) { + struct amdgpu_hive_info *hive = amdgpu_get_xgmi_hive(adev); - list_for_each_entry(remote_adev, - device_list_handle, gmc.xgmi.head) { - amdgpu_ras_query_err_status(remote_adev); - amdgpu_ras_log_on_err_counter(remote_adev); + /* Build list of devices to query RAS related errors */ + if (hive && adev->gmc.xgmi.num_physical_nodes > 1) { + device_list_handle = &hive->device_list; + } else { + INIT_LIST_HEAD(&device_list); + list_add_tail(&adev->gmc.xgmi.head, &device_list); + device_list_handle = &device_list; + } + + list_for_each_entry(remote_adev, + device_list_handle, gmc.xgmi.head) { + amdgpu_ras_query_err_status(remote_adev); + amdgpu_ras_log_on_err_counter(remote_adev); + } + + amdgpu_put_xgmi_hive(hive); } - amdgpu_put_xgmi_hive(hive); - } + if (amdgpu_device_should_recover_gpu(ras->adev)) + amdgpu_device_gpu_recover(ras->adev, NULL); + atomic_set(&ras->in_recovery, 0); - if (amdgpu_device_should_recover_gpu(ras->adev)) - amdgpu_device_gpu_recover(ras->adev, NULL); - atomic_set(&ras->in_recovery, 0); + drm_dev_exit(idx); + } } /* alloc/realloc bps array */ diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_uvd.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_uvd.c index df47f5ffa08f..19790afd2893 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_uvd.c +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_uvd.c @@ -1234,29 +1234,36 @@ static void amdgpu_uvd_idle_work_handler(struct work_struct *work) struct amdgpu_device *adev = container_of(work, struct amdgpu_device, uvd.idle_work.work); unsigned fences = 0, i, j; + int idx; - for (i = 0; i < adev->uvd.num_uvd_inst; ++i) { - if (adev->uvd.harvest_config & (1 << i)) - continue; - fences += amdgpu_fence_count_emitted(&adev->uvd.inst[i].ring); - for (j = 0; j < adev->uvd.num_enc_rings; ++j) { - fences += amdgpu_fence_count_emitted(&adev->uvd.inst[i].ring_enc[j]); + if (drm_dev_enter(&adev->ddev, &idx)) { + + for (i = 0; i < adev->uvd.num_uvd_inst; ++i) { + if (adev->uvd.harvest_config & (1 << i)) + continue; + fences += amdgpu_fence_count_emitted(&adev->uvd.inst[i].ring); + for (j = 0; j < adev->uvd.num_enc_rings; ++j) { + fences += amdgpu_fence_count_emitted(&adev->uvd.inst[i].ring_enc[j]); + } } - } - if (fences == 0) { - if (adev->pm.dpm_enabled) { - amdgpu_dpm_enable_uvd(adev, false); + if (fences == 0) { + if (adev->pm.dpm_enabled) { + amdgpu_dpm_enable_uvd(adev, false); + } else { + amdgpu_asic_set_uvd_clocks(adev, 0, 0); + /* shutdown the UVD block */ + amdgpu_device_ip_set_powergating_state(adev, AMD_IP_BLOCK_TYPE_UVD, + AMD_PG_STATE_GATE); + amdgpu_device_ip_set_clockgating_state(adev, AMD_IP_BLOCK_TYPE_UVD, + AMD_CG_STATE_GATE); + } } else { - amdgpu_asic_set_uvd_clocks(adev, 0, 0); - /* shutdown the UVD block */ - amdgpu_device_ip_set_powergating_state(adev, AMD_IP_BLOCK_TYPE_UVD, - AMD_PG_STATE_GATE); - amdgpu_device_ip_set_clockgating_state(adev, AMD_IP_BLOCK_TYPE_UVD, - AMD_CG_STATE_GATE); + schedule_delayed_work(&adev->uvd.idle_work, UVD_IDLE_TIMEOUT); } - } else { - schedule_delayed_work(&adev->uvd.idle_work, UVD_IDLE_TIMEOUT); + + drm_dev_exit(idx); + } } diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_vce.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_vce.c index 833203401ef4..81ad937936bb 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_vce.c +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_vce.c @@ -340,22 +340,28 @@ static void amdgpu_vce_idle_work_handler(struct work_struct *work) struct amdgpu_device *adev = container_of(work, struct amdgpu_device, vce.idle_work.work); unsigned i, count = 0; + int idx; - for (i = 0; i < adev->vce.num_rings; i++) - count += amdgpu_fence_count_emitted(&adev->vce.ring[i]); + if (drm_dev_enter(&adev->ddev, &idx)) { - if (count == 0) { - if (adev->pm.dpm_enabled) { - amdgpu_dpm_enable_vce(adev, false); + for (i = 0; i < adev->vce.num_rings; i++) + count += amdgpu_fence_count_emitted(&adev->vce.ring[i]); + + if (count == 0) { + if (adev->pm.dpm_enabled) { + amdgpu_dpm_enable_vce(adev, false); + } else { + amdgpu_asic_set_vce_clocks(adev, 0, 0); + amdgpu_device_ip_set_powergating_state(adev, AMD_IP_BLOCK_TYPE_VCE, + AMD_PG_STATE_GATE); + amdgpu_device_ip_set_clockgating_state(adev, AMD_IP_BLOCK_TYPE_VCE, + AMD_CG_STATE_GATE); + } } else { - amdgpu_asic_set_vce_clocks(adev, 0, 0); - amdgpu_device_ip_set_powergating_state(adev, AMD_IP_BLOCK_TYPE_VCE, - AMD_PG_STATE_GATE); - amdgpu_device_ip_set_clockgating_state(adev, AMD_IP_BLOCK_TYPE_VCE, - AMD_CG_STATE_GATE); + schedule_delayed_work(&adev->vce.idle_work, VCE_IDLE_TIMEOUT); } - } else { - schedule_delayed_work(&adev->vce.idle_work, VCE_IDLE_TIMEOUT); + + drm_dev_exit(idx); } } diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_vcn.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_vcn.c index b42db22761b8..0e7404653ac5 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_vcn.c +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_vcn.c @@ -342,42 +342,47 @@ static void amdgpu_vcn_idle_work_handler(struct work_struct *work) container_of(work, struct amdgpu_device, vcn.idle_work.work); unsigned int fences = 0, fence[AMDGPU_MAX_VCN_INSTANCES] = {0}; unsigned int i, j; - int r = 0; + int idx, r = 0; - for (j = 0; j < adev->vcn.num_vcn_inst; ++j) { - if (adev->vcn.harvest_config & (1 << j)) - continue; + if (drm_dev_enter(&adev->ddev, &idx)) { - for (i = 0; i < adev->vcn.num_enc_rings; ++i) { - fence[j] += amdgpu_fence_count_emitted(&adev->vcn.inst[j].ring_enc[i]); - } + for (j = 0; j < adev->vcn.num_vcn_inst; ++j) { + if (adev->vcn.harvest_config & (1 << j)) + continue; - if (adev->pg_flags & AMD_PG_SUPPORT_VCN_DPG) { - struct dpg_pause_state new_state; + for (i = 0; i < adev->vcn.num_enc_rings; ++i) + fence[j] += amdgpu_fence_count_emitted(&adev->vcn.inst[j].ring_enc[i]); - if (fence[j] || - unlikely(atomic_read(&adev->vcn.inst[j].dpg_enc_submission_cnt))) - new_state.fw_based = VCN_DPG_STATE__PAUSE; - else - new_state.fw_based = VCN_DPG_STATE__UNPAUSE; + if (adev->pg_flags & AMD_PG_SUPPORT_VCN_DPG) { + struct dpg_pause_state new_state; + + if (fence[j] || + unlikely(atomic_read(&adev->vcn.inst[j].dpg_enc_submission_cnt))) + new_state.fw_based = VCN_DPG_STATE__PAUSE; + else + new_state.fw_based = VCN_DPG_STATE__UNPAUSE; - adev->vcn.pause_dpg_mode(adev, j, &new_state); + adev->vcn.pause_dpg_mode(adev, j, &new_state); + } + + fence[j] += amdgpu_fence_count_emitted(&adev->vcn.inst[j].ring_dec); + fences += fence[j]; } - fence[j] += amdgpu_fence_count_emitted(&adev->vcn.inst[j].ring_dec); - fences += fence[j]; - } + if (!fences && !atomic_read(&adev->vcn.total_submission_cnt)) { + amdgpu_gfx_off_ctrl(adev, true); + amdgpu_device_ip_set_powergating_state(adev, AMD_IP_BLOCK_TYPE_VCN, + AMD_PG_STATE_GATE); + r = amdgpu_dpm_switch_power_profile(adev, PP_SMC_POWER_PROFILE_VIDEO, + false); + if (r) + dev_warn(adev->dev, "(%d) failed to disable video power profile mode\n", r); + } else { + schedule_delayed_work(&adev->vcn.idle_work, VCN_IDLE_TIMEOUT); + } + + drm_dev_exit(idx); - if (!fences && !atomic_read(&adev->vcn.total_submission_cnt)) { - amdgpu_gfx_off_ctrl(adev, true); - amdgpu_device_ip_set_powergating_state(adev, AMD_IP_BLOCK_TYPE_VCN, - AMD_PG_STATE_GATE); - r = amdgpu_dpm_switch_power_profile(adev, PP_SMC_POWER_PROFILE_VIDEO, - false); - if (r) - dev_warn(adev->dev, "(%d) failed to disable video power profile mode\n", r); - } else { - schedule_delayed_work(&adev->vcn.idle_work, VCN_IDLE_TIMEOUT); } } diff --git a/drivers/gpu/drm/amd/amdgpu/gfx_v8_0.c b/drivers/gpu/drm/amd/amdgpu/gfx_v8_0.c index 84d2eaa38101..4799290e5625 100644 --- a/drivers/gpu/drm/amd/amdgpu/gfx_v8_0.c +++ b/drivers/gpu/drm/amd/amdgpu/gfx_v8_0.c @@ -57,6 +57,8 @@ #include "ivsrcid/ivsrcid_vislands30.h" +#include + #define GFX8_NUM_GFX_RINGS 1 #define GFX8_MEC_HPD_SIZE 4096 @@ -6793,8 +6795,14 @@ static void gfx_v8_0_sq_irq_work_func(struct work_struct *work) struct amdgpu_device *adev = container_of(work, struct amdgpu_device, gfx.sq_work.work); struct sq_work *sq_work = container_of(work, struct sq_work, work); + int idx; + + if (drm_dev_enter(&adev->ddev, &idx)) { - gfx_v8_0_parse_sq_irq(adev, sq_work->ih_data); + gfx_v8_0_parse_sq_irq(adev, sq_work->ih_data); + + drm_dev_exit(idx); + } } static int gfx_v8_0_sq_irq(struct amdgpu_device *adev, From patchwork Wed Apr 28 15:12:06 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrey Grodzovsky X-Patchwork-Id: 12229249 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.5 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,MSGID_FROM_MTA_HEADER,SPF_HELO_NONE,SPF_PASS, USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id B8A02C43461 for ; Wed, 28 Apr 2021 15:13:32 +0000 (UTC) Received: from gabe.freedesktop.org (gabe.freedesktop.org [131.252.210.177]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 61DA76143A for ; Wed, 28 Apr 2021 15:13:32 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 61DA76143A Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=amd.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=dri-devel-bounces@lists.freedesktop.org Received: from gabe.freedesktop.org (localhost [127.0.0.1]) by gabe.freedesktop.org (Postfix) with ESMTP id AA5666EB7A; Wed, 28 Apr 2021 15:13:29 +0000 (UTC) Received: from NAM11-CO1-obe.outbound.protection.outlook.com (mail-co1nam11on2043.outbound.protection.outlook.com [40.107.220.43]) by gabe.freedesktop.org (Postfix) with ESMTPS id 42C466EB75; Wed, 28 Apr 2021 15:13:20 +0000 (UTC) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=XcroSFWPYOc08gzEasYvlbcxvPgzvp+FlayonchB94aC8X2v1H3dZhiuXyOdGWaynYdX99p2W4U3y6YEt7HNCRyOnmJGZuBPeaRgdw2u9TmIspx6mMEceohtFoM+zKoa1oWABVWxZ5ZAWZfjqB0DmTQ66AhgmXXROrbVSv1CUQi+6kA2QG2Gz4A2TGMGBgsuqA0vkc7rWua3FhfxnTGAZKn0RGRww6IspHYsCDElOhV3VWAvnJGw5clRAqlxIsiCPvulmEBG/jbGYrd2YeLZ0OGBb+DwuhLk+sE6JjTOmjQSCHt43EcO97ut7SyuscVIgljvzk97EtapbsAPVpb16g== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=SOwkjT7+g+P7X9KE+MhAHVmjtjutqkkv/Olu4nEjCzI=; b=k8amwkmBGI50/cvVjEDmgXRbDZAMulfcX+8NEq0klFZr2hv+KWrI36Fkv/94dlf6L1A4W47CLBVH3R0UQr4Hpe5pYJA4OEbpuraztxjs51/H7t2z+gbS846hSrLDA6xDwXAq2Yk0lJjZT1HOdOZEig1MmKyoja9u7w7vjQkZkCz1Mc062SwuDstGb6CCq23pur4Iwnoz041cCpFdbJQq0yCuaHGxAzGqxRmSsHp6HtXEonLrz7yXExhbNujOUSV+I+0K0L427DAVXcxnehlU972ENvJBsy3iHcGgyAxvN1DJxPid0dbgSsplgnsJLKmK5Vk4G0hJ2s+y9YV0K9NGqw== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=amd.com; dmarc=pass action=none header.from=amd.com; dkim=pass header.d=amd.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=SOwkjT7+g+P7X9KE+MhAHVmjtjutqkkv/Olu4nEjCzI=; b=iiuewlepIIbpaIM/7SW9cLySS9qoBm13KxH5xGljeFgSgs4mnfcWT7WTNzq16GBpyYnoNF2vFa4shqboi6Xj5QkNRQPczgJWp3sCub/bg7Vu+M3sClQ3iU6/tB4N5D95CYBT8fVA118UZyHTZann/s8ywizfr6qWms6av6V5sdw= Authentication-Results: lists.freedesktop.org; dkim=none (message not signed) header.d=none; lists.freedesktop.org; dmarc=none action=none header.from=amd.com; Received: from SN6PR12MB4623.namprd12.prod.outlook.com (2603:10b6:805:e9::17) by SN6PR12MB2749.namprd12.prod.outlook.com (2603:10b6:805:6d::18) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4065.22; Wed, 28 Apr 2021 15:13:18 +0000 Received: from SN6PR12MB4623.namprd12.prod.outlook.com ([fe80::ad51:8c49:b171:856c]) by SN6PR12MB4623.namprd12.prod.outlook.com ([fe80::ad51:8c49:b171:856c%7]) with mapi id 15.20.4065.026; Wed, 28 Apr 2021 15:13:18 +0000 From: Andrey Grodzovsky To: dri-devel@lists.freedesktop.org, amd-gfx@lists.freedesktop.org, linux-pci@vger.kernel.org, ckoenig.leichtzumerken@gmail.com, daniel.vetter@ffwll.ch, Harry.Wentland@amd.com Subject: [PATCH v5 26/27] drm/amd/display: Remove superflous drm_mode_config_cleanup Date: Wed, 28 Apr 2021 11:12:06 -0400 Message-Id: <20210428151207.1212258-27-andrey.grodzovsky@amd.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210428151207.1212258-1-andrey.grodzovsky@amd.com> References: <20210428151207.1212258-1-andrey.grodzovsky@amd.com> X-Originating-IP: [2607:fea8:3edf:49b0:7212:f93a:73b0:8f23] X-ClientProxiedBy: YT1PR01CA0142.CANPRD01.PROD.OUTLOOK.COM (2603:10b6:b01:2f::21) To SN6PR12MB4623.namprd12.prod.outlook.com (2603:10b6:805:e9::17) MIME-Version: 1.0 X-MS-Exchange-MessageSentRepresentingType: 1 Received: from agrodzovsky-All-Series.hitronhub.home (2607:fea8:3edf:49b0:7212:f93a:73b0:8f23) by YT1PR01CA0142.CANPRD01.PROD.OUTLOOK.COM (2603:10b6:b01:2f::21) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4065.25 via Frontend Transport; Wed, 28 Apr 2021 15:13:17 +0000 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 776e3725-2352-45d1-f083-08d90a582769 X-MS-TrafficTypeDiagnostic: SN6PR12MB2749: X-MS-Exchange-Transport-Forked: True X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:25; X-MS-Exchange-SenderADCheck: 1 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: NvKVH3tjJyf6MBKmxdadxmPIbZKRKiWpT+wa7qDGE0tHWJtVQAQtgAXMXAh1vOH2/l2FOYKnPyehdrmBgXu8/tRuK3u/zvETCkhjVNSuSGrI4jlUeEUBMVZaIlw3/Ln3tGrPMTcp1YQOevnZwKia5l3h5UqUVrHffJaOWEMa+4OB9hGwiiTBnMwv6yhOR+9pCAFyGKr1lNzLFAaxS+2KelA76HsH8EuXuKj7bapgq1vkn7uZgeGNuXHjHufCFmyRIk9mNfYyV7hwrPilJDQVMAsQdJS9tj2PHh3Sobt0EnKZP6DRadwy+uLVnELFybtYBHv7E4g8pr3otOgeVcbXYgQOUgVFozqqjZ+IZv7KEGlDOkZ9LR6IcYNW3HQ6s9AAQ39FbIHxJDwGCyN0rXGAAuinh7rutRYu/Bq0RTFJx7W0XUKonVYZDFYX4+AZ3OMOrsJq5CC9CUuKZ07suzFJg9Q0OT4paHRHql8phiJViXoY0IuicimsnHVyXP2Pv6f9wLubxVHKdlIf0Vvh6Gpsl6tla3zCtcQOoWrL+FPCG5+aMCmYJvHJgt0kkpMqSP7SrZHoo3ABKms1NgnF8otf1+yxjeYw/emhoQPRT1ZGS/w= X-Forefront-Antispam-Report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:SN6PR12MB4623.namprd12.prod.outlook.com; PTR:; CAT:NONE; SFS:(4636009)(376002)(346002)(39860400002)(136003)(396003)(366004)(16526019)(4744005)(6666004)(6512007)(2616005)(1076003)(2906002)(6636002)(66556008)(186003)(86362001)(5660300002)(8936002)(83380400001)(6486002)(478600001)(44832011)(38100700002)(66946007)(52116002)(6506007)(66476007)(8676002)(4326008)(36756003)(316002); DIR:OUT; SFP:1101; X-MS-Exchange-AntiSpam-MessageData: hKveRRbaOauvQ9omh08SvW27oGvQeHarmCMSPTPLHPpAO5XHBm/b70F/m4NcWz+I5KE0lm7HGjvTEM8LIFNBYESrvSA+CiOVoIB2qNIC3FI0G3hoF66Wc4z746imhWSpPXUwzUHUoC9mUNdBdl8NG5Bm8vJO+nHwCBYKhjb9r/Fx/vpCyM9cbzjk6rdrwpgwRzquWvdYj2Np/7rOoaVOoFGjU6gEyBcmobEX2k6/VpIztDMdFsnvcoFPNnIfybc8MnwG7Fi6xAgzmVm3VQG7e7mUz0tx9aDuqDP13/0p0MQgcJVDDRDuuKNpmlDIt3IbAUgmsvdxR868h/TuTdpn37myB4/QtNkf+mMh+WRp3ZQTMoJ6vx3nJJ10V/ahg1kpRTNr75LJZQlfszrgx1VgOlGre8x5x0p0MJapNuEQSOG4BAbJzZMfca3j8sFLFPpVHyRxbD962ZsrcN5Lb4MLy0+N5PgglUrbRDI7yZX0qPk07ZKcIJtsBKM1ZfR87QD348sdVLWC+ibN+8knJL8jBTIT5+TSAWv9FnMYIbd8vf7R617MfB4vpTG7LbNhpNtyZADhgg/eRKNulI7Qvvlq2GPNDTWwoUhNuqCeLGNgkGxUqKJ3XZWE927pnDwT5zzf1QTHTsrDg8geUIvyOs0C2N4LbbSD+EZgZEEgW/a0TyN4otpVcBZQFgSqR6/uzakUTGAagOW7iZqeBXQEYnw3TrWvm47t+VBj6pfFSDXgxGOggAXLc1So1qL2xtZYDFW3nj3NnUQsONb7DdcauNjkJUuWPksfmbxYG/XIZHcECxViZ7BTPWRtatJKF9f4M/NjftOyj/A+juUI9a2ZizFkQOWhT3KssKcDvYdxUgJntUumLxEx34A7nVlVFecWuLqtPqAJsXrRwzm9HvtT0BBSBk5BFKIxZcGfvpX9+cCadOKLzSjDUcYgzMzWh1v+QT7dZhVivO4W2oIY2U1LTtbVVMnZQHl0f8QRHcw9t3vsewR80QPudDtB8dglHbefVudUrMuIHZDssEAWElPaozWX77UcCjxV+dHwIbzSTKR6E5gpWCOlNeYGJ3CG/3gHgqbat6jMYc7DWon2fRt3mzznkJCRqp+ulmT4PELYFuH1ShiNgIPndC5Fhi0Tb5jFvQP+WYeSXiJSy2sWZbhuLoZ/rxNtdyYFuO1Rhi+0n/XOizN83jWGxtIIXFFLlnZzImrmlVIbBZw3owRZ03wdQiHDVZ+7yoJXuPzVGweZKaqQj3ZsweuMQm0m3CudD13hQqrTyxK5fwuu2latB2r7VKfE0tRa6rMYHauWnfZQzRTwx9P57okeVE+hm57dJBuwPA1Vw7OFxMnK5UU7EHtjsmJ4oTzY45b+KkrXc2/fTBOeBkfdkN1H76QuytUFeGVwBk3X X-OriginatorOrg: amd.com X-MS-Exchange-CrossTenant-Network-Message-Id: 776e3725-2352-45d1-f083-08d90a582769 X-MS-Exchange-CrossTenant-AuthSource: SN6PR12MB4623.namprd12.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 28 Apr 2021 15:13:18.6948 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: D8AhbSkPZTjXltqmYFmv44lt3gN0KfmfDmYtGkpPJASH3S/bNuJ4/ZEsuUK/4zjw9S5VvQkLvgcUKcspA6SJUQ== X-MS-Exchange-Transport-CrossTenantHeadersStamped: SN6PR12MB2749 X-BeenThere: dri-devel@lists.freedesktop.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Direct Rendering Infrastructure - Development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: gregkh@linuxfoundation.org, Felix.Kuehling@amd.com, helgaas@kernel.org, Alexander.Deucher@amd.com Errors-To: dri-devel-bounces@lists.freedesktop.org Sender: "dri-devel" It's already being released by DRM core through devm Signed-off-by: Andrey Grodzovsky --- drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm.c | 1 - 1 file changed, 1 deletion(-) diff --git a/drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm.c b/drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm.c index c0b9abb773a4..b9aa15f22cfc 100644 --- a/drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm.c +++ b/drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm.c @@ -3590,7 +3590,6 @@ static int amdgpu_dm_initialize_drm_device(struct amdgpu_device *adev) static void amdgpu_dm_destroy_drm_device(struct amdgpu_display_manager *dm) { - drm_mode_config_cleanup(dm->ddev); drm_atomic_private_obj_fini(&dm->atomic_obj); return; } From patchwork Wed Apr 28 15:12:07 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrey Grodzovsky X-Patchwork-Id: 12229281 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.5 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,MSGID_FROM_MTA_HEADER,SPF_HELO_NONE,SPF_PASS, USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id ED8C0C433ED for ; Wed, 28 Apr 2021 15:14:38 +0000 (UTC) Received: from gabe.freedesktop.org (gabe.freedesktop.org [131.252.210.177]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 85AE1613F1 for ; Wed, 28 Apr 2021 15:14:38 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 85AE1613F1 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=amd.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=dri-devel-bounces@lists.freedesktop.org Received: from gabe.freedesktop.org (localhost [127.0.0.1]) by gabe.freedesktop.org (Postfix) with ESMTP id A84AB6EB9E; Wed, 28 Apr 2021 15:14:37 +0000 (UTC) Received: from NAM11-CO1-obe.outbound.protection.outlook.com (mail-co1nam11on2070.outbound.protection.outlook.com [40.107.220.70]) by gabe.freedesktop.org (Postfix) with ESMTPS id 912636EB7A; Wed, 28 Apr 2021 15:13:22 +0000 (UTC) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=SaHOJckFRjLU6pUB94C+ynE+/Io+pSimJXv25+ZvA0xiI3R+s56u5y6z4I5lqCucBwlPIpXkIvW2NI71fO9kEdFIQTkkUQksI2WI5reoAq9YsBiwPyDOHRhD/D27kzX9jeT7sBCdU9v+owbJfrQjaQHnsWbrBO3Q9XNbkUixAMEfs8oXGg+C59UTodUL2ciKm/DcwM21c5Y54ZAwgldDgYrXMe6G2JW+KNwt1ckHyw/xhPDB3JOWT9kLopOJch4bZzG3e3sNy0CHrAxzqfJnKo1Ozip0PMJOhhO5iWzFTSPPd4yZx/hWtygrlwOGQvHbha+zJEsTDqEe2oYTkAo8tA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=5FEL0be6dJV8J4R16Kq8tDPjvIpvPEZ1uLevXHJGFeI=; b=XP4amdCIZX1c5LY8ey2tpCX3GCCrjigCWxYuUIianBr+QAHTrCSJ9kqag5ROzI5IinV9p5I5/MG8SY+ycrEoCyx3df+q5B+qFSSNQ/ScnD20TkBQafjFXuTuiwUmaExvmQ4U0PVCXyedf/Dgpemz7SnRqOJcbPF7c4PB83Pb0EW5OszQIsplPUEvtYXXPecWhAMPGXipFFiQGSOTO22NBLiDPMkWx5Gf7gAqT6NFRH/TW+AdRq5hN7zEoW9IywrNxTPQtPU0HjgSk6TKsWzpW8RQMckwOVaWGxIw+oqdYq7UJ+iwPiH0WPCg4diI1vMu6liJpXtuoo7nfZIzv1BKMA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=amd.com; dmarc=pass action=none header.from=amd.com; dkim=pass header.d=amd.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=5FEL0be6dJV8J4R16Kq8tDPjvIpvPEZ1uLevXHJGFeI=; b=rUQ9I5La8HDRikIZ64dEAX7TBB8BX2uH8RtX1mq4dtfwFWNQWLBJBwPZwBvz2CpOaVKR33BypbkXXSUxDZiBJczzmP3mQdtvymc/88eoNCFlhssgMYKZMnXPBzqLMfjZKMufCgISgF/LWiLHPiuJQcx98LoaDDoyZeIhqLruczw= Authentication-Results: lists.freedesktop.org; dkim=none (message not signed) header.d=none; lists.freedesktop.org; dmarc=none action=none header.from=amd.com; Received: from SN6PR12MB4623.namprd12.prod.outlook.com (2603:10b6:805:e9::17) by SN6PR12MB2749.namprd12.prod.outlook.com (2603:10b6:805:6d::18) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4065.22; Wed, 28 Apr 2021 15:13:20 +0000 Received: from SN6PR12MB4623.namprd12.prod.outlook.com ([fe80::ad51:8c49:b171:856c]) by SN6PR12MB4623.namprd12.prod.outlook.com ([fe80::ad51:8c49:b171:856c%7]) with mapi id 15.20.4065.026; Wed, 28 Apr 2021 15:13:20 +0000 From: Andrey Grodzovsky To: dri-devel@lists.freedesktop.org, amd-gfx@lists.freedesktop.org, linux-pci@vger.kernel.org, ckoenig.leichtzumerken@gmail.com, daniel.vetter@ffwll.ch, Harry.Wentland@amd.com Subject: [PATCH v5 27/27] drm/amdgpu: Verify DMA opearations from device are done Date: Wed, 28 Apr 2021 11:12:07 -0400 Message-Id: <20210428151207.1212258-28-andrey.grodzovsky@amd.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210428151207.1212258-1-andrey.grodzovsky@amd.com> References: <20210428151207.1212258-1-andrey.grodzovsky@amd.com> X-Originating-IP: [2607:fea8:3edf:49b0:7212:f93a:73b0:8f23] X-ClientProxiedBy: YT1PR01CA0142.CANPRD01.PROD.OUTLOOK.COM (2603:10b6:b01:2f::21) To SN6PR12MB4623.namprd12.prod.outlook.com (2603:10b6:805:e9::17) MIME-Version: 1.0 X-MS-Exchange-MessageSentRepresentingType: 1 Received: from agrodzovsky-All-Series.hitronhub.home (2607:fea8:3edf:49b0:7212:f93a:73b0:8f23) by YT1PR01CA0142.CANPRD01.PROD.OUTLOOK.COM (2603:10b6:b01:2f::21) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4065.25 via Frontend Transport; Wed, 28 Apr 2021 15:13:19 +0000 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: d9d01b08-cd96-4b03-eeb3-08d90a582881 X-MS-TrafficTypeDiagnostic: SN6PR12MB2749: X-MS-Exchange-Transport-Forked: True X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:5516; X-MS-Exchange-SenderADCheck: 1 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: G6GAeYG7R8XbIJYNM79bE/7UfV1goPj15EZtgbmcBeHch34F7xsTyeODyCLBBjNw7HIpJa9xQr2dMMQOS8cO3GVqIDLbTOdAP8gVdSwoyrJ1Si2wR+1FoSCTAgJo3XAdTpzkCrMe6LPUlZnUCKgCr47mbiHQ14XJDaVcxFFL2OqnO62ImL04yr2wjlUDC8kUakieE1GBjIvJ1rJC/JheKWE/s4eHuA7vtVxe1cDpK/RCw2uq+yjBv20L3rfL9Jjo6GmEAUclAHcdA0gs74xNe3dpq6e2zUQuFfUuCBb1TUU3kKM5/Jga02IjdMIppmHdqfxlX3a5j2WgDyr1JHCuLHgPN8f0uclx+r+169U26iog+a9VKH7r36GKTbF56tYddii7qC4+WeDVr+Dq6GSs349Ib5zYbEDWAGLu4W/KNi5O3pZhAOSm4OXamHmMtF5EgyQ3GLLC9MS0TyoTzpuMpBAyWzCDAIUcovKLZrensv5VRqifpZ/rAGTDRhildJCxiGL0ckQns/Cz0fOmshWU0pttqLTpGkWzyMtdo3KeV6ytD7gKfMjklubODLBb48Gie0/Eo3gH2fgZVJqcGisfr4hBnA7PBQQzH7WK4i7MDxo= X-Forefront-Antispam-Report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:SN6PR12MB4623.namprd12.prod.outlook.com; PTR:; CAT:NONE; SFS:(4636009)(376002)(346002)(39860400002)(136003)(396003)(366004)(16526019)(4744005)(6666004)(6512007)(2616005)(1076003)(15650500001)(2906002)(6636002)(66556008)(186003)(86362001)(5660300002)(8936002)(83380400001)(6486002)(478600001)(44832011)(38100700002)(66946007)(52116002)(6506007)(66476007)(8676002)(4326008)(36756003)(316002); DIR:OUT; SFP:1101; X-MS-Exchange-AntiSpam-MessageData: PrGeHb5s5c+48U0wNqWAwlYImvCey+jXYHPeXzfAlOVEvGVJ2R73e82tsOWbD58KR6qA7+t7J4pA8CTpH+xh/ERoHIAhUANa+fryxw5+FRI19Js5SJkpCWHmtyLpWe3yfvG44I1KWQmAiKIxY0ewZHuvwNlrTw3pZBQEDLnSx9yMc3zaBPwxXwzCWLgKyNGw1WRKIXv6tsEo8LeKIKsspBQseBRNR+wZMQN5zknOTzfmNT0sRa89IXDlJXSlOGJRaAKC7LyFtQ091PERNo7WSXekh7wugTAaCujJzsenZ2VozHQCNx+/IN2iW4f6wAu0G+o/LHf+fcKwYmjpaTkb+FNVa4RCukx4LzaT8TaTrunaprSvM893bcIw6opGWCgDdZXO3p3x2D1hSidTHwDoYDxFx2d2xAamwIZcbQHYC2B8zN/yteGbKy0kpKOk6FRAfWd91CM0V1HI3TEbEmWaqw+koW/5uXFKNk1KcUJyZRg6CkRGCVYQQpJckxYH/qqgCuGVbY/IYTc8769OdWK1t+pM15cQIi/aDJNupXGhoOAmfAJeAlaQoyq4KHSuX0Jg3azLETK1QbB23SoCrIcncQBJCkj8LJTkyO+rwVYdTh/LcZZF2ph5Ke7/GKHprhc+X5yZIfdJ4QMofsOubjQbqdgfYwkdvpKl5ukMXR69t65ffOBlZe/FGR0h5/aKVjaQ3x37LcGNm30rSiALbuCygEKUQi29NrUDX4MbgVMPiczzHrjEs1D3ArJvyvJLJctrD3hW5uj3YNTO0KvNATkyW5bKLoLFA2ZbZUOar+AJZa5iHldliPFs+KWt0a9Te3VxH7AWYX3pBGRaLERSjy5in9qGB41d2Rb/hLPhxY+nrPTKuTLYuG+eMwjIh/2Bfbxf35oa3qV6tksYKFsQPT40bSMTCqulCHjX8D2RT8PPDAfV186oQbo+9UiFW2/x2TnMjYwytjau/C70EuqLmzWsPcdAicDdemZVJXE5lm5qqbAdtMK2At0/JcwQXXnhgEtHFE65Bem2cPLlnNU3tMYV4o66blO31RHMk0CtgO2fMHxPUDXOxBy7gF+kxr5TwH7a99xgm8cdNsw2Q+rWPix4I3DGBrpfLezBy/QuqhCdwcDzqqeFHh5MK0DpqaD2am2xHzSutSBBWG145AF4MAt2sHR4BbH8RZvwZ6SN+WimXpaRNK4Cb7OHMS7fdEWauAK9cD0/fBEoX1IkLBZfWuxcNwHBqBTwCn3HhJ71u6sSS9IDwzrADZ/OZFkNdhg40AkeM8sOKfaVBkHBIMkKavLdsYUnpP0SRhfI7bjE9U6hUCUc7a4ZaWwBH8nsofaIhhfjC4u0MlJ9oETO2w3r4VULqUBvXNy69XEP0jKNeFvMEt/7dK+5ciGcug5/xa9CmcBG X-OriginatorOrg: amd.com X-MS-Exchange-CrossTenant-Network-Message-Id: d9d01b08-cd96-4b03-eeb3-08d90a582881 X-MS-Exchange-CrossTenant-AuthSource: SN6PR12MB4623.namprd12.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 28 Apr 2021 15:13:20.4808 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: Z70l5v/cgqpWRsTmNRO1i3fEHSy2TBiVb0g3AuHgnPBrwKwDfhXzRJBox8ySQcM4NXg0aY5jibBj9ynxmOR2ug== X-MS-Exchange-Transport-CrossTenantHeadersStamped: SN6PR12MB2749 X-BeenThere: dri-devel@lists.freedesktop.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Direct Rendering Infrastructure - Development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: gregkh@linuxfoundation.org, Felix.Kuehling@amd.com, helgaas@kernel.org, Alexander.Deucher@amd.com Errors-To: dri-devel-bounces@lists.freedesktop.org Sender: "dri-devel" In case device remove is just simualted by sysfs then verify device doesn't keep doing DMA to the released memory after pci_remove is done. Signed-off-by: Andrey Grodzovsky --- drivers/gpu/drm/amd/amdgpu/amdgpu_drv.c | 6 ++++++ 1 file changed, 6 insertions(+) diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_drv.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_drv.c index d0f34f230ef3..f3e8fbde62a0 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_drv.c +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_drv.c @@ -1258,7 +1258,13 @@ amdgpu_pci_remove(struct pci_dev *pdev) drm_dev_unplug(dev); amdgpu_driver_unload_kms(dev); + /* + * Flush any in flight DMA operations from device. + * Clear the Bus Master Enable bit and then wait on the PCIe Device + * StatusTransactions Pending bit. + */ pci_disable_device(pdev); + pci_wait_for_pending_transaction(pdev); } static void