From patchwork Mon May 10 16:36:10 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrey Grodzovsky X-Patchwork-Id: 12248663 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-18.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,MSGID_FROM_MTA_HEADER,SPF_HELO_NONE, SPF_PASS,USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 7ECF7C43460 for ; Mon, 10 May 2021 16:36:51 +0000 (UTC) Received: from gabe.freedesktop.org (gabe.freedesktop.org [131.252.210.177]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 4281F6115C for ; Mon, 10 May 2021 16:36:51 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 4281F6115C Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=amd.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=dri-devel-bounces@lists.freedesktop.org Received: from gabe.freedesktop.org (localhost [127.0.0.1]) by gabe.freedesktop.org (Postfix) with ESMTP id 412886E8A9; Mon, 10 May 2021 16:36:46 +0000 (UTC) Received: from NAM10-MW2-obe.outbound.protection.outlook.com (mail-mw2nam10on2089.outbound.protection.outlook.com [40.107.94.89]) by gabe.freedesktop.org (Postfix) with ESMTPS id 3E5D06E88B; Mon, 10 May 2021 16:36:45 +0000 (UTC) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=Uv+1MnRWnMteCZwc4yplDrgqfPyNPSPe+8xu7PfgadEMxK2pzKT5xi9IuYuKCQ+Ubg8S/5fNlkdvZxpXXdaqVhRFFWT24mgm76iukB3UR49yMmeiCvDCfB5QIFB/0U4NjbG9+dmgtNTv5cw8dokTvwSN2fekT+pMvHImzlOAOveasM6LYZ0UUKRIvgeFDCw3aXN3byyoMOAS2fq7DOkJgY4BkEEWDlML+1Sa35wrrHVfXVDJNn1qUiXEU6IIYuUWSp0C6TK2aL3vV/bYsEfsaahvfHPMIL1XjwLJtWTuCIRa11oWJnvoFoYrtx7GWgeOujDcE4xB1n+Voy69KVZmpg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=PG9hxNn2BX9GkWOFWtKfFZQJq6AU3DE2aXsSC0AMjT4=; b=c9ZjCwlX8iNH1StIr5yks4dq/UVAmTqH55xdxygmOGugzG2i1FswbCrfGgMp9IlVSgKeRRMNkN5s1Jum61Z7Mfrfo1355cxmP6b+fbZnntsV5g4Vd1cRb4kurvb19guQgfNBjvOGoCzruIglpHNKvwmpzPIZ6Bnr5EQLVNpvyPeJOsuJeAFzyWgzx8hgzor4DzksbUyasx7OoRGCH6FMo6+hE9Pnkz7/sX7GrLTfkW252c9EBlQjSN+e7KXRGdXWXhl5xrQ8w5pOFWrulGubGqci33ZiwVq02j5uMXWDbMaSg5NvZfZTKqFIek9f6CcCy1jpCd1thnMJJS7U/jMisQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=amd.com; dmarc=pass action=none header.from=amd.com; dkim=pass header.d=amd.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=PG9hxNn2BX9GkWOFWtKfFZQJq6AU3DE2aXsSC0AMjT4=; b=aXXEHFZBqDgWwUbsTKHkZFkKlO9fv2h8vPXg2q5qQCBT0uxpbcbXoOe1eQYmyP4k8Qk6AdIf4p9+xY4fVNlyzIM0T9nBv2yuj2ZUZp22/pgrWQjPS9tfPG2HBFO4ph70bWpq4gYEVY4ce9xYkfXO7XNULtPC51ndkS9It0YiG04= Authentication-Results: lists.freedesktop.org; dkim=none (message not signed) header.d=none; lists.freedesktop.org; dmarc=none action=none header.from=amd.com; Received: from SN6PR12MB4623.namprd12.prod.outlook.com (2603:10b6:805:e9::17) by SA0PR12MB4462.namprd12.prod.outlook.com (2603:10b6:806:95::22) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4108.25; Mon, 10 May 2021 16:36:43 +0000 Received: from SN6PR12MB4623.namprd12.prod.outlook.com ([fe80::ad51:8c49:b171:856c]) by SN6PR12MB4623.namprd12.prod.outlook.com ([fe80::ad51:8c49:b171:856c%7]) with mapi id 15.20.4108.031; Mon, 10 May 2021 16:36:43 +0000 From: Andrey Grodzovsky To: dri-devel@lists.freedesktop.org, amd-gfx@lists.freedesktop.org, linux-pci@vger.kernel.org, ckoenig.leichtzumerken@gmail.com, daniel.vetter@ffwll.ch, Harry.Wentland@amd.com Subject: [PATCH v6 01/16] drm/ttm: Remap all page faults to per process dummy page. Date: Mon, 10 May 2021 12:36:10 -0400 Message-Id: <20210510163625.407105-2-andrey.grodzovsky@amd.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210510163625.407105-1-andrey.grodzovsky@amd.com> References: <20210510163625.407105-1-andrey.grodzovsky@amd.com> X-Originating-IP: [2607:fea8:3edf:49b0:6a5:47b8:e610:f6a3] X-ClientProxiedBy: YT2PR01CA0020.CANPRD01.PROD.OUTLOOK.COM (2603:10b6:b01:38::25) To SN6PR12MB4623.namprd12.prod.outlook.com (2603:10b6:805:e9::17) MIME-Version: 1.0 X-MS-Exchange-MessageSentRepresentingType: 1 Received: from agrodzovsky-All-Series.hitronhub.home (2607:fea8:3edf:49b0:6a5:47b8:e610:f6a3) by YT2PR01CA0020.CANPRD01.PROD.OUTLOOK.COM (2603:10b6:b01:38::25) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4108.31 via Frontend Transport; Mon, 10 May 2021 16:36:42 +0000 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 1335a1e4-36ba-4dba-2763-08d913d1cbc0 X-MS-TrafficTypeDiagnostic: SA0PR12MB4462: X-MS-Exchange-Transport-Forked: True X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:6790; X-MS-Exchange-SenderADCheck: 1 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: xa49JixO2HyZhn/YFsiz1JAdN8GXcTMH8zwC0lJTSZVVHTVBcK6h9jp9Vsj/psb+STTqioj2JECxYKM9jGRLvLiqAqfz8wxX64GBife7181T1fyAg3goTA2Cey+Zu7kXCo1OYH1yKRWhBNX9NhlTPjYf7ItJShVSBJWOGcSY4V4Px1MxZ2vI97mUAygxlCXoBlpdXclo7ZxmyR1rnwj9A4lpFaaCR0d6sos5vySVffdSttVLcEguKyDnPa/O2H7SDyPXMieWOdy0W/S+ZZQu6x+Yu1cqMog0aVFlfDp7jSX7h9/gfd4WHyOevx22Huq2K8NBEq9YawJOg2r2jJmZMUjR3K9NGYntMKLPy9a8tMOcg9mD6p2vGA3to9X5zFrAtYUEzLfaiDnmdyIWt+69niuyWFUH/DdAI1O7By4rkP8UraWmlbkHJiWmVDUZaFlHEK/mIV/F4/6Fj2F4MPXx//9reCoiOKIuX9kyMHWgJACUv/WfjZ/8RSYSfpe6+6QRm38Zb6cofkCy4yQGVu+Nm+4LxxclNCfHsNhWR0/ieWsLD61Hfeey+emLx+y3qusUL7Jk+RRfb5+cCoFg4h3yQnVKXXwbuySYYC6sqOwDBoI= X-Forefront-Antispam-Report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:SN6PR12MB4623.namprd12.prod.outlook.com; PTR:; CAT:NONE; SFS:(4636009)(366004)(136003)(39860400002)(396003)(376002)(346002)(86362001)(6506007)(38100700002)(52116002)(6666004)(44832011)(8676002)(36756003)(5660300002)(478600001)(8936002)(2906002)(6486002)(66946007)(2616005)(66476007)(66556008)(1076003)(316002)(6636002)(6512007)(16526019)(186003)(83380400001)(4326008); DIR:OUT; SFP:1101; X-MS-Exchange-AntiSpam-MessageData: CTZ1QAS1hUwICbdgwHg6EJwQVrfYz5pGwiiFpLoGXHfwwxY1QeE6GLyQl9totOP08i6OVkIVixaGQbb0fTkLncLhnQ2VK683DiSFhtMQsTBt4351opQ1acKfdT2XMNoiRylqJ6iY+v4EU5WjrMJxO1H1jamjIawDgwMYANpBg1CpCJLKJndRTcT/VKw3ollP4zweR6yzHxztoIBjcGMParKhpX9jet9sC46IxhVCQ9PiG0aM4AP+Bg9/vaBe4Cq0w3mS3z5ujK09ImMWyNjmkHEl8nb2v3HFtPfbejuQPzWQbnftXlRj90dGJHhBQw6sRysMm0bM0/3USazOW3X6y45vICzcVrS6xKAQIYA10kieDVfsqoTfXx/hhjJ/j52qeC9oaEHRYnDgP8vduAZrccw0SuwCe2ABjf5/dYgiEX5d61c0ZhRkpUPHoe8bfRf8WXGuFOC5vubh+kUd/fg3EzMI3j4AC4GyFMuZPz6QznHhK8hQS1CcEsnAt/+9DgGjni2Fcdxx8KqTI2POjM4aY/FKiPYXYB1EOPnm+m0xHuIhZJeTmRY98EVnVNqw+9SXjCSX3QPrObvfIoTvgOMlYcbh3JS72sML2tL4hD382baWBcjwDay/xGZhEsjtb5ldxe1s5UgkgMwJFv3RK0IQRpXY6Ei1WylByqcty6I1zOHuF6r2xd9l/EE6OXS5sYXG867nlFaCDLCOOlKzVknPY0ueltqmG8JOBImrcBu67vA247gBvXqW/BUJKJGdAkM58juRtB/7SZbmUX8Sb7gEiwAGmMd6hvo8byGwIpUVLZuhZ8iUzTBaHsxYthlztElVjhouBSVduQVS5axmP9wdF697dhRsK+dwVgIK7BmeywAeEepEg5lyEy1W8blnpkLPXszx8NcqOhZkjlJI4r1DYh9DZZZvVwlM11N+Bz502JNA3la01/w1Iifk0Diz1S0pJuUVeyUpONDqTyLj11bKM55KR5eWNIPXubL8ytteCQSXC3jeUEseR6fjySrQeVB7tLmQ2EDbN0Z5P3KeYz8am6JXeDYWwyLuoCdiwwYylcFoSXDumNuqVvlHaxMz+J1k7ltheYAFYPxLSReTvOMWdFMS2zQ2EUHIbpoKMCt2Y0VsRwG3RHE17TfNUIxyG+7OfMUfM4CpwyD/ocf8h8ZuVg36E7ecynSN5Lh9ETUhg9xB3roFg7EaEllenyAUGd98NwTY7mss6WNedkI1rndl1+XSBpBd47FBLXhwKLj9XrAAtIRwVsVddiuGtMJBlgTx8yE5FHEc3c5mcPlzg/7zVfx3HLMUIa/34Ekxu2cS9Quhp45nu0OF7n+6U23ujPC0wIYylAcev0JapDuUdDGfmMXPnv9gPSpffgA0lOpg05AAYrMuNFCfUjiSfSpFi0KX X-OriginatorOrg: amd.com X-MS-Exchange-CrossTenant-Network-Message-Id: 1335a1e4-36ba-4dba-2763-08d913d1cbc0 X-MS-Exchange-CrossTenant-AuthSource: SN6PR12MB4623.namprd12.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 10 May 2021 16:36:43.8465 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: BXBVsxul2R5Kj0vjqikQvLrfjP3SLadrsPLHvTNrbEodEOsBZvIinsOYqKsdJGeG0egX3LpsG62qkYw0UUzgQw== X-MS-Exchange-Transport-CrossTenantHeadersStamped: SA0PR12MB4462 X-BeenThere: dri-devel@lists.freedesktop.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Direct Rendering Infrastructure - Development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: gregkh@linuxfoundation.org, Felix.Kuehling@amd.com, helgaas@kernel.org, Alexander.Deucher@amd.com Errors-To: dri-devel-bounces@lists.freedesktop.org Sender: "dri-devel" On device removal reroute all CPU mappings to dummy page. v3: Remove loop to find DRM file and instead access it by vma->vm_file->private_data. Move dummy page installation into a separate function. v4: Map the entire BOs VA space into on demand allocated dummy page on the first fault for that BO. v5: Remove duplicate return. v6: Polish ttm_bo_vm_dummy_page, remove superflous code. Signed-off-by: Andrey Grodzovsky --- drivers/gpu/drm/ttm/ttm_bo_vm.c | 57 ++++++++++++++++++++++++++++++++- include/drm/ttm/ttm_bo_api.h | 2 ++ 2 files changed, 58 insertions(+), 1 deletion(-) diff --git a/drivers/gpu/drm/ttm/ttm_bo_vm.c b/drivers/gpu/drm/ttm/ttm_bo_vm.c index b31b18058965..e5a9615519d1 100644 --- a/drivers/gpu/drm/ttm/ttm_bo_vm.c +++ b/drivers/gpu/drm/ttm/ttm_bo_vm.c @@ -34,6 +34,8 @@ #include #include #include +#include +#include #include #include #include @@ -380,19 +382,72 @@ vm_fault_t ttm_bo_vm_fault_reserved(struct vm_fault *vmf, } EXPORT_SYMBOL(ttm_bo_vm_fault_reserved); +static void ttm_bo_release_dummy_page(struct drm_device *dev, void *res) +{ + struct page *dummy_page = (struct page *)res; + + __free_page(dummy_page); +} + +vm_fault_t ttm_bo_vm_dummy_page(struct vm_fault *vmf, pgprot_t prot) +{ + struct vm_area_struct *vma = vmf->vma; + struct ttm_buffer_object *bo = vma->vm_private_data; + struct drm_device *ddev = bo->base.dev; + vm_fault_t ret = VM_FAULT_NOPAGE; + unsigned long address; + unsigned long pfn; + struct page *page; + + /* Allocate new dummy page to map all the VA range in this VMA to it*/ + page = alloc_page(GFP_KERNEL | __GFP_ZERO); + if (!page) + return VM_FAULT_OOM; + + pfn = page_to_pfn(page); + + /* Prefault the entire VMA range right away to avoid further faults */ + for (address = vma->vm_start; address < vma->vm_end; address += PAGE_SIZE) { + + if (unlikely(address >= vma->vm_end)) + break; + + if (vma->vm_flags & VM_MIXEDMAP) + ret = vmf_insert_mixed_prot(vma, address, + __pfn_to_pfn_t(pfn, PFN_DEV), + prot); + else + ret = vmf_insert_pfn_prot(vma, address, pfn, prot); + } + + /* Set the page to be freed using drmm release action */ + if (drmm_add_action_or_reset(ddev, ttm_bo_release_dummy_page, page)) + return VM_FAULT_OOM; + + return ret; +} +EXPORT_SYMBOL(ttm_bo_vm_dummy_page); + vm_fault_t ttm_bo_vm_fault(struct vm_fault *vmf) { struct vm_area_struct *vma = vmf->vma; pgprot_t prot; struct ttm_buffer_object *bo = vma->vm_private_data; + struct drm_device *ddev = bo->base.dev; vm_fault_t ret; + int idx; ret = ttm_bo_vm_reserve(bo, vmf); if (ret) return ret; prot = vma->vm_page_prot; - ret = ttm_bo_vm_fault_reserved(vmf, prot, TTM_BO_VM_NUM_PREFAULT, 1); + if (drm_dev_enter(ddev, &idx)) { + ret = ttm_bo_vm_fault_reserved(vmf, prot, TTM_BO_VM_NUM_PREFAULT, 1); + drm_dev_exit(idx); + } else { + ret = ttm_bo_vm_dummy_page(vmf, prot); + } if (ret == VM_FAULT_RETRY && !(vmf->flags & FAULT_FLAG_RETRY_NOWAIT)) return ret; diff --git a/include/drm/ttm/ttm_bo_api.h b/include/drm/ttm/ttm_bo_api.h index 639521880c29..254ede97f8e3 100644 --- a/include/drm/ttm/ttm_bo_api.h +++ b/include/drm/ttm/ttm_bo_api.h @@ -620,4 +620,6 @@ int ttm_bo_vm_access(struct vm_area_struct *vma, unsigned long addr, void *buf, int len, int write); bool ttm_bo_delayed_delete(struct ttm_device *bdev, bool remove_all); +vm_fault_t ttm_bo_vm_dummy_page(struct vm_fault *vmf, pgprot_t prot); + #endif From patchwork Mon May 10 16:36:11 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrey Grodzovsky X-Patchwork-Id: 12248665 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-18.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,MSGID_FROM_MTA_HEADER,SPF_HELO_NONE, SPF_PASS,USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 2565AC433ED for ; Mon, 10 May 2021 16:36:54 +0000 (UTC) Received: from gabe.freedesktop.org (gabe.freedesktop.org [131.252.210.177]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id C9F576115C for ; Mon, 10 May 2021 16:36:53 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org C9F576115C Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=amd.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=dri-devel-bounces@lists.freedesktop.org Received: from gabe.freedesktop.org (localhost [127.0.0.1]) by gabe.freedesktop.org (Postfix) with ESMTP id CEDD66E89C; Mon, 10 May 2021 16:36:47 +0000 (UTC) Received: from NAM10-MW2-obe.outbound.protection.outlook.com (mail-mw2nam10on2078.outbound.protection.outlook.com [40.107.94.78]) by gabe.freedesktop.org (Postfix) with ESMTPS id E0D806E8AC; Mon, 10 May 2021 16:36:46 +0000 (UTC) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=MQV9iUMCxqOKhtAsOR+uaTWWPmechG103HPlWN9QNdroRoQ5NjLkbm/RzgibckDN9RlAAA3U9ZtlcZRxrtwvLhE0sjwfX+B3KFTrWXlsJXPy0GO2TGJHUuUWLBzlyM8IQ2q+tFBfpFPxvY2DjeKEtCX0bQ6E+uglWKXL0CpTef4V9NWziy955XT0fIKSMmreRKaUQ8nzl63F2NCIpXQ9ih1Ni/6tTu9ZSLbKChQvVxUQ9rvS0afqC+BGz7k0Iyf4lYm1U83l9w+8N6HH3DeEFJ9okbqjVWwMkt+0Cl5qDWYKVoalfojVkhux8TpPlGrBtllkreIV0Rap1PiM5Wtmnw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=BPUKus8UfUiKGKixY69zLZagZ3iEboaQcf3OLBd+5uo=; b=U+KUouJruF6iv4GXsBgcSUzYhmHCKUfL5NGJNLkbPUHRnpqk9TWXXqdQyeSVUAWgffboB0dJHftcw7qiGzCyxEZsOPsNVa/gduNoRmcUJJ1qSvxGpvHXaMLUMfKrVH2MW3zotyrLJofeMyhiEo5khsP4qR7tv16Ad7g061ZSbQ+LVQvXDLE8w0Ja6BY1IWx+6h3l05yurYDU4IOH8+SRynwHqcRKVt4+ls53CCew/ZsMkprsTCdIGldfZLOMpJr1u3FruMxvfasmk6auDPyb2R23ZT2/H+s8zdu92+9vMVJ9KKhYO5AAb+9+yijsggDRgj78l7Cg/Wfyt+8Q3p8UBg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=amd.com; dmarc=pass action=none header.from=amd.com; dkim=pass header.d=amd.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=BPUKus8UfUiKGKixY69zLZagZ3iEboaQcf3OLBd+5uo=; b=KOkWf5CKi2GRJR3PYi1xU0pwRDn4wRMY4wu2c31vxdXTMxozhZspBPGCBPj/X4qhCRLVPCPcntUk+9IDqMQS654Nr+umR1v6ZWfUiALKe1cJ7dlLcmIreRTj4toDHMFb1AM0ArZ3B/bCYvC7xtbF36MvWia/yQcwzwLmKE3ElTg= Authentication-Results: lists.freedesktop.org; dkim=none (message not signed) header.d=none; lists.freedesktop.org; dmarc=none action=none header.from=amd.com; Received: from SN6PR12MB4623.namprd12.prod.outlook.com (2603:10b6:805:e9::17) by SA0PR12MB4462.namprd12.prod.outlook.com (2603:10b6:806:95::22) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4108.25; Mon, 10 May 2021 16:36:45 +0000 Received: from SN6PR12MB4623.namprd12.prod.outlook.com ([fe80::ad51:8c49:b171:856c]) by SN6PR12MB4623.namprd12.prod.outlook.com ([fe80::ad51:8c49:b171:856c%7]) with mapi id 15.20.4108.031; Mon, 10 May 2021 16:36:45 +0000 From: Andrey Grodzovsky To: dri-devel@lists.freedesktop.org, amd-gfx@lists.freedesktop.org, linux-pci@vger.kernel.org, ckoenig.leichtzumerken@gmail.com, daniel.vetter@ffwll.ch, Harry.Wentland@amd.com Subject: [PATCH v6 02/16] drm/ttm: Expose ttm_tt_unpopulate for driver use Date: Mon, 10 May 2021 12:36:11 -0400 Message-Id: <20210510163625.407105-3-andrey.grodzovsky@amd.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210510163625.407105-1-andrey.grodzovsky@amd.com> References: <20210510163625.407105-1-andrey.grodzovsky@amd.com> X-Originating-IP: [2607:fea8:3edf:49b0:6a5:47b8:e610:f6a3] X-ClientProxiedBy: YT2PR01CA0020.CANPRD01.PROD.OUTLOOK.COM (2603:10b6:b01:38::25) To SN6PR12MB4623.namprd12.prod.outlook.com (2603:10b6:805:e9::17) MIME-Version: 1.0 X-MS-Exchange-MessageSentRepresentingType: 1 Received: from agrodzovsky-All-Series.hitronhub.home (2607:fea8:3edf:49b0:6a5:47b8:e610:f6a3) by YT2PR01CA0020.CANPRD01.PROD.OUTLOOK.COM (2603:10b6:b01:38::25) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4108.31 via Frontend Transport; Mon, 10 May 2021 16:36:44 +0000 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 015c91f1-5c66-42bd-7447-08d913d1cccf X-MS-TrafficTypeDiagnostic: SA0PR12MB4462: X-MS-Exchange-Transport-Forked: True X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:489; X-MS-Exchange-SenderADCheck: 1 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: q1Z800tohbKW7zvn4Bbh1XA7Fftu3U5Amrz9uyWw7SKDQ7lfKhrCA4dDddHSUHbRnyCxc/piArC0L3liubpcRgdni9YaC1FRa7tNusmMlRhhHXynnK0PDtH+2XmLbKeR3NA/ZmURLptvDo17HMaqFv3vfRv4lDJ4kD3vlIx44qdWQVx14EqWvnbta6g8HDVhEkCml3BiXdQ4LKpzTk17afRJsvueaCXpbIkUlBorML4sGZ7QpyE+FEGOo7Fvo20E/IhBjck5M5vh1iQle03TbwKlsQ92m4364cgXHTSLaLvxp018kOmkO6Rln2zSA/9u6XwAmjN+F1gPo9ZrvYXQSDFyfv7WbJgdb0gMD6nzSrJZeiMPojBqlyA9W6s81lcE1pK471u8MGxs9ONYZD5+I569nhllqRC1YZH+MtMR57Ni+BRdO2qrs0mAFsO0OaFSFVN1ocmAG/Ne32Vhfuux/AHqbnnz2rSQYHq9j2KLABLliCfkZ6UYZ0kjONNxUMiOYQBUnOpKIASzs+dchSWQ35Yxg6CNNP0DGgoDjHkP25JOSYwoMfIiE+S6MsDY2MMFVmSsJ23AkI0o1QPzLVyqnVDwxj1RKfWy8R2KvrqGGGg= X-Forefront-Antispam-Report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:SN6PR12MB4623.namprd12.prod.outlook.com; PTR:; CAT:NONE; SFS:(4636009)(366004)(136003)(39860400002)(396003)(376002)(346002)(86362001)(6506007)(38100700002)(52116002)(6666004)(44832011)(8676002)(36756003)(5660300002)(478600001)(8936002)(2906002)(6486002)(66946007)(4744005)(2616005)(66476007)(66556008)(1076003)(316002)(6636002)(6512007)(16526019)(186003)(83380400001)(4326008); DIR:OUT; SFP:1101; X-MS-Exchange-AntiSpam-MessageData: HmudEYT8XJ81BJAeUtRMYGEf+SXkrhMkGC7sxN+XsekuG+Q1GmdYFDrMB7YMnmABSdVyLAi4gPpZE32g/rzxX8J2cDEYPlazN8mUL7+o6rKDXbCABzLKZ6s4eiVrUxWlwT2W3y3LSqalKY91/SZGbjM/UnIm9/r/QwP2RaP6zQX4p349eBVE1MYNfHxNlxZq2abSY9U9Mnz+fLUYYarSCMvr7KLxajDueG3Ve3NpO/M66HVax6stCfREVtRMn8fCQs3GgoTRt9fyQY2rqOHVRyg8qa50Z2+XsKVcbfPYjayz6jJU+ugOz98YkAdklB4dibDF5zbPpA1cleHtdkKpodk1vNrmKkLICpbwjncY3Y5vfFGUk+wl+XaNwCgEy/C42ydflMnUxZWzzIhYIUHfzW5i5GjQG4QivRuaJOizZvnmcBMX4gdrxjq1C9BpnpCbkU+fHyWEJGr7Q0f0P5R2FCrk87xyN2BeiZgBv6PPMiSCaShRtw/f3OY5mG2BgSS/CMzRMMXfo9OP3BeYdfqxwnrkXlG4QVwonYjS4/wt7mXzn4R2uKSZlaFBB3q+DQJwdqXkbEDWXtliBEn906aN2EmMVnWmImjvn+rTSogN20FUVfMqtyvbgzlOYuZMZflMTEmYkx1dB7C+UcK/hiVcy0gHw77BMu/9NxwYZ24K5Ev/GYaMC5YH34IiJfiiAr4cxG35jxm0mKh35I3J5kfrtfreGxt/cMooYNW3XDPZKe7wOtHwj2tHbQ55ZkxPZUxHVXYNlQu6V9hqpsr/J9vkSj+eWbgXpgW2Ni1Sn2e1OQyXQCr9o2EE6UKr2BZF49vNCDSCNnOqnU6P/QhrVR1RLPG6O+OHHjIZoQl6FZ5VBcAnpFywFtOldIa4oZc17Izk3ChsMxdlggdyZdyCIS0ET3FKqKty5Nu4q0czEIC9StbONDbZcTfCOq0CGTKhds7PuSlSQnZgjCePKwqyilSN2wHXjzFwGyH5HPMtxZk58tl2j18yZIPOJ5ppx+7NlfuL1qTNEUEPd3erIo5YnevdKoOLGJkHLQYS8c+89cbh4WVYXc+souVxwG5pmM0ZK70q1OUdsvRfQ1ZAO1M3LUJd9dEBbVSsyVoyVN0vOCX1H5RiquCYSaPNwMmuGhkETEXylk99HI4k0iYLVG2JenUgcM8qFkxJYto2zjDTDkzV3oM38t1Yl612Qj5VroUcrMMmqt4SgnxpuMrNkAji/K5Z+zuGiYW1DRQskx+WSjNt2VwHlJlLc/PEKJ7SYt6ATWuDAoKaxurIGK3dRjWJzEzfrjh2GU3iD8VVs59NV2J4HezCaNwCNV9IkYgFUe2OWMNXirlLCTYEGIZ5WwRwaIzbwHoEbRqYFU0TvU1IJvY/dkCdmrXCPTb8fspDeE5IhQ28 X-OriginatorOrg: amd.com X-MS-Exchange-CrossTenant-Network-Message-Id: 015c91f1-5c66-42bd-7447-08d913d1cccf X-MS-Exchange-CrossTenant-AuthSource: SN6PR12MB4623.namprd12.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 10 May 2021 16:36:45.6965 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: ZBWfBQ3Q3dFKR0PaerZXYIbOO8wRaOI2BvOadpf/U8rLOQnCHVHUidcP4+C5cvGXAaH3cSX5ixtMt04uERtJqQ== X-MS-Exchange-Transport-CrossTenantHeadersStamped: SA0PR12MB4462 X-BeenThere: dri-devel@lists.freedesktop.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Direct Rendering Infrastructure - Development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: gregkh@linuxfoundation.org, Felix.Kuehling@amd.com, helgaas@kernel.org, Alexander.Deucher@amd.com Errors-To: dri-devel-bounces@lists.freedesktop.org Sender: "dri-devel" It's needed to drop iommu backed pages on device unplug before device's IOMMU group is released. Signed-off-by: Andrey Grodzovsky --- drivers/gpu/drm/ttm/ttm_tt.c | 1 + 1 file changed, 1 insertion(+) diff --git a/drivers/gpu/drm/ttm/ttm_tt.c b/drivers/gpu/drm/ttm/ttm_tt.c index 539e0232cb3b..dfbe1ea8763f 100644 --- a/drivers/gpu/drm/ttm/ttm_tt.c +++ b/drivers/gpu/drm/ttm/ttm_tt.c @@ -433,3 +433,4 @@ void ttm_tt_mgr_init(unsigned long num_pages, unsigned long num_dma32_pages) if (!ttm_dma32_pages_limit) ttm_dma32_pages_limit = num_dma32_pages; } +EXPORT_SYMBOL(ttm_tt_unpopulate); From patchwork Mon May 10 16:36:12 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Andrey Grodzovsky X-Patchwork-Id: 12248667 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-18.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,MSGID_FROM_MTA_HEADER,SPF_HELO_NONE, SPF_PASS,USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id AB688C43460 for ; Mon, 10 May 2021 16:36:57 +0000 (UTC) Received: from gabe.freedesktop.org (gabe.freedesktop.org [131.252.210.177]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 72B9F61107 for ; Mon, 10 May 2021 16:36:57 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 72B9F61107 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=amd.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=dri-devel-bounces@lists.freedesktop.org Received: from gabe.freedesktop.org (localhost [127.0.0.1]) by gabe.freedesktop.org (Postfix) with ESMTP id 82F116E8AC; Mon, 10 May 2021 16:36:50 +0000 (UTC) Received: from NAM11-CO1-obe.outbound.protection.outlook.com (mail-co1nam11on2050.outbound.protection.outlook.com [40.107.220.50]) by gabe.freedesktop.org (Postfix) with ESMTPS id 17C5C6E88B; Mon, 10 May 2021 16:36:49 +0000 (UTC) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=DTe+9Al9HPlBMoLwvE/CRIAu3vCgw3JyUnpdLKjRJSA/BxFkoMpnmn2xIThK1KpRdljEshno9hw+9yO84hdxzI5fVMqC+012/KYJFytA/da50iPXGU5dn5lwJoxP24IuqSQf53MSQFVQBCpOrNL/2Z6O4d+mxUGwdyACbxCFmgl2ZH/dORBmc7cwm63vd4x0XrLL5LzLs7DIt3TvKEfrQGAlf+BTDIX3flBRB8jFPAdD20hMFFA2L0SHWLOeeHWrf7EvsmDb9yosW9dxJWvnjQ7zqziImST/uMFI4EDWG9aGzhfNk91MaSbCNB2xJnZ7z4JnNTYPgMoiHuZ64egZdg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=9x9LOr31QXQh26mL0MRx+84J3JNefHzW2myk9ZEcHMw=; b=Mf6DhPPQRHmqOgfLc2fsOWZ6kXTbDV5HoNUdJFGLFemDpWfBmeYsToJAxnqIel66PZLjDwXcReyaqwbVoPybpyihyAIm4dTNzDHBUiqTnwkWLfHDRwGpn5wNopnckLgv31YlebsNPVBNVs5KEZcH8oRIGn8QqRfQsC5YewZN7LiPPmE/l+hY1mki2G4dLD3WuTGHIxg4Rnwbx5i69vnf6uJ3ZUTANXRIwQUsaskWRLsXxTRS0AD+Rzt5nlj43TsA51n6CeMaBukPOuqJBHAhfQQhtoZkGIKqFK4qfw8ACQuwxKs9FFhM6YZm49lEx4juQBindShj1Yj/ye+6z6NHMQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=amd.com; dmarc=pass action=none header.from=amd.com; dkim=pass header.d=amd.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=9x9LOr31QXQh26mL0MRx+84J3JNefHzW2myk9ZEcHMw=; b=32/tcg0Fri9wi1kijUGM2yxYG85sOgVUjsRMKF1JhoA37wA2oVC6A+FgE1y4L699/ZCdAVIoe6L1L8iTz7zTsVXFS59IaRSkgWfFrD75PWomQ5qVesm3RM95D725ybV1fd7jmq2l2/ojn1XvnHcpHPwGfe0NbSKZ2YfFPoA7BOo= Authentication-Results: lists.freedesktop.org; dkim=none (message not signed) header.d=none; lists.freedesktop.org; dmarc=none action=none header.from=amd.com; Received: from SN6PR12MB4623.namprd12.prod.outlook.com (2603:10b6:805:e9::17) by SA0PR12MB4462.namprd12.prod.outlook.com (2603:10b6:806:95::22) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4108.25; Mon, 10 May 2021 16:36:47 +0000 Received: from SN6PR12MB4623.namprd12.prod.outlook.com ([fe80::ad51:8c49:b171:856c]) by SN6PR12MB4623.namprd12.prod.outlook.com ([fe80::ad51:8c49:b171:856c%7]) with mapi id 15.20.4108.031; Mon, 10 May 2021 16:36:47 +0000 From: Andrey Grodzovsky To: dri-devel@lists.freedesktop.org, amd-gfx@lists.freedesktop.org, linux-pci@vger.kernel.org, ckoenig.leichtzumerken@gmail.com, daniel.vetter@ffwll.ch, Harry.Wentland@amd.com Subject: [PATCH v6 03/16] drm/amdgpu: Split amdgpu_device_fini into early and late Date: Mon, 10 May 2021 12:36:12 -0400 Message-Id: <20210510163625.407105-4-andrey.grodzovsky@amd.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210510163625.407105-1-andrey.grodzovsky@amd.com> References: <20210510163625.407105-1-andrey.grodzovsky@amd.com> X-Originating-IP: [2607:fea8:3edf:49b0:6a5:47b8:e610:f6a3] X-ClientProxiedBy: YT2PR01CA0020.CANPRD01.PROD.OUTLOOK.COM (2603:10b6:b01:38::25) To SN6PR12MB4623.namprd12.prod.outlook.com (2603:10b6:805:e9::17) MIME-Version: 1.0 X-MS-Exchange-MessageSentRepresentingType: 1 Received: from agrodzovsky-All-Series.hitronhub.home (2607:fea8:3edf:49b0:6a5:47b8:e610:f6a3) by YT2PR01CA0020.CANPRD01.PROD.OUTLOOK.COM (2603:10b6:b01:38::25) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4108.31 via Frontend Transport; Mon, 10 May 2021 16:36:46 +0000 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 9d56f206-11c4-4c5c-5ff2-08d913d1cdf8 X-MS-TrafficTypeDiagnostic: SA0PR12MB4462: X-MS-Exchange-Transport-Forked: True X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:862; X-MS-Exchange-SenderADCheck: 1 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: //jvJRMNpu2/zegt8mfyEyGF+wbYwmMlWgImVwMYu/JUTAQ4mMawAFaASQKzJyhXYyUOKV8rGpsXEwK/FVv+8mu2bkFeeOM1AYqkDwgloTJiHlqraHxsjjkc+w51ZsIhLrILFooQM8aOLN1mRll2kKBr6FBWZFH4BCPC1M6O/YR5AYIpWtHRDdnAMp5/6mDlg+ltl0TyF5aVcpQ4jmlYgoQ9jZqq0OLrKDTBAHsT+YgEQx2oVa5p82X68GePeoU9qeN4FLLyIf0bR2pmHUKG0m+RdX198J9BJHE4LIwFf6nN+kYOTjBgfTjqgp9rvxo9Nsz9CehLuEFipTIsXQ9JJw0952FkVyOGjjHujV7TQZE3MV8PMNN5HFLW8bKGQJqJa6VtVdGhnosMFAzi8hVd+nYKSgkdQ5M5Y+NpBEI3XyB0YgriaovKIjN7PASFdh5CuE2BufPiCDge81VikiuKZLu0/mZHEDMtG+aEjOGsBDD1m4yboZ27zBEKcyk9DRiidyMaGiJBKgyZC1CeXsm9Td6oXbEaucS9yTc+UVGopfLfYuOE+4e+ljGC2D1kRLe0MC+oONua53D31VNRSFOY4p7YX2nCaceT+eboSogXle0= X-Forefront-Antispam-Report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:SN6PR12MB4623.namprd12.prod.outlook.com; PTR:; CAT:NONE; SFS:(4636009)(366004)(136003)(39860400002)(396003)(376002)(346002)(86362001)(6506007)(38100700002)(52116002)(6666004)(44832011)(8676002)(36756003)(5660300002)(478600001)(8936002)(2906002)(6486002)(54906003)(66946007)(30864003)(2616005)(66476007)(66556008)(1076003)(316002)(6636002)(6512007)(16526019)(186003)(83380400001)(4326008)(66574015); DIR:OUT; SFP:1101; X-MS-Exchange-AntiSpam-MessageData: =?utf-8?q?ulA3EmRTvEOsHmh0x8se+ESUQ8LQ19?= =?utf-8?q?kQ6euLP6AA+Hbqd98Hhdtk/mzN91bmvNFTCcIq0FvTUiepgLqI8XqmgidXzoxf/AO?= =?utf-8?q?YlBln4jPWzXY7ERI8UUsBPGmi0R++/9Qtm4fBzwbwoEG5VHrGwT2NEuhdLJkYs63E?= =?utf-8?q?DJVSmItOWCYxKMgS2YprzPurxFgDNJp2CeMyGueVlhkKl8VCZjd32F0E+OnSyxivn?= =?utf-8?q?XNDpgQ1K9QHZdtnnYgYfwwqsLHmrWv4Z7SFlZFnCCLDJsnL3AECJQFaP28Z70UZzo?= =?utf-8?q?FCSmuTjEzGaDtDIfgdyT0jPzbulbWSDtCCN/5ZF12PTG+gjZLoDCfFRQvrN/ZrKbR?= =?utf-8?q?QQC+0o9ZPWaja+DcuzKq9u0CFKyyiorD0oA50iPFFr5oWUfgs2+xWFS1dofs229II?= =?utf-8?q?ziqePzAh+++TPgK3YAVfubsJo/4GFFNvxiErL7rX4kf3syEBhHVpNzKUSrBmOcyuX?= =?utf-8?q?dWPieaHapAnjSfPLNw4PHLTt9GHPfDFZqs+raAnR+kJIR7OcI2GCuwezErAvRdbca?= =?utf-8?q?a+iO60SK0j42MvZuLhet4V3fxXhStow0bO5O6WP2+yrSt4OZHrsA+JQF4AW4EGpaj?= =?utf-8?q?sMCgSSXkyhr3BDOGlWkCPW55K3eiViSoOqBoSZLZHgyqYZs7x2OnjEGerUo444FdT?= =?utf-8?q?3w9s1MYxvfA7eQE6N6udTgw+w1kjNWPbq50AWOkxrCUJ7Esl0MkWafghppXRPz9x6?= =?utf-8?q?Y1xx+y1kwS2jofMLlR8K3psSc14XMAKMm2Mav8+4yLGpKVF1+fyt8dmXcyeTnMqDx?= =?utf-8?q?RKmcbq/iigA1gyDsuJt98/k7prdXhhe8eXxXCR297Edoir/e7D4BdrGUmONK9PXXe?= =?utf-8?q?O1i0wbLtbqOrWls6tB8fFHhyJbDoHDTX1ybhmSh9zIT+48pGOe8LuTIq0lKzaKNkU?= =?utf-8?q?Q5s76y2mZSWpQlpNaUopI6KAEJmRT8MJc8ss8xSU2fgjWsLIYg+XIpFLEgzzkxECI?= =?utf-8?q?i7sOcQrx4fB5rYDWZ3vTFdrMZPRRQNbZoFhM9DFVc+RHd+F1XOl4Wt4qaGIu5HiJg?= =?utf-8?q?ToMc2YWZ/c2NcR84J+iE5PnLg/ZFlhjQsVPyuI3fLRiT+aN7QG+0NRw9I/JAxAgV7?= =?utf-8?q?oi799zT0EPmTIoZjeOVIWd2rqS7Cbqdl7CufAihaWNGNt/R/xBWfu0C6K3TWk6uqB?= =?utf-8?q?fGR1JxJfw+fiWZL4jNgwilzEdD1xrTPeky0fWMt5Vnas8R0TTgqnJdGmZInuYtht8?= =?utf-8?q?ev8f8w2yS51v2TP6PfKk1lrGKC1vLF3/pGf/w4doAlLQtLZMk88JMxE82RY7jP4nt?= =?utf-8?q?YqcC8FqUlTBD+r4mTqetdkyIGiI3xD6stSvi1jEv1hSw/wtl/NVX5E//N8wgjRfQb?= =?utf-8?q?ac92mJ96WrzGw?= X-OriginatorOrg: amd.com X-MS-Exchange-CrossTenant-Network-Message-Id: 9d56f206-11c4-4c5c-5ff2-08d913d1cdf8 X-MS-Exchange-CrossTenant-AuthSource: SN6PR12MB4623.namprd12.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 10 May 2021 16:36:47.5914 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: zqG7TWnqpjvUYDDVFzdQwFsOiej0qFZbchn4M4gVIa85XMhYjPUKVKMHGguEFntyZjNhI2C+bLHGK2f3p3n/RQ== X-MS-Exchange-Transport-CrossTenantHeadersStamped: SA0PR12MB4462 X-BeenThere: dri-devel@lists.freedesktop.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Direct Rendering Infrastructure - Development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: gregkh@linuxfoundation.org, Felix.Kuehling@amd.com, helgaas@kernel.org, Alex Deucher , =?utf-8?q?Christian_K=C3=B6nig?= Errors-To: dri-devel-bounces@lists.freedesktop.org Sender: "dri-devel" Some of the stuff in amdgpu_device_fini such as HW interrupts disable and pending fences finilization must be done right away on pci_remove while most of the stuff which relates to finilizing and releasing driver data structures can be kept until drm_driver.release hook is called, i.e. when the last device reference is dropped. v4: Change functions prefix early->hw and late->sw Signed-off-by: Andrey Grodzovsky Acked-by: Christian König Reviewed-by: Alex Deucher Reported-by: kernel test robot --- drivers/gpu/drm/amd/amdgpu/amdgpu.h | 6 ++++- drivers/gpu/drm/amd/amdgpu/amdgpu_device.c | 26 +++++++++++++++------- drivers/gpu/drm/amd/amdgpu/amdgpu_drv.c | 7 ++---- drivers/gpu/drm/amd/amdgpu/amdgpu_fence.c | 15 ++++++++++++- drivers/gpu/drm/amd/amdgpu/amdgpu_irq.c | 26 +++++++++++++--------- drivers/gpu/drm/amd/amdgpu/amdgpu_irq.h | 3 ++- drivers/gpu/drm/amd/amdgpu/amdgpu_kms.c | 12 +++++++++- drivers/gpu/drm/amd/amdgpu/amdgpu_ras.c | 1 + drivers/gpu/drm/amd/amdgpu/amdgpu_ring.h | 3 ++- drivers/gpu/drm/amd/amdgpu/cik_ih.c | 2 +- drivers/gpu/drm/amd/amdgpu/cz_ih.c | 2 +- drivers/gpu/drm/amd/amdgpu/iceland_ih.c | 2 +- drivers/gpu/drm/amd/amdgpu/navi10_ih.c | 2 +- drivers/gpu/drm/amd/amdgpu/si_ih.c | 2 +- drivers/gpu/drm/amd/amdgpu/tonga_ih.c | 2 +- drivers/gpu/drm/amd/amdgpu/vega10_ih.c | 2 +- drivers/gpu/drm/amd/amdgpu/vega20_ih.c | 2 +- 17 files changed, 79 insertions(+), 36 deletions(-) diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu.h b/drivers/gpu/drm/amd/amdgpu/amdgpu.h index 380801b59b07..d830a541ba89 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu.h +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu.h @@ -1099,7 +1099,9 @@ static inline struct amdgpu_device *amdgpu_ttm_adev(struct ttm_device *bdev) int amdgpu_device_init(struct amdgpu_device *adev, uint32_t flags); -void amdgpu_device_fini(struct amdgpu_device *adev); +void amdgpu_device_fini_hw(struct amdgpu_device *adev); +void amdgpu_device_fini_sw(struct amdgpu_device *adev); + int amdgpu_gpu_wait_for_idle(struct amdgpu_device *adev); void amdgpu_device_vram_access(struct amdgpu_device *adev, loff_t pos, @@ -1319,6 +1321,8 @@ void amdgpu_driver_lastclose_kms(struct drm_device *dev); int amdgpu_driver_open_kms(struct drm_device *dev, struct drm_file *file_priv); void amdgpu_driver_postclose_kms(struct drm_device *dev, struct drm_file *file_priv); +void amdgpu_driver_release_kms(struct drm_device *dev); + int amdgpu_device_ip_suspend(struct amdgpu_device *adev); int amdgpu_device_suspend(struct drm_device *dev, bool fbcon); int amdgpu_device_resume(struct drm_device *dev, bool fbcon); diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c index b4ad1c055c70..3760ce7d8ff8 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c @@ -3648,15 +3648,13 @@ int amdgpu_device_init(struct amdgpu_device *adev, * Tear down the driver info (all asics). * Called at driver shutdown. */ -void amdgpu_device_fini(struct amdgpu_device *adev) +void amdgpu_device_fini_hw(struct amdgpu_device *adev) { dev_info(adev->dev, "amdgpu: finishing device.\n"); flush_delayed_work(&adev->delayed_init_work); ttm_bo_lock_delayed_workqueue(&adev->mman.bdev); adev->shutdown = true; - kfree(adev->pci_state); - /* make sure IB test finished before entering exclusive mode * to avoid preemption on IB test * */ @@ -3673,11 +3671,24 @@ void amdgpu_device_fini(struct amdgpu_device *adev) else drm_atomic_helper_shutdown(adev_to_drm(adev)); } - amdgpu_fence_driver_fini(adev); + amdgpu_fence_driver_fini_hw(adev); + if (adev->pm_sysfs_en) amdgpu_pm_sysfs_fini(adev); + if (adev->ucode_sysfs_en) + amdgpu_ucode_sysfs_fini(adev); + sysfs_remove_files(&adev->dev->kobj, amdgpu_dev_attributes); + + amdgpu_fbdev_fini(adev); + + amdgpu_irq_fini_hw(adev); +} + +void amdgpu_device_fini_sw(struct amdgpu_device *adev) +{ amdgpu_device_ip_fini(adev); + amdgpu_fence_driver_fini_sw(adev); release_firmware(adev->firmware.gpu_info_fw); adev->firmware.gpu_info_fw = NULL; adev->accel_working = false; @@ -3703,14 +3714,13 @@ void amdgpu_device_fini(struct amdgpu_device *adev) adev->rmmio = NULL; amdgpu_device_doorbell_fini(adev); - if (adev->ucode_sysfs_en) - amdgpu_ucode_sysfs_fini(adev); - - sysfs_remove_files(&adev->dev->kobj, amdgpu_dev_attributes); if (IS_ENABLED(CONFIG_PERF_EVENTS)) amdgpu_pmu_fini(adev); if (adev->mman.discovery_bin) amdgpu_discovery_fini(adev); + + kfree(adev->pci_state); + } diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_drv.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_drv.c index 6cf573293823..5ebed4c7d9c0 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_drv.c +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_drv.c @@ -1311,14 +1311,10 @@ amdgpu_pci_remove(struct pci_dev *pdev) { struct drm_device *dev = pci_get_drvdata(pdev); -#ifdef MODULE - if (THIS_MODULE->state != MODULE_STATE_GOING) -#endif - DRM_ERROR("Hotplug removal is not supported\n"); drm_dev_unplug(dev); amdgpu_driver_unload_kms(dev); + pci_disable_device(pdev); - pci_set_drvdata(pdev, NULL); } static void @@ -1748,6 +1744,7 @@ static const struct drm_driver amdgpu_kms_driver = { .dumb_create = amdgpu_mode_dumb_create, .dumb_map_offset = amdgpu_mode_dumb_mmap, .fops = &amdgpu_driver_kms_fops, + .release = &amdgpu_driver_release_kms, .prime_handle_to_fd = drm_gem_prime_handle_to_fd, .prime_fd_to_handle = drm_gem_prime_fd_to_handle, diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_fence.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_fence.c index 47ea46859618..1ffb36bd0b19 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_fence.c +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_fence.c @@ -523,7 +523,7 @@ int amdgpu_fence_driver_init(struct amdgpu_device *adev) * * Tear down the fence driver for all possible rings (all asics). */ -void amdgpu_fence_driver_fini(struct amdgpu_device *adev) +void amdgpu_fence_driver_fini_hw(struct amdgpu_device *adev) { unsigned i, j; int r; @@ -545,6 +545,19 @@ void amdgpu_fence_driver_fini(struct amdgpu_device *adev) ring->fence_drv.irq_type); del_timer_sync(&ring->fence_drv.fallback_timer); + } +} + +void amdgpu_fence_driver_fini_sw(struct amdgpu_device *adev) +{ + unsigned int i, j; + + for (i = 0; i < AMDGPU_MAX_RINGS; i++) { + struct amdgpu_ring *ring = adev->rings[i]; + + if (!ring || !ring->fence_drv.initialized) + continue; + for (j = 0; j <= ring->fence_drv.num_fences_mask; ++j) dma_fence_put(ring->fence_drv.fences[j]); kfree(ring->fence_drv.fences); diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_irq.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_irq.c index 90f50561b43a..233b64dab94b 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_irq.c +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_irq.c @@ -49,6 +49,7 @@ #include #include #include +#include #include "amdgpu.h" #include "amdgpu_ih.h" #include "atom.h" @@ -348,6 +349,20 @@ int amdgpu_irq_init(struct amdgpu_device *adev) return 0; } + +void amdgpu_irq_fini_hw(struct amdgpu_device *adev) +{ + if (adev->irq.installed) { + drm_irq_uninstall(&adev->ddev); + adev->irq.installed = false; + if (adev->irq.msi_enabled) + pci_free_irq_vectors(adev->pdev); + + if (!amdgpu_device_has_dc_support(adev)) + flush_work(&adev->hotplug_work); + } +} + /** * amdgpu_irq_fini - shut down interrupt handling * @@ -357,19 +372,10 @@ int amdgpu_irq_init(struct amdgpu_device *adev) * functionality, shuts down vblank, hotplug and reset interrupt handling, * turns off interrupts from all sources (all ASICs). */ -void amdgpu_irq_fini(struct amdgpu_device *adev) +void amdgpu_irq_fini_sw(struct amdgpu_device *adev) { unsigned i, j; - if (adev->irq.installed) { - drm_irq_uninstall(adev_to_drm(adev)); - adev->irq.installed = false; - if (adev->irq.msi_enabled) - pci_free_irq_vectors(adev->pdev); - if (!amdgpu_device_has_dc_support(adev)) - flush_work(&adev->hotplug_work); - } - for (i = 0; i < AMDGPU_IRQ_CLIENTID_MAX; ++i) { if (!adev->irq.client[i].sources) continue; diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_irq.h b/drivers/gpu/drm/amd/amdgpu/amdgpu_irq.h index cf6116648322..78ad4784cc74 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_irq.h +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_irq.h @@ -103,7 +103,8 @@ void amdgpu_irq_disable_all(struct amdgpu_device *adev); irqreturn_t amdgpu_irq_handler(int irq, void *arg); int amdgpu_irq_init(struct amdgpu_device *adev); -void amdgpu_irq_fini(struct amdgpu_device *adev); +void amdgpu_irq_fini_sw(struct amdgpu_device *adev); +void amdgpu_irq_fini_hw(struct amdgpu_device *adev); int amdgpu_irq_add_id(struct amdgpu_device *adev, unsigned client_id, unsigned src_id, struct amdgpu_irq_src *source); diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_kms.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_kms.c index 39ee88d29cca..f3ecada208b0 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_kms.c +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_kms.c @@ -28,6 +28,7 @@ #include "amdgpu.h" #include +#include #include "amdgpu_uvd.h" #include "amdgpu_vce.h" #include "atom.h" @@ -92,7 +93,7 @@ void amdgpu_driver_unload_kms(struct drm_device *dev) } amdgpu_acpi_fini(adev); - amdgpu_device_fini(adev); + amdgpu_device_fini_hw(adev); } void amdgpu_register_gpu_instance(struct amdgpu_device *adev) @@ -1219,6 +1220,15 @@ void amdgpu_driver_postclose_kms(struct drm_device *dev, pm_runtime_put_autosuspend(dev->dev); } + +void amdgpu_driver_release_kms(struct drm_device *dev) +{ + struct amdgpu_device *adev = drm_to_adev(dev); + + amdgpu_device_fini_sw(adev); + pci_set_drvdata(adev->pdev, NULL); +} + /* * VBlank related functions. */ diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_ras.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_ras.c index 0541196ae1ed..844a667f655b 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_ras.c +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_ras.c @@ -2325,6 +2325,7 @@ int amdgpu_ras_pre_fini(struct amdgpu_device *adev) if (!adev->ras_features || !con) return 0; + /* Need disable ras on all IPs here before ip [hw/sw]fini */ amdgpu_ras_disable_all_features(adev, 0); amdgpu_ras_recovery_fini(adev); diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_ring.h b/drivers/gpu/drm/amd/amdgpu/amdgpu_ring.h index ca1622835296..e7d3d0dbdd96 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_ring.h +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_ring.h @@ -107,7 +107,8 @@ struct amdgpu_fence_driver { }; int amdgpu_fence_driver_init(struct amdgpu_device *adev); -void amdgpu_fence_driver_fini(struct amdgpu_device *adev); +void amdgpu_fence_driver_fini_hw(struct amdgpu_device *adev); +void amdgpu_fence_driver_fini_sw(struct amdgpu_device *adev); void amdgpu_fence_driver_force_completion(struct amdgpu_ring *ring); int amdgpu_fence_driver_init_ring(struct amdgpu_ring *ring, diff --git a/drivers/gpu/drm/amd/amdgpu/cik_ih.c b/drivers/gpu/drm/amd/amdgpu/cik_ih.c index d3745711d55f..183d44a6583c 100644 --- a/drivers/gpu/drm/amd/amdgpu/cik_ih.c +++ b/drivers/gpu/drm/amd/amdgpu/cik_ih.c @@ -309,7 +309,7 @@ static int cik_ih_sw_fini(void *handle) { struct amdgpu_device *adev = (struct amdgpu_device *)handle; - amdgpu_irq_fini(adev); + amdgpu_irq_fini_sw(adev); amdgpu_ih_ring_fini(adev, &adev->irq.ih); amdgpu_irq_remove_domain(adev); diff --git a/drivers/gpu/drm/amd/amdgpu/cz_ih.c b/drivers/gpu/drm/amd/amdgpu/cz_ih.c index 307c01301c87..d32743949003 100644 --- a/drivers/gpu/drm/amd/amdgpu/cz_ih.c +++ b/drivers/gpu/drm/amd/amdgpu/cz_ih.c @@ -301,7 +301,7 @@ static int cz_ih_sw_fini(void *handle) { struct amdgpu_device *adev = (struct amdgpu_device *)handle; - amdgpu_irq_fini(adev); + amdgpu_irq_fini_sw(adev); amdgpu_ih_ring_fini(adev, &adev->irq.ih); amdgpu_irq_remove_domain(adev); diff --git a/drivers/gpu/drm/amd/amdgpu/iceland_ih.c b/drivers/gpu/drm/amd/amdgpu/iceland_ih.c index cc957471f31e..da96c6013477 100644 --- a/drivers/gpu/drm/amd/amdgpu/iceland_ih.c +++ b/drivers/gpu/drm/amd/amdgpu/iceland_ih.c @@ -300,7 +300,7 @@ static int iceland_ih_sw_fini(void *handle) { struct amdgpu_device *adev = (struct amdgpu_device *)handle; - amdgpu_irq_fini(adev); + amdgpu_irq_fini_sw(adev); amdgpu_ih_ring_fini(adev, &adev->irq.ih); amdgpu_irq_remove_domain(adev); diff --git a/drivers/gpu/drm/amd/amdgpu/navi10_ih.c b/drivers/gpu/drm/amd/amdgpu/navi10_ih.c index f4e4040bbd25..5eea4550b856 100644 --- a/drivers/gpu/drm/amd/amdgpu/navi10_ih.c +++ b/drivers/gpu/drm/amd/amdgpu/navi10_ih.c @@ -569,7 +569,7 @@ static int navi10_ih_sw_fini(void *handle) { struct amdgpu_device *adev = (struct amdgpu_device *)handle; - amdgpu_irq_fini(adev); + amdgpu_irq_fini_sw(adev); amdgpu_ih_ring_fini(adev, &adev->irq.ih_soft); amdgpu_ih_ring_fini(adev, &adev->irq.ih2); amdgpu_ih_ring_fini(adev, &adev->irq.ih1); diff --git a/drivers/gpu/drm/amd/amdgpu/si_ih.c b/drivers/gpu/drm/amd/amdgpu/si_ih.c index 51880f6ef634..751307f3252c 100644 --- a/drivers/gpu/drm/amd/amdgpu/si_ih.c +++ b/drivers/gpu/drm/amd/amdgpu/si_ih.c @@ -175,7 +175,7 @@ static int si_ih_sw_fini(void *handle) { struct amdgpu_device *adev = (struct amdgpu_device *)handle; - amdgpu_irq_fini(adev); + amdgpu_irq_fini_sw(adev); amdgpu_ih_ring_fini(adev, &adev->irq.ih); return 0; diff --git a/drivers/gpu/drm/amd/amdgpu/tonga_ih.c b/drivers/gpu/drm/amd/amdgpu/tonga_ih.c index 249fcbee7871..973d80ec7f6c 100644 --- a/drivers/gpu/drm/amd/amdgpu/tonga_ih.c +++ b/drivers/gpu/drm/amd/amdgpu/tonga_ih.c @@ -312,7 +312,7 @@ static int tonga_ih_sw_fini(void *handle) { struct amdgpu_device *adev = (struct amdgpu_device *)handle; - amdgpu_irq_fini(adev); + amdgpu_irq_fini_sw(adev); amdgpu_ih_ring_fini(adev, &adev->irq.ih); amdgpu_irq_remove_domain(adev); diff --git a/drivers/gpu/drm/amd/amdgpu/vega10_ih.c b/drivers/gpu/drm/amd/amdgpu/vega10_ih.c index ca8efa5c6978..dead9c2fbd4c 100644 --- a/drivers/gpu/drm/amd/amdgpu/vega10_ih.c +++ b/drivers/gpu/drm/amd/amdgpu/vega10_ih.c @@ -513,7 +513,7 @@ static int vega10_ih_sw_fini(void *handle) { struct amdgpu_device *adev = (struct amdgpu_device *)handle; - amdgpu_irq_fini(adev); + amdgpu_irq_fini_sw(adev); amdgpu_ih_ring_fini(adev, &adev->irq.ih_soft); amdgpu_ih_ring_fini(adev, &adev->irq.ih2); amdgpu_ih_ring_fini(adev, &adev->irq.ih1); diff --git a/drivers/gpu/drm/amd/amdgpu/vega20_ih.c b/drivers/gpu/drm/amd/amdgpu/vega20_ih.c index 8a122b413bf5..58993ae1fe11 100644 --- a/drivers/gpu/drm/amd/amdgpu/vega20_ih.c +++ b/drivers/gpu/drm/amd/amdgpu/vega20_ih.c @@ -565,7 +565,7 @@ static int vega20_ih_sw_fini(void *handle) { struct amdgpu_device *adev = (struct amdgpu_device *)handle; - amdgpu_irq_fini(adev); + amdgpu_irq_fini_sw(adev); amdgpu_ih_ring_fini(adev, &adev->irq.ih_soft); amdgpu_ih_ring_fini(adev, &adev->irq.ih2); amdgpu_ih_ring_fini(adev, &adev->irq.ih1); From patchwork Mon May 10 16:36:13 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrey Grodzovsky X-Patchwork-Id: 12248669 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-18.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,MSGID_FROM_MTA_HEADER,SPF_HELO_NONE, SPF_PASS,USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id BC03BC43461 for ; Mon, 10 May 2021 16:37:00 +0000 (UTC) Received: from gabe.freedesktop.org (gabe.freedesktop.org [131.252.210.177]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 7AE4E61107 for ; Mon, 10 May 2021 16:37:00 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 7AE4E61107 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=amd.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=dri-devel-bounces@lists.freedesktop.org Received: from gabe.freedesktop.org (localhost [127.0.0.1]) by gabe.freedesktop.org (Postfix) with ESMTP id F251F6E8AF; Mon, 10 May 2021 16:36:52 +0000 (UTC) Received: from NAM10-MW2-obe.outbound.protection.outlook.com (mail-mw2nam10on2068.outbound.protection.outlook.com [40.107.94.68]) by gabe.freedesktop.org (Postfix) with ESMTPS id 619236E0AC; Mon, 10 May 2021 16:36:50 +0000 (UTC) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=YQJB6kRWnfk/qa9n5dU2mlcBeMQyCeY8j5zs62D5vN/0o17RcU8A2hGd8ECxoS6xyxZS7ljHbWFnq+cHwmWlNm1PpOnVrP4YTBQIHh7W8f1i9Ibj3p7zNhod93Nsm/4AG3c+5ZOrzTub1jSDYRc6Gl5O+ZomG0ZqsOL6t2saZmkQ+o6xdEIgcGKY9lSOgfoaGhWoo5nJWA3w91i6Ep/szIJkco6pfbakXFQhpYmaxX7dgKSY3Vzp3o6/gMeTeSv1/uXwC8b51GTd0JfdUUEaqS9euSE24/CEIMET4yBsizGdSieTFc/VShr5XtH5zbGSFz+1sgQzIXSM9ztmpoN5TA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=+8ub1TT4Why1wfUTgqnHi8urBMdsD729TcqEvDbve44=; b=Gokryek0pmhOtSGc8HtkrdKxsL1vX+RV48Lj45xzDqW2wCwE+AfcZcMaGOlnlVEOkzKPAMNmJPVerK59Y4oh7l+mbSmpNTTYenCNQ1iB+Ha3SsnVLDvjMJ3OkQ8s1UlLZwajLoCyzs4EV3/argcp8G8C/aoajNs3qIcUL5Nv5lUvXyOJLgS8jl3vIxGFVzi1lY0u8b3PGsmQ9UsmsK2IPFtxEaotoJ86AVMDMpIADeLaQrgEAbJDN8KQaLB8NTQ2JQy+5Z2VlkZ8hPpHf+ks69shP2LoaH4CGuXAHEdJtsVowMMrmNcnulFOddEJjU1/Be3tw8UHRk5g4jYnthnrYQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=amd.com; dmarc=pass action=none header.from=amd.com; dkim=pass header.d=amd.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=+8ub1TT4Why1wfUTgqnHi8urBMdsD729TcqEvDbve44=; b=JQdVYKYD2g4fvoEM0JTnXUyY6CW08s+SmeDu6ayHiTeJ4Hjrur2hbukO9d6BeL6VzaCEuWuqgyi+uefFlYRmmWPx9C3tTtrlKUhEFPa7nSQtsZoFj3Prdq1w9S1c1H3cAx+dFheAtC+qoRnmpDAk/c93LovDOmFW8komxO3RFAw= Authentication-Results: lists.freedesktop.org; dkim=none (message not signed) header.d=none; lists.freedesktop.org; dmarc=none action=none header.from=amd.com; Received: from SN6PR12MB4623.namprd12.prod.outlook.com (2603:10b6:805:e9::17) by SA0PR12MB4462.namprd12.prod.outlook.com (2603:10b6:806:95::22) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4108.25; Mon, 10 May 2021 16:36:49 +0000 Received: from SN6PR12MB4623.namprd12.prod.outlook.com ([fe80::ad51:8c49:b171:856c]) by SN6PR12MB4623.namprd12.prod.outlook.com ([fe80::ad51:8c49:b171:856c%7]) with mapi id 15.20.4108.031; Mon, 10 May 2021 16:36:49 +0000 From: Andrey Grodzovsky To: dri-devel@lists.freedesktop.org, amd-gfx@lists.freedesktop.org, linux-pci@vger.kernel.org, ckoenig.leichtzumerken@gmail.com, daniel.vetter@ffwll.ch, Harry.Wentland@amd.com Subject: [PATCH v6 04/16] drm/amdkfd: Split kfd suspend from devie exit Date: Mon, 10 May 2021 12:36:13 -0400 Message-Id: <20210510163625.407105-5-andrey.grodzovsky@amd.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210510163625.407105-1-andrey.grodzovsky@amd.com> References: <20210510163625.407105-1-andrey.grodzovsky@amd.com> X-Originating-IP: [2607:fea8:3edf:49b0:6a5:47b8:e610:f6a3] X-ClientProxiedBy: YT2PR01CA0020.CANPRD01.PROD.OUTLOOK.COM (2603:10b6:b01:38::25) To SN6PR12MB4623.namprd12.prod.outlook.com (2603:10b6:805:e9::17) MIME-Version: 1.0 X-MS-Exchange-MessageSentRepresentingType: 1 Received: from agrodzovsky-All-Series.hitronhub.home (2607:fea8:3edf:49b0:6a5:47b8:e610:f6a3) by YT2PR01CA0020.CANPRD01.PROD.OUTLOOK.COM (2603:10b6:b01:38::25) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4108.31 via Frontend Transport; Mon, 10 May 2021 16:36:48 +0000 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: bf2489ae-498b-47f2-152c-08d913d1cf05 X-MS-TrafficTypeDiagnostic: SA0PR12MB4462: X-MS-Exchange-Transport-Forked: True X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:510; X-MS-Exchange-SenderADCheck: 1 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: gWM+hN349McEyOKV1Sw84YoBN7wZWNZREnlAdBzX3K+xAwZeFK3wRwJgkUI5fPV/uuvvCGD2xiUDT0Tl4TfPmGyfidz9Y2/v5Tg6BCuyGCHeQeyccapzQK3tMdi5gnkWuAPipurUBIKUBzvRncgsZqopnHs0cr8DJFUkDaguQ+LIFBP75e3CDbej6Sa1jfxbBvWaMCml5v5LnESX/IO8ZJEXU3aWhDgHZ3IccB3pvRVJiTT98dKHxYZkFC0luOa3l7SaOxXKjDZPS6+/zOkZ1Dy06XD08aAxQWx38IDwn5allMv0fgepidzuSl6s+OaRbPik3Q2tC7nn4uieXuBgYZjd4dy0Rq6+XONAiX4kzr6IYAL1A4VvviaPBKAzchErMewaZ/x3+0vgT6buVH7GNGBahYZsOwAwn0Xio1/vX8Z/RLOTscvz4XIrCoLTbPUoZzQaKP1JRmfmfKzjAU0NH2ABVuQAq+h6SALi+alWkqyhSkIyMvYBF/aCI4NRvxFsup7DQ4JWriUNMh1f36LtSiGjy7rx5XixWAHG5krsC/aA6ZfbRNwlPmokH9XWUGmIbOq4glHHjDqFx57NzrkfE4g9SdgIRdf/D3RgMtg6vsU= X-Forefront-Antispam-Report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:SN6PR12MB4623.namprd12.prod.outlook.com; PTR:; CAT:NONE; SFS:(4636009)(366004)(136003)(39860400002)(396003)(376002)(346002)(86362001)(6506007)(38100700002)(52116002)(6666004)(44832011)(8676002)(36756003)(5660300002)(478600001)(8936002)(2906002)(6486002)(66946007)(2616005)(66476007)(66556008)(1076003)(316002)(6636002)(6512007)(16526019)(186003)(83380400001)(4326008); DIR:OUT; SFP:1101; X-MS-Exchange-AntiSpam-MessageData: +KXj8mFm173VQufNManNabSmaSu1k2hO0gbBuuCaPDb/Gx0o6Jben9AXzfRmr2ilxwtBN5rlYElYfT2shyiVERk0lipnjlUer8OOfTz0oKLOPXrbvTPL18kySSpc6I+CTC9egVG0jZunGkxUw3MfzYIBzITeDnuq4nTaeHWaXGkhCmdXvHpw0kYaHUx0JjYCWcE2rlSw062DdpzOfF0YgHC38spRl63lwggQZXo9D+yH5BTYwwxs9jw6qvPFTM5Y1dqvHjiyEvZ+NLfDlI/fvlscKNT74SgdZfo5PvMAd4TFfV/5L278BVrOvC7Idxb+/hxUu1PjnHzITvHJ1REldbyjpCaKXhCFclflprT2vuxN1Pd1NDWgFyPntydKZpqm+V7kAyUXzaqMAwHJ05hN9rVO6TVwXyrvtL7X21M+AGYzNHgK3owqzcWlkgSkfBrWp18tI1UH8NO9PtwL0cdT4YDCNBDP1vLC+i+/cjHnai0b5If37myzqFKrAEBrtlEIc3uzxMRgB8yhAEx30EkbM5DC9FLBpJ3+P+upErEMCuay1yAhuJ0TkPcAsiRvfRDUIePCXRi5jjrI/9DISubnjZmSTcGf5srWTu5zXecJ/UKSR+mek3Gvf4erqxLwGYaILgbiESByiB+DZAZknWKXeYvarOplRxfU7NEyWwmybTrvMVMdzz5Jd//jd/QYNv6/2l4k7nwDr2r8eLs+cRa33rL+PLUHhp0hYRDADWaFQc00zUMiiwmAZW7ao2zLbUqcHrDhHnNU8r115RO3u+diMtHUwzxGV8kZRhyqOUe/0nqu+5ZWWebLYLWdUA3kBNmj+NK0lsMBLb+yRg6Rer4ki+Lo2y2QQjMa8h1MLnSg1QsoJ6zWmvw1hjPpXefFi7zwXylrYq/UR8y7aqqe3X8zYR8gg3ZrscpcnSbZiS+R2QLU+iLr+1b9oD3aAKZxBKTDj4kZpiCWWJPgaya3jj663xMxIVPhyAvn1QdC9aVVDp+rcmcUA6jGVkuQm9UR2fR585DHm9H9Sbvgkl2FKWvFYUyN8opjiZy7ec+9Zquq9gufGHTfPNLfRKQt1lc6QhkTCpthf5iuqbxPq3Mh6VkUho0PaPI099ctec1Z3TXCb/4L6rRu9KfapKrNoSq0LUHGAP7XstBqiOzZndlCufkMVc1+aCaTzTMZ2VyZabzFEtN7DQHWlfxrCw+zjWSlFHDsDGUbt9/GMciCCCFaujksDW/UplaO5PCAofTBGhM57OmXhHTK9POTsmroGcu2pFLhmroqp23yQ9DD6vDJKDKpUv23XHECmfxj74C4bXpbIsg6W7WPWJ6anEv8jzYXY/F1XLpkm0KPE6yjYPicchGTbDf/OeaE28PvM5igNXHLGqiRPigbru7THKiNR9Y0JHwe X-OriginatorOrg: amd.com X-MS-Exchange-CrossTenant-Network-Message-Id: bf2489ae-498b-47f2-152c-08d913d1cf05 X-MS-Exchange-CrossTenant-AuthSource: SN6PR12MB4623.namprd12.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 10 May 2021 16:36:49.3444 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: 39Zlf1dLoYZ/0knZyd8EgoM8NrA92vRi63R6C5R54A5RzyNJorkN7ZkLBQ4YiMPOje0Qmahg0fVWwKEEcNRNJA== X-MS-Exchange-Transport-CrossTenantHeadersStamped: SA0PR12MB4462 X-BeenThere: dri-devel@lists.freedesktop.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Direct Rendering Infrastructure - Development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: gregkh@linuxfoundation.org, Felix.Kuehling@amd.com, helgaas@kernel.org, Alexander.Deucher@amd.com Errors-To: dri-devel-bounces@lists.freedesktop.org Sender: "dri-devel" Helps to expdite HW related stuff to amdgpu_pci_remove Signed-off-by: Andrey Grodzovsky --- drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd.c | 2 +- drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd.h | 2 +- drivers/gpu/drm/amd/amdkfd/kfd_device.c | 3 ++- 3 files changed, 4 insertions(+), 3 deletions(-) diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd.c index 5f6696a3c778..2b06dee9a0ce 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd.c +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd.c @@ -170,7 +170,7 @@ void amdgpu_amdkfd_device_init(struct amdgpu_device *adev) } } -void amdgpu_amdkfd_device_fini(struct amdgpu_device *adev) +void amdgpu_amdkfd_device_fini_sw(struct amdgpu_device *adev) { if (adev->kfd.dev) { kgd2kfd_device_exit(adev->kfd.dev); diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd.h b/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd.h index 14f68c028126..f8e10af99c28 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd.h +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd.h @@ -127,7 +127,7 @@ void amdgpu_amdkfd_interrupt(struct amdgpu_device *adev, const void *ih_ring_entry); void amdgpu_amdkfd_device_probe(struct amdgpu_device *adev); void amdgpu_amdkfd_device_init(struct amdgpu_device *adev); -void amdgpu_amdkfd_device_fini(struct amdgpu_device *adev); +void amdgpu_amdkfd_device_fini_sw(struct amdgpu_device *adev); int amdgpu_amdkfd_submit_ib(struct kgd_dev *kgd, enum kgd_engine_type engine, uint32_t vmid, uint64_t gpu_addr, uint32_t *ib_cmd, uint32_t ib_len); diff --git a/drivers/gpu/drm/amd/amdkfd/kfd_device.c b/drivers/gpu/drm/amd/amdkfd/kfd_device.c index 357b9bf62a1c..ab6d2a43c9a3 100644 --- a/drivers/gpu/drm/amd/amdkfd/kfd_device.c +++ b/drivers/gpu/drm/amd/amdkfd/kfd_device.c @@ -858,10 +858,11 @@ bool kgd2kfd_device_init(struct kfd_dev *kfd, return kfd->init_complete; } + + void kgd2kfd_device_exit(struct kfd_dev *kfd) { if (kfd->init_complete) { - kgd2kfd_suspend(kfd, false); device_queue_manager_uninit(kfd->dqm); kfd_interrupt_exit(kfd); kfd_topology_remove_device(kfd); From patchwork Mon May 10 16:36:14 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Andrey Grodzovsky X-Patchwork-Id: 12248671 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-18.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,MSGID_FROM_MTA_HEADER,SPF_HELO_NONE, SPF_PASS,USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id ACF2DC43460 for ; Mon, 10 May 2021 16:37:03 +0000 (UTC) Received: from gabe.freedesktop.org (gabe.freedesktop.org [131.252.210.177]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 66B7461155 for ; Mon, 10 May 2021 16:37:03 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 66B7461155 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=amd.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=dri-devel-bounces@lists.freedesktop.org Received: from gabe.freedesktop.org (localhost [127.0.0.1]) by gabe.freedesktop.org (Postfix) with ESMTP id C0B566E8AE; Mon, 10 May 2021 16:36:53 +0000 (UTC) Received: from NAM10-MW2-obe.outbound.protection.outlook.com (mail-mw2nam10on2062.outbound.protection.outlook.com [40.107.94.62]) by gabe.freedesktop.org (Postfix) with ESMTPS id 8C3116E8AE; Mon, 10 May 2021 16:36:52 +0000 (UTC) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=l+xg8caui8Crz2ivL5gOhWTG9t00NCa8E4nSBLhAMVVQtQz1gUM8bDNX34FZQMGpbvEmJ9e6Kzu1eGy2kH+kvaLphlJtFmGAATTfwf1XHzwT+1mXXFv7HXdWh7kIugHMVtNeLnm8IkbUJKHB+BPtA++tYddXpuKI4ydxO/V6c42wL44e7NVH1DLKyMMgGMFCOWJO09iCqCr8kiRnlxt0yCTYhfUlnFZP9atVehyl7QKSP/51TvRjKuPh8rGNiSh7elON2/ZmjTZErpCSkbb3g9mmj4W8t1sps0aNezYpWOXnxjaJ+24FLhmfDTbB5RMQm7Ox17ddUqbZuqGD4/fSGQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=5oboIsoonZokR9RC85cS+UAEHnSUecYlSuUIK9niN/Q=; b=BvXZ+v401Q05JPrdxez6F6KVg8Krc8KiCeF0zGbEyCukM13j+v81Cal/GObF78pnMd1fszfwWT/3f3cP6prBfu+GV5aoFDpzFqebQ4z7CzfC6HETRRAbAiNiXcNK+Qy05xgelELOo/u0Mo4gL64gbHcGOeTCvnpwAXa1H+s4V+97+oYgmfSQ08UgL4fNgqgDeCOo2JvpyZQ1r+Au8GC2YiWsAAMGF02E+bcpeQGHStke5zftHZ41mcMdt8+/svZDiLJaU9N6EMQ2Z2iQ070BvnNOOcs1rVp2KIZh6qXYmZ9nX0PUQgIllRY1ikx3fHnSDnxukQ22FrxJbSAdkm4Nfg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=amd.com; dmarc=pass action=none header.from=amd.com; dkim=pass header.d=amd.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=5oboIsoonZokR9RC85cS+UAEHnSUecYlSuUIK9niN/Q=; b=b8/gzU70HVoSFImEN3sHSl9rVdmhr/Oxai2QJS7Rz8I8PVKVcErBVrobb2RcFx/k5pv5I2cA8NE8DvccdBiZ3MDWZRA/MCz2S99iUzcZx1PKoywARL6/95NUaOMFkIO4zkduxKpUAcKH5+2gleGpktUXXVVxyVRxisbjHa1Jor8= Authentication-Results: lists.freedesktop.org; dkim=none (message not signed) header.d=none; lists.freedesktop.org; dmarc=none action=none header.from=amd.com; Received: from SN6PR12MB4623.namprd12.prod.outlook.com (2603:10b6:805:e9::17) by SA0PR12MB4462.namprd12.prod.outlook.com (2603:10b6:806:95::22) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4108.25; Mon, 10 May 2021 16:36:51 +0000 Received: from SN6PR12MB4623.namprd12.prod.outlook.com ([fe80::ad51:8c49:b171:856c]) by SN6PR12MB4623.namprd12.prod.outlook.com ([fe80::ad51:8c49:b171:856c%7]) with mapi id 15.20.4108.031; Mon, 10 May 2021 16:36:51 +0000 From: Andrey Grodzovsky To: dri-devel@lists.freedesktop.org, amd-gfx@lists.freedesktop.org, linux-pci@vger.kernel.org, ckoenig.leichtzumerken@gmail.com, daniel.vetter@ffwll.ch, Harry.Wentland@amd.com Subject: [PATCH v6 05/16] drm/amdgpu: Add early fini callback Date: Mon, 10 May 2021 12:36:14 -0400 Message-Id: <20210510163625.407105-6-andrey.grodzovsky@amd.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210510163625.407105-1-andrey.grodzovsky@amd.com> References: <20210510163625.407105-1-andrey.grodzovsky@amd.com> X-Originating-IP: [2607:fea8:3edf:49b0:6a5:47b8:e610:f6a3] X-ClientProxiedBy: YT2PR01CA0020.CANPRD01.PROD.OUTLOOK.COM (2603:10b6:b01:38::25) To SN6PR12MB4623.namprd12.prod.outlook.com (2603:10b6:805:e9::17) MIME-Version: 1.0 X-MS-Exchange-MessageSentRepresentingType: 1 Received: from agrodzovsky-All-Series.hitronhub.home (2607:fea8:3edf:49b0:6a5:47b8:e610:f6a3) by YT2PR01CA0020.CANPRD01.PROD.OUTLOOK.COM (2603:10b6:b01:38::25) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4108.31 via Frontend Transport; Mon, 10 May 2021 16:36:50 +0000 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 37f87a4e-35a4-46df-89aa-08d913d1d00b X-MS-TrafficTypeDiagnostic: SA0PR12MB4462: X-MS-Exchange-Transport-Forked: True X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:5797; X-MS-Exchange-SenderADCheck: 1 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: WaTr+1ZEA2JBI2VTaScNLDsxs8sJKNGLh4s1yS4e0G6o5Kye9/iprTQMn/6/caF7hqLyveOvhJOEnlxlB4fwUJz3xgF/baYexmq4KyrIG4vkL/d3CwJWJBPnqXdOxlMCflBA1d7aS7829PTf/T5yy57WhzhWaMcmjmjf9vWGxWLOZXcxzZ4DAoUAhSrtF/BsTXAGYQ8GnN/AlWMzku3F+9w3FpAFaXAOu0cz7ioQa2K9jUoX2mi17vR/nLElyO6K3kYym8Xp/p0vLtXwmX8pws0/NqyyhJzomffgDW2Hi3pZxdylA5jaO4FFC6awSFsoqtYmeXIXN15O5x8V88FwKqu7k1iR/5vbfd8nPmI49HJh9fyAW65AX8XdPjERN3fCyh1t7mzK7S/rtOJ5rkoyLkCYWzxDc89BFqgvYGd2BMBByObLgOoUyxihIErlaKPoJ8QiBUdefZ2m7VNuLRPOoQGk2Nm6stzIfFpDhTptxWChHqFcseIjj8NVEJdvjgb3zE4OtCkPRuy6eQJzyPKdExcI/AgKCJ/nvQ99lpdw9eEPvW17w4xn+/UNFflaz1aHpUwdrFGsH7LLDkJbk8WOSg== X-Forefront-Antispam-Report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:SN6PR12MB4623.namprd12.prod.outlook.com; PTR:; CAT:NONE; SFS:(4636009)(366004)(136003)(39860400002)(396003)(376002)(346002)(86362001)(6506007)(38100700002)(52116002)(6666004)(44832011)(8676002)(36756003)(5660300002)(478600001)(8936002)(2906002)(6486002)(66946007)(2616005)(66476007)(66556008)(1076003)(316002)(6636002)(6512007)(16526019)(186003)(83380400001)(4326008); DIR:OUT; SFP:1101; X-MS-Exchange-AntiSpam-MessageData: 2RXdW5czk8FR010tgBz0mSaRPXtKiR5KD+aZ1B84vjpgEkLGYKJB/Eg4rscWXGgoVSx++Lw2b948CIpPJ9yL9AyJ2qBHB7PQ2VuX64Nq+jtoDJFHdmBu+wN3BRTze/nEG/ASBa6ADzhB+Ji/iWA60yuZPTXDVTZUQi0fbG5ErJeRVQGvCgitI91mitc21D0IgmT5SWtVyjWd2usw10wO09DvkHvRY6IAWFMqd7irxHcpWFZrieDhFwgFoFxOxLCU3qJPWSd1tRBI/zwCbS5DHJvju6+IqQ+RNbjOlp/2aHCYWiBwlJ3gPQGxTxoNckcs80qn1v10avYRgqr5ru0nN4nVrd7b6APXwOn+MSO9aigCYOXR6S7uR+cqV4Mq9PmF0FajNPaoIaI3KxynrnLgjg9wr0bRqbQZB9raG9fVvfXhlYA6QnyD7ash5QwmBbURI2EKWZcoaEexKT8xeJ6K1vm+KnrgAEgV7ncqXDG27qN5dkEB6ODosZSlqEzUTOk/PKRlf70AsHY47BBaUQQbyfMw5WeGv6ZQsoWbpxIyVJ/Bxa2jx2/1EmVfGbzcYGELYzU1Z7kvr09BL7vqf4Fqb4cpEyMOBUM6lCCrqYeATWHqbYThFNSDLodwzLmCcRKfXrn7mgh2GdDTIpKqxjNoEFgnpCcCq+fTlIcGKP6f8EU49H4cnl0EaFg58s9wU+rGs21WJc/VVHoH9JbyxE0K1x5O9Um1rE6dOeNbnPhUrspCDyaonJ9n9XGATu+J+f8bZE8s5puAUyoJmFLyip6c7iG0MJPYC7W1Pf2gEloQtm8x0b3PMtQeqakJu6eHQUWCddLG9uiClOPVe/+RMQGMY+NrlSL9apu4k1zDglc0YzBM3387y7hJLYAeBMYk7E4pZUD3j/npVLQ+RcBwouDDmGBnolUaJxDBQpp05OlcEI81BrQ49UexN528OYVGEnlh/nkMqyajNroGE+qX98JFzLqS3ApVlgINwzW6Mq1WLouwLFYj9oz6gzIe8V7G8oTSClXdHIRAxKKgal600dmZL88/wdyod3grrR1cVpodSkeo4rrfI3nMau8ckJGjiuJRBHfhiVyfMwbckI/Pg4sQAb6GJif4syQ0WLOq8TbNgVThNg/gVmmRWhUPC8KKgchgdsNa1A5eog5ciCbWMXiUFoF+N8yrVAUMapoxP5wI/GEWhsFUHrqrTwIhy4k/nF/mcbHtiYuDODNKnRL7gJ3EJzLk5Oae+NOOucdHnRCPLxZS/xHwpxKm6q1rsCRK24EcDKNQyEuWOzNSBtHu4zhiILQa2HTjCpnAUhglPcrq7aW//OeVLqzwihb4kPEKRsVldaPvegU8y9GQNjjNi7H7E0PGRc5kfJkJOciZhYVQoDK07n8p89KOJVAT9z9cVBeP X-OriginatorOrg: amd.com X-MS-Exchange-CrossTenant-Network-Message-Id: 37f87a4e-35a4-46df-89aa-08d913d1d00b X-MS-Exchange-CrossTenant-AuthSource: SN6PR12MB4623.namprd12.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 10 May 2021 16:36:51.0554 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: Dp2eCxzcItBtix8hiXvxoyOAK/N7OfIXYfk5jVA7hoomdxMGuceiCrB1kFnPSjIk2VAMi6hwHSNXnt7dcL8vbQ== X-MS-Exchange-Transport-CrossTenantHeadersStamped: SA0PR12MB4462 X-BeenThere: dri-devel@lists.freedesktop.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Direct Rendering Infrastructure - Development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: gregkh@linuxfoundation.org, Felix.Kuehling@amd.com, helgaas@kernel.org, Alexander.Deucher@amd.com Errors-To: dri-devel-bounces@lists.freedesktop.org Sender: "dri-devel" Use it to call disply code dependent on device->drv_data before it's set to NULL on device unplug v5: Move HW finilization into this callback to prevent MMIO accesses post cpi remove. Signed-off-by: Andrey Grodzovsky Acked-by: Christian König --- drivers/gpu/drm/amd/amdgpu/amdgpu_device.c | 59 +++++++++++++------ .../gpu/drm/amd/display/amdgpu_dm/amdgpu_dm.c | 12 +++- drivers/gpu/drm/amd/include/amd_shared.h | 2 + 3 files changed, 52 insertions(+), 21 deletions(-) diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c index 3760ce7d8ff8..18598eda18f6 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c @@ -2558,34 +2558,26 @@ static int amdgpu_device_ip_late_init(struct amdgpu_device *adev) return 0; } -/** - * amdgpu_device_ip_fini - run fini for hardware IPs - * - * @adev: amdgpu_device pointer - * - * Main teardown pass for hardware IPs. The list of all the hardware - * IPs that make up the asic is walked and the hw_fini and sw_fini callbacks - * are run. hw_fini tears down the hardware associated with each IP - * and sw_fini tears down any software state associated with each IP. - * Returns 0 on success, negative error code on failure. - */ -static int amdgpu_device_ip_fini(struct amdgpu_device *adev) +static int amdgpu_device_ip_fini_early(struct amdgpu_device *adev) { int i, r; - if (amdgpu_sriov_vf(adev) && adev->virt.ras_init_done) - amdgpu_virt_release_ras_err_handler_data(adev); + for (i = 0; i < adev->num_ip_blocks; i++) { + if (!adev->ip_blocks[i].version->funcs->early_fini) + continue; - amdgpu_ras_pre_fini(adev); + r = adev->ip_blocks[i].version->funcs->early_fini((void *)adev); + if (r) { + DRM_DEBUG("early_fini of IP block <%s> failed %d\n", + adev->ip_blocks[i].version->funcs->name, r); + } + } - if (adev->gmc.xgmi.num_physical_nodes > 1) - amdgpu_xgmi_remove_device(adev); + amdgpu_amdkfd_suspend(adev, false); amdgpu_device_set_pg_state(adev, AMD_PG_STATE_UNGATE); amdgpu_device_set_cg_state(adev, AMD_CG_STATE_UNGATE); - amdgpu_amdkfd_device_fini(adev); - /* need to disable SMC first */ for (i = 0; i < adev->num_ip_blocks; i++) { if (!adev->ip_blocks[i].status.hw) @@ -2616,6 +2608,33 @@ static int amdgpu_device_ip_fini(struct amdgpu_device *adev) adev->ip_blocks[i].status.hw = false; } + return 0; +} + +/** + * amdgpu_device_ip_fini - run fini for hardware IPs + * + * @adev: amdgpu_device pointer + * + * Main teardown pass for hardware IPs. The list of all the hardware + * IPs that make up the asic is walked and the hw_fini and sw_fini callbacks + * are run. hw_fini tears down the hardware associated with each IP + * and sw_fini tears down any software state associated with each IP. + * Returns 0 on success, negative error code on failure. + */ +static int amdgpu_device_ip_fini(struct amdgpu_device *adev) +{ + int i, r; + + if (amdgpu_sriov_vf(adev) && adev->virt.ras_init_done) + amdgpu_virt_release_ras_err_handler_data(adev); + + amdgpu_ras_pre_fini(adev); + + if (adev->gmc.xgmi.num_physical_nodes > 1) + amdgpu_xgmi_remove_device(adev); + + amdgpu_amdkfd_device_fini_sw(adev); for (i = adev->num_ip_blocks - 1; i >= 0; i--) { if (!adev->ip_blocks[i].status.sw) @@ -3683,6 +3702,8 @@ void amdgpu_device_fini_hw(struct amdgpu_device *adev) amdgpu_fbdev_fini(adev); amdgpu_irq_fini_hw(adev); + + amdgpu_device_ip_fini_early(adev); } void amdgpu_device_fini_sw(struct amdgpu_device *adev) diff --git a/drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm.c b/drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm.c index 296704ce3768..6c2c6a51ce6c 100644 --- a/drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm.c +++ b/drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm.c @@ -1251,6 +1251,15 @@ static int amdgpu_dm_init(struct amdgpu_device *adev) return -EINVAL; } +static int amdgpu_dm_early_fini(void *handle) +{ + struct amdgpu_device *adev = (struct amdgpu_device *)handle; + + amdgpu_dm_audio_fini(adev); + + return 0; +} + static void amdgpu_dm_fini(struct amdgpu_device *adev) { int i; @@ -1259,8 +1268,6 @@ static void amdgpu_dm_fini(struct amdgpu_device *adev) drm_encoder_cleanup(&adev->dm.mst_encoders[i].base); } - amdgpu_dm_audio_fini(adev); - amdgpu_dm_destroy_drm_device(&adev->dm); #if defined(CONFIG_DRM_AMD_SECURE_DISPLAY) @@ -2298,6 +2305,7 @@ static const struct amd_ip_funcs amdgpu_dm_funcs = { .late_init = dm_late_init, .sw_init = dm_sw_init, .sw_fini = dm_sw_fini, + .early_fini = amdgpu_dm_early_fini, .hw_init = dm_hw_init, .hw_fini = dm_hw_fini, .suspend = dm_suspend, diff --git a/drivers/gpu/drm/amd/include/amd_shared.h b/drivers/gpu/drm/amd/include/amd_shared.h index 43ed6291b2b8..1ad56da486e4 100644 --- a/drivers/gpu/drm/amd/include/amd_shared.h +++ b/drivers/gpu/drm/amd/include/amd_shared.h @@ -240,6 +240,7 @@ enum amd_dpm_forced_level; * @late_init: sets up late driver/hw state (post hw_init) - Optional * @sw_init: sets up driver state, does not configure hw * @sw_fini: tears down driver state, does not configure hw + * @early_fini: tears down stuff before dev detached from driver * @hw_init: sets up the hw state * @hw_fini: tears down the hw state * @late_fini: final cleanup @@ -268,6 +269,7 @@ struct amd_ip_funcs { int (*late_init)(void *handle); int (*sw_init)(void *handle); int (*sw_fini)(void *handle); + int (*early_fini)(void *handle); int (*hw_init)(void *handle); int (*hw_fini)(void *handle); void (*late_fini)(void *handle); From patchwork Mon May 10 16:36:15 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrey Grodzovsky X-Patchwork-Id: 12248675 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-18.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,MSGID_FROM_MTA_HEADER,SPF_HELO_NONE, SPF_PASS,USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 06B78C433ED for ; Mon, 10 May 2021 16:37:08 +0000 (UTC) Received: from gabe.freedesktop.org (gabe.freedesktop.org [131.252.210.177]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id B87C861107 for ; Mon, 10 May 2021 16:37:07 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org B87C861107 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=amd.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=dri-devel-bounces@lists.freedesktop.org Received: from gabe.freedesktop.org (localhost [127.0.0.1]) by gabe.freedesktop.org (Postfix) with ESMTP id D98926E8AD; Mon, 10 May 2021 16:36:56 +0000 (UTC) Received: from NAM10-MW2-obe.outbound.protection.outlook.com (mail-mw2nam10on2088.outbound.protection.outlook.com [40.107.94.88]) by gabe.freedesktop.org (Postfix) with ESMTPS id CF7B26E8B1; Mon, 10 May 2021 16:36:53 +0000 (UTC) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=Vg7o3wp+DC4aUqq18I66eCiemL+Twx/1x2OmdKqFUpD7aG5bmmoJE2paQpm+6sXunsAyJWcaJvbHxQYE6x4Zk0OjVGXSuR/ZMx8LBQR581ogjw8F/U8pLiGHZHe8kKXjrEHmeAzMDJfFdLOuFXkV5raTZ6K4hrsqcWMNGBiswWLk7tLeXF2tVvotkilRoNDiK72fbgM/2vAmYBlhIxc6ZutIsIFGtBc+Fs+E5OYuuqU8ix1xhsvUN1oi6tHWNhrUxbaY3dD1lJr1m50tkkIpi4hlsoeyAL7uttIjHl25wAgDuZzZGMKAUofszW1AOaHIRNNprT/d+P8l3sW6mTW+Vw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=mzzgbK6At32mjHbaASDvGlUhyVoxCDoZ8d3LjhCC41A=; b=KIaqcBUbmUfHy/ZJzpS9rvlbr+FRzQRUH3X4qoNTBDBv4G2nWD9K0Coi68Bu1OYArzM3XJ6C86A3pqCL09XqsSVnWC1BbWh2XfwNrRDkLoqRuHhxThX9FqK0qkdPix3epCqAPDltgpajIOQsedX+qhn5JvdFEqBVNeUw9FDDmxNH3gIomenv1ZTtf3yj5LMYfwAKbDMUG08YwQ9PhTEEFuNa6LzTKEbvdMV90pkZ5f2o5RV17SI7NaHJ3131UR3OWCZN7t5fTKZghayjYfLVTMkHTtKhtxWScLss6QyBQ7CBPvRF4E/wAvyA1OJb364x36a1vYStSP572KPXYACoIA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=amd.com; dmarc=pass action=none header.from=amd.com; dkim=pass header.d=amd.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=mzzgbK6At32mjHbaASDvGlUhyVoxCDoZ8d3LjhCC41A=; b=WTJlSUqmAh9MQpkzu2IRtkfWcOKMk2X40d433nVEf/RoySLGPYq6+bmfy9yXf4Jo86E1rXwigQBEHeUIV1Nuoo/wen+igZK0pyCIr9F32h23EtuNf1afk2/IYkGbfOLAzevY4eWsXTXHwiX49ewEKbmugl8bwJsZLeC3n6Wsm+k= Authentication-Results: lists.freedesktop.org; dkim=none (message not signed) header.d=none; lists.freedesktop.org; dmarc=none action=none header.from=amd.com; Received: from SN6PR12MB4623.namprd12.prod.outlook.com (2603:10b6:805:e9::17) by SA0PR12MB4462.namprd12.prod.outlook.com (2603:10b6:806:95::22) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4108.25; Mon, 10 May 2021 16:36:52 +0000 Received: from SN6PR12MB4623.namprd12.prod.outlook.com ([fe80::ad51:8c49:b171:856c]) by SN6PR12MB4623.namprd12.prod.outlook.com ([fe80::ad51:8c49:b171:856c%7]) with mapi id 15.20.4108.031; Mon, 10 May 2021 16:36:52 +0000 From: Andrey Grodzovsky To: dri-devel@lists.freedesktop.org, amd-gfx@lists.freedesktop.org, linux-pci@vger.kernel.org, ckoenig.leichtzumerken@gmail.com, daniel.vetter@ffwll.ch, Harry.Wentland@amd.com Subject: [PATCH v6 06/16] drm/amdgpu: Handle IOMMU enabled case. Date: Mon, 10 May 2021 12:36:15 -0400 Message-Id: <20210510163625.407105-7-andrey.grodzovsky@amd.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210510163625.407105-1-andrey.grodzovsky@amd.com> References: <20210510163625.407105-1-andrey.grodzovsky@amd.com> X-Originating-IP: [2607:fea8:3edf:49b0:6a5:47b8:e610:f6a3] X-ClientProxiedBy: YT2PR01CA0020.CANPRD01.PROD.OUTLOOK.COM (2603:10b6:b01:38::25) To SN6PR12MB4623.namprd12.prod.outlook.com (2603:10b6:805:e9::17) MIME-Version: 1.0 X-MS-Exchange-MessageSentRepresentingType: 1 Received: from agrodzovsky-All-Series.hitronhub.home (2607:fea8:3edf:49b0:6a5:47b8:e610:f6a3) by YT2PR01CA0020.CANPRD01.PROD.OUTLOOK.COM (2603:10b6:b01:38::25) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4108.31 via Frontend Transport; Mon, 10 May 2021 16:36:51 +0000 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 48ba7869-5ef1-4baa-f203-08d913d1d105 X-MS-TrafficTypeDiagnostic: SA0PR12MB4462: X-MS-Exchange-Transport-Forked: True X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:1013; X-MS-Exchange-SenderADCheck: 1 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: q56mg5686nQa4vwuDM19EDLE29beIMJcRI85NU2P8MTFsSlOKqKQ4NmPtY9lm8HcsDXwhOJ9ordmzMxj6Ji96ZIbQwLUgtYDarVMsAzOAiEuX8dSA7d1XUIYzP/Ww0Alg06AZ/R+gdrqo39m6vSWo52jX1OfBbGd0Ivf4vNSDS4d/u5mKDFmVbGgh/l9Cd+JhZNcY3fhuJcXO4QFczqyqfh004ESDqzim3j6H2bKXldzYurc3EgNXosA8xC23N75zqvqqqO9tUfBfnjlqfKOLLeNEj3eqpbKBOu7NWj3PvrtP+SdHPmijJnembmO+rttMzQiGiMn+jqPdtf9r0km1px/VyJWOCkxyKDpzbtsY08RVSbsM+m0zNxk1rsxMt/bh39O0TqdKQ0ebB7MzICzdsRiCUD0LjsLHnq2w9LH949rTGIStN2FlSCuU8X47wrR0tUPDXo0irZkqPQE8ew3Ko8H5ch5DrJ4Zuskb1sqyUVePnL2o7HGr9CeF90TaNUH4Gqh6J2TbDtrArQWlQXp4Zf4fnEUhZjp0CVaAeFpfD8q51qY759UcBG9WRalqWkam/mAxEP9gqwvrhBwccJD07HykFqLTIKLfEsIWP8O4Ik= X-Forefront-Antispam-Report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:SN6PR12MB4623.namprd12.prod.outlook.com; PTR:; CAT:NONE; SFS:(4636009)(366004)(136003)(39860400002)(396003)(376002)(346002)(86362001)(6506007)(38100700002)(52116002)(6666004)(44832011)(8676002)(36756003)(5660300002)(478600001)(8936002)(2906002)(6486002)(66946007)(2616005)(66476007)(66556008)(1076003)(316002)(6636002)(6512007)(16526019)(186003)(83380400001)(4326008); DIR:OUT; SFP:1101; X-MS-Exchange-AntiSpam-MessageData: RrG2OfyILByiEQD4omNy1BzCZ1NpGMkFDOAqZ7tNFXHCDdSX95xaJ6wI9JpWaoJKkep7Y6RO0yL3KRqc/4YmMYy/0VBm/vh00V+YRp2KLOVPjfBunPg8I+kTP3i1WZcqN3v/gEcv6RkcJHXNB5LQjJkkKeA1RXPxUXRRzZTVsdD0zIauF8plku6wC07yBlWj1TyLOMyIdEF+fD5sOUSzJsVH09OuwKMGKonKqTPUZQNfaOUd1e46SqoXB9gUz6Fr3wibJD4EcNQvu+GaalgCdryyGd0AgmJQuRVph3YbeSBjCf7M66EV0mgtlNpvjO5LgJhGt5f+ONPPjHlH9z/gvdvi+GAHvRgI8MLlSI/NlRFd6A2n2SeYAXEgi81JPbFWGMd2TvyUsTB6YQqBGHkz4nq/fXfPYoVp8Fylks42lvAUB5hNV0EkewVPBPVR6Yp6RBTA3luhn0wG+/At9RZZZMak4lZXScPYMQjvhpLvRBPnTk/5qjHHAMniS1K6AdydEEjmtQAp8n+aGYnnaqFwBVWDkm1cs0vAbbnHl+fx7hM8t0Zew/5hMjotuIOTtn6XEn2KiXI+PhzGP1xUJ16UKltaIRfjMS4p55pmM4uKKp8XNX4KTHnWOKwSYSi2ojwncwPKOaXZFAzBY+0vWeclb0+JSsTbSU2cyyimwRbF79cMPs7qZ4KhxJ/HuYSXtPry8V8KyqqE0c3T4kG2+3CZrZVW0iCZhNd7+9hlv7GE2sP8kKsCOArhXpu/QipFmtExGhjkkRhXRyvxoE5Tr6Cg01zy/ksIAtap8DwybR7BCNo5CK/ZLBxlZmagyS2IsaUm+o9OGsdqkcY3FXWLiPSQAGQSBHVvzF/nSSL5ICLIcII8ECiD8h5isxVpCVgn0egMKDzhPFqmb1dILSngjawZprGvazVU4xDFuSf05D35no16gVjowzBYCvV85gGMuhrVgKRujGhAdA73+zuY6FyWELQKJBUFQGNNNd3JG8i0L0alqWMbpaECHyCwkAM9B7PCvtygz1KGON3pTgYN/BZxCBGUUncrMohcm1hTZCEmd7XoF2t8lnK/GZtzdNRmxX0Q3Xbd+fXYdDiuQBxpHdeOj1H3aHbvoWAx2uQbUlnwlZ92kcdjdLZVNblFZA93g5BSSdt11FIOW4rEDYrcRv7XxxFJVS2rjEtloqL54u4Og0AMMPZRjdI5mxpDf2G4L1P7uGIxbFr0bg8gDsYj6hzawnDNpmBLaczTYrlIFthqUmsdySJUBkB5Btr0v+Y133mIS88UCHlDRGWz6pjRa/YTK3kl5OKnEYi8SHmJc+u1340R7ObajYlaZ/G1Dgdvd3F2Zo9D/bqAeLY12s2mX82DmDC4Z157TD4AYLIqh6G6iipUbBD+zviwBog+dsMAqo9O X-OriginatorOrg: amd.com X-MS-Exchange-CrossTenant-Network-Message-Id: 48ba7869-5ef1-4baa-f203-08d913d1d105 X-MS-Exchange-CrossTenant-AuthSource: SN6PR12MB4623.namprd12.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 10 May 2021 16:36:52.6945 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: pe4s73ZAdV4GfxrtmK+QZmKVCSpVyWIUxbdjEykzLJ9SO4uM6PntCMoDV3Fh2idfkcTEF/0QUq3zhHmX8xtcxg== X-MS-Exchange-Transport-CrossTenantHeadersStamped: SA0PR12MB4462 X-BeenThere: dri-devel@lists.freedesktop.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Direct Rendering Infrastructure - Development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: gregkh@linuxfoundation.org, Felix.Kuehling@amd.com, helgaas@kernel.org, Alexander.Deucher@amd.com Errors-To: dri-devel-bounces@lists.freedesktop.org Sender: "dri-devel" Handle all DMA IOMMU gropup related dependencies before the group is removed. v5: Drop IOMMU notifier and switch to lockless call to ttm_tt_unpopulate v6: Drop the BO unamp list Signed-off-by: Andrey Grodzovsky --- drivers/gpu/drm/amd/amdgpu/amdgpu_device.c | 4 ++-- drivers/gpu/drm/amd/amdgpu/amdgpu_gart.c | 3 +-- drivers/gpu/drm/amd/amdgpu/amdgpu_gart.h | 1 + drivers/gpu/drm/amd/amdgpu/amdgpu_irq.c | 9 +++++++++ drivers/gpu/drm/amd/amdgpu/cik_ih.c | 1 - drivers/gpu/drm/amd/amdgpu/cz_ih.c | 1 - drivers/gpu/drm/amd/amdgpu/iceland_ih.c | 1 - drivers/gpu/drm/amd/amdgpu/navi10_ih.c | 3 --- drivers/gpu/drm/amd/amdgpu/si_ih.c | 1 - drivers/gpu/drm/amd/amdgpu/tonga_ih.c | 1 - drivers/gpu/drm/amd/amdgpu/vega10_ih.c | 3 --- 11 files changed, 13 insertions(+), 15 deletions(-) diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c index 18598eda18f6..a0bff4713672 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c @@ -3256,7 +3256,6 @@ static const struct attribute *amdgpu_dev_attributes[] = { NULL }; - /** * amdgpu_device_init - initialize the driver * @@ -3698,12 +3697,13 @@ void amdgpu_device_fini_hw(struct amdgpu_device *adev) amdgpu_ucode_sysfs_fini(adev); sysfs_remove_files(&adev->dev->kobj, amdgpu_dev_attributes); - amdgpu_fbdev_fini(adev); amdgpu_irq_fini_hw(adev); amdgpu_device_ip_fini_early(adev); + + amdgpu_gart_dummy_page_fini(adev); } void amdgpu_device_fini_sw(struct amdgpu_device *adev) diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_gart.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_gart.c index c5a9a4fb10d2..354e68081b53 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_gart.c +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_gart.c @@ -92,7 +92,7 @@ static int amdgpu_gart_dummy_page_init(struct amdgpu_device *adev) * * Frees the dummy page used by the driver (all asics). */ -static void amdgpu_gart_dummy_page_fini(struct amdgpu_device *adev) +void amdgpu_gart_dummy_page_fini(struct amdgpu_device *adev) { if (!adev->dummy_page_addr) return; @@ -375,5 +375,4 @@ int amdgpu_gart_init(struct amdgpu_device *adev) */ void amdgpu_gart_fini(struct amdgpu_device *adev) { - amdgpu_gart_dummy_page_fini(adev); } diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_gart.h b/drivers/gpu/drm/amd/amdgpu/amdgpu_gart.h index a25fe97b0196..78dc7a23da56 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_gart.h +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_gart.h @@ -58,6 +58,7 @@ int amdgpu_gart_table_vram_pin(struct amdgpu_device *adev); void amdgpu_gart_table_vram_unpin(struct amdgpu_device *adev); int amdgpu_gart_init(struct amdgpu_device *adev); void amdgpu_gart_fini(struct amdgpu_device *adev); +void amdgpu_gart_dummy_page_fini(struct amdgpu_device *adev); int amdgpu_gart_unbind(struct amdgpu_device *adev, uint64_t offset, int pages); int amdgpu_gart_map(struct amdgpu_device *adev, uint64_t offset, diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_irq.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_irq.c index 233b64dab94b..a14973a7a9c9 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_irq.c +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_irq.c @@ -361,6 +361,15 @@ void amdgpu_irq_fini_hw(struct amdgpu_device *adev) if (!amdgpu_device_has_dc_support(adev)) flush_work(&adev->hotplug_work); } + + if (adev->irq.ih_soft.ring) + amdgpu_ih_ring_fini(adev, &adev->irq.ih_soft); + if (adev->irq.ih.ring) + amdgpu_ih_ring_fini(adev, &adev->irq.ih); + if (adev->irq.ih1.ring) + amdgpu_ih_ring_fini(adev, &adev->irq.ih1); + if (adev->irq.ih2.ring) + amdgpu_ih_ring_fini(adev, &adev->irq.ih2); } /** diff --git a/drivers/gpu/drm/amd/amdgpu/cik_ih.c b/drivers/gpu/drm/amd/amdgpu/cik_ih.c index 183d44a6583c..df385ffc9768 100644 --- a/drivers/gpu/drm/amd/amdgpu/cik_ih.c +++ b/drivers/gpu/drm/amd/amdgpu/cik_ih.c @@ -310,7 +310,6 @@ static int cik_ih_sw_fini(void *handle) struct amdgpu_device *adev = (struct amdgpu_device *)handle; amdgpu_irq_fini_sw(adev); - amdgpu_ih_ring_fini(adev, &adev->irq.ih); amdgpu_irq_remove_domain(adev); return 0; diff --git a/drivers/gpu/drm/amd/amdgpu/cz_ih.c b/drivers/gpu/drm/amd/amdgpu/cz_ih.c index d32743949003..b8c47e0cf37a 100644 --- a/drivers/gpu/drm/amd/amdgpu/cz_ih.c +++ b/drivers/gpu/drm/amd/amdgpu/cz_ih.c @@ -302,7 +302,6 @@ static int cz_ih_sw_fini(void *handle) struct amdgpu_device *adev = (struct amdgpu_device *)handle; amdgpu_irq_fini_sw(adev); - amdgpu_ih_ring_fini(adev, &adev->irq.ih); amdgpu_irq_remove_domain(adev); return 0; diff --git a/drivers/gpu/drm/amd/amdgpu/iceland_ih.c b/drivers/gpu/drm/amd/amdgpu/iceland_ih.c index da96c6013477..ddfe4eaeea05 100644 --- a/drivers/gpu/drm/amd/amdgpu/iceland_ih.c +++ b/drivers/gpu/drm/amd/amdgpu/iceland_ih.c @@ -301,7 +301,6 @@ static int iceland_ih_sw_fini(void *handle) struct amdgpu_device *adev = (struct amdgpu_device *)handle; amdgpu_irq_fini_sw(adev); - amdgpu_ih_ring_fini(adev, &adev->irq.ih); amdgpu_irq_remove_domain(adev); return 0; diff --git a/drivers/gpu/drm/amd/amdgpu/navi10_ih.c b/drivers/gpu/drm/amd/amdgpu/navi10_ih.c index 5eea4550b856..e171a9e78544 100644 --- a/drivers/gpu/drm/amd/amdgpu/navi10_ih.c +++ b/drivers/gpu/drm/amd/amdgpu/navi10_ih.c @@ -571,9 +571,6 @@ static int navi10_ih_sw_fini(void *handle) amdgpu_irq_fini_sw(adev); amdgpu_ih_ring_fini(adev, &adev->irq.ih_soft); - amdgpu_ih_ring_fini(adev, &adev->irq.ih2); - amdgpu_ih_ring_fini(adev, &adev->irq.ih1); - amdgpu_ih_ring_fini(adev, &adev->irq.ih); return 0; } diff --git a/drivers/gpu/drm/amd/amdgpu/si_ih.c b/drivers/gpu/drm/amd/amdgpu/si_ih.c index 751307f3252c..9a24f17a5750 100644 --- a/drivers/gpu/drm/amd/amdgpu/si_ih.c +++ b/drivers/gpu/drm/amd/amdgpu/si_ih.c @@ -176,7 +176,6 @@ static int si_ih_sw_fini(void *handle) struct amdgpu_device *adev = (struct amdgpu_device *)handle; amdgpu_irq_fini_sw(adev); - amdgpu_ih_ring_fini(adev, &adev->irq.ih); return 0; } diff --git a/drivers/gpu/drm/amd/amdgpu/tonga_ih.c b/drivers/gpu/drm/amd/amdgpu/tonga_ih.c index 973d80ec7f6c..b08905d1c00f 100644 --- a/drivers/gpu/drm/amd/amdgpu/tonga_ih.c +++ b/drivers/gpu/drm/amd/amdgpu/tonga_ih.c @@ -313,7 +313,6 @@ static int tonga_ih_sw_fini(void *handle) struct amdgpu_device *adev = (struct amdgpu_device *)handle; amdgpu_irq_fini_sw(adev); - amdgpu_ih_ring_fini(adev, &adev->irq.ih); amdgpu_irq_remove_domain(adev); return 0; diff --git a/drivers/gpu/drm/amd/amdgpu/vega10_ih.c b/drivers/gpu/drm/amd/amdgpu/vega10_ih.c index dead9c2fbd4c..d78b8abe993a 100644 --- a/drivers/gpu/drm/amd/amdgpu/vega10_ih.c +++ b/drivers/gpu/drm/amd/amdgpu/vega10_ih.c @@ -515,9 +515,6 @@ static int vega10_ih_sw_fini(void *handle) amdgpu_irq_fini_sw(adev); amdgpu_ih_ring_fini(adev, &adev->irq.ih_soft); - amdgpu_ih_ring_fini(adev, &adev->irq.ih2); - amdgpu_ih_ring_fini(adev, &adev->irq.ih1); - amdgpu_ih_ring_fini(adev, &adev->irq.ih); return 0; } From patchwork Mon May 10 16:36:16 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Andrey Grodzovsky X-Patchwork-Id: 12248673 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-18.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,MSGID_FROM_MTA_HEADER,SPF_HELO_NONE, SPF_PASS,USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 2D94FC433B4 for ; Mon, 10 May 2021 16:37:06 +0000 (UTC) Received: from gabe.freedesktop.org (gabe.freedesktop.org [131.252.210.177]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id DC73161155 for ; Mon, 10 May 2021 16:37:05 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org DC73161155 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=amd.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=dri-devel-bounces@lists.freedesktop.org Received: from gabe.freedesktop.org (localhost [127.0.0.1]) by gabe.freedesktop.org (Postfix) with ESMTP id 214D26E8B5; Mon, 10 May 2021 16:36:57 +0000 (UTC) Received: from NAM11-CO1-obe.outbound.protection.outlook.com (mail-co1nam11on2081.outbound.protection.outlook.com [40.107.220.81]) by gabe.freedesktop.org (Postfix) with ESMTPS id 6B89C6E8AD; Mon, 10 May 2021 16:36:55 +0000 (UTC) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=DY0+Pxbnuj+jRO5JeW71jQVuQsWzUk6njv3luA+k0/kLdytnhtqfTEqRZyk5HLltrmaj7t8mnAbfdeHBW8ekif0SMtPLRxglYuiNR/3uBzVlJRc/qCu4RknJeOxlH/QlZkgWKtEG4EmNHAyPizibrYwFmEl0sk+2aBLEIwkRHVmMeJe3/lLJnt1xq2NKsN69P4ox3575YNl5ccOscnXiZ2/Wfz5bUErHn/San51o67LMIVPss/3qdCUk7SR40j6sAAudcnR2XeQnArlQmfDujSpvDP9IcqjeWlQzIFdZah+zpNvbjyo3OXOd1/WRfrltvT5Ubt7RLmtNHK/ypY0ZFw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=ImiQhpPoXfItMc84rz4U+sl7ukQBW3gycLm8/fXkr9o=; b=FVkfI/ZvATN/g8uXINeVRxzj1XMLglvQ3hjQQ5eSpDhlxFIDkjQauDCu4u3jNaJeM6QWeL1dq1oJ6x9zQGJaM809Wlu9t8syDPXMTt5ZLs2an1Lgo6ZPSm/EPOu+ogEo+t8OZ090so2Stqn2YiA4Bae+kje/t5BdPg70f72PGs+GXHfX9Tjtbx0kgucajLppYCVSZ+73WvQZLBA159aEycKmMrpF45orbp7EJeqBigJ3WKv0bbIoGwZlokeougqsdcNTVe1g1hc9UlmH4HQroGPdLJpyjsRc0AiV4jWhe0wenQL7tQGUqqriTjBJya25RULAP4d4ZHWFVUc3Cfm13w== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=amd.com; dmarc=pass action=none header.from=amd.com; dkim=pass header.d=amd.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=ImiQhpPoXfItMc84rz4U+sl7ukQBW3gycLm8/fXkr9o=; b=QvX5zpMwKJS7tM33SuWHN5dgn4f9sHuvaFU0paoYM3fi9C+YinJGg0g6wdW4yvXjbysnMB2uo1bwuI7zFeXNamy38XFb9Yk3UcK0KkGNV8vueUtE8LaXi/a/7foIdX3v2V+wPjMYdES0u2NPbQ4NjcNKttKcUGqdUy84Is1R7MY= Authentication-Results: lists.freedesktop.org; dkim=none (message not signed) header.d=none; lists.freedesktop.org; dmarc=none action=none header.from=amd.com; Received: from SN6PR12MB4623.namprd12.prod.outlook.com (2603:10b6:805:e9::17) by SA0PR12MB4462.namprd12.prod.outlook.com (2603:10b6:806:95::22) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4108.25; Mon, 10 May 2021 16:36:54 +0000 Received: from SN6PR12MB4623.namprd12.prod.outlook.com ([fe80::ad51:8c49:b171:856c]) by SN6PR12MB4623.namprd12.prod.outlook.com ([fe80::ad51:8c49:b171:856c%7]) with mapi id 15.20.4108.031; Mon, 10 May 2021 16:36:54 +0000 From: Andrey Grodzovsky To: dri-devel@lists.freedesktop.org, amd-gfx@lists.freedesktop.org, linux-pci@vger.kernel.org, ckoenig.leichtzumerken@gmail.com, daniel.vetter@ffwll.ch, Harry.Wentland@amd.com Subject: [PATCH v6 07/16] drm/amdgpu: Remap all page faults to per process dummy page. Date: Mon, 10 May 2021 12:36:16 -0400 Message-Id: <20210510163625.407105-8-andrey.grodzovsky@amd.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210510163625.407105-1-andrey.grodzovsky@amd.com> References: <20210510163625.407105-1-andrey.grodzovsky@amd.com> X-Originating-IP: [2607:fea8:3edf:49b0:6a5:47b8:e610:f6a3] X-ClientProxiedBy: YT2PR01CA0020.CANPRD01.PROD.OUTLOOK.COM (2603:10b6:b01:38::25) To SN6PR12MB4623.namprd12.prod.outlook.com (2603:10b6:805:e9::17) MIME-Version: 1.0 X-MS-Exchange-MessageSentRepresentingType: 1 Received: from agrodzovsky-All-Series.hitronhub.home (2607:fea8:3edf:49b0:6a5:47b8:e610:f6a3) by YT2PR01CA0020.CANPRD01.PROD.OUTLOOK.COM (2603:10b6:b01:38::25) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4108.31 via Frontend Transport; Mon, 10 May 2021 16:36:53 +0000 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 2f7cca66-b0cd-4978-2a5e-08d913d1d204 X-MS-TrafficTypeDiagnostic: SA0PR12MB4462: X-MS-Exchange-Transport-Forked: True X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:2000; X-MS-Exchange-SenderADCheck: 1 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: TyrhCVAyZhOiuh1JFseEDpL96lsuEALzha62o53XF1TZIS9fz05CKHqPO/w7S2rMGdiKbT4nBlnQH6XkQYoQ+l3aKSvD8XVeERoAN9CGXgvToUoFN9vbIkrBY7SANrORlNySyQXOZEx93BTMMyoCrZK++AVE3Mur8cWvy+SWR5R7xdHP+x/y31kABsMJA73hTMHct6RMd6fUvqqToLAGpskQAijKLJHIr3XhmqeYLsQJWtJkIX/mUN3dbNWme40kEnJf1YIiXpiVYTLfhZ47b5Vlfqk1jvgFQd0Z8+LG4oZAIGiORdtluKMzAFBHCRtq8P2C/uPM2vkOu1ybtaJkc+WieG9QbGOdTxJ+4tSC5BzNXr6XjgfV0NN6QzCG7rlCHFS9KXwNBKWb9zDYCvb4RabS5rNkkNPgi5kYW5/5kai3QmBRPR9oxqPR7LhuaVhS6MNyRsoNPvHuF+EV6fs0+VYtjIM/3+tJccWPK9nGMtj7pZvnTjenGUKLgVJTF5mk3C9yGz6+g4vN87YceLDEh8Gwmxm+0Q8tWhbJ8YQIC1Kzxe7l7FcMV9pvFRNNU+eQnjP30G6G4+9Iva3LQJN80W6n0aXMSWyGL2vvLOg0D0c= X-Forefront-Antispam-Report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:SN6PR12MB4623.namprd12.prod.outlook.com; PTR:; CAT:NONE; SFS:(4636009)(366004)(136003)(39860400002)(396003)(376002)(346002)(86362001)(6506007)(38100700002)(52116002)(6666004)(44832011)(8676002)(36756003)(5660300002)(478600001)(8936002)(2906002)(6486002)(54906003)(66946007)(2616005)(66476007)(66556008)(1076003)(316002)(6636002)(6512007)(16526019)(186003)(83380400001)(4326008)(66574015); DIR:OUT; SFP:1101; X-MS-Exchange-AntiSpam-MessageData: =?utf-8?q?7lcGkH1qx3rB3FwJenFAIFtopW5W4b?= =?utf-8?q?h4dXiXZ9MxmxsFldiItDPZW3G9Fw/TjT7MnLBNUjKIIolIxaX6ZRQDbV7V7siwDZq?= =?utf-8?q?DiGe8d7iLApXnCt9aYqlEH72Jkh+7Z4maglHvzMbTo0jRErRQUGiH8uT2lrBZ0QAD?= =?utf-8?q?lroWhW99gSlWO5YbVXyQKf/YFKfQJ78LNpwowJWKd+pi3qfRHvC7lpSZ3pw6ilwN/?= =?utf-8?q?JDDeyrMwcN1/il+cPLZ8c/xKMTwbjRebUu+JbBvX/GwgZVwFWRqeSbp56zjbMfESn?= =?utf-8?q?LBCjISkCCQnlDztO9Cadx+p4Qj2x/yEXu6zZv9iJVOxRmdpmhjeDoB6giv4bPLwi1?= =?utf-8?q?G5I3qhmlOfZeQLIbsu+HluLHiic2bem8iRui8uTZy0gFnDyWGhqUxyAhfcpG+bYvi?= =?utf-8?q?G64Qzh3w9U3/+KaD+lPeLxWLQDdejikFT/J/qcYY0XI30fmD5kAMGPkkijAokfDZ6?= =?utf-8?q?f+qk1SG2+jYU7Vm4d9Pe9kOlzk9AOu+odDPv2OTz0t8ggTXRkOH379iyqKGcLJa2B?= =?utf-8?q?6A328CSThJscUuNsuz1G1Qu0HQdM8hNk7WH27bj0jY5IAog+z0MlhV31rh0aN9pKD?= =?utf-8?q?xnEFHpCrntn+nqwbxF6ZoPqyOxncQ17Mfu8VMcjncYeQA0qtVhZTOrX8+8/AR5ZVQ?= =?utf-8?q?0fyDCsx8Hbb/EPE93poQ47kTbajL+ywQAoWLHziDWaNy3+tiIKZnJ1rlab7xGvoLm?= =?utf-8?q?K+ArScv6ewRc7wuNQshSVDfJxshRDd7ax8NKxEZAS3etpcNmPLqNoeREXVfSUjwp+?= =?utf-8?q?FrAPTSUIVcRBQwVGEyXIuXAGMPDK/+llLwgN5mLLo5Hdt40CJt3kOlYCBf6R3/9/w?= =?utf-8?q?ptzjkBVcUiF7rYIt9pHr8QC9UvE16EaLrD5jZjyPcJCIetntlFxsORXkxneLiUM9b?= =?utf-8?q?VyqFCn7LYnmxiv7TVv0N3e8lkAHlltisHHNTs1JjitzRfEqC5X0J9vW9uwpIHiEmu?= =?utf-8?q?Cs1AY/cUYeDqdajAvu8GW/+RIHAypesnW7Zn4EyuRvrztK1TCrWL+ay4DCmaD6uK5?= =?utf-8?q?FmLo58n0L/VtKwvO7/G/peaJADvMGasoix4ZqC3YjoG1ofbW3jqzqmoBDm7PyssKB?= =?utf-8?q?wl3v37wxI0yqWlOcyAl7pHNlVmecYmglxAP1qZcDwvcRpwWGjl7voNqQLX6Hstn+Q?= =?utf-8?q?B4QqhaOv1FC/HoxtHQbAV0mqdqHWZFC5vSPXHALTY2iwLCTCFiqmCUtZ69+wP4KRi?= =?utf-8?q?7X0vGbdfX7BPBYetNb1XHbI37qjcN/ZHbwXK/+mv+gTSnbJq7u0CPZBzdstKarrf+?= =?utf-8?q?28mIam48cD9jpEpMd19tfNCvN8QxIBHOV4+Qf0NUcJx3LkiV69X3SKpnLZjdtuOG1?= =?utf-8?q?T6wumU1K+jlPt?= X-OriginatorOrg: amd.com X-MS-Exchange-CrossTenant-Network-Message-Id: 2f7cca66-b0cd-4978-2a5e-08d913d1d204 X-MS-Exchange-CrossTenant-AuthSource: SN6PR12MB4623.namprd12.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 10 May 2021 16:36:54.3576 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: VS9pumjICUHa4+Ie6QsUhWCdFqtGW7OD4ZFxNvJzxjjbZ8RTEZPL99ybOfGKTjc3TYxx18MarfkiEmvWsWa7qg== X-MS-Exchange-Transport-CrossTenantHeadersStamped: SA0PR12MB4462 X-BeenThere: dri-devel@lists.freedesktop.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Direct Rendering Infrastructure - Development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: gregkh@linuxfoundation.org, Felix.Kuehling@amd.com, helgaas@kernel.org, Alexander.Deucher@amd.com, =?utf-8?q?Christian_K=C3=B6nig?= Errors-To: dri-devel-bounces@lists.freedesktop.org Sender: "dri-devel" On device removal reroute all CPU mappings to dummy page per drm_file instance or imported GEM object. v4: Update for modified ttm_bo_vm_dummy_page Signed-off-by: Andrey Grodzovsky Reviewed-by: Christian König --- drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.c | 21 ++++++++++++++++----- 1 file changed, 16 insertions(+), 5 deletions(-) diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.c index 8c7ec09eb1a4..0d54e70278ca 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.c +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.c @@ -48,6 +48,7 @@ #include #include +#include #include "amdgpu.h" #include "amdgpu_object.h" @@ -1905,18 +1906,28 @@ void amdgpu_ttm_set_buffer_funcs_status(struct amdgpu_device *adev, bool enable) static vm_fault_t amdgpu_ttm_fault(struct vm_fault *vmf) { struct ttm_buffer_object *bo = vmf->vma->vm_private_data; + struct drm_device *ddev = bo->base.dev; vm_fault_t ret; + int idx; ret = ttm_bo_vm_reserve(bo, vmf); if (ret) return ret; - ret = amdgpu_bo_fault_reserve_notify(bo); - if (ret) - goto unlock; + if (drm_dev_enter(ddev, &idx)) { + ret = amdgpu_bo_fault_reserve_notify(bo); + if (ret) { + drm_dev_exit(idx); + goto unlock; + } - ret = ttm_bo_vm_fault_reserved(vmf, vmf->vma->vm_page_prot, - TTM_BO_VM_NUM_PREFAULT, 1); + ret = ttm_bo_vm_fault_reserved(vmf, vmf->vma->vm_page_prot, + TTM_BO_VM_NUM_PREFAULT, 1); + + drm_dev_exit(idx); + } else { + ret = ttm_bo_vm_dummy_page(vmf, vmf->vma->vm_page_prot); + } if (ret == VM_FAULT_RETRY && !(vmf->flags & FAULT_FLAG_RETRY_NOWAIT)) return ret; From patchwork Mon May 10 16:36:17 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrey Grodzovsky X-Patchwork-Id: 12248677 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-18.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,MSGID_FROM_MTA_HEADER,SPF_HELO_NONE, SPF_PASS,USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id BF0D7C43461 for ; Mon, 10 May 2021 16:37:10 +0000 (UTC) Received: from gabe.freedesktop.org (gabe.freedesktop.org [131.252.210.177]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 6F97D61165 for ; Mon, 10 May 2021 16:37:10 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 6F97D61165 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=amd.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=dri-devel-bounces@lists.freedesktop.org Received: from gabe.freedesktop.org (localhost [127.0.0.1]) by gabe.freedesktop.org (Postfix) with ESMTP id E52D36E8CB; Mon, 10 May 2021 16:36:59 +0000 (UTC) Received: from NAM11-CO1-obe.outbound.protection.outlook.com (mail-co1nam11on2069.outbound.protection.outlook.com [40.107.220.69]) by gabe.freedesktop.org (Postfix) with ESMTPS id C3DD16E8B7; Mon, 10 May 2021 16:36:57 +0000 (UTC) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=H3e5yK5h3vvdy8IRUflO8u8BHkgDi/50MOQTLqa/gr5ek6Hy8Fo1Mq/n7oLBfSUPvwAQkE9IA1qfi3gkVsutKHYLmQ4SddNZ/yO4u1YhBCtXIQYK34t/Vc21T5JOdLosOZsPRA7hQZ78n7Y1NcePXhW1nzssISgmrnRyQG3Sz7urfFfg3ASZiEZiF6YgXzdOEgff2N43080C0Vae5GvTtaX9zIJNjZpxV9sc1v1JxQbeMSGa1io/SCRIdp38TaO2+y/hHUim7FHbtByBhtuUlPRqjydSGmmKgmLDb7cALTyGsf/nsZ6vlke1UazF9vDrrtdrRyFCOlvS4zTDHv/VOA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=YfW+qrJpKhR5p+rD+wQdoutlcOFsn4QAumXGO4q0St4=; b=iVO8SOdt+pJPzP3jEsQ0ra86fEE+VwFp4+Ek/qPqLqDrVucy/Om9tcOwvHOij1AxOZOWNzOrE+pEle2LIaKhJR9pIwmbrXsxh7FuVWmxtApYGOy3HUcQVKfu1qz/YVSImkD7pm8m2gr0b7muCp1x3z+EvNdEUbIzLySFolynLyL4Ee5KS4KdgBBojW5i5PtMIEhNqxejhcUInz3nFbGj64NIfmVRiWWDiligDRCv3cX9FYdNP55yaeGymLugVmmoC6NR509b03nUywD3SiNU8Q+J9vy2ardpkPk6xh2iqNISec54TEIhjcuKqvbkTVnVV3hKvcQGHN7U1/HAJQfD3w== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=amd.com; dmarc=pass action=none header.from=amd.com; dkim=pass header.d=amd.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=YfW+qrJpKhR5p+rD+wQdoutlcOFsn4QAumXGO4q0St4=; b=sF0hCAFaWC0Asz9gQgqn6dAvKG8zbe5z99v/sPPmYYUVoX9N7VgF4IUDyBxCblbKSr1WhanFlR5s8Cd0An5hYYt/oXzgrxQJ8m4vpsBj/UZBzGsP6EJjtyBhDoq81Bl2O0elB9k6LWz87SmC+70Bxdp4dmxggHgeU7kK3p8seDg= Authentication-Results: lists.freedesktop.org; dkim=none (message not signed) header.d=none; lists.freedesktop.org; dmarc=none action=none header.from=amd.com; Received: from SN6PR12MB4623.namprd12.prod.outlook.com (2603:10b6:805:e9::17) by SN1PR12MB2397.namprd12.prod.outlook.com (2603:10b6:802:27::21) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4108.29; Mon, 10 May 2021 16:36:56 +0000 Received: from SN6PR12MB4623.namprd12.prod.outlook.com ([fe80::ad51:8c49:b171:856c]) by SN6PR12MB4623.namprd12.prod.outlook.com ([fe80::ad51:8c49:b171:856c%7]) with mapi id 15.20.4108.031; Mon, 10 May 2021 16:36:56 +0000 From: Andrey Grodzovsky To: dri-devel@lists.freedesktop.org, amd-gfx@lists.freedesktop.org, linux-pci@vger.kernel.org, ckoenig.leichtzumerken@gmail.com, daniel.vetter@ffwll.ch, Harry.Wentland@amd.com Subject: [PATCH v6 08/16] PCI: Add support for dev_groups to struct pci_device_driver Date: Mon, 10 May 2021 12:36:17 -0400 Message-Id: <20210510163625.407105-9-andrey.grodzovsky@amd.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210510163625.407105-1-andrey.grodzovsky@amd.com> References: <20210510163625.407105-1-andrey.grodzovsky@amd.com> X-Originating-IP: [2607:fea8:3edf:49b0:6a5:47b8:e610:f6a3] X-ClientProxiedBy: YT2PR01CA0020.CANPRD01.PROD.OUTLOOK.COM (2603:10b6:b01:38::25) To SN6PR12MB4623.namprd12.prod.outlook.com (2603:10b6:805:e9::17) MIME-Version: 1.0 X-MS-Exchange-MessageSentRepresentingType: 1 Received: from agrodzovsky-All-Series.hitronhub.home (2607:fea8:3edf:49b0:6a5:47b8:e610:f6a3) by YT2PR01CA0020.CANPRD01.PROD.OUTLOOK.COM (2603:10b6:b01:38::25) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4108.31 via Frontend Transport; Mon, 10 May 2021 16:36:54 +0000 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 54a98bf7-4684-4c6a-4d8f-08d913d1d2fa X-MS-TrafficTypeDiagnostic: SN1PR12MB2397: X-MS-Exchange-Transport-Forked: True X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:5797; X-MS-Exchange-SenderADCheck: 1 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: ooGQP1ZYap/yZnT2kSe2/8VAfjYDRKlfk5c2c51WNy1B43JxxGg1mWH2rDo1Wi+acCNZCr3ieqXlDNANnpl1W89uN3oM9pNUv1fdQX3ZL62CXqyb6n0b8oOinOSQZOw93SPWvne6ZmlIJdOTvkj59SZlyQeUXgeag0+Hx8j5GAGf9AbTzp3+Dr9980z4TcQ9WKCM4MN3SQ2zEj8fAB2EM747O/WMI8PMTeFzT7k3ULfbZRzmUfVQE9i4LHy8aMdJkjqPa7Q0efdmFO7d+9JOD0lhz57awWt4aCdukQ97L2A80v0USR/K95c5J9Gw1ZKnkAaK5H28SmYyWWQgAlGUwu7iMJl7DJJ5PcJETkml4o8SFIvRJogQEF7OJYxDlS2ZNu0y0MAavazZayRsw/DcoSPnESUEwIVMgDqPEro5AYgXtmvDsbyRk3RqBbRSb3v52arGsls40f/Hox6x4UaNNNIWipoNsGYp9k3QIrxMlrGmnVkotlCvuG+H2vRrwwJxSfjBq7CxufrtTo83ap6Bwm7qohwmNgf8SlwJPX0agRnsP0gAmlSZCaM9V0o5udIa0dqIs2mQHr7CD1QwxXihIx9WCJE6k87FNJfDQ7X+/kU= X-Forefront-Antispam-Report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:SN6PR12MB4623.namprd12.prod.outlook.com; PTR:; CAT:NONE; SFS:(4636009)(366004)(1076003)(6636002)(44832011)(83380400001)(6512007)(66946007)(66556008)(36756003)(2906002)(66476007)(6666004)(16526019)(5660300002)(52116002)(4326008)(8676002)(6506007)(2616005)(8936002)(6486002)(86362001)(38100700002)(186003)(498600001); DIR:OUT; SFP:1101; X-MS-Exchange-AntiSpam-MessageData: U7QcdvGikTgz/2gQXDqKBUOCyshZlNEESXLaPyL4cMrfQ+W+AzCI0aw6xlZ0qfBndWRCPysZeF62vadEdthVzla4lZIV0NbZ1TE7ks54wHazAmzibtqHtg8mQxyBVdE185I+KtFronX/b4MJLU0dT0/pLgy1WBJ1pIfCuv9qn9HWH1QIfrvwmodKUKJX5CSBq5Tcp7kkGkXbxkAU2oO1l2hibrrdthnTfZ2HF/2x03ZUuboJKLZGylcdt4eEOp+mRjblXf3eeF3QYuM9GurJzo7Uy5OO5N4txMmfcmHhl5/3VKiGieuqwI1s4JT6tO5zt4Dq5btTaP0slMj0Dv9cB4Mcq5arQFK+RR6b8F1kUhhBXGcKBigGfTRHDLGaWTpI9jyxuaShlSqjty0oJXWRP1fFWoiYdaRBDCJ4FC2P5qNjHct1r+igSbgJzHfG9slQZ4DpxmmWjhUG0UCnC43S9uQkbOjMS4BkMNP5+O3d375Ts3kdU3icERjtQCb965Y2wcd3HrLbfejg3bHhF+ITkCmStWJaVpkzc1WHBv+6ZxcAk1/Vmn4q+Bw0AViI5uAWzx5w2BUX+PVNYUto22UzI/NWXKJ7tQnyyhX91DGaXR6y/InuwdLahDE4eFeszDBjI1S2LKpagwE6NHI5Va71cQC+SXl0A3G/06QURGjk1BDmUXsMYoJ/ESK48GQkZGEmk1B/EgYJILd7gdp6Ppilba2NxsguhWl4mengiBmeY8G6T0YRbfQVItgVvSFTz58g+2U2QW1DeRwzvzvTBB4KzlA4HnlU2x2JMVO5+0r0F2vkdX8FKmExwXIknl1oUuhSgKmotqh29P128t4c0Imyg8kiwhu2jfpLwyrSLD+OydbKAeqDKzAOk2DejHedL+Ao6hM1vZzUTrjVnLVuKIfpvqltw9PLCprGCouW7gv7katmB7rNwxrBbRX+YnBTHWrpicuKJOtlYTCKZAUHqMqq8YufBl3YZFgd0LcH6bhXz91kIqEzKhRQ+aYBP8VubDQ1gARqvxjpiTyIwvz/qrELshu47dvtdCbmQljL5fQMa6gphm5GLm3TbmBjjVg8A19dgnsCNijOF1z1DaSWEMRZJeRuWNtxd4ZI6gItHUrW/c3mxM5v3ZV/mFCuWhhxQ1zJ/F4HJSxTzQwVyv4ikw2vCs3sRiqeXiMYpXUfOC4pOwz3HQYZplol2i/QtAgBNPmIUKfdrFa0WQlWzvfTYHG6uelWyJtNkvtR+mqaPdDsXn+ythZF1aUAPDWhKo5VWSdcrZzaqL6ArwgLet8KW4mGSYTBaCsRFrp9gLECZ8ice9aSjTtuq3FPSJiOqLNcbGfFxvSfEJpGZuXM36/Gmj+eFs0KpDN439X+T+LqJE9O5OQ5IqtsO+aCeXpPabJRUMl5 X-OriginatorOrg: amd.com X-MS-Exchange-CrossTenant-Network-Message-Id: 54a98bf7-4684-4c6a-4d8f-08d913d1d2fa X-MS-Exchange-CrossTenant-AuthSource: SN6PR12MB4623.namprd12.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 10 May 2021 16:36:55.9936 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: oE+CKvLaIncFOC7/5g1e7bId0gtA6dKYOQpMkQMhZfRb+/VxlFHSkv2t9uzc3L0AomBwB1efIMHPbSuuYftdSg== X-MS-Exchange-Transport-CrossTenantHeadersStamped: SN1PR12MB2397 X-BeenThere: dri-devel@lists.freedesktop.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Direct Rendering Infrastructure - Development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: gregkh@linuxfoundation.org, Felix.Kuehling@amd.com, helgaas@kernel.org, Alexander.Deucher@amd.com Errors-To: dri-devel-bounces@lists.freedesktop.org Sender: "dri-devel" This helps converting PCI drivers sysfs attributes to static. Analogous to b71b283e3d6d ("USB: add support for dev_groups to struct usb_driver") Signed-off-by: Andrey Grodzovsky Suggested-by: Greg Kroah-Hartman Acked-by: Bjorn Helgaas --- drivers/pci/pci-driver.c | 1 + include/linux/pci.h | 3 +++ 2 files changed, 4 insertions(+) diff --git a/drivers/pci/pci-driver.c b/drivers/pci/pci-driver.c index ec44a79e951a..3a72352aa5cf 100644 --- a/drivers/pci/pci-driver.c +++ b/drivers/pci/pci-driver.c @@ -1385,6 +1385,7 @@ int __pci_register_driver(struct pci_driver *drv, struct module *owner, drv->driver.owner = owner; drv->driver.mod_name = mod_name; drv->driver.groups = drv->groups; + drv->driver.dev_groups = drv->dev_groups; spin_lock_init(&drv->dynids.lock); INIT_LIST_HEAD(&drv->dynids.list); diff --git a/include/linux/pci.h b/include/linux/pci.h index 86c799c97b77..b57755b03009 100644 --- a/include/linux/pci.h +++ b/include/linux/pci.h @@ -858,6 +858,8 @@ struct module; * number of VFs to enable via sysfs "sriov_numvfs" file. * @err_handler: See Documentation/PCI/pci-error-recovery.rst * @groups: Sysfs attribute groups. + * @dev_groups: Attributes attached to the device that will be + * created once it is bound to the driver. * @driver: Driver model structure. * @dynids: List of dynamically added device IDs. */ @@ -873,6 +875,7 @@ struct pci_driver { int (*sriov_configure)(struct pci_dev *dev, int num_vfs); /* On PF */ const struct pci_error_handlers *err_handler; const struct attribute_group **groups; + const struct attribute_group **dev_groups; struct device_driver driver; struct pci_dynids dynids; }; From patchwork Mon May 10 16:36:18 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Andrey Grodzovsky X-Patchwork-Id: 12248679 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-18.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,MSGID_FROM_MTA_HEADER,SPF_HELO_NONE, SPF_PASS,USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 9F4BFC43462 for ; Mon, 10 May 2021 16:37:12 +0000 (UTC) Received: from gabe.freedesktop.org (gabe.freedesktop.org [131.252.210.177]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 58EF2611AB for ; Mon, 10 May 2021 16:37:12 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 58EF2611AB Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=amd.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=dri-devel-bounces@lists.freedesktop.org Received: from gabe.freedesktop.org (localhost [127.0.0.1]) by gabe.freedesktop.org (Postfix) with ESMTP id C155C6E8B4; Mon, 10 May 2021 16:37:01 +0000 (UTC) Received: from NAM11-CO1-obe.outbound.protection.outlook.com (mail-co1nam11on2063.outbound.protection.outlook.com [40.107.220.63]) by gabe.freedesktop.org (Postfix) with ESMTPS id F2F496E8C3; Mon, 10 May 2021 16:37:00 +0000 (UTC) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=eJecyp0HC854m+XK0+qt88BhlgqD+iLvsSRmPOA46KVsRemVkOpUXNT6BODvGPJNWpnOHcLxSAVHcgr5cTaTkLoWgaEkhzokVkycsI6YHHlSVdiXSKpOXYd5++7t/oplptgHuQ2sQSQKCC8MPMnrW2DccztjPdwtoQ/gvedmqDFvi+BRTG93PAlbz65bjVzGQvyzeWMaWbtJ/X86FMHQci7fd3T+ekHZvndUXfM2sfXDdUXPXeBV6Br1mET4XTF3B+g23II4MkzdKWWsMg7Yo9orvK22W1YgGEwsJTia6rd/03gQUhkU5eejJAMl1bRzSkentX5iecbwSidCLFusEw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=+j+bM1q4z3pMJEkudykjSW5dJG+JuOaAr7e0M6/ckSw=; b=GGQqgOdfIW5e2ds3aLPTz868enYuenbbNB5UiZ0Oa/4wGZ+nKvcHIbT3yalZ5JP1THlIRII0Fu28O3/wEoGIQHbl9Q/aXFMbsrVjxSdY/2hG+pr1ukrtalr8+FQNiQRN7X2k4p/bBXsT9IpXB1KJBPWYkm4OUqwQeMuHusL7/5XY+ZiAt+iVeypnif+2kyaBRMx+ooJ+KrxEmJZmEK2T3dfSLdHTyld2jzg1UFZAQFwtI4MMoEsTE7k8TFAyKFdZFoBtPUNwlOGI5Bl/391pn/9fAT1fTe2OPQuNw9gmHuIlSORxDqXvy4cUr00M5DJBQaHjr74/7ltFF0dPBvW0Gw== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=amd.com; dmarc=pass action=none header.from=amd.com; dkim=pass header.d=amd.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=+j+bM1q4z3pMJEkudykjSW5dJG+JuOaAr7e0M6/ckSw=; b=xFWcdoK4E2+CY+pSvb3iFoPz0W7zLvXEY37/Q8zx+4N2qoD2A6p96kY5H47Glu0wK2pWRmyM+F9Sv/G2qZeu1QD2tzVvkPtawyM1eB/vjcKGfncV30EywNNfM8ikkMmyBXElS6CZGjrlldQYXvbE0jCS3j+8De+dgnG7R4nVYaA= Authentication-Results: lists.freedesktop.org; dkim=none (message not signed) header.d=none; lists.freedesktop.org; dmarc=none action=none header.from=amd.com; Received: from SN6PR12MB4623.namprd12.prod.outlook.com (2603:10b6:805:e9::17) by SN1PR12MB2397.namprd12.prod.outlook.com (2603:10b6:802:27::21) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4108.29; Mon, 10 May 2021 16:36:57 +0000 Received: from SN6PR12MB4623.namprd12.prod.outlook.com ([fe80::ad51:8c49:b171:856c]) by SN6PR12MB4623.namprd12.prod.outlook.com ([fe80::ad51:8c49:b171:856c%7]) with mapi id 15.20.4108.031; Mon, 10 May 2021 16:36:57 +0000 From: Andrey Grodzovsky To: dri-devel@lists.freedesktop.org, amd-gfx@lists.freedesktop.org, linux-pci@vger.kernel.org, ckoenig.leichtzumerken@gmail.com, daniel.vetter@ffwll.ch, Harry.Wentland@amd.com Subject: [PATCH v6 09/16] drm/amdgpu: Convert driver sysfs attributes to static attributes Date: Mon, 10 May 2021 12:36:18 -0400 Message-Id: <20210510163625.407105-10-andrey.grodzovsky@amd.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210510163625.407105-1-andrey.grodzovsky@amd.com> References: <20210510163625.407105-1-andrey.grodzovsky@amd.com> X-Originating-IP: [2607:fea8:3edf:49b0:6a5:47b8:e610:f6a3] X-ClientProxiedBy: YT2PR01CA0020.CANPRD01.PROD.OUTLOOK.COM (2603:10b6:b01:38::25) To SN6PR12MB4623.namprd12.prod.outlook.com (2603:10b6:805:e9::17) MIME-Version: 1.0 X-MS-Exchange-MessageSentRepresentingType: 1 Received: from agrodzovsky-All-Series.hitronhub.home (2607:fea8:3edf:49b0:6a5:47b8:e610:f6a3) by YT2PR01CA0020.CANPRD01.PROD.OUTLOOK.COM (2603:10b6:b01:38::25) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4108.31 via Frontend Transport; Mon, 10 May 2021 16:36:56 +0000 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 5562cbb1-a149-4a0d-abbe-08d913d1d40a X-MS-TrafficTypeDiagnostic: SN1PR12MB2397: X-MS-Exchange-Transport-Forked: True X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:6108; X-MS-Exchange-SenderADCheck: 1 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: iI/X83UHzIDbDtdbgfFOykfsbVFyf4IiDgE+YPs0cNh3qgCdEud/ALSQKT+a04H7O2BqrucywVXyOb5o/h5q/N2JEK/wZ7OTaR8/Fvwof2n70CXZcZndCXTFHEDSBLHLm5poBa2oqkfx7z+KIzZJBRopgQQohs3j7PmF83bTQYgpBAcbXHjtn8khPshF+Oz48/IkTWD74L9EL3FuIzPjqx2kfjRnhvW2dkWPW+7UoGPPPE9g3SZhUnNelOLuLGu5YOS9sHYPjYaC6xnSVfO3yFHN6/Na8Ls7puQyG9I/hLd2c7l73se+51znxujueK1c7NhbP8rI2IJgdp+pBon/ZlWh3ju/g0gfXOcVe75x/ym7TbnqK1buoKDkCF/gFb0zaOxqD+MVJDf8ckSBAY3X2T0xwgImg9/aNTddtM45UZ7J8G5XDk1K+ZudoLxfYPWegLhzbVT9RKlDsW+XgvECMfWLaE05a+9/1bzIs1uYAH8DrSroHR8inzZQDYdTr7qrJCDG1Z2hnRA2rjMJSz+QSZ3Tkbk6Mzm7ely8FhknawxtfDI8FNWl2xU+1HOF5PxF0jJUKyLwvTZEHDQUnAdqamH4qD9ce4P+M11BXSpfJv8= X-Forefront-Antispam-Report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:SN6PR12MB4623.namprd12.prod.outlook.com; PTR:; CAT:NONE; SFS:(4636009)(366004)(1076003)(54906003)(6636002)(44832011)(83380400001)(6512007)(66946007)(66556008)(36756003)(2906002)(66476007)(6666004)(16526019)(5660300002)(52116002)(4326008)(8676002)(6506007)(2616005)(66574015)(8936002)(6486002)(86362001)(38100700002)(186003)(498600001); DIR:OUT; SFP:1101; X-MS-Exchange-AntiSpam-MessageData: =?utf-8?q?nqft+7P0tydxQ9sVH9Lc6FQA5jWwfB?= =?utf-8?q?Faq1HcRBf/JqQI/2sR3u6fGDxF9LKKJsSCHS3MsPfsSDonlgB4U9AOmOnQXsYs36L?= =?utf-8?q?S5D7sr3E8JD1EAAcYIAuPhy3ul1lKk6zkhovZv2b9NW7k2f2id+Yn+v6AQ63tEMk5?= =?utf-8?q?RvgWO77gkh9l1PLUjgkvVm2oHVH2kt92a8/BKMpBPwTQagfG86D1wVvFKaGtoZvBy?= =?utf-8?q?fKIJPqNZQUqeWzUKUyb8od1liFSq0akObDvpN9ptbQ4VdQ7VlcZHOY7j6WP3nPgiS?= =?utf-8?q?gb+gZaWwlI/uPF4Le8APYYwUFifwiaWYXjCtRnFrED/o/AjYQdQsDZWroNSB2MtDK?= =?utf-8?q?1fthmGxe8sxg/SfOFq0Ive3zI10iCo1qxyVzxNTt21g7YZbr30OZM5wLaZGfH6+H9?= =?utf-8?q?8nVp4ksSyXWnY0b3Ce6oQ+aDrvyhDtu07ZpkD5BzU155BlRPqeFTuMoqV/Dt6ZE2F?= =?utf-8?q?wiuINk3sCd+k1dGsIiY2yoyvlrVK+XMm18Vr5/xltCxZwpcnl0kuexUCDLc40pzuz?= =?utf-8?q?Fs1ZEZhVXEa5N0gFjcidyL7Nak4LA9MfYkzdG7om9vTDvLuMsYFEX1zwK2tjbg9eO?= =?utf-8?q?Kh6qab36q/jIc5tdauLsRbGtlVS853pIFMWDzIrXbOhKyI/HQ2AvhDuP39K6r9Jmm?= =?utf-8?q?wYxnwnyNfbdNe3p+rfRpTmSMJmWnMPhM3s07BCuvxzgmFsaPc7twMJDifExF6Ykxn?= =?utf-8?q?4FfyGm3KP4aHFwRo3SDgfue69fGDPQztjLrDyoW4ObMzLDtrTtHMGt01cTbm/Nfuq?= =?utf-8?q?OfOY20LMlW5GQ4C4hiX4XgIYkHVpsz9zjvEbZLanJUrJRGtzsvvh7G7u/gjQp911x?= =?utf-8?q?0xGnQRUa65D5lGkpPEJHelpRKXmBGSEuHeUDvS+F/mURM63dZ5McxNSAYdywQtu+W?= =?utf-8?q?L82knxBAQcQ9u3MLeZymCv56EY6y5sg18HxsEUAgNAR59osnO1A5nEAyrBV0kiSGD?= =?utf-8?q?60jWbTLMRzEV3DWH18GaNERQn8wV7R6QIZWvNROG+Du1GhdkB1wAE4P3YreUK/7LT?= =?utf-8?q?9iBNRdNikdRn15zJpPXAsfqJmy2Orxrr3n3sSn3+Awt656rsno32OYad2JxV203UQ?= =?utf-8?q?+rhMHoPjBFvz7XOpjbwRMisR700JccAKoEJihOPK5EzF8i1i9M1VCSzPp9qGTQEUU?= =?utf-8?q?SKwf0xd02glLuuE5exGKtJSyCgwhvAvCEx6M+QqewB7Sjs9/7mB/KVpr6eLNe7Lxb?= =?utf-8?q?yZtCB4oeqLzJzamCLEkFEF2pAA0wL8YCws6c6GKHTdOH/B8leTbjS9UoiIG4x00Z6?= =?utf-8?q?JrL04hDC8FkrZ2EbmawgQfDkBTe7HZzSm9ciyVlLVIwxen8vJAikuLzXw/9QZVHdR?= =?utf-8?q?VOnk0X9mc+K1G?= X-OriginatorOrg: amd.com X-MS-Exchange-CrossTenant-Network-Message-Id: 5562cbb1-a149-4a0d-abbe-08d913d1d40a X-MS-Exchange-CrossTenant-AuthSource: SN6PR12MB4623.namprd12.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 10 May 2021 16:36:57.7746 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: O1iTQk5HvTkMxppOrl2tuuJXMzR4CiAKEbqgS2vce1cDqKYFgkj100fXtKnSFRyZ10YYZ2w8tvWhDWf8SLBS/g== X-MS-Exchange-Transport-CrossTenantHeadersStamped: SN1PR12MB2397 X-BeenThere: dri-devel@lists.freedesktop.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Direct Rendering Infrastructure - Development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: gregkh@linuxfoundation.org, Felix.Kuehling@amd.com, helgaas@kernel.org, Alexander.Deucher@amd.com, =?utf-8?q?Christian_K=C3=B6nig?= Errors-To: dri-devel-bounces@lists.freedesktop.org Sender: "dri-devel" This allows to remove explicit creation and destruction of those attrs and by this avoids warnings on device finalizing post physical device extraction. v5: Use newly added pci_driver.dev_groups directly Signed-off-by: Andrey Grodzovsky Acked-by: Christian König --- drivers/gpu/drm/amd/amdgpu/amdgpu_atombios.c | 17 ++++++------- drivers/gpu/drm/amd/amdgpu/amdgpu_drv.c | 13 ++++++++++ drivers/gpu/drm/amd/amdgpu/amdgpu_gtt_mgr.c | 25 ++++++++------------ drivers/gpu/drm/amd/amdgpu/amdgpu_vram_mgr.c | 14 ++++------- 4 files changed, 37 insertions(+), 32 deletions(-) diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_atombios.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_atombios.c index 494b2e1717d5..879ed3e50a6e 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_atombios.c +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_atombios.c @@ -1768,6 +1768,15 @@ static ssize_t amdgpu_atombios_get_vbios_version(struct device *dev, static DEVICE_ATTR(vbios_version, 0444, amdgpu_atombios_get_vbios_version, NULL); +static struct attribute *amdgpu_vbios_version_attrs[] = { + &dev_attr_vbios_version.attr, + NULL +}; + +const struct attribute_group amdgpu_vbios_version_attr_group = { + .attrs = amdgpu_vbios_version_attrs +}; + /** * amdgpu_atombios_fini - free the driver info and callbacks for atombios * @@ -1787,7 +1796,6 @@ void amdgpu_atombios_fini(struct amdgpu_device *adev) adev->mode_info.atom_context = NULL; kfree(adev->mode_info.atom_card_info); adev->mode_info.atom_card_info = NULL; - device_remove_file(adev->dev, &dev_attr_vbios_version); } /** @@ -1804,7 +1812,6 @@ int amdgpu_atombios_init(struct amdgpu_device *adev) { struct card_info *atom_card_info = kzalloc(sizeof(struct card_info), GFP_KERNEL); - int ret; if (!atom_card_info) return -ENOMEM; @@ -1833,12 +1840,6 @@ int amdgpu_atombios_init(struct amdgpu_device *adev) amdgpu_atombios_allocate_fb_scratch(adev); } - ret = device_create_file(adev->dev, &dev_attr_vbios_version); - if (ret) { - DRM_ERROR("Failed to create device file for VBIOS version\n"); - return ret; - } - return 0; } diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_drv.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_drv.c index 5ebed4c7d9c0..83006f45b10b 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_drv.c +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_drv.c @@ -1766,6 +1766,18 @@ static struct pci_error_handlers amdgpu_pci_err_handler = { .resume = amdgpu_pci_resume, }; +extern const struct attribute_group amdgpu_vram_mgr_attr_group; +extern const struct attribute_group amdgpu_gtt_mgr_attr_group; +extern const struct attribute_group amdgpu_vbios_version_attr_group; + +static const struct attribute_group *amdgpu_sysfs_groups[] = { + &amdgpu_vram_mgr_attr_group, + &amdgpu_gtt_mgr_attr_group, + &amdgpu_vbios_version_attr_group, + NULL, +}; + + static struct pci_driver amdgpu_kms_pci_driver = { .name = DRIVER_NAME, .id_table = pciidlist, @@ -1774,6 +1786,7 @@ static struct pci_driver amdgpu_kms_pci_driver = { .shutdown = amdgpu_pci_shutdown, .driver.pm = &amdgpu_pm_ops, .err_handler = &amdgpu_pci_err_handler, + .dev_groups = amdgpu_sysfs_groups, }; static int __init amdgpu_init(void) diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_gtt_mgr.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_gtt_mgr.c index 72962de4c04c..a4404da8ca6d 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_gtt_mgr.c +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_gtt_mgr.c @@ -75,6 +75,16 @@ static DEVICE_ATTR(mem_info_gtt_total, S_IRUGO, static DEVICE_ATTR(mem_info_gtt_used, S_IRUGO, amdgpu_mem_info_gtt_used_show, NULL); +static struct attribute *amdgpu_gtt_mgr_attributes[] = { + &dev_attr_mem_info_gtt_total.attr, + &dev_attr_mem_info_gtt_used.attr, + NULL +}; + +const struct attribute_group amdgpu_gtt_mgr_attr_group = { + .attrs = amdgpu_gtt_mgr_attributes +}; + static const struct ttm_resource_manager_func amdgpu_gtt_mgr_func; /** * amdgpu_gtt_mgr_init - init GTT manager and DRM MM @@ -89,7 +99,6 @@ int amdgpu_gtt_mgr_init(struct amdgpu_device *adev, uint64_t gtt_size) struct amdgpu_gtt_mgr *mgr = &adev->mman.gtt_mgr; struct ttm_resource_manager *man = &mgr->manager; uint64_t start, size; - int ret; man->use_tt = true; man->func = &amdgpu_gtt_mgr_func; @@ -102,17 +111,6 @@ int amdgpu_gtt_mgr_init(struct amdgpu_device *adev, uint64_t gtt_size) spin_lock_init(&mgr->lock); atomic64_set(&mgr->available, gtt_size >> PAGE_SHIFT); - ret = device_create_file(adev->dev, &dev_attr_mem_info_gtt_total); - if (ret) { - DRM_ERROR("Failed to create device file mem_info_gtt_total\n"); - return ret; - } - ret = device_create_file(adev->dev, &dev_attr_mem_info_gtt_used); - if (ret) { - DRM_ERROR("Failed to create device file mem_info_gtt_used\n"); - return ret; - } - ttm_set_driver_manager(&adev->mman.bdev, TTM_PL_TT, &mgr->manager); ttm_resource_manager_set_used(man, true); return 0; @@ -142,9 +140,6 @@ void amdgpu_gtt_mgr_fini(struct amdgpu_device *adev) drm_mm_takedown(&mgr->mm); spin_unlock(&mgr->lock); - device_remove_file(adev->dev, &dev_attr_mem_info_gtt_total); - device_remove_file(adev->dev, &dev_attr_mem_info_gtt_used); - ttm_resource_manager_cleanup(man); ttm_set_driver_manager(&adev->mman.bdev, TTM_PL_TT, NULL); } diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_vram_mgr.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_vram_mgr.c index 2344aba9dca3..8543d6486018 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_vram_mgr.c +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_vram_mgr.c @@ -152,7 +152,7 @@ static DEVICE_ATTR(mem_info_vis_vram_used, S_IRUGO, static DEVICE_ATTR(mem_info_vram_vendor, S_IRUGO, amdgpu_mem_info_vram_vendor, NULL); -static const struct attribute *amdgpu_vram_mgr_attributes[] = { +static struct attribute *amdgpu_vram_mgr_attributes[] = { &dev_attr_mem_info_vram_total.attr, &dev_attr_mem_info_vis_vram_total.attr, &dev_attr_mem_info_vram_used.attr, @@ -161,6 +161,10 @@ static const struct attribute *amdgpu_vram_mgr_attributes[] = { NULL }; +const struct attribute_group amdgpu_vram_mgr_attr_group = { + .attrs = amdgpu_vram_mgr_attributes +}; + static const struct ttm_resource_manager_func amdgpu_vram_mgr_func; /** @@ -174,7 +178,6 @@ int amdgpu_vram_mgr_init(struct amdgpu_device *adev) { struct amdgpu_vram_mgr *mgr = &adev->mman.vram_mgr; struct ttm_resource_manager *man = &mgr->manager; - int ret; ttm_resource_manager_init(man, adev->gmc.real_vram_size >> PAGE_SHIFT); @@ -185,11 +188,6 @@ int amdgpu_vram_mgr_init(struct amdgpu_device *adev) INIT_LIST_HEAD(&mgr->reservations_pending); INIT_LIST_HEAD(&mgr->reserved_pages); - /* Add the two VRAM-related sysfs files */ - ret = sysfs_create_files(&adev->dev->kobj, amdgpu_vram_mgr_attributes); - if (ret) - DRM_ERROR("Failed to register sysfs\n"); - ttm_set_driver_manager(&adev->mman.bdev, TTM_PL_VRAM, &mgr->manager); ttm_resource_manager_set_used(man, true); return 0; @@ -227,8 +225,6 @@ void amdgpu_vram_mgr_fini(struct amdgpu_device *adev) drm_mm_takedown(&mgr->mm); spin_unlock(&mgr->lock); - sysfs_remove_files(&adev->dev->kobj, amdgpu_vram_mgr_attributes); - ttm_resource_manager_cleanup(man); ttm_set_driver_manager(&adev->mman.bdev, TTM_PL_VRAM, NULL); } From patchwork Mon May 10 16:36:19 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrey Grodzovsky X-Patchwork-Id: 12248683 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-18.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,MSGID_FROM_MTA_HEADER,SPF_HELO_NONE, SPF_PASS,USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id C95B4C433ED for ; Mon, 10 May 2021 16:37:14 +0000 (UTC) Received: from gabe.freedesktop.org (gabe.freedesktop.org [131.252.210.177]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 869B361155 for ; Mon, 10 May 2021 16:37:14 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 869B361155 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=amd.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=dri-devel-bounces@lists.freedesktop.org Received: from gabe.freedesktop.org (localhost [127.0.0.1]) by gabe.freedesktop.org (Postfix) with ESMTP id 7D79B6E8B9; Mon, 10 May 2021 16:37:04 +0000 (UTC) Received: from NAM11-CO1-obe.outbound.protection.outlook.com (mail-co1nam11on2050.outbound.protection.outlook.com [40.107.220.50]) by gabe.freedesktop.org (Postfix) with ESMTPS id 8B6136E8BD; Mon, 10 May 2021 16:37:02 +0000 (UTC) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=TNiOhVsJN61CPRtei+G0z7dS8rHFIb56yD/Yi3bff0EZ7qI1a+HMkflaaf9oQZ/l03byeFNpnJtOEnWK6N/Ux6ytp/KZYCKIFBKKbJeH4NKA0l1uIuWEoYLsi9DmdQm5g4qODsI3+1LJAjaqacp2BcbGDb/t24Wnk88ti74rRb90PSYZYGVvXxKMzKYP9Cq3YY4zajQL00qx6dmSrZrO3rrE4nZT4WVjTUpt88EUyL6pROWXttCMw3DeR66GnRRBXh0eo60bchMxBtX32wE43Y/orROzs+0i4uIjtQZGJupd5GXee/nicBP0/9v4HoabmTYeyqTTfR7MIvyqspL9PQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=DstkaeNteEXkR2rRWOuxwCTbNBJAyOp8exCoBjZvTNw=; b=eLacPs1zskEDr5hnSLOtY+6r+Lqg+nTudDYoNG+28Br0BRo1HXJjCnxdM51ROWq7Upmjxsytu5Bba2mczcq9qa5dOU67za8VVFoxiStuTOST6NfoMkhBf5v3b+0saAKatbfRnng2o9BHcYjoM6fKz8GkPHfk3Wv/YY8miKpcUF/IiFLax1Y/HPIeB+J+1SiF7eL2bW+tlC6/LYlnk68wKEuJcIt84n2eH2mngKo91X7MVCOb2W8VP2OutAe1xT2HxfggXh1nGWgAmDDE+az+AJG15CV2TXB6G4OmzR2ioSWKKO6rCpB7caQmbjmOv6Qa/HMwOY4ltHFzUIkYGYm1zg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=amd.com; dmarc=pass action=none header.from=amd.com; dkim=pass header.d=amd.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=DstkaeNteEXkR2rRWOuxwCTbNBJAyOp8exCoBjZvTNw=; b=poL5sTkHVMACH6XPnfO7qODf4vRcJO9yKy02QUOq333vr8MLn2qQxDLHlNXWDfs7szzn4imdt+IHUXM0k9hKGWtRpT2mCK9SvsA4DTIc2waE7rmYvZ/0mTLZ3lVmDMVC2arHMGoz4j+lhUYUQrMT1RA9knIPshFO9tt0hxj/UCU= Authentication-Results: lists.freedesktop.org; dkim=none (message not signed) header.d=none; lists.freedesktop.org; dmarc=none action=none header.from=amd.com; Received: from SN6PR12MB4623.namprd12.prod.outlook.com (2603:10b6:805:e9::17) by SN1PR12MB2397.namprd12.prod.outlook.com (2603:10b6:802:27::21) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4108.29; Mon, 10 May 2021 16:36:59 +0000 Received: from SN6PR12MB4623.namprd12.prod.outlook.com ([fe80::ad51:8c49:b171:856c]) by SN6PR12MB4623.namprd12.prod.outlook.com ([fe80::ad51:8c49:b171:856c%7]) with mapi id 15.20.4108.031; Mon, 10 May 2021 16:36:59 +0000 From: Andrey Grodzovsky To: dri-devel@lists.freedesktop.org, amd-gfx@lists.freedesktop.org, linux-pci@vger.kernel.org, ckoenig.leichtzumerken@gmail.com, daniel.vetter@ffwll.ch, Harry.Wentland@amd.com Subject: [PATCH v6 10/16] drm/amdgpu: Guard against write accesses after device removal Date: Mon, 10 May 2021 12:36:19 -0400 Message-Id: <20210510163625.407105-11-andrey.grodzovsky@amd.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210510163625.407105-1-andrey.grodzovsky@amd.com> References: <20210510163625.407105-1-andrey.grodzovsky@amd.com> X-Originating-IP: [2607:fea8:3edf:49b0:6a5:47b8:e610:f6a3] X-ClientProxiedBy: YT2PR01CA0020.CANPRD01.PROD.OUTLOOK.COM (2603:10b6:b01:38::25) To SN6PR12MB4623.namprd12.prod.outlook.com (2603:10b6:805:e9::17) MIME-Version: 1.0 X-MS-Exchange-MessageSentRepresentingType: 1 Received: from agrodzovsky-All-Series.hitronhub.home (2607:fea8:3edf:49b0:6a5:47b8:e610:f6a3) by YT2PR01CA0020.CANPRD01.PROD.OUTLOOK.COM (2603:10b6:b01:38::25) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4108.31 via Frontend Transport; Mon, 10 May 2021 16:36:58 +0000 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 8472f9cc-06ca-4b62-c189-08d913d1d506 X-MS-TrafficTypeDiagnostic: SN1PR12MB2397: X-MS-Exchange-Transport-Forked: True X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:644; X-MS-Exchange-SenderADCheck: 1 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: FG15+OyWUcVlblH8YeNmkHD7KTVougqlIt6EK6QB+VKXjJg6yr68HE5B/QN0O7tTuQI7UP3Lca5tx43x07tfrj34G11IBuzh8LF+HUGn5HhXjIaeHHt5nGqt98PRnsvkuNMU45WSRQUs8bC6pX2Us7gO1xuX5ejvHJHYBErRcXXJcKoyohZJEyVy8Lye+dcKwA5YlVyD88JsaW8boIs1nRG19FECJnaJE9PTXgVSlq31HHAakvUqs/y5/8p7aawNCAXEAqb8Pf2MlLhu/crMYd50/K16mojmmlVBnqPhIwnubIwM2umUxS28tSssh9HQksywXmXDwj2QqndrkfCkkGsmyroKuAn+11RWB+PPih1tfuVLMN/ysFn8AUOeYyON4VDM3CytB/O2dv5rW/fYRi5AR1LXqsIpSGWAQxPjKzUEOwik0VNtjLmfsnBT6/OUW2LI1jmGJnzusX16djxLxgNs3ecLBV9FPcFxhPZFlPLza9iYZsZU6etNBcoAUdmcTWuvo9fCes+mybOusoIUckOEVJf8uAV2ZuKYV/e+TjDOfg2p8/nDhJzM194zmexPDUIiVIG16mCzqVAuIpBKHiLf/W+HNOCwevGchLn5QWQ= X-Forefront-Antispam-Report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:SN6PR12MB4623.namprd12.prod.outlook.com; PTR:; CAT:NONE; SFS:(4636009)(136003)(346002)(39860400002)(376002)(366004)(396003)(1076003)(6636002)(44832011)(83380400001)(30864003)(6512007)(66946007)(66556008)(36756003)(2906002)(66476007)(6666004)(316002)(16526019)(5660300002)(52116002)(4326008)(8676002)(6506007)(2616005)(8936002)(6486002)(86362001)(38100700002)(186003)(478600001); DIR:OUT; SFP:1101; X-MS-Exchange-AntiSpam-MessageData: z/9xJLz5ecfZ9cbqsIWU9Y3YuM8aNPVoslTeIeJZR8Dpat7dLrR7704nBuXrD7/aXZX+JDFQlx74yGhL0WTlb7QnC7RrP0BmYKlBsaasCUJzwK3ZsGkQEWpsZJnUmqi91oi9BcOtaFPQ8ucl8GpPrTSt81kq/yQCs24TBM7nLS/9Vjn49SOV/HucSYq1qhVt0psQkd8cT81tC+WRahV9gLtfO4CIoh/jtUawaz8BOMFV89ATtZW703VbGczUso2HKDSoeDFo/LFpt7pn5+UrPbIFsSMVsTpYPs+PU0lhBloZCCIEGo2ZEpg/UylKfJWZ/1+lx7lzJTb93h1B3Kq4v1zN/IRt2gHnC3hyKHnBNtIjD5pfCBnIJ+TZAVVQbxU6fgSv+U5XmRt/5y8IO+HolOmo2spopyzN0LdrIWvgjz5S2CG0D0gNGn4NL+1ztQkC+w/4OLjJYbQPHdWQJyuGQC5d1laeFvHOlfLJtkXOJFhMqRZKg4VLjr2+lHxiNMKwIdV63Tv3D9T55KzVsC5xXghts4vEkJ+xBSS6QImBmRnDKcxt+2wopUWzqooyretNnJAR9dr/jclaekk9DzhBPc8AxH36qa5KCvqbnOV5auv20hDKOLMquSuoPk29e8r0sUeJxj+J1v9L89QMnkVRFqhn8Lmd3v8PqbCHIHQI0xJchZS+7NIShqG0k6AoJSdxd6mpwmDMqXIE1siM27wdtjS0kuf4W6oPwC+Te3gnm5dSnH5unhUcsVXAGrXT5ioQYhZU5B8nHLWKoKFbWd39MDdad5EvF01OR6/hA+05eVbvvYL8aPIkNuln7tF457scd0xsgFU4IPVxJGXVNHtL+rZ2xH7pkeIZDkZq9r8NbTIuhmcrzHDoPlyVDebvoLeXhsJpGjlftBsQwNLdJ63tmXHlO4MS3R6DPqnsATIV22FztxIa4Yog37QAqHrPqO6g5RnPSvh8X5yghGPZ0duY/vfD+3E/Kml1oG2ObL1v8mr7rSkUOD/pt+uLH76GJhlpPqg9uGLXvldx7Reteotlq/Cw41ZnDelT9f0CSDnYDG/PTttEsEGDLMuvSGXL+fHwcWQ94c+ifSdz4iWuyqCs1WCwGBqwGJdYzLyEDlnB933B/JOS2R82SA9jspPxfiU/Am1whkY/1sQL+nTf11L/g2eNOOg+c4UoaS382R7IiRlu6SdoZySZOwWYXzxzqmxX+H8GMnW+m2o19wXW7a9yZk1BCcgJWm1x1rlX4hB1V/amuN9yYQGoGHJxSNiKS4Gwnnb0y3nEHLphfSyXq6NL2MxUjhIyHNYIytVx4no1+j9TzfJourjJ8+1Rk4lqZD6KXEWqKhktpWm8L40zVHbhV0zZyfbyoo+xYU/OMiPJTV8marwUXqEit8QNJrYulRNL X-OriginatorOrg: amd.com X-MS-Exchange-CrossTenant-Network-Message-Id: 8472f9cc-06ca-4b62-c189-08d913d1d506 X-MS-Exchange-CrossTenant-AuthSource: SN6PR12MB4623.namprd12.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 10 May 2021 16:36:59.4817 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: aLFqzNQmETsXfKEb6SYbVd8EMqYv4XCcCL8bo4hw3zKPGZMYVnC99Gxwq8gV1PtCeGDQYAe8C551M40fyL81LA== X-MS-Exchange-Transport-CrossTenantHeadersStamped: SN1PR12MB2397 X-BeenThere: dri-devel@lists.freedesktop.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Direct Rendering Infrastructure - Development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: gregkh@linuxfoundation.org, Felix.Kuehling@amd.com, helgaas@kernel.org, Alexander.Deucher@amd.com Errors-To: dri-devel-bounces@lists.freedesktop.org Sender: "dri-devel" This should prevent writing to memory or IO ranges possibly already allocated for other uses after our device is removed. v5: Protect more places wher memcopy_to/form_io takes place Protect IB submissions v6: Switch to !drm_dev_enter instead of scoping entire code with brackets. Signed-off-by: Andrey Grodzovsky --- drivers/gpu/drm/amd/amdgpu/amdgpu_device.c | 11 ++- drivers/gpu/drm/amd/amdgpu/amdgpu_gmc.c | 9 +++ drivers/gpu/drm/amd/amdgpu/amdgpu_ib.c | 17 +++-- drivers/gpu/drm/amd/amdgpu/amdgpu_psp.c | 63 +++++++++++------ drivers/gpu/drm/amd/amdgpu/amdgpu_psp.h | 2 + drivers/gpu/drm/amd/amdgpu/amdgpu_ring.c | 70 +++++++++++++++++++ drivers/gpu/drm/amd/amdgpu/amdgpu_ring.h | 49 ++----------- drivers/gpu/drm/amd/amdgpu/amdgpu_uvd.c | 31 +++++--- drivers/gpu/drm/amd/amdgpu/amdgpu_vce.c | 11 ++- drivers/gpu/drm/amd/amdgpu/amdgpu_vcn.c | 22 ++++-- drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c | 7 +- drivers/gpu/drm/amd/amdgpu/psp_v11_0.c | 44 ++++++------ drivers/gpu/drm/amd/amdgpu/psp_v12_0.c | 8 +-- drivers/gpu/drm/amd/amdgpu/psp_v3_1.c | 8 +-- drivers/gpu/drm/amd/amdgpu/vce_v4_0.c | 26 ++++--- drivers/gpu/drm/amd/amdgpu/vcn_v3_0.c | 22 +++--- .../drm/amd/pm/powerplay/smumgr/smu7_smumgr.c | 2 + 17 files changed, 257 insertions(+), 145 deletions(-) diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c index a0bff4713672..94c415176cdc 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c @@ -71,6 +71,8 @@ #include #include +#include + MODULE_FIRMWARE("amdgpu/vega10_gpu_info.bin"); MODULE_FIRMWARE("amdgpu/vega12_gpu_info.bin"); MODULE_FIRMWARE("amdgpu/raven_gpu_info.bin"); @@ -281,7 +283,10 @@ void amdgpu_device_vram_access(struct amdgpu_device *adev, loff_t pos, unsigned long flags; uint32_t hi = ~0; uint64_t last; + int idx; + if (!drm_dev_enter(&adev->ddev, &idx)) + return; #ifdef CONFIG_64BIT last = min(pos + size, adev->gmc.visible_vram_size); @@ -299,8 +304,10 @@ void amdgpu_device_vram_access(struct amdgpu_device *adev, loff_t pos, memcpy_fromio(buf, addr, count); } - if (count == size) + if (count == size) { + drm_dev_exit(idx); return; + } pos += count; buf += count / 4; @@ -323,6 +330,8 @@ void amdgpu_device_vram_access(struct amdgpu_device *adev, loff_t pos, *buf++ = RREG32_NO_KIQ(mmMM_DATA); } spin_unlock_irqrestore(&adev->mmio_idx_lock, flags); + + drm_dev_exit(idx); } /* diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_gmc.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_gmc.c index 4d32233cde92..04ba5eef1e88 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_gmc.c +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_gmc.c @@ -31,6 +31,8 @@ #include "amdgpu_ras.h" #include "amdgpu_xgmi.h" +#include + /** * amdgpu_gmc_pdb0_alloc - allocate vram for pdb0 * @@ -151,6 +153,10 @@ int amdgpu_gmc_set_pte_pde(struct amdgpu_device *adev, void *cpu_pt_addr, { void __iomem *ptr = (void *)cpu_pt_addr; uint64_t value; + int idx; + + if (!drm_dev_enter(&adev->ddev, &idx)) + return 0; /* * The following is for PTE only. GART does not have PDEs. @@ -158,6 +164,9 @@ int amdgpu_gmc_set_pte_pde(struct amdgpu_device *adev, void *cpu_pt_addr, value = addr & 0x0000FFFFFFFFF000ULL; value |= flags; writeq(value, ptr + (gpu_page_idx * 8)); + + drm_dev_exit(idx); + return 0; } diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_ib.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_ib.c index 148a3b481b12..62fcbd446c71 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_ib.c +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_ib.c @@ -30,6 +30,7 @@ #include #include +#include #include "amdgpu.h" #include "atom.h" @@ -137,7 +138,7 @@ int amdgpu_ib_schedule(struct amdgpu_ring *ring, unsigned num_ibs, bool secure; unsigned i; - int r = 0; + int idx, r = 0; bool need_pipe_sync = false; if (num_ibs == 0) @@ -169,13 +170,16 @@ int amdgpu_ib_schedule(struct amdgpu_ring *ring, unsigned num_ibs, return -EINVAL; } + if (!drm_dev_enter(&adev->ddev, &idx)) + return -ENODEV; + alloc_size = ring->funcs->emit_frame_size + num_ibs * ring->funcs->emit_ib_size; r = amdgpu_ring_alloc(ring, alloc_size); if (r) { dev_err(adev->dev, "scheduling IB failed (%d).\n", r); - return r; + goto exit; } need_ctx_switch = ring->current_ctx != fence_ctx; @@ -205,7 +209,7 @@ int amdgpu_ib_schedule(struct amdgpu_ring *ring, unsigned num_ibs, r = amdgpu_vm_flush(ring, job, need_pipe_sync); if (r) { amdgpu_ring_undo(ring); - return r; + goto exit; } } @@ -286,7 +290,7 @@ int amdgpu_ib_schedule(struct amdgpu_ring *ring, unsigned num_ibs, if (job && job->vmid) amdgpu_vmid_reset(adev, ring->funcs->vmhub, job->vmid); amdgpu_ring_undo(ring); - return r; + goto exit; } if (ring->funcs->insert_end) @@ -304,7 +308,10 @@ int amdgpu_ib_schedule(struct amdgpu_ring *ring, unsigned num_ibs, ring->funcs->emit_wave_limit(ring, false); amdgpu_ring_commit(ring); - return 0; + +exit: + drm_dev_exit(idx); + return r; } /** diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_psp.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_psp.c index 9e769cf6095b..bb6afee61666 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_psp.c +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_psp.c @@ -25,6 +25,7 @@ #include #include +#include #include "amdgpu.h" #include "amdgpu_psp.h" @@ -39,6 +40,8 @@ #include "amdgpu_ras.h" #include "amdgpu_securedisplay.h" +#include + static int psp_sysfs_init(struct amdgpu_device *adev); static void psp_sysfs_fini(struct amdgpu_device *adev); @@ -253,7 +256,7 @@ psp_cmd_submit_buf(struct psp_context *psp, struct psp_gfx_cmd_resp *cmd, uint64_t fence_mc_addr) { int ret; - int index; + int index, idx; int timeout = 20000; bool ras_intr = false; bool skip_unsupport = false; @@ -261,6 +264,9 @@ psp_cmd_submit_buf(struct psp_context *psp, if (psp->adev->in_pci_err_recovery) return 0; + if (!drm_dev_enter(&psp->adev->ddev, &idx)) + return 0; + mutex_lock(&psp->mutex); memset(psp->cmd_buf_mem, 0, PSP_CMD_BUFFER_SIZE); @@ -271,8 +277,7 @@ psp_cmd_submit_buf(struct psp_context *psp, ret = psp_ring_cmd_submit(psp, psp->cmd_buf_mc_addr, fence_mc_addr, index); if (ret) { atomic_dec(&psp->fence_value); - mutex_unlock(&psp->mutex); - return ret; + goto exit; } amdgpu_asic_invalidate_hdp(psp->adev, NULL); @@ -312,8 +317,8 @@ psp_cmd_submit_buf(struct psp_context *psp, psp->cmd_buf_mem->cmd_id, psp->cmd_buf_mem->resp.status); if (!timeout) { - mutex_unlock(&psp->mutex); - return -EINVAL; + ret = -EINVAL; + goto exit; } } @@ -321,8 +326,10 @@ psp_cmd_submit_buf(struct psp_context *psp, ucode->tmr_mc_addr_lo = psp->cmd_buf_mem->resp.fw_addr_lo; ucode->tmr_mc_addr_hi = psp->cmd_buf_mem->resp.fw_addr_hi; } - mutex_unlock(&psp->mutex); +exit: + mutex_unlock(&psp->mutex); + drm_dev_exit(idx); return ret; } @@ -359,8 +366,7 @@ static int psp_load_toc(struct psp_context *psp, if (!cmd) return -ENOMEM; /* Copy toc to psp firmware private buffer */ - memset(psp->fw_pri_buf, 0, PSP_1_MEG); - memcpy(psp->fw_pri_buf, psp->toc_start_addr, psp->toc_bin_size); + psp_copy_fw(psp, psp->toc_start_addr, psp->toc_bin_size); psp_prep_load_toc_cmd_buf(cmd, psp->fw_pri_mc_addr, psp->toc_bin_size); @@ -625,8 +631,7 @@ static int psp_asd_load(struct psp_context *psp) if (!cmd) return -ENOMEM; - memset(psp->fw_pri_buf, 0, PSP_1_MEG); - memcpy(psp->fw_pri_buf, psp->asd_start_addr, psp->asd_ucode_size); + psp_copy_fw(psp, psp->asd_start_addr, psp->asd_ucode_size); psp_prep_asd_load_cmd_buf(cmd, psp->fw_pri_mc_addr, psp->asd_ucode_size); @@ -781,8 +786,7 @@ static int psp_xgmi_load(struct psp_context *psp) if (!cmd) return -ENOMEM; - memset(psp->fw_pri_buf, 0, PSP_1_MEG); - memcpy(psp->fw_pri_buf, psp->ta_xgmi_start_addr, psp->ta_xgmi_ucode_size); + psp_copy_fw(psp, psp->ta_xgmi_start_addr, psp->ta_xgmi_ucode_size); psp_prep_ta_load_cmd_buf(cmd, psp->fw_pri_mc_addr, @@ -1038,8 +1042,7 @@ static int psp_ras_load(struct psp_context *psp) if (!cmd) return -ENOMEM; - memset(psp->fw_pri_buf, 0, PSP_1_MEG); - memcpy(psp->fw_pri_buf, psp->ta_ras_start_addr, psp->ta_ras_ucode_size); + psp_copy_fw(psp, psp->ta_ras_start_addr, psp->ta_ras_ucode_size); psp_prep_ta_load_cmd_buf(cmd, psp->fw_pri_mc_addr, @@ -1275,8 +1278,7 @@ static int psp_hdcp_load(struct psp_context *psp) if (!cmd) return -ENOMEM; - memset(psp->fw_pri_buf, 0, PSP_1_MEG); - memcpy(psp->fw_pri_buf, psp->ta_hdcp_start_addr, + psp_copy_fw(psp, psp->ta_hdcp_start_addr, psp->ta_hdcp_ucode_size); psp_prep_ta_load_cmd_buf(cmd, @@ -1427,8 +1429,7 @@ static int psp_dtm_load(struct psp_context *psp) if (!cmd) return -ENOMEM; - memset(psp->fw_pri_buf, 0, PSP_1_MEG); - memcpy(psp->fw_pri_buf, psp->ta_dtm_start_addr, psp->ta_dtm_ucode_size); + psp_copy_fw(psp, psp->ta_dtm_start_addr, psp->ta_dtm_ucode_size); psp_prep_ta_load_cmd_buf(cmd, psp->fw_pri_mc_addr, @@ -1573,8 +1574,7 @@ static int psp_rap_load(struct psp_context *psp) if (!cmd) return -ENOMEM; - memset(psp->fw_pri_buf, 0, PSP_1_MEG); - memcpy(psp->fw_pri_buf, psp->ta_rap_start_addr, psp->ta_rap_ucode_size); + psp_copy_fw(psp, psp->ta_rap_start_addr, psp->ta_rap_ucode_size); psp_prep_ta_load_cmd_buf(cmd, psp->fw_pri_mc_addr, @@ -3022,7 +3022,7 @@ static ssize_t psp_usbc_pd_fw_sysfs_write(struct device *dev, struct amdgpu_device *adev = drm_to_adev(ddev); void *cpu_addr; dma_addr_t dma_addr; - int ret; + int ret, idx; char fw_name[100]; const struct firmware *usbc_pd_fw; @@ -3031,6 +3031,9 @@ static ssize_t psp_usbc_pd_fw_sysfs_write(struct device *dev, return -EBUSY; } + if (!drm_dev_enter(ddev, &idx)) + return -ENODEV; + snprintf(fw_name, sizeof(fw_name), "amdgpu/%s", buf); ret = request_firmware(&usbc_pd_fw, fw_name, adev->dev); if (ret) @@ -3062,16 +3065,30 @@ static ssize_t psp_usbc_pd_fw_sysfs_write(struct device *dev, rel_buf: dma_free_coherent(adev->dev, usbc_pd_fw->size, cpu_addr, dma_addr); release_firmware(usbc_pd_fw); - fail: if (ret) { DRM_ERROR("Failed to load USBC PD FW, err = %d", ret); - return ret; + count = ret; } + drm_dev_exit(idx); return count; } +void psp_copy_fw(struct psp_context *psp, uint8_t *start_addr, uint32_t bin_size) +{ + int idx; + + if (!drm_dev_enter(&psp->adev->ddev, &idx)) + return; + + memset(psp->fw_pri_buf, 0, PSP_1_MEG); + memcpy(psp->fw_pri_buf, start_addr, bin_size); + + drm_dev_exit(idx); +} + + static DEVICE_ATTR(usbc_pd_fw, S_IRUGO | S_IWUSR, psp_usbc_pd_fw_sysfs_read, psp_usbc_pd_fw_sysfs_write); diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_psp.h b/drivers/gpu/drm/amd/amdgpu/amdgpu_psp.h index 46a5328e00e0..2bfdc278817f 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_psp.h +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_psp.h @@ -423,4 +423,6 @@ int psp_get_fw_attestation_records_addr(struct psp_context *psp, int psp_load_fw_list(struct psp_context *psp, struct amdgpu_firmware_info **ucode_list, int ucode_count); +void psp_copy_fw(struct psp_context *psp, uint8_t *start_addr, uint32_t bin_size); + #endif diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_ring.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_ring.c index 688624ebe421..e1985bc34436 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_ring.c +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_ring.c @@ -35,6 +35,8 @@ #include "amdgpu.h" #include "atom.h" +#include + /* * Rings * Most engines on the GPU are fed via ring buffers. Ring @@ -461,3 +463,71 @@ int amdgpu_ring_test_helper(struct amdgpu_ring *ring) ring->sched.ready = !r; return r; } + +void amdgpu_ring_clear_ring(struct amdgpu_ring *ring) +{ + int idx; + int i = 0; + + if (!drm_dev_enter(&ring->adev->ddev, &idx)) + return; + + while (i <= ring->buf_mask) + ring->ring[i++] = ring->funcs->nop; + + drm_dev_exit(idx); + +} + +void amdgpu_ring_write(struct amdgpu_ring *ring, uint32_t v) +{ + int idx; + + if (!drm_dev_enter(&ring->adev->ddev, &idx)) + return; + + if (ring->count_dw <= 0) + DRM_ERROR("amdgpu: writing more dwords to the ring than expected!\n"); + ring->ring[ring->wptr++ & ring->buf_mask] = v; + ring->wptr &= ring->ptr_mask; + ring->count_dw--; + + drm_dev_exit(idx); +} + +void amdgpu_ring_write_multiple(struct amdgpu_ring *ring, + void *src, int count_dw) +{ + unsigned occupied, chunk1, chunk2; + void *dst; + int idx; + + if (!drm_dev_enter(&ring->adev->ddev, &idx)) + return; + + if (unlikely(ring->count_dw < count_dw)) + DRM_ERROR("amdgpu: writing more dwords to the ring than expected!\n"); + + occupied = ring->wptr & ring->buf_mask; + dst = (void *)&ring->ring[occupied]; + chunk1 = ring->buf_mask + 1 - occupied; + chunk1 = (chunk1 >= count_dw) ? count_dw: chunk1; + chunk2 = count_dw - chunk1; + chunk1 <<= 2; + chunk2 <<= 2; + + if (chunk1) + memcpy(dst, src, chunk1); + + if (chunk2) { + src += chunk1; + dst = (void *)ring->ring; + memcpy(dst, src, chunk2); + } + + ring->wptr += count_dw; + ring->wptr &= ring->ptr_mask; + ring->count_dw -= count_dw; + + drm_dev_exit(idx); +} diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_ring.h b/drivers/gpu/drm/amd/amdgpu/amdgpu_ring.h index e7d3d0dbdd96..c67bc6d3d039 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_ring.h +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_ring.h @@ -299,53 +299,12 @@ static inline void amdgpu_ring_set_preempt_cond_exec(struct amdgpu_ring *ring, *ring->cond_exe_cpu_addr = cond_exec; } -static inline void amdgpu_ring_clear_ring(struct amdgpu_ring *ring) -{ - int i = 0; - while (i <= ring->buf_mask) - ring->ring[i++] = ring->funcs->nop; - -} - -static inline void amdgpu_ring_write(struct amdgpu_ring *ring, uint32_t v) -{ - if (ring->count_dw <= 0) - DRM_ERROR("amdgpu: writing more dwords to the ring than expected!\n"); - ring->ring[ring->wptr++ & ring->buf_mask] = v; - ring->wptr &= ring->ptr_mask; - ring->count_dw--; -} +void amdgpu_ring_clear_ring(struct amdgpu_ring *ring); -static inline void amdgpu_ring_write_multiple(struct amdgpu_ring *ring, - void *src, int count_dw) -{ - unsigned occupied, chunk1, chunk2; - void *dst; - - if (unlikely(ring->count_dw < count_dw)) - DRM_ERROR("amdgpu: writing more dwords to the ring than expected!\n"); - - occupied = ring->wptr & ring->buf_mask; - dst = (void *)&ring->ring[occupied]; - chunk1 = ring->buf_mask + 1 - occupied; - chunk1 = (chunk1 >= count_dw) ? count_dw: chunk1; - chunk2 = count_dw - chunk1; - chunk1 <<= 2; - chunk2 <<= 2; +void amdgpu_ring_write(struct amdgpu_ring *ring, uint32_t v); - if (chunk1) - memcpy(dst, src, chunk1); - - if (chunk2) { - src += chunk1; - dst = (void *)ring->ring; - memcpy(dst, src, chunk2); - } - - ring->wptr += count_dw; - ring->wptr &= ring->ptr_mask; - ring->count_dw -= count_dw; -} +void amdgpu_ring_write_multiple(struct amdgpu_ring *ring, + void *src, int count_dw); int amdgpu_ring_test_helper(struct amdgpu_ring *ring); diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_uvd.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_uvd.c index c6dbc0801604..82f0542c7792 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_uvd.c +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_uvd.c @@ -32,6 +32,7 @@ #include #include +#include #include "amdgpu.h" #include "amdgpu_pm.h" @@ -375,7 +376,7 @@ int amdgpu_uvd_suspend(struct amdgpu_device *adev) { unsigned size; void *ptr; - int i, j; + int i, j, idx; bool in_ras_intr = amdgpu_ras_intr_triggered(); cancel_delayed_work_sync(&adev->uvd.idle_work); @@ -403,11 +404,15 @@ int amdgpu_uvd_suspend(struct amdgpu_device *adev) if (!adev->uvd.inst[j].saved_bo) return -ENOMEM; - /* re-write 0 since err_event_athub will corrupt VCPU buffer */ - if (in_ras_intr) - memset(adev->uvd.inst[j].saved_bo, 0, size); - else - memcpy_fromio(adev->uvd.inst[j].saved_bo, ptr, size); + if (drm_dev_enter(&adev->ddev, &idx)) { + /* re-write 0 since err_event_athub will corrupt VCPU buffer */ + if (in_ras_intr) + memset(adev->uvd.inst[j].saved_bo, 0, size); + else + memcpy_fromio(adev->uvd.inst[j].saved_bo, ptr, size); + + drm_dev_exit(idx); + } } if (in_ras_intr) @@ -420,7 +425,7 @@ int amdgpu_uvd_resume(struct amdgpu_device *adev) { unsigned size; void *ptr; - int i; + int i, idx; for (i = 0; i < adev->uvd.num_uvd_inst; i++) { if (adev->uvd.harvest_config & (1 << i)) @@ -432,7 +437,10 @@ int amdgpu_uvd_resume(struct amdgpu_device *adev) ptr = adev->uvd.inst[i].cpu_addr; if (adev->uvd.inst[i].saved_bo != NULL) { - memcpy_toio(ptr, adev->uvd.inst[i].saved_bo, size); + if (drm_dev_enter(&adev->ddev, &idx)) { + memcpy_toio(ptr, adev->uvd.inst[i].saved_bo, size); + drm_dev_exit(idx); + } kvfree(adev->uvd.inst[i].saved_bo); adev->uvd.inst[i].saved_bo = NULL; } else { @@ -442,8 +450,11 @@ int amdgpu_uvd_resume(struct amdgpu_device *adev) hdr = (const struct common_firmware_header *)adev->uvd.fw->data; if (adev->firmware.load_type != AMDGPU_FW_LOAD_PSP) { offset = le32_to_cpu(hdr->ucode_array_offset_bytes); - memcpy_toio(adev->uvd.inst[i].cpu_addr, adev->uvd.fw->data + offset, - le32_to_cpu(hdr->ucode_size_bytes)); + if (drm_dev_enter(&adev->ddev, &idx)) { + memcpy_toio(adev->uvd.inst[i].cpu_addr, adev->uvd.fw->data + offset, + le32_to_cpu(hdr->ucode_size_bytes)); + drm_dev_exit(idx); + } size -= le32_to_cpu(hdr->ucode_size_bytes); ptr += le32_to_cpu(hdr->ucode_size_bytes); } diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_vce.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_vce.c index ea6a62f67e38..833203401ef4 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_vce.c +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_vce.c @@ -29,6 +29,7 @@ #include #include +#include #include "amdgpu.h" #include "amdgpu_pm.h" @@ -293,7 +294,7 @@ int amdgpu_vce_resume(struct amdgpu_device *adev) void *cpu_addr; const struct common_firmware_header *hdr; unsigned offset; - int r; + int r, idx; if (adev->vce.vcpu_bo == NULL) return -EINVAL; @@ -313,8 +314,12 @@ int amdgpu_vce_resume(struct amdgpu_device *adev) hdr = (const struct common_firmware_header *)adev->vce.fw->data; offset = le32_to_cpu(hdr->ucode_array_offset_bytes); - memcpy_toio(cpu_addr, adev->vce.fw->data + offset, - adev->vce.fw->size - offset); + + if (drm_dev_enter(&adev->ddev, &idx)) { + memcpy_toio(cpu_addr, adev->vce.fw->data + offset, + adev->vce.fw->size - offset); + drm_dev_exit(idx); + } amdgpu_bo_kunmap(adev->vce.vcpu_bo); diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_vcn.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_vcn.c index 201645963ba5..21f7d3644d70 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_vcn.c +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_vcn.c @@ -27,6 +27,7 @@ #include #include #include +#include #include "amdgpu.h" #include "amdgpu_pm.h" @@ -275,7 +276,7 @@ int amdgpu_vcn_suspend(struct amdgpu_device *adev) { unsigned size; void *ptr; - int i; + int i, idx; cancel_delayed_work_sync(&adev->vcn.idle_work); @@ -292,7 +293,10 @@ int amdgpu_vcn_suspend(struct amdgpu_device *adev) if (!adev->vcn.inst[i].saved_bo) return -ENOMEM; - memcpy_fromio(adev->vcn.inst[i].saved_bo, ptr, size); + if (drm_dev_enter(&adev->ddev, &idx)) { + memcpy_fromio(adev->vcn.inst[i].saved_bo, ptr, size); + drm_dev_exit(idx); + } } return 0; } @@ -301,7 +305,7 @@ int amdgpu_vcn_resume(struct amdgpu_device *adev) { unsigned size; void *ptr; - int i; + int i, idx; for (i = 0; i < adev->vcn.num_vcn_inst; ++i) { if (adev->vcn.harvest_config & (1 << i)) @@ -313,7 +317,10 @@ int amdgpu_vcn_resume(struct amdgpu_device *adev) ptr = adev->vcn.inst[i].cpu_addr; if (adev->vcn.inst[i].saved_bo != NULL) { - memcpy_toio(ptr, adev->vcn.inst[i].saved_bo, size); + if (drm_dev_enter(&adev->ddev, &idx)) { + memcpy_toio(ptr, adev->vcn.inst[i].saved_bo, size); + drm_dev_exit(idx); + } kvfree(adev->vcn.inst[i].saved_bo); adev->vcn.inst[i].saved_bo = NULL; } else { @@ -323,8 +330,11 @@ int amdgpu_vcn_resume(struct amdgpu_device *adev) hdr = (const struct common_firmware_header *)adev->vcn.fw->data; if (adev->firmware.load_type != AMDGPU_FW_LOAD_PSP) { offset = le32_to_cpu(hdr->ucode_array_offset_bytes); - memcpy_toio(adev->vcn.inst[i].cpu_addr, adev->vcn.fw->data + offset, - le32_to_cpu(hdr->ucode_size_bytes)); + if (drm_dev_enter(&adev->ddev, &idx)) { + memcpy_toio(adev->vcn.inst[i].cpu_addr, adev->vcn.fw->data + offset, + le32_to_cpu(hdr->ucode_size_bytes)); + drm_dev_exit(idx); + } size -= le32_to_cpu(hdr->ucode_size_bytes); ptr += le32_to_cpu(hdr->ucode_size_bytes); } diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c index 9f868cf3b832..7dd5f10ab570 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c @@ -32,6 +32,7 @@ #include #include +#include #include "amdgpu.h" #include "amdgpu_trace.h" #include "amdgpu_amdkfd.h" @@ -1606,7 +1607,10 @@ static int amdgpu_vm_bo_update_mapping(struct amdgpu_device *adev, struct amdgpu_vm_update_params params; enum amdgpu_sync_mode sync_mode; uint64_t pfn; - int r; + int r, idx; + + if (!drm_dev_enter(&adev->ddev, &idx)) + return -ENODEV; memset(¶ms, 0, sizeof(params)); params.adev = adev; @@ -1715,6 +1719,7 @@ static int amdgpu_vm_bo_update_mapping(struct amdgpu_device *adev, error_unlock: amdgpu_vm_eviction_unlock(vm); + drm_dev_exit(idx); return r; } diff --git a/drivers/gpu/drm/amd/amdgpu/psp_v11_0.c b/drivers/gpu/drm/amd/amdgpu/psp_v11_0.c index 589410c32d09..2cec71e823f5 100644 --- a/drivers/gpu/drm/amd/amdgpu/psp_v11_0.c +++ b/drivers/gpu/drm/amd/amdgpu/psp_v11_0.c @@ -23,6 +23,7 @@ #include #include #include +#include #include "amdgpu.h" #include "amdgpu_psp.h" @@ -269,10 +270,8 @@ static int psp_v11_0_bootloader_load_kdb(struct psp_context *psp) if (ret) return ret; - memset(psp->fw_pri_buf, 0, PSP_1_MEG); - /* Copy PSP KDB binary to memory */ - memcpy(psp->fw_pri_buf, psp->kdb_start_addr, psp->kdb_bin_size); + psp_copy_fw(psp, psp->kdb_start_addr, psp->kdb_bin_size); /* Provide the PSP KDB to bootloader */ WREG32_SOC15(MP0, 0, mmMP0_SMN_C2PMSG_36, @@ -302,10 +301,8 @@ static int psp_v11_0_bootloader_load_spl(struct psp_context *psp) if (ret) return ret; - memset(psp->fw_pri_buf, 0, PSP_1_MEG); - /* Copy PSP SPL binary to memory */ - memcpy(psp->fw_pri_buf, psp->spl_start_addr, psp->spl_bin_size); + psp_copy_fw(psp, psp->spl_start_addr, psp->spl_bin_size); /* Provide the PSP SPL to bootloader */ WREG32_SOC15(MP0, 0, mmMP0_SMN_C2PMSG_36, @@ -335,10 +332,8 @@ static int psp_v11_0_bootloader_load_sysdrv(struct psp_context *psp) if (ret) return ret; - memset(psp->fw_pri_buf, 0, PSP_1_MEG); - /* Copy PSP System Driver binary to memory */ - memcpy(psp->fw_pri_buf, psp->sys_start_addr, psp->sys_bin_size); + psp_copy_fw(psp, psp->sys_start_addr, psp->sys_bin_size); /* Provide the sys driver to bootloader */ WREG32_SOC15(MP0, 0, mmMP0_SMN_C2PMSG_36, @@ -371,10 +366,8 @@ static int psp_v11_0_bootloader_load_sos(struct psp_context *psp) if (ret) return ret; - memset(psp->fw_pri_buf, 0, PSP_1_MEG); - /* Copy Secure OS binary to PSP memory */ - memcpy(psp->fw_pri_buf, psp->sos_start_addr, psp->sos_bin_size); + psp_copy_fw(psp, psp->sos_start_addr, psp->sos_bin_size); /* Provide the PSP secure OS to bootloader */ WREG32_SOC15(MP0, 0, mmMP0_SMN_C2PMSG_36, @@ -608,7 +601,7 @@ static int psp_v11_0_memory_training(struct psp_context *psp, uint32_t ops) uint32_t p2c_header[4]; uint32_t sz; void *buf; - int ret; + int ret, idx; if (ctx->init == PSP_MEM_TRAIN_NOT_SUPPORT) { DRM_DEBUG("Memory training is not supported.\n"); @@ -681,17 +674,24 @@ static int psp_v11_0_memory_training(struct psp_context *psp, uint32_t ops) return -ENOMEM; } - memcpy_fromio(buf, adev->mman.aper_base_kaddr, sz); - ret = psp_v11_0_memory_training_send_msg(psp, PSP_BL__DRAM_LONG_TRAIN); - if (ret) { - DRM_ERROR("Send long training msg failed.\n"); + if (drm_dev_enter(&adev->ddev, &idx)) { + memcpy_fromio(buf, adev->mman.aper_base_kaddr, sz); + ret = psp_v11_0_memory_training_send_msg(psp, PSP_BL__DRAM_LONG_TRAIN); + if (ret) { + DRM_ERROR("Send long training msg failed.\n"); + vfree(buf); + drm_dev_exit(idx); + return ret; + } + + memcpy_toio(adev->mman.aper_base_kaddr, buf, sz); + adev->hdp.funcs->flush_hdp(adev, NULL); vfree(buf); - return ret; + drm_dev_exit(idx); + } else { + vfree(buf); + return -ENODEV; } - - memcpy_toio(adev->mman.aper_base_kaddr, buf, sz); - adev->hdp.funcs->flush_hdp(adev, NULL); - vfree(buf); } if (ops & PSP_MEM_TRAIN_SAVE) { diff --git a/drivers/gpu/drm/amd/amdgpu/psp_v12_0.c b/drivers/gpu/drm/amd/amdgpu/psp_v12_0.c index c4828bd3264b..618e5b6b85d9 100644 --- a/drivers/gpu/drm/amd/amdgpu/psp_v12_0.c +++ b/drivers/gpu/drm/amd/amdgpu/psp_v12_0.c @@ -138,10 +138,8 @@ static int psp_v12_0_bootloader_load_sysdrv(struct psp_context *psp) if (ret) return ret; - memset(psp->fw_pri_buf, 0, PSP_1_MEG); - /* Copy PSP System Driver binary to memory */ - memcpy(psp->fw_pri_buf, psp->sys_start_addr, psp->sys_bin_size); + psp_copy_fw(psp, psp->sys_start_addr, psp->sys_bin_size); /* Provide the sys driver to bootloader */ WREG32_SOC15(MP0, 0, mmMP0_SMN_C2PMSG_36, @@ -179,10 +177,8 @@ static int psp_v12_0_bootloader_load_sos(struct psp_context *psp) if (ret) return ret; - memset(psp->fw_pri_buf, 0, PSP_1_MEG); - /* Copy Secure OS binary to PSP memory */ - memcpy(psp->fw_pri_buf, psp->sos_start_addr, psp->sos_bin_size); + psp_copy_fw(psp, psp->sos_start_addr, psp->sos_bin_size); /* Provide the PSP secure OS to bootloader */ WREG32_SOC15(MP0, 0, mmMP0_SMN_C2PMSG_36, diff --git a/drivers/gpu/drm/amd/amdgpu/psp_v3_1.c b/drivers/gpu/drm/amd/amdgpu/psp_v3_1.c index f2e725f72d2f..d0a6cccd0897 100644 --- a/drivers/gpu/drm/amd/amdgpu/psp_v3_1.c +++ b/drivers/gpu/drm/amd/amdgpu/psp_v3_1.c @@ -102,10 +102,8 @@ static int psp_v3_1_bootloader_load_sysdrv(struct psp_context *psp) if (ret) return ret; - memset(psp->fw_pri_buf, 0, PSP_1_MEG); - /* Copy PSP System Driver binary to memory */ - memcpy(psp->fw_pri_buf, psp->sys_start_addr, psp->sys_bin_size); + psp_copy_fw(psp, psp->sys_start_addr, psp->sys_bin_size); /* Provide the sys driver to bootloader */ WREG32_SOC15(MP0, 0, mmMP0_SMN_C2PMSG_36, @@ -143,10 +141,8 @@ static int psp_v3_1_bootloader_load_sos(struct psp_context *psp) if (ret) return ret; - memset(psp->fw_pri_buf, 0, PSP_1_MEG); - /* Copy Secure OS binary to PSP memory */ - memcpy(psp->fw_pri_buf, psp->sos_start_addr, psp->sos_bin_size); + psp_copy_fw(psp, psp->sos_start_addr, psp->sos_bin_size); /* Provide the PSP secure OS to bootloader */ WREG32_SOC15(MP0, 0, mmMP0_SMN_C2PMSG_36, diff --git a/drivers/gpu/drm/amd/amdgpu/vce_v4_0.c b/drivers/gpu/drm/amd/amdgpu/vce_v4_0.c index 8e238dea7bef..90910d19db12 100644 --- a/drivers/gpu/drm/amd/amdgpu/vce_v4_0.c +++ b/drivers/gpu/drm/amd/amdgpu/vce_v4_0.c @@ -25,6 +25,7 @@ */ #include +#include #include "amdgpu.h" #include "amdgpu_vce.h" @@ -555,16 +556,19 @@ static int vce_v4_0_hw_fini(void *handle) static int vce_v4_0_suspend(void *handle) { struct amdgpu_device *adev = (struct amdgpu_device *)handle; - int r; + int r, idx; if (adev->vce.vcpu_bo == NULL) return 0; - if (adev->firmware.load_type == AMDGPU_FW_LOAD_PSP) { - unsigned size = amdgpu_bo_size(adev->vce.vcpu_bo); - void *ptr = adev->vce.cpu_addr; + if (drm_dev_enter(&adev->ddev, &idx)) { + if (adev->firmware.load_type == AMDGPU_FW_LOAD_PSP) { + unsigned size = amdgpu_bo_size(adev->vce.vcpu_bo); + void *ptr = adev->vce.cpu_addr; - memcpy_fromio(adev->vce.saved_bo, ptr, size); + memcpy_fromio(adev->vce.saved_bo, ptr, size); + } + drm_dev_exit(idx); } r = vce_v4_0_hw_fini(adev); @@ -577,16 +581,20 @@ static int vce_v4_0_suspend(void *handle) static int vce_v4_0_resume(void *handle) { struct amdgpu_device *adev = (struct amdgpu_device *)handle; - int r; + int r, idx; if (adev->vce.vcpu_bo == NULL) return -EINVAL; if (adev->firmware.load_type == AMDGPU_FW_LOAD_PSP) { - unsigned size = amdgpu_bo_size(adev->vce.vcpu_bo); - void *ptr = adev->vce.cpu_addr; - memcpy_toio(ptr, adev->vce.saved_bo, size); + if (drm_dev_enter(&adev->ddev, &idx)) { + unsigned size = amdgpu_bo_size(adev->vce.vcpu_bo); + void *ptr = adev->vce.cpu_addr; + + memcpy_toio(ptr, adev->vce.saved_bo, size); + drm_dev_exit(idx); + } } else { r = amdgpu_vce_resume(adev); if (r) diff --git a/drivers/gpu/drm/amd/amdgpu/vcn_v3_0.c b/drivers/gpu/drm/amd/amdgpu/vcn_v3_0.c index 3f15bf34123a..df34be8ec82d 100644 --- a/drivers/gpu/drm/amd/amdgpu/vcn_v3_0.c +++ b/drivers/gpu/drm/amd/amdgpu/vcn_v3_0.c @@ -34,6 +34,8 @@ #include "vcn/vcn_3_0_0_sh_mask.h" #include "ivsrcid/vcn/irqsrcs_vcn_2_0.h" +#include + #define mmUVD_CONTEXT_ID_INTERNAL_OFFSET 0x27 #define mmUVD_GPCOM_VCPU_CMD_INTERNAL_OFFSET 0x0f #define mmUVD_GPCOM_VCPU_DATA0_INTERNAL_OFFSET 0x10 @@ -268,16 +270,20 @@ static int vcn_v3_0_sw_init(void *handle) static int vcn_v3_0_sw_fini(void *handle) { struct amdgpu_device *adev = (struct amdgpu_device *)handle; - int i, r; + int i, r, idx; - for (i = 0; i < adev->vcn.num_vcn_inst; i++) { - volatile struct amdgpu_fw_shared *fw_shared; + if (drm_dev_enter(&adev->ddev, &idx)) { + for (i = 0; i < adev->vcn.num_vcn_inst; i++) { + volatile struct amdgpu_fw_shared *fw_shared; - if (adev->vcn.harvest_config & (1 << i)) - continue; - fw_shared = adev->vcn.inst[i].fw_shared_cpu_addr; - fw_shared->present_flag_0 = 0; - fw_shared->sw_ring.is_enabled = false; + if (adev->vcn.harvest_config & (1 << i)) + continue; + fw_shared = adev->vcn.inst[i].fw_shared_cpu_addr; + fw_shared->present_flag_0 = 0; + fw_shared->sw_ring.is_enabled = false; + } + + drm_dev_exit(idx); } if (amdgpu_sriov_vf(adev)) diff --git a/drivers/gpu/drm/amd/pm/powerplay/smumgr/smu7_smumgr.c b/drivers/gpu/drm/amd/pm/powerplay/smumgr/smu7_smumgr.c index aae25243eb10..d628b91846c9 100644 --- a/drivers/gpu/drm/amd/pm/powerplay/smumgr/smu7_smumgr.c +++ b/drivers/gpu/drm/amd/pm/powerplay/smumgr/smu7_smumgr.c @@ -405,6 +405,8 @@ int smu7_request_smu_load_fw(struct pp_hwmgr *hwmgr) UCODE_ID_MEC_STORAGE, &toc->entry[toc->num_entries++]), "Failed to Get Firmware Entry.", r = -EINVAL; goto failed); } + + /* AG TODO Can't call drm_dev_enter/exit because access adev->ddev here ... */ memcpy_toio(smu_data->header_buffer.kaddr, smu_data->toc, sizeof(struct SMU_DRAMData_TOC)); smum_send_msg_to_smc_with_parameter(hwmgr, From patchwork Mon May 10 16:36:20 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Andrey Grodzovsky X-Patchwork-Id: 12248681 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-18.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,MSGID_FROM_MTA_HEADER,SPF_HELO_NONE, SPF_PASS,USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 59673C433B4 for ; Mon, 10 May 2021 16:37:16 +0000 (UTC) Received: from gabe.freedesktop.org (gabe.freedesktop.org [131.252.210.177]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 16A0561155 for ; Mon, 10 May 2021 16:37:16 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 16A0561155 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=amd.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=dri-devel-bounces@lists.freedesktop.org Received: from gabe.freedesktop.org (localhost [127.0.0.1]) by gabe.freedesktop.org (Postfix) with ESMTP id 1D7EE6E8C1; Mon, 10 May 2021 16:37:05 +0000 (UTC) Received: from NAM11-DM6-obe.outbound.protection.outlook.com (mail-dm6nam11on2045.outbound.protection.outlook.com [40.107.223.45]) by gabe.freedesktop.org (Postfix) with ESMTPS id DCC246E8BF; Mon, 10 May 2021 16:37:02 +0000 (UTC) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=NOZBAXuF2IP5dVyOee0G03YJTmc6tyD/OuE/m9muKxdoE0PFbe0lDtZXWlti3lnYn5LAmO6rK64lgAn0Kbku+4TWpsZ7IrNpdmnoWgtQz0p2hLXLNyT7YJpUFcS4LFEQOLcguJ5QXO9RV4pohkjyAwQIF8DnYmk4upZbhCy7dti19VTrQStxCVvF6CNQrdOtBTmBZh+qJmM/+RPKs/5RubIw4A+LW8jaT2evS7GIIK1CbTenUVTOm3odnX16VqqTpZiKKXxDFFWP6IlpHAfdei5Hmlqvh3FqbGYuig+bgp3kO7esVw5Paa8WruAPidsDXkuwRyCOmi9t5t7VulB1Jw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=GyxO3UGIf0UkObd+Uj0vO9NCfXhm2KLCGxrpgYsDd8c=; b=muH/Y22v57R62+YWZ5MuhXFQs/HrKc+msilVDZv7XrTRfNqr4cXQI3VdS1gcazYi3X8wu7qIhnH9vkIKKmy7xY0N796W8t/IpIV9IWjpajktJWP98y8gcnLFyKvFvzKDvY8QyXh6EQhy4+J8o1AhNX/xQKqoumat1vDhgdn3Fd325VgyNY3iTaQyvB/yTg0d4CoGv1sqyRImBpk0ue1B8YIkgOo9Hm6XeX4C+L4vqbFu95cOtG3lzwuHsGkk7W10IaO4htwZvJ2RTzLcekgY/tirUVzpaC4bql8DHS9rCwq0vpfT9avnMeWjhpoXVwFyzojQGc4KA/h2H7vG8hdQEw== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=amd.com; dmarc=pass action=none header.from=amd.com; dkim=pass header.d=amd.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=GyxO3UGIf0UkObd+Uj0vO9NCfXhm2KLCGxrpgYsDd8c=; b=0677oPdLSVpt0iiIu3DsZniCcKQVMlq6O3IlbgpBr1Hz+BEJdSnzNpWn+/JSEtiEQ0FQvrkg3RJVV84X2fEusTtCAZ61ati1VDuF2+BAfjIo+CWDWaWNP7N1k5BVcsC8vx2NQDU2OtaePbaKqyy7poEkGEs5yzeywBhzF0FKhdA= Authentication-Results: lists.freedesktop.org; dkim=none (message not signed) header.d=none; lists.freedesktop.org; dmarc=none action=none header.from=amd.com; Received: from SN6PR12MB4623.namprd12.prod.outlook.com (2603:10b6:805:e9::17) by SN6PR12MB4717.namprd12.prod.outlook.com (2603:10b6:805:e2::31) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4108.24; Mon, 10 May 2021 16:37:01 +0000 Received: from SN6PR12MB4623.namprd12.prod.outlook.com ([fe80::ad51:8c49:b171:856c]) by SN6PR12MB4623.namprd12.prod.outlook.com ([fe80::ad51:8c49:b171:856c%7]) with mapi id 15.20.4108.031; Mon, 10 May 2021 16:37:01 +0000 From: Andrey Grodzovsky To: dri-devel@lists.freedesktop.org, amd-gfx@lists.freedesktop.org, linux-pci@vger.kernel.org, ckoenig.leichtzumerken@gmail.com, daniel.vetter@ffwll.ch, Harry.Wentland@amd.com Subject: [PATCH v6 11/16] drm/sched: Make timeout timer rearm conditional. Date: Mon, 10 May 2021 12:36:20 -0400 Message-Id: <20210510163625.407105-12-andrey.grodzovsky@amd.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210510163625.407105-1-andrey.grodzovsky@amd.com> References: <20210510163625.407105-1-andrey.grodzovsky@amd.com> X-Originating-IP: [2607:fea8:3edf:49b0:6a5:47b8:e610:f6a3] X-ClientProxiedBy: YT2PR01CA0020.CANPRD01.PROD.OUTLOOK.COM (2603:10b6:b01:38::25) To SN6PR12MB4623.namprd12.prod.outlook.com (2603:10b6:805:e9::17) MIME-Version: 1.0 X-MS-Exchange-MessageSentRepresentingType: 1 Received: from agrodzovsky-All-Series.hitronhub.home (2607:fea8:3edf:49b0:6a5:47b8:e610:f6a3) by YT2PR01CA0020.CANPRD01.PROD.OUTLOOK.COM (2603:10b6:b01:38::25) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4108.31 via Frontend Transport; Mon, 10 May 2021 16:37:00 +0000 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 68631d47-e3b6-4ac2-20eb-08d913d1d605 X-MS-TrafficTypeDiagnostic: SN6PR12MB4717: X-MS-Exchange-Transport-Forked: True X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:6108; X-MS-Exchange-SenderADCheck: 1 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: 9nXckeHMcc2IIh3fiwDUHYyakSAZ9BTelkU2Jcg5HkXygKAkQYn0KZwvlz3U9VkVfPWPqa6jHc6X+piWJPA+dbkhYRn7yBS4zt/G95ZzNSZKKbaqF8JtZTYYhGzwoGTYGAc7gk/h834wbvvyW9FBNmFLTiAtJDhNOyyJhe3Yscxu8XaiSkmK2CD6K65Pyt38zBgGdR72WgeTzy7qGZnoY5UHqSvV2EXbKRUMt1J6ayRRJDpTEwlClez5R4tsGLNyoStmCr25QtaSD9WnOtfEOnL15+gp/1c4Il/MxzJ1k8bw3Eb8wrnR+GRMKd3FjbtXI55o+1eZAj+kal6ex3qap5J2brH5oJ7Y49D2Cgs9l3nier7I2/6TIou8quqMVR/B+UAVT9d5dleszO4djsEHn6ich29BOo7srgicKiYStcRXFaFVH8ipG5dGb4sdNb1LHNSCUtLud1f65WwNe16RejpYz6LixKDULZnPoWI6oqZwV4eD3SR/jl+IErxajAIRIS0FTIt63fmwIHiZyCFpsEUS/GfWqM2aKzTGTkoAVmLT0e6IAZJQy0ml1W4s4sGFy9Di/3kiBIK3AvDZXe8+xgrBM+T8H5auvXXIhfq53Ns= X-Forefront-Antispam-Report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:SN6PR12MB4623.namprd12.prod.outlook.com; PTR:; CAT:NONE; SFS:(4636009)(39860400002)(376002)(346002)(396003)(136003)(366004)(6486002)(44832011)(83380400001)(5660300002)(2616005)(8676002)(1076003)(86362001)(6506007)(478600001)(186003)(16526019)(8936002)(6512007)(6666004)(38100700002)(36756003)(52116002)(2906002)(316002)(4326008)(66946007)(6636002)(66476007)(66556008); DIR:OUT; SFP:1101; X-MS-Exchange-AntiSpam-MessageData: 8jZ9JMKP7H798Pvee5DDPxaQoBE8WB5hNg0d9ZPbbXbT64hJ2mkabII1ItmcIM0Kx965w9az62zzjaCnno3kfqZk/7nmAqNKbMo8G2GpHK73R3Uk0nuXebZWVmaVtSST+T2pqK9DPdhCCQPfNz/wWIj2V9R0HYwdpUjiVksFgf/+1Yr8dFvYfLUt/haviUnYJqLhKGkIH5FS+25YfIuwwAmfUoKnXHHFrE48kKFr2sJKAgRixEP6NFiFPCpqUlPp2mFumm+efATJeJKoYcv/53wijdO5zv8iKFjBy4KGncqAhQnpaHjqkRubIfxoE+eWmH4xPUVIVmSVywZHGAo6FmWpFKgyumVmt08mxGvG7KnecGkdg90Vu63zFp6L9pLUPEkYoTkTaGunwWjSovxye9iIhYvKLpV2dAXupO8r4n160bGWYPAy076ZWZBVXPUJxLfQgRL+4lvWJbrfkK/pNM0pelvqje/w8LNFJs9OYIHSjdZae6zNIHzP23vMCKHn1sNm7Yh7kqPe0QTkxCuf0vUM8N0BnIW+56g8lN0fxDIQvBqAxsAnhS7i4UC6K0TCivtOROtOZ488bHAvu83QJ6l+laJaHPYPvylCvWbtam2+MwHLzfawkfZ8ckOHw5ceC/3BHHkeJV4xF/VfpzZ8+ShsdwyGpeQGFFT3pidt1Ylzr5YYxL+qDTCWFpgNtoWnF5uabROgaWf85zzLBZlQHznl7HjBdAx8/MVsXTJ0ZTPL2Ue1HGor7DlNofcPxO6WuYqW5fEY7QrN4FbMogjpkaEISPH8HcyXuUccjwo9gOL2Hdd4KimVZVt1Ciect1ZsEZAo1yirPmoakvZPdH8DYmBUUI6JtCF9j2flsCWgA/1G0rT78iV9vVyZ4IajtDeuTb1ZBqjOumwSIAHfcuRpc9K9WJ/q9P/mTVvlT9+4iki2FzNZ+xO7WhYojdUel/ZiK59cLsof0fnbB2ndfaEOHNeoYcF2koJ8hkCTd/p0CCEzeRhMyIFJDgElYeh/tAZoHcwY1EO5JnYWxxGdehFtBAGnnZgMPi78cISDAT4u8IOubJ9ZCVUSAlAJleBB/qgk86ZLjp9ElJUavW1+juOpLqLTtyiPE3MKvSMBMLaWbQoEyWxBMvPLES10h/nGwctnaBx+SLCZfoTxk0g0x2qAl2ev9VT//sLP0l2s3vFqGxNjw85qrPwiw187WjjZTU1ClbZtjdKmVJ/PIg5Cqdy+1tzlMIqTWtW8uqJFMiGkaIatTHVihbX7KYqtYu67/pNViUQ4kVF3s3Y8y4vqLeGkDF30ef1Qs4CnV/ASoqHmWlWam342EQRs3uthmF+7BQdX8SQcQYC4KnBi/PLLNpOZCbYWBDAU1pVkWRTHQdF6Vi/yU4xIj5G5y0kbgQyFEMKL X-OriginatorOrg: amd.com X-MS-Exchange-CrossTenant-Network-Message-Id: 68631d47-e3b6-4ac2-20eb-08d913d1d605 X-MS-Exchange-CrossTenant-AuthSource: SN6PR12MB4623.namprd12.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 10 May 2021 16:37:01.0977 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: sAOdle7BN79BoNWoHzjm5uf8Aq82XFCSNyIRAFCi0SlyKmJALIANDUP6OAfw7tn2aifmRqDaLXmF4uKDP1sEoQ== X-MS-Exchange-Transport-CrossTenantHeadersStamped: SN6PR12MB4717 X-BeenThere: dri-devel@lists.freedesktop.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Direct Rendering Infrastructure - Development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: gregkh@linuxfoundation.org, Felix.Kuehling@amd.com, helgaas@kernel.org, Alexander.Deucher@amd.com Errors-To: dri-devel-bounces@lists.freedesktop.org Sender: "dri-devel" We don't want to rearm the timer if driver hook reports that the device is gone. v5: Update drm_gpu_sched_stat values in code. Signed-off-by: Andrey Grodzovsky Reviewed-by: Christian König --- drivers/gpu/drm/scheduler/sched_main.c | 11 +++++++---- 1 file changed, 7 insertions(+), 4 deletions(-) diff --git a/drivers/gpu/drm/scheduler/sched_main.c b/drivers/gpu/drm/scheduler/sched_main.c index f4f474944169..8d1211e87101 100644 --- a/drivers/gpu/drm/scheduler/sched_main.c +++ b/drivers/gpu/drm/scheduler/sched_main.c @@ -314,6 +314,7 @@ static void drm_sched_job_timedout(struct work_struct *work) { struct drm_gpu_scheduler *sched; struct drm_sched_job *job; + enum drm_gpu_sched_stat status = DRM_GPU_SCHED_STAT_NOMINAL; sched = container_of(work, struct drm_gpu_scheduler, work_tdr.work); @@ -331,7 +332,7 @@ static void drm_sched_job_timedout(struct work_struct *work) list_del_init(&job->list); spin_unlock(&sched->job_list_lock); - job->sched->ops->timedout_job(job); + status = job->sched->ops->timedout_job(job); /* * Guilty job did complete and hence needs to be manually removed @@ -345,9 +346,11 @@ static void drm_sched_job_timedout(struct work_struct *work) spin_unlock(&sched->job_list_lock); } - spin_lock(&sched->job_list_lock); - drm_sched_start_timeout(sched); - spin_unlock(&sched->job_list_lock); + if (status != DRM_GPU_SCHED_STAT_ENODEV) { + spin_lock(&sched->job_list_lock); + drm_sched_start_timeout(sched); + spin_unlock(&sched->job_list_lock); + } } /** From patchwork Mon May 10 16:36:21 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Andrey Grodzovsky X-Patchwork-Id: 12248685 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-18.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,MSGID_FROM_MTA_HEADER,SPF_HELO_NONE, SPF_PASS,USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id DA263C43470 for ; Mon, 10 May 2021 16:37:18 +0000 (UTC) Received: from gabe.freedesktop.org (gabe.freedesktop.org [131.252.210.177]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 96A4A611AB for ; Mon, 10 May 2021 16:37:18 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 96A4A611AB Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=amd.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=dri-devel-bounces@lists.freedesktop.org Received: from gabe.freedesktop.org (localhost [127.0.0.1]) by gabe.freedesktop.org (Postfix) with ESMTP id AF6096E8C2; Mon, 10 May 2021 16:37:06 +0000 (UTC) Received: from NAM04-MW2-obe.outbound.protection.outlook.com (mail-mw2nam08on2067.outbound.protection.outlook.com [40.107.101.67]) by gabe.freedesktop.org (Postfix) with ESMTPS id 5F0CA6E8BB; Mon, 10 May 2021 16:37:04 +0000 (UTC) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=Ad73/V65/bNixlr7E4bGVfLiP2RBaz/Dye+lh9MEmQ6i83Jk98NQpWVY67VOro7zc+nEG/2M9rEe6V/a9LRm7GdurDgcqvl4jac8IJ1XSmNWYAyxDt7yvkgk3lzhYdgb69aOsEkz3Hp3Vf9IqjHUxROmfH4niBXY87PW8XZRgX8f182CbbIOdDJ0qAViEqHVKbfre/RFmX02WzglT49LUep9+NvGbLu74+icnpiJuS3g/7zZKbVYxB9Mc3fMy6VXLtr3a3wcSsYkEdhvgJx3e8H9hCvg4o1dmF+ay9BkYHyiz61oTpW7RsuLMbguhsqQm2WLLvVQrseYJFaTP5NUVg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=Yq2Nfd7RRClQQ37WYASWsolamzag+TDEAOgKHpaDDxg=; b=SmeYE+wYDrfTYHOHcyLaWkl/S8qQsfG+0lhsWaaLbAJgIYUENhS6l3ynjM9o3Dptu/sp2ecMhDVhl1kZmyCP9XhKemTsxIm9Gz7laGoI5bpzy/8bcsfA+9J9nPgUfm3DHvdI8im5MW74eO20s6vAZq4vufzMcvMYQnc/L6Jwkbc0qi03pzAQp7tsB1iPGRyszfTR3FjTzTlimluQqV017KpaSv0bJZW1E26MryuuW+Hn2y/8CeF5k78y4Ac63/hE+f930hBkiIh64IceWNy0/XKH8SV+dd/0R9iIoWMgQxIcNvUlDlRiKnneB9m1XpfX06aRKfdLZwIitWj+WyE04Q== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=amd.com; dmarc=pass action=none header.from=amd.com; dkim=pass header.d=amd.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=Yq2Nfd7RRClQQ37WYASWsolamzag+TDEAOgKHpaDDxg=; b=HyNwLimS1rBtXSCPrPhA2dG2pjU6yZgR+Tq1s5VhHAC2WgQvCugVvD0fkZScEFeEXuxsEw/79sOZxN9xbF3kjKJBvW1xYAHTU6DXi4EpczbkBYO+FqqYDcUKXQHpaK46vSSvj7KAmFwP9SR5GDixfNlBoknRNUaPEV63vehHWdM= Authentication-Results: lists.freedesktop.org; dkim=none (message not signed) header.d=none; lists.freedesktop.org; dmarc=none action=none header.from=amd.com; Received: from SN6PR12MB4623.namprd12.prod.outlook.com (2603:10b6:805:e9::17) by SN6PR12MB4717.namprd12.prod.outlook.com (2603:10b6:805:e2::31) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4108.24; Mon, 10 May 2021 16:37:03 +0000 Received: from SN6PR12MB4623.namprd12.prod.outlook.com ([fe80::ad51:8c49:b171:856c]) by SN6PR12MB4623.namprd12.prod.outlook.com ([fe80::ad51:8c49:b171:856c%7]) with mapi id 15.20.4108.031; Mon, 10 May 2021 16:37:03 +0000 From: Andrey Grodzovsky To: dri-devel@lists.freedesktop.org, amd-gfx@lists.freedesktop.org, linux-pci@vger.kernel.org, ckoenig.leichtzumerken@gmail.com, daniel.vetter@ffwll.ch, Harry.Wentland@amd.com Subject: [PATCH v6 12/16] drm/amdgpu: Prevent any job recoveries after device is unplugged. Date: Mon, 10 May 2021 12:36:21 -0400 Message-Id: <20210510163625.407105-13-andrey.grodzovsky@amd.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210510163625.407105-1-andrey.grodzovsky@amd.com> References: <20210510163625.407105-1-andrey.grodzovsky@amd.com> X-Originating-IP: [2607:fea8:3edf:49b0:6a5:47b8:e610:f6a3] X-ClientProxiedBy: YT2PR01CA0020.CANPRD01.PROD.OUTLOOK.COM (2603:10b6:b01:38::25) To SN6PR12MB4623.namprd12.prod.outlook.com (2603:10b6:805:e9::17) MIME-Version: 1.0 X-MS-Exchange-MessageSentRepresentingType: 1 Received: from agrodzovsky-All-Series.hitronhub.home (2607:fea8:3edf:49b0:6a5:47b8:e610:f6a3) by YT2PR01CA0020.CANPRD01.PROD.OUTLOOK.COM (2603:10b6:b01:38::25) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4108.31 via Frontend Transport; Mon, 10 May 2021 16:37:01 +0000 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: fa2848a6-f9f8-463a-a7fa-08d913d1d6fa X-MS-TrafficTypeDiagnostic: SN6PR12MB4717: X-MS-Exchange-Transport-Forked: True X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:5797; X-MS-Exchange-SenderADCheck: 1 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: oRSRjvYl4qPVeoQ2x7n3yX+DEPJDxE9Qpg1hAN0LX076qS0a282XvFIs2aYfMu7WgWMZdkrYz6Wqwxbh8h6/YAV3laaJorTPAJeRvWm6d6a4a1CPh0HsMANJpVhiCV+TDaHy8NdgxJcBM2Q4OwH35lRu5ta2Uw9PFmlA3N9T/EVQ68ypdc1xYS032C/CBtElP7zlDeKr0dWyHOh/ERL84FJGRj97QYXSvv5V/RENf71+dEgc9K56RW/ZNyCfV4o6lSBMT6Z3Kw96V2tuuvlcCMX5L2905vOfFzo2WXnjxqcGHxMZgQR81aO097XMuQOHtLqVlikUPU2LzbFcZb8UE9cNM7SAkgYGsOEEya1LWYHDCpZjATs44aBxqWAh9jX20yc5zEa2M9KtfBgKiZpB+vZU3yIsp8PphoerXd74ILK6DDFekQivLZkJ4/gBXxTG9l4/eRsRxk0UutUjSzdWS7UGCj011uULwMeEqsi7Hrrz4BkZsq2Od31jPa+6ytSVtEMxLkWBMxxWwbY0XlkMHv/hbI8Wx56lEVmey5aWcveY62Ul6hvTsV6GlpZCvMoUoeeiIbzV4omQkHqJ0VcDXAUA7/+ILRGJcij9CAuihhs= X-Forefront-Antispam-Report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:SN6PR12MB4623.namprd12.prod.outlook.com; PTR:; CAT:NONE; SFS:(4636009)(39860400002)(376002)(346002)(396003)(136003)(366004)(6486002)(44832011)(83380400001)(5660300002)(2616005)(8676002)(1076003)(86362001)(6506007)(478600001)(186003)(16526019)(8936002)(6512007)(6666004)(38100700002)(36756003)(52116002)(2906002)(316002)(4326008)(66946007)(6636002)(66476007)(66556008); DIR:OUT; SFP:1101; X-MS-Exchange-AntiSpam-MessageData: 4uZjXku+5LDOlQhicl7snFE+crf9H4fiWGkMsCRTOcrr8c1hj8qQV6/Fp8UNrPEuejYZqSKTYiEIFT1rotWExFsNwvB4ZgS1rr7L9tJP2yjGqFJhJSnGKhDg/QZJbiOAKANZ6FdqbNfT6TTDL8Qyi1oyZE0cJSKW2S2NhHDCiY+Bhsy+qLoXR9kSOEpHO+AOy9pjPnMT4RGqsFPrkd+CSCN1byT6aoHk5mIuBfhJfwf7ATYcQA+DSwQrp4ivST4uDS3QO+bV1NDVa1CX2RgB78TwilgA0LR2fA809nwxy9uH92a511X7g/rbC66jrlaQlvzvk7zb2Z3tlfc/iaPHNjNN5cVjmKhfRqSg+Iv61LcPHyI9ffmvoWsBwd1nXYJFDjjc0EW6tq89erZYKboRMXZ18yYJd3e9HQSnXkpTcYTmZ6/fkQyDmGgkudKMe9fgwzNA5GhXqlDefVe5e2wluewaedYsXyFSvgq88HhI0UNIn5Z5sNth5QSCDi07llmP/MFXPejJEqLPWMOkAQ3qjkp1fY0QNr0QMF+VRjCsJ67sC+iP0YAwakLfV4sWUp7+gx4pX0EfSfUXxPa3ntGp2OYeHzKnmjN7Wxi2DEJWyqloI3i4MzxKNoFqb4pB0MFfeZO/0y3U9rq4JVlzCkVm5XHpsFeqDYSbUgfKscOeO2ROWDYhdETpEoyUNFM1hUMEhmT0+E1ngDRYVU4OOciTI05M8qA532243l9P3Nm+WZG1+dsjKvUKjDETsFtlkEPyMpvqFrokeUnxzUoBNRpA08EBF69kjpjhk4Ic8XxWKbsBwUkWgYgZkDu923XyUJB5H88yN0zannELvdIC16F9Gela2zO3Y3Ai5EOS0Sigg0c27bfAyY6h72M+49JgNST7u7XOzQ8tqXLYHg4tZX1v9ZX/kM0VWw1WFWtvWvw+tpqaMB3vP4SDSBWlL8wXvEQYiNpIZr/eCAU41OMdGHQnRO57OAhQAns7Q7NxqGKfnxjvZhZF1Rx8hli0kX0Gew9JmTwOQkN5Ma1N6tm1pjl2lTSI0+Hq4I/3fC91wq6eeyGKtREJlhHl+MrFSAfmSBS2H8lJfzKSFCtnpWmV9eOBZg0KQmTcd+MdXKfpmm9kuPUtn88VhIEFAqVcb0xM24Nh8fbqG3dZXBV8Bv5YfPi5mbmXs6cRfcr8AjS5g5ba706vxYvvXIyor8ZrAHIWI0MC8MGifM8/HrPcunGtxACIwfO/9AWZVp1fUErA2GR4VDxCGiEKdAvU8NmlH3XvyeTYPKYpyRbJ21t8snwcf09D0EjO/L/iTeLDudqQ3Oscw+pNnrfXRuk6s9xMJOln3rVISG5OjAz4Re8bws5luVs8A8sOUKFtoVo+YYj5ys6P6oZIZ2mhOHrXBvxlLf6CmHuF X-OriginatorOrg: amd.com X-MS-Exchange-CrossTenant-Network-Message-Id: fa2848a6-f9f8-463a-a7fa-08d913d1d6fa X-MS-Exchange-CrossTenant-AuthSource: SN6PR12MB4623.namprd12.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 10 May 2021 16:37:02.7618 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: Nw/jGj0HDHAP8J/HC/+VG8bql9xMWEWonFlg5WNiwAkM4INIfCPFo1d0MzuhMUOj0aCPfBUmD2eanEvMok33tA== X-MS-Exchange-Transport-CrossTenantHeadersStamped: SN6PR12MB4717 X-BeenThere: dri-devel@lists.freedesktop.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Direct Rendering Infrastructure - Development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: gregkh@linuxfoundation.org, Felix.Kuehling@amd.com, helgaas@kernel.org, Alexander.Deucher@amd.com Errors-To: dri-devel-bounces@lists.freedesktop.org Sender: "dri-devel" Return DRM_TASK_STATUS_ENODEV back to the scheduler when device is not present so they timeout timer will not be rearmed. v5: Update to match updated return values in enum drm_gpu_sched_stat Signed-off-by: Andrey Grodzovsky Reviewed-by: Christian König --- drivers/gpu/drm/amd/amdgpu/amdgpu_job.c | 19 ++++++++++++++++--- 1 file changed, 16 insertions(+), 3 deletions(-) diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_job.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_job.c index 759b34799221..d33e6d97cc89 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_job.c +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_job.c @@ -25,6 +25,8 @@ #include #include +#include + #include "amdgpu.h" #include "amdgpu_trace.h" @@ -34,6 +36,15 @@ static enum drm_gpu_sched_stat amdgpu_job_timedout(struct drm_sched_job *s_job) struct amdgpu_job *job = to_amdgpu_job(s_job); struct amdgpu_task_info ti; struct amdgpu_device *adev = ring->adev; + int idx; + + if (!drm_dev_enter(&adev->ddev, &idx)) { + DRM_INFO("%s - device unplugged skipping recovery on scheduler:%s", + __func__, s_job->sched->name); + + /* Effectively the job is aborted as the device is gone */ + return DRM_GPU_SCHED_STAT_ENODEV; + } memset(&ti, 0, sizeof(struct amdgpu_task_info)); @@ -41,7 +52,7 @@ static enum drm_gpu_sched_stat amdgpu_job_timedout(struct drm_sched_job *s_job) amdgpu_ring_soft_recovery(ring, job->vmid, s_job->s_fence->parent)) { DRM_ERROR("ring %s timeout, but soft recovered\n", s_job->sched->name); - return DRM_GPU_SCHED_STAT_NOMINAL; + goto exit; } amdgpu_vm_get_task_info(ring->adev, job->pasid, &ti); @@ -53,13 +64,15 @@ static enum drm_gpu_sched_stat amdgpu_job_timedout(struct drm_sched_job *s_job) if (amdgpu_device_should_recover_gpu(ring->adev)) { amdgpu_device_gpu_recover(ring->adev, job); - return DRM_GPU_SCHED_STAT_NOMINAL; } else { drm_sched_suspend_timeout(&ring->sched); if (amdgpu_sriov_vf(adev)) adev->virt.tdr_debug = true; - return DRM_GPU_SCHED_STAT_NOMINAL; } + +exit: + drm_dev_exit(idx); + return DRM_GPU_SCHED_STAT_NOMINAL; } int amdgpu_job_alloc(struct amdgpu_device *adev, unsigned num_ibs, From patchwork Mon May 10 16:36:22 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrey Grodzovsky X-Patchwork-Id: 12248687 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-18.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,MSGID_FROM_MTA_HEADER,SPF_HELO_NONE, SPF_PASS,USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 0BBD4C4360C for ; Mon, 10 May 2021 16:37:20 +0000 (UTC) Received: from gabe.freedesktop.org (gabe.freedesktop.org [131.252.210.177]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id C77B4611AB for ; Mon, 10 May 2021 16:37:19 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org C77B4611AB Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=amd.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=dri-devel-bounces@lists.freedesktop.org Received: from gabe.freedesktop.org (localhost [127.0.0.1]) by gabe.freedesktop.org (Postfix) with ESMTP id B221A6E8BE; Mon, 10 May 2021 16:37:09 +0000 (UTC) Received: from NAM04-MW2-obe.outbound.protection.outlook.com (mail-mw2nam08on2089.outbound.protection.outlook.com [40.107.101.89]) by gabe.freedesktop.org (Postfix) with ESMTPS id A48A56E8B6; Mon, 10 May 2021 16:37:05 +0000 (UTC) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=CrjU57sG6/H847qVMEAtqh9wPXUaWlPoOOJtZdDWsaZvOKL4LkzR0p6TWkbCSXi/rFxb4umcxzbokMZX7rK5QVR4ffIQ7tydSWyiiwU8UWESQvjdMYtMpoKEF3AseuMJQ++FjDEdBT3h88HTTh2NDbO7vwwGSwn57ugbwBPfXMh3eH5J4WvzVH8M5yDOXl0s7Im9sT+Vl8Et9Ed/BY/cRB6Nn3DjkgnoUBgLFAXXiVhXV8oJCWNZp2mIi/On+GNI2zkaN5yAACXDe+v+4+DJWcuYMLSpVYRtafvnXW0hZaAhtfDdV00V24loRcx3BrhlEozfXBGKBNF8tL+LvM295A== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=9AZGyPcKDSup40OWapnZWYHJum+sOEaVRemzO6xhrCs=; b=LQl7rV4xnC1ITkJRWIf9WGHlNslWcnmqd5FgMTv4EbCHg9m6yPYcNoR8Jc+KuxYNRE78eqHWvSrYk1n2VMbwdgFh8x84dd62YYRtR/LDxMvlM+digwkNIO/H87yKoKLYhLhWKXB2QLjCDRXrzZDJCoUnok+/PhRnmFAk8SgE9vQBLDP+ck/yS9iJ4pQ4pMO/d5W0y+EFdf/V7OUU7Gon+GPBIfLZh0MxOBObBmjo3pfHHV1CmqYy4sk6DZWRl+son27Uxmg2c1aChiH283H4EG4jNy4dedfoLm1Td2lRpYsqcyUCXBp2CONbrccquxicu0/H8Ogp5rSChwAWET4t2g== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=amd.com; dmarc=pass action=none header.from=amd.com; dkim=pass header.d=amd.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=9AZGyPcKDSup40OWapnZWYHJum+sOEaVRemzO6xhrCs=; b=ndkXorKJAdpo4wp8qGD5Xcrnxy5yhhUn7WKwsDzFUxva0Bpx9fGpb3mDjdOJSUgO4lkEpA4aV++/OBHiOo3+EqdH7w5tp1b7FSa5/hNHkPPpIp3SLIt9D7hBvm9127WELtkwY4vxuosoOPpQwOCjOuPQQNuh/LUGibbDbvLj7Bc= Authentication-Results: lists.freedesktop.org; dkim=none (message not signed) header.d=none; lists.freedesktop.org; dmarc=none action=none header.from=amd.com; Received: from SN6PR12MB4623.namprd12.prod.outlook.com (2603:10b6:805:e9::17) by SN6PR12MB4717.namprd12.prod.outlook.com (2603:10b6:805:e2::31) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4108.24; Mon, 10 May 2021 16:37:04 +0000 Received: from SN6PR12MB4623.namprd12.prod.outlook.com ([fe80::ad51:8c49:b171:856c]) by SN6PR12MB4623.namprd12.prod.outlook.com ([fe80::ad51:8c49:b171:856c%7]) with mapi id 15.20.4108.031; Mon, 10 May 2021 16:37:04 +0000 From: Andrey Grodzovsky To: dri-devel@lists.freedesktop.org, amd-gfx@lists.freedesktop.org, linux-pci@vger.kernel.org, ckoenig.leichtzumerken@gmail.com, daniel.vetter@ffwll.ch, Harry.Wentland@amd.com Subject: [PATCH v6 13/16] drm/amdgpu: Fix hang on device removal. Date: Mon, 10 May 2021 12:36:22 -0400 Message-Id: <20210510163625.407105-14-andrey.grodzovsky@amd.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210510163625.407105-1-andrey.grodzovsky@amd.com> References: <20210510163625.407105-1-andrey.grodzovsky@amd.com> X-Originating-IP: [2607:fea8:3edf:49b0:6a5:47b8:e610:f6a3] X-ClientProxiedBy: YT2PR01CA0020.CANPRD01.PROD.OUTLOOK.COM (2603:10b6:b01:38::25) To SN6PR12MB4623.namprd12.prod.outlook.com (2603:10b6:805:e9::17) MIME-Version: 1.0 X-MS-Exchange-MessageSentRepresentingType: 1 Received: from agrodzovsky-All-Series.hitronhub.home (2607:fea8:3edf:49b0:6a5:47b8:e610:f6a3) by YT2PR01CA0020.CANPRD01.PROD.OUTLOOK.COM (2603:10b6:b01:38::25) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4108.31 via Frontend Transport; Mon, 10 May 2021 16:37:03 +0000 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: e811cac2-a2d1-488d-db3f-08d913d1d803 X-MS-TrafficTypeDiagnostic: SN6PR12MB4717: X-MS-Exchange-Transport-Forked: True X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:3383; X-MS-Exchange-SenderADCheck: 1 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: ah4hYblDMYbeXyWe3XSF0vET7g1uAs8i04g+4mHAFZ4GufMiD9yQ4sAWowvp1eLDFHiIJkEeNCkCSxBSQsO7+iIFfELdfFw9wZfctsyvDGLmoG9NFuUHiJr53xXSymMqWQEsYe1YMXabuuwlsdjcMy+FuSx7DJzJ+b/oiywjgpBXqIH67RtQ5bQJ0G4IMDQZLU4hlzRLh7ZFwSbdYfjg/z9eXFxoF9OYWpFD1u432e7ach8mlXuw+WymJEfONWAT9gqxcJF5FqgRqgzVef84/Yj9zRj6TWdmEwPsxNg60rnnWdoJnG4Nq4T2rY9ZNMcjDfX3m495kFyn364J0fc5X9fiL0q5nZnyW3foQ/+QSXRkVbo0YcQnMDaesmNruvhVFp/xiPsBRQBfB4arKspi9B7dANsHSJvqHJa2Xso77jekmZZR5pczGInGhXp7mRhgEAfCtkc5I/DXvf61j6R9+PGTJtlIOtC7A0JfEJqNpxTbsWeDN/JQpp2cTkPt0mD7AdA3stV3rvKc66+6baYXBP0oZhivoWyFSMy+jindwPx6wrkTCQeKEvmu7OZ70JvQS19YUSwFHIeYF0sWFoHTEgy0MEagpdA8uhb0J3e6hyw= X-Forefront-Antispam-Report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:SN6PR12MB4623.namprd12.prod.outlook.com; PTR:; CAT:NONE; SFS:(4636009)(39860400002)(376002)(346002)(396003)(136003)(366004)(6486002)(44832011)(83380400001)(5660300002)(2616005)(8676002)(1076003)(86362001)(6506007)(478600001)(186003)(16526019)(8936002)(6512007)(6666004)(38100700002)(36756003)(52116002)(2906002)(316002)(4326008)(66946007)(6636002)(66476007)(66556008); DIR:OUT; SFP:1101; X-MS-Exchange-AntiSpam-MessageData: egRm+9vCKocacMLv3KPgPm0o7UjWGS1TbKhW0BbvS5Cg0sf/NeqwMufghgdBj+JQ2xB7yMqvdGdgaaHBe0yt6in0NyhfpYZg72Y8cMeUUzuHtraYS32/QwAaCh/XWSa6K8ChifK0fZBMhkXBdjjUABEifCBx1Ypp5ivj7UyfY3UnW8gpJbsx2TnYwsdkHmgMWnMhYNsZmljjma+gtrrzmEnWpHbYpVXGgmfn+mHqQhKLQ1GURG5Me+vA3myPHxg3Hxk606vvF9bB7mjhoS2mtHPR+3yyEwkYFZubMwXdGnBNI66Esg7pSfibaK6KD5oi3LI2kOz7kwtYWEUkjTw3dbLdvpsXrxmoEd4a4+1LrJfGLvdjsXQncs8dLeQt7UttmPlup3TNJEbv5bjQHwyReOcljYZOnmUVoukFJxYvRyOe4LRH5YWef7+OTQ3aTMg6/GpTvrFLkp7jgMxX8+2I5KLPAFpcz7nbZGdoO8Pr3FXyPvCV59K9Wk/x0oPkYs2Ltg/rxTgo7bx25ngH7rVLElPPT9sDiBEF+bREwoPtIbkN70LHLA2XUrXtFoAxeJ6tznVArSDzIApw7U3EL/O10Q8HzNwUKoakI8d5tr6bCsna6XpQNlpOmAZxN/WGcBeryzqQckOvzlj6yrDiRTBXPaPJicCOQFMMXzro6ABVhns4OgGvk5lkXVib9ziYfgHmCiKlUCRVRoey0qXHMfbAW44NG5gMRNMwAPsVcKy49gmWnhDubFv8x+l+l2dX9VhDkiWH6Gn2TGapk0oUEuQMHGaFap4weFeenB9oTyMsDB18YZethAGc3TgDmvBPUF/V51z49nP74AXRqH2A/wEjIikcy3pjEFhcCRM6CXBq76mOhRen3Cjw6E4cgdsDp6WlLhw8ivZ6jJViC6SETRokDGaQi6WESHZEK+QdlrXWXdgMbeU4OFksMYVxMqjcKdqLR9/1Fhw2WvSSBPGta7snuHZNFyoPl9p6911H8CXFm6FB3OpmvjeNgXgMUEHqjYL48ta2sZkkTC2q0PC6hVjSjT/IQ4NmiIPrCBghAC8aWiHmr3LVJ2QpiepYBm5VTLFzbwP+rwZ7oRMFZm31XpWQfxdkg+xX/lWSliV0MmyGEcKLRIUmPsEdHqM+anRJi5yjQdT6CBNI3z9B2J+Ne+8uFAJmhhE1HBQPHdT378JRdRAiboN5UpJiWxGO65j2UHFpc3XTf5o8TBilzHXygs5Se/zHi6TNqJh4o5jAjRNyofBidvMNus6urg2zVT2t+qfGxSibKw68v+hsCFTr0u2O4czfIriqah5A0spt/bcgabTOnHN4jqL5Cmq9D1JTz+0X1M8/dQWcooBYK69+Ky7VjZsNlHFfp9IvFgEf6hQwug7rKny8A8YM6ZJsKrszoLb4 X-OriginatorOrg: amd.com X-MS-Exchange-CrossTenant-Network-Message-Id: e811cac2-a2d1-488d-db3f-08d913d1d803 X-MS-Exchange-CrossTenant-AuthSource: SN6PR12MB4623.namprd12.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 10 May 2021 16:37:04.4129 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: 1JKVk545nufP9ikbyr/rONDQiActZ68yHORm7npuzg3y3kd+q1QY7m0z3CeDgs8JZFXKywPDtd+9HLwlOgE2AA== X-MS-Exchange-Transport-CrossTenantHeadersStamped: SN6PR12MB4717 X-BeenThere: dri-devel@lists.freedesktop.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Direct Rendering Infrastructure - Development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: gregkh@linuxfoundation.org, Felix.Kuehling@amd.com, helgaas@kernel.org, Alexander.Deucher@amd.com Errors-To: dri-devel-bounces@lists.freedesktop.org Sender: "dri-devel" If removing while commands in flight you cannot wait to flush the HW fences on a ring since the device is gone. Signed-off-by: Andrey Grodzovsky --- drivers/gpu/drm/amd/amdgpu/amdgpu_fence.c | 16 ++++++++++------ 1 file changed, 10 insertions(+), 6 deletions(-) diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_fence.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_fence.c index 1ffb36bd0b19..fa03702ecbfb 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_fence.c +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_fence.c @@ -36,6 +36,7 @@ #include #include +#include #include "amdgpu.h" #include "amdgpu_trace.h" @@ -525,8 +526,7 @@ int amdgpu_fence_driver_init(struct amdgpu_device *adev) */ void amdgpu_fence_driver_fini_hw(struct amdgpu_device *adev) { - unsigned i, j; - int r; + int i, r; for (i = 0; i < AMDGPU_MAX_RINGS; i++) { struct amdgpu_ring *ring = adev->rings[i]; @@ -535,11 +535,15 @@ void amdgpu_fence_driver_fini_hw(struct amdgpu_device *adev) continue; if (!ring->no_scheduler) drm_sched_fini(&ring->sched); - r = amdgpu_fence_wait_empty(ring); - if (r) { - /* no need to trigger GPU reset as we are unloading */ + /* You can't wait for HW to signal if it's gone */ + if (!drm_dev_is_unplugged(&adev->ddev)) + r = amdgpu_fence_wait_empty(ring); + else + r = -ENODEV; + /* no need to trigger GPU reset as we are unloading */ + if (r) amdgpu_fence_driver_force_completion(ring); - } + if (ring->fence_drv.irq_src) amdgpu_irq_put(adev, ring->fence_drv.irq_src, ring->fence_drv.irq_type); From patchwork Mon May 10 16:36:23 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Andrey Grodzovsky X-Patchwork-Id: 12248689 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-18.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,MSGID_FROM_MTA_HEADER,SPF_HELO_NONE, SPF_PASS,USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 6ECC1C433B4 for ; Mon, 10 May 2021 16:37:21 +0000 (UTC) Received: from gabe.freedesktop.org (gabe.freedesktop.org [131.252.210.177]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 34758611AB for ; Mon, 10 May 2021 16:37:21 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 34758611AB Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=amd.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=dri-devel-bounces@lists.freedesktop.org Received: from gabe.freedesktop.org (localhost [127.0.0.1]) by gabe.freedesktop.org (Postfix) with ESMTP id 7350D6E8C8; Mon, 10 May 2021 16:37:10 +0000 (UTC) Received: from NAM10-MW2-obe.outbound.protection.outlook.com (mail-mw2nam10on2086.outbound.protection.outlook.com [40.107.94.86]) by gabe.freedesktop.org (Postfix) with ESMTPS id 27DDE6E8BE; Mon, 10 May 2021 16:37:09 +0000 (UTC) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=BYMfnsN52o4+zaE7HpWvmwHiYfycXD5b56nyw/U3HNkIzwOcXQxzpXJ4PbIcEML3fyx6I1YTGFIvA9g7KSUE48/I+s13h5FrfPK8lFH/r0f9cDxwezW2cg0o7THnmagioV+tqdShBMX6VfA7tBCHjSNE/zanqhymLS+jw78FA/b9IMQl4ANdg8YQcHt7ZeKAwX5SnBzLuGRk7oW6MhM2YodgzevlAuoLwPWjUwatFjp/oeOpjValBSOoH2ZtAD75S+uVkp01ga/x1wte7BzUGBQFG5qTvYwVF8ehG+EhvnfA5t79g10s4O4A1ZIPLVyL7PqtqSkHQlNKFlk464HX4Q== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=H/uPjWV0fg0dVVvp8px//s3OoPIuYbV9UKVinTXXgyM=; b=S4U2Qo66i9XLZtjXmXqoLiC7yNacRv2/sEHls891C4BLkqUn5f79Tgdhj2wzk8GjZ1xz0zy9RYA0dzdrava5rUPLB5BwxZAvLkN3iZ38wV4Nt3Eg6wliuWDmJOdywVS6JbiEVThjHPBH4QaArQhvaqoO8szxmDnUeocKpUkVC4q36p68qOT9B//aPzpJELJSMdUZzsBmmRLhX5GNpMMAUFHmAGXoxJm+D5aYSOrg04PTQp/2g4YMzd3maxlpjxPrpQ0IzllvsPNoIjdJtjVFLJ4NJZ8YrVKYSktAntXkJfsCQJczkOO6sDaymUrAWg6rimjv0N9TVGXHnTm1DLFmaA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=amd.com; dmarc=pass action=none header.from=amd.com; dkim=pass header.d=amd.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=H/uPjWV0fg0dVVvp8px//s3OoPIuYbV9UKVinTXXgyM=; b=MxKUWsIKIIaK/ZXfaerPAxNtzoeGAN8orCb4LGwFZ8lJ7VwMlVi+7H4e13NA+FS+9stRUZdgydvioheOHwxrXIPQnaPjmL1yQ4b7Z8f3TseaGrAXrMZZcEzTI+lxMbHjeLGibiWDrxgA20CHTI7B4bn7sFTAFDpqBhhdhFTrb/Y= Authentication-Results: lists.freedesktop.org; dkim=none (message not signed) header.d=none; lists.freedesktop.org; dmarc=none action=none header.from=amd.com; Received: from SN6PR12MB4623.namprd12.prod.outlook.com (2603:10b6:805:e9::17) by SN6PR12MB4717.namprd12.prod.outlook.com (2603:10b6:805:e2::31) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4108.24; Mon, 10 May 2021 16:37:07 +0000 Received: from SN6PR12MB4623.namprd12.prod.outlook.com ([fe80::ad51:8c49:b171:856c]) by SN6PR12MB4623.namprd12.prod.outlook.com ([fe80::ad51:8c49:b171:856c%7]) with mapi id 15.20.4108.031; Mon, 10 May 2021 16:37:07 +0000 From: Andrey Grodzovsky To: dri-devel@lists.freedesktop.org, amd-gfx@lists.freedesktop.org, linux-pci@vger.kernel.org, ckoenig.leichtzumerken@gmail.com, daniel.vetter@ffwll.ch, Harry.Wentland@amd.com Subject: [PATCH v6 14/16] drm/scheduler: Fix hang when sched_entity released Date: Mon, 10 May 2021 12:36:23 -0400 Message-Id: <20210510163625.407105-15-andrey.grodzovsky@amd.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210510163625.407105-1-andrey.grodzovsky@amd.com> References: <20210510163625.407105-1-andrey.grodzovsky@amd.com> X-Originating-IP: [2607:fea8:3edf:49b0:6a5:47b8:e610:f6a3] X-ClientProxiedBy: YT2PR01CA0020.CANPRD01.PROD.OUTLOOK.COM (2603:10b6:b01:38::25) To SN6PR12MB4623.namprd12.prod.outlook.com (2603:10b6:805:e9::17) MIME-Version: 1.0 X-MS-Exchange-MessageSentRepresentingType: 1 Received: from agrodzovsky-All-Series.hitronhub.home (2607:fea8:3edf:49b0:6a5:47b8:e610:f6a3) by YT2PR01CA0020.CANPRD01.PROD.OUTLOOK.COM (2603:10b6:b01:38::25) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4108.31 via Frontend Transport; Mon, 10 May 2021 16:37:06 +0000 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 27eed853-578c-4555-9117-08d913d1da04 X-MS-TrafficTypeDiagnostic: SN6PR12MB4717: X-MS-Exchange-Transport-Forked: True X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:663; X-MS-Exchange-SenderADCheck: 1 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: Cj65LdDYZi6Q/yMXUb0SwW15TOKbBTNgSCXO6DmjPbUpdFPwjgQtNVmgFr3EoRuwWCfXV1/pK/4zQ/bUsbzLtXkBu3yje+RmGB29z1LyIjkrJPF1kZjGQHn/I642Z+RkdpHrDhlwB27Bn5IpnzvXVij1hUR4b3oFbaRF2INdCng+KRtGi1TGncHHScebBr9y/FVqEGwKyrCifqxsc1kR7MGR2X4LysTL4lF5XDrp2SOxiNUeNXhm1lLsFvu1GH7Qw4CRfhnx18kszOD+foyBhq6kpQB0QJqlyXPvAzzpOjnaN8uT9UGbZ78pYwxo+uUwjynMKtwGUT70OLzo1hl1nQOqC7VGi98ZzfeLvMdje5jaoZx1+rvA2HN0WTJX5COUbcJvUUwQaO/xELXrgjQt57tGLVhvFEcZ9XADZqbUMVOxJGatc6BUqiA4vvnIYI8AS15b7bSYNLi8jbUkmrnhHSDCnL6rq49+HCrWz7TTtTXPzkAUdKU85v8YHmLuJ0G3CUP04q4Ih1OMnnjXzvtYFl6e7LP167DngEHkeACpoh3uuDki0/YAV9SAOL9SGa4+l4h1S8xoe/uVLHpjHSH6ocHyLb0RPW+7mkyT2JZs/G8= X-Forefront-Antispam-Report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:SN6PR12MB4623.namprd12.prod.outlook.com; PTR:; CAT:NONE; SFS:(4636009)(39860400002)(376002)(346002)(396003)(136003)(366004)(6486002)(44832011)(83380400001)(5660300002)(66574015)(2616005)(8676002)(1076003)(86362001)(6506007)(478600001)(186003)(16526019)(8936002)(6512007)(6666004)(38100700002)(36756003)(52116002)(2906002)(316002)(4326008)(54906003)(66946007)(6636002)(66476007)(66556008); DIR:OUT; SFP:1101; X-MS-Exchange-AntiSpam-MessageData: =?utf-8?q?un9Ia759PO/+RBzPGUNzP7OL8uwjis?= =?utf-8?q?TxM7GpPMGfKK90E9KvSMUmsMVasBNdXnByPMg7PYSbZmCsL0dUpvAzEPFUsFeEO/2?= =?utf-8?q?7QGU2ANerL0XyE7DpNbO2XR5fnIphuFcbVPBw5ViWxB8KvDyEubFhgHshPF5dwMq+?= =?utf-8?q?Zo5zqEFoplQATUexV1q42m1ZgjO5RKzURplCkv0R/sX2TouKWkh6Q2xf76XeQGYWQ?= =?utf-8?q?zChW48i5ASUxfcm5sgnB7DR9jG36ANhdhWt2wrCNPobV+qX4BDSKwO9fUkAxJhBtu?= =?utf-8?q?9wbH3zSG4g9XTrGmxngF6VhzxeeZbjCs0rKDt3sK8C0H4bFNEpeV8Pc6PAeuZX/bX?= =?utf-8?q?d3R1FtC1lSUp18843utVk4Y9fDv2OgxwGTWkZC+jQaVXjw3ZabnVBPxhNZQbk1auY?= =?utf-8?q?xbmh3cUFiY74Qxi3j1vWwflBmyY4eeSu2Yw8hjRVa8pulLVRLxwIhS7ckM7h854VC?= =?utf-8?q?u50r5LMaMo4YkwMbbA+Hwi5a/+GBXZ67FMxNk4JCLdpCAFdBfVErHvc5Er1La49NH?= =?utf-8?q?CZLy9ZGj/APVeEjLptNi7HXUSRViQU4Ic/w2R4BfbiPkLpr+YsxIUcIRoO8xDAsyH?= =?utf-8?q?IRoKCaljTQqyuj2tJuXZXLNLzrEQsIeub0yU/zKLbuUpwPcGlLckWgZuDZO44vc9B?= =?utf-8?q?f6mo4ubFrl+Y5cX8aku9YQUx5hyEVhak8iN8X3E81jv6qfU+Xz+eU/hpkb+epSucq?= =?utf-8?q?gA8+Gd0mBMsj3MO5tnDJaE0VsDf2lYasuV4dH3ZMTqg4zO1uHxbmc0QiqU3mJ0HZw?= =?utf-8?q?eeufSV9rag+BmDI9YwibamKwZBoyw4q8ZzwykSFyH1V20naLXPG9n/khuNuR/FK5P?= =?utf-8?q?og/Arcb9owCjFQIo/4LqQ2HUTF4/HPh9GK6ruCFPhviF8g8o6KFW/FjiMzqHsIR/L?= =?utf-8?q?Y7U7E+jgvhQI7QOqYiwPkXhiRI3yrXmL6UgARB03fOcloJ/zgB6dtbP/o07DGL2JO?= =?utf-8?q?QkYw2N+H1Y9n3alxn5isgIrxDnKuFWqXINmdq9UNqo9iq9PkFtgyZq/8ZsYHodzAF?= =?utf-8?q?Nhz8oBuDEHFxi1CBHuHELNRsRX4uax0MRw9NmKBPyCULMbIdCYOkV9kAl/NVpwuCz?= =?utf-8?q?eL44S0wMElA5IRXnEMQWl2j8cnB9tgFi8g7DqJO8VBaZUChkDqR1QQlnQmj/hI+7I?= =?utf-8?q?UqCt5HJkDGuON34ho89gJs9FMeqPBOgXcloEWH8gLYjciRPY7UtNBCG5x7YDnyYZX?= =?utf-8?q?ohZf4zfU6u99Z7qogFyowtkU6PkMtS5mTFSi7gqAMtEefOYg2nmQSi32/9zSPhrX8?= =?utf-8?q?DmxMd5NtAiv/50pujF31RVTlgC2i/PGGuakHzu0BGgnJefe2X2RuyA7KJK1rL3n2J?= =?utf-8?q?2CpOs3AbBkiCD?= X-OriginatorOrg: amd.com X-MS-Exchange-CrossTenant-Network-Message-Id: 27eed853-578c-4555-9117-08d913d1da04 X-MS-Exchange-CrossTenant-AuthSource: SN6PR12MB4623.namprd12.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 10 May 2021 16:37:07.7780 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: C9+m+liBrIy8ToO8d38g3LfgnXZbJBwV1Jds+wbhPOzg8hV03pkZJ3jPCSLcYsK8r8kXoyp+i2JMygXRqUM+6w== X-MS-Exchange-Transport-CrossTenantHeadersStamped: SN6PR12MB4717 X-BeenThere: dri-devel@lists.freedesktop.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Direct Rendering Infrastructure - Development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: gregkh@linuxfoundation.org, Felix.Kuehling@amd.com, helgaas@kernel.org, Alexander.Deucher@amd.com, =?utf-8?q?Christian_K=C3=B6nig?= Errors-To: dri-devel-bounces@lists.freedesktop.org Sender: "dri-devel" Problem: If scheduler is already stopped by the time sched_entity is released and entity's job_queue not empty I encountred a hang in drm_sched_entity_flush. This is because drm_sched_entity_is_idle never becomes false. Fix: In drm_sched_fini detach all sched_entities from the scheduler's run queues. This will satisfy drm_sched_entity_is_idle. Also wakeup all those processes stuck in sched_entity flushing as the scheduler main thread which wakes them up is stopped by now. v2: Reverse order of drm_sched_rq_remove_entity and marking s_entity as stopped to prevent reinserion back to rq due to race. v3: Drop drm_sched_rq_remove_entity, only modify entity->stopped and check for it in drm_sched_entity_is_idle Signed-off-by: Andrey Grodzovsky Reviewed-by: Christian König --- drivers/gpu/drm/scheduler/sched_entity.c | 3 ++- drivers/gpu/drm/scheduler/sched_main.c | 24 ++++++++++++++++++++++++ 2 files changed, 26 insertions(+), 1 deletion(-) diff --git a/drivers/gpu/drm/scheduler/sched_entity.c b/drivers/gpu/drm/scheduler/sched_entity.c index 0249c7450188..2e93e881b65f 100644 --- a/drivers/gpu/drm/scheduler/sched_entity.c +++ b/drivers/gpu/drm/scheduler/sched_entity.c @@ -116,7 +116,8 @@ static bool drm_sched_entity_is_idle(struct drm_sched_entity *entity) rmb(); /* for list_empty to work without lock */ if (list_empty(&entity->list) || - spsc_queue_count(&entity->job_queue) == 0) + spsc_queue_count(&entity->job_queue) == 0 || + entity->stopped) return true; return false; diff --git a/drivers/gpu/drm/scheduler/sched_main.c b/drivers/gpu/drm/scheduler/sched_main.c index 8d1211e87101..a2a953693b45 100644 --- a/drivers/gpu/drm/scheduler/sched_main.c +++ b/drivers/gpu/drm/scheduler/sched_main.c @@ -898,9 +898,33 @@ EXPORT_SYMBOL(drm_sched_init); */ void drm_sched_fini(struct drm_gpu_scheduler *sched) { + struct drm_sched_entity *s_entity; + int i; + if (sched->thread) kthread_stop(sched->thread); + for (i = DRM_SCHED_PRIORITY_COUNT - 1; i >= DRM_SCHED_PRIORITY_MIN; i--) { + struct drm_sched_rq *rq = &sched->sched_rq[i]; + + if (!rq) + continue; + + spin_lock(&rq->lock); + list_for_each_entry(s_entity, &rq->entities, list) + /* + * Prevents reinsertion and marks job_queue as idle, + * it will removed from rq in drm_sched_entity_fini + * eventually + */ + s_entity->stopped = true; + spin_unlock(&rq->lock); + + } + + /* Wakeup everyone stuck in drm_sched_entity_flush for this scheduler */ + wake_up_all(&sched->job_scheduled); + /* Confirm no work left behind accessing device structures */ cancel_delayed_work_sync(&sched->work_tdr); From patchwork Mon May 10 16:36:24 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrey Grodzovsky X-Patchwork-Id: 12248691 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-18.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,MSGID_FROM_MTA_HEADER,SPF_HELO_NONE, SPF_PASS,USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id EF417C43603 for ; Mon, 10 May 2021 16:37:22 +0000 (UTC) Received: from gabe.freedesktop.org (gabe.freedesktop.org [131.252.210.177]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id C089B6115C for ; Mon, 10 May 2021 16:37:22 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org C089B6115C Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=amd.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=dri-devel-bounces@lists.freedesktop.org Received: from gabe.freedesktop.org (localhost [127.0.0.1]) by gabe.freedesktop.org (Postfix) with ESMTP id 276E86E8C4; Mon, 10 May 2021 16:37:14 +0000 (UTC) Received: from NAM10-MW2-obe.outbound.protection.outlook.com (mail-mw2nam10on2072.outbound.protection.outlook.com [40.107.94.72]) by gabe.freedesktop.org (Postfix) with ESMTPS id 8900E6E8BB; Mon, 10 May 2021 16:37:12 +0000 (UTC) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=MBaDXq0efsnM1gVCKGGCcYZaJ5oz3qXdBWL93JNuKPc3qKG+xDLngokHVt4riZlXyUNKwtjVmXCwHWQ5J0AwMlpvLzLOxYWV4FMTav22BtcpXxiMT1THTOYlaeS2+VnaPPHGnXowVwn9LNwIQo+HX3qhYblyV7WztUet6C1e/cjjn+b4Rd32PukHNiqVqIPBtF/ggEHK75uoddE419SVw7CR+kiGvy2R/Z8TrZvR8XDXuzk7csb0tme6G58tmZmf2HE4IhQ34NudAEo5SUj8x9Nn7H5O4qqoVC4Tm0nX0r2fl1Y+XY/Tk6RP3Ox0rw7S55SAvMCevzOaS4Vbg2C7rA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=WgvjxGBOwqJefZXd6MzvbwldJ32F5m+yZB9fIe0OTKM=; b=J7UdecaNnSuuGPbxz1PNPK52KNGDXlwxyh7M6FTfGL3nnxzfahqKqPnET2c/EYRwhB9zgRElp3dSlJWP6K7X0lNMOCb9F4iRNR18aZYNcldgq1CXyMbnU8UuWIxF3nHsFWV2jtN/7Tq5sSAn4uYQgCm3rWolPE6fZ21Y73qot72NxWEUZwm9h7Tvj+utC2UaDzrM04BZzUq4QNv9Gg55BeEPZKS52RXOU+F8LwWbJLe3R2Xkm1tW4zils6mIA/Wy2R9CelurxF7Zs2mfp7zkU4G9ZTnNG6fL2FaJFCDhYeuGFxc4TkD4/8THsdoDDCCf7UIwPkOY7aV87nLDRskkWQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=amd.com; dmarc=pass action=none header.from=amd.com; dkim=pass header.d=amd.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=WgvjxGBOwqJefZXd6MzvbwldJ32F5m+yZB9fIe0OTKM=; b=Gm9rFbvcSIJ+6OfngNJVmUvEQwZ44FDzUn7d8ZmkTsmjabAlQJTRR8Knub0Nlf02JT6JKCzkVnf1n8b/8GkrFX/TTdckgQnqwD2D8ynbMIkpDKSOWRCNRI1vFxJwmLFqZdzedl4UXYxWKa4gRzJXdqZI9rbYn3A28qxjY2Paq7o= Authentication-Results: lists.freedesktop.org; dkim=none (message not signed) header.d=none; lists.freedesktop.org; dmarc=none action=none header.from=amd.com; Received: from SN6PR12MB4623.namprd12.prod.outlook.com (2603:10b6:805:e9::17) by SN6PR12MB4717.namprd12.prod.outlook.com (2603:10b6:805:e2::31) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4108.24; Mon, 10 May 2021 16:37:11 +0000 Received: from SN6PR12MB4623.namprd12.prod.outlook.com ([fe80::ad51:8c49:b171:856c]) by SN6PR12MB4623.namprd12.prod.outlook.com ([fe80::ad51:8c49:b171:856c%7]) with mapi id 15.20.4108.031; Mon, 10 May 2021 16:37:11 +0000 From: Andrey Grodzovsky To: dri-devel@lists.freedesktop.org, amd-gfx@lists.freedesktop.org, linux-pci@vger.kernel.org, ckoenig.leichtzumerken@gmail.com, daniel.vetter@ffwll.ch, Harry.Wentland@amd.com Subject: [PATCH v6 15/16] drm/amd/display: Remove superflous drm_mode_config_cleanup Date: Mon, 10 May 2021 12:36:24 -0400 Message-Id: <20210510163625.407105-16-andrey.grodzovsky@amd.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210510163625.407105-1-andrey.grodzovsky@amd.com> References: <20210510163625.407105-1-andrey.grodzovsky@amd.com> X-Originating-IP: [2607:fea8:3edf:49b0:6a5:47b8:e610:f6a3] X-ClientProxiedBy: YT2PR01CA0020.CANPRD01.PROD.OUTLOOK.COM (2603:10b6:b01:38::25) To SN6PR12MB4623.namprd12.prod.outlook.com (2603:10b6:805:e9::17) MIME-Version: 1.0 X-MS-Exchange-MessageSentRepresentingType: 1 Received: from agrodzovsky-All-Series.hitronhub.home (2607:fea8:3edf:49b0:6a5:47b8:e610:f6a3) by YT2PR01CA0020.CANPRD01.PROD.OUTLOOK.COM (2603:10b6:b01:38::25) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4108.31 via Frontend Transport; Mon, 10 May 2021 16:37:10 +0000 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: ebd2fb5b-3e59-4d37-54f5-08d913d1dbf6 X-MS-TrafficTypeDiagnostic: SN6PR12MB4717: X-MS-Exchange-Transport-Forked: True X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:25; X-MS-Exchange-SenderADCheck: 1 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: ngs82JYkOa6ip7PcLESJB8O3CNCZ3v6TWboyLg1cclrNe/IPTEUSo/soVW9Ub9Zv4PaWQ5m0Jqec8CIOjAAwJC1iSstc26XRftruqEwKZcvOsq9QCaec8LUtpbAOm3Ytx6ZWPQZBQ6GPrhPQEWT5svaQpyoAOo4zOxikWjLraEVboG8Q+tmIKizG/VKBFPVvk+Av1UVr8sTteUinV1F/KFUEknQ5zE52odvBIaZaCkUITwfuHAVroJ4UHklH1LaxrZ0NvutSzRYG4xhwwdYZm150N2g08wGnnxs0QNucl85ECjpb9DFct1Cq3LY/jw63DaakKLPm0K9OCTXxJoW1I6YhW1fAgeA0xA+SwJSl5MNSYBHRtCbBpm5PdQExv8xnrmZ2lORdcTltE2QPwPTg1xJHX3zBiTppHrZVNm/l593PIs4bclWOxXHxrGpDLY1Fnfyb1ijO/cVUaD7dLc8K2RsSQ/75s2JFCNNGpYxHn9+uyWKf2DaWXnh/W/nhnqxEOTEPd+2XGN/i/4OjDOQjdz37xCDH5VItoxbDjxU4tHt4RpOtKiHp/IFIrERHbJCQuaULJMUzE7gstG4Qw6cBp+iNXQ30h2+yYUTehhAMafQ= X-Forefront-Antispam-Report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:SN6PR12MB4623.namprd12.prod.outlook.com; PTR:; CAT:NONE; SFS:(4636009)(39860400002)(376002)(346002)(396003)(136003)(366004)(6486002)(44832011)(83380400001)(5660300002)(2616005)(8676002)(1076003)(86362001)(6506007)(478600001)(186003)(16526019)(8936002)(6512007)(6666004)(38100700002)(36756003)(4744005)(52116002)(2906002)(316002)(4326008)(66946007)(6636002)(66476007)(66556008); DIR:OUT; SFP:1101; X-MS-Exchange-AntiSpam-MessageData: Hz0EU/4KhscPqZxZ0oI6rJ3WHHTO4mEwFm6Mje9wg1WBvOqZ7EHBx2TfAgqe/c+BBVIzqDPuVRF4anIPrINQfEyNXDDtZYWMbDRaUjgc170gJiWYY5XNAeTJzYKmV5xVMDnDxd+vEboabhei27GiMu47u4AZgFap5Zhz+11dgyq/QdRPVDTNNTEAdIKHmlrbon0rQD6THaUrx7AmI2ONE2e1p0M10Mm3br22akE7yDwWal+vurluMp1Y6Z0F6M7pahF0mZYCozhFceJpaxQRV0H8dGlHVMsRV5ok7JE32hNh6UdurtZhSV2WTpUg6avLEFFTw6twmOper02If2EK7Oslu2yBJwWNgwbtYdJDdUT2KnnaTiiQZ5QwMPJ8Hs0UcQxeg22ACKPSEYmNMGR7dXqSTK98kWvC5EmKLFxW7yKHSJFjbP/M1HHUSci9BUC78rwsRgj5Uqf7vwLy/30OwSOoU/xCVUkMwuWrj95mBdjvYYUA1w0TCeTa0Pch87ONQlU6ST/EBoLaPEOyF5L573qY/JPx3D8gx7NSCQ8an3e2DuyltZ8YMoOC204rhl3moGSIOx+w4jULpx1765w0KpEb9n27hQ/Y/HwfRGvexBdnNYOKRSCAKm3f/lpA61enWcA9hEqIlgg/w7qeG+I3DcHKUC3PTNwFQFYtEC2i5PCugBrdZp1fezKMcT8FVsWGUVqlHLveUuqM2u9TIm8Axb5zgNiGFU2PBMLBlBkDhz+TTpEfUrXNmqhzRpqOTigm5E5zK++TDWg8/o2zhIdsFndjdm5p21LGfnSOwWOCmWkMCkRPH4RlmnjyHezf4SmTy78tQNy3dQJH3lfcJhVNBD+XqxHdc6Bp3cj+02LSiF0BgwFv1UC4LZc2RyisfIDd12vC+QYNuSacoesd7KSaTpYcF8nAT7bZdoxr4iu7QwG3Y1OLtghucQYoJiLcAFvjGm4++eEzac4rWw+BjYrgLetGeuNBPAe45rSS0e/QubMrQ5THyq9cIDQx3jYFaZfEDcuB7kU5ZtEBHp8TFcDCFN+mzrg+iCWOViAm07kNYsG91RT37jsU1DNsKVFMRvNO4oeCKTBkmE7zgAH2cxgmgCWo2y/P5gcG1q9B4AQOFLEH+zzDQJy4M+Mt1Mb6oNP59JJCIABuTN92Jv35l939I3BaItGqEZLB00PqP4R+vQfR+PaRKUBMMdX9ZxcEc29zOofDuuWDYQZf0EcbOEgk2eqzysH792PDREwMpyZJbnwuTWSU0/Wc9w0ffEkIxH6VQMwRB/l//Jki0UfZvOVLky24PZbt2ZEIVzy+A/OyvvqwoHTYIQkO+bD0oFmwErtcfp3AlChiUAVWRcZxnnHeFdpbZvj59WNGACp9C0UVIBaY9DZBkQZ4EjVD31PE7UrW X-OriginatorOrg: amd.com X-MS-Exchange-CrossTenant-Network-Message-Id: ebd2fb5b-3e59-4d37-54f5-08d913d1dbf6 X-MS-Exchange-CrossTenant-AuthSource: SN6PR12MB4623.namprd12.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 10 May 2021 16:37:11.1201 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: FUFmkpj812liLPH7VEVWGeCKCHZHMv2LgonB+Q7ysk3dRW2BXHeci9efPCE2zU05bOfOUSunOjCJerhP6IuHYg== X-MS-Exchange-Transport-CrossTenantHeadersStamped: SN6PR12MB4717 X-BeenThere: dri-devel@lists.freedesktop.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Direct Rendering Infrastructure - Development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: gregkh@linuxfoundation.org, Felix.Kuehling@amd.com, helgaas@kernel.org, Alexander.Deucher@amd.com Errors-To: dri-devel-bounces@lists.freedesktop.org Sender: "dri-devel" It's already being released by DRM core through devm Signed-off-by: Andrey Grodzovsky Reviewed-by: Rodrigo Siqueira --- drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm.c | 1 - 1 file changed, 1 deletion(-) diff --git a/drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm.c b/drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm.c index 6c2c6a51ce6c..9728a0158bcb 100644 --- a/drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm.c +++ b/drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm.c @@ -3757,7 +3757,6 @@ static int amdgpu_dm_initialize_drm_device(struct amdgpu_device *adev) static void amdgpu_dm_destroy_drm_device(struct amdgpu_display_manager *dm) { - drm_mode_config_cleanup(dm->ddev); drm_atomic_private_obj_fini(&dm->atomic_obj); return; } From patchwork Mon May 10 16:36:25 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Andrey Grodzovsky X-Patchwork-Id: 12248693 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-18.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,MSGID_FROM_MTA_HEADER,SPF_HELO_NONE, SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 60DBCC43611 for ; Mon, 10 May 2021 16:37:24 +0000 (UTC) Received: from gabe.freedesktop.org (gabe.freedesktop.org [131.252.210.177]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 314D5611AB for ; Mon, 10 May 2021 16:37:24 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 314D5611AB Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=amd.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=dri-devel-bounces@lists.freedesktop.org Received: from gabe.freedesktop.org (localhost [127.0.0.1]) by gabe.freedesktop.org (Postfix) with ESMTP id 307E96E8C7; Mon, 10 May 2021 16:37:16 +0000 (UTC) Received: from NAM04-MW2-obe.outbound.protection.outlook.com (mail-mw2nam08on2083.outbound.protection.outlook.com [40.107.101.83]) by gabe.freedesktop.org (Postfix) with ESMTPS id 88F7F6E8C5; Mon, 10 May 2021 16:37:14 +0000 (UTC) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=ObCBA/iEVVokAklFfgiqo5mEDQqpuCAWZ8mKNVwZpGaFN/g8xY6aUZW+m/s9gyvJ9Jk7rfA/J+rB7Xikd7OLzWA6OETHanJrnAnxOkRbriZOd14Kl5ql29ByGXIyQNOQuSWTNbDDPtwJbwuZHNd1vOmbo0NRKQYqZNGMfDsmgT/NHrRJ819cULumkUSCzHrNSO4LlAQBFVUT3URxfIVwukYGhiH17BBXmjdZGQ0oNZ/olCYTHK2jOIYQLo1joJsOBaFdePhGykpbQ48H2TxGRID9yAGjDlRjrdAtI8IZTxF9OV5IUwBaGNglG3UWo/oPZ2jbdRHfECvgEItdktIxEw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=TW/TKpFRu1X4rRTXhKdo8M0et9MXFShEHfYdvlK6R68=; b=Hj1bOzu3i9NlOs5NAyRV71Aa3fwe7eKZ6YJu/9rQZd8FbZNMnrRfecoh9O9ZvrHWd0kkeBp9U8dY7iKvjXzdhxgmz04+9Xqcx3Xi1XnMt0z7x60VyXBOpGiVxwEk5sbQr/x6MTJUAowC0AO9NqnsF2zwzHwJS+DyIZnmrRylmWB4Zm8eZgGmraOkwnczfLPV1ETnG+i3ONN508fi5MNk3QUpUu2pxE2PN5fi1owE5TmzocVP6YY4hGaZylWzwRj7knWi2Qb4HInKyoMSSNyFI18F8NSIRQqsfmBinHztMy3lw7FwF+0TssVJkSdJeaPp+aGgcDzetSQo6GydpWZRVA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=amd.com; dmarc=pass action=none header.from=amd.com; dkim=pass header.d=amd.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=TW/TKpFRu1X4rRTXhKdo8M0et9MXFShEHfYdvlK6R68=; b=EOriBtda1zGRrKxlaTjyVau2EXODh1ZYQUJguN0aQK7K3eoo/qNVOQjRvPJMJbUuubR/JZPbJXe8rJJLho72Oam4kRrGhT54lMTjtlhPxaFELSuPrB6WRM36/u4QhDur/nV+NIH2MzwEXhb/oIKWdaIt0a++Gt5rB5z5P1zE1yA= Authentication-Results: lists.freedesktop.org; dkim=none (message not signed) header.d=none; lists.freedesktop.org; dmarc=none action=none header.from=amd.com; Received: from SN6PR12MB4623.namprd12.prod.outlook.com (2603:10b6:805:e9::17) by SN6PR12MB4717.namprd12.prod.outlook.com (2603:10b6:805:e2::31) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4108.24; Mon, 10 May 2021 16:37:13 +0000 Received: from SN6PR12MB4623.namprd12.prod.outlook.com ([fe80::ad51:8c49:b171:856c]) by SN6PR12MB4623.namprd12.prod.outlook.com ([fe80::ad51:8c49:b171:856c%7]) with mapi id 15.20.4108.031; Mon, 10 May 2021 16:37:13 +0000 From: Andrey Grodzovsky To: dri-devel@lists.freedesktop.org, amd-gfx@lists.freedesktop.org, linux-pci@vger.kernel.org, ckoenig.leichtzumerken@gmail.com, daniel.vetter@ffwll.ch, Harry.Wentland@amd.com Subject: [PATCH v6 16/16] drm/amdgpu: Verify DMA opearations from device are done Date: Mon, 10 May 2021 12:36:25 -0400 Message-Id: <20210510163625.407105-17-andrey.grodzovsky@amd.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210510163625.407105-1-andrey.grodzovsky@amd.com> References: <20210510163625.407105-1-andrey.grodzovsky@amd.com> X-Originating-IP: [2607:fea8:3edf:49b0:6a5:47b8:e610:f6a3] X-ClientProxiedBy: YT2PR01CA0020.CANPRD01.PROD.OUTLOOK.COM (2603:10b6:b01:38::25) To SN6PR12MB4623.namprd12.prod.outlook.com (2603:10b6:805:e9::17) MIME-Version: 1.0 X-MS-Exchange-MessageSentRepresentingType: 1 Received: from agrodzovsky-All-Series.hitronhub.home (2607:fea8:3edf:49b0:6a5:47b8:e610:f6a3) by YT2PR01CA0020.CANPRD01.PROD.OUTLOOK.COM (2603:10b6:b01:38::25) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4108.31 via Frontend Transport; Mon, 10 May 2021 16:37:12 +0000 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 511bbdbc-0f0e-4b6b-2529-08d913d1dd34 X-MS-TrafficTypeDiagnostic: SN6PR12MB4717: X-MS-Exchange-Transport-Forked: True X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:5516; X-MS-Exchange-SenderADCheck: 1 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: /L9CmCspkysqpaQBheakBvPIM26S+qtk+vh0d+N+6DbkQtKNIsDCxUJN3YilM3t+nbBa6c3w40Kh7kSknxhr/Dwkjo55D0Gzv2jzwu/ANJMcq1r35vNOcYEUBD9Uz3j4P3c6sSBHXGim2fBPCk2mQlT/satTSL4YAjP45kIjKcEVOoxbZrtWz8Y/TYYzkh7XqPeNvMRrOolg7VbY7Ikt+PJACuNYiIaUrtS+RzzRdJcXMU04BK8H/+f/ysWft9H3H6FjhJz+zXXcTchEo7e2ys5loLDq695grqluW5uzBixtEg+xAbZgFpm8dGIYVHO1b8QgpkWqA5wshFR/vVTJCAlXipF6U/5TiqQXzbR3T6CC5dhIe3lDYg6WNT35NgBlKDvhs5RXRwKMx7Du/BTwIdu9mN/aC1MVTMRXuvdM/zT5Oda6wG6bDkmv7rC9rjNNm3ai462E39j9Vu52+3ByhUTpR5CUrCFKUybhGvM2/sfi+t2ZnGsOds5KZotivLSvD3jMXHJFySUtMh4ggA/FvgZ0mUN9ts+Zah1BiaweIWyoEzEQOo6FOMOZ5aBp1+kaXFdUMcJ8/MZVIBWTCURF2LFceSt5qUrrHyKajwOh7DU= X-Forefront-Antispam-Report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:SN6PR12MB4623.namprd12.prod.outlook.com; PTR:; CAT:NONE; SFS:(4636009)(39860400002)(376002)(346002)(396003)(136003)(366004)(6486002)(44832011)(83380400001)(5660300002)(2616005)(8676002)(1076003)(86362001)(6506007)(478600001)(186003)(16526019)(8936002)(6512007)(6666004)(38100700002)(36756003)(4744005)(15650500001)(52116002)(2906002)(316002)(4326008)(66946007)(6636002)(66476007)(66556008); DIR:OUT; SFP:1101; X-MS-Exchange-AntiSpam-MessageData: ENnE1pL4eiFcR5j57zgoDX9ao6yk8PB+wJbFBSAR1ve1rab4skd3QnSBRlcwIFPUMqLw2uPMFCz+XfXPyfHYeUCBX46kb9ZLb27KsSoPirSFdMio/d4fQ5JR7dIZjsRVOxYr+QaiK3OC3d43msyndGONX0icAOel0NM0faIUHjtylRNQZjwILv4LI7mfaWCnUKtUQRYSflV1GpRECxCHLUT+DzTzXFe6s/aR5ek3uOv3d6euvjvrAXMOLpXLfWYvC+mQjp3keNBtLFoCK9ihRBhcmynKkxxc5v70sHaByVEV71gqvq8U6Wq1Q8L9rdo3b3g4XcrFX9ZLZuschnt4fbJLGh4/blFrwfB3qgT8gtKKQMUFuvUz304+TD5hTlAnepLNgagXC66Sm+TcQIXtQa7AafQ4jE8UuAoeZqyWrppqvG5Kw2nqPYgN/r/Ux98BlKxZImhn6MUuo+rEK3IugsFK9aYWeVKi9VJIDFU0yhPvXJXpd3yikZV/ph2AnrPN6eDeRhn6KoNzTvoX00aKKTuj4w5nUZegWorDm1+RoMEHHTPM5M3w9BU4kgCsiwzuIXrPjQXSoCzOiqEQvaJXcoeVZ/rssoS6Hcg8SFuAlqa5I/uVWj2T2lC3yH0eBJnGdgQd2lbZhkqOfB/G7AJjVrogD+WFF4hVZiyBPckCb+4FU/7yW3Y4kX5YawYjq3B6IAffdNkZYz0XY3sxqqlLiLTMpFuXOSwGfCthDJ4FoGcBGQnvAMDD4NNaFFlZgrO86+E7qrpW+LofPUX/JPVhfleeUN2es6FvklAaybVDx6FMgG7zLPFK4QivpwMfEK3sJmgOa+rVVoK+JXREBZz2MVJuEfG2ffqHLhAO/mTzpDUEi6rnxmP1aJMcd+18Cakjz/l17dJZySDBze+ExBRPDYzNQgtGY6jkqd2XJNUawGu1ZIBJnzgNC1A1DGnjATGqquBtjPF/w5VWSomz3xek4Tb7EbWfwKC5FY0wNekrxAK4v1hatgKcoxDeuR38ZHtbulSxP6vMK14E28CoRORmG77x+HHOv/+6DrvL86qbHvK7+8hTfZFA6+Sk1KAQMT26IVahdFTzzaj/7Nuiyt1TmatjoyIimmKaphGs1H6LsdODCTaBhhDuIRB/OAMh2PeYDfLr9xBc2SmsOYxNqUlm69EoNSybfxFteQd0C+PT3SJI/U028DArIjlDKXF79I3mC6QgGpSOUl+JMzeMDPr3XOGWRQvnJpRy5vdVTZYoP0qbnfCdJheyKhRMmhVZDEHPm2JrisqN9hOEDs0vFFy207EMKVT/yKpuXmpMKzNFUaBLAfGH573YtIdEqeSDz+WDPS9Jf5zwECV3vEpdJmOX3YD9eJqozYPmFzH/90HpSVNq8ZtJKKHFX4YBEIo1ek5s X-OriginatorOrg: amd.com X-MS-Exchange-CrossTenant-Network-Message-Id: 511bbdbc-0f0e-4b6b-2529-08d913d1dd34 X-MS-Exchange-CrossTenant-AuthSource: SN6PR12MB4623.namprd12.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 10 May 2021 16:37:13.2089 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: 8pDQbXBjLz/JM3DU7CAZO4gsYsS9X8xJkbIFcM5o06SMbK1ULnUTyWUuFy80rk83tMc+smPDQh7ganAC4hI9gQ== X-MS-Exchange-Transport-CrossTenantHeadersStamped: SN6PR12MB4717 X-BeenThere: dri-devel@lists.freedesktop.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Direct Rendering Infrastructure - Development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: gregkh@linuxfoundation.org, Felix.Kuehling@amd.com, helgaas@kernel.org, Alexander.Deucher@amd.com Errors-To: dri-devel-bounces@lists.freedesktop.org Sender: "dri-devel" In case device remove is just simualted by sysfs then verify device doesn't keep doing DMA to the released memory after pci_remove is done. Signed-off-by: Andrey Grodzovsky Acked-by: Christian König --- drivers/gpu/drm/amd/amdgpu/amdgpu_drv.c | 6 ++++++ 1 file changed, 6 insertions(+) diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_drv.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_drv.c index 83006f45b10b..5e6af9e0b7bf 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_drv.c +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_drv.c @@ -1314,7 +1314,13 @@ amdgpu_pci_remove(struct pci_dev *pdev) drm_dev_unplug(dev); amdgpu_driver_unload_kms(dev); + /* + * Flush any in flight DMA operations from device. + * Clear the Bus Master Enable bit and then wait on the PCIe Device + * StatusTransactions Pending bit. + */ pci_disable_device(pdev); + pci_wait_for_pending_transaction(pdev); } static void