From patchwork Wed May 12 14:26:33 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Andrey Grodzovsky X-Patchwork-Id: 12253869 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-18.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,MSGID_FROM_MTA_HEADER,SPF_HELO_NONE, SPF_PASS,USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 0E567C43461 for ; Wed, 12 May 2021 14:27:17 +0000 (UTC) Received: from gabe.freedesktop.org (gabe.freedesktop.org [131.252.210.177]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 930A961407 for ; Wed, 12 May 2021 14:27:16 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 930A961407 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=amd.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=dri-devel-bounces@lists.freedesktop.org Received: from gabe.freedesktop.org (localhost [127.0.0.1]) by gabe.freedesktop.org (Postfix) with ESMTP id 08809899F0; Wed, 12 May 2021 14:27:12 +0000 (UTC) Received: from NAM04-DM6-obe.outbound.protection.outlook.com (mail-dm6nam08on2040.outbound.protection.outlook.com [40.107.102.40]) by gabe.freedesktop.org (Postfix) with ESMTPS id BBBBA6E079; Wed, 12 May 2021 14:27:08 +0000 (UTC) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=jS5RAF8o5mrHr9N1fu1aZIRQlgsHFjuMpUIBEpCrTafHH6nTO8iNwazNVupEyg+Z79D/+vSIsGu1m3r89fX5I+IhbXULVTQsE8yrYarkAxW+1TubW0UWEKVGjd0vCSk0x8/vwne6mIBAgjx5Bre4g34nZZMXl3SCN7m/A8EBNTCGj1rKQxHEbstf8M9Ad3DlIsyDmL7I6eaAkxxvLyz49g1hMBkjVZRV/n84qQdene3obYb18Z+X/gezj4Dsf9Ocjha/tixq1AJEBlhSxvtjlgz+WxH1wKqpxSTPXaXJtZZBeIGqQgQK1IkiQGYqvfGb0lDNLUmxBUJMEFOLCaS5zA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=j8CHomDLolHWYr6vXq3Rl7552WLkWchm5KRWiP0lHhI=; b=UnSgnzomTf/Uvl8LCWJAF8oEo6eEBA2sO2x7SFtS+1vI38X8KndCis4hf+v2rz13xOEuArVcKgmGjyDg7tzpvYROY48YaJrsr1TvqcGdHlj0DHP0Ol8OYKIBhWaFXj+uHr3OOWuAX7BOqPX30Ar3J4Zh96GUUSyR6cbB9JH76PCp4Xz5pG8B04he4T/9t7NWzxZ1g3Vg6ruhQEtJI2JJtyAAUC1N7q7aKYmIQ8mV4VjQvBX2Ox/uGerxuwYaDpVySADtxMYKEzD5Jfbhl8EzAisehQqPxEt2uwHU7Pfcal9xQoVndpsMf+9l50cfyz9FZPTHrfFlLwhh4MNp37ksHA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=amd.com; dmarc=pass action=none header.from=amd.com; dkim=pass header.d=amd.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=j8CHomDLolHWYr6vXq3Rl7552WLkWchm5KRWiP0lHhI=; b=MElGLQMI/LtmlnN1NF/t0kC4QD2NtiXxqxBhhdrWOULFfbiK/fsH5vywCJ685aaQBjt0ddVTOZunL924TRgNF+B1JIKxnwfAAp8Ax8pB4f67XlfCPp3P96c9nupCsSkrFzmVvbK6Jmii7DgxrgSBhb2h7Tf6DW4vqTPAqLFgGEI= Authentication-Results: lists.freedesktop.org; dkim=none (message not signed) header.d=none; lists.freedesktop.org; dmarc=none action=none header.from=amd.com; Received: from SN6PR12MB4623.namprd12.prod.outlook.com (2603:10b6:805:e9::17) by SA0PR12MB4576.namprd12.prod.outlook.com (2603:10b6:806:93::13) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4108.27; Wed, 12 May 2021 14:27:07 +0000 Received: from SN6PR12MB4623.namprd12.prod.outlook.com ([fe80::ad51:8c49:b171:856c]) by SN6PR12MB4623.namprd12.prod.outlook.com ([fe80::ad51:8c49:b171:856c%7]) with mapi id 15.20.4129.026; Wed, 12 May 2021 14:27:07 +0000 From: Andrey Grodzovsky To: dri-devel@lists.freedesktop.org, amd-gfx@lists.freedesktop.org, linux-pci@vger.kernel.org, ckoenig.leichtzumerken@gmail.com, daniel.vetter@ffwll.ch, Harry.Wentland@amd.com Subject: [PATCH v7 01/16] drm/ttm: Remap all page faults to per process dummy page. Date: Wed, 12 May 2021 10:26:33 -0400 Message-Id: <20210512142648.666476-2-andrey.grodzovsky@amd.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210512142648.666476-1-andrey.grodzovsky@amd.com> References: <20210512142648.666476-1-andrey.grodzovsky@amd.com> X-Originating-IP: [2607:fea8:3edf:49b0:7576:4f76:97d8:1487] X-ClientProxiedBy: YTOPR0101CA0032.CANPRD01.PROD.OUTLOOK.COM (2603:10b6:b00:15::45) To SN6PR12MB4623.namprd12.prod.outlook.com (2603:10b6:805:e9::17) MIME-Version: 1.0 X-MS-Exchange-MessageSentRepresentingType: 1 Received: from agrodzovsky-All-Series.hitronhub.home (2607:fea8:3edf:49b0:7576:4f76:97d8:1487) by YTOPR0101CA0032.CANPRD01.PROD.OUTLOOK.COM (2603:10b6:b00:15::45) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4108.30 via Frontend Transport; Wed, 12 May 2021 14:27:06 +0000 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 975d7753-45e7-4525-b5b4-08d915520562 X-MS-TrafficTypeDiagnostic: SA0PR12MB4576: X-MS-Exchange-Transport-Forked: True X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:6790; X-MS-Exchange-SenderADCheck: 1 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: etRnEnff2z2ZPq+iDnCEn6lAqBYiED72ykSUMSeOX/aYUCJF4AXk1i7ykQgIsxkTepOwTquWwDGKf+7gqXiiAmZFoh49JBzhWijeuHaYXs9j5M9Pdw8gE6pO/Vfi6JOIvRg4JlR7xOJYo21sighKrL9ljmXzzdaPV8RTV8O+nHFNqPwSQZCjc707IhmJ5T/nuRYKQ6xcpUBjLTl8RFOYqQdwbQAw2PWRF1Kapok7nkTydnEqN1Sia66E58Or1L26RD+Wvz2c2Enbe6/hmqtesE0dSeWSFO+9xYMS5X1W0B/olq/7nCsFiP+yEKg6ZdMlUancXt5HV8oNKKgi7V/g5bieXor0dO94Nt1NMCs46ilMNak0U2D0hajIf8hGl2lU9G5jIjhxSjMvqY3Z3o1r9jF7WPK1qMnLZULUQkhOPeWUz3Mvslrjb5k6ZmWb73Q6BxqXpgObxJOjyPUyD6vyKCHDZ23CLSTvb6azTT12DzCCzIcddZGMgWrEWYfCfIPHMpxnRcBQNp8q3h17H+gkE5A5E9eqcpEb1nBccJwtWHOWETJzwhc1xobe7lRJbu62JehEKdbBieOlnN4v8hgQuDf7qCR8Fs99MWQT6m2A0NQ= X-Forefront-Antispam-Report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:SN6PR12MB4623.namprd12.prod.outlook.com; PTR:; CAT:NONE; SFS:(4636009)(376002)(39860400002)(366004)(346002)(396003)(136003)(66574015)(38100700002)(5660300002)(1076003)(6666004)(186003)(316002)(16526019)(8676002)(83380400001)(66476007)(2906002)(6636002)(86362001)(52116002)(36756003)(54906003)(2616005)(66946007)(66556008)(8936002)(6512007)(6506007)(44832011)(478600001)(4326008)(6486002); DIR:OUT; SFP:1101; X-MS-Exchange-AntiSpam-MessageData: =?utf-8?q?U7/ZKW7qf2RG1f0yVNNQoi/4/CRvGF?= =?utf-8?q?FHOvUIiTVyktrV69a1DUbPTtZf/iLNL1CqhvrnZjhUix7jnkjrTM3Lh+WuzhhXuAr?= =?utf-8?q?RjzIQojoT+jQSBiVW8J1Q+/7KPQJlNkoFrsaZ+MBv72wUZpRCa/nNN6app+abWdao?= =?utf-8?q?9P6o+X3VVa4wZFH6fL06LOIKg/EzIe+g5NFwlngODnlK2q2GTUWl4KA1o0VnB/rYe?= =?utf-8?q?xJkvwqzoRkauGT9iu53Bfxh81GTP6xzCcsC871U3WcVpwmfX1ILS1wmAGk8LvKKpy?= =?utf-8?q?EhJcYJarEideUk2cZA9WPpxU56VTeXwzD6oHnukBsZTStVN5W7eIOdJeq2Rzwpwzj?= =?utf-8?q?kJFyUQgjZOH1uqCItGukVcBs1oZBr2wDjht5TZRWtdx23eyd+jqNh6kvXUtaH9MsE?= =?utf-8?q?0YhwZPAJUITkTKRmTxhoPTN0T6FAwGMXAeFcRNFWk1VD41xG+VtGD6KJ+JDil1v+w?= =?utf-8?q?SlBQZFhpCWqceYfHoKVGBQHsHcwWOqPRcNHzcB6+9Im1dfCjB0fRXeDLbbhFxyL8S?= =?utf-8?q?LJv8rnHOtUfT2IcVtq0CQLivE9L0GMpJXsYkPBEDf2d10kEK9KkZXak7O0Y6YDPlR?= =?utf-8?q?KSJ9P5wTqJ1HuLIc0pzfSRJksKQjTR0EA0QvFOFB1LgtmB+Fzu46Yv+y57suOu33A?= =?utf-8?q?N/JYy4S8VFga2+FDc2CYDhIHMXZz0G1932RrsmBVqta8jqKM6e+8SW+M5KVSnRkTw?= =?utf-8?q?3X6ouWnyqpW9yBhOECnA/fuz2s9GJ0ed1OGMwx3VoQAC/Yj7mQndcMBjg9at/owHL?= =?utf-8?q?m4gc0XeUbvRzh5vnGQTfBITXUcYJxoAjNFv+oUMbkbknVM4wGvFgUgVBnTqP6c89X?= =?utf-8?q?XpP3VgpqzZKHua2lKtWf7LttSvzwk1uvHym6wJzqpzgmtA0KErfB4sEEjs+uYysbq?= =?utf-8?q?i41dOM3U7eTHmB9JpegEpTRbih6WuptAYyr+Nq16CBfALTbrQNKlFZxwDGmCgxHHu?= =?utf-8?q?QKbwUhVsPP6W3I8dZ7BZ3nCwqc/DSVisW9MFvIKBZYVLXjmqtPBp/+xP3+Ad+8F/g?= =?utf-8?q?bHdztOKvFqci4phHnXOfaqHX5vz4aT44zkiHzvvrGWKaeCutcIv/RLXioikCJqno9?= =?utf-8?q?Ah/L5gxMCh81CnuQym8QXcprroIglZ1jvC/uF2fh6XX59UDbSPrp+/k4iXm7kc6nF?= =?utf-8?q?3L6yv/22JmSG/FyBp54oqsurM0W0LdezD4aD6HtFOrSHQwuwgZQeQA9bFE6vstEkI?= =?utf-8?q?I78OhPCmQ2Ean0fD4KYQh7LLIhPWPGSJM1/ST7ZPJ7wtisckfd9JOjA13ITm+xzBO?= =?utf-8?q?sS9sQcSZ6luQESiG1N17urowCIwxQ+Ajum9+gb3ACpN6aK3FmazcLXaipurDwz/cc?= =?utf-8?q?Wxc41HIjO5NKi?= X-OriginatorOrg: amd.com X-MS-Exchange-CrossTenant-Network-Message-Id: 975d7753-45e7-4525-b5b4-08d915520562 X-MS-Exchange-CrossTenant-AuthSource: SN6PR12MB4623.namprd12.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 12 May 2021 14:27:07.3141 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: CdMMo6DCYooIsq5NApc7PlP9izUHeDiqnCKOpVX2DIKb/wqPVcJw6nG6c0ALf9FFAfw0mi0nZ+dKFtSvgDfsrw== X-MS-Exchange-Transport-CrossTenantHeadersStamped: SA0PR12MB4576 X-BeenThere: dri-devel@lists.freedesktop.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Direct Rendering Infrastructure - Development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: gregkh@linuxfoundation.org, Felix.Kuehling@amd.com, helgaas@kernel.org, Alexander.Deucher@amd.com, =?utf-8?q?Christian_K=C3=B6nig?= Errors-To: dri-devel-bounces@lists.freedesktop.org Sender: "dri-devel" On device removal reroute all CPU mappings to dummy page. v3: Remove loop to find DRM file and instead access it by vma->vm_file->private_data. Move dummy page installation into a separate function. v4: Map the entire BOs VA space into on demand allocated dummy page on the first fault for that BO. v5: Remove duplicate return. v6: Polish ttm_bo_vm_dummy_page, remove superfluous code. Signed-off-by: Andrey Grodzovsky Reviewed-by: Christian König --- drivers/gpu/drm/ttm/ttm_bo_vm.c | 54 ++++++++++++++++++++++++++++++++- include/drm/ttm/ttm_bo_api.h | 2 ++ 2 files changed, 55 insertions(+), 1 deletion(-) diff --git a/drivers/gpu/drm/ttm/ttm_bo_vm.c b/drivers/gpu/drm/ttm/ttm_bo_vm.c index b31b18058965..7ff9fd551357 100644 --- a/drivers/gpu/drm/ttm/ttm_bo_vm.c +++ b/drivers/gpu/drm/ttm/ttm_bo_vm.c @@ -34,6 +34,8 @@ #include #include #include +#include +#include #include #include #include @@ -380,19 +382,69 @@ vm_fault_t ttm_bo_vm_fault_reserved(struct vm_fault *vmf, } EXPORT_SYMBOL(ttm_bo_vm_fault_reserved); +static void ttm_bo_release_dummy_page(struct drm_device *dev, void *res) +{ + struct page *dummy_page = (struct page *)res; + + __free_page(dummy_page); +} + +vm_fault_t ttm_bo_vm_dummy_page(struct vm_fault *vmf, pgprot_t prot) +{ + struct vm_area_struct *vma = vmf->vma; + struct ttm_buffer_object *bo = vma->vm_private_data; + struct drm_device *ddev = bo->base.dev; + vm_fault_t ret = VM_FAULT_NOPAGE; + unsigned long address; + unsigned long pfn; + struct page *page; + + /* Allocate new dummy page to map all the VA range in this VMA to it*/ + page = alloc_page(GFP_KERNEL | __GFP_ZERO); + if (!page) + return VM_FAULT_OOM; + + /* Set the page to be freed using drmm release action */ + if (drmm_add_action_or_reset(ddev, ttm_bo_release_dummy_page, page)) + return VM_FAULT_OOM; + + pfn = page_to_pfn(page); + + /* Prefault the entire VMA range right away to avoid further faults */ + for (address = vma->vm_start; address < vma->vm_end; address += PAGE_SIZE) { + + if (vma->vm_flags & VM_MIXEDMAP) + ret = vmf_insert_mixed_prot(vma, address, + __pfn_to_pfn_t(pfn, PFN_DEV), + prot); + else + ret = vmf_insert_pfn_prot(vma, address, pfn, prot); + } + + return ret; +} +EXPORT_SYMBOL(ttm_bo_vm_dummy_page); + vm_fault_t ttm_bo_vm_fault(struct vm_fault *vmf) { struct vm_area_struct *vma = vmf->vma; pgprot_t prot; struct ttm_buffer_object *bo = vma->vm_private_data; + struct drm_device *ddev = bo->base.dev; vm_fault_t ret; + int idx; ret = ttm_bo_vm_reserve(bo, vmf); if (ret) return ret; prot = vma->vm_page_prot; - ret = ttm_bo_vm_fault_reserved(vmf, prot, TTM_BO_VM_NUM_PREFAULT, 1); + if (drm_dev_enter(ddev, &idx)) { + ret = ttm_bo_vm_fault_reserved(vmf, prot, TTM_BO_VM_NUM_PREFAULT, 1); + drm_dev_exit(idx); + } else { + ret = ttm_bo_vm_dummy_page(vmf, prot); + } if (ret == VM_FAULT_RETRY && !(vmf->flags & FAULT_FLAG_RETRY_NOWAIT)) return ret; diff --git a/include/drm/ttm/ttm_bo_api.h b/include/drm/ttm/ttm_bo_api.h index 639521880c29..254ede97f8e3 100644 --- a/include/drm/ttm/ttm_bo_api.h +++ b/include/drm/ttm/ttm_bo_api.h @@ -620,4 +620,6 @@ int ttm_bo_vm_access(struct vm_area_struct *vma, unsigned long addr, void *buf, int len, int write); bool ttm_bo_delayed_delete(struct ttm_device *bdev, bool remove_all); +vm_fault_t ttm_bo_vm_dummy_page(struct vm_fault *vmf, pgprot_t prot); + #endif From patchwork Wed May 12 14:26:34 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Andrey Grodzovsky X-Patchwork-Id: 12253873 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-18.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,MSGID_FROM_MTA_HEADER,SPF_HELO_NONE, SPF_PASS,USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 9F334C433B4 for ; Wed, 12 May 2021 14:27:22 +0000 (UTC) Received: from gabe.freedesktop.org (gabe.freedesktop.org [131.252.210.177]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 430FB61407 for ; Wed, 12 May 2021 14:27:22 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 430FB61407 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=amd.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=dri-devel-bounces@lists.freedesktop.org Received: from gabe.freedesktop.org (localhost [127.0.0.1]) by gabe.freedesktop.org (Postfix) with ESMTP id 76CB86E507; Wed, 12 May 2021 14:27:14 +0000 (UTC) Received: from NAM04-DM6-obe.outbound.protection.outlook.com (mail-dm6nam08on2046.outbound.protection.outlook.com [40.107.102.46]) by gabe.freedesktop.org (Postfix) with ESMTPS id 449D5899F0; Wed, 12 May 2021 14:27:11 +0000 (UTC) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=lDjKdmFDx6J4sJvCAv8mGUSpjQpWWUqFNWafsEh7aP9+OSfprhWD96ceGHyCYTLyoZSQfhWMdENfZnUy5YeblOOB8mrF6hqzkOba0IjBcB0cBUSjJCuLdoaneRFekUW/eQkPwrJKMexx6YTabHeYwg50nr5iAL07VI9Ov62cXk/mfQIrQpj5QRc/Fo2gVpht5ghS2aMXbo+APncC6htNq1BU+rj39CePbdxf4NhFARdYFPTLngD47rz/01wYRQtnlYUKV/ItV79qYdjejFav2206gb2aWKEz2Hre9JBPGbj5b+7+Wxd5lrQNHVzdQ/t5uFt/VdwAnbnKrRF3nHZeQA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=9x9LOr31QXQh26mL0MRx+84J3JNefHzW2myk9ZEcHMw=; b=QdeaMp/tfFyo8bSnV2j6s0zYMCp229n4aURnGQ5UJVbtbItYYzYwcqJ00baqz82zkkQ0vIprGJg7YG5umYNhargNpWEAqtSn4QaOqUOLvw/D0EE4uVZWphCsaDIqIv3rNwPM3Wt/ESivFupsSQUhtjKEOF8TAXLs3sf4/iqzBxqcy47/inCBktcVH176nToC7wPq9GVi5WPaQJAdrVnk94a69ZzD2z9XLuZMS1HZ16j1psP7MiJ1ixuSUEyCRTcGizUWhmq+1hpTPs4XlEYzvNon0F1LlmDiS87eED0kiBUCV0x/76qG9G9VBbTYpm+7Gx0p2tl0mNB7rjQ6pwlwdg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=amd.com; dmarc=pass action=none header.from=amd.com; dkim=pass header.d=amd.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=9x9LOr31QXQh26mL0MRx+84J3JNefHzW2myk9ZEcHMw=; b=ofNVIaJIvhWUP4Vj7LrIRfI7si31+afqaT5VoCUPuOhQncIHJah4Mn9pf7aPz8xzL0UZPI6P+QItuRrIIWZQ7J4pMntdFeAQj4C4ZsnLUGE7DaW3Zgx8jj9zveYxhWKRVIH6KihEfZXAYmUP4oCawd8+0Ikb4qxyujWfDj6gjFQ= Authentication-Results: lists.freedesktop.org; dkim=none (message not signed) header.d=none; lists.freedesktop.org; dmarc=none action=none header.from=amd.com; Received: from SN6PR12MB4623.namprd12.prod.outlook.com (2603:10b6:805:e9::17) by SA0PR12MB4576.namprd12.prod.outlook.com (2603:10b6:806:93::13) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4108.27; Wed, 12 May 2021 14:27:09 +0000 Received: from SN6PR12MB4623.namprd12.prod.outlook.com ([fe80::ad51:8c49:b171:856c]) by SN6PR12MB4623.namprd12.prod.outlook.com ([fe80::ad51:8c49:b171:856c%7]) with mapi id 15.20.4129.026; Wed, 12 May 2021 14:27:09 +0000 From: Andrey Grodzovsky To: dri-devel@lists.freedesktop.org, amd-gfx@lists.freedesktop.org, linux-pci@vger.kernel.org, ckoenig.leichtzumerken@gmail.com, daniel.vetter@ffwll.ch, Harry.Wentland@amd.com Subject: [PATCH v7 02/16] drm/amdgpu: Split amdgpu_device_fini into early and late Date: Wed, 12 May 2021 10:26:34 -0400 Message-Id: <20210512142648.666476-3-andrey.grodzovsky@amd.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210512142648.666476-1-andrey.grodzovsky@amd.com> References: <20210512142648.666476-1-andrey.grodzovsky@amd.com> X-Originating-IP: [2607:fea8:3edf:49b0:7576:4f76:97d8:1487] X-ClientProxiedBy: YTOPR0101CA0032.CANPRD01.PROD.OUTLOOK.COM (2603:10b6:b00:15::45) To SN6PR12MB4623.namprd12.prod.outlook.com (2603:10b6:805:e9::17) MIME-Version: 1.0 X-MS-Exchange-MessageSentRepresentingType: 1 Received: from agrodzovsky-All-Series.hitronhub.home (2607:fea8:3edf:49b0:7576:4f76:97d8:1487) by YTOPR0101CA0032.CANPRD01.PROD.OUTLOOK.COM (2603:10b6:b00:15::45) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4108.30 via Frontend Transport; Wed, 12 May 2021 14:27:08 +0000 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 0253bf37-5a2c-4721-510c-08d915520672 X-MS-TrafficTypeDiagnostic: SA0PR12MB4576: X-MS-Exchange-Transport-Forked: True X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:862; X-MS-Exchange-SenderADCheck: 1 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: KUk+g7dHrI/2cpfTZMzVHbrE68z9KQCltBgvxkrHHkQZLPm3U+TSkvl8SmsKqSfvX4lXXj7xS9Ho1ncs3YnzLGUifHwd9F3h/GGCgGLfvBGjM3rIWCC9Cp+luHcLbJVUJry1et6LbHPQDL3RW9edjuFxK8XXn4s0bCNVeeqN+zgWkqbVJ6aXj6RHxqAOsVA9cZt0dkBpK55/foIW799VORZawC13914WJqxV094xFDpYruNwtJSKB9hmPyWmBhZ4Hvlaqi32q2Bow/BKjHw+AIDsfdMny4Jf51/1MeGZyZJIozZd2w6XeNv48KCGsqVMa0O7F0ZI0NPf3rhzbbnBcR8a2OPJlowOIsHgbFyBTK7tpfJjSlImczmeulsK00L/k5w0B5VshKhQrvqCgB7j2DCJUdselzTiNvB5x+4ndTUw0lmCz3vYn5uIgbddjGqUtNLYGBYBSNBR6+6prR2WUJsMcy9//+EOLYqj/gmag9kEUQ291m1tji8B2WbivGAaLQ3VtU7bzipuF8MoGOMlIA0vUNY7hcAOdPDTttGI4DcWFJuH1W+bn6sfMmXTzSq7G0cpWPFbSrOUb5YJZs6alk49x8Q6oxnw54oRzeBmEUs= X-Forefront-Antispam-Report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:SN6PR12MB4623.namprd12.prod.outlook.com; PTR:; CAT:NONE; SFS:(4636009)(376002)(39860400002)(366004)(346002)(396003)(136003)(66574015)(38100700002)(5660300002)(1076003)(6666004)(186003)(316002)(16526019)(8676002)(83380400001)(66476007)(2906002)(6636002)(86362001)(30864003)(52116002)(36756003)(54906003)(2616005)(66946007)(66556008)(8936002)(6512007)(6506007)(44832011)(478600001)(4326008)(6486002); DIR:OUT; SFP:1101; X-MS-Exchange-AntiSpam-MessageData: =?utf-8?q?vWz0sIROGLdCDIHdNK9D24Aze8d/63?= =?utf-8?q?IP/ldKaV4HeDx7eQZ4Y3RhvCojx6B2cQiyCyz7ontA4IvPww4BZNnxaFkwiVPyhLw?= =?utf-8?q?Q2m2QGmyGlgILkYBYHQ2J/lKh06oYH6TU6ItgmQEMKJKHZBRH9lpn2fCq9N02QDko?= =?utf-8?q?XmaGCV0eZRprL5KYSiz4ZC7gidIgKQ0GIdL0mIl2Yx34yKlMJOVmHVqf1TmE4/An+?= =?utf-8?q?AS0P5pG4firAj73+1nnwaSmBavWnV/p3FNKuy0JoPkVgu4FzqmQHmZzPJYW5KwDG5?= =?utf-8?q?qOcvjWwE/8Py7F1V90KNkUIF8pvEAhtQu0vUuRZYasMqtO5WPJesRkaV/Nfe5y/hm?= =?utf-8?q?YyG8P7MK9rLX3uOPLcI5LAQQU8RO+9FY4PgkeB73YiZHZiuHKfwixbOaL+tmunwd/?= =?utf-8?q?yu0Fn05SxhwyDKyOk38rpl0/b41/7Xcde5OW7QOhDjf8dyNQdMxhCjeMX94m6Ik1x?= =?utf-8?q?TCtsMrDIec+ghHF7hntQluxf9gNIYoYdPCWRXZKUPnuzAO95E4b9Bucs6/fWoV4Dy?= =?utf-8?q?+sZCl9THB2nx1aaYRKdgZvop00/sbaL4bWO676MR/WV/aR3jAt54ZrigQqqhW8bU/?= =?utf-8?q?IPcSUVM92B9ZKqITVMShIQwmNwjUS3D4JpRhFFF2sL6mQhWuUt4EBSO29YI+uvBjw?= =?utf-8?q?ZnJAVVaoBuo3NNRMIikirMeW9m7qsiCdmC51C4SLwQsdbfZPM+vwZynUUGz4M0Fnz?= =?utf-8?q?yF6btwH1RSN2KyraZh65gjEG9A+dM2izPy44FqfmLTkoNVePIj7JgOBbl7E/zAbWI?= =?utf-8?q?JS5v493q+AQGfVBL2hUaxIFAoY4YvuK54zWM5t0nam02BpRBAuCzrxJE4oVKBZZNt?= =?utf-8?q?aBl2uS7u3u/teLo+LGeQ9VXR5m0lzjkUUYQj0YSS5yfirOTvmedxzCHPmdClvyAk2?= =?utf-8?q?iNAwwTXeVKNlvwU7es1LFo9HIFOYPJWJXNwYk4giIczLq8JnUXnmcUDzdGVYbt1Pi?= =?utf-8?q?v1xQ5p3XUp3l9Jn0SoJLiZHBoWypiui7xOYhL3Le5DFjBbBj+NVt2I9v36ydFI3An?= =?utf-8?q?MdN93/FoBTkgWCL9euQg0qOOVwMzbUP6Wd06CeN7wnZu3dHRY3wwJOLy/B8szq+zY?= =?utf-8?q?4C3sVXJpvG48Gho/hk20nwQog7Ke9lc/4ZTy1H8kN1iJp8WxQBaK9SZMgPBkYuWga?= =?utf-8?q?5EVW+4TFA2L4xLb62RVsBdTS3s/TGKUOdmabUPSd1K0A5ujaNZpFaX13iH8rp/Ffv?= =?utf-8?q?GHLacd8YGrUgfAUelAdmfOaoDDJNDg8IJHWwFqxa10RKSTUbZYrrf2cwIdRAih244?= =?utf-8?q?70Txpw8zF2pCq4Mfez2VJMN/QF36u8mq6wuM8NLcIx2DXcFPteuDSDcdOmUeKbhY2?= =?utf-8?q?PtsYntI+vc0nk?= X-OriginatorOrg: amd.com X-MS-Exchange-CrossTenant-Network-Message-Id: 0253bf37-5a2c-4721-510c-08d915520672 X-MS-Exchange-CrossTenant-AuthSource: SN6PR12MB4623.namprd12.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 12 May 2021 14:27:09.3090 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: Dqpu5joRz6BGJNehmMl+PCCBrx7jQbRLMZE7yu9B29TKGAaVHPoeo31BumGwC1Og8ScfzV7utj9JeSGIeV4PmA== X-MS-Exchange-Transport-CrossTenantHeadersStamped: SA0PR12MB4576 X-BeenThere: dri-devel@lists.freedesktop.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Direct Rendering Infrastructure - Development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: gregkh@linuxfoundation.org, Felix.Kuehling@amd.com, helgaas@kernel.org, Alex Deucher , =?utf-8?q?Christian_K=C3=B6nig?= Errors-To: dri-devel-bounces@lists.freedesktop.org Sender: "dri-devel" Some of the stuff in amdgpu_device_fini such as HW interrupts disable and pending fences finilization must be done right away on pci_remove while most of the stuff which relates to finilizing and releasing driver data structures can be kept until drm_driver.release hook is called, i.e. when the last device reference is dropped. v4: Change functions prefix early->hw and late->sw Signed-off-by: Andrey Grodzovsky Acked-by: Christian König Reviewed-by: Alex Deucher --- drivers/gpu/drm/amd/amdgpu/amdgpu.h | 6 ++++- drivers/gpu/drm/amd/amdgpu/amdgpu_device.c | 26 +++++++++++++++------- drivers/gpu/drm/amd/amdgpu/amdgpu_drv.c | 7 ++---- drivers/gpu/drm/amd/amdgpu/amdgpu_fence.c | 15 ++++++++++++- drivers/gpu/drm/amd/amdgpu/amdgpu_irq.c | 26 +++++++++++++--------- drivers/gpu/drm/amd/amdgpu/amdgpu_irq.h | 3 ++- drivers/gpu/drm/amd/amdgpu/amdgpu_kms.c | 12 +++++++++- drivers/gpu/drm/amd/amdgpu/amdgpu_ras.c | 1 + drivers/gpu/drm/amd/amdgpu/amdgpu_ring.h | 3 ++- drivers/gpu/drm/amd/amdgpu/cik_ih.c | 2 +- drivers/gpu/drm/amd/amdgpu/cz_ih.c | 2 +- drivers/gpu/drm/amd/amdgpu/iceland_ih.c | 2 +- drivers/gpu/drm/amd/amdgpu/navi10_ih.c | 2 +- drivers/gpu/drm/amd/amdgpu/si_ih.c | 2 +- drivers/gpu/drm/amd/amdgpu/tonga_ih.c | 2 +- drivers/gpu/drm/amd/amdgpu/vega10_ih.c | 2 +- drivers/gpu/drm/amd/amdgpu/vega20_ih.c | 2 +- 17 files changed, 79 insertions(+), 36 deletions(-) diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu.h b/drivers/gpu/drm/amd/amdgpu/amdgpu.h index 380801b59b07..d830a541ba89 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu.h +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu.h @@ -1099,7 +1099,9 @@ static inline struct amdgpu_device *amdgpu_ttm_adev(struct ttm_device *bdev) int amdgpu_device_init(struct amdgpu_device *adev, uint32_t flags); -void amdgpu_device_fini(struct amdgpu_device *adev); +void amdgpu_device_fini_hw(struct amdgpu_device *adev); +void amdgpu_device_fini_sw(struct amdgpu_device *adev); + int amdgpu_gpu_wait_for_idle(struct amdgpu_device *adev); void amdgpu_device_vram_access(struct amdgpu_device *adev, loff_t pos, @@ -1319,6 +1321,8 @@ void amdgpu_driver_lastclose_kms(struct drm_device *dev); int amdgpu_driver_open_kms(struct drm_device *dev, struct drm_file *file_priv); void amdgpu_driver_postclose_kms(struct drm_device *dev, struct drm_file *file_priv); +void amdgpu_driver_release_kms(struct drm_device *dev); + int amdgpu_device_ip_suspend(struct amdgpu_device *adev); int amdgpu_device_suspend(struct drm_device *dev, bool fbcon); int amdgpu_device_resume(struct drm_device *dev, bool fbcon); diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c index b4ad1c055c70..3760ce7d8ff8 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c @@ -3648,15 +3648,13 @@ int amdgpu_device_init(struct amdgpu_device *adev, * Tear down the driver info (all asics). * Called at driver shutdown. */ -void amdgpu_device_fini(struct amdgpu_device *adev) +void amdgpu_device_fini_hw(struct amdgpu_device *adev) { dev_info(adev->dev, "amdgpu: finishing device.\n"); flush_delayed_work(&adev->delayed_init_work); ttm_bo_lock_delayed_workqueue(&adev->mman.bdev); adev->shutdown = true; - kfree(adev->pci_state); - /* make sure IB test finished before entering exclusive mode * to avoid preemption on IB test * */ @@ -3673,11 +3671,24 @@ void amdgpu_device_fini(struct amdgpu_device *adev) else drm_atomic_helper_shutdown(adev_to_drm(adev)); } - amdgpu_fence_driver_fini(adev); + amdgpu_fence_driver_fini_hw(adev); + if (adev->pm_sysfs_en) amdgpu_pm_sysfs_fini(adev); + if (adev->ucode_sysfs_en) + amdgpu_ucode_sysfs_fini(adev); + sysfs_remove_files(&adev->dev->kobj, amdgpu_dev_attributes); + + amdgpu_fbdev_fini(adev); + + amdgpu_irq_fini_hw(adev); +} + +void amdgpu_device_fini_sw(struct amdgpu_device *adev) +{ amdgpu_device_ip_fini(adev); + amdgpu_fence_driver_fini_sw(adev); release_firmware(adev->firmware.gpu_info_fw); adev->firmware.gpu_info_fw = NULL; adev->accel_working = false; @@ -3703,14 +3714,13 @@ void amdgpu_device_fini(struct amdgpu_device *adev) adev->rmmio = NULL; amdgpu_device_doorbell_fini(adev); - if (adev->ucode_sysfs_en) - amdgpu_ucode_sysfs_fini(adev); - - sysfs_remove_files(&adev->dev->kobj, amdgpu_dev_attributes); if (IS_ENABLED(CONFIG_PERF_EVENTS)) amdgpu_pmu_fini(adev); if (adev->mman.discovery_bin) amdgpu_discovery_fini(adev); + + kfree(adev->pci_state); + } diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_drv.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_drv.c index 6cf573293823..5ebed4c7d9c0 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_drv.c +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_drv.c @@ -1311,14 +1311,10 @@ amdgpu_pci_remove(struct pci_dev *pdev) { struct drm_device *dev = pci_get_drvdata(pdev); -#ifdef MODULE - if (THIS_MODULE->state != MODULE_STATE_GOING) -#endif - DRM_ERROR("Hotplug removal is not supported\n"); drm_dev_unplug(dev); amdgpu_driver_unload_kms(dev); + pci_disable_device(pdev); - pci_set_drvdata(pdev, NULL); } static void @@ -1748,6 +1744,7 @@ static const struct drm_driver amdgpu_kms_driver = { .dumb_create = amdgpu_mode_dumb_create, .dumb_map_offset = amdgpu_mode_dumb_mmap, .fops = &amdgpu_driver_kms_fops, + .release = &amdgpu_driver_release_kms, .prime_handle_to_fd = drm_gem_prime_handle_to_fd, .prime_fd_to_handle = drm_gem_prime_fd_to_handle, diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_fence.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_fence.c index 47ea46859618..1ffb36bd0b19 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_fence.c +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_fence.c @@ -523,7 +523,7 @@ int amdgpu_fence_driver_init(struct amdgpu_device *adev) * * Tear down the fence driver for all possible rings (all asics). */ -void amdgpu_fence_driver_fini(struct amdgpu_device *adev) +void amdgpu_fence_driver_fini_hw(struct amdgpu_device *adev) { unsigned i, j; int r; @@ -545,6 +545,19 @@ void amdgpu_fence_driver_fini(struct amdgpu_device *adev) ring->fence_drv.irq_type); del_timer_sync(&ring->fence_drv.fallback_timer); + } +} + +void amdgpu_fence_driver_fini_sw(struct amdgpu_device *adev) +{ + unsigned int i, j; + + for (i = 0; i < AMDGPU_MAX_RINGS; i++) { + struct amdgpu_ring *ring = adev->rings[i]; + + if (!ring || !ring->fence_drv.initialized) + continue; + for (j = 0; j <= ring->fence_drv.num_fences_mask; ++j) dma_fence_put(ring->fence_drv.fences[j]); kfree(ring->fence_drv.fences); diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_irq.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_irq.c index 90f50561b43a..233b64dab94b 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_irq.c +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_irq.c @@ -49,6 +49,7 @@ #include #include #include +#include #include "amdgpu.h" #include "amdgpu_ih.h" #include "atom.h" @@ -348,6 +349,20 @@ int amdgpu_irq_init(struct amdgpu_device *adev) return 0; } + +void amdgpu_irq_fini_hw(struct amdgpu_device *adev) +{ + if (adev->irq.installed) { + drm_irq_uninstall(&adev->ddev); + adev->irq.installed = false; + if (adev->irq.msi_enabled) + pci_free_irq_vectors(adev->pdev); + + if (!amdgpu_device_has_dc_support(adev)) + flush_work(&adev->hotplug_work); + } +} + /** * amdgpu_irq_fini - shut down interrupt handling * @@ -357,19 +372,10 @@ int amdgpu_irq_init(struct amdgpu_device *adev) * functionality, shuts down vblank, hotplug and reset interrupt handling, * turns off interrupts from all sources (all ASICs). */ -void amdgpu_irq_fini(struct amdgpu_device *adev) +void amdgpu_irq_fini_sw(struct amdgpu_device *adev) { unsigned i, j; - if (adev->irq.installed) { - drm_irq_uninstall(adev_to_drm(adev)); - adev->irq.installed = false; - if (adev->irq.msi_enabled) - pci_free_irq_vectors(adev->pdev); - if (!amdgpu_device_has_dc_support(adev)) - flush_work(&adev->hotplug_work); - } - for (i = 0; i < AMDGPU_IRQ_CLIENTID_MAX; ++i) { if (!adev->irq.client[i].sources) continue; diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_irq.h b/drivers/gpu/drm/amd/amdgpu/amdgpu_irq.h index cf6116648322..78ad4784cc74 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_irq.h +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_irq.h @@ -103,7 +103,8 @@ void amdgpu_irq_disable_all(struct amdgpu_device *adev); irqreturn_t amdgpu_irq_handler(int irq, void *arg); int amdgpu_irq_init(struct amdgpu_device *adev); -void amdgpu_irq_fini(struct amdgpu_device *adev); +void amdgpu_irq_fini_sw(struct amdgpu_device *adev); +void amdgpu_irq_fini_hw(struct amdgpu_device *adev); int amdgpu_irq_add_id(struct amdgpu_device *adev, unsigned client_id, unsigned src_id, struct amdgpu_irq_src *source); diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_kms.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_kms.c index 39ee88d29cca..f3ecada208b0 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_kms.c +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_kms.c @@ -28,6 +28,7 @@ #include "amdgpu.h" #include +#include #include "amdgpu_uvd.h" #include "amdgpu_vce.h" #include "atom.h" @@ -92,7 +93,7 @@ void amdgpu_driver_unload_kms(struct drm_device *dev) } amdgpu_acpi_fini(adev); - amdgpu_device_fini(adev); + amdgpu_device_fini_hw(adev); } void amdgpu_register_gpu_instance(struct amdgpu_device *adev) @@ -1219,6 +1220,15 @@ void amdgpu_driver_postclose_kms(struct drm_device *dev, pm_runtime_put_autosuspend(dev->dev); } + +void amdgpu_driver_release_kms(struct drm_device *dev) +{ + struct amdgpu_device *adev = drm_to_adev(dev); + + amdgpu_device_fini_sw(adev); + pci_set_drvdata(adev->pdev, NULL); +} + /* * VBlank related functions. */ diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_ras.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_ras.c index 0541196ae1ed..844a667f655b 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_ras.c +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_ras.c @@ -2325,6 +2325,7 @@ int amdgpu_ras_pre_fini(struct amdgpu_device *adev) if (!adev->ras_features || !con) return 0; + /* Need disable ras on all IPs here before ip [hw/sw]fini */ amdgpu_ras_disable_all_features(adev, 0); amdgpu_ras_recovery_fini(adev); diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_ring.h b/drivers/gpu/drm/amd/amdgpu/amdgpu_ring.h index ca1622835296..e7d3d0dbdd96 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_ring.h +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_ring.h @@ -107,7 +107,8 @@ struct amdgpu_fence_driver { }; int amdgpu_fence_driver_init(struct amdgpu_device *adev); -void amdgpu_fence_driver_fini(struct amdgpu_device *adev); +void amdgpu_fence_driver_fini_hw(struct amdgpu_device *adev); +void amdgpu_fence_driver_fini_sw(struct amdgpu_device *adev); void amdgpu_fence_driver_force_completion(struct amdgpu_ring *ring); int amdgpu_fence_driver_init_ring(struct amdgpu_ring *ring, diff --git a/drivers/gpu/drm/amd/amdgpu/cik_ih.c b/drivers/gpu/drm/amd/amdgpu/cik_ih.c index d3745711d55f..183d44a6583c 100644 --- a/drivers/gpu/drm/amd/amdgpu/cik_ih.c +++ b/drivers/gpu/drm/amd/amdgpu/cik_ih.c @@ -309,7 +309,7 @@ static int cik_ih_sw_fini(void *handle) { struct amdgpu_device *adev = (struct amdgpu_device *)handle; - amdgpu_irq_fini(adev); + amdgpu_irq_fini_sw(adev); amdgpu_ih_ring_fini(adev, &adev->irq.ih); amdgpu_irq_remove_domain(adev); diff --git a/drivers/gpu/drm/amd/amdgpu/cz_ih.c b/drivers/gpu/drm/amd/amdgpu/cz_ih.c index 307c01301c87..d32743949003 100644 --- a/drivers/gpu/drm/amd/amdgpu/cz_ih.c +++ b/drivers/gpu/drm/amd/amdgpu/cz_ih.c @@ -301,7 +301,7 @@ static int cz_ih_sw_fini(void *handle) { struct amdgpu_device *adev = (struct amdgpu_device *)handle; - amdgpu_irq_fini(adev); + amdgpu_irq_fini_sw(adev); amdgpu_ih_ring_fini(adev, &adev->irq.ih); amdgpu_irq_remove_domain(adev); diff --git a/drivers/gpu/drm/amd/amdgpu/iceland_ih.c b/drivers/gpu/drm/amd/amdgpu/iceland_ih.c index cc957471f31e..da96c6013477 100644 --- a/drivers/gpu/drm/amd/amdgpu/iceland_ih.c +++ b/drivers/gpu/drm/amd/amdgpu/iceland_ih.c @@ -300,7 +300,7 @@ static int iceland_ih_sw_fini(void *handle) { struct amdgpu_device *adev = (struct amdgpu_device *)handle; - amdgpu_irq_fini(adev); + amdgpu_irq_fini_sw(adev); amdgpu_ih_ring_fini(adev, &adev->irq.ih); amdgpu_irq_remove_domain(adev); diff --git a/drivers/gpu/drm/amd/amdgpu/navi10_ih.c b/drivers/gpu/drm/amd/amdgpu/navi10_ih.c index f4e4040bbd25..5eea4550b856 100644 --- a/drivers/gpu/drm/amd/amdgpu/navi10_ih.c +++ b/drivers/gpu/drm/amd/amdgpu/navi10_ih.c @@ -569,7 +569,7 @@ static int navi10_ih_sw_fini(void *handle) { struct amdgpu_device *adev = (struct amdgpu_device *)handle; - amdgpu_irq_fini(adev); + amdgpu_irq_fini_sw(adev); amdgpu_ih_ring_fini(adev, &adev->irq.ih_soft); amdgpu_ih_ring_fini(adev, &adev->irq.ih2); amdgpu_ih_ring_fini(adev, &adev->irq.ih1); diff --git a/drivers/gpu/drm/amd/amdgpu/si_ih.c b/drivers/gpu/drm/amd/amdgpu/si_ih.c index 51880f6ef634..751307f3252c 100644 --- a/drivers/gpu/drm/amd/amdgpu/si_ih.c +++ b/drivers/gpu/drm/amd/amdgpu/si_ih.c @@ -175,7 +175,7 @@ static int si_ih_sw_fini(void *handle) { struct amdgpu_device *adev = (struct amdgpu_device *)handle; - amdgpu_irq_fini(adev); + amdgpu_irq_fini_sw(adev); amdgpu_ih_ring_fini(adev, &adev->irq.ih); return 0; diff --git a/drivers/gpu/drm/amd/amdgpu/tonga_ih.c b/drivers/gpu/drm/amd/amdgpu/tonga_ih.c index 249fcbee7871..973d80ec7f6c 100644 --- a/drivers/gpu/drm/amd/amdgpu/tonga_ih.c +++ b/drivers/gpu/drm/amd/amdgpu/tonga_ih.c @@ -312,7 +312,7 @@ static int tonga_ih_sw_fini(void *handle) { struct amdgpu_device *adev = (struct amdgpu_device *)handle; - amdgpu_irq_fini(adev); + amdgpu_irq_fini_sw(adev); amdgpu_ih_ring_fini(adev, &adev->irq.ih); amdgpu_irq_remove_domain(adev); diff --git a/drivers/gpu/drm/amd/amdgpu/vega10_ih.c b/drivers/gpu/drm/amd/amdgpu/vega10_ih.c index ca8efa5c6978..dead9c2fbd4c 100644 --- a/drivers/gpu/drm/amd/amdgpu/vega10_ih.c +++ b/drivers/gpu/drm/amd/amdgpu/vega10_ih.c @@ -513,7 +513,7 @@ static int vega10_ih_sw_fini(void *handle) { struct amdgpu_device *adev = (struct amdgpu_device *)handle; - amdgpu_irq_fini(adev); + amdgpu_irq_fini_sw(adev); amdgpu_ih_ring_fini(adev, &adev->irq.ih_soft); amdgpu_ih_ring_fini(adev, &adev->irq.ih2); amdgpu_ih_ring_fini(adev, &adev->irq.ih1); diff --git a/drivers/gpu/drm/amd/amdgpu/vega20_ih.c b/drivers/gpu/drm/amd/amdgpu/vega20_ih.c index 8a122b413bf5..58993ae1fe11 100644 --- a/drivers/gpu/drm/amd/amdgpu/vega20_ih.c +++ b/drivers/gpu/drm/amd/amdgpu/vega20_ih.c @@ -565,7 +565,7 @@ static int vega20_ih_sw_fini(void *handle) { struct amdgpu_device *adev = (struct amdgpu_device *)handle; - amdgpu_irq_fini(adev); + amdgpu_irq_fini_sw(adev); amdgpu_ih_ring_fini(adev, &adev->irq.ih_soft); amdgpu_ih_ring_fini(adev, &adev->irq.ih2); amdgpu_ih_ring_fini(adev, &adev->irq.ih1); From patchwork Wed May 12 14:26:35 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrey Grodzovsky X-Patchwork-Id: 12253871 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-18.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,MSGID_FROM_MTA_HEADER,SPF_HELO_NONE, SPF_PASS,USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 01723C43461 for ; Wed, 12 May 2021 14:27:20 +0000 (UTC) Received: from gabe.freedesktop.org (gabe.freedesktop.org [131.252.210.177]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 992F4613E9 for ; Wed, 12 May 2021 14:27:19 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 992F4613E9 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=amd.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=dri-devel-bounces@lists.freedesktop.org Received: from gabe.freedesktop.org (localhost [127.0.0.1]) by gabe.freedesktop.org (Postfix) with ESMTP id DB3736E1D5; Wed, 12 May 2021 14:27:13 +0000 (UTC) Received: from NAM04-DM6-obe.outbound.protection.outlook.com (mail-dm6nam08on2083.outbound.protection.outlook.com [40.107.102.83]) by gabe.freedesktop.org (Postfix) with ESMTPS id B083C6E210; Wed, 12 May 2021 14:27:12 +0000 (UTC) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=iml+pX7Rmj+BMTRh3EsLGG+hWP1uo+N3rifDXZOB8gwwP8y1IJDWpOLliJP+kVrx2Qy8lgNRrWh1oYTi1keeeHBF75HVRCF2GmSRi+nypEVqD3cZtcbOyIrmgbkIRL+ikpCRjDSbpGJpmlZ/mnUSzeO4KTnpmsGEq/nRrEltPezvvL2nPwx/XjJukLXsep2nTgFzfh7rgMcfDDXvXMIKv7gVtRIrbTH6UAgEavNiE8MWCoF19TJWuH5Xt3MEliWFZ5nLukFq4VVAmEwp9PkOQE7gwRisg2iLfzOWGemXRDbmUJwor3bWnAxwFTyawu91bypE3hkczgCDjQ3d6jSaiA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=+8ub1TT4Why1wfUTgqnHi8urBMdsD729TcqEvDbve44=; b=OZYnez1Eoaqwya0ydUJKWiGOD4565pWqCsprFCeukbJW6Lwjy0NlreQuC240jT2jUjkYt7vDDBCM6nBaTrg2gI8fckng3CdQVhNKnTVsjPeD/JOAugAJHBkCVm/WXnahuyqPN7zpJd4OdyLX/RbOnNFQbJJnd0ELn0dt/snP1wO/xYqchL+cAtdtEBNOm1x9cM8Iw4hHV5Cn7SbeV2rVNl1xmL5lINq9nO7dYb9OGsdAW+ywc3sZ6Ii2QTm0ZCWuWY7Otsz20jOYY3Ur/pcyw8ZROBgeufvak4Rv6KPNqpAto8u+BtIAX9hzWlExWuO7qAEfCVZGpmjxJj6/0ePNvQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=amd.com; dmarc=pass action=none header.from=amd.com; dkim=pass header.d=amd.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=+8ub1TT4Why1wfUTgqnHi8urBMdsD729TcqEvDbve44=; b=deKB3rPt/WzsM/zddxXQn1kg9wGvvNemZt3P+EcZRU38onurmhQcCal73LmIoe5VvKLl8nO0kafQb4OGIXvEhE8JM9T+dZCmZNK5HmQCcM/p8I95OoEfQHJZDHqO5eBAyM8r4fSKfJ/N3Mg0X731StvZL/99JHLfbtap8Rgc+w0= Authentication-Results: lists.freedesktop.org; dkim=none (message not signed) header.d=none; lists.freedesktop.org; dmarc=none action=none header.from=amd.com; Received: from SN6PR12MB4623.namprd12.prod.outlook.com (2603:10b6:805:e9::17) by SA0PR12MB4576.namprd12.prod.outlook.com (2603:10b6:806:93::13) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4108.27; Wed, 12 May 2021 14:27:11 +0000 Received: from SN6PR12MB4623.namprd12.prod.outlook.com ([fe80::ad51:8c49:b171:856c]) by SN6PR12MB4623.namprd12.prod.outlook.com ([fe80::ad51:8c49:b171:856c%7]) with mapi id 15.20.4129.026; Wed, 12 May 2021 14:27:11 +0000 From: Andrey Grodzovsky To: dri-devel@lists.freedesktop.org, amd-gfx@lists.freedesktop.org, linux-pci@vger.kernel.org, ckoenig.leichtzumerken@gmail.com, daniel.vetter@ffwll.ch, Harry.Wentland@amd.com Subject: [PATCH v7 03/16] drm/amdkfd: Split kfd suspend from device exit Date: Wed, 12 May 2021 10:26:35 -0400 Message-Id: <20210512142648.666476-4-andrey.grodzovsky@amd.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210512142648.666476-1-andrey.grodzovsky@amd.com> References: <20210512142648.666476-1-andrey.grodzovsky@amd.com> X-Originating-IP: [2607:fea8:3edf:49b0:7576:4f76:97d8:1487] X-ClientProxiedBy: YTOPR0101CA0032.CANPRD01.PROD.OUTLOOK.COM (2603:10b6:b00:15::45) To SN6PR12MB4623.namprd12.prod.outlook.com (2603:10b6:805:e9::17) MIME-Version: 1.0 X-MS-Exchange-MessageSentRepresentingType: 1 Received: from agrodzovsky-All-Series.hitronhub.home (2607:fea8:3edf:49b0:7576:4f76:97d8:1487) by YTOPR0101CA0032.CANPRD01.PROD.OUTLOOK.COM (2603:10b6:b00:15::45) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4108.30 via Frontend Transport; Wed, 12 May 2021 14:27:10 +0000 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: e13d4333-6ddc-428a-54ee-08d9155207bc X-MS-TrafficTypeDiagnostic: SA0PR12MB4576: X-MS-Exchange-Transport-Forked: True X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:510; X-MS-Exchange-SenderADCheck: 1 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: SxfFz1bzt9qxNw6BYhX3PyFfbY1sjd9jJoXdO7Q1561W0MYFQblFM53ISCAYDaKt5NvtOOzMRh+HMXSrXXx6xlYf6NRA1fpW1+gvKm/JMYO7taGQvM+GBM2zOJPsjC2R6L8wMvQ47EOdyz6VVRR/D8F/HHf1g0Sh+zoBtnua3Pbf0QvGumTepgOSOY93JBAl6a5nrfiII4RXjHVx4YLVpT1TWW26x9SfpXpIG8cdVDKgfW52on79of68pMRQDJ9AnKTw6pYcAfKTDMbqwZjKNDQX7zjWFG5VhoEcPG56yKZs/cyS0oR4cfQ8k4RuWLQG3ze2+Fr333rAPWzGi9OBLpdkoxrKwtFZvuxHXvmKRhGTm1VytuvOSZ64wg2cW6VkfrugEVMn1Pg2N0jBx7w7EK2SZ0l2IC4nmA6zDBzjsfn6Vn5kOqTOn1O6dR3CK430SErzfYqMtTAYuphjPB48o1Rgq6qJV6Ai4z9Y/prYXvjV/YGuDRbWi0zxReXtz/yVTowrbsOPpHww1JJkPsdm7FGkEqPIb9zeoNbC58m7CrX3YNUQEt8p7+HDH8bt4lrbw8zgklD6UQfktuNV5HmL2KSNJ8A3JdwKpR2wGVoySak= X-Forefront-Antispam-Report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:SN6PR12MB4623.namprd12.prod.outlook.com; PTR:; CAT:NONE; SFS:(4636009)(376002)(39860400002)(366004)(346002)(396003)(136003)(38100700002)(5660300002)(1076003)(6666004)(186003)(316002)(16526019)(8676002)(83380400001)(66476007)(2906002)(6636002)(86362001)(52116002)(36756003)(2616005)(66946007)(66556008)(8936002)(6512007)(6506007)(44832011)(478600001)(4326008)(6486002); DIR:OUT; SFP:1101; X-MS-Exchange-AntiSpam-MessageData: NuPcrtPD5jlyoDjH8rfNGPbXLj/bUs3hTUt/fdTH9GpdL9l9Fxnu50WR1lBkvI1GcWAXlfXDkJ3yeiZkvARPKMwU5DRYkI/5GVvOETLaair8QAos0ilksSshQ1O1LR9g0aX7vaIkwOUz2aiQ1S8psClAjj091VpzmDjplkZfqu06x9VuT4xIkaOxCpeD7/Y0iHW93l7IeVGD7GPpJ+sLMvzpI96JLNgjOMc8crsap5echWJ0TC8/k+ETGORFezfUYlV1nNEqGdZzDLYVvQxSRyhpjExoYqHwHkKUfxWPAWQw1rns1B+lwWmIKaFdmDF6pgzMTYaiZvsGLxQhkDq90fpdBcBE2xp94kfhMxbQkkt2VE528IT/7uMw+IXlAgxR4zfmkudNQOQdnfdmaG0gOAc2Qtxbr54M7LpwipeXgokklbhctuNGmNm7Ns08FGqnVPH1Ek6nt+2PE1/EnNLQ4id25uVjcT2Qk7XD8d7QogvcK9/IkAkqmlhGTirO5ZbPfePgK/u/8Gy36At0R1pXktOKDZzA1miuRehYWZ9aXKPmr8Ab8qwEW1IdID/qZi2/XT1kNm+MLkAp9vs/yhHcoa93qEX9DyywNejauU6qq+7O0SscVAQw/QkmOWJKvToJ8FsfxrX8YTVuvCNwztXApUasg2e8P0SX5X4DnwiVniA4BXL/GzgN3AAiGy2eK1XOEPd9ZixqJz2+LGXzhhL2FYrK9Af9RKVVaigTO1rQP6ypmaOjYmQvlkWIWppN1isWINNm1nAkDji4u5rLN5NwgC1M7PyKyI+rf7HXigkSJiwAMjaPDGHJUIr7NYNYZFS2UQRe7QGxKkrg9iV0D31nMBSwoLMEKzP4HEr2KWlzrte4d0FtzWFiRlYr3CkhW6sSwiue3zsZ1F5iZ8yXEajvqySiilldu0CqiEK/deuEfuu9/3cWby1NdJiXwtc3az8dUcwVOrhpIzMmS+U+UAtKMAAYIjx7oAQtrn2s75c+Y82w9MdO7Gc5MBcyVqxlIkAWk+gD6MCKJThpnkM1JlEGSwkzYLwGnXwVybhxuhi8sDQMo4EQt5bHUlSRy1WClPGdlZ7NqIVw4ofbyeVCf2p3DByLxvrTsC4cy9ieQEyAThMAmhL4hldW6rDzITSsctESzFkFQL7VqPjN7EICsBnARxJBNqmYNEruDpG+TaHzDdQbeDAMeIiiVqA2fcAV57kp7y3MXkzizd8whtBqvRy8MiR23t5Vrjr24hmVCVPbwreDR6lTyXCFsR79ThDWBYTpiEOg8m2RGgGRXf4PtWfy6gWRuWyC1pAXKHRm1J84Tu/lUXWDIbVFeInxUMIOrBLP9uswfUr/Ywy/htols8SIGh5h0hn83WN1W+dr1jlh3pRMRKm0kbxEUuM49W2ENJC8+/77RzGy2DLtsBg3ScvnWA== X-OriginatorOrg: amd.com X-MS-Exchange-CrossTenant-Network-Message-Id: e13d4333-6ddc-428a-54ee-08d9155207bc X-MS-Exchange-CrossTenant-AuthSource: SN6PR12MB4623.namprd12.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 12 May 2021 14:27:11.3019 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: 4mo4G/7TvGFqkJIms5YkMNozgRFj7HQB1AP24bgHBpxyTrVm/0IROlt+xCWlABd5/nKwiAPO3NhRBsPCk5W1FA== X-MS-Exchange-Transport-CrossTenantHeadersStamped: SA0PR12MB4576 X-BeenThere: dri-devel@lists.freedesktop.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Direct Rendering Infrastructure - Development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: gregkh@linuxfoundation.org, Felix.Kuehling@amd.com, helgaas@kernel.org, Alexander.Deucher@amd.com Errors-To: dri-devel-bounces@lists.freedesktop.org Sender: "dri-devel" Helps to expdite HW related stuff to amdgpu_pci_remove Signed-off-by: Andrey Grodzovsky --- drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd.c | 2 +- drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd.h | 2 +- drivers/gpu/drm/amd/amdkfd/kfd_device.c | 3 ++- 3 files changed, 4 insertions(+), 3 deletions(-) diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd.c index 5f6696a3c778..2b06dee9a0ce 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd.c +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd.c @@ -170,7 +170,7 @@ void amdgpu_amdkfd_device_init(struct amdgpu_device *adev) } } -void amdgpu_amdkfd_device_fini(struct amdgpu_device *adev) +void amdgpu_amdkfd_device_fini_sw(struct amdgpu_device *adev) { if (adev->kfd.dev) { kgd2kfd_device_exit(adev->kfd.dev); diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd.h b/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd.h index 14f68c028126..f8e10af99c28 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd.h +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd.h @@ -127,7 +127,7 @@ void amdgpu_amdkfd_interrupt(struct amdgpu_device *adev, const void *ih_ring_entry); void amdgpu_amdkfd_device_probe(struct amdgpu_device *adev); void amdgpu_amdkfd_device_init(struct amdgpu_device *adev); -void amdgpu_amdkfd_device_fini(struct amdgpu_device *adev); +void amdgpu_amdkfd_device_fini_sw(struct amdgpu_device *adev); int amdgpu_amdkfd_submit_ib(struct kgd_dev *kgd, enum kgd_engine_type engine, uint32_t vmid, uint64_t gpu_addr, uint32_t *ib_cmd, uint32_t ib_len); diff --git a/drivers/gpu/drm/amd/amdkfd/kfd_device.c b/drivers/gpu/drm/amd/amdkfd/kfd_device.c index 357b9bf62a1c..ab6d2a43c9a3 100644 --- a/drivers/gpu/drm/amd/amdkfd/kfd_device.c +++ b/drivers/gpu/drm/amd/amdkfd/kfd_device.c @@ -858,10 +858,11 @@ bool kgd2kfd_device_init(struct kfd_dev *kfd, return kfd->init_complete; } + + void kgd2kfd_device_exit(struct kfd_dev *kfd) { if (kfd->init_complete) { - kgd2kfd_suspend(kfd, false); device_queue_manager_uninit(kfd->dqm); kfd_interrupt_exit(kfd); kfd_topology_remove_device(kfd); From patchwork Wed May 12 14:26:36 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Andrey Grodzovsky X-Patchwork-Id: 12253875 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-18.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,MSGID_FROM_MTA_HEADER,SPF_HELO_NONE, SPF_PASS,USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 7B98FC43462 for ; Wed, 12 May 2021 14:27:25 +0000 (UTC) Received: from gabe.freedesktop.org (gabe.freedesktop.org [131.252.210.177]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id BCB6961411 for ; Wed, 12 May 2021 14:27:24 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org BCB6961411 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=amd.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=dri-devel-bounces@lists.freedesktop.org Received: from gabe.freedesktop.org (localhost [127.0.0.1]) by gabe.freedesktop.org (Postfix) with ESMTP id F1B996E529; Wed, 12 May 2021 14:27:18 +0000 (UTC) Received: from NAM04-DM6-obe.outbound.protection.outlook.com (mail-dm6nam08on2081.outbound.protection.outlook.com [40.107.102.81]) by gabe.freedesktop.org (Postfix) with ESMTPS id 0A1226E079; Wed, 12 May 2021 14:27:17 +0000 (UTC) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=Qwf5r8AfWrRbfO0kWP9Z3w8tP55sHFPhLqGDcbohMp+jzONLHzbCS9jBtPViP5BZgtzyKMFh936Qr2TEGpcxhm8/W/wOqk2i/PA6U19R8vERL8JKCNwXA1QiHanTQ2IWeky972pD+i4i8cjzIuoQxhgX6Z7yAozQ3jzjJUeT+Hrbg/WFsKRacWUW4IxhtPZJnzXxeBpkO6XVGR2tY33gxCj0oS9jXyhll/SgV/TolY8/AXXXMnHpcIjCeWC3/U8nUQr7BguWSeK3dO/Ztnkx/5bJLvq/FRHqRwfZzwnIUGj1VW44SJkxVagD9DH1PBZghspawgcrdUAfrypdIAcq8w== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=4f76bDjmlOpO1JyP2h+5eU+QH8A/4mNsiKFTRoZXVXM=; b=L0HTceAVIYPoSOgcP0SamgoUazIyguYMCkCC2mpBgTmejK7ankvtEE035z8fFBy892Cfe1CSMwHZdk7m7Ff1yb1Av0s449et6RFzANLGYEgjrHkO3OhNagyL2wRXRp0EddC+Ho04BgOhpA9h8rx/sPuNFQIr6uasAA9O3H14scpRqu0ritQIG0DRorDWP1q1E+Yat9DhaydHa46u4Yam024VYt4lUkUzPQUDHqxWU+oHY5E3mTD3UHVDlPOuVd+JfW3wByw2cvX+DGvbGkSoUA3B+uPmlsII0edKZ0ZAjwxV2RRKlcZAcW+sewi4D/7gg54SePjXT3/YGa9ct2uQcQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=amd.com; dmarc=pass action=none header.from=amd.com; dkim=pass header.d=amd.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=4f76bDjmlOpO1JyP2h+5eU+QH8A/4mNsiKFTRoZXVXM=; b=1hkQt4g3bDxNe3an6O7ZbvX4PCX8YOw7/a/4xupN0RkJ4KXfNxS3duWx1pP3BAm6BVbZa59uPViFrglbn5rTqRdC6djZJ6wTawo6smnUa7+GOciuKDktw4ap63JuTc+VIQodbixvzQUgfPnztPPf1DtJgVz6BvP9ziTJNUZdzP8= Authentication-Results: lists.freedesktop.org; dkim=none (message not signed) header.d=none; lists.freedesktop.org; dmarc=none action=none header.from=amd.com; Received: from SN6PR12MB4623.namprd12.prod.outlook.com (2603:10b6:805:e9::17) by SA0PR12MB4576.namprd12.prod.outlook.com (2603:10b6:806:93::13) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4108.27; Wed, 12 May 2021 14:27:15 +0000 Received: from SN6PR12MB4623.namprd12.prod.outlook.com ([fe80::ad51:8c49:b171:856c]) by SN6PR12MB4623.namprd12.prod.outlook.com ([fe80::ad51:8c49:b171:856c%7]) with mapi id 15.20.4129.026; Wed, 12 May 2021 14:27:15 +0000 From: Andrey Grodzovsky To: dri-devel@lists.freedesktop.org, amd-gfx@lists.freedesktop.org, linux-pci@vger.kernel.org, ckoenig.leichtzumerken@gmail.com, daniel.vetter@ffwll.ch, Harry.Wentland@amd.com Subject: [PATCH v7 04/16] drm/amdgpu: Add early fini callback Date: Wed, 12 May 2021 10:26:36 -0400 Message-Id: <20210512142648.666476-5-andrey.grodzovsky@amd.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210512142648.666476-1-andrey.grodzovsky@amd.com> References: <20210512142648.666476-1-andrey.grodzovsky@amd.com> X-Originating-IP: [2607:fea8:3edf:49b0:7576:4f76:97d8:1487] X-ClientProxiedBy: YTOPR0101CA0032.CANPRD01.PROD.OUTLOOK.COM (2603:10b6:b00:15::45) To SN6PR12MB4623.namprd12.prod.outlook.com (2603:10b6:805:e9::17) MIME-Version: 1.0 X-MS-Exchange-MessageSentRepresentingType: 1 Received: from agrodzovsky-All-Series.hitronhub.home (2607:fea8:3edf:49b0:7576:4f76:97d8:1487) by YTOPR0101CA0032.CANPRD01.PROD.OUTLOOK.COM (2603:10b6:b00:15::45) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4108.30 via Frontend Transport; Wed, 12 May 2021 14:27:14 +0000 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 9392f4b3-d6d9-4062-2df4-08d915520a42 X-MS-TrafficTypeDiagnostic: SA0PR12MB4576: X-MS-Exchange-Transport-Forked: True X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:5797; X-MS-Exchange-SenderADCheck: 1 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: UFHUdEwcyTwLchxUswSHmi1j6WFGVrMRTJi4PnqCwCNZLAgRZl+JbFPbLhoeX2crB47CQgIaYHThMZrTjg1kyyv3gNSkC3MydPKkELX4pm1KsVYUNlNyZwUUU0DIPImE8ulpkfOsr7vAtLQ4uVTS9/l7PXXvloFuX15RTVSZD5URfAfPNwzscvttYRcs2E1wX7ILqUByPyRLJXwcVFlYwWZEfUvALbWmmjce+5nskBV5V9/RhUtppQG6rrgB7QxfZfH3aVzm/EeWRAeQFwHUiyposg3FQ4YaYsYXxd3yP43nzjWOzLQExIjWPQk8ABqy7U/8yh8Zp5YAJwWTbGijeLl1ZjSOFC0BNfgOrfxjIiKSPbdcKR4BpRhRfG9zLyQGEmMWhVPZ08bBwsNNawg0iqlSMLrQ70o/k/tPwkO4D5HJ2zZHzGts/xwZBdV3WBEiw9/AoSC8r8zNL8utBZmyTWBxlO3jmVgttKiAu11V/3eulXUhMZBLmloqOMgzPSKQ9bJI+3WDssyprQ9exGgylvQUmHWS6aAh0W5j8K8L0BeacsDlz6WscYGFl0r8h4yNBwID2oTWgSjhpWTJiQjayw== X-Forefront-Antispam-Report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:SN6PR12MB4623.namprd12.prod.outlook.com; PTR:; CAT:NONE; SFS:(4636009)(376002)(39860400002)(366004)(346002)(396003)(136003)(66574015)(38100700002)(5660300002)(1076003)(6666004)(186003)(316002)(16526019)(8676002)(83380400001)(66476007)(2906002)(6636002)(86362001)(52116002)(36756003)(54906003)(2616005)(66946007)(66556008)(8936002)(6512007)(6506007)(44832011)(478600001)(4326008)(6486002); DIR:OUT; SFP:1101; X-MS-Exchange-AntiSpam-MessageData: =?utf-8?q?ijmZxnt0/xf6z6sRkuYnoGr2CBTrvM?= =?utf-8?q?Jgv8xDl9IvL115cpgtz6rBb7r87nNSoyRWsP4jtl80rZB1F+Jenjb0x76V4pFwYgH?= =?utf-8?q?++jTfySs7fr3flakJiy7WrmZuMtHn8aApJSI9Xeo8AcHpY0qOeKdqh47TjU8BuidC?= =?utf-8?q?yurp+87swhPR9asuOgPCzgUcSykIehjmOG2rCJgO5wt0xQbQjVVCtVV2L5mJt3yhZ?= =?utf-8?q?5JlY4FapYcZv2WbOZz7bQEgYWUEqY6XjXlVfeGRE9+8nLzL2fsmUxVhlpc/IlHwSN?= =?utf-8?q?vuMv40NpiqqMkrnLSZLgEF4bp84nT15Djk6idHRS14s+Rt+qWPbj4MyJ/UUK2tLJm?= =?utf-8?q?XINtrmwnXAu8iQdgBAw/BIhbzWqfVA89GBhPrilU2OBT1zKCwYoefyYvaDzFUghnm?= =?utf-8?q?lxxN/DvZy87mHHsUTcl/WtpMJenFwJQh81ndus1ut2no/d/OifzmEAxP85F44eJsN?= =?utf-8?q?xxyqBZNkWykvpI5x8RNsqkR/Hrw29RN7pXNb1rCKCODb5szWFmTJ5FweORBEvAABi?= =?utf-8?q?nGC7QUKZjxY55kcT38hs08YPOXdPmDWYCHGh9lz4cSAa7EhUuwsst9yG+bcSVwrkr?= =?utf-8?q?nfrFE2HJZcEebiDnwYJNThHQJHYjfHkjdv0GT00wj1+/zE37S4Mb8I9bET0cFuM0+?= =?utf-8?q?hGxP1PVvXK6dwd5l06317tHnvfR92JUY/eFJbklMEsoDZNw++u6tVAwsJxpYeresQ?= =?utf-8?q?nW13x+/RiE0w1PheQ7ztdw/azhNNYyW8wAPla8G7zVHogmr1WHn8OKOinkx3m25AC?= =?utf-8?q?7ADC34BwXCp17VuC5tqWHrnP2s4CuMHHecC6J6wZlgA8FbeEAyWMtfYFT2Mtkq4Vl?= =?utf-8?q?3EnPsDJhv/Kx21IzYQGK1hJL793CXJ99etbme9cRsHzwEdsk3dKVKNAh2VufY4Kv+?= =?utf-8?q?luV7CTM148XtL31I2dvMIturgeZeBQL4p+R+uA7VH/u1bSbHE67huIve8FRSDIqED?= =?utf-8?q?ELnddKdb6LnMVZzQGDCtnnflLYJkWienJw8kJpHJ6eQX6Xt+EThjf9M4sXj33bIlP?= =?utf-8?q?tgYXD5QE3gyoj0WIgOAOaFhfiuDjbb2m9uuZaSonXs3bg8vN0hiAqMvZr5QJn21uV?= =?utf-8?q?qAMk0bvBP4rD1D/JPU3jhZTAaWZM6RvoJSz7sXrF59yGy/VKKST6e1H6uTPXfdGMY?= =?utf-8?q?gabWk1DMdFz7F+68pZL7NaexcHXr/wt0eG0RdQvSsHyDnzr696dFbxrp1Xd7BvQgy?= =?utf-8?q?nbZYheh/Si24VTogF24oUq1VLA6r+bLvvRaYYfuNLY1aJXi/5SqC3HcGJvWNDIhnC?= =?utf-8?q?V9/tOm0XcYUHXdY4hiI0dIhJg0Heb4OXHl0f1y6poG2jy3keFw5NGSiHnYq2aLV/P?= =?utf-8?q?kTmFweeTIOdmVL7NwOqKEZhTDxs/sAfp3Lw=3D=3D?= X-OriginatorOrg: amd.com X-MS-Exchange-CrossTenant-Network-Message-Id: 9392f4b3-d6d9-4062-2df4-08d915520a42 X-MS-Exchange-CrossTenant-AuthSource: SN6PR12MB4623.namprd12.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 12 May 2021 14:27:15.4885 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: liFYfae1W6+s8/aIwfxYDzjFQSW5hCvgMd17cUKymWclMlydyqDz4/6rAhmhL9ZeOwG8Q1tLnq/uByJjuLgvNg== X-MS-Exchange-Transport-CrossTenantHeadersStamped: SA0PR12MB4576 X-BeenThere: dri-devel@lists.freedesktop.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Direct Rendering Infrastructure - Development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: gregkh@linuxfoundation.org, Felix.Kuehling@amd.com, helgaas@kernel.org, Alexander.Deucher@amd.com, =?utf-8?q?Christian_K=C3=B6nig?= Errors-To: dri-devel-bounces@lists.freedesktop.org Sender: "dri-devel" Use it to call disply code dependent on device->drv_data before it's set to NULL on device unplug v5: Move HW finilization into this callback to prevent MMIO accesses post cpi remove. Signed-off-by: Andrey Grodzovsky Acked-by: Christian König --- drivers/gpu/drm/amd/amdgpu/amdgpu_device.c | 59 +++++++++++++------ .../gpu/drm/amd/display/amdgpu_dm/amdgpu_dm.c | 12 +++- drivers/gpu/drm/amd/include/amd_shared.h | 2 + 3 files changed, 52 insertions(+), 21 deletions(-) diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c index 3760ce7d8ff8..18598eda18f6 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c @@ -2558,34 +2558,26 @@ static int amdgpu_device_ip_late_init(struct amdgpu_device *adev) return 0; } -/** - * amdgpu_device_ip_fini - run fini for hardware IPs - * - * @adev: amdgpu_device pointer - * - * Main teardown pass for hardware IPs. The list of all the hardware - * IPs that make up the asic is walked and the hw_fini and sw_fini callbacks - * are run. hw_fini tears down the hardware associated with each IP - * and sw_fini tears down any software state associated with each IP. - * Returns 0 on success, negative error code on failure. - */ -static int amdgpu_device_ip_fini(struct amdgpu_device *adev) +static int amdgpu_device_ip_fini_early(struct amdgpu_device *adev) { int i, r; - if (amdgpu_sriov_vf(adev) && adev->virt.ras_init_done) - amdgpu_virt_release_ras_err_handler_data(adev); + for (i = 0; i < adev->num_ip_blocks; i++) { + if (!adev->ip_blocks[i].version->funcs->early_fini) + continue; - amdgpu_ras_pre_fini(adev); + r = adev->ip_blocks[i].version->funcs->early_fini((void *)adev); + if (r) { + DRM_DEBUG("early_fini of IP block <%s> failed %d\n", + adev->ip_blocks[i].version->funcs->name, r); + } + } - if (adev->gmc.xgmi.num_physical_nodes > 1) - amdgpu_xgmi_remove_device(adev); + amdgpu_amdkfd_suspend(adev, false); amdgpu_device_set_pg_state(adev, AMD_PG_STATE_UNGATE); amdgpu_device_set_cg_state(adev, AMD_CG_STATE_UNGATE); - amdgpu_amdkfd_device_fini(adev); - /* need to disable SMC first */ for (i = 0; i < adev->num_ip_blocks; i++) { if (!adev->ip_blocks[i].status.hw) @@ -2616,6 +2608,33 @@ static int amdgpu_device_ip_fini(struct amdgpu_device *adev) adev->ip_blocks[i].status.hw = false; } + return 0; +} + +/** + * amdgpu_device_ip_fini - run fini for hardware IPs + * + * @adev: amdgpu_device pointer + * + * Main teardown pass for hardware IPs. The list of all the hardware + * IPs that make up the asic is walked and the hw_fini and sw_fini callbacks + * are run. hw_fini tears down the hardware associated with each IP + * and sw_fini tears down any software state associated with each IP. + * Returns 0 on success, negative error code on failure. + */ +static int amdgpu_device_ip_fini(struct amdgpu_device *adev) +{ + int i, r; + + if (amdgpu_sriov_vf(adev) && adev->virt.ras_init_done) + amdgpu_virt_release_ras_err_handler_data(adev); + + amdgpu_ras_pre_fini(adev); + + if (adev->gmc.xgmi.num_physical_nodes > 1) + amdgpu_xgmi_remove_device(adev); + + amdgpu_amdkfd_device_fini_sw(adev); for (i = adev->num_ip_blocks - 1; i >= 0; i--) { if (!adev->ip_blocks[i].status.sw) @@ -3683,6 +3702,8 @@ void amdgpu_device_fini_hw(struct amdgpu_device *adev) amdgpu_fbdev_fini(adev); amdgpu_irq_fini_hw(adev); + + amdgpu_device_ip_fini_early(adev); } void amdgpu_device_fini_sw(struct amdgpu_device *adev) diff --git a/drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm.c b/drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm.c index 296704ce3768..6c2c6a51ce6c 100644 --- a/drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm.c +++ b/drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm.c @@ -1251,6 +1251,15 @@ static int amdgpu_dm_init(struct amdgpu_device *adev) return -EINVAL; } +static int amdgpu_dm_early_fini(void *handle) +{ + struct amdgpu_device *adev = (struct amdgpu_device *)handle; + + amdgpu_dm_audio_fini(adev); + + return 0; +} + static void amdgpu_dm_fini(struct amdgpu_device *adev) { int i; @@ -1259,8 +1268,6 @@ static void amdgpu_dm_fini(struct amdgpu_device *adev) drm_encoder_cleanup(&adev->dm.mst_encoders[i].base); } - amdgpu_dm_audio_fini(adev); - amdgpu_dm_destroy_drm_device(&adev->dm); #if defined(CONFIG_DRM_AMD_SECURE_DISPLAY) @@ -2298,6 +2305,7 @@ static const struct amd_ip_funcs amdgpu_dm_funcs = { .late_init = dm_late_init, .sw_init = dm_sw_init, .sw_fini = dm_sw_fini, + .early_fini = amdgpu_dm_early_fini, .hw_init = dm_hw_init, .hw_fini = dm_hw_fini, .suspend = dm_suspend, diff --git a/drivers/gpu/drm/amd/include/amd_shared.h b/drivers/gpu/drm/amd/include/amd_shared.h index 43ed6291b2b8..1ad56da486e4 100644 --- a/drivers/gpu/drm/amd/include/amd_shared.h +++ b/drivers/gpu/drm/amd/include/amd_shared.h @@ -240,6 +240,7 @@ enum amd_dpm_forced_level; * @late_init: sets up late driver/hw state (post hw_init) - Optional * @sw_init: sets up driver state, does not configure hw * @sw_fini: tears down driver state, does not configure hw + * @early_fini: tears down stuff before dev detached from driver * @hw_init: sets up the hw state * @hw_fini: tears down the hw state * @late_fini: final cleanup @@ -268,6 +269,7 @@ struct amd_ip_funcs { int (*late_init)(void *handle); int (*sw_init)(void *handle); int (*sw_fini)(void *handle); + int (*early_fini)(void *handle); int (*hw_init)(void *handle); int (*hw_fini)(void *handle); void (*late_fini)(void *handle); From patchwork Wed May 12 14:26:37 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrey Grodzovsky X-Patchwork-Id: 12253877 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-18.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,MSGID_FROM_MTA_HEADER,SPF_HELO_NONE, SPF_PASS,USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 6C888C433ED for ; Wed, 12 May 2021 14:27:27 +0000 (UTC) Received: from gabe.freedesktop.org (gabe.freedesktop.org [131.252.210.177]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 1734B613E9 for ; Wed, 12 May 2021 14:27:27 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 1734B613E9 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=amd.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=dri-devel-bounces@lists.freedesktop.org Received: from gabe.freedesktop.org (localhost [127.0.0.1]) by gabe.freedesktop.org (Postfix) with ESMTP id 98F6B6E804; Wed, 12 May 2021 14:27:22 +0000 (UTC) Received: from NAM04-DM6-obe.outbound.protection.outlook.com (mail-dm6nam08on2060.outbound.protection.outlook.com [40.107.102.60]) by gabe.freedesktop.org (Postfix) with ESMTPS id E1D336EC05; Wed, 12 May 2021 14:27:20 +0000 (UTC) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=ieHFv+rfK5AZWoqRmsGIjaFM21mm9Vu8QHpZYyVTFKKA2+54P0DSg7gj0nh/rDAsB6lENp8qh30LTdG5wlFzqn1vlrtN2Wozsx1TFiSbxqh75z+wmxB0QQDAy04SGROJBxEMhyQkWR1v5XGiCib8Wg/vvjAaID7vOHd5WAeyz6haAUpCX4ka1la79CZ9Y88/opB3BcfWFJV4GJBK1bMQSmO6iwNJ8zD3rWrD5SDYykpy3ZMTmGnp0+LWhn/+Zqyb43c8nTBG6E4JUGFzqRrAKl5az7Lfiq/+GvNSzzsfR9rjptu5xFh73aKlFtEUKMJlkg9sHJSlvOawMzWwwaX9Yg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=s0dwKENLsURuCDAMt3HBqU82krOzXD1UOWvf5Glanbw=; b=etNqqK5Zeqn/2NXyt8h/QTwxNmpEQdzIzBodmKv915DQAGxLJ8HQsw7RuWOMQnYXcV48lqyACK5X3jn50BzTYbS5M99wvwA+jpWTRUy31EB601WNCX3k4zGFVmM6M7LA9ncjFGkmD+SUqd2eGG6TZFAbbkkK80yrACvOfn7sZuN+R3vHgo3Xgre0JFjwNVbhXwO1vSGDgw9yEx6xdBQm+w76XIJwBRSpEqSnHvmxMy+erEML/pVPIosRzGr5Jd88Le6U6fEDQ83EWv+CvcEicOJnwV3sDoRAww9D6ak6gp3jEMsN900eBTlXkts4KS12BEloBiwPf+KHD8Qq25tCAw== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=amd.com; dmarc=pass action=none header.from=amd.com; dkim=pass header.d=amd.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=s0dwKENLsURuCDAMt3HBqU82krOzXD1UOWvf5Glanbw=; b=XS1cqmMkuvrEafch1LlsiNBGe1KCXxbdUMskhu37LBDb57Lf/tnOXpM3L8CoNbysAtM+MRE2ntngHEirsu7ZO/Z5bplpYHPS3tOLKL0fwzt+TutvHNZFwmHTa3hDXuoi48pZysSblWEC84ypNvYOGdYQNFqU2G5C/o37n/9ywBw= Authentication-Results: lists.freedesktop.org; dkim=none (message not signed) header.d=none; lists.freedesktop.org; dmarc=none action=none header.from=amd.com; Received: from SN6PR12MB4623.namprd12.prod.outlook.com (2603:10b6:805:e9::17) by SA0PR12MB4576.namprd12.prod.outlook.com (2603:10b6:806:93::13) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4108.27; Wed, 12 May 2021 14:27:19 +0000 Received: from SN6PR12MB4623.namprd12.prod.outlook.com ([fe80::ad51:8c49:b171:856c]) by SN6PR12MB4623.namprd12.prod.outlook.com ([fe80::ad51:8c49:b171:856c%7]) with mapi id 15.20.4129.026; Wed, 12 May 2021 14:27:19 +0000 From: Andrey Grodzovsky To: dri-devel@lists.freedesktop.org, amd-gfx@lists.freedesktop.org, linux-pci@vger.kernel.org, ckoenig.leichtzumerken@gmail.com, daniel.vetter@ffwll.ch, Harry.Wentland@amd.com Subject: [PATCH v7 05/16] drm/amdgpu: Handle IOMMU enabled case. Date: Wed, 12 May 2021 10:26:37 -0400 Message-Id: <20210512142648.666476-6-andrey.grodzovsky@amd.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210512142648.666476-1-andrey.grodzovsky@amd.com> References: <20210512142648.666476-1-andrey.grodzovsky@amd.com> X-Originating-IP: [2607:fea8:3edf:49b0:7576:4f76:97d8:1487] X-ClientProxiedBy: YTOPR0101CA0032.CANPRD01.PROD.OUTLOOK.COM (2603:10b6:b00:15::45) To SN6PR12MB4623.namprd12.prod.outlook.com (2603:10b6:805:e9::17) MIME-Version: 1.0 X-MS-Exchange-MessageSentRepresentingType: 1 Received: from agrodzovsky-All-Series.hitronhub.home (2607:fea8:3edf:49b0:7576:4f76:97d8:1487) by YTOPR0101CA0032.CANPRD01.PROD.OUTLOOK.COM (2603:10b6:b00:15::45) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4108.30 via Frontend Transport; Wed, 12 May 2021 14:27:18 +0000 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 768fc8e9-dd41-4618-e0d4-08d915520c78 X-MS-TrafficTypeDiagnostic: SA0PR12MB4576: X-MS-Exchange-Transport-Forked: True X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:3826; X-MS-Exchange-SenderADCheck: 1 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: FR6rPtghGwM3J/XyYqrsIWf99XEkkziKNH+qJJNwfpZD+ba6jiOPrbSFNWXDo7ad4l+7pEpucLUZTCcLmLNS2IOqlZAtBFDn+iaR97tlyiUJGtNhvvK7Zv/qomDYUCLOX39Xhh+xYFSyDNzkQ3qb/mY3FwVI43hm4S2tQITxg4SmHUg5zBRN+27yR/CvkvjUc+n8grF1os4lAMiH8cisCP+XIupG/5q2jy9/mKrwC1tXNPFn8To2wpaVNjP38jRdDYdq6n//xCfMMghYu89/mi9fN/1teC/Y1b6nplbnAczwvzIP/c4EeFMpqSZwbIJLzfqGmMCfl1zp6iVUyMobjw0+a1sN3dwIOYqOu3rTRXDOuJga+BRHpoAwagPxKGGc4kIlmzReFBsBLUSEWpHPgimTOwVWlk7Hgg7ueEBIwzrGNmKtCwIm9lzmGxNOYkenYB3zuzi73D6ixQlTiHXUkzXbdq1f/hOSAdMQGmTJoKxANaduX3JJRRJu9+w3I21dBDKvoNpRGXLNRw6LzY6wKFrnO9QjHeoxbZHIAP5B4Ul0iffOqCyLU6lqiCieTffproW9JnZ8+jAVleDz4HfFfTMPj4L0dOyKnp9/CNiwNyY= X-Forefront-Antispam-Report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:SN6PR12MB4623.namprd12.prod.outlook.com; PTR:; CAT:NONE; SFS:(4636009)(376002)(39860400002)(366004)(346002)(396003)(136003)(38100700002)(5660300002)(1076003)(6666004)(186003)(316002)(16526019)(8676002)(83380400001)(66476007)(2906002)(6636002)(86362001)(30864003)(52116002)(36756003)(2616005)(66946007)(66556008)(8936002)(6512007)(6506007)(44832011)(478600001)(4326008)(6486002); DIR:OUT; SFP:1101; X-MS-Exchange-AntiSpam-MessageData: N3qV0GEhJxKNAaEmxbJpQJURRDDa8KIIA5qdLMWfSpC+KV9CSptgfQTu+96Za7dxCr47PTJPLUqryDooCn1TKGHB2tR6ZXA1EpZDkPJwT0hLOaC91XS1ioQdq7pH2AeYz3AazW5iB/Igu9P1fhYLIcX0P4jVTUfyB6dYGjNMj5TfJdJmYKWBpC9LQD2C/qgZl6LSJpcvA/4mZPHi4GZZjFBubMoHRxTW0DFwhEf7/FGlldHMTll7Rcvgt+DiRBVPNJH1SxAQIu0VJ1QpL+SBYGlfZ0uLS3WkSDgdWyNYnK1NP6QY6buyPiGyCGq7MhHQV3SS0GBuCiqzuHcOCFsvZbnHjgoVjlKywvK05IlOGhSGVSkArnaj8hrGC07zzE9Izx0tdhPQjM5F0/Src+c45zj6v4lVxvTKgthqsBK8wr/FfuhJdXnxB0O3920jUw7vwSc/Pdzwjc/oNNHtU1rIYZ0JenHVHWfG2o9twgOiyV0AKKnerI0DCHEV4XYpEpPUYosS1dks+GDWeZ1ueUZcB3CE08NipAo6gHK2LPwosNQB/NUOm2vRq2Qp3O3A5NQG0klNQBqBZHD4AhfRqijgfIvrnV9Sh7g0UxWeZt5Dhsvt2l96XiF/I41y+moWmb2tbjhCyrdBR//hA6k6zhCHwRCRO8EskkEiTcFo8nayo66WESXAMSgD09FyUFruDYuqjQ9Ywaogo+aymDRasUjkLJGLMeHxfWNfwJAcOPY/EG21kJA2xGlKxVJnCgi8ZW1F57/yMpR4aTJB10MTlGPxDd7yVVg+yy+TDLSRE+9pek5Akdtu9L3PMaPrPws7z4YkjrH5RYJtPGUQxABowdjoKxkBulD7u+NCi/YKHG2RU6H84fQpqhwlgC58Nv4lM0hUzqficSeqbGXC0aha4F5h8+FngyzYqNldKU74B/NaOQ6jpxIyQNp7DOmFKlwdkeGuu0nuU5XxucfNjk5xzIxj4Y746c59NazL6zwAd7tpJh3jCcbzJQZHyq9lfn2y8C1oSE9lP2W8FEnnXqSDioNjlxF4s3nOW3chelnW4njjATWZcr0hqIo7JYYP76k4XTqFejWbLJphfF6I5U+2pBNsImT028lk9PbaEUyCLYIrofgxegwc2PSQzjptdQ42BmEkOkg2TQ5yBRC75M92+RXeCPt1zPCCxur7lwBVn2S0zQWlsN6iqzHYXl/2tBDX+e4FgytgmtiwM1jfFEgGj5FJ2bl3TGzxrHXpXjnzmpa0vKuwRoHpm5Z+A+cxb6j5xh0k8AwNokhUzKvgqv42XW0t/zoOhko+N78uZcEnQ2gz8wPNksSn7YhTTYNjkPMwOvcqod+13l1LiRB97MY7kYcQKjYv16LXxzubntjnEJpugY9q+FCggFE9PGTWFQJTq213 X-OriginatorOrg: amd.com X-MS-Exchange-CrossTenant-Network-Message-Id: 768fc8e9-dd41-4618-e0d4-08d915520c78 X-MS-Exchange-CrossTenant-AuthSource: SN6PR12MB4623.namprd12.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 12 May 2021 14:27:19.2834 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: Qc2zuFjldOwwMW2isi6YVIvxnIcTurPbWXuQX6CGA7vVPxuYJ4Vd4FHjFoVrAWn6YMXoMYBnRkObz4C7LEvEhA== X-MS-Exchange-Transport-CrossTenantHeadersStamped: SA0PR12MB4576 X-BeenThere: dri-devel@lists.freedesktop.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Direct Rendering Infrastructure - Development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: gregkh@linuxfoundation.org, Felix.Kuehling@amd.com, helgaas@kernel.org, Alexander.Deucher@amd.com Errors-To: dri-devel-bounces@lists.freedesktop.org Sender: "dri-devel" Handle all DMA IOMMU gropup related dependencies before the group is removed. v5: Drop IOMMU notifier and switch to lockless call to ttm_tt_unpopulate v6: Drop the BO unamp list v7: Drop amdgpu_gart_fini In amdgpu_ih_ring_fini do uncinditional check (!ih->ring) to avoid freeing uniniitalized rings. Call amdgpu_ih_ring_fini unconditionally. Signed-off-by: Andrey Grodzovsky --- drivers/gpu/drm/amd/amdgpu/amdgpu_device.c | 4 ++-- drivers/gpu/drm/amd/amdgpu/amdgpu_gart.c | 14 +------------- drivers/gpu/drm/amd/amdgpu/amdgpu_gart.h | 2 +- drivers/gpu/drm/amd/amdgpu/amdgpu_ih.c | 6 ++++-- drivers/gpu/drm/amd/amdgpu/amdgpu_irq.c | 5 +++++ drivers/gpu/drm/amd/amdgpu/cik_ih.c | 1 - drivers/gpu/drm/amd/amdgpu/cz_ih.c | 1 - drivers/gpu/drm/amd/amdgpu/gmc_v10_0.c | 1 - drivers/gpu/drm/amd/amdgpu/gmc_v6_0.c | 1 - drivers/gpu/drm/amd/amdgpu/gmc_v7_0.c | 1 - drivers/gpu/drm/amd/amdgpu/gmc_v8_0.c | 1 - drivers/gpu/drm/amd/amdgpu/gmc_v9_0.c | 1 - drivers/gpu/drm/amd/amdgpu/iceland_ih.c | 1 - drivers/gpu/drm/amd/amdgpu/navi10_ih.c | 4 ---- drivers/gpu/drm/amd/amdgpu/si_ih.c | 1 - drivers/gpu/drm/amd/amdgpu/tonga_ih.c | 1 - drivers/gpu/drm/amd/amdgpu/vega10_ih.c | 4 ---- drivers/gpu/drm/amd/amdgpu/vega20_ih.c | 4 ---- 18 files changed, 13 insertions(+), 40 deletions(-) diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c index 18598eda18f6..a0bff4713672 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c @@ -3256,7 +3256,6 @@ static const struct attribute *amdgpu_dev_attributes[] = { NULL }; - /** * amdgpu_device_init - initialize the driver * @@ -3698,12 +3697,13 @@ void amdgpu_device_fini_hw(struct amdgpu_device *adev) amdgpu_ucode_sysfs_fini(adev); sysfs_remove_files(&adev->dev->kobj, amdgpu_dev_attributes); - amdgpu_fbdev_fini(adev); amdgpu_irq_fini_hw(adev); amdgpu_device_ip_fini_early(adev); + + amdgpu_gart_dummy_page_fini(adev); } void amdgpu_device_fini_sw(struct amdgpu_device *adev) diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_gart.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_gart.c index c5a9a4fb10d2..6460cf723f0a 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_gart.c +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_gart.c @@ -92,7 +92,7 @@ static int amdgpu_gart_dummy_page_init(struct amdgpu_device *adev) * * Frees the dummy page used by the driver (all asics). */ -static void amdgpu_gart_dummy_page_fini(struct amdgpu_device *adev) +void amdgpu_gart_dummy_page_fini(struct amdgpu_device *adev) { if (!adev->dummy_page_addr) return; @@ -365,15 +365,3 @@ int amdgpu_gart_init(struct amdgpu_device *adev) return 0; } - -/** - * amdgpu_gart_fini - tear down the driver info for managing the gart - * - * @adev: amdgpu_device pointer - * - * Tear down the gart driver info and free the dummy page (all asics). - */ -void amdgpu_gart_fini(struct amdgpu_device *adev) -{ - amdgpu_gart_dummy_page_fini(adev); -} diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_gart.h b/drivers/gpu/drm/amd/amdgpu/amdgpu_gart.h index a25fe97b0196..030b9d4c736a 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_gart.h +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_gart.h @@ -57,7 +57,7 @@ void amdgpu_gart_table_vram_free(struct amdgpu_device *adev); int amdgpu_gart_table_vram_pin(struct amdgpu_device *adev); void amdgpu_gart_table_vram_unpin(struct amdgpu_device *adev); int amdgpu_gart_init(struct amdgpu_device *adev); -void amdgpu_gart_fini(struct amdgpu_device *adev); +void amdgpu_gart_dummy_page_fini(struct amdgpu_device *adev); int amdgpu_gart_unbind(struct amdgpu_device *adev, uint64_t offset, int pages); int amdgpu_gart_map(struct amdgpu_device *adev, uint64_t offset, diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_ih.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_ih.c index faaa6aa2faaf..433469ace6f4 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_ih.c +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_ih.c @@ -115,9 +115,11 @@ int amdgpu_ih_ring_init(struct amdgpu_device *adev, struct amdgpu_ih_ring *ih, */ void amdgpu_ih_ring_fini(struct amdgpu_device *adev, struct amdgpu_ih_ring *ih) { + + if (!ih->ring) + return; + if (ih->use_bus_addr) { - if (!ih->ring) - return; /* add 8 bytes for the rptr/wptr shadows and * add them to the end of the ring allocation. diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_irq.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_irq.c index 233b64dab94b..32ce0e679dc7 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_irq.c +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_irq.c @@ -361,6 +361,11 @@ void amdgpu_irq_fini_hw(struct amdgpu_device *adev) if (!amdgpu_device_has_dc_support(adev)) flush_work(&adev->hotplug_work); } + + amdgpu_ih_ring_fini(adev, &adev->irq.ih_soft); + amdgpu_ih_ring_fini(adev, &adev->irq.ih); + amdgpu_ih_ring_fini(adev, &adev->irq.ih1); + amdgpu_ih_ring_fini(adev, &adev->irq.ih2); } /** diff --git a/drivers/gpu/drm/amd/amdgpu/cik_ih.c b/drivers/gpu/drm/amd/amdgpu/cik_ih.c index 183d44a6583c..df385ffc9768 100644 --- a/drivers/gpu/drm/amd/amdgpu/cik_ih.c +++ b/drivers/gpu/drm/amd/amdgpu/cik_ih.c @@ -310,7 +310,6 @@ static int cik_ih_sw_fini(void *handle) struct amdgpu_device *adev = (struct amdgpu_device *)handle; amdgpu_irq_fini_sw(adev); - amdgpu_ih_ring_fini(adev, &adev->irq.ih); amdgpu_irq_remove_domain(adev); return 0; diff --git a/drivers/gpu/drm/amd/amdgpu/cz_ih.c b/drivers/gpu/drm/amd/amdgpu/cz_ih.c index d32743949003..b8c47e0cf37a 100644 --- a/drivers/gpu/drm/amd/amdgpu/cz_ih.c +++ b/drivers/gpu/drm/amd/amdgpu/cz_ih.c @@ -302,7 +302,6 @@ static int cz_ih_sw_fini(void *handle) struct amdgpu_device *adev = (struct amdgpu_device *)handle; amdgpu_irq_fini_sw(adev); - amdgpu_ih_ring_fini(adev, &adev->irq.ih); amdgpu_irq_remove_domain(adev); return 0; diff --git a/drivers/gpu/drm/amd/amdgpu/gmc_v10_0.c b/drivers/gpu/drm/amd/amdgpu/gmc_v10_0.c index 2bfd620576f2..5e8bfcdd422e 100644 --- a/drivers/gpu/drm/amd/amdgpu/gmc_v10_0.c +++ b/drivers/gpu/drm/amd/amdgpu/gmc_v10_0.c @@ -954,7 +954,6 @@ static int gmc_v10_0_sw_init(void *handle) static void gmc_v10_0_gart_fini(struct amdgpu_device *adev) { amdgpu_gart_table_vram_free(adev); - amdgpu_gart_fini(adev); } static int gmc_v10_0_sw_fini(void *handle) diff --git a/drivers/gpu/drm/amd/amdgpu/gmc_v6_0.c b/drivers/gpu/drm/amd/amdgpu/gmc_v6_0.c index 405d6ad09022..0e81e03e9b49 100644 --- a/drivers/gpu/drm/amd/amdgpu/gmc_v6_0.c +++ b/drivers/gpu/drm/amd/amdgpu/gmc_v6_0.c @@ -898,7 +898,6 @@ static int gmc_v6_0_sw_fini(void *handle) amdgpu_vm_manager_fini(adev); amdgpu_gart_table_vram_free(adev); amdgpu_bo_fini(adev); - amdgpu_gart_fini(adev); release_firmware(adev->gmc.fw); adev->gmc.fw = NULL; diff --git a/drivers/gpu/drm/amd/amdgpu/gmc_v7_0.c b/drivers/gpu/drm/amd/amdgpu/gmc_v7_0.c index 210ada2289ec..0795ea736573 100644 --- a/drivers/gpu/drm/amd/amdgpu/gmc_v7_0.c +++ b/drivers/gpu/drm/amd/amdgpu/gmc_v7_0.c @@ -1085,7 +1085,6 @@ static int gmc_v7_0_sw_fini(void *handle) kfree(adev->gmc.vm_fault_info); amdgpu_gart_table_vram_free(adev); amdgpu_bo_fini(adev); - amdgpu_gart_fini(adev); release_firmware(adev->gmc.fw); adev->gmc.fw = NULL; diff --git a/drivers/gpu/drm/amd/amdgpu/gmc_v8_0.c b/drivers/gpu/drm/amd/amdgpu/gmc_v8_0.c index c1bd190841f8..dbf2e5472069 100644 --- a/drivers/gpu/drm/amd/amdgpu/gmc_v8_0.c +++ b/drivers/gpu/drm/amd/amdgpu/gmc_v8_0.c @@ -1194,7 +1194,6 @@ static int gmc_v8_0_sw_fini(void *handle) kfree(adev->gmc.vm_fault_info); amdgpu_gart_table_vram_free(adev); amdgpu_bo_fini(adev); - amdgpu_gart_fini(adev); release_firmware(adev->gmc.fw); adev->gmc.fw = NULL; diff --git a/drivers/gpu/drm/amd/amdgpu/gmc_v9_0.c b/drivers/gpu/drm/amd/amdgpu/gmc_v9_0.c index c82d82da2c73..5ed0adae05cf 100644 --- a/drivers/gpu/drm/amd/amdgpu/gmc_v9_0.c +++ b/drivers/gpu/drm/amd/amdgpu/gmc_v9_0.c @@ -1601,7 +1601,6 @@ static int gmc_v9_0_sw_fini(void *handle) amdgpu_gart_table_vram_free(adev); amdgpu_bo_unref(&adev->gmc.pdb0_bo); amdgpu_bo_fini(adev); - amdgpu_gart_fini(adev); return 0; } diff --git a/drivers/gpu/drm/amd/amdgpu/iceland_ih.c b/drivers/gpu/drm/amd/amdgpu/iceland_ih.c index da96c6013477..ddfe4eaeea05 100644 --- a/drivers/gpu/drm/amd/amdgpu/iceland_ih.c +++ b/drivers/gpu/drm/amd/amdgpu/iceland_ih.c @@ -301,7 +301,6 @@ static int iceland_ih_sw_fini(void *handle) struct amdgpu_device *adev = (struct amdgpu_device *)handle; amdgpu_irq_fini_sw(adev); - amdgpu_ih_ring_fini(adev, &adev->irq.ih); amdgpu_irq_remove_domain(adev); return 0; diff --git a/drivers/gpu/drm/amd/amdgpu/navi10_ih.c b/drivers/gpu/drm/amd/amdgpu/navi10_ih.c index 5eea4550b856..941d464a2b47 100644 --- a/drivers/gpu/drm/amd/amdgpu/navi10_ih.c +++ b/drivers/gpu/drm/amd/amdgpu/navi10_ih.c @@ -570,10 +570,6 @@ static int navi10_ih_sw_fini(void *handle) struct amdgpu_device *adev = (struct amdgpu_device *)handle; amdgpu_irq_fini_sw(adev); - amdgpu_ih_ring_fini(adev, &adev->irq.ih_soft); - amdgpu_ih_ring_fini(adev, &adev->irq.ih2); - amdgpu_ih_ring_fini(adev, &adev->irq.ih1); - amdgpu_ih_ring_fini(adev, &adev->irq.ih); return 0; } diff --git a/drivers/gpu/drm/amd/amdgpu/si_ih.c b/drivers/gpu/drm/amd/amdgpu/si_ih.c index 751307f3252c..9a24f17a5750 100644 --- a/drivers/gpu/drm/amd/amdgpu/si_ih.c +++ b/drivers/gpu/drm/amd/amdgpu/si_ih.c @@ -176,7 +176,6 @@ static int si_ih_sw_fini(void *handle) struct amdgpu_device *adev = (struct amdgpu_device *)handle; amdgpu_irq_fini_sw(adev); - amdgpu_ih_ring_fini(adev, &adev->irq.ih); return 0; } diff --git a/drivers/gpu/drm/amd/amdgpu/tonga_ih.c b/drivers/gpu/drm/amd/amdgpu/tonga_ih.c index 973d80ec7f6c..b08905d1c00f 100644 --- a/drivers/gpu/drm/amd/amdgpu/tonga_ih.c +++ b/drivers/gpu/drm/amd/amdgpu/tonga_ih.c @@ -313,7 +313,6 @@ static int tonga_ih_sw_fini(void *handle) struct amdgpu_device *adev = (struct amdgpu_device *)handle; amdgpu_irq_fini_sw(adev); - amdgpu_ih_ring_fini(adev, &adev->irq.ih); amdgpu_irq_remove_domain(adev); return 0; diff --git a/drivers/gpu/drm/amd/amdgpu/vega10_ih.c b/drivers/gpu/drm/amd/amdgpu/vega10_ih.c index dead9c2fbd4c..32ec4b8e806a 100644 --- a/drivers/gpu/drm/amd/amdgpu/vega10_ih.c +++ b/drivers/gpu/drm/amd/amdgpu/vega10_ih.c @@ -514,10 +514,6 @@ static int vega10_ih_sw_fini(void *handle) struct amdgpu_device *adev = (struct amdgpu_device *)handle; amdgpu_irq_fini_sw(adev); - amdgpu_ih_ring_fini(adev, &adev->irq.ih_soft); - amdgpu_ih_ring_fini(adev, &adev->irq.ih2); - amdgpu_ih_ring_fini(adev, &adev->irq.ih1); - amdgpu_ih_ring_fini(adev, &adev->irq.ih); return 0; } diff --git a/drivers/gpu/drm/amd/amdgpu/vega20_ih.c b/drivers/gpu/drm/amd/amdgpu/vega20_ih.c index 58993ae1fe11..f51dfc38ac65 100644 --- a/drivers/gpu/drm/amd/amdgpu/vega20_ih.c +++ b/drivers/gpu/drm/amd/amdgpu/vega20_ih.c @@ -566,10 +566,6 @@ static int vega20_ih_sw_fini(void *handle) struct amdgpu_device *adev = (struct amdgpu_device *)handle; amdgpu_irq_fini_sw(adev); - amdgpu_ih_ring_fini(adev, &adev->irq.ih_soft); - amdgpu_ih_ring_fini(adev, &adev->irq.ih2); - amdgpu_ih_ring_fini(adev, &adev->irq.ih1); - amdgpu_ih_ring_fini(adev, &adev->irq.ih); return 0; } From patchwork Wed May 12 14:26:38 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Andrey Grodzovsky X-Patchwork-Id: 12253881 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-18.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,MSGID_FROM_MTA_HEADER,SPF_HELO_NONE, SPF_PASS,USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 273C1C43460 for ; Wed, 12 May 2021 14:27:31 +0000 (UTC) Received: from gabe.freedesktop.org (gabe.freedesktop.org [131.252.210.177]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 3E73B61418 for ; Wed, 12 May 2021 14:27:29 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 3E73B61418 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=amd.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=dri-devel-bounces@lists.freedesktop.org Received: from gabe.freedesktop.org (localhost [127.0.0.1]) by gabe.freedesktop.org (Postfix) with ESMTP id CEAD76EC12; Wed, 12 May 2021 14:27:25 +0000 (UTC) Received: from NAM12-BN8-obe.outbound.protection.outlook.com (mail-bn8nam12on2086.outbound.protection.outlook.com [40.107.237.86]) by gabe.freedesktop.org (Postfix) with ESMTPS id AE08D6EC0C; Wed, 12 May 2021 14:27:23 +0000 (UTC) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=SpNDX9420W8P1KRsWpRaOfmCxf4mLm4PL9PeO1OKVRM3yiznowwyIgzRdC7ys4m1tykbbGDNgyEHxzf9ePyRggWncnxLxI2VKP3XePpOLhr6EASbYOVLSOiHIOxMEqgaiO1MXSFnDorLtjk+LBKRk/G+7SYyk0M/dObbFGBirWw0UfNlSrVJh20Gu9s4Tz/y9P8FR61MVEL0iEaAM42oylFoIvi4TicRKr05TWTdJA08IJN6JAc+R16HXG5fIDXjN8/U3d/VFqRIU2RKNauWqEHFLRmt4jc7zgc9OVScEryjdxwCy7wJe7uta5Mughzt1BgVFAAcn9qYRGrXeWF7DA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=ImiQhpPoXfItMc84rz4U+sl7ukQBW3gycLm8/fXkr9o=; b=gWTzaIXAJ31jbrWawfH1P/KCJs9mhHnqIIS0Dw+Q1nJ4JNcqJhh5Vn8CnZqqUNwpihQbojvYDH7gEzV0ldovc7zZmzuau26CjS3JpAm/zuQ86qAvWkOEONPc2gcWPp3nx1FTepclH1jdO5MNjHKpbtPKWI9IpLfrZziKKyIwrrBELKnGEn7K3GzNmwBbFVSbpRF2jFzSTGPFYZPvgWtUEgjc1LUsEQ+/ZO6fu4Zu0A+2fp68RWl1TxWHAz2JVlk/TjNFHGn8zw909sd4N0koaEZT/dkR5LBPSppWtwG9mggti5sazE2RLrykIZTGgubnMvuw7/evnVpSE9ipvGhVrA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=amd.com; dmarc=pass action=none header.from=amd.com; dkim=pass header.d=amd.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=ImiQhpPoXfItMc84rz4U+sl7ukQBW3gycLm8/fXkr9o=; b=x8z4WG+qWMtdNnH/khw7DZxsDYMTaE9m5Hl05qGM+1+QgNG6Lsrrp2s26o3QjusmJrtld/g5iqMtbV6PLNnhbOfBnO/KbR5zwQo+CLN8+ni8F9ng72i+xURnrV74UStwJSuouHaGSjdNmr6jycpujzMT1nbOJN+xKCtGIzSmM8k= Authentication-Results: lists.freedesktop.org; dkim=none (message not signed) header.d=none; lists.freedesktop.org; dmarc=none action=none header.from=amd.com; Received: from SN6PR12MB4623.namprd12.prod.outlook.com (2603:10b6:805:e9::17) by SN6PR12MB2845.namprd12.prod.outlook.com (2603:10b6:805:75::33) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4129.25; Wed, 12 May 2021 14:27:21 +0000 Received: from SN6PR12MB4623.namprd12.prod.outlook.com ([fe80::ad51:8c49:b171:856c]) by SN6PR12MB4623.namprd12.prod.outlook.com ([fe80::ad51:8c49:b171:856c%7]) with mapi id 15.20.4129.026; Wed, 12 May 2021 14:27:21 +0000 From: Andrey Grodzovsky To: dri-devel@lists.freedesktop.org, amd-gfx@lists.freedesktop.org, linux-pci@vger.kernel.org, ckoenig.leichtzumerken@gmail.com, daniel.vetter@ffwll.ch, Harry.Wentland@amd.com Subject: [PATCH v7 06/16] drm/amdgpu: Remap all page faults to per process dummy page. Date: Wed, 12 May 2021 10:26:38 -0400 Message-Id: <20210512142648.666476-7-andrey.grodzovsky@amd.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210512142648.666476-1-andrey.grodzovsky@amd.com> References: <20210512142648.666476-1-andrey.grodzovsky@amd.com> X-Originating-IP: [2607:fea8:3edf:49b0:7576:4f76:97d8:1487] X-ClientProxiedBy: YTOPR0101CA0032.CANPRD01.PROD.OUTLOOK.COM (2603:10b6:b00:15::45) To SN6PR12MB4623.namprd12.prod.outlook.com (2603:10b6:805:e9::17) MIME-Version: 1.0 X-MS-Exchange-MessageSentRepresentingType: 1 Received: from agrodzovsky-All-Series.hitronhub.home (2607:fea8:3edf:49b0:7576:4f76:97d8:1487) by YTOPR0101CA0032.CANPRD01.PROD.OUTLOOK.COM (2603:10b6:b00:15::45) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4108.30 via Frontend Transport; Wed, 12 May 2021 14:27:20 +0000 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 4f0ef048-18cd-4925-9568-08d915520dd4 X-MS-TrafficTypeDiagnostic: SN6PR12MB2845: X-MS-Exchange-Transport-Forked: True X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:2000; X-MS-Exchange-SenderADCheck: 1 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: w6+Kysu0z59GAH1eWP+zeLVLHVStGr0KRMW4OWeORaHz/JJIXMluUbNXL0Udsr3d4ZOjwNr9fr/e9CdNBFrM4npUk7yF1fxNvYOtKImJZahveSiK1hqdgZlR6MtBeExDdtSWLSsd4DjJ9oYsG5ayZICu2hFrwHgZRMeLJKtxE1gP8zy5EMV8BAjbhOqZX2PLUrU/EXUVCtQbwakEH+Y8Iwhy0LKUrmvPoLJVi2HjvTaCRx1g7qNLwBSAG7yhIC5WSME5P2+309D99g46TxVSpmd61ln9HH+j5pTDGh2cnwGSEqjyADyQFxbTeDBpse7lOUX8zOe3ZlCp/hupZ3+Lxc3GVp+PWKYg4lZ1dZg9O+GxCHPILwO+zVJ7FYD3sJbCEcS1mUWTRFpYZbJgrrjI0ANDOncPOd2GeuUQVg2ZkR77jC+bb9ts/6R3GjRUIvDprt81R/2ZroahydZJo/X9zNlx8MXV+jESHh+cxR+PiIBo4TXZWwGwD2jYWB23S21JmfzHz38LJ5IrTsa/77ydV0/HZryf5XZ9KJZaRZowyrv6sN4CzTDR/ZuU8W6s5UQvn5H0gNDZI4X+ldmUgrFsdXldSagAz/yPEDiPmK0fpHo= X-Forefront-Antispam-Report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:SN6PR12MB4623.namprd12.prod.outlook.com; PTR:; CAT:NONE; SFS:(4636009)(376002)(39860400002)(396003)(346002)(136003)(366004)(6486002)(8676002)(54906003)(316002)(6512007)(16526019)(6636002)(478600001)(1076003)(186003)(4326008)(44832011)(2906002)(6506007)(8936002)(5660300002)(2616005)(38100700002)(66574015)(36756003)(86362001)(6666004)(66476007)(66946007)(66556008)(83380400001)(52116002); DIR:OUT; SFP:1101; X-MS-Exchange-AntiSpam-MessageData: =?utf-8?q?3P+HcZmqKXvEjGv+RDjcd0WsSeHgj5?= =?utf-8?q?tyty/xhNsjDG6xsKSDyKSBozXWKz8/MVr//64xNDcZXnh0S1clo3n+GxS4noXWutS?= =?utf-8?q?/kKb0iIT3VVWNwxRZhpQvT+WKLlnAaUrZDyqk4QvpbYU8Mv/AvugTQRWGTYr4hIOR?= =?utf-8?q?6fVX17ndHJuqgOhCCkm8X4nj6u9npZGZH2+38JX8brlDkkYvWE2bCfwYeWE9wrTDj?= =?utf-8?q?jD6rtcbjIjFlVbs5GC/IiHObhh9GzcRZ51s5CdAlDqd5t97OgWBsMK0y9padof8+4?= =?utf-8?q?7WPVP75jXchZvGU3VBTLydGude1JDooFW3VRKQYeyAGtX3Y8zKWOOw/XJjEzox/7d?= =?utf-8?q?d7oHgzm3j2vMct9UIHYAIH3AToutP0QMH+Nku2w6oni14QpPFBhpWB3bGeagTrL3+?= =?utf-8?q?Jh8lBZB3v6IE6K/k+wESaDbGQl5joT4r62FSRrAH+8SuXby0uo47vNGsOX5YvChul?= =?utf-8?q?geOPNSJTwNL4D5AgN457zPIC4L5egkVF/Wuex2/WpEViBZ0XzezmoMqQDcVkU5ESA?= =?utf-8?q?44oTESwuMKvKDzeEhKgAECPxeMGvonKDYwtZViRSSdY07F6SwKmIS+d9YKe3DNbZQ?= =?utf-8?q?WKvVByHJV54aMpkUaQGnstXLBsIFj5dsOPkQfDMQtVtukIBe2W3D6u+n/XoV/soU3?= =?utf-8?q?z1RYVqkDp7j8fLhOcpbEnstUJpiZEfF3VBoYwQS5aPnavlOqnZ7TViUnIW8hTFhm4?= =?utf-8?q?064Ixr9P/gaoCSKMK8VXcdUjHcceu1kESyI0ROnxdkG1OS6FO7BJakpnRcJN7UvjR?= =?utf-8?q?VAtmF/qEDbUuFJXkeGTv84Mhn4+RDEKLTyzKTcTx8cVe60O2aFIdjkDLlBplrcIQ4?= =?utf-8?q?NRZC0ZLH8Tr3B2+XSXsRJwZZd/NMCZDsLMOgBB78JwpJyqSEdNObrGMhkGOuTcTYU?= =?utf-8?q?725zsjw2F90yEHR1Pe/66xBKMgd8X9ZF3+bbp4uJrw66bA46ihxNGYY/KMKrhsWNw?= =?utf-8?q?O4rKL97FO5lk4WudDTX6+ERoCR5PmeT7SEFvXPez9opSnyjz8LoonMm/sGKz+tCJd?= =?utf-8?q?2pf84eJzhOS9xFj0oo5b3R5JLOGnZ0Ksn0ce3wE/Kk398ofz3XEL/zwnSa1ahxGIJ?= =?utf-8?q?GmUNNLrET8I/UkwSony86ibDe3IzmyIhRGt7wPAJrqv+lKhfaPm0d9y7QV93qJ6PS?= =?utf-8?q?25v3H9ESRvO/cmbCJABdFHx0zPZ+mv/QYbW1HycgBP3CudOgGOA/VeB62UaEHs5rZ?= =?utf-8?q?+R8cCt+MHWC6QJcB25N5pEGaLgtF0KPri4rFhkbdRTdxWmFYRJiRds+6jWZD3t0bT?= =?utf-8?q?GYgLgfKYMeRrvyLuewC8325p5UWeUTChXesA1v4lr9t2PyFD/j4VUwQGV/CfIszQk?= =?utf-8?q?8ezZXNQEBrJCD?= X-OriginatorOrg: amd.com X-MS-Exchange-CrossTenant-Network-Message-Id: 4f0ef048-18cd-4925-9568-08d915520dd4 X-MS-Exchange-CrossTenant-AuthSource: SN6PR12MB4623.namprd12.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 12 May 2021 14:27:21.4732 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: ZL1LntA3e3ALoedGO+5LZMq9jWQ7a09q2F+oWuGY77O3DbeMwwK1UXrzv+CHEhV4EU8uxcRRx2G0gJ9VkVzF+w== X-MS-Exchange-Transport-CrossTenantHeadersStamped: SN6PR12MB2845 X-BeenThere: dri-devel@lists.freedesktop.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Direct Rendering Infrastructure - Development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: gregkh@linuxfoundation.org, Felix.Kuehling@amd.com, helgaas@kernel.org, Alexander.Deucher@amd.com, =?utf-8?q?Christian_K=C3=B6nig?= Errors-To: dri-devel-bounces@lists.freedesktop.org Sender: "dri-devel" On device removal reroute all CPU mappings to dummy page per drm_file instance or imported GEM object. v4: Update for modified ttm_bo_vm_dummy_page Signed-off-by: Andrey Grodzovsky Reviewed-by: Christian König --- drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.c | 21 ++++++++++++++++----- 1 file changed, 16 insertions(+), 5 deletions(-) diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.c index 8c7ec09eb1a4..0d54e70278ca 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.c +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.c @@ -48,6 +48,7 @@ #include #include +#include #include "amdgpu.h" #include "amdgpu_object.h" @@ -1905,18 +1906,28 @@ void amdgpu_ttm_set_buffer_funcs_status(struct amdgpu_device *adev, bool enable) static vm_fault_t amdgpu_ttm_fault(struct vm_fault *vmf) { struct ttm_buffer_object *bo = vmf->vma->vm_private_data; + struct drm_device *ddev = bo->base.dev; vm_fault_t ret; + int idx; ret = ttm_bo_vm_reserve(bo, vmf); if (ret) return ret; - ret = amdgpu_bo_fault_reserve_notify(bo); - if (ret) - goto unlock; + if (drm_dev_enter(ddev, &idx)) { + ret = amdgpu_bo_fault_reserve_notify(bo); + if (ret) { + drm_dev_exit(idx); + goto unlock; + } - ret = ttm_bo_vm_fault_reserved(vmf, vmf->vma->vm_page_prot, - TTM_BO_VM_NUM_PREFAULT, 1); + ret = ttm_bo_vm_fault_reserved(vmf, vmf->vma->vm_page_prot, + TTM_BO_VM_NUM_PREFAULT, 1); + + drm_dev_exit(idx); + } else { + ret = ttm_bo_vm_dummy_page(vmf, vmf->vma->vm_page_prot); + } if (ret == VM_FAULT_RETRY && !(vmf->flags & FAULT_FLAG_RETRY_NOWAIT)) return ret; From patchwork Wed May 12 14:26:39 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrey Grodzovsky X-Patchwork-Id: 12253879 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-18.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,MSGID_FROM_MTA_HEADER,SPF_HELO_NONE, SPF_PASS,USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id A6DE3C43616 for ; Wed, 12 May 2021 14:27:31 +0000 (UTC) Received: from gabe.freedesktop.org (gabe.freedesktop.org [131.252.210.177]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 37ED2613E9 for ; Wed, 12 May 2021 14:27:31 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 37ED2613E9 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=amd.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=dri-devel-bounces@lists.freedesktop.org Received: from gabe.freedesktop.org (localhost [127.0.0.1]) by gabe.freedesktop.org (Postfix) with ESMTP id D8F786EC11; Wed, 12 May 2021 14:27:27 +0000 (UTC) Received: from NAM11-DM6-obe.outbound.protection.outlook.com (mail-dm6nam11on2072.outbound.protection.outlook.com [40.107.223.72]) by gabe.freedesktop.org (Postfix) with ESMTPS id CA0B96EC11; Wed, 12 May 2021 14:27:25 +0000 (UTC) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=CmkQr74taIp1Iw7KaANpctHA35hNoUXNDLkt4FDMNor/p4IEBV8X/4bj+V776nqpSAwdyhWi9YhgwmFyQ7D+p2cUwtQ1qRtrdpArR3jo2puWzB7i/YRfaM5nWO44XeZVeEjHltCp2vYqo+T8nBfxdN/mKP9ZBg4z7hoS9wqLhKIJGPI/p6QfAoqKpRxqDEkUS/1bpAbp6uo4ZPsJVtAJEq7W+XwBpolhDZM6og3yaOtOyM6KWeR2hAkARoafHirv7X0lYaYObKr0mIFiI9X6nzRZUFhVu7vkeCMQSWDOkK5qFW2Wk1OIjqkHDy+SKBMf5GXZFebMqkGey/Jck+C2PQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=Ib9JMbJcQ89S3QqH2/zvtmoA13b9zTTfzZEYQOgJxs4=; b=jpzUTj/AYiDtyZ5sUU16lKenuffbZedAu8u+j453UWRIHUN9Dw68Jy9FDuMyGSTBs4ETT/QTZRQ2S6OphfdQ7Ci4ESprgJQY6BBemRIYxvJ+Dkui3NgjN6a+jDyddOvKDDZ1E0VPgA+7P69VXz4IboD0emFfvbEmnPA6lqMe9DlGwRIE+xRNpwGbzXtVSpWKwTYb/qKnLhdPdJ3hAIElmLHItUMLIvaGTvjPafBaM8hhF8MxHk77AmJm89bT2Jc5ZHqE+XDavcxmWCTuAE4QPqZnbcFy97yeIL/eze/JCsHYH08PbDw6SHD5GrS6uGnABcg+sEilaJuCNq9mRgoY9w== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=amd.com; dmarc=pass action=none header.from=amd.com; dkim=pass header.d=amd.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=Ib9JMbJcQ89S3QqH2/zvtmoA13b9zTTfzZEYQOgJxs4=; b=dejeCAViCbUGefjdpMBG1HI/5+wnIryalwnu33JMf5cKQBeHCp5SDAPwiOKSP+Su9iltYdIfu3a+yxtww2i4w9h6kj8PlFXLSPZSewSNqZwMXaWT/lnEnRi97De9mKAPOhTkgt5hav3eoXmZd/JV2ixf4PiMXQtVq4sP7PCkg/k= Authentication-Results: lists.freedesktop.org; dkim=none (message not signed) header.d=none; lists.freedesktop.org; dmarc=none action=none header.from=amd.com; Received: from SN6PR12MB4623.namprd12.prod.outlook.com (2603:10b6:805:e9::17) by SN6PR12MB2845.namprd12.prod.outlook.com (2603:10b6:805:75::33) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4129.25; Wed, 12 May 2021 14:27:24 +0000 Received: from SN6PR12MB4623.namprd12.prod.outlook.com ([fe80::ad51:8c49:b171:856c]) by SN6PR12MB4623.namprd12.prod.outlook.com ([fe80::ad51:8c49:b171:856c%7]) with mapi id 15.20.4129.026; Wed, 12 May 2021 14:27:24 +0000 From: Andrey Grodzovsky To: dri-devel@lists.freedesktop.org, amd-gfx@lists.freedesktop.org, linux-pci@vger.kernel.org, ckoenig.leichtzumerken@gmail.com, daniel.vetter@ffwll.ch, Harry.Wentland@amd.com Subject: [PATCH v7 07/16] PCI: Add support for dev_groups to struct pci_driver Date: Wed, 12 May 2021 10:26:39 -0400 Message-Id: <20210512142648.666476-8-andrey.grodzovsky@amd.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210512142648.666476-1-andrey.grodzovsky@amd.com> References: <20210512142648.666476-1-andrey.grodzovsky@amd.com> X-Originating-IP: [2607:fea8:3edf:49b0:7576:4f76:97d8:1487] X-ClientProxiedBy: YTOPR0101CA0032.CANPRD01.PROD.OUTLOOK.COM (2603:10b6:b00:15::45) To SN6PR12MB4623.namprd12.prod.outlook.com (2603:10b6:805:e9::17) MIME-Version: 1.0 X-MS-Exchange-MessageSentRepresentingType: 1 Received: from agrodzovsky-All-Series.hitronhub.home (2607:fea8:3edf:49b0:7576:4f76:97d8:1487) by YTOPR0101CA0032.CANPRD01.PROD.OUTLOOK.COM (2603:10b6:b00:15::45) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4108.30 via Frontend Transport; Wed, 12 May 2021 14:27:23 +0000 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 9cc5f59e-5043-4ff6-6096-08d915520f63 X-MS-TrafficTypeDiagnostic: SN6PR12MB2845: X-MS-Exchange-Transport-Forked: True X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:551; X-MS-Exchange-SenderADCheck: 1 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: FIdWCEba5PvMihH9V8v4zF3bQkbJWxh0oJMl0zP5dXIjUH8x12scsTLi2HxMJlb/QssNik2mDoiozhwFMmcYkOOuhplMcOAaYpYIg/fzMC1l1iz/r3f9n/jYpqj6O2ysHYMSi4F5gIt0vKnn0psqEbPQndEdbX6s77AXywHwW5UBlVJYhrlcWwtW5rnHzPCemAzWPVHiTjihFxT9RFup/FjxqwhBgLonj1Ias5cxbidttQhmg6U4QJUetmjOziUOFCkmmBokjvMScUsOIP9kEm+LPCzAPfHpISK4umn4mk5K9WRuRUHts5m50ey5rG9OFy1CMYr/mCmHdxXh4MLn1qQ4Juhx0V3R6o8LkpQuPRivt86ZvnhCuIrN3lLRKvoYdzkfzzyM+r9v6atmRLTsBtQYK2XZLKLoom3mZLAaBueGMem06bvSBULa664jx0a6Y/Ss6PdzRHVKVsZ6+/bdCQKi4v509nWsXCNBIP5GtwNFoUUozRP7Eap4EixCpcXuLLY/xwaDHEaWIv7P9Nl0tvSxW2JpD2xyjUWoqj5sFiKjpnpT/0BAw1JNnzYFvUtGcNGumz0l0ze+RyiBL6UHE3bC8hP6NDCZ6zCXyE+HCzo= X-Forefront-Antispam-Report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:SN6PR12MB4623.namprd12.prod.outlook.com; PTR:; CAT:NONE; SFS:(4636009)(376002)(39860400002)(396003)(346002)(136003)(366004)(6486002)(8676002)(54906003)(316002)(6512007)(16526019)(6636002)(478600001)(1076003)(186003)(4326008)(44832011)(2906002)(6506007)(8936002)(5660300002)(2616005)(38100700002)(36756003)(86362001)(6666004)(66476007)(66946007)(66556008)(83380400001)(52116002); DIR:OUT; SFP:1101; X-MS-Exchange-AntiSpam-MessageData: u8iHqv8tz9BHH08oKkW0lWy+cVTbTQr+sNaKsxJOFnpNCoHi6ienOySoSYOtdAsqtZWqURt0kPBQn8bW/cO7UvWGWVxk/7MN4LL4cAgY0Ey0BvXLazUbVY7reJIczrZThoQXR9eVTD67J8QQFuy/OUwyD39E1Zis4m+LvFlhanVLffF5juJkg7MSzN3P7niWvBNibx0a3fDmjFIUnfl2nkyvlB06P5s1OqM3ZJiYgNiMi+zfPJvaTHBGVO8A5NxBYJ6XSIQEIJUYHr70Stjt1tSWNMLn3CLxvX5uDNag9BKeDvscL4acYDhm0Mg8VGaSMII4Fb5Qthd3YzufXp2M+7+l/8qveJukSWe/jf6+JJ88HZEaKHKKZMjTP0fOcoxu0jYcUJwnB7fGByHyfqbknXzBW293697/N0euPVTv4fcmHJjbPrA1M9yaAfEGvks476KhrWgkFKLpr4Xjm3vbTuAC/SwpIYfgpQrwymZCoZpRHFkTh7JuCPzgDHwO1X/zGJCyGh3TJqnKaLfpGrTdahBXhvnyIQaVeN2zUcikn+uLHFw/IKDfs7Q+JaEfGh1gF4KT/q7utxmPmmTCTlowDt9gEzLhLi7pjQiLKH746Tr2vOwZxsNZrIkL0msPwAC2UpycnIws5FfPKBiqVrjVWYN3bucLL6f8b0hF9ySUR2Swl8XXvgLJ9UDC5xJdkSoRqJW+g/8ZhAnAb11g33pS2CX3ZHuh8ND6Jfa0t9eXYpxPnY4p/E5Y6hLDIinQFHF1o/A4kPiPfuG177XkS081kXR8l5f4q8RTcZdECGyvGhQp39XRDzoOE5sRwAkD2If8kfuQP3Yq9aMYFhnX1seDDbjleUMuHxKfcVfsby459WkgJYvxMPpfGetzSnjLLB3+DtbBXiBfCBeE69f+CaBODO0FDqUFZ81U9ql1lSZHeRcfYtM9iOQ9SM6jLm8DsnWvDc60yb2AVu880FrSJonHv8HJc30spBvDqYu0MZpcMzih/uukCmWws8zQy0THDhH0o1aIy9NUm+WbLq/VjgWFU0EwvpoJBVec/JjV6QdKLblhWZj4g2zD4MpLYEuGePx9LMqH8coX8xmNv6nZoEMADVnZGKQDCTUOpbDbOwaVZzji8JoYORKBt9CYiSoalEAM9iDXQRmyH6PHFlqpvtVBU7niz+JJt8muSKNpqzqQum73v83BAgctJuO41MKbMsCe3CkR/W3+Xwi3Po8C54lpUZky6XAAwSL+YndZQgWEcgX7disj7xdIzrUOdT0X70onRkByA5LEikpaV9cSIAnLC3iPZfYrK5aF2lQydMuHvEHjgQeSbfDRmsN/Wl/1qvhpjIGPnKFNtQ/2zrL6cf3QYXXXpQoSc6CsOQALjgb12A88sGsyl8p8aUuUKPHPc0s9 X-OriginatorOrg: amd.com X-MS-Exchange-CrossTenant-Network-Message-Id: 9cc5f59e-5043-4ff6-6096-08d915520f63 X-MS-Exchange-CrossTenant-AuthSource: SN6PR12MB4623.namprd12.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 12 May 2021 14:27:24.0827 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: ZQtc9Q04ARiFQzokZZctQsHV7H4uApNgsNfGDqe/v8sPsmekPREdT+07h+OaIp+NAnyCvBq2yHew9QcmHKJubw== X-MS-Exchange-Transport-CrossTenantHeadersStamped: SN6PR12MB2845 X-BeenThere: dri-devel@lists.freedesktop.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Direct Rendering Infrastructure - Development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: gregkh@linuxfoundation.org, Felix.Kuehling@amd.com, Bjorn Helgaas , helgaas@kernel.org, Alexander.Deucher@amd.com Errors-To: dri-devel-bounces@lists.freedesktop.org Sender: "dri-devel" This helps converting PCI drivers sysfs attributes to static. Analogous to' commit b71b283e3d6d ("USB: add support for dev_groups to struct usb_driver")' Signed-off-by: Andrey Grodzovsky Suggested-by: Greg Kroah-Hartman Acked-by: Bjorn Helgaas --- drivers/pci/pci-driver.c | 1 + include/linux/pci.h | 3 +++ 2 files changed, 4 insertions(+) diff --git a/drivers/pci/pci-driver.c b/drivers/pci/pci-driver.c index ec44a79e951a..3a72352aa5cf 100644 --- a/drivers/pci/pci-driver.c +++ b/drivers/pci/pci-driver.c @@ -1385,6 +1385,7 @@ int __pci_register_driver(struct pci_driver *drv, struct module *owner, drv->driver.owner = owner; drv->driver.mod_name = mod_name; drv->driver.groups = drv->groups; + drv->driver.dev_groups = drv->dev_groups; spin_lock_init(&drv->dynids.lock); INIT_LIST_HEAD(&drv->dynids.list); diff --git a/include/linux/pci.h b/include/linux/pci.h index 86c799c97b77..b57755b03009 100644 --- a/include/linux/pci.h +++ b/include/linux/pci.h @@ -858,6 +858,8 @@ struct module; * number of VFs to enable via sysfs "sriov_numvfs" file. * @err_handler: See Documentation/PCI/pci-error-recovery.rst * @groups: Sysfs attribute groups. + * @dev_groups: Attributes attached to the device that will be + * created once it is bound to the driver. * @driver: Driver model structure. * @dynids: List of dynamically added device IDs. */ @@ -873,6 +875,7 @@ struct pci_driver { int (*sriov_configure)(struct pci_dev *dev, int num_vfs); /* On PF */ const struct pci_error_handlers *err_handler; const struct attribute_group **groups; + const struct attribute_group **dev_groups; struct device_driver driver; struct pci_dynids dynids; }; From patchwork Wed May 12 14:26:40 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Andrey Grodzovsky X-Patchwork-Id: 12253883 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-18.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,MSGID_FROM_MTA_HEADER,SPF_HELO_NONE, SPF_PASS,USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id C7B81C43462 for ; Wed, 12 May 2021 14:27:33 +0000 (UTC) Received: from gabe.freedesktop.org (gabe.freedesktop.org [131.252.210.177]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 65537613E9 for ; Wed, 12 May 2021 14:27:33 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 65537613E9 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=amd.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=dri-devel-bounces@lists.freedesktop.org Received: from gabe.freedesktop.org (localhost [127.0.0.1]) by gabe.freedesktop.org (Postfix) with ESMTP id A56996EC0F; Wed, 12 May 2021 14:27:31 +0000 (UTC) Received: from NAM11-DM6-obe.outbound.protection.outlook.com (mail-dm6nam11on2044.outbound.protection.outlook.com [40.107.223.44]) by gabe.freedesktop.org (Postfix) with ESMTPS id BE95B6EC14; Wed, 12 May 2021 14:27:28 +0000 (UTC) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=Jlohzi4jlJFCfQ+AbqnLz7TSTFLiT9vWHJoJWR59lWod+84YqXifufwXfvEgGQpy3zqAU1vaKcQ4U6H5W8aCf74b9c6tTbRZhtP/oMM16JA7b7CynijVc7oc0am+ZCufm/Ub2NDu5ovsdpDqTE6KL5VYZeEw8OrJXuGnqzfFz3gKqs17Lw9sBBTNERo7wRVtSLWgynF0JBayacOVwylX6e1fOPUYp22yvkgO47pbP5FgY9X0U9A5+P9SSh2OLQFcvXIDm50u6mmN0Z6qSJxQaC8h3aZpDC/VYsIhqCtT1L0r92RhKNScTXHogWw/PU6av9OQWHYTakuqvGDRXBYJNA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=+j+bM1q4z3pMJEkudykjSW5dJG+JuOaAr7e0M6/ckSw=; b=biGjpNvl7b82q+PwL9wI8EAepzxD3HsxeZfODkC2mE+0FKTOdqWv5Gh2ldi+ZZQ8vAsXVaRO5JZqyWMwA0lFPcaIv0alIq7y1DIqKAetjev88BlIiPaLsT6Op3755uHAz/3R2iV7jJr348cz3+JuVD2s/Fn4iorKAy0/zJOEZCvNRvhuxs65PA8J0BV49SFfhdXt246RlA8EWEwlhBS164x3jRCzkdWoawkxnSW5wXlV8ZLlJTWvhQM6lZjA3BGXZV24caAwp04dSgzrYeb0XDReGO6Dhwj3T8gJyXeK/1jOi/YN78e8S4d3Sm0vGnx2woIfTS21YR5DdTlHiyuR/g== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=amd.com; dmarc=pass action=none header.from=amd.com; dkim=pass header.d=amd.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=+j+bM1q4z3pMJEkudykjSW5dJG+JuOaAr7e0M6/ckSw=; b=ddjX09QaEOh1ra+Wyrm7sDzzGIWUBL4l2kZKVNMmrWjenI1ySrSTqeL7QJHR/lMS0XW7aJ7Vatspab4QN0Rg0uDLIC++eqgppd5iwjiCnpak4V12xjyRCSyi8LIZUhPvAkHDogDLGaWQYx4DbA/6jHe0QV62hy+0zv1/8f9zOxg= Authentication-Results: lists.freedesktop.org; dkim=none (message not signed) header.d=none; lists.freedesktop.org; dmarc=none action=none header.from=amd.com; Received: from SN6PR12MB4623.namprd12.prod.outlook.com (2603:10b6:805:e9::17) by SN6PR12MB2845.namprd12.prod.outlook.com (2603:10b6:805:75::33) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4129.25; Wed, 12 May 2021 14:27:26 +0000 Received: from SN6PR12MB4623.namprd12.prod.outlook.com ([fe80::ad51:8c49:b171:856c]) by SN6PR12MB4623.namprd12.prod.outlook.com ([fe80::ad51:8c49:b171:856c%7]) with mapi id 15.20.4129.026; Wed, 12 May 2021 14:27:26 +0000 From: Andrey Grodzovsky To: dri-devel@lists.freedesktop.org, amd-gfx@lists.freedesktop.org, linux-pci@vger.kernel.org, ckoenig.leichtzumerken@gmail.com, daniel.vetter@ffwll.ch, Harry.Wentland@amd.com Subject: [PATCH v7 08/16] drm/amdgpu: Convert driver sysfs attributes to static attributes Date: Wed, 12 May 2021 10:26:40 -0400 Message-Id: <20210512142648.666476-9-andrey.grodzovsky@amd.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210512142648.666476-1-andrey.grodzovsky@amd.com> References: <20210512142648.666476-1-andrey.grodzovsky@amd.com> X-Originating-IP: [2607:fea8:3edf:49b0:7576:4f76:97d8:1487] X-ClientProxiedBy: YTOPR0101CA0032.CANPRD01.PROD.OUTLOOK.COM (2603:10b6:b00:15::45) To SN6PR12MB4623.namprd12.prod.outlook.com (2603:10b6:805:e9::17) MIME-Version: 1.0 X-MS-Exchange-MessageSentRepresentingType: 1 Received: from agrodzovsky-All-Series.hitronhub.home (2607:fea8:3edf:49b0:7576:4f76:97d8:1487) by YTOPR0101CA0032.CANPRD01.PROD.OUTLOOK.COM (2603:10b6:b00:15::45) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4108.30 via Frontend Transport; Wed, 12 May 2021 14:27:25 +0000 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 6e3753d6-d2bd-40ed-d4b9-08d9155210f5 X-MS-TrafficTypeDiagnostic: SN6PR12MB2845: X-MS-Exchange-Transport-Forked: True X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:6108; X-MS-Exchange-SenderADCheck: 1 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: NuHBhvPgseDyf/4H1ld5FZxFb7YL93e4NSR0VdOVJIvO4ZNcyQpXfBlfY4KC6V8e5z6OOaEp817+jghKDylxUrFxOyf7Ayjzl8hUZ32AkkXLVbBMKh+MvMEIxBI/epyIYkHVssBUJS65xBvpGVL9QhARV99WFhtWvXg4E4ATxKJXa6M47+l5P0fDfuLTUzfUZlxXAhrW5Yb5+UuWxT62NdTt5nElgRQlL1z/jGAhZqbHX8RGPamG6CeFVAC+b6HmeMZK8PAa/nfghMGGNGUFeqwjqWou3w1VfLhPzCFlurkg9V7WMnAhjqw0rIVg+nK3Ky0mjZL6nlg4Jw2sAfbIEGrHSZ3uFYt7dxagjViMcWQi7stWCXE4XnafDW6JrBhTU5/3RnCrjR2M/VY27S62/Px97+90CjeU68tyebjU1929JojzKLnpJWhedwANLHOd9UvYj8c84ncAhrh5sNkAfZ2DGTckHMvrhLrV5almfZmTyW6d+Sv8w62pY5chlEgQlzKZ59/LbzZa06+jSb7WzgQUQmKjn4Lw8GtgsowegP/uHZv94wyyGWxcFAzjyTFklAIjFt6w3pTbtjtrGLHBWzAe73sb0+uvvrlVRJj0TcE= X-Forefront-Antispam-Report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:SN6PR12MB4623.namprd12.prod.outlook.com; PTR:; CAT:NONE; SFS:(4636009)(376002)(39860400002)(396003)(346002)(136003)(366004)(6486002)(8676002)(54906003)(316002)(6512007)(16526019)(6636002)(478600001)(1076003)(186003)(4326008)(44832011)(2906002)(6506007)(8936002)(5660300002)(2616005)(38100700002)(66574015)(36756003)(86362001)(6666004)(66476007)(66946007)(66556008)(83380400001)(52116002); DIR:OUT; SFP:1101; X-MS-Exchange-AntiSpam-MessageData: =?utf-8?q?eTC0QvD4vSrL4t0QkMf/NqvVrz398n?= =?utf-8?q?62FLDx1L+BC+JK6PCsd1zOgClgN8JI9elyBxqZgqNo3g28znCzbs9j/P4JcV+zlcb?= =?utf-8?q?IATk9DaNcH4MO0tJSuevV3R1bonY2AyvoujrzUoL25EQI5Vle3RIVt5Qn9rMtzJsK?= =?utf-8?q?+eVmDCoSJtBTmHlV8gbqlVmDxeQj09HwRQdZsZjd9azzMzwpVWBkyPNSvcwE/VSmo?= =?utf-8?q?6f94D0Epr2NYvFIAQHIRxe0PLyJFlukSJ6kgxa8KxrNVO+HdO4qjBvylbCZkwOatP?= =?utf-8?q?u77UmxKPKm0/ED9UQ+Qpga9DIENdYTlp8QTm/TgpxGMRe/gfAqSatB7UnQYKvNZqe?= =?utf-8?q?tFiFkH0X1SPeQswN0L7p55QLdCl8v32J+VCUN6vo1zMXJSXxK6+l/2y7CnaYIxj3s?= =?utf-8?q?r8IUcXbQwWTwx2+HgC07TdxbFoy5s061JaLWjqb8U92ZyYUBwEc+Fges6Op4c067B?= =?utf-8?q?8SgrtTmqM4FBAJYA4mCFceoaqwjrXkj/GV3T6QtXleY06u3InLTPnkGv/waOTCINB?= =?utf-8?q?sf4v8+e4Ddj9OA/j5AOIeOU4vSDEuA8UzqFbK1If57yESd7exzYmwW7EJg+Q8ROu/?= =?utf-8?q?ZhXblN5T/pA8NQyegGVBnaMx673WeawxAOyuBXjruhw3IJDNew7/2P4Oq4UMQZsAR?= =?utf-8?q?1RyVoQwA1oYnTU9xnEgAIQ9mJ/fUUatsxXGsKo2uXYYVbcJqtIFh9UhQJd81DeT2g?= =?utf-8?q?NOetZPnx4v91oVa9z09iZumon8Mu6DZCillVudXLAL9pmIcYIlAS+YpKwgAsDwNT0?= =?utf-8?q?cBMlwMagLDy5f40k+tZgFJRgaqTRK8cn8g2rL1S7/Kz1tECZrfVfMImxUPPp/+VSI?= =?utf-8?q?k2XaZ/N1CJxSXqmHzAPs+KYJNqP7z5ocI7OIlmSBXePp/EZxUxNgO89HoJAFVZowy?= =?utf-8?q?58VeDJKR2jD7J09EQ/Y0/feiWe65rom0kET88N8GVaK/ld1Ai2vNXGrmzPI7OAqi3?= =?utf-8?q?wSYtduN7NIq0TPpHicyRQGW1WdEBQ+qcQABY4hakH8E+mBs1w8/1L8ijqeWp7miAh?= =?utf-8?q?9Jp7z5iZBCA+3HC0/Usukcs5/Jjx5dGS4u40PR2/Q5V9aLATC0J213jZIVMHGF0Pe?= =?utf-8?q?aYeFTp8mf0RguJCcD7hWUjqxhMihrDy3qeDd4J4EpjeowWLSfgWnXHYFUlx+gCk+N?= =?utf-8?q?uY0JHSvPhGwi3auGOAG633ARnM78qUubW4yLK0dsppbqWvrqSb35spA3Y/pIwLA41?= =?utf-8?q?eApSOVXbzItM63Oo4Utp8gji8XXEJupF9doMNauBcj1w7keCV5kRAh4mELUXOn8zY?= =?utf-8?q?QBeht6Rsxekls2uvLrWMgIVDOWWa+7QoIk40qP2uRaJ3gqbk0f8i2FV5D8rTmSEPF?= =?utf-8?q?LrPQv3ljBY9mo?= X-OriginatorOrg: amd.com X-MS-Exchange-CrossTenant-Network-Message-Id: 6e3753d6-d2bd-40ed-d4b9-08d9155210f5 X-MS-Exchange-CrossTenant-AuthSource: SN6PR12MB4623.namprd12.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 12 May 2021 14:27:26.7072 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: OGm5T98Pb2GAiFQY/WpOdHyPNf/bkiMz44IsiNS+64QUwhrzZCSaZ/f4nTjekOleGhYOCmiLntMtDOFN3psVAA== X-MS-Exchange-Transport-CrossTenantHeadersStamped: SN6PR12MB2845 X-BeenThere: dri-devel@lists.freedesktop.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Direct Rendering Infrastructure - Development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: gregkh@linuxfoundation.org, Felix.Kuehling@amd.com, helgaas@kernel.org, Alexander.Deucher@amd.com, =?utf-8?q?Christian_K=C3=B6nig?= Errors-To: dri-devel-bounces@lists.freedesktop.org Sender: "dri-devel" This allows to remove explicit creation and destruction of those attrs and by this avoids warnings on device finalizing post physical device extraction. v5: Use newly added pci_driver.dev_groups directly Signed-off-by: Andrey Grodzovsky Acked-by: Christian König --- drivers/gpu/drm/amd/amdgpu/amdgpu_atombios.c | 17 ++++++------- drivers/gpu/drm/amd/amdgpu/amdgpu_drv.c | 13 ++++++++++ drivers/gpu/drm/amd/amdgpu/amdgpu_gtt_mgr.c | 25 ++++++++------------ drivers/gpu/drm/amd/amdgpu/amdgpu_vram_mgr.c | 14 ++++------- 4 files changed, 37 insertions(+), 32 deletions(-) diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_atombios.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_atombios.c index 494b2e1717d5..879ed3e50a6e 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_atombios.c +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_atombios.c @@ -1768,6 +1768,15 @@ static ssize_t amdgpu_atombios_get_vbios_version(struct device *dev, static DEVICE_ATTR(vbios_version, 0444, amdgpu_atombios_get_vbios_version, NULL); +static struct attribute *amdgpu_vbios_version_attrs[] = { + &dev_attr_vbios_version.attr, + NULL +}; + +const struct attribute_group amdgpu_vbios_version_attr_group = { + .attrs = amdgpu_vbios_version_attrs +}; + /** * amdgpu_atombios_fini - free the driver info and callbacks for atombios * @@ -1787,7 +1796,6 @@ void amdgpu_atombios_fini(struct amdgpu_device *adev) adev->mode_info.atom_context = NULL; kfree(adev->mode_info.atom_card_info); adev->mode_info.atom_card_info = NULL; - device_remove_file(adev->dev, &dev_attr_vbios_version); } /** @@ -1804,7 +1812,6 @@ int amdgpu_atombios_init(struct amdgpu_device *adev) { struct card_info *atom_card_info = kzalloc(sizeof(struct card_info), GFP_KERNEL); - int ret; if (!atom_card_info) return -ENOMEM; @@ -1833,12 +1840,6 @@ int amdgpu_atombios_init(struct amdgpu_device *adev) amdgpu_atombios_allocate_fb_scratch(adev); } - ret = device_create_file(adev->dev, &dev_attr_vbios_version); - if (ret) { - DRM_ERROR("Failed to create device file for VBIOS version\n"); - return ret; - } - return 0; } diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_drv.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_drv.c index 5ebed4c7d9c0..83006f45b10b 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_drv.c +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_drv.c @@ -1766,6 +1766,18 @@ static struct pci_error_handlers amdgpu_pci_err_handler = { .resume = amdgpu_pci_resume, }; +extern const struct attribute_group amdgpu_vram_mgr_attr_group; +extern const struct attribute_group amdgpu_gtt_mgr_attr_group; +extern const struct attribute_group amdgpu_vbios_version_attr_group; + +static const struct attribute_group *amdgpu_sysfs_groups[] = { + &amdgpu_vram_mgr_attr_group, + &amdgpu_gtt_mgr_attr_group, + &amdgpu_vbios_version_attr_group, + NULL, +}; + + static struct pci_driver amdgpu_kms_pci_driver = { .name = DRIVER_NAME, .id_table = pciidlist, @@ -1774,6 +1786,7 @@ static struct pci_driver amdgpu_kms_pci_driver = { .shutdown = amdgpu_pci_shutdown, .driver.pm = &amdgpu_pm_ops, .err_handler = &amdgpu_pci_err_handler, + .dev_groups = amdgpu_sysfs_groups, }; static int __init amdgpu_init(void) diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_gtt_mgr.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_gtt_mgr.c index 72962de4c04c..a4404da8ca6d 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_gtt_mgr.c +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_gtt_mgr.c @@ -75,6 +75,16 @@ static DEVICE_ATTR(mem_info_gtt_total, S_IRUGO, static DEVICE_ATTR(mem_info_gtt_used, S_IRUGO, amdgpu_mem_info_gtt_used_show, NULL); +static struct attribute *amdgpu_gtt_mgr_attributes[] = { + &dev_attr_mem_info_gtt_total.attr, + &dev_attr_mem_info_gtt_used.attr, + NULL +}; + +const struct attribute_group amdgpu_gtt_mgr_attr_group = { + .attrs = amdgpu_gtt_mgr_attributes +}; + static const struct ttm_resource_manager_func amdgpu_gtt_mgr_func; /** * amdgpu_gtt_mgr_init - init GTT manager and DRM MM @@ -89,7 +99,6 @@ int amdgpu_gtt_mgr_init(struct amdgpu_device *adev, uint64_t gtt_size) struct amdgpu_gtt_mgr *mgr = &adev->mman.gtt_mgr; struct ttm_resource_manager *man = &mgr->manager; uint64_t start, size; - int ret; man->use_tt = true; man->func = &amdgpu_gtt_mgr_func; @@ -102,17 +111,6 @@ int amdgpu_gtt_mgr_init(struct amdgpu_device *adev, uint64_t gtt_size) spin_lock_init(&mgr->lock); atomic64_set(&mgr->available, gtt_size >> PAGE_SHIFT); - ret = device_create_file(adev->dev, &dev_attr_mem_info_gtt_total); - if (ret) { - DRM_ERROR("Failed to create device file mem_info_gtt_total\n"); - return ret; - } - ret = device_create_file(adev->dev, &dev_attr_mem_info_gtt_used); - if (ret) { - DRM_ERROR("Failed to create device file mem_info_gtt_used\n"); - return ret; - } - ttm_set_driver_manager(&adev->mman.bdev, TTM_PL_TT, &mgr->manager); ttm_resource_manager_set_used(man, true); return 0; @@ -142,9 +140,6 @@ void amdgpu_gtt_mgr_fini(struct amdgpu_device *adev) drm_mm_takedown(&mgr->mm); spin_unlock(&mgr->lock); - device_remove_file(adev->dev, &dev_attr_mem_info_gtt_total); - device_remove_file(adev->dev, &dev_attr_mem_info_gtt_used); - ttm_resource_manager_cleanup(man); ttm_set_driver_manager(&adev->mman.bdev, TTM_PL_TT, NULL); } diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_vram_mgr.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_vram_mgr.c index 2344aba9dca3..8543d6486018 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_vram_mgr.c +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_vram_mgr.c @@ -152,7 +152,7 @@ static DEVICE_ATTR(mem_info_vis_vram_used, S_IRUGO, static DEVICE_ATTR(mem_info_vram_vendor, S_IRUGO, amdgpu_mem_info_vram_vendor, NULL); -static const struct attribute *amdgpu_vram_mgr_attributes[] = { +static struct attribute *amdgpu_vram_mgr_attributes[] = { &dev_attr_mem_info_vram_total.attr, &dev_attr_mem_info_vis_vram_total.attr, &dev_attr_mem_info_vram_used.attr, @@ -161,6 +161,10 @@ static const struct attribute *amdgpu_vram_mgr_attributes[] = { NULL }; +const struct attribute_group amdgpu_vram_mgr_attr_group = { + .attrs = amdgpu_vram_mgr_attributes +}; + static const struct ttm_resource_manager_func amdgpu_vram_mgr_func; /** @@ -174,7 +178,6 @@ int amdgpu_vram_mgr_init(struct amdgpu_device *adev) { struct amdgpu_vram_mgr *mgr = &adev->mman.vram_mgr; struct ttm_resource_manager *man = &mgr->manager; - int ret; ttm_resource_manager_init(man, adev->gmc.real_vram_size >> PAGE_SHIFT); @@ -185,11 +188,6 @@ int amdgpu_vram_mgr_init(struct amdgpu_device *adev) INIT_LIST_HEAD(&mgr->reservations_pending); INIT_LIST_HEAD(&mgr->reserved_pages); - /* Add the two VRAM-related sysfs files */ - ret = sysfs_create_files(&adev->dev->kobj, amdgpu_vram_mgr_attributes); - if (ret) - DRM_ERROR("Failed to register sysfs\n"); - ttm_set_driver_manager(&adev->mman.bdev, TTM_PL_VRAM, &mgr->manager); ttm_resource_manager_set_used(man, true); return 0; @@ -227,8 +225,6 @@ void amdgpu_vram_mgr_fini(struct amdgpu_device *adev) drm_mm_takedown(&mgr->mm); spin_unlock(&mgr->lock); - sysfs_remove_files(&adev->dev->kobj, amdgpu_vram_mgr_attributes); - ttm_resource_manager_cleanup(man); ttm_set_driver_manager(&adev->mman.bdev, TTM_PL_VRAM, NULL); } From patchwork Wed May 12 14:26:41 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrey Grodzovsky X-Patchwork-Id: 12253885 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-18.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,MSGID_FROM_MTA_HEADER,SPF_HELO_NONE, SPF_PASS,USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id CA138C433ED for ; Wed, 12 May 2021 14:27:35 +0000 (UTC) Received: from gabe.freedesktop.org (gabe.freedesktop.org [131.252.210.177]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 742EC613E9 for ; Wed, 12 May 2021 14:27:35 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 742EC613E9 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=amd.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=dri-devel-bounces@lists.freedesktop.org Received: from gabe.freedesktop.org (localhost [127.0.0.1]) by gabe.freedesktop.org (Postfix) with ESMTP id C7D7F6EC16; Wed, 12 May 2021 14:27:33 +0000 (UTC) Received: from NAM11-DM6-obe.outbound.protection.outlook.com (mail-dm6nam11on2062.outbound.protection.outlook.com [40.107.223.62]) by gabe.freedesktop.org (Postfix) with ESMTPS id A73376EC14; Wed, 12 May 2021 14:27:32 +0000 (UTC) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=UUDnD8aUh9uFuLyepEr7/rghueXXXW7tBBXC4D/l9B0Ia+jhAx15YjTqQyJvwvrJcTLien8sdWvXfQE/X8r8+TQq8lcDlPEzY4N98zXFQQ1X9S7KbjJU38bbkk0XxiNBzm0AFwl6hNIvtijGKAJW/e09i64aDQ72Cbq9FRNvY57Ply7j9rH6Gs8/fn+yaydCClPw18FlLvkWhDOOhNN/3e1Y582EAXvSJqUi3t1CjFQEFpwK499T/L9wVZ2KEDtZ0X8FOXeY/IDzSlZGNchSacCbM7dqgfBxZW3413+t6+4+KjL6xU+ICINhmYTAGbjwsSOkgj+7k+wgcnbAWqoJxQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=soskPfGUn90+ncZcVR2G7+mzePSgk09w9c8SsAGJQ5Q=; b=PpwpcXV7NSVp7TXimTUerh4MBNbN7aT2QSKxtqAfqh9LYAnylXkxi4dk5mzjyI4fcTeUAqYPV2iYXLlKv6Ndmo2JiKzTXT6nhNsE44aedDe/O73AkFImEZFIFGEKawtLe7iTTL4sUBcx4ttEGuLVaQ/eqRzZsd/ZRFyeKv3JNdJJKgnHWBJTlOrJ5jfswVF2D1ITtavqWlaoEwae8RoZWRWnJp7wP+IHBee8jpHpWwOI4igbxReFkUQRpsl84JwcsuK68y/zxbZ2AkFdKQ5d3MbqnvhsoUFifJIPfo52YSTaB8wX/iJGetdC6yTBWOdHvgCkHu3ufHnpThPICfDMTA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=amd.com; dmarc=pass action=none header.from=amd.com; dkim=pass header.d=amd.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=soskPfGUn90+ncZcVR2G7+mzePSgk09w9c8SsAGJQ5Q=; b=ZJJsYpKJ5xDRMBxuk7cEHhMiaxOIFOCcjxwukS7c5+QYeffKowUGy6lnJgdsANTp2FruSzCkOnZENhmf1Pwuz7VsoJpkP8OxPKNK3XQzmsiW9jNo+Kgh2hsaHpSC7sSBq85DOJTGnJhN87NQQ5pHBjFS4DN09ZdgRjIfQWPwLoU= Authentication-Results: lists.freedesktop.org; dkim=none (message not signed) header.d=none; lists.freedesktop.org; dmarc=none action=none header.from=amd.com; Received: from SN6PR12MB4623.namprd12.prod.outlook.com (2603:10b6:805:e9::17) by SN6PR12MB2845.namprd12.prod.outlook.com (2603:10b6:805:75::33) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4129.25; Wed, 12 May 2021 14:27:30 +0000 Received: from SN6PR12MB4623.namprd12.prod.outlook.com ([fe80::ad51:8c49:b171:856c]) by SN6PR12MB4623.namprd12.prod.outlook.com ([fe80::ad51:8c49:b171:856c%7]) with mapi id 15.20.4129.026; Wed, 12 May 2021 14:27:30 +0000 From: Andrey Grodzovsky To: dri-devel@lists.freedesktop.org, amd-gfx@lists.freedesktop.org, linux-pci@vger.kernel.org, ckoenig.leichtzumerken@gmail.com, daniel.vetter@ffwll.ch, Harry.Wentland@amd.com Subject: [PATCH v7 09/16] drm/amdgpu: Guard against write accesses after device removal Date: Wed, 12 May 2021 10:26:41 -0400 Message-Id: <20210512142648.666476-10-andrey.grodzovsky@amd.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210512142648.666476-1-andrey.grodzovsky@amd.com> References: <20210512142648.666476-1-andrey.grodzovsky@amd.com> X-Originating-IP: [2607:fea8:3edf:49b0:7576:4f76:97d8:1487] X-ClientProxiedBy: YTOPR0101CA0032.CANPRD01.PROD.OUTLOOK.COM (2603:10b6:b00:15::45) To SN6PR12MB4623.namprd12.prod.outlook.com (2603:10b6:805:e9::17) MIME-Version: 1.0 X-MS-Exchange-MessageSentRepresentingType: 1 Received: from agrodzovsky-All-Series.hitronhub.home (2607:fea8:3edf:49b0:7576:4f76:97d8:1487) by YTOPR0101CA0032.CANPRD01.PROD.OUTLOOK.COM (2603:10b6:b00:15::45) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4108.30 via Frontend Transport; Wed, 12 May 2021 14:27:29 +0000 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: c3ac0936-ebf8-4e83-41d9-08d915521331 X-MS-TrafficTypeDiagnostic: SN6PR12MB2845: X-MS-Exchange-Transport-Forked: True X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:5797; X-MS-Exchange-SenderADCheck: 1 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: 3fLZ+ieGmlOW+WHqGCd62VAqTNlmYoEhrL0diyKQunx/wabQ3icaitCfWvegyAn9CuvCgq3X/yQmjd+Ewafu2UZ/0exSLw5pNTR5ahiLUUaWPoZo49rS0NDCIWb4zCDCZPGEtoA63ks++dtHWpcWN5SMGG6QdB3xY4Lz2bbsgMOo+dqZehRrsFOBT7WZCThSuLnd5EhcbjZJa/qsePEn5Y7czO7EnWPnXQ84LCwuGm/tUtLI804KL2WFTjFoQLLEcYgDZ7Q/lJdQXhjO+5DkkVSrnrSC2b0rgKR6MEd/4skQb+aMFwQPwq8Gv8m23/w3zyWpwhjhore5p+I4+KFy/fhJ3qLyCoXXmYSGp5wyM0+qeRkNu9nbmvnhlddh/ke4uco652/x/cacjJ25HYh9L5+xdXwhiZEzlBAzlzF7P9SxI6ah4V8JctuaQ+MuSI4QbtELDRWj/G9L6vQgyFS4onaEr//apeoFYmviMshpCOOVNgWqlc7K5RtfvyC4mQ01s5vxppmJMWq4toD8/cUMrq+ERtc8zo3dCimZ7yoDHxdVUNjZO+GzlK72+L9t+fIcrQz0DIsht9QPjZ4F9etAp1hhSaQa/muaJRPJTv6nHRI= X-Forefront-Antispam-Report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:SN6PR12MB4623.namprd12.prod.outlook.com; PTR:; CAT:NONE; SFS:(4636009)(376002)(39860400002)(396003)(346002)(136003)(366004)(6486002)(8676002)(316002)(6512007)(16526019)(6636002)(478600001)(1076003)(186003)(4326008)(44832011)(2906002)(6506007)(30864003)(8936002)(5660300002)(2616005)(38100700002)(36756003)(86362001)(6666004)(66476007)(66946007)(66556008)(83380400001)(52116002); DIR:OUT; SFP:1101; X-MS-Exchange-AntiSpam-MessageData: 2nWnqX114st//nm3yad7D2G/78o3In0I21F43KJc7sGUisUjGOB972QU3zMOeOqny+jIsZR2yI7dZidqq1B7KZXnzLEsD6hI0Xa3d7HRtPJw6oOf+lMGXpl93yGAAY2m+cbuz3fNaZ9A6zClUYtM1R/AY/XJ8OXM/u1lbl3KUPjqvDjcQydQAeVHv3IvWpcaWspnmOUBdXDHh5UKX6HM6weoRcLc+S22/hRXLmV+CGub7IFJdhvvjsWjWWDAFnWUNfc923PXNUR7GTYhozopubzN3NrQD8eUpAxmYSCnSgHpMB54BrLfwg8JmvlBvNfzBJ3TrzOTbCbrrB7t0O0eKo/RN0mFu3t0lifedLfJGclImH9iQ5BEErzKyIrD6LXSYzN34ZxAlWgrq5TWgEq4mYmKovv0jFnPjwgiKn6SUDryGMHm1sXfuSx5HXoPEg25nafUWKlGAPYX0nYk+1VOT50+V0zPoqVNWx/vTC/eQqUBI/pJ2kirUP+x88M680pI1Q4q9XSE3B92p0qURXGdJFkgl9neNqtrajIFot2up+1/Edx/mz2K+uOIdbEHYROhM7aB46q+zX2L1DxMNbl55kwao3JYGcPf53XZSdgkzSBIEkJvzhY7z5Y2E1iYjnq8dMBfWLDC2nBW6AqPkdU1F7+7im9ahZrZLFWjGVLWHQ/Bw/AGk2bAE6B+bK1kIxPKO+MhlKeUXduS8mcqNSpc8EVYcNgffRiTi8kBSU6yaxcbvEv5FnkbXydSUOnyruThU2DuoKs3DfZNb6q1LB8ua3v7LUyqPjr3MeH/yJFp/5yt5kFl4ZN/YefSpQf0qpVJ/Pz+aheJGazKlbsNlYq+bE1vVcYUIgus5Dgbn7kRajYuXZCSod8u/aZpQsPCgodpJTmpYnzOXPcoAZjMsO/ZIv0YwAI2e4SVp/ztXAi9WaQUsdD+piXuLKQnt3NIOFxUhz82uhEM7lE2hBY4uUU0SGg3Ccso+NRwQSO8G0uNGYXxpSmKAL2attV9cW4DkjH+lxEio58aWsR/fCoPTjHXXF6axqVXW3AYsv4w2RLbkKpVmmbyhRzUzzFEjG6RrjzsIHavbYNirMn4fomEGVfS51LGwMm1eBY8xKtX6q2Suwodf2yz0LkbSuZYcVOTdL3fC83/ER11h4u3H9/oZC6g3kQC4vyHuLbhDXX/JriKkR3JCyxXwEgLTTotGyaVehCaGFLm6BMfu5L7k6uqjIbjTfDRMU224wXUJZuaE7Our9rIpYX9kWVpvdRoibuNA646q5LSh1cpYbbvB04nE/Y7jNAHtCkFIRomMwWNjGSML3Zhiz/XbcMTfBdexFv4COVvnp51RYoANUvL5X+PHQ8DVOIHO0iB9PNgaooJrgfj7lxj3AyDZQ9g9+Imh6Zkauv1 X-OriginatorOrg: amd.com X-MS-Exchange-CrossTenant-Network-Message-Id: c3ac0936-ebf8-4e83-41d9-08d915521331 X-MS-Exchange-CrossTenant-AuthSource: SN6PR12MB4623.namprd12.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 12 May 2021 14:27:30.4741 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: mOSHeOuU0cvGG+tD3WgVrWqL/hHlc1Hy0Bjz0ZcvN0FCbo18vOtkWaNAjBTL49W68ry1CbjxaMInND1tpK3ZJg== X-MS-Exchange-Transport-CrossTenantHeadersStamped: SN6PR12MB2845 X-BeenThere: dri-devel@lists.freedesktop.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Direct Rendering Infrastructure - Development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: gregkh@linuxfoundation.org, Felix.Kuehling@amd.com, helgaas@kernel.org, Alexander.Deucher@amd.com Errors-To: dri-devel-bounces@lists.freedesktop.org Sender: "dri-devel" This should prevent writing to memory or IO ranges possibly already allocated for other uses after our device is removed. v5: Protect more places wher memcopy_to/form_io takes place Protect IB submissions v6: Switch to !drm_dev_enter instead of scoping entire code with brackets. v7: Drop guard of HW ring commands emission protection since they are in GART and not in MMIO. Signed-off-by: Andrey Grodzovsky Reviewed-by: Alex Deucher --- drivers/gpu/drm/amd/amdgpu/amdgpu_device.c | 10 +++- drivers/gpu/drm/amd/amdgpu/amdgpu_gmc.c | 9 ++++ drivers/gpu/drm/amd/amdgpu/amdgpu_psp.c | 63 ++++++++++++++-------- drivers/gpu/drm/amd/amdgpu/amdgpu_psp.h | 2 + drivers/gpu/drm/amd/amdgpu/amdgpu_uvd.c | 31 +++++++---- drivers/gpu/drm/amd/amdgpu/amdgpu_vce.c | 11 ++-- drivers/gpu/drm/amd/amdgpu/amdgpu_vcn.c | 22 +++++--- drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c | 7 ++- drivers/gpu/drm/amd/amdgpu/psp_v11_0.c | 44 +++++++-------- drivers/gpu/drm/amd/amdgpu/psp_v12_0.c | 8 +-- drivers/gpu/drm/amd/amdgpu/psp_v3_1.c | 8 +-- drivers/gpu/drm/amd/amdgpu/vce_v4_0.c | 26 +++++---- drivers/gpu/drm/amd/amdgpu/vcn_v3_0.c | 22 +++++--- 13 files changed, 168 insertions(+), 95 deletions(-) diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c index a0bff4713672..f7cca25c0fa0 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c @@ -71,6 +71,8 @@ #include #include +#include + MODULE_FIRMWARE("amdgpu/vega10_gpu_info.bin"); MODULE_FIRMWARE("amdgpu/vega12_gpu_info.bin"); MODULE_FIRMWARE("amdgpu/raven_gpu_info.bin"); @@ -281,7 +283,10 @@ void amdgpu_device_vram_access(struct amdgpu_device *adev, loff_t pos, unsigned long flags; uint32_t hi = ~0; uint64_t last; + int idx; + if (!drm_dev_enter(&adev->ddev, &idx)) + return; #ifdef CONFIG_64BIT last = min(pos + size, adev->gmc.visible_vram_size); @@ -300,7 +305,7 @@ void amdgpu_device_vram_access(struct amdgpu_device *adev, loff_t pos, } if (count == size) - return; + goto exit; pos += count; buf += count / 4; @@ -323,6 +328,9 @@ void amdgpu_device_vram_access(struct amdgpu_device *adev, loff_t pos, *buf++ = RREG32_NO_KIQ(mmMM_DATA); } spin_unlock_irqrestore(&adev->mmio_idx_lock, flags); + +exit: + drm_dev_exit(idx); } /* diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_gmc.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_gmc.c index 4d32233cde92..04ba5eef1e88 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_gmc.c +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_gmc.c @@ -31,6 +31,8 @@ #include "amdgpu_ras.h" #include "amdgpu_xgmi.h" +#include + /** * amdgpu_gmc_pdb0_alloc - allocate vram for pdb0 * @@ -151,6 +153,10 @@ int amdgpu_gmc_set_pte_pde(struct amdgpu_device *adev, void *cpu_pt_addr, { void __iomem *ptr = (void *)cpu_pt_addr; uint64_t value; + int idx; + + if (!drm_dev_enter(&adev->ddev, &idx)) + return 0; /* * The following is for PTE only. GART does not have PDEs. @@ -158,6 +164,9 @@ int amdgpu_gmc_set_pte_pde(struct amdgpu_device *adev, void *cpu_pt_addr, value = addr & 0x0000FFFFFFFFF000ULL; value |= flags; writeq(value, ptr + (gpu_page_idx * 8)); + + drm_dev_exit(idx); + return 0; } diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_psp.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_psp.c index 9e769cf6095b..bb6afee61666 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_psp.c +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_psp.c @@ -25,6 +25,7 @@ #include #include +#include #include "amdgpu.h" #include "amdgpu_psp.h" @@ -39,6 +40,8 @@ #include "amdgpu_ras.h" #include "amdgpu_securedisplay.h" +#include + static int psp_sysfs_init(struct amdgpu_device *adev); static void psp_sysfs_fini(struct amdgpu_device *adev); @@ -253,7 +256,7 @@ psp_cmd_submit_buf(struct psp_context *psp, struct psp_gfx_cmd_resp *cmd, uint64_t fence_mc_addr) { int ret; - int index; + int index, idx; int timeout = 20000; bool ras_intr = false; bool skip_unsupport = false; @@ -261,6 +264,9 @@ psp_cmd_submit_buf(struct psp_context *psp, if (psp->adev->in_pci_err_recovery) return 0; + if (!drm_dev_enter(&psp->adev->ddev, &idx)) + return 0; + mutex_lock(&psp->mutex); memset(psp->cmd_buf_mem, 0, PSP_CMD_BUFFER_SIZE); @@ -271,8 +277,7 @@ psp_cmd_submit_buf(struct psp_context *psp, ret = psp_ring_cmd_submit(psp, psp->cmd_buf_mc_addr, fence_mc_addr, index); if (ret) { atomic_dec(&psp->fence_value); - mutex_unlock(&psp->mutex); - return ret; + goto exit; } amdgpu_asic_invalidate_hdp(psp->adev, NULL); @@ -312,8 +317,8 @@ psp_cmd_submit_buf(struct psp_context *psp, psp->cmd_buf_mem->cmd_id, psp->cmd_buf_mem->resp.status); if (!timeout) { - mutex_unlock(&psp->mutex); - return -EINVAL; + ret = -EINVAL; + goto exit; } } @@ -321,8 +326,10 @@ psp_cmd_submit_buf(struct psp_context *psp, ucode->tmr_mc_addr_lo = psp->cmd_buf_mem->resp.fw_addr_lo; ucode->tmr_mc_addr_hi = psp->cmd_buf_mem->resp.fw_addr_hi; } - mutex_unlock(&psp->mutex); +exit: + mutex_unlock(&psp->mutex); + drm_dev_exit(idx); return ret; } @@ -359,8 +366,7 @@ static int psp_load_toc(struct psp_context *psp, if (!cmd) return -ENOMEM; /* Copy toc to psp firmware private buffer */ - memset(psp->fw_pri_buf, 0, PSP_1_MEG); - memcpy(psp->fw_pri_buf, psp->toc_start_addr, psp->toc_bin_size); + psp_copy_fw(psp, psp->toc_start_addr, psp->toc_bin_size); psp_prep_load_toc_cmd_buf(cmd, psp->fw_pri_mc_addr, psp->toc_bin_size); @@ -625,8 +631,7 @@ static int psp_asd_load(struct psp_context *psp) if (!cmd) return -ENOMEM; - memset(psp->fw_pri_buf, 0, PSP_1_MEG); - memcpy(psp->fw_pri_buf, psp->asd_start_addr, psp->asd_ucode_size); + psp_copy_fw(psp, psp->asd_start_addr, psp->asd_ucode_size); psp_prep_asd_load_cmd_buf(cmd, psp->fw_pri_mc_addr, psp->asd_ucode_size); @@ -781,8 +786,7 @@ static int psp_xgmi_load(struct psp_context *psp) if (!cmd) return -ENOMEM; - memset(psp->fw_pri_buf, 0, PSP_1_MEG); - memcpy(psp->fw_pri_buf, psp->ta_xgmi_start_addr, psp->ta_xgmi_ucode_size); + psp_copy_fw(psp, psp->ta_xgmi_start_addr, psp->ta_xgmi_ucode_size); psp_prep_ta_load_cmd_buf(cmd, psp->fw_pri_mc_addr, @@ -1038,8 +1042,7 @@ static int psp_ras_load(struct psp_context *psp) if (!cmd) return -ENOMEM; - memset(psp->fw_pri_buf, 0, PSP_1_MEG); - memcpy(psp->fw_pri_buf, psp->ta_ras_start_addr, psp->ta_ras_ucode_size); + psp_copy_fw(psp, psp->ta_ras_start_addr, psp->ta_ras_ucode_size); psp_prep_ta_load_cmd_buf(cmd, psp->fw_pri_mc_addr, @@ -1275,8 +1278,7 @@ static int psp_hdcp_load(struct psp_context *psp) if (!cmd) return -ENOMEM; - memset(psp->fw_pri_buf, 0, PSP_1_MEG); - memcpy(psp->fw_pri_buf, psp->ta_hdcp_start_addr, + psp_copy_fw(psp, psp->ta_hdcp_start_addr, psp->ta_hdcp_ucode_size); psp_prep_ta_load_cmd_buf(cmd, @@ -1427,8 +1429,7 @@ static int psp_dtm_load(struct psp_context *psp) if (!cmd) return -ENOMEM; - memset(psp->fw_pri_buf, 0, PSP_1_MEG); - memcpy(psp->fw_pri_buf, psp->ta_dtm_start_addr, psp->ta_dtm_ucode_size); + psp_copy_fw(psp, psp->ta_dtm_start_addr, psp->ta_dtm_ucode_size); psp_prep_ta_load_cmd_buf(cmd, psp->fw_pri_mc_addr, @@ -1573,8 +1574,7 @@ static int psp_rap_load(struct psp_context *psp) if (!cmd) return -ENOMEM; - memset(psp->fw_pri_buf, 0, PSP_1_MEG); - memcpy(psp->fw_pri_buf, psp->ta_rap_start_addr, psp->ta_rap_ucode_size); + psp_copy_fw(psp, psp->ta_rap_start_addr, psp->ta_rap_ucode_size); psp_prep_ta_load_cmd_buf(cmd, psp->fw_pri_mc_addr, @@ -3022,7 +3022,7 @@ static ssize_t psp_usbc_pd_fw_sysfs_write(struct device *dev, struct amdgpu_device *adev = drm_to_adev(ddev); void *cpu_addr; dma_addr_t dma_addr; - int ret; + int ret, idx; char fw_name[100]; const struct firmware *usbc_pd_fw; @@ -3031,6 +3031,9 @@ static ssize_t psp_usbc_pd_fw_sysfs_write(struct device *dev, return -EBUSY; } + if (!drm_dev_enter(ddev, &idx)) + return -ENODEV; + snprintf(fw_name, sizeof(fw_name), "amdgpu/%s", buf); ret = request_firmware(&usbc_pd_fw, fw_name, adev->dev); if (ret) @@ -3062,16 +3065,30 @@ static ssize_t psp_usbc_pd_fw_sysfs_write(struct device *dev, rel_buf: dma_free_coherent(adev->dev, usbc_pd_fw->size, cpu_addr, dma_addr); release_firmware(usbc_pd_fw); - fail: if (ret) { DRM_ERROR("Failed to load USBC PD FW, err = %d", ret); - return ret; + count = ret; } + drm_dev_exit(idx); return count; } +void psp_copy_fw(struct psp_context *psp, uint8_t *start_addr, uint32_t bin_size) +{ + int idx; + + if (!drm_dev_enter(&psp->adev->ddev, &idx)) + return; + + memset(psp->fw_pri_buf, 0, PSP_1_MEG); + memcpy(psp->fw_pri_buf, start_addr, bin_size); + + drm_dev_exit(idx); +} + + static DEVICE_ATTR(usbc_pd_fw, S_IRUGO | S_IWUSR, psp_usbc_pd_fw_sysfs_read, psp_usbc_pd_fw_sysfs_write); diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_psp.h b/drivers/gpu/drm/amd/amdgpu/amdgpu_psp.h index 46a5328e00e0..2bfdc278817f 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_psp.h +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_psp.h @@ -423,4 +423,6 @@ int psp_get_fw_attestation_records_addr(struct psp_context *psp, int psp_load_fw_list(struct psp_context *psp, struct amdgpu_firmware_info **ucode_list, int ucode_count); +void psp_copy_fw(struct psp_context *psp, uint8_t *start_addr, uint32_t bin_size); + #endif diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_uvd.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_uvd.c index c6dbc0801604..82f0542c7792 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_uvd.c +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_uvd.c @@ -32,6 +32,7 @@ #include #include +#include #include "amdgpu.h" #include "amdgpu_pm.h" @@ -375,7 +376,7 @@ int amdgpu_uvd_suspend(struct amdgpu_device *adev) { unsigned size; void *ptr; - int i, j; + int i, j, idx; bool in_ras_intr = amdgpu_ras_intr_triggered(); cancel_delayed_work_sync(&adev->uvd.idle_work); @@ -403,11 +404,15 @@ int amdgpu_uvd_suspend(struct amdgpu_device *adev) if (!adev->uvd.inst[j].saved_bo) return -ENOMEM; - /* re-write 0 since err_event_athub will corrupt VCPU buffer */ - if (in_ras_intr) - memset(adev->uvd.inst[j].saved_bo, 0, size); - else - memcpy_fromio(adev->uvd.inst[j].saved_bo, ptr, size); + if (drm_dev_enter(&adev->ddev, &idx)) { + /* re-write 0 since err_event_athub will corrupt VCPU buffer */ + if (in_ras_intr) + memset(adev->uvd.inst[j].saved_bo, 0, size); + else + memcpy_fromio(adev->uvd.inst[j].saved_bo, ptr, size); + + drm_dev_exit(idx); + } } if (in_ras_intr) @@ -420,7 +425,7 @@ int amdgpu_uvd_resume(struct amdgpu_device *adev) { unsigned size; void *ptr; - int i; + int i, idx; for (i = 0; i < adev->uvd.num_uvd_inst; i++) { if (adev->uvd.harvest_config & (1 << i)) @@ -432,7 +437,10 @@ int amdgpu_uvd_resume(struct amdgpu_device *adev) ptr = adev->uvd.inst[i].cpu_addr; if (adev->uvd.inst[i].saved_bo != NULL) { - memcpy_toio(ptr, adev->uvd.inst[i].saved_bo, size); + if (drm_dev_enter(&adev->ddev, &idx)) { + memcpy_toio(ptr, adev->uvd.inst[i].saved_bo, size); + drm_dev_exit(idx); + } kvfree(adev->uvd.inst[i].saved_bo); adev->uvd.inst[i].saved_bo = NULL; } else { @@ -442,8 +450,11 @@ int amdgpu_uvd_resume(struct amdgpu_device *adev) hdr = (const struct common_firmware_header *)adev->uvd.fw->data; if (adev->firmware.load_type != AMDGPU_FW_LOAD_PSP) { offset = le32_to_cpu(hdr->ucode_array_offset_bytes); - memcpy_toio(adev->uvd.inst[i].cpu_addr, adev->uvd.fw->data + offset, - le32_to_cpu(hdr->ucode_size_bytes)); + if (drm_dev_enter(&adev->ddev, &idx)) { + memcpy_toio(adev->uvd.inst[i].cpu_addr, adev->uvd.fw->data + offset, + le32_to_cpu(hdr->ucode_size_bytes)); + drm_dev_exit(idx); + } size -= le32_to_cpu(hdr->ucode_size_bytes); ptr += le32_to_cpu(hdr->ucode_size_bytes); } diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_vce.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_vce.c index ea6a62f67e38..833203401ef4 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_vce.c +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_vce.c @@ -29,6 +29,7 @@ #include #include +#include #include "amdgpu.h" #include "amdgpu_pm.h" @@ -293,7 +294,7 @@ int amdgpu_vce_resume(struct amdgpu_device *adev) void *cpu_addr; const struct common_firmware_header *hdr; unsigned offset; - int r; + int r, idx; if (adev->vce.vcpu_bo == NULL) return -EINVAL; @@ -313,8 +314,12 @@ int amdgpu_vce_resume(struct amdgpu_device *adev) hdr = (const struct common_firmware_header *)adev->vce.fw->data; offset = le32_to_cpu(hdr->ucode_array_offset_bytes); - memcpy_toio(cpu_addr, adev->vce.fw->data + offset, - adev->vce.fw->size - offset); + + if (drm_dev_enter(&adev->ddev, &idx)) { + memcpy_toio(cpu_addr, adev->vce.fw->data + offset, + adev->vce.fw->size - offset); + drm_dev_exit(idx); + } amdgpu_bo_kunmap(adev->vce.vcpu_bo); diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_vcn.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_vcn.c index 201645963ba5..21f7d3644d70 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_vcn.c +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_vcn.c @@ -27,6 +27,7 @@ #include #include #include +#include #include "amdgpu.h" #include "amdgpu_pm.h" @@ -275,7 +276,7 @@ int amdgpu_vcn_suspend(struct amdgpu_device *adev) { unsigned size; void *ptr; - int i; + int i, idx; cancel_delayed_work_sync(&adev->vcn.idle_work); @@ -292,7 +293,10 @@ int amdgpu_vcn_suspend(struct amdgpu_device *adev) if (!adev->vcn.inst[i].saved_bo) return -ENOMEM; - memcpy_fromio(adev->vcn.inst[i].saved_bo, ptr, size); + if (drm_dev_enter(&adev->ddev, &idx)) { + memcpy_fromio(adev->vcn.inst[i].saved_bo, ptr, size); + drm_dev_exit(idx); + } } return 0; } @@ -301,7 +305,7 @@ int amdgpu_vcn_resume(struct amdgpu_device *adev) { unsigned size; void *ptr; - int i; + int i, idx; for (i = 0; i < adev->vcn.num_vcn_inst; ++i) { if (adev->vcn.harvest_config & (1 << i)) @@ -313,7 +317,10 @@ int amdgpu_vcn_resume(struct amdgpu_device *adev) ptr = adev->vcn.inst[i].cpu_addr; if (adev->vcn.inst[i].saved_bo != NULL) { - memcpy_toio(ptr, adev->vcn.inst[i].saved_bo, size); + if (drm_dev_enter(&adev->ddev, &idx)) { + memcpy_toio(ptr, adev->vcn.inst[i].saved_bo, size); + drm_dev_exit(idx); + } kvfree(adev->vcn.inst[i].saved_bo); adev->vcn.inst[i].saved_bo = NULL; } else { @@ -323,8 +330,11 @@ int amdgpu_vcn_resume(struct amdgpu_device *adev) hdr = (const struct common_firmware_header *)adev->vcn.fw->data; if (adev->firmware.load_type != AMDGPU_FW_LOAD_PSP) { offset = le32_to_cpu(hdr->ucode_array_offset_bytes); - memcpy_toio(adev->vcn.inst[i].cpu_addr, adev->vcn.fw->data + offset, - le32_to_cpu(hdr->ucode_size_bytes)); + if (drm_dev_enter(&adev->ddev, &idx)) { + memcpy_toio(adev->vcn.inst[i].cpu_addr, adev->vcn.fw->data + offset, + le32_to_cpu(hdr->ucode_size_bytes)); + drm_dev_exit(idx); + } size -= le32_to_cpu(hdr->ucode_size_bytes); ptr += le32_to_cpu(hdr->ucode_size_bytes); } diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c index 9f868cf3b832..7dd5f10ab570 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c @@ -32,6 +32,7 @@ #include #include +#include #include "amdgpu.h" #include "amdgpu_trace.h" #include "amdgpu_amdkfd.h" @@ -1606,7 +1607,10 @@ static int amdgpu_vm_bo_update_mapping(struct amdgpu_device *adev, struct amdgpu_vm_update_params params; enum amdgpu_sync_mode sync_mode; uint64_t pfn; - int r; + int r, idx; + + if (!drm_dev_enter(&adev->ddev, &idx)) + return -ENODEV; memset(¶ms, 0, sizeof(params)); params.adev = adev; @@ -1715,6 +1719,7 @@ static int amdgpu_vm_bo_update_mapping(struct amdgpu_device *adev, error_unlock: amdgpu_vm_eviction_unlock(vm); + drm_dev_exit(idx); return r; } diff --git a/drivers/gpu/drm/amd/amdgpu/psp_v11_0.c b/drivers/gpu/drm/amd/amdgpu/psp_v11_0.c index 589410c32d09..2cec71e823f5 100644 --- a/drivers/gpu/drm/amd/amdgpu/psp_v11_0.c +++ b/drivers/gpu/drm/amd/amdgpu/psp_v11_0.c @@ -23,6 +23,7 @@ #include #include #include +#include #include "amdgpu.h" #include "amdgpu_psp.h" @@ -269,10 +270,8 @@ static int psp_v11_0_bootloader_load_kdb(struct psp_context *psp) if (ret) return ret; - memset(psp->fw_pri_buf, 0, PSP_1_MEG); - /* Copy PSP KDB binary to memory */ - memcpy(psp->fw_pri_buf, psp->kdb_start_addr, psp->kdb_bin_size); + psp_copy_fw(psp, psp->kdb_start_addr, psp->kdb_bin_size); /* Provide the PSP KDB to bootloader */ WREG32_SOC15(MP0, 0, mmMP0_SMN_C2PMSG_36, @@ -302,10 +301,8 @@ static int psp_v11_0_bootloader_load_spl(struct psp_context *psp) if (ret) return ret; - memset(psp->fw_pri_buf, 0, PSP_1_MEG); - /* Copy PSP SPL binary to memory */ - memcpy(psp->fw_pri_buf, psp->spl_start_addr, psp->spl_bin_size); + psp_copy_fw(psp, psp->spl_start_addr, psp->spl_bin_size); /* Provide the PSP SPL to bootloader */ WREG32_SOC15(MP0, 0, mmMP0_SMN_C2PMSG_36, @@ -335,10 +332,8 @@ static int psp_v11_0_bootloader_load_sysdrv(struct psp_context *psp) if (ret) return ret; - memset(psp->fw_pri_buf, 0, PSP_1_MEG); - /* Copy PSP System Driver binary to memory */ - memcpy(psp->fw_pri_buf, psp->sys_start_addr, psp->sys_bin_size); + psp_copy_fw(psp, psp->sys_start_addr, psp->sys_bin_size); /* Provide the sys driver to bootloader */ WREG32_SOC15(MP0, 0, mmMP0_SMN_C2PMSG_36, @@ -371,10 +366,8 @@ static int psp_v11_0_bootloader_load_sos(struct psp_context *psp) if (ret) return ret; - memset(psp->fw_pri_buf, 0, PSP_1_MEG); - /* Copy Secure OS binary to PSP memory */ - memcpy(psp->fw_pri_buf, psp->sos_start_addr, psp->sos_bin_size); + psp_copy_fw(psp, psp->sos_start_addr, psp->sos_bin_size); /* Provide the PSP secure OS to bootloader */ WREG32_SOC15(MP0, 0, mmMP0_SMN_C2PMSG_36, @@ -608,7 +601,7 @@ static int psp_v11_0_memory_training(struct psp_context *psp, uint32_t ops) uint32_t p2c_header[4]; uint32_t sz; void *buf; - int ret; + int ret, idx; if (ctx->init == PSP_MEM_TRAIN_NOT_SUPPORT) { DRM_DEBUG("Memory training is not supported.\n"); @@ -681,17 +674,24 @@ static int psp_v11_0_memory_training(struct psp_context *psp, uint32_t ops) return -ENOMEM; } - memcpy_fromio(buf, adev->mman.aper_base_kaddr, sz); - ret = psp_v11_0_memory_training_send_msg(psp, PSP_BL__DRAM_LONG_TRAIN); - if (ret) { - DRM_ERROR("Send long training msg failed.\n"); + if (drm_dev_enter(&adev->ddev, &idx)) { + memcpy_fromio(buf, adev->mman.aper_base_kaddr, sz); + ret = psp_v11_0_memory_training_send_msg(psp, PSP_BL__DRAM_LONG_TRAIN); + if (ret) { + DRM_ERROR("Send long training msg failed.\n"); + vfree(buf); + drm_dev_exit(idx); + return ret; + } + + memcpy_toio(adev->mman.aper_base_kaddr, buf, sz); + adev->hdp.funcs->flush_hdp(adev, NULL); vfree(buf); - return ret; + drm_dev_exit(idx); + } else { + vfree(buf); + return -ENODEV; } - - memcpy_toio(adev->mman.aper_base_kaddr, buf, sz); - adev->hdp.funcs->flush_hdp(adev, NULL); - vfree(buf); } if (ops & PSP_MEM_TRAIN_SAVE) { diff --git a/drivers/gpu/drm/amd/amdgpu/psp_v12_0.c b/drivers/gpu/drm/amd/amdgpu/psp_v12_0.c index c4828bd3264b..618e5b6b85d9 100644 --- a/drivers/gpu/drm/amd/amdgpu/psp_v12_0.c +++ b/drivers/gpu/drm/amd/amdgpu/psp_v12_0.c @@ -138,10 +138,8 @@ static int psp_v12_0_bootloader_load_sysdrv(struct psp_context *psp) if (ret) return ret; - memset(psp->fw_pri_buf, 0, PSP_1_MEG); - /* Copy PSP System Driver binary to memory */ - memcpy(psp->fw_pri_buf, psp->sys_start_addr, psp->sys_bin_size); + psp_copy_fw(psp, psp->sys_start_addr, psp->sys_bin_size); /* Provide the sys driver to bootloader */ WREG32_SOC15(MP0, 0, mmMP0_SMN_C2PMSG_36, @@ -179,10 +177,8 @@ static int psp_v12_0_bootloader_load_sos(struct psp_context *psp) if (ret) return ret; - memset(psp->fw_pri_buf, 0, PSP_1_MEG); - /* Copy Secure OS binary to PSP memory */ - memcpy(psp->fw_pri_buf, psp->sos_start_addr, psp->sos_bin_size); + psp_copy_fw(psp, psp->sos_start_addr, psp->sos_bin_size); /* Provide the PSP secure OS to bootloader */ WREG32_SOC15(MP0, 0, mmMP0_SMN_C2PMSG_36, diff --git a/drivers/gpu/drm/amd/amdgpu/psp_v3_1.c b/drivers/gpu/drm/amd/amdgpu/psp_v3_1.c index f2e725f72d2f..d0a6cccd0897 100644 --- a/drivers/gpu/drm/amd/amdgpu/psp_v3_1.c +++ b/drivers/gpu/drm/amd/amdgpu/psp_v3_1.c @@ -102,10 +102,8 @@ static int psp_v3_1_bootloader_load_sysdrv(struct psp_context *psp) if (ret) return ret; - memset(psp->fw_pri_buf, 0, PSP_1_MEG); - /* Copy PSP System Driver binary to memory */ - memcpy(psp->fw_pri_buf, psp->sys_start_addr, psp->sys_bin_size); + psp_copy_fw(psp, psp->sys_start_addr, psp->sys_bin_size); /* Provide the sys driver to bootloader */ WREG32_SOC15(MP0, 0, mmMP0_SMN_C2PMSG_36, @@ -143,10 +141,8 @@ static int psp_v3_1_bootloader_load_sos(struct psp_context *psp) if (ret) return ret; - memset(psp->fw_pri_buf, 0, PSP_1_MEG); - /* Copy Secure OS binary to PSP memory */ - memcpy(psp->fw_pri_buf, psp->sos_start_addr, psp->sos_bin_size); + psp_copy_fw(psp, psp->sos_start_addr, psp->sos_bin_size); /* Provide the PSP secure OS to bootloader */ WREG32_SOC15(MP0, 0, mmMP0_SMN_C2PMSG_36, diff --git a/drivers/gpu/drm/amd/amdgpu/vce_v4_0.c b/drivers/gpu/drm/amd/amdgpu/vce_v4_0.c index 8e238dea7bef..90910d19db12 100644 --- a/drivers/gpu/drm/amd/amdgpu/vce_v4_0.c +++ b/drivers/gpu/drm/amd/amdgpu/vce_v4_0.c @@ -25,6 +25,7 @@ */ #include +#include #include "amdgpu.h" #include "amdgpu_vce.h" @@ -555,16 +556,19 @@ static int vce_v4_0_hw_fini(void *handle) static int vce_v4_0_suspend(void *handle) { struct amdgpu_device *adev = (struct amdgpu_device *)handle; - int r; + int r, idx; if (adev->vce.vcpu_bo == NULL) return 0; - if (adev->firmware.load_type == AMDGPU_FW_LOAD_PSP) { - unsigned size = amdgpu_bo_size(adev->vce.vcpu_bo); - void *ptr = adev->vce.cpu_addr; + if (drm_dev_enter(&adev->ddev, &idx)) { + if (adev->firmware.load_type == AMDGPU_FW_LOAD_PSP) { + unsigned size = amdgpu_bo_size(adev->vce.vcpu_bo); + void *ptr = adev->vce.cpu_addr; - memcpy_fromio(adev->vce.saved_bo, ptr, size); + memcpy_fromio(adev->vce.saved_bo, ptr, size); + } + drm_dev_exit(idx); } r = vce_v4_0_hw_fini(adev); @@ -577,16 +581,20 @@ static int vce_v4_0_suspend(void *handle) static int vce_v4_0_resume(void *handle) { struct amdgpu_device *adev = (struct amdgpu_device *)handle; - int r; + int r, idx; if (adev->vce.vcpu_bo == NULL) return -EINVAL; if (adev->firmware.load_type == AMDGPU_FW_LOAD_PSP) { - unsigned size = amdgpu_bo_size(adev->vce.vcpu_bo); - void *ptr = adev->vce.cpu_addr; - memcpy_toio(ptr, adev->vce.saved_bo, size); + if (drm_dev_enter(&adev->ddev, &idx)) { + unsigned size = amdgpu_bo_size(adev->vce.vcpu_bo); + void *ptr = adev->vce.cpu_addr; + + memcpy_toio(ptr, adev->vce.saved_bo, size); + drm_dev_exit(idx); + } } else { r = amdgpu_vce_resume(adev); if (r) diff --git a/drivers/gpu/drm/amd/amdgpu/vcn_v3_0.c b/drivers/gpu/drm/amd/amdgpu/vcn_v3_0.c index 3f15bf34123a..df34be8ec82d 100644 --- a/drivers/gpu/drm/amd/amdgpu/vcn_v3_0.c +++ b/drivers/gpu/drm/amd/amdgpu/vcn_v3_0.c @@ -34,6 +34,8 @@ #include "vcn/vcn_3_0_0_sh_mask.h" #include "ivsrcid/vcn/irqsrcs_vcn_2_0.h" +#include + #define mmUVD_CONTEXT_ID_INTERNAL_OFFSET 0x27 #define mmUVD_GPCOM_VCPU_CMD_INTERNAL_OFFSET 0x0f #define mmUVD_GPCOM_VCPU_DATA0_INTERNAL_OFFSET 0x10 @@ -268,16 +270,20 @@ static int vcn_v3_0_sw_init(void *handle) static int vcn_v3_0_sw_fini(void *handle) { struct amdgpu_device *adev = (struct amdgpu_device *)handle; - int i, r; + int i, r, idx; - for (i = 0; i < adev->vcn.num_vcn_inst; i++) { - volatile struct amdgpu_fw_shared *fw_shared; + if (drm_dev_enter(&adev->ddev, &idx)) { + for (i = 0; i < adev->vcn.num_vcn_inst; i++) { + volatile struct amdgpu_fw_shared *fw_shared; - if (adev->vcn.harvest_config & (1 << i)) - continue; - fw_shared = adev->vcn.inst[i].fw_shared_cpu_addr; - fw_shared->present_flag_0 = 0; - fw_shared->sw_ring.is_enabled = false; + if (adev->vcn.harvest_config & (1 << i)) + continue; + fw_shared = adev->vcn.inst[i].fw_shared_cpu_addr; + fw_shared->present_flag_0 = 0; + fw_shared->sw_ring.is_enabled = false; + } + + drm_dev_exit(idx); } if (amdgpu_sriov_vf(adev)) From patchwork Wed May 12 14:26:42 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Andrey Grodzovsky X-Patchwork-Id: 12253889 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-18.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,MSGID_FROM_MTA_HEADER,SPF_HELO_NONE, SPF_PASS,USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 15914C43600 for ; Wed, 12 May 2021 14:27:40 +0000 (UTC) Received: from gabe.freedesktop.org (gabe.freedesktop.org [131.252.210.177]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id B4DAF61411 for ; Wed, 12 May 2021 14:27:39 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org B4DAF61411 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=amd.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=dri-devel-bounces@lists.freedesktop.org Received: from gabe.freedesktop.org (localhost [127.0.0.1]) by gabe.freedesktop.org (Postfix) with ESMTP id 2386B6EC1A; Wed, 12 May 2021 14:27:38 +0000 (UTC) Received: from NAM11-DM6-obe.outbound.protection.outlook.com (mail-dm6nam11on2060.outbound.protection.outlook.com [40.107.223.60]) by gabe.freedesktop.org (Postfix) with ESMTPS id 971846EC14; Wed, 12 May 2021 14:27:36 +0000 (UTC) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=U+Eewd0CLuSwSrxK2iNd+N11Ag2uXXSflAUMvxU58wk/3a8nRqswUDjkXSihqO4RcikcJepDLP0yoh0vazSKq7r5syxS/WOUHcB70kN1rxf8jJv3+wVmF3gLirQ/P1K6CNkLkA7gVH2ECM5fqEwYx0zHjyg2kZ1m77fpBMPWBVwwQXIW1l4g7Tc1ojGxxPB1WlxHIbNHK6R+Awgbc3/5gwE6QrKyN8chv9RO1SlIOMqIOTV2XhH+hbKaSPyr9vc+4wr6pvlgvJMsKpch31WTsNkphvIplt8XbnNpyZZjV0Jk2cxAQDapvNyMXFHGyh3HHv5K+lGkmPJ3yjvJbzq4RA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=WvCJ3p4J5b6j3Y4MhS7JBFeYIKw8tX02HYjT2myyNew=; b=NClw1aosdmAom2YbdRgfYIU24BgaLP+kgESmVxFQBbjzAY/KYYkSGbYcZYD3RgX65BSz9cvDalAXMqu/sLeafv3s5TiWxsoi3SlzP0KSUFVYW8AWMO+zkObdrhb4lXzILyMTsYQAV4BLZS/RMgEhV6RXaRobxMdaEf2c1wBgKy7UwqzX29+8yNPAAlyxyGspDvs/l8pvApZs9QfPJLtSz6eo0rc2R8EY5FlBE2w7v1TgTfq4QgBxnIJUR1syFGNSt/kLO3FCYy4+/3zMcMCFhL6QrJCncU+GbgY9PUOB7jAnTQjwHPOiN8CdgbfdujO8NgIzbkvBlJ8FEHxLMNYL8Q== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=amd.com; dmarc=pass action=none header.from=amd.com; dkim=pass header.d=amd.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=WvCJ3p4J5b6j3Y4MhS7JBFeYIKw8tX02HYjT2myyNew=; b=nuzT6bWjXxYiWGuUlPvhCLkzrocNnKXBxeykUb8ZHUJA22yXqLzNURmZJ5B25myLUh1ss5V5YTxsxl0TBD8cAkn1INPtYwnTpEzalCJqxwN2NKsf7sNTJc+XTh0H/nv19TtT9YQubOuNbTdDfz1bVwNMte1diF1vXxJX/G4wwbI= Authentication-Results: lists.freedesktop.org; dkim=none (message not signed) header.d=none; lists.freedesktop.org; dmarc=none action=none header.from=amd.com; Received: from SN6PR12MB4623.namprd12.prod.outlook.com (2603:10b6:805:e9::17) by SN6PR12MB2845.namprd12.prod.outlook.com (2603:10b6:805:75::33) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4129.25; Wed, 12 May 2021 14:27:33 +0000 Received: from SN6PR12MB4623.namprd12.prod.outlook.com ([fe80::ad51:8c49:b171:856c]) by SN6PR12MB4623.namprd12.prod.outlook.com ([fe80::ad51:8c49:b171:856c%7]) with mapi id 15.20.4129.026; Wed, 12 May 2021 14:27:33 +0000 From: Andrey Grodzovsky To: dri-devel@lists.freedesktop.org, amd-gfx@lists.freedesktop.org, linux-pci@vger.kernel.org, ckoenig.leichtzumerken@gmail.com, daniel.vetter@ffwll.ch, Harry.Wentland@amd.com Subject: [PATCH v7 10/16] drm/sched: Make timeout timer rearm conditional. Date: Wed, 12 May 2021 10:26:42 -0400 Message-Id: <20210512142648.666476-11-andrey.grodzovsky@amd.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210512142648.666476-1-andrey.grodzovsky@amd.com> References: <20210512142648.666476-1-andrey.grodzovsky@amd.com> X-Originating-IP: [2607:fea8:3edf:49b0:7576:4f76:97d8:1487] X-ClientProxiedBy: YTOPR0101CA0032.CANPRD01.PROD.OUTLOOK.COM (2603:10b6:b00:15::45) To SN6PR12MB4623.namprd12.prod.outlook.com (2603:10b6:805:e9::17) MIME-Version: 1.0 X-MS-Exchange-MessageSentRepresentingType: 1 Received: from agrodzovsky-All-Series.hitronhub.home (2607:fea8:3edf:49b0:7576:4f76:97d8:1487) by YTOPR0101CA0032.CANPRD01.PROD.OUTLOOK.COM (2603:10b6:b00:15::45) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4108.30 via Frontend Transport; Wed, 12 May 2021 14:27:32 +0000 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: c7617c67-b5ff-49a0-b218-08d9155214bc X-MS-TrafficTypeDiagnostic: SN6PR12MB2845: X-MS-Exchange-Transport-Forked: True X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:6108; X-MS-Exchange-SenderADCheck: 1 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: KhyuVcsMoQnPkmDdtW1RBzrl3t+epjt0mjR37Gb5T3amsd9DFfRP6189FvF7IQghX39P5HgtZfyF0yoGJrZE/RPGjW8b2xosncgsEF136E+3wHuSh3163ujmXV+lKMIvPNVhQITmlK6vc56CPt7FX2LNGLPh0drXsEfwPtyq38+hsCifRucPtiCDWKnhU7RU0N7A20FqWMc277KoGT0LtrW+9plHP/iI2GP0iQ8iAdnxK7eQxkV8obGvhrKY5p+huUlctCmFhd3bOZCFySSbCha11mK1ZMwSS1AlAWV/Ad4a9+bLHuvwwyQal5D1OJ8nQxokfreCFT9Fc4kQ0qqCeUNeHiPFr1/2YSn0GBGVenJMd+coF3gZg1NXWx094y5qWC5w8pDQrXaTKKznqz1SmrgU8/leRMAvn0TXsIbgf7GIMcmQ4wTYvoeSr+8iZeVyUkZhq/TKwHvEybm8GuYta3yOYhxrJOS1yDQPkw48CnuMJxUBvb3uO0U1F0fGIoc0AGZv86WyywQFweFaGjr8EFMX+Y71+JR6BnDpvsF4Ez7I5KLH1bEutQ6xhao9OAG5pB6kAb+8KKF7AnAJyqWXmd/rW+ELefisxrRxUJ6oUgc= X-Forefront-Antispam-Report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:SN6PR12MB4623.namprd12.prod.outlook.com; PTR:; CAT:NONE; SFS:(4636009)(376002)(39860400002)(396003)(346002)(136003)(366004)(6486002)(8676002)(54906003)(316002)(6512007)(16526019)(6636002)(478600001)(1076003)(186003)(4326008)(44832011)(2906002)(6506007)(8936002)(5660300002)(2616005)(38100700002)(66574015)(36756003)(86362001)(6666004)(66476007)(66946007)(66556008)(83380400001)(52116002); DIR:OUT; SFP:1101; X-MS-Exchange-AntiSpam-MessageData: =?utf-8?q?Zp46b6ZgS+zr26iBldjJXQLdwSDsSX?= =?utf-8?q?D1eus1rn9gtCSaU4Y9nUZ1MO1O3W8HxEn0DYLxI/WXzr4apNpSIVc+7z+cUt3qrAp?= =?utf-8?q?ENWifcnw/giWT4hwaSiH9OcmqVNTpED5BVE66EG6kCg9JCep9Wrxd66E+c3tsTcUR?= =?utf-8?q?w6khxFP4UtHaF2Ky9bd9RIxBWQkJVjHYHyDxbmyqjuQKbe5B/rsOOLoTbgCmcUebg?= =?utf-8?q?xB7WJzfOCSPt85r1MhdXc3/apKYWxOcymceynTvc8ZCJa2pbqOLNaiM6m+78GTTHe?= =?utf-8?q?vWU8+un6p7XyOEEJyatOdFiGJ2R5qm2eQ9QFr7YhHZxdQZ5l+HTQsrdUcGNV2qTwD?= =?utf-8?q?h08kEXX9LxaoluLsJIxZa3h6ri/ePvZtICV5SAPvUq3TudFwKpfKWbc3lXMfKUDzK?= =?utf-8?q?tN7yqv9U18IUWUSnIXER78U05+BwJ0zutaK0H/NIwAurrAakfT38Zff2lfXjCjKuD?= =?utf-8?q?NYzcACvU3TOjy6gEiiLX+l4IbjFgXCR4b59+/odIgZ+XlTG3JoOwNldBRAB4vCAut?= =?utf-8?q?SVDyFF/m9/LmxR6zKNZ/3BtNF2hX6dxaR+g2cb8y80vwNNGy+tcH3+Axtb62mugbY?= =?utf-8?q?BG6dq8oxphOY85kZxSFvqZOWPDwVErQKfYi0IFMO1c9xBwxt/K2qkj3OlPompp8au?= =?utf-8?q?jdkvH2sMp2vCH+exbmH2x9eJixayTbPKVinKVFqF7bZjJAddx9i4sbEsITEi3vLGh?= =?utf-8?q?KTZm4fg8ebWeSI7fMOO8ANEHdH8vb+ksIIQ87ux9ZGbEKC8XLSQZCw/bju6zuxhMZ?= =?utf-8?q?fusDuOR6J3bhnsYEl+ltLHqhJdmptYgYY1K7sdhIvNKRnHjqriIcGZ7y139y5w83v?= =?utf-8?q?UvdSBStd+HsMJFDLLVQglo6v4XmeIkKLn77vo+PpXVzD7Kw/WFzImSeVeNv6yM0MO?= =?utf-8?q?IrVrk5yDy29sGGSk6M4qOrhw6twexo57s5RoqgPAFrFLSP0DqKzEwt/oCXZwEnOrR?= =?utf-8?q?P78R2n5nhMsAeLiZ/XNlzDXP3G+CBMqApsC7ZduU0ly5GjjeOSFh6zu8VFj2RGuCH?= =?utf-8?q?2a17d92557OZi+FQYnzqs06TjTIkpKkzSGw5CbDFFd1f5GVbdxXXYvPdttp9DHlcq?= =?utf-8?q?2CmlDf3FydafMqBzcMqZ1P4ywvMMYfDwMacRn0mSCJY9AsDVTgNCnEqvNOr73JzuG?= =?utf-8?q?DAXo2DU6OWMP5sBLwOMgJOJIrFOc8SShz4pHu56ZhxlNn1ufXLElAnkIshHWWr3zH?= =?utf-8?q?SI/dMzX9UVu8mw9q6NoVHKeIhmqf89iS4kMzChcV1ss0ikurzTNTOue7DyIT3O16j?= =?utf-8?q?FDOiDWCTifeaWmwsimHJg80rxKvPO43qkK6uW6O/CCWngsLKnjGLsuGxRP+iL0mBX?= =?utf-8?q?sm1Zh9fAAQps3?= X-OriginatorOrg: amd.com X-MS-Exchange-CrossTenant-Network-Message-Id: c7617c67-b5ff-49a0-b218-08d9155214bc X-MS-Exchange-CrossTenant-AuthSource: SN6PR12MB4623.namprd12.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 12 May 2021 14:27:33.1106 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: MSzpb4eWgClP/ko3EO32H9TwmiF7WT7dWTws8GSqK+BnTDsgBsii8mhwdYouToTo2sSwFo+RDOHj4ifOEO2Bhg== X-MS-Exchange-Transport-CrossTenantHeadersStamped: SN6PR12MB2845 X-BeenThere: dri-devel@lists.freedesktop.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Direct Rendering Infrastructure - Development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: gregkh@linuxfoundation.org, Felix.Kuehling@amd.com, helgaas@kernel.org, Alexander.Deucher@amd.com, =?utf-8?q?Christian_K=C3=B6nig?= Errors-To: dri-devel-bounces@lists.freedesktop.org Sender: "dri-devel" We don't want to rearm the timer if driver hook reports that the device is gone. v5: Update drm_gpu_sched_stat values in code. Signed-off-by: Andrey Grodzovsky Reviewed-by: Christian König --- drivers/gpu/drm/scheduler/sched_main.c | 11 +++++++---- 1 file changed, 7 insertions(+), 4 deletions(-) diff --git a/drivers/gpu/drm/scheduler/sched_main.c b/drivers/gpu/drm/scheduler/sched_main.c index f4f474944169..8d1211e87101 100644 --- a/drivers/gpu/drm/scheduler/sched_main.c +++ b/drivers/gpu/drm/scheduler/sched_main.c @@ -314,6 +314,7 @@ static void drm_sched_job_timedout(struct work_struct *work) { struct drm_gpu_scheduler *sched; struct drm_sched_job *job; + enum drm_gpu_sched_stat status = DRM_GPU_SCHED_STAT_NOMINAL; sched = container_of(work, struct drm_gpu_scheduler, work_tdr.work); @@ -331,7 +332,7 @@ static void drm_sched_job_timedout(struct work_struct *work) list_del_init(&job->list); spin_unlock(&sched->job_list_lock); - job->sched->ops->timedout_job(job); + status = job->sched->ops->timedout_job(job); /* * Guilty job did complete and hence needs to be manually removed @@ -345,9 +346,11 @@ static void drm_sched_job_timedout(struct work_struct *work) spin_unlock(&sched->job_list_lock); } - spin_lock(&sched->job_list_lock); - drm_sched_start_timeout(sched); - spin_unlock(&sched->job_list_lock); + if (status != DRM_GPU_SCHED_STAT_ENODEV) { + spin_lock(&sched->job_list_lock); + drm_sched_start_timeout(sched); + spin_unlock(&sched->job_list_lock); + } } /** From patchwork Wed May 12 14:26:43 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Andrey Grodzovsky X-Patchwork-Id: 12253887 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-18.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,MSGID_FROM_MTA_HEADER,SPF_HELO_NONE, SPF_PASS,USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 1F024C43462 for ; Wed, 12 May 2021 14:27:39 +0000 (UTC) Received: from gabe.freedesktop.org (gabe.freedesktop.org [131.252.210.177]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id C453561417 for ; Wed, 12 May 2021 14:27:38 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org C453561417 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=amd.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=dri-devel-bounces@lists.freedesktop.org Received: from gabe.freedesktop.org (localhost [127.0.0.1]) by gabe.freedesktop.org (Postfix) with ESMTP id AE4D06EC05; Wed, 12 May 2021 14:27:37 +0000 (UTC) Received: from NAM11-DM6-obe.outbound.protection.outlook.com (mail-dm6nam11on2060.outbound.protection.outlook.com [40.107.223.60]) by gabe.freedesktop.org (Postfix) with ESMTPS id BE6446EC18; Wed, 12 May 2021 14:27:36 +0000 (UTC) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=Ktk1Bp3p2ZH/zwOCuurfcUnlFQXMAhj06P29c3MCZU6ArpsKIMpWPtvVH8M44Gh4Cj2Zzp3o15F5Y9HQohsH/sx6VDS4VyeVsXPdi251qkMDPDHTbazYE7gtj+YAK+Gr1N4gO049iT5IwnvCb+TMSox4BBNyjPJ5OnDiF+bH48Ujkq44nOfMffqlKF/ZWZSq8i8N+oe1X0BZyoWWM3tQYIoFq+Eps4EicMYHO6Vj57ORfMnNDVusvzG2jZ6LLKwNJVVjicbMFGeSSkYFhzlOOzik+Icto8qTbtLp7OgGBluBkEa7llHXivl6zxQ0kOEojWK5XhF6mPqZW56FzVtnxQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=PsuHwoXxG2GPArc939kNaSlXDwLw+Fkof8ti3WmEEZA=; b=TNZ2FXE9QjUl1Xe4Ck/f4obAOyFv8MUe54VDvOY5qAyVn4+ifQq8jby1giUNrNFeXg86AF02TrT6acWB3QE0fRVpGo7WSafTDW7qQLWSxvktV54MG6BcHiOmo+VM+l6sCoP2qFxY9IdA5qz7QTgp6+018MIwsgjYxoX56bO8jum6+5zct4bEQZ5QDCkU/H8nsHVm6qVzqFuB/WRKExGjTQuzGQV8mVlrrKRfBbo7E+8CcqoYuQi/g2g5QV8aD0s1OFPX4GWNtuAFALHvIcuDpG1Yn+7ctU1kzt+yeIznTt7WDacxIu+wLkCInggbYWtn4vtBTwaVLSlPEclNNCQjXg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=amd.com; dmarc=pass action=none header.from=amd.com; dkim=pass header.d=amd.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=PsuHwoXxG2GPArc939kNaSlXDwLw+Fkof8ti3WmEEZA=; b=z2Z8DPhzF4Ztx63weg4NF1yA/a0sSiYhQPMy8ULpPIiCqcl3t59QbWPyDoojpMeBjkSlAZ2aIfm5zi7I7UhAPszzVDz+b5gn32eibXPX/5ZJdoLuQ2lLME/mSljYGWQ7sv7Nrm6JSAZZITKyBSxmqFpX1Y6WkjZovApyYs+6a14= Authentication-Results: lists.freedesktop.org; dkim=none (message not signed) header.d=none; lists.freedesktop.org; dmarc=none action=none header.from=amd.com; Received: from SN6PR12MB4623.namprd12.prod.outlook.com (2603:10b6:805:e9::17) by SN6PR12MB2845.namprd12.prod.outlook.com (2603:10b6:805:75::33) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4129.25; Wed, 12 May 2021 14:27:35 +0000 Received: from SN6PR12MB4623.namprd12.prod.outlook.com ([fe80::ad51:8c49:b171:856c]) by SN6PR12MB4623.namprd12.prod.outlook.com ([fe80::ad51:8c49:b171:856c%7]) with mapi id 15.20.4129.026; Wed, 12 May 2021 14:27:35 +0000 From: Andrey Grodzovsky To: dri-devel@lists.freedesktop.org, amd-gfx@lists.freedesktop.org, linux-pci@vger.kernel.org, ckoenig.leichtzumerken@gmail.com, daniel.vetter@ffwll.ch, Harry.Wentland@amd.com Subject: [PATCH v7 11/16] drm/amdgpu: Prevent any job recoveries after device is unplugged. Date: Wed, 12 May 2021 10:26:43 -0400 Message-Id: <20210512142648.666476-12-andrey.grodzovsky@amd.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210512142648.666476-1-andrey.grodzovsky@amd.com> References: <20210512142648.666476-1-andrey.grodzovsky@amd.com> X-Originating-IP: [2607:fea8:3edf:49b0:7576:4f76:97d8:1487] X-ClientProxiedBy: YTOPR0101CA0032.CANPRD01.PROD.OUTLOOK.COM (2603:10b6:b00:15::45) To SN6PR12MB4623.namprd12.prod.outlook.com (2603:10b6:805:e9::17) MIME-Version: 1.0 X-MS-Exchange-MessageSentRepresentingType: 1 Received: from agrodzovsky-All-Series.hitronhub.home (2607:fea8:3edf:49b0:7576:4f76:97d8:1487) by YTOPR0101CA0032.CANPRD01.PROD.OUTLOOK.COM (2603:10b6:b00:15::45) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4108.30 via Frontend Transport; Wed, 12 May 2021 14:27:34 +0000 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: bec6a8e7-d976-4c93-9d8e-08d9155215ee X-MS-TrafficTypeDiagnostic: SN6PR12MB2845: X-MS-Exchange-Transport-Forked: True X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:5797; X-MS-Exchange-SenderADCheck: 1 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: aOnQuY4GtXVhAzs2cVIDlW15ErGYjE9IiIOxxvp9z3jvdEctnikmIYr3Wy3jdWv0n6zUUscxqbUViOO4hCMCX8TL2VaWMzhu7FXEIvt8C6hbRQ/Za4LntQGGWEozvyhGYEuFNUTbr6zjMOlMvuo4Zeu86OZmMhhqsVro1EUX45uGIXVJvSIM4pUnMKUCbQR99NgpTupY72AW+Qsr3YUjpe6X5A7FJaezPT210MWl+MqffgZu5PD8fhR76O5APoHO9dhrgOEpWmvsWnyOGWvevLgj2SPUy70/BSsLuEJcchbGqFgVgAIfgnedG6NNf+EBBzYy9Bwn7dCp9nTJY9VFtNebdH3PewFyBilgcyskZkcAPP3IAMZGYEuyaN8kS5kYoqA//6gTNXi6IMyjWHOiRZ1pn7cdTtEVqL/zaDYU1gl1GL8kOtnmmBbG1y0HoJ3Gj7IynnQf+6CZ5NduuNz4vMEyaiiSQYqdslXUc6HIbwjkHyBADTidUNkhtwmxfGkoc6uJGLEAPPhStsyWcXIhNGOOMmqkcL8fi2+slM2KtS/OzaVpBwffKPAK8IDJIOVRFBGrTOkEk+0GO/5dVWkESmtQN93IHGSLUqrYaX3jryY= X-Forefront-Antispam-Report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:SN6PR12MB4623.namprd12.prod.outlook.com; PTR:; CAT:NONE; SFS:(4636009)(376002)(39860400002)(396003)(346002)(136003)(366004)(6486002)(8676002)(54906003)(316002)(6512007)(16526019)(6636002)(478600001)(1076003)(186003)(4326008)(44832011)(2906002)(6506007)(8936002)(5660300002)(2616005)(38100700002)(66574015)(36756003)(86362001)(6666004)(66476007)(66946007)(66556008)(83380400001)(52116002); DIR:OUT; SFP:1101; X-MS-Exchange-AntiSpam-MessageData: =?utf-8?q?1PoivChCVqO6Juez1W4546bV6ZINHM?= =?utf-8?q?HiFIuJlYpF7+/1baYrwFLavtPXTUp5d/TjS/4NkaZsismEzbCM1MImFlsYbtzmc1J?= =?utf-8?q?kM3YiAAzdEK2fjfsCa7xk/GbP9ji1dKNoogfEDVXo5Had1DdOHNXobrCEjSMRoLp1?= =?utf-8?q?N+UPGehF2vnWqXCOeZ80W9tK1jNpybI+L+s3HLUfEBhO2TrYJEcgB+9qYTdJSSbKC?= =?utf-8?q?VSdxFTXmrtbhu3MMSYxPPNRNdyvtO/azGTxT4jS2Ky2InRSKA9b9NKoAHk6hqY/5B?= =?utf-8?q?0guO9BZea6cljB4RUjPTjauzP3JkDisl2VpG6mDWRrNbi1u5NJ3jLJVaJ5xtr4Dix?= =?utf-8?q?F5+Cw4AEWr3HiGwkz6izf7DCC/XEoQz3Q1LmLZGVHHcKoiF4RekbRxw7KnC21C2lK?= =?utf-8?q?M1/UAT5ha12VoagF+NIjpYgFQB9ZhZ3l4T5AVIlhpcyLyYQaeD1VbggyKw3MUa3Qh?= =?utf-8?q?DgAD3Dp92bnnMLZ49CJdp5OpBCQSpg1g0khQBZyFlCXjxRzNQrGLGobFuV/BYVQ2N?= =?utf-8?q?yWyYojpmHjlfrkRCChnOsanxT6qtaWzxJRcNY72ROpnoXPMvY6Td1j0cb53q84zk+?= =?utf-8?q?MkO/mRnYTrumkiE3foaHui00TsF4oM2Y2LMpkL1GH2KEJIboA+xSFpQjGBHXDFkCX?= =?utf-8?q?pu80nCMno4hcD2cg9kWI7kF9PD9LvzrmHTUwx1+NcfBx+ADlS5fqPb/FS6jv0nDfn?= =?utf-8?q?zdFSC/tabqgpK6bnDl0xAxLGxCAH71B4z8cQA+IYSUe5mAdHXNrJbHPjD/cAi28CW?= =?utf-8?q?auyiHQyADXWfqLbg0Nqb1d7oUCM5T+L4yPEbgo9Cwo7R0GP789ZBjsNwRXfgweb5v?= =?utf-8?q?kGmJ0eSDlokziw40DVwc7/DYodFwOk0ITDUk9s+swALIZx+Ndit+PHBKmjgZYB7oA?= =?utf-8?q?43M5UoS8WZfmfkHNXiK/YQXaMd5MBKj8Zb3JiDe0f6gR3uSw5Z4yupdaGqr9c2h3T?= =?utf-8?q?uSATUkhmgY26uZZvx/WY4A03s5iaX2i5HRoh1JumHAVze+xLM1NfzwghQW6YcXV4W?= =?utf-8?q?SV7guhwsPkTjrPDmcq9ESTweV03zHxxBkoRI19EVBsMBuFyRFt6+/n5KLZPiqlFrM?= =?utf-8?q?Hz2ZLKNJadrgu9XrJdYdFoHulxupkqLhinDhAOsz/MH+zi9XAmvHsc51SgUtr4L3n?= =?utf-8?q?kVdrQIPPE1NbQzzf5oOyFwHMEgGeV3T9TUlUJoC49zz+BAcEFkcBFufgc0ldR+DAj?= =?utf-8?q?jT1V4MJww9BFnXX7DTTQdeE6Mo4EuncYkFxp2C6VjewmwKVcUjDqMF9J7HkVkYDmY?= =?utf-8?q?IdQnsNQhzK+pTGaCx9ERqFo0SZHAZ4OBBc+PqCawjvYMzQFKrozPRsB2fsovzIeKe?= =?utf-8?q?9IJ19RJfPzUM0?= X-OriginatorOrg: amd.com X-MS-Exchange-CrossTenant-Network-Message-Id: bec6a8e7-d976-4c93-9d8e-08d9155215ee X-MS-Exchange-CrossTenant-AuthSource: SN6PR12MB4623.namprd12.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 12 May 2021 14:27:35.1045 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: oOrrNCm3WrW2wSyBewI0vkE/bro2UGoAG67AedeQw7YjH0exhdt5Sbj2FNTAONp6UcR2niWdvgsBp0dP2jnOWw== X-MS-Exchange-Transport-CrossTenantHeadersStamped: SN6PR12MB2845 X-BeenThere: dri-devel@lists.freedesktop.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Direct Rendering Infrastructure - Development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: gregkh@linuxfoundation.org, Felix.Kuehling@amd.com, helgaas@kernel.org, Alexander.Deucher@amd.com, =?utf-8?q?Christian_K=C3=B6nig?= Errors-To: dri-devel-bounces@lists.freedesktop.org Sender: "dri-devel" Return DRM_TASK_STATUS_ENODEV back to the scheduler when device is not present so they timeout timer will not be rearmed. v5: Update to match updated return values in enum drm_gpu_sched_stat Signed-off-by: Andrey Grodzovsky Reviewed-by: Christian König --- drivers/gpu/drm/amd/amdgpu/amdgpu_job.c | 19 ++++++++++++++++--- 1 file changed, 16 insertions(+), 3 deletions(-) diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_job.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_job.c index 759b34799221..d33e6d97cc89 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_job.c +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_job.c @@ -25,6 +25,8 @@ #include #include +#include + #include "amdgpu.h" #include "amdgpu_trace.h" @@ -34,6 +36,15 @@ static enum drm_gpu_sched_stat amdgpu_job_timedout(struct drm_sched_job *s_job) struct amdgpu_job *job = to_amdgpu_job(s_job); struct amdgpu_task_info ti; struct amdgpu_device *adev = ring->adev; + int idx; + + if (!drm_dev_enter(&adev->ddev, &idx)) { + DRM_INFO("%s - device unplugged skipping recovery on scheduler:%s", + __func__, s_job->sched->name); + + /* Effectively the job is aborted as the device is gone */ + return DRM_GPU_SCHED_STAT_ENODEV; + } memset(&ti, 0, sizeof(struct amdgpu_task_info)); @@ -41,7 +52,7 @@ static enum drm_gpu_sched_stat amdgpu_job_timedout(struct drm_sched_job *s_job) amdgpu_ring_soft_recovery(ring, job->vmid, s_job->s_fence->parent)) { DRM_ERROR("ring %s timeout, but soft recovered\n", s_job->sched->name); - return DRM_GPU_SCHED_STAT_NOMINAL; + goto exit; } amdgpu_vm_get_task_info(ring->adev, job->pasid, &ti); @@ -53,13 +64,15 @@ static enum drm_gpu_sched_stat amdgpu_job_timedout(struct drm_sched_job *s_job) if (amdgpu_device_should_recover_gpu(ring->adev)) { amdgpu_device_gpu_recover(ring->adev, job); - return DRM_GPU_SCHED_STAT_NOMINAL; } else { drm_sched_suspend_timeout(&ring->sched); if (amdgpu_sriov_vf(adev)) adev->virt.tdr_debug = true; - return DRM_GPU_SCHED_STAT_NOMINAL; } + +exit: + drm_dev_exit(idx); + return DRM_GPU_SCHED_STAT_NOMINAL; } int amdgpu_job_alloc(struct amdgpu_device *adev, unsigned num_ibs, From patchwork Wed May 12 14:26:44 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrey Grodzovsky X-Patchwork-Id: 12253893 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-18.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,MSGID_FROM_MTA_HEADER,SPF_HELO_NONE, SPF_PASS,USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 077C8C43470 for ; Wed, 12 May 2021 14:27:47 +0000 (UTC) Received: from gabe.freedesktop.org (gabe.freedesktop.org [131.252.210.177]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id A2D7A61407 for ; Wed, 12 May 2021 14:27:46 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org A2D7A61407 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=amd.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=dri-devel-bounces@lists.freedesktop.org Received: from gabe.freedesktop.org (localhost [127.0.0.1]) by gabe.freedesktop.org (Postfix) with ESMTP id 2B3BC6EC1E; Wed, 12 May 2021 14:27:45 +0000 (UTC) Received: from NAM11-DM6-obe.outbound.protection.outlook.com (mail-dm6nam11on2072.outbound.protection.outlook.com [40.107.223.72]) by gabe.freedesktop.org (Postfix) with ESMTPS id A3D036EC1C; Wed, 12 May 2021 14:27:38 +0000 (UTC) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=Al+MD/yUii9sgCr+WqTDUSva3TJENFoIdlCDKmAhC4wJKo1lFhFk7uaOOIaKKZu8KCCffPbwtB+t5LVZRPYdLxh5uqFl42E/oPrEstdHGlCNgE/scx5tmY4dEDzDcxAIYGkpoertr7zysexi2ZOJ9Un2MMnXH8VscC286uHnyNyLeY5Mbof+JvVpsZNCy1m4SMknt1vjnPXVWvxqDE2NzHC+Vc+wjZsJa9nI6rVI3jnHf/C8CkgT3+u3O8TDZHl6gmwRCWUlqZqYqerj8dj+3U5Oz9KjBrcFdzsS2Bp+Mztqb6WUv7RDOygmpIppPKaKJcGn/wYAI0EwI0OTA02UTQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=9AZGyPcKDSup40OWapnZWYHJum+sOEaVRemzO6xhrCs=; b=XCopPD0T3zndcAwmLyQjDiBulcMCKGh3nd+eqBbk40Oxj/8fm9jYVeUUjskMixJeoQd/YBACTfVEf7agzzR3iVnIt80X6PiV84Pt/Ge0kW+K6+qOlWeRZltRRRqG6EW7XMkOR2E54rofW0dJ8b9Zo5nkKKfwHiUEywgOvIo1mgQt5EE9C7JLVOfl0+HBNLRER0xViUGm4GU0dsL+1jdo864GKZcu83MUtQG/EUOQflqueGfAa6XqlQFK67mvR5swLTzbyHpksscyHEO8SwZv+0ZTpnrxTVOpsjwoNUmc7sJn+j2K5tPO3MZMGQ32RuzW1aWKDgT5ghvW/VC1iroA/A== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=amd.com; dmarc=pass action=none header.from=amd.com; dkim=pass header.d=amd.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=9AZGyPcKDSup40OWapnZWYHJum+sOEaVRemzO6xhrCs=; b=cVfJUYrhZUrhI7/bpuEeT4MQqwdsNUYFKOSYgTbptxizTLPkffwHuhnBQcLf+ZNzGuKgjtQ+lK0Y4UxfsSSe166AjE/7+nopDdYveqGYuXqDceCgCS9OaRebuF0XbUoIRU0FVPfF8i9L2+5FEhe7ck94O1UI3K4wD7wH72l1c5c= Authentication-Results: lists.freedesktop.org; dkim=none (message not signed) header.d=none; lists.freedesktop.org; dmarc=none action=none header.from=amd.com; Received: from SN6PR12MB4623.namprd12.prod.outlook.com (2603:10b6:805:e9::17) by SN6PR12MB2845.namprd12.prod.outlook.com (2603:10b6:805:75::33) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4129.25; Wed, 12 May 2021 14:27:36 +0000 Received: from SN6PR12MB4623.namprd12.prod.outlook.com ([fe80::ad51:8c49:b171:856c]) by SN6PR12MB4623.namprd12.prod.outlook.com ([fe80::ad51:8c49:b171:856c%7]) with mapi id 15.20.4129.026; Wed, 12 May 2021 14:27:36 +0000 From: Andrey Grodzovsky To: dri-devel@lists.freedesktop.org, amd-gfx@lists.freedesktop.org, linux-pci@vger.kernel.org, ckoenig.leichtzumerken@gmail.com, daniel.vetter@ffwll.ch, Harry.Wentland@amd.com Subject: [PATCH v7 12/16] drm/amdgpu: Fix hang on device removal. Date: Wed, 12 May 2021 10:26:44 -0400 Message-Id: <20210512142648.666476-13-andrey.grodzovsky@amd.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210512142648.666476-1-andrey.grodzovsky@amd.com> References: <20210512142648.666476-1-andrey.grodzovsky@amd.com> X-Originating-IP: [2607:fea8:3edf:49b0:7576:4f76:97d8:1487] X-ClientProxiedBy: YTOPR0101CA0032.CANPRD01.PROD.OUTLOOK.COM (2603:10b6:b00:15::45) To SN6PR12MB4623.namprd12.prod.outlook.com (2603:10b6:805:e9::17) MIME-Version: 1.0 X-MS-Exchange-MessageSentRepresentingType: 1 Received: from agrodzovsky-All-Series.hitronhub.home (2607:fea8:3edf:49b0:7576:4f76:97d8:1487) by YTOPR0101CA0032.CANPRD01.PROD.OUTLOOK.COM (2603:10b6:b00:15::45) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4108.30 via Frontend Transport; Wed, 12 May 2021 14:27:35 +0000 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 6ecd6c91-2c03-4633-2824-08d9155216f4 X-MS-TrafficTypeDiagnostic: SN6PR12MB2845: X-MS-Exchange-Transport-Forked: True X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:3383; X-MS-Exchange-SenderADCheck: 1 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: LMM2BcB1boHQI6oPuGTSTPtATz0rpQBmEBbb2Z0ZTOJ/G0CelHp6khTk7jIh+yTdtGniONnNGujTCYWqZPGUSnoM4q+i5P/2b/m4cic4qSesk+nvYoKNCshokOjTP1q6/l00NAdfi8beN999e3XkqCXVlYJ+SG34Fp5563rpD9KsdPkQlG96L7mEpbuiQHTouv8JHREu/aHCq9ULlr6gDxjhp3BdwcHgn5+oZB8IDAW+yrwTm90qd3QVx5sN6qitEE5KFWfHRS1LAgOEsT/JpHL942vnsioASdTF/vRh4ebDFCXitLf7WyxVV5GYByx4ovhDdaXMmk2yOh/9Gz3uThvmQPkfOw5jWP62pzjHSh8BCj8AJcwTVhpJwf19e4u/sPZwjLSrZUSNZafMOGqM6nMB8smvnlzKC54LO9H1JJSWqExcV4afxmHzd9q/y3TaQv/7FH7JObrdFBDn78gcdPUEpfv/TDOyO7lhEge6EfhoRrgEsvTjxng4RjHXwoaVZueIINwAttYyIw0tLIJ2P5wIGR4FJJ/6cFoS2Z20cNr1+rJILnQv2nK9lDloFxVl7bWalmOWxLzmYYcenW8bjcbZw3nHhMwq2yw/Ziuh7Uw= X-Forefront-Antispam-Report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:SN6PR12MB4623.namprd12.prod.outlook.com; PTR:; CAT:NONE; SFS:(4636009)(376002)(39860400002)(396003)(346002)(136003)(366004)(6486002)(8676002)(316002)(6512007)(16526019)(6636002)(478600001)(1076003)(186003)(4326008)(44832011)(2906002)(6506007)(8936002)(5660300002)(2616005)(38100700002)(36756003)(86362001)(6666004)(66476007)(66946007)(66556008)(83380400001)(52116002); DIR:OUT; SFP:1101; X-MS-Exchange-AntiSpam-MessageData: SMnLgxxK8HjKnvv2Ec5ufYHHScQZ82VNJFsftwb+3xGUAqtOzmgfaPn/uImfXV7XzpuA1qkgrKg2ht98kYvGZiHmdbpGsQNr4xJ2j+P+ckXKC8GnVMdLFd2xmBH5Ml7FCuqvNV/r0hiGgjgU2oROosgLKcRvBjx9eIbJJmXXpkiAAIX26u6TqQE/DSuZe7fDHPMzs9qTQhU00aBUqh4ywnKbUfP+Pfn//ySO1LPEsafj3hQfq0A1nFfgPecu3smPF+1/zZeOqTzIiuRenCcMOkByGsimafzwCg04DZjj0A6aCqjD7BpvZCbbzESYzad6JR9TBAWqvd1SLuczodysAyVxmvOQj0HDAHHWxwLJMVj3OPgNHQKBbMqQKPDJ7pRtI4/+AYu8FIa3i3lGfrUk30lIxEy6B894LhGeDC+8ZyWPjY5pk3PwsDAvzc8NDxnKfYMOLg/o6Zqoj6KBxKJ5dGEYQk4KLCIxvs4gMG0qSpUPbfK7JL59XxJJR96TRpXPsf146b4UZ/hUMkfSAkZanR51nW6neMvZ5Wxwc399MRQp0PO+fNzRyra0W2Poejcir5EWuF54yFy+p9s3elfrw+U3rQl9OEGipAzAQ2+YGV+lH6Q3ivWibCp1G7KXrEIfqLMwQ39qe5kFfk7+9BDZSrqaAIDtUSuqZFiBTSTLRG4rq3EcaD5d2HlPbQxCStHvj14A41s5GkO3kbtRZ8OxV8uY7psIedBDEadgdV4Olp3pbLqvNoVElMWNH/KIvOrd200xcmi6pY8gVccRM0eJjzg03I9Bua5QLEpZyEZspWf5gclnqXDMh7htlM7W9dRtNyMVrpHXp5g7fiCK5/ZnwXVGsA8aKgZyyz8lhzRuNIVDCrIFYwB5TrnrlhGQw/h+f848ne+TfT8iFOJE7Ve8PFJacHqJ9T8DTjzE2ve2x/EdHWznnSiWwAcAIBqkjaBtbUfghRsP//e6WEGzDx/Q5BzP0v/ZM6Eb2iQ7kjksHeezdvxpqks+1aOA2p0mciWCiKc9PKrb3xeu263gZNtTMCqODxEU9+5kP1Hx8Y+8bH3mS8MVT9C14oojl3hvUKhsiSXIrNlhMxgFZNBHzaidJxEBlQAKQkedQoH4CcrZ8N/n3YVoDvo9f3YFd9qliMp/AeCtevD09Z1eQWTB1L6rH46alrsJTxnTLApXrndLXhXWeMD+PXiz71AqWPOWRhIDI3IkG0ay/BeVBLlLRJD+0Kzr6RQy0UGQToYN5q25ylPuXpuwa9EeiyvDdmW4oaBEFXoHNw/RsfCUYHjOVMWCdFt3y+tEdWHHMKUxMNhNFGFsgH7UXqodxZF/XalkdfGye7uG97CwPPG6gHheDJLGLidtMnbi4SgzRDWUI3rZsX3/Sjv9nlYziGp1RoIDsUae X-OriginatorOrg: amd.com X-MS-Exchange-CrossTenant-Network-Message-Id: 6ecd6c91-2c03-4633-2824-08d9155216f4 X-MS-Exchange-CrossTenant-AuthSource: SN6PR12MB4623.namprd12.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 12 May 2021 14:27:36.7696 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: 9ZJOFopSndo3JaLcVl2DwttKLFuCc20VJsTougAJxhQ8bSbokMC9KaORUqnZuAI9TuRkOWAAMS2FhSNb9Ys9Xg== X-MS-Exchange-Transport-CrossTenantHeadersStamped: SN6PR12MB2845 X-BeenThere: dri-devel@lists.freedesktop.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Direct Rendering Infrastructure - Development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: gregkh@linuxfoundation.org, Felix.Kuehling@amd.com, helgaas@kernel.org, Alexander.Deucher@amd.com Errors-To: dri-devel-bounces@lists.freedesktop.org Sender: "dri-devel" If removing while commands in flight you cannot wait to flush the HW fences on a ring since the device is gone. Signed-off-by: Andrey Grodzovsky Reviewed-by: Alex Deucher --- drivers/gpu/drm/amd/amdgpu/amdgpu_fence.c | 16 ++++++++++------ 1 file changed, 10 insertions(+), 6 deletions(-) diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_fence.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_fence.c index 1ffb36bd0b19..fa03702ecbfb 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_fence.c +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_fence.c @@ -36,6 +36,7 @@ #include #include +#include #include "amdgpu.h" #include "amdgpu_trace.h" @@ -525,8 +526,7 @@ int amdgpu_fence_driver_init(struct amdgpu_device *adev) */ void amdgpu_fence_driver_fini_hw(struct amdgpu_device *adev) { - unsigned i, j; - int r; + int i, r; for (i = 0; i < AMDGPU_MAX_RINGS; i++) { struct amdgpu_ring *ring = adev->rings[i]; @@ -535,11 +535,15 @@ void amdgpu_fence_driver_fini_hw(struct amdgpu_device *adev) continue; if (!ring->no_scheduler) drm_sched_fini(&ring->sched); - r = amdgpu_fence_wait_empty(ring); - if (r) { - /* no need to trigger GPU reset as we are unloading */ + /* You can't wait for HW to signal if it's gone */ + if (!drm_dev_is_unplugged(&adev->ddev)) + r = amdgpu_fence_wait_empty(ring); + else + r = -ENODEV; + /* no need to trigger GPU reset as we are unloading */ + if (r) amdgpu_fence_driver_force_completion(ring); - } + if (ring->fence_drv.irq_src) amdgpu_irq_put(adev, ring->fence_drv.irq_src, ring->fence_drv.irq_type); From patchwork Wed May 12 14:26:45 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Andrey Grodzovsky X-Patchwork-Id: 12253891 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-18.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,MSGID_FROM_MTA_HEADER,SPF_HELO_NONE, SPF_PASS,USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 2F125C43460 for ; Wed, 12 May 2021 14:27:44 +0000 (UTC) Received: from gabe.freedesktop.org (gabe.freedesktop.org [131.252.210.177]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id D0237613E9 for ; Wed, 12 May 2021 14:27:43 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org D0237613E9 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=amd.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=dri-devel-bounces@lists.freedesktop.org Received: from gabe.freedesktop.org (localhost [127.0.0.1]) by gabe.freedesktop.org (Postfix) with ESMTP id DEA726EC17; Wed, 12 May 2021 14:27:42 +0000 (UTC) Received: from NAM11-DM6-obe.outbound.protection.outlook.com (mail-dm6nam11on2076.outbound.protection.outlook.com [40.107.223.76]) by gabe.freedesktop.org (Postfix) with ESMTPS id D86F66EC1C; Wed, 12 May 2021 14:27:40 +0000 (UTC) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=Q8uJ/P4kEJylA6M5wShhbSBD4LzFcEWvzb7aXLDIRp6DgWanorX4Oz8/GytJlSd3v7dt+b5Ts+/BHbIMeQFMiHTCEiEdXPy8bybEniOC+BcxzI++VZQt5r3/rarGeXhZ+xd3+grssFdDINwL5bvkWXXKagXBR9p4hhX6zgmU3WfOgT7ih7eWu+GadtCk21yOoygbI1sI9JqKqYsWERs4IbaTOw/1ahy4lgrDT1ZqutIWccM5K52Jht0S+MIzbnt/SHPUVPSwW/tvx7x3u/bF7j9vFHDoUQhDDkM0vygqnYMz1Qx7E6P/bbIh6FGHzYzEu/QwLsANSwnHQdwT1KBZuQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=H/uPjWV0fg0dVVvp8px//s3OoPIuYbV9UKVinTXXgyM=; b=b6lerkiA7/BnD7wc3ITilvmwfkWtEt+ZCXZQEJN9SC4bVFSM9pbCNFxBfNoFkUaKG4jnVUR4CgOr0ufueGfvznUJisWxf1WhY7UBED9ruVuzSC1kG6q6BKzKY3SZXF26vQ24q2MfiOg8+icYt6e2irENyp7AwFd6PmnEtZdEAzJuvvRcM/5w/TNGFLJtv+5eKyCB/keTgLJakMpzhs3p5KaKyqVkpb1+MzQptxdzigngzgN+wt0BvJYG9KWkX2/Ki/C9Og6/K9y0eTo36tkTsPUCMZWYPsbeTbKShGVmn+/tjhnSOq/LIquEV9m5vtpIRmh9RZDeJNZbacnpktgamw== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=amd.com; dmarc=pass action=none header.from=amd.com; dkim=pass header.d=amd.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=H/uPjWV0fg0dVVvp8px//s3OoPIuYbV9UKVinTXXgyM=; b=Q1lhmhAFDXI/aTiHDu8ODto3TIDliCYTi6PtbzhMteZTLwuA2Ym4jywAMiFwGEiKgDF+NBz1wdJXe7wk2g4IpAzqrV8LbRJF3GDJctWOCmKJucy+HS1LPSuYj2ck4dgyQPHqZ+oLSZi2NmImO4GvXfhaNi5mZc2LX35etICN4cA= Authentication-Results: lists.freedesktop.org; dkim=none (message not signed) header.d=none; lists.freedesktop.org; dmarc=none action=none header.from=amd.com; Received: from SN6PR12MB4623.namprd12.prod.outlook.com (2603:10b6:805:e9::17) by SN6PR12MB2655.namprd12.prod.outlook.com (2603:10b6:805:72::17) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4108.27; Wed, 12 May 2021 14:27:38 +0000 Received: from SN6PR12MB4623.namprd12.prod.outlook.com ([fe80::ad51:8c49:b171:856c]) by SN6PR12MB4623.namprd12.prod.outlook.com ([fe80::ad51:8c49:b171:856c%7]) with mapi id 15.20.4129.026; Wed, 12 May 2021 14:27:38 +0000 From: Andrey Grodzovsky To: dri-devel@lists.freedesktop.org, amd-gfx@lists.freedesktop.org, linux-pci@vger.kernel.org, ckoenig.leichtzumerken@gmail.com, daniel.vetter@ffwll.ch, Harry.Wentland@amd.com Subject: [PATCH v7 13/16] drm/scheduler: Fix hang when sched_entity released Date: Wed, 12 May 2021 10:26:45 -0400 Message-Id: <20210512142648.666476-14-andrey.grodzovsky@amd.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210512142648.666476-1-andrey.grodzovsky@amd.com> References: <20210512142648.666476-1-andrey.grodzovsky@amd.com> X-Originating-IP: [2607:fea8:3edf:49b0:7576:4f76:97d8:1487] X-ClientProxiedBy: YTOPR0101CA0032.CANPRD01.PROD.OUTLOOK.COM (2603:10b6:b00:15::45) To SN6PR12MB4623.namprd12.prod.outlook.com (2603:10b6:805:e9::17) MIME-Version: 1.0 X-MS-Exchange-MessageSentRepresentingType: 1 Received: from agrodzovsky-All-Series.hitronhub.home (2607:fea8:3edf:49b0:7576:4f76:97d8:1487) by YTOPR0101CA0032.CANPRD01.PROD.OUTLOOK.COM (2603:10b6:b00:15::45) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4108.30 via Frontend Transport; Wed, 12 May 2021 14:27:37 +0000 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 3ce3949c-dc28-4dc4-7178-08d9155217fc X-MS-TrafficTypeDiagnostic: SN6PR12MB2655: X-MS-Exchange-Transport-Forked: True X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:663; X-MS-Exchange-SenderADCheck: 1 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: ZjodDNVbmuIH5JjKRtrRHLX1HTVK05REV2vn70KgDg58hJD+13RcCei+iEwJULjW35ktZGrUPe1LyY/S/4Rqk2zdRsY2XHDxgCcJNtxvB8Xm+xSj3As9vghir7t2hA95PaqZB3gtQ+CiEVaW2cqQb4NasjnvHp7AwVN/H9Ovp1dKQjlG9DkxOg1SZ64fsrRS24a4Yszpkc5NFepZlVWKPnyS3jGwXzuuFokYYb7Sa7atLUSkILwaOjimMvcKCUVvzav1NPgXXWqtkRo0AQVunCMzJplkGIE+ifP72zMOnhlWHWFzCR24kW/ihSb0SU+iqwTB+WtmuAgL2mNRI2A/C6b9mBp96RVyzCI0rv46vfTADmkqrTchfNeyligoONtdK4YGyebXPBvZzn47Yw1Yx6UKFg33CyV8QvD5ZeUIGPc0Z55kvHckgTJSTW9NFCSjZZY75sQsoEKKlpWLrc0biU0ohw0kInfZIvDshbN6W8XIUfXEC+ir7s7ISEDQlWp1T9yHwJGivpzEHRG30DLDZTdXaYw+9AcqT8wcsVl9lAIPPnSHNbz6W3khaDYAWgUvN9TCnbWcB/uMq3bqO9iRi0KgdjmudHjYNtH2+CZhPLg= X-Forefront-Antispam-Report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:SN6PR12MB4623.namprd12.prod.outlook.com; PTR:; CAT:NONE; SFS:(4636009)(136003)(39860400002)(366004)(376002)(346002)(396003)(54906003)(66556008)(38100700002)(6512007)(66946007)(66574015)(8936002)(4326008)(83380400001)(2616005)(44832011)(6506007)(1076003)(316002)(86362001)(186003)(16526019)(66476007)(8676002)(6666004)(2906002)(5660300002)(478600001)(6636002)(36756003)(6486002)(52116002); DIR:OUT; SFP:1101; X-MS-Exchange-AntiSpam-MessageData: =?utf-8?q?TvGKIKti8W7F0wJRvksdfGp2Hy9Ujk?= =?utf-8?q?TjKSMw6FRvCF5u3u2MhedJmuFPGxUCvpGPqT5QURcJMYxKBrHexB0fBeAZr3pumkG?= =?utf-8?q?LwHF2zycyCigvk0K+0gBOl8vPUfoaGOrYmMKo2L85SBBcSHMRDtuZ86ch5nzl7Ols?= =?utf-8?q?swSa7jZi9VHNQLLw3u5GHOmBCF4gxsdUBqclFQsifEIw4CsdOqY2gkXtRbjP0zkKj?= =?utf-8?q?BtD3D41Mm87LDGrikMXNLqTsLjkgGhtGJKmbHh3aKL3wOwCeWNQWK+6Z6j0ZruoGk?= =?utf-8?q?h3zN47+bLhojFxrFeVPgUrqO+1rsz9AT1knMxKnU00N+KrqqplMPtHUv2DvcAJc8h?= =?utf-8?q?3O0HIfCvkb5cBdkkv2C2KQx75F7y/MxvxRkGgR2ci4tXS+IMLL3G9Py+wtybIu9Sm?= =?utf-8?q?thM6MLKtHWJY1EdC0BB3QUeGYeN00MckiTklooHnl242X1MnbXi8SFdtPiv9LHUXF?= =?utf-8?q?9kGisM4YsjfR5CJTAHibWpWcXScg6psZhKEh0R0J5E9ULlJsiZeSxEkyrsBGqI7NP?= =?utf-8?q?YTow0dLzZePjTSD/7qoDHQZ4QCBx24VFacya6vGC1mRgInG4hb933kZ3ZAC8uGr/Z?= =?utf-8?q?Q8B/GRBz7K0NlAFc5KeHJACC/hUy3m77luJbvF0ll2AvPxI4NU3truz1aSog53gnl?= =?utf-8?q?Pv5w4HbUN5e5CiXA7MNHhbS2yYXR1VyiJ3o2RDAD0425b76t5NQkQ6U/tOaz5q/jc?= =?utf-8?q?+Prgd09rvG/v1tzwxxZtHMJEETFhk7FJH19AzBCqK1qSYAMvYXfw2w2fv2BgwlsWF?= =?utf-8?q?GnOaDWv9HNaaeNSV9lX/YA5ReBEXG0UExgEfqbruENL521584JZxtpd8igG+OTDsf?= =?utf-8?q?JHvWV//HRi4jlGtjCsZO6xAhqK0jspaHjEzIrS33P4MniwL6piOp+cTk98/THYR3s?= =?utf-8?q?NGJ2VHMfxYDV5R4POTakK8dB/sKAAHhaE1Pgw/4b40XCw4lxSSyzp92Nws8/IAEb+?= =?utf-8?q?FINwHWIRxkcx3bs2/qqR0MtLy0S9pMN6w73r8QpYcJl2rKCqbq0VZUihR0eeQ0RjT?= =?utf-8?q?A3jlfJbNUth+8f5ssOAphTWfnzTB2BSbTBZxKtMvr/0xnWq3t4gQrD4NT2Lh6cdX1?= =?utf-8?q?5SyzqB/WTy4H4jAQ3VFGVWmknEVrJq1UmA+GY0GNbfd8aanf6L8Zip1Xlgh3Vucc9?= =?utf-8?q?WK+qV7ju05xRrVW/VGDJ86wSGL3pjeaqBPanqrb1qGyz6oSxQFzK4on1tKjDi4Mez?= =?utf-8?q?qZByIg1v3MpV/xE5zdhcu0WCJzutIrOIhsVvO+/DAYqkaH3h56DZGLhd+MLwueaL+?= =?utf-8?q?2SPXxxXxamdNclae4B2Yr46my8uA5D9t1BplXeocSweE3QQoclCk1Ssn4yHdVPjas?= =?utf-8?q?tjSiYznSg0gLr?= X-OriginatorOrg: amd.com X-MS-Exchange-CrossTenant-Network-Message-Id: 3ce3949c-dc28-4dc4-7178-08d9155217fc X-MS-Exchange-CrossTenant-AuthSource: SN6PR12MB4623.namprd12.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 12 May 2021 14:27:38.4856 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: 6xHyO1wMWcGgBOWbhgkHoR0RY5/ooGfgTSLHNOV6MI91Z0+eBA8clj6mQRkTTkJjtwtF5QKp3YkfbCSnrnQ1Pw== X-MS-Exchange-Transport-CrossTenantHeadersStamped: SN6PR12MB2655 X-BeenThere: dri-devel@lists.freedesktop.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Direct Rendering Infrastructure - Development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: gregkh@linuxfoundation.org, Felix.Kuehling@amd.com, helgaas@kernel.org, Alexander.Deucher@amd.com, =?utf-8?q?Christian_K=C3=B6nig?= Errors-To: dri-devel-bounces@lists.freedesktop.org Sender: "dri-devel" Problem: If scheduler is already stopped by the time sched_entity is released and entity's job_queue not empty I encountred a hang in drm_sched_entity_flush. This is because drm_sched_entity_is_idle never becomes false. Fix: In drm_sched_fini detach all sched_entities from the scheduler's run queues. This will satisfy drm_sched_entity_is_idle. Also wakeup all those processes stuck in sched_entity flushing as the scheduler main thread which wakes them up is stopped by now. v2: Reverse order of drm_sched_rq_remove_entity and marking s_entity as stopped to prevent reinserion back to rq due to race. v3: Drop drm_sched_rq_remove_entity, only modify entity->stopped and check for it in drm_sched_entity_is_idle Signed-off-by: Andrey Grodzovsky Reviewed-by: Christian König --- drivers/gpu/drm/scheduler/sched_entity.c | 3 ++- drivers/gpu/drm/scheduler/sched_main.c | 24 ++++++++++++++++++++++++ 2 files changed, 26 insertions(+), 1 deletion(-) diff --git a/drivers/gpu/drm/scheduler/sched_entity.c b/drivers/gpu/drm/scheduler/sched_entity.c index 0249c7450188..2e93e881b65f 100644 --- a/drivers/gpu/drm/scheduler/sched_entity.c +++ b/drivers/gpu/drm/scheduler/sched_entity.c @@ -116,7 +116,8 @@ static bool drm_sched_entity_is_idle(struct drm_sched_entity *entity) rmb(); /* for list_empty to work without lock */ if (list_empty(&entity->list) || - spsc_queue_count(&entity->job_queue) == 0) + spsc_queue_count(&entity->job_queue) == 0 || + entity->stopped) return true; return false; diff --git a/drivers/gpu/drm/scheduler/sched_main.c b/drivers/gpu/drm/scheduler/sched_main.c index 8d1211e87101..a2a953693b45 100644 --- a/drivers/gpu/drm/scheduler/sched_main.c +++ b/drivers/gpu/drm/scheduler/sched_main.c @@ -898,9 +898,33 @@ EXPORT_SYMBOL(drm_sched_init); */ void drm_sched_fini(struct drm_gpu_scheduler *sched) { + struct drm_sched_entity *s_entity; + int i; + if (sched->thread) kthread_stop(sched->thread); + for (i = DRM_SCHED_PRIORITY_COUNT - 1; i >= DRM_SCHED_PRIORITY_MIN; i--) { + struct drm_sched_rq *rq = &sched->sched_rq[i]; + + if (!rq) + continue; + + spin_lock(&rq->lock); + list_for_each_entry(s_entity, &rq->entities, list) + /* + * Prevents reinsertion and marks job_queue as idle, + * it will removed from rq in drm_sched_entity_fini + * eventually + */ + s_entity->stopped = true; + spin_unlock(&rq->lock); + + } + + /* Wakeup everyone stuck in drm_sched_entity_flush for this scheduler */ + wake_up_all(&sched->job_scheduled); + /* Confirm no work left behind accessing device structures */ cancel_delayed_work_sync(&sched->work_tdr); From patchwork Wed May 12 14:26:46 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrey Grodzovsky X-Patchwork-Id: 12253899 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-18.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,MSGID_FROM_MTA_HEADER,SPF_HELO_NONE, SPF_PASS,USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 97679C43462 for ; Wed, 12 May 2021 14:27:53 +0000 (UTC) Received: from gabe.freedesktop.org (gabe.freedesktop.org [131.252.210.177]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 408A7613E9 for ; Wed, 12 May 2021 14:27:53 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 408A7613E9 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=amd.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=dri-devel-bounces@lists.freedesktop.org Received: from gabe.freedesktop.org (localhost [127.0.0.1]) by gabe.freedesktop.org (Postfix) with ESMTP id 59F7F6EC28; Wed, 12 May 2021 14:27:52 +0000 (UTC) Received: from NAM12-BN8-obe.outbound.protection.outlook.com (mail-bn8nam12on2072.outbound.protection.outlook.com [40.107.237.72]) by gabe.freedesktop.org (Postfix) with ESMTPS id 6FB666EC1C; Wed, 12 May 2021 14:27:42 +0000 (UTC) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=FxzF+XeNdq06tnZ2Kw6krWA1gSxrJv2whU9aZFTp+bSP1P81htYKGZpVdClNBFLctcIarQt6NdujxY/9KVT9segJHv/hqJmmu7RRZ9tSU45Zd4KgOMELNesHCFj9fBi70axl4Y1pMbm1jk8029bJ/6yU7RwtnHGMWoYCTfX6DlcHNdfkO9as64d+JGjXmARc0YIKvcMovw2wK6LFUJ1fYcht3AaDp7EQAxGkOEUBzsTF0ECSfMFXJpCjnK7cmLU6xjxR81iHfm7fbsxfXesprykSEJj+XXSfjF8mXEWD5V5OHpKPMoK3g65Nu68vNHmInw5SfH/KFmN9vlXgDfwf4g== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=y2srJ+addgQ1ntwlTW/iL8rlklYo9LRnsGnuD5iu8B0=; b=i7yEjEm1a03GPLfzkaevAyTwzYg8dbPR6cD70PTA8pF1yDEznC/Pm48HLzGxMSLcn0mxrxKBWgYXAW6HPcaZgF7WUPSepj6lWb+6O/37O9LI/l5gaXgJhuBI0BAnO3XlTBFxwHStzAiGi4QILFK/V6GxdB08lSLxnE5QeDB8E2Cd7d/eiWOpE98AGeyj8YUKQc0MwU8KZc75CSd94GGGyaRcC9lEdXKx9LQoYkRDh0xAYJZD9gzL/ViZOqfhw2RfA/W8M5S1fnaxkrW4ArAwFwkPV9yRhiXdQ5ulrupIhegA29f2z2TZFldsjcggOFT+YCGAuCvokeByld66m2+vsQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=amd.com; dmarc=pass action=none header.from=amd.com; dkim=pass header.d=amd.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=y2srJ+addgQ1ntwlTW/iL8rlklYo9LRnsGnuD5iu8B0=; b=pSvLr/I+1ySGS2SduquKV6Yu/H4URcEOGoSvN04c0ZhPyMoyOnqqtw33DMUpKlhXv3L07k0gHXJvcCNQdfbr/WBqgbdLlmK29TkBLg2TRCKbDB6DCzKjrT3f0939wqLbLdA458zKl0WFWW9YzomRs9Gw/1SyQ+njAQ0idrw5oNA= Authentication-Results: lists.freedesktop.org; dkim=none (message not signed) header.d=none; lists.freedesktop.org; dmarc=none action=none header.from=amd.com; Received: from SN6PR12MB4623.namprd12.prod.outlook.com (2603:10b6:805:e9::17) by SN6PR12MB2655.namprd12.prod.outlook.com (2603:10b6:805:72::17) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4108.27; Wed, 12 May 2021 14:27:40 +0000 Received: from SN6PR12MB4623.namprd12.prod.outlook.com ([fe80::ad51:8c49:b171:856c]) by SN6PR12MB4623.namprd12.prod.outlook.com ([fe80::ad51:8c49:b171:856c%7]) with mapi id 15.20.4129.026; Wed, 12 May 2021 14:27:40 +0000 From: Andrey Grodzovsky To: dri-devel@lists.freedesktop.org, amd-gfx@lists.freedesktop.org, linux-pci@vger.kernel.org, ckoenig.leichtzumerken@gmail.com, daniel.vetter@ffwll.ch, Harry.Wentland@amd.com Subject: [PATCH v7 14/16] drm/amd/display: Remove superfluous drm_mode_config_cleanup Date: Wed, 12 May 2021 10:26:46 -0400 Message-Id: <20210512142648.666476-15-andrey.grodzovsky@amd.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210512142648.666476-1-andrey.grodzovsky@amd.com> References: <20210512142648.666476-1-andrey.grodzovsky@amd.com> X-Originating-IP: [2607:fea8:3edf:49b0:7576:4f76:97d8:1487] X-ClientProxiedBy: YTOPR0101CA0032.CANPRD01.PROD.OUTLOOK.COM (2603:10b6:b00:15::45) To SN6PR12MB4623.namprd12.prod.outlook.com (2603:10b6:805:e9::17) MIME-Version: 1.0 X-MS-Exchange-MessageSentRepresentingType: 1 Received: from agrodzovsky-All-Series.hitronhub.home (2607:fea8:3edf:49b0:7576:4f76:97d8:1487) by YTOPR0101CA0032.CANPRD01.PROD.OUTLOOK.COM (2603:10b6:b00:15::45) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4108.30 via Frontend Transport; Wed, 12 May 2021 14:27:39 +0000 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: a6754147-f416-4ddd-9536-08d91552192f X-MS-TrafficTypeDiagnostic: SN6PR12MB2655: X-MS-Exchange-Transport-Forked: True X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:25; X-MS-Exchange-SenderADCheck: 1 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: 2yi6jAlKUlFql6CfwdEP2W84RtF2Go0lO3q/6qGRxQ4LBjLtI0c1u2ObHCl+3JzaKnfcMib6Nlh3HfL29Q6lyZirIwwVWbr1E+/tRCe8ot7gBAQHZNFQ1bpNp6ddJpMrxdyUxCQSku4e6HaXgWhy0m12hf5YVCatuALMc0uRzpFZTIf9QGTVex6hjcOai2l/FdKPlk9qoNRAsYLlXuEp6HaGI1CEqXfGv7CorQSos0wgMY/Ms3E8pBUjDNtfwR77cCk7/A7vjKklOc+xmdNGDw3fd8u308IBiPtJKe69M7M2/g/r0sGYwunXSAtuA/xvPm1qZdtEvZzYbAoSNcBscu1ZgtTgGBJpJ5PwFeRcWUCZ0VpVDezEl6O2MvCgPdD92CRs7Lrx/u0Fyav5FTaCGdPVvxHVcp19AsuLHOdyD/lscpya8cCzEtQeXilIvJ5G1nNw5Po4aP9TVK01dG6MoPkjUv/JfYST+XEj3Y9mtZ24n2+QDaHd3bZoq2fnCiqpBd2BqkPSpYq3sHKdw3UDkkj+kUPNH7534ZF9lipwQkaJU5tTtx+OO1BG5lffXZbE3Gz3Q1NiAoKPhLV2xP+8k2tAE/KCQ5Ls+AFsaXrqzfg= X-Forefront-Antispam-Report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:SN6PR12MB4623.namprd12.prod.outlook.com; PTR:; CAT:NONE; SFS:(4636009)(396003)(376002)(346002)(366004)(39860400002)(136003)(86362001)(316002)(186003)(16526019)(66476007)(1076003)(4744005)(6486002)(36756003)(52116002)(2906002)(5660300002)(8676002)(6666004)(6636002)(478600001)(6512007)(38100700002)(54906003)(66556008)(66946007)(4326008)(8936002)(6506007)(44832011)(2616005)(83380400001); DIR:OUT; SFP:1101; X-MS-Exchange-AntiSpam-MessageData: rZ8XKE3UzDLyBeOlFfHbEYfhvYHYpaH3C7kQZp2wbhrwsnwN34o4SvKFYgkOOi559GNtuTV+vhJPEcEmFnnJmLhu+xquqTcKyKm9+5DXnogVt0EHXArbcIjh6IOHhi5d+JiwsSUon2dQsTOhfj5oVbBOGHIc3JlrmXsWXqQyBr0HeQsgK74jOXeGgX4K43BKyfaeznisTfTz6GVEP9Gtix/SmBAUP4f68GRIOLPD+2VvY1Fhms3F08UgWm/m9W6xCxXlTWvfJDVKzgZflEkTbP06jMaoydFhhvG7E/pUF5PKY35J+tdYzJTMSgDvbwcoVDdCsLtSdZ/hm3P5iOKgnxSWaQBKs7Jym7rLLlYLOP1vBF20zmNHTXezoUru2Iz4dCG4Ro+h2p1u75vmTAQzOsmXTA12i5MXxJ7ij5DKZf8hOofw5ptBAm4IdVLUQ2HLh6A9Mdvqw+LvtO0uUwnXfPQkKgDrTO34ZZ6KnKX9d7G4hiFGnaKTRuSaxNNiKQLtfTCGiBYFm4ReDBMhxP0vm5B+5ER8mTmd1L3GOHsqwoh4BBOE1hU4eavdqa3gMM51foASGF4moJIqL3L4bgQKfE27znuhn3peeN6cRUJcLx/4kAct3DETtbivl46kx8NXYfLImAcXSRiIpT1zhESnKNabdakwh234khoCEKIgcjYAp8Ry5JtIxNywZo6L03xuYIdfnSQ1kbNsark0T1LQV+5QUVfBqjs0cplVTgIiHdOOQ31KDYlGSEpehaCdErlDio/Ba6M2vCnYa9EmSaS+h6Y0VuZkS1K6CJuMjNm4KenP5ZXWKHz0KGEd4YziYA6sdFjwGlOok7jIzGcJkp5m2vnpx6+TxxFELfS74ers9I2r3T+MQxDDSulzOaxVwCYV4v5QlZPSQGYKvT46iZLvcNGc5NSFNQu1auBLu79rWygx1UpNWa6o0Auw691bFrFfmKvR0BlSYUU9TBcc6XU3XRGtTMb2WfQjB3RXIZNjYyAfBupIqtYU0QUAOohUIhrAmHErkAg06043vbIUs10I0CIFqlbGrVr/+l+qrXcuTZBrDoaUa+tXcxyNBlTpUzAd1nU+EIzHxcLHLKut5S4flM78NS1iiEx66z/UbVGTDG/WDfUsRU9Eanb/FlD9bH3zC7NL25FgxyAYgUedICNeDKHy6OYkuG38WsVBqJ8BcusA8UuDb9P/Bt5YAXIluuTzCmW49eWzAuSGCb+Y3NLFIj+BQCYxEhEl47zDIxbQXTBWCJpKmsVc5GwmGjtHTnEzZ6Px7yjXwqaV/ClsrCIvg6rgiv/OrZEd6DtMN28xVzCR18RcLN0vCOmXW6yMl8K/45Q/9da3wK71YPgKRDny8B+yG51yOOc2GJAJyAjYKG9qMlo4F9JQDYjIIZjE9FqL X-OriginatorOrg: amd.com X-MS-Exchange-CrossTenant-Network-Message-Id: a6754147-f416-4ddd-9536-08d91552192f X-MS-Exchange-CrossTenant-AuthSource: SN6PR12MB4623.namprd12.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 12 May 2021 14:27:40.6434 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: YZrQGXi+q/WwCCl5y6fU7M4fdDPHb/gVUlhT0PjfQDDyWdx20d84WAvvAASa2NgtP9L7qfXZawZERDPpDQ8d4A== X-MS-Exchange-Transport-CrossTenantHeadersStamped: SN6PR12MB2655 X-BeenThere: dri-devel@lists.freedesktop.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Direct Rendering Infrastructure - Development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: gregkh@linuxfoundation.org, Felix.Kuehling@amd.com, Rodrigo Siqueira , helgaas@kernel.org, Alexander.Deucher@amd.com Errors-To: dri-devel-bounces@lists.freedesktop.org Sender: "dri-devel" It's already being released by DRM core through devm Signed-off-by: Andrey Grodzovsky Reviewed-by: Rodrigo Siqueira --- drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm.c | 1 - 1 file changed, 1 deletion(-) diff --git a/drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm.c b/drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm.c index 6c2c6a51ce6c..9728a0158bcb 100644 --- a/drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm.c +++ b/drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm.c @@ -3757,7 +3757,6 @@ static int amdgpu_dm_initialize_drm_device(struct amdgpu_device *adev) static void amdgpu_dm_destroy_drm_device(struct amdgpu_display_manager *dm) { - drm_mode_config_cleanup(dm->ddev); drm_atomic_private_obj_fini(&dm->atomic_obj); return; } From patchwork Wed May 12 14:26:47 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Andrey Grodzovsky X-Patchwork-Id: 12253895 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-18.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,MSGID_FROM_MTA_HEADER,SPF_HELO_NONE, SPF_PASS,USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id DAB83C433B4 for ; Wed, 12 May 2021 14:27:48 +0000 (UTC) Received: from gabe.freedesktop.org (gabe.freedesktop.org [131.252.210.177]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 8653E613E9 for ; Wed, 12 May 2021 14:27:48 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 8653E613E9 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=amd.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=dri-devel-bounces@lists.freedesktop.org Received: from gabe.freedesktop.org (localhost [127.0.0.1]) by gabe.freedesktop.org (Postfix) with ESMTP id A791A6EC25; Wed, 12 May 2021 14:27:46 +0000 (UTC) Received: from NAM11-DM6-obe.outbound.protection.outlook.com (mail-dm6nam11on2051.outbound.protection.outlook.com [40.107.223.51]) by gabe.freedesktop.org (Postfix) with ESMTPS id 033DA6EC21; Wed, 12 May 2021 14:27:43 +0000 (UTC) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=iKmbXIVIz740lvjNIQ0mLeWN2GULnqI0leMhUy8lsRkI/LG/26YZhEWHdN2ZoUIBsdS/MA1W4OqrSR4sKfuclcJZTe1NG0Dj1aLjK9QLnIFmcC8Su1+QBDLh7FWQvW2fjILOu8UjasOnMORDE9hs+thKEfLF24OV/TKxG1+foJNRycudfcjp856Rzx7Stnl+hhk1Gx4gdDMRRyGqOKDT6c006kkVHMOFJVVfc+SJops7mdBuBlAe60lcNXlmvFK8pEFvuRJDC9m/aDFBw1u0iNIEh/5jGMFSNV4oKMHviqPbsjghh00oyH7WiR+xg0lW8zzzvgc0KAQTlF9Xd0cPsg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=y9T0dd+0o5MpTwVzYXgN4zXaKXsjwhA7tqNN2ODWw44=; b=fidbEuDx4TBBgumRdkQyuJ39qIL9bYt/jWC9R2WPoLxEdAHcSYlkBzWFjogHpVG9Cy6t0JemHnsQnXn9oCpqK28n2xs5Y3fAThM56eifh2PqfuQb/k3lGOf879nzVsafWr4HnSE0+GP3YssCLLcWFV0VGxM0lCWURCYLyowJiOCTvdQs8JEo7CeNWXGtbOLgEBwikrDw9c4K4d1avCJwp8ISCIMH85fT0DuwFJz/8K3hGuCo6NBD0xFcG7adZ+b1GlDa6W5d325rb5n0bNaIIM8dXw9UQqhv8IjAkjVhyWDA/pl3RmNhmeG3cVpcSbEr4o1YzrvKLfxMa2hqkML34Q== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=amd.com; dmarc=pass action=none header.from=amd.com; dkim=pass header.d=amd.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=y9T0dd+0o5MpTwVzYXgN4zXaKXsjwhA7tqNN2ODWw44=; b=GGGmrT9MA/Y1PBzi6advQiXfPOBzUgweUmn4Og6SbED6EiBSslkjNOJk24TFn0Dg8qOg7afWcfJv9VQsSj85ftKiFxsP8dSIGdnCfUj+SfmHYWZkUJUmiogKogztbIL/JBYUO807hBltX5U4FQnjeuyZSPE5Hw6VqFd14Q3ZuMk= Authentication-Results: lists.freedesktop.org; dkim=none (message not signed) header.d=none; lists.freedesktop.org; dmarc=none action=none header.from=amd.com; Received: from SN6PR12MB4623.namprd12.prod.outlook.com (2603:10b6:805:e9::17) by SN6PR12MB2655.namprd12.prod.outlook.com (2603:10b6:805:72::17) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4108.27; Wed, 12 May 2021 14:27:42 +0000 Received: from SN6PR12MB4623.namprd12.prod.outlook.com ([fe80::ad51:8c49:b171:856c]) by SN6PR12MB4623.namprd12.prod.outlook.com ([fe80::ad51:8c49:b171:856c%7]) with mapi id 15.20.4129.026; Wed, 12 May 2021 14:27:42 +0000 From: Andrey Grodzovsky To: dri-devel@lists.freedesktop.org, amd-gfx@lists.freedesktop.org, linux-pci@vger.kernel.org, ckoenig.leichtzumerken@gmail.com, daniel.vetter@ffwll.ch, Harry.Wentland@amd.com Subject: [PATCH v7 15/16] drm/amdgpu: Verify DMA opearations from device are done Date: Wed, 12 May 2021 10:26:47 -0400 Message-Id: <20210512142648.666476-16-andrey.grodzovsky@amd.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210512142648.666476-1-andrey.grodzovsky@amd.com> References: <20210512142648.666476-1-andrey.grodzovsky@amd.com> X-Originating-IP: [2607:fea8:3edf:49b0:7576:4f76:97d8:1487] X-ClientProxiedBy: YTOPR0101CA0032.CANPRD01.PROD.OUTLOOK.COM (2603:10b6:b00:15::45) To SN6PR12MB4623.namprd12.prod.outlook.com (2603:10b6:805:e9::17) MIME-Version: 1.0 X-MS-Exchange-MessageSentRepresentingType: 1 Received: from agrodzovsky-All-Series.hitronhub.home (2607:fea8:3edf:49b0:7576:4f76:97d8:1487) by YTOPR0101CA0032.CANPRD01.PROD.OUTLOOK.COM (2603:10b6:b00:15::45) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4108.30 via Frontend Transport; Wed, 12 May 2021 14:27:41 +0000 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: e7fc7ee3-4694-4272-4a18-08d915521a45 X-MS-TrafficTypeDiagnostic: SN6PR12MB2655: X-MS-Exchange-Transport-Forked: True X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:5516; X-MS-Exchange-SenderADCheck: 1 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: 35aEVVycR3xugV1itAF2X1WDzwEQWf2grfqI+lMSpat1x1SJJa0NeWGgPlWyCPWIqTwiUzOyWQSEcqHLzcgmOcxZiqi9aqn7KM5lG71hpDt0ET6KWCEYVYnKu8hwgpgzfM3N6qWmPoBYhCD6q7SoTMfATg+Xq7w4JbRFMMLbeoKbB6EikH57yej6wkVIbbV3rNjJKjVjiKArMJjmE5veMO8TsBFwg43GkqKOqofvULv6gymvR/NGV0a6k0XOrZjeTHuEZG6rwv17TI1snI4P8J9IMRUWvAlLQ3pGOJ78dlyL4vaQW3qs7YXYPRN7tQLp/WJBwReWQCd3zaSqM3xaqJRqI3ZtDpWnfsTtcUrw94dRtdHuEYy09E9L6Sb9UMm7gdrTEqXkLVo/LN99MstQnnkn/BC6lCxUS8Bzoxxtd+1e6DKnpwPzfi5yVgEWmA6YwdyJEMr3ongdEmzK2R+WJzkQYHiBRHFMDd92/PGhjkQb1wrl3N/fihS91se4ngcPb1U4WfGiZbbthHP/XSiACDAQZWi47Q3ULLxWzmRQYbgJqQsSuzrU2jA10UX9zzznjhJm1/jSJF7f2TrQzRs2Q9LMESJPtYm2FIZPXaWfeJs= X-Forefront-Antispam-Report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:SN6PR12MB4623.namprd12.prod.outlook.com; PTR:; CAT:NONE; SFS:(4636009)(396003)(376002)(346002)(366004)(39860400002)(136003)(86362001)(316002)(186003)(16526019)(66476007)(1076003)(4744005)(6486002)(36756003)(52116002)(2906002)(5660300002)(8676002)(6636002)(478600001)(6512007)(38100700002)(54906003)(66556008)(66946007)(66574015)(4326008)(8936002)(6506007)(44832011)(15650500001)(2616005)(83380400001); DIR:OUT; SFP:1101; X-MS-Exchange-AntiSpam-MessageData: =?utf-8?q?cEEmD6wOze1Kgn0Nit88sO6bJrmkVl?= =?utf-8?q?wjWop82vYXpWQx/r7LgJyCZDEg/6xA0MRRTdpLIpDT7wPV0dsBCr4lwFzLX959DOu?= =?utf-8?q?r6+2IvKtGvsAHU9Y2o1DUZkc06vIYokmjh4sHUU5DPEHqj/WC+wHkVIXmD7eqb2uY?= =?utf-8?q?yxAT06+B2/ppyPGXma4bhLQwJSxfldt3/hbdZ5DDrFlNStjFFPRdHRohFInbcQvp6?= =?utf-8?q?xOf0XB2p37Dlqg6SemM6+OQCwJusrFkYvPyaxeELThyl7tIVHdRvGbeN1dCFWe891?= =?utf-8?q?AddV4gIusT/in3xkh7ZGhqPVSN+BIm1wdWqQcesvPAN4CnEbVbvSETtX5fI8TtoAO?= =?utf-8?q?6qJt9Yw5RCBsHch2I8QslrBosNiOH9uhy4srr7oVIoUF6oG4Jfbklr3FkwJBcInvo?= =?utf-8?q?XVw0AztWTCQJoFPVD6xVLTvvsOR85tLFegH7WFJUkrGE6nSXML/nNbOk3TD0zOs2o?= =?utf-8?q?z/crosJjn+ziSdhMJKAWxZZ4GkrRiuGI4Bu+y7QAIzIkEwWdKhiM+yYpyazPmOOEN?= =?utf-8?q?E8MUmi5SPayUh+4RICUTpuD6rYkGBUsMBVrBNreimvDLUrbbXGyNPKQCtnjInfhVt?= =?utf-8?q?3tzBndCtb/tAKt2aay8uoiziXZJo3W6xprMMaai58R24xzVIO6rtArAweI0z99/o6?= =?utf-8?q?8hSatIcGB9LzHyrigwBzziGu5U1QoeDRTTB9A5N8WRlYDOqtAXJid75nkmga9dH0p?= =?utf-8?q?Ote4YK5vouRuHFZaSUQ3ZPam1/xDJ10JO0c8wYrr81RLsngme28FkwrYGlIW6XA3m?= =?utf-8?q?6/IbFY73NbPrVGfD5LFcU1i7frWZfLZoZEG4qtTsbi0/XSAZD+kNX6FY3wUL2xvIc?= =?utf-8?q?UUQqkABXWS5+aEJlgKaB6XDuZRU3zZnYqT7INNuh8lM0y42S2T1ZZhiZxBd/Vi21B?= =?utf-8?q?izY0S8i1owYZEYP4kmXDlLKdDo8dngFws/hQKAiFtxUR+E2Pw/c9cjrZ8DfgYfXz8?= =?utf-8?q?CmRLRvmA0gYPYiA59z+qCYDN3Hi1eGfpE+cFFEip/8K/r+X22LwOziy6EKaM0GcoJ?= =?utf-8?q?IvNksTld8emzHdUnB0tpl23m7D/i717TrKcG2HlCWPKbWy6ckhk5AeE9I1E9VxAwt?= =?utf-8?q?ZVUnvtEKk1+86Va88OIJwNAQj8+AkNIjRpzuPn0VTPPMyFStopRJui4t07txvWEIT?= =?utf-8?q?cJREc4WVDMSC+wCuHxAwJRHqx1Y5GztUJuSFosPOZKBeKtDiqEKdLKSBWFPHQW8UH?= =?utf-8?q?RVYNmJNj9i9bvHzSmjBYTH1bRjYZxY0voc/YersqfQQYTswnrK346PTNc48rm6XBx?= =?utf-8?q?dvO+AITVC0LvPEOuyy+Wxd1nsgMBoq6C/apK5Q7x4/Ilr0on9GuoIFgXkoLg24H2U?= =?utf-8?q?/JD4XCtEoF5AC?= X-OriginatorOrg: amd.com X-MS-Exchange-CrossTenant-Network-Message-Id: e7fc7ee3-4694-4272-4a18-08d915521a45 X-MS-Exchange-CrossTenant-AuthSource: SN6PR12MB4623.namprd12.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 12 May 2021 14:27:42.3854 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: rsEIMopaMfYvp77SD6zSyECvOmb5s2/E/sryRK/pm2DIsy0hFkRp+WyHowgklbJw5i8h5WhZm5+P2SilLXQq5Q== X-MS-Exchange-Transport-CrossTenantHeadersStamped: SN6PR12MB2655 X-BeenThere: dri-devel@lists.freedesktop.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Direct Rendering Infrastructure - Development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: gregkh@linuxfoundation.org, Felix.Kuehling@amd.com, helgaas@kernel.org, Alexander.Deucher@amd.com, =?utf-8?q?Christian_K=C3=B6nig?= Errors-To: dri-devel-bounces@lists.freedesktop.org Sender: "dri-devel" In case device remove is just simualted by sysfs then verify device doesn't keep doing DMA to the released memory after pci_remove is done. Signed-off-by: Andrey Grodzovsky Acked-by: Christian König --- drivers/gpu/drm/amd/amdgpu/amdgpu_drv.c | 6 ++++++ 1 file changed, 6 insertions(+) diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_drv.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_drv.c index 83006f45b10b..5e6af9e0b7bf 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_drv.c +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_drv.c @@ -1314,7 +1314,13 @@ amdgpu_pci_remove(struct pci_dev *pdev) drm_dev_unplug(dev); amdgpu_driver_unload_kms(dev); + /* + * Flush any in flight DMA operations from device. + * Clear the Bus Master Enable bit and then wait on the PCIe Device + * StatusTransactions Pending bit. + */ pci_disable_device(pdev); + pci_wait_for_pending_transaction(pdev); } static void From patchwork Wed May 12 14:26:48 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrey Grodzovsky X-Patchwork-Id: 12253897 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-18.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,MSGID_FROM_MTA_HEADER,SPF_HELO_NONE, SPF_PASS,USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 6BF1EC43460 for ; Wed, 12 May 2021 14:27:50 +0000 (UTC) Received: from gabe.freedesktop.org (gabe.freedesktop.org [131.252.210.177]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 14839613E9 for ; Wed, 12 May 2021 14:27:50 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 14839613E9 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=amd.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=dri-devel-bounces@lists.freedesktop.org Received: from gabe.freedesktop.org (localhost [127.0.0.1]) by gabe.freedesktop.org (Postfix) with ESMTP id 0704E6EC23; Wed, 12 May 2021 14:27:49 +0000 (UTC) Received: from NAM12-BN8-obe.outbound.protection.outlook.com (mail-bn8nam12on2048.outbound.protection.outlook.com [40.107.237.48]) by gabe.freedesktop.org (Postfix) with ESMTPS id 5FA096EC24; Wed, 12 May 2021 14:27:46 +0000 (UTC) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=deTgeZwP3+Wb/n0Zkxc1CkgFhsxd9lU38sNGFdk+CmesjSyFCk1gIG3qf7vi0v81fO9DsnK2J6T2U0TMbd2sNcrXf3otD5jvQYKB6dig/8Y1SohgvmIeD/YZKniPPFdYDeKCN2hEQ47lIkFswIicByqLTahXQOmdquI0nT9CNuEPg0s6um3x1EX23lNoayrc9uCR4oDzcwj1dPXeoK4/DuYMUBtW/VmURr9/gYfIS5owQBTZdQ9yR3rINz7sK38oZH3iTvwsm4TV7Czpld5qKNxP+aislMtkLuRHbArUE7XTlEVSqr0dc+CATnMBcx2q9CAdjZ5+I6UrBKoJCylIeg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=7NnAB5+Ww9eG0XsaV8ZDYC8M60EqyqkYdAAVZSiV2A0=; b=TgPlZfcKjvB9+qMWkNoFAalyl9kj9LVLcK22BCEArdElvjB3p0iS0X7V/onOEksZuQ7Lw3PgmCNZtw8ui71npzKIfKT3We3f+WOAx20ZzWtGCDm1ZHvo4j4m94c9x7vG6wJ3FzRA7KgO5/gA8vFaXIefS2WnGJjhGq8VYl6+50ngtozkbUOgnieFArWa+9DPlw0Nt5HbDJk8mLwiYAtw9XvnZDJIFCxmOCjj5R9suk84hlvY+cJ2KnIecNkykHVKAEvdrqqmu3MwHom4bZRqILDt2ms04d2ABnWiupYMwJwhXR7ktMxC0mAIfRmCqZWbIo1JshG9RbnZcYlEYMb+kQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=amd.com; dmarc=pass action=none header.from=amd.com; dkim=pass header.d=amd.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=7NnAB5+Ww9eG0XsaV8ZDYC8M60EqyqkYdAAVZSiV2A0=; b=tTdXw8lp418J/vNQyU/MJy550CQmdXYyA0DnjQ77/NEZ6nJ/ahNBjKJz2/6de23xJGJX7Rf+Tnro8ehfiTiyPUZhU7UdzRX1uEjgyRAIpjshIfilATfJM6eK1bEonweFYXnPH59Ubj9CHnBPHWnByiNnoafHFffGS5lBpko6fkw= Authentication-Results: lists.freedesktop.org; dkim=none (message not signed) header.d=none; lists.freedesktop.org; dmarc=none action=none header.from=amd.com; Received: from SN6PR12MB4623.namprd12.prod.outlook.com (2603:10b6:805:e9::17) by SN6PR12MB2655.namprd12.prod.outlook.com (2603:10b6:805:72::17) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4108.27; Wed, 12 May 2021 14:27:44 +0000 Received: from SN6PR12MB4623.namprd12.prod.outlook.com ([fe80::ad51:8c49:b171:856c]) by SN6PR12MB4623.namprd12.prod.outlook.com ([fe80::ad51:8c49:b171:856c%7]) with mapi id 15.20.4129.026; Wed, 12 May 2021 14:27:44 +0000 From: Andrey Grodzovsky To: dri-devel@lists.freedesktop.org, amd-gfx@lists.freedesktop.org, linux-pci@vger.kernel.org, ckoenig.leichtzumerken@gmail.com, daniel.vetter@ffwll.ch, Harry.Wentland@amd.com Subject: [PATCH v7 16/16] drm/amdgpu: Unmap all MMIO mappings Date: Wed, 12 May 2021 10:26:48 -0400 Message-Id: <20210512142648.666476-17-andrey.grodzovsky@amd.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210512142648.666476-1-andrey.grodzovsky@amd.com> References: <20210512142648.666476-1-andrey.grodzovsky@amd.com> X-Originating-IP: [2607:fea8:3edf:49b0:7576:4f76:97d8:1487] X-ClientProxiedBy: YTOPR0101CA0032.CANPRD01.PROD.OUTLOOK.COM (2603:10b6:b00:15::45) To SN6PR12MB4623.namprd12.prod.outlook.com (2603:10b6:805:e9::17) MIME-Version: 1.0 X-MS-Exchange-MessageSentRepresentingType: 1 Received: from agrodzovsky-All-Series.hitronhub.home (2607:fea8:3edf:49b0:7576:4f76:97d8:1487) by YTOPR0101CA0032.CANPRD01.PROD.OUTLOOK.COM (2603:10b6:b00:15::45) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4108.30 via Frontend Transport; Wed, 12 May 2021 14:27:43 +0000 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: dcabf876-1041-47ff-84cf-08d915521b65 X-MS-TrafficTypeDiagnostic: SN6PR12MB2655: X-MS-Exchange-Transport-Forked: True X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:632; X-MS-Exchange-SenderADCheck: 1 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: GHIgZw0bCU+e5LPxxYCPoFpOLfblmadpSunkm4SL/uaCncgpa8xBffu30WkV+/SZh4Q3bYu3Rk/8T6x0sVBnBG3F5bUmP8ii5Zq+yXg8bCQILaI3CjTeldmaDnlvm8EgO3ULq0JBwIPV6GctebubQ1zOnHeuammf0PmjxOIsktMss55fLKo+RO22PUaDl/yOG1dFNIwVVUfSWAGeSz9CWx+wW2k4HdCP5i1tL175g4im50j0DfozxoPMiJ7HDGHffJbu0P4y63zQ3i30zf+7bp3fyj1kX4VTtY04L5IMsNReHBJdunWmukosRAloVdb6JHBQXh8Em0P8zv2nCNF2LW7AnsEzhJyOeEQT2v4GNgujDIUVTKpG2bTyj/wdaqDadFInmmyjsIJe073gFWa3xxC7sCJQRyW27/gZPeYxoxlckE0Cmi+s9/pfhsAB3Uel+M1NopGwCnpKKESQu0y5AOiklW1dpX7+N5SpfUNHBNdx6w5oPb+NSatLAch9/qsQDlYRlUYqxwjhj91UINV7Y905JzjGDGnuj0BKNupYYInzV56UUi5AjP/1J6Fkx/d12RRfxeelDSmDX+2opqSXgqPTqqcWMDNnFLwbg5+ymY4= X-Forefront-Antispam-Report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:SN6PR12MB4623.namprd12.prod.outlook.com; PTR:; CAT:NONE; SFS:(4636009)(396003)(376002)(346002)(366004)(39860400002)(136003)(86362001)(316002)(186003)(16526019)(66476007)(1076003)(6486002)(36756003)(52116002)(2906002)(5660300002)(8676002)(6636002)(478600001)(6512007)(38100700002)(66556008)(66946007)(4326008)(8936002)(6506007)(44832011)(2616005)(83380400001); DIR:OUT; SFP:1101; X-MS-Exchange-AntiSpam-MessageData: w0r2yDCUioNpiEusHbk3eUAgHiF2sJvUDzmJ6eQR4hhz55Xtqopqh+TrwjL6M8D1m9SP33Ka+FuT6zimg4ZfENJAa4hVgqIVupU7WmhMvyRBNziADkCIHfweda5C+2rxEJeALJ/MxvnJyutdwI8UPwfKQkEr9AwmrjftFrVghbZpvcwbJutdXD1i2xQDB2wiBhppSP52+A//vUpbuvj2lvEUbq6SvtOSHwQxJU3iLkiprJqFLTd03zBhNZJtAJ02+N0U8ONMu7uhCzhkqZ7IsIZYzA9MAYctCULapoVBACqDtx599vAaEgBZQ4roWTvDElAl1cJDUA7qP2cy9RXnYUSUWEkzTJ+OXc+iejH0tCVWc1pfX1WaooqvRCsShXn1kUMLyR+MxO7a40mTKG7/Jyzwzl1SfKM3sHvUtCx7YJjKFZ4dIF5Q0XJosDWj3ItjPYKmfEjKSR4gG5Tw4O/2Pa3+Jac0amT8H3J50M3KNUUXjJLPNL/UC8pg/kAAftM3ziyEi4RYgNOMAT+9cJrYj1UCXzaDoCgZ9vf3sLtnW9+daq0L4JMF9sgiQr//MH1fjH2BxhQ7lAVR+FDBhVezO+6YCWMHva9H2G5PHhod1wxvCNeBObzzVVL9iqYtGXyQlyHontT5LPxSXq0pkQKhLzrCuogZKLEuG66u9geMHRhVleVHp4PVtRcA47zgHPV51mFqfSJ/z1SzzmENFMT2NvY6hKNqT6sX1fETwtGf61zxTZFT1UqotbP/TB0+48OAKp65cXbL2rBgnppfwu2LL0WiOc6/uZ8rVEwizona8N8/Eg5GJDOLduDpaFgj3nGzx1jGM+CE8pY51wgcNbKyN/XhdXAsr78N8rWqi30lz27i6MrX4WsAS4QpNter/lrQBCKc60rs6TYx2yyrRSIkCgxbfOlPVC9b3meTGedemknM01+cXRdF1LUU5Gpi5U5raeqlpDP7Y9qmE4X2Ws8T/pu4zH9ZAA2ssf//NnJ2slP0QgcbjL11J0ecSKxWiot7erZpK/aDJUlWJaYR0qTLGd5zDLyY9J11vriKfxjFJdtqVL+ix2vFEtGqVPpIZA4oD4yAMRNNi3pVowL7T7wM0Te1irMIKoGG34L6Eu6MfzeIBkjGE4oMJ5DCljTtqHMvhfVPySp08VY98F0v7HlvNqyiTWTFdtGQ8wQkbxOlheRFwnM1E8wPPyQB8ku13uBnCNhKpDriXrNXwQgA7Ynq3fYkr61MGd2YUA3Z0kIzULbfYNlZolyda7CdoxzwQTxKuQ3XbVhMpXgmXwTXEJ+vAw+G29PjmYbcHDMWGZ1ygO0+monuiIanCgFl+bl6o154dRUE+cqeIkMwQ3E1jJCTkcyUs8WO7trsLr6quyF1IY1wP82NKETe9YHC7kTbMKvC X-OriginatorOrg: amd.com X-MS-Exchange-CrossTenant-Network-Message-Id: dcabf876-1041-47ff-84cf-08d915521b65 X-MS-Exchange-CrossTenant-AuthSource: SN6PR12MB4623.namprd12.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 12 May 2021 14:27:44.2484 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: +bfjgrLBjIFeZqvkyt/QvyRz6ce+owinfKVhc2jOmFqO02Y/4suI7oA/Ab31XoOjy+mQjnrLyqnZl66XqjcY5Q== X-MS-Exchange-Transport-CrossTenantHeadersStamped: SN6PR12MB2655 X-BeenThere: dri-devel@lists.freedesktop.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Direct Rendering Infrastructure - Development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: gregkh@linuxfoundation.org, Felix.Kuehling@amd.com, helgaas@kernel.org, Alexander.Deucher@amd.com Errors-To: dri-devel-bounces@lists.freedesktop.org Sender: "dri-devel" Access to those must be prevented post pci_remove v6: Drop BOs list, unampping VRAM BAR is enough. Signed-off-by: Andrey Grodzovsky --- drivers/gpu/drm/amd/amdgpu/amdgpu_device.c | 24 +++++++++++++++++++--- drivers/gpu/drm/amd/amdgpu/amdgpu_object.c | 1 + drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.c | 4 ---- 3 files changed, 22 insertions(+), 7 deletions(-) diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c index f7cca25c0fa0..73cbc3c7453f 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c @@ -3666,6 +3666,25 @@ int amdgpu_device_init(struct amdgpu_device *adev, return r; } +static void amdgpu_device_unmap_mmio(struct amdgpu_device *adev) +{ + /* Clear all CPU mappings pointing to this device */ + unmap_mapping_range(adev->ddev.anon_inode->i_mapping, 0, 0, 1); + + /* Unmap all mapped bars - Doorbell, registers and VRAM */ + amdgpu_device_doorbell_fini(adev); + + iounmap(adev->rmmio); + adev->rmmio = NULL; + if (adev->mman.aper_base_kaddr) + iounmap(adev->mman.aper_base_kaddr); + adev->mman.aper_base_kaddr = NULL; + + /* Memory manager related */ + arch_phys_wc_del(adev->gmc.vram_mtrr); + arch_io_free_memtype_wc(adev->gmc.aper_base, adev->gmc.aper_size); +} + /** * amdgpu_device_fini - tear down the driver * @@ -3712,6 +3731,8 @@ void amdgpu_device_fini_hw(struct amdgpu_device *adev) amdgpu_device_ip_fini_early(adev); amdgpu_gart_dummy_page_fini(adev); + + amdgpu_device_unmap_mmio(adev); } void amdgpu_device_fini_sw(struct amdgpu_device *adev) @@ -3739,9 +3760,6 @@ void amdgpu_device_fini_sw(struct amdgpu_device *adev) } if ((adev->pdev->class >> 8) == PCI_CLASS_DISPLAY_VGA) vga_client_register(adev->pdev, NULL, NULL, NULL); - iounmap(adev->rmmio); - adev->rmmio = NULL; - amdgpu_device_doorbell_fini(adev); if (IS_ENABLED(CONFIG_PERF_EVENTS)) amdgpu_pmu_fini(adev); diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_object.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_object.c index 0adffcace326..882fb49f3c41 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_object.c +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_object.c @@ -533,6 +533,7 @@ static int amdgpu_bo_do_create(struct amdgpu_device *adev, return -ENOMEM; drm_gem_private_object_init(adev_to_drm(adev), &bo->tbo.base, size); INIT_LIST_HEAD(&bo->shadow_list); + bo->vm_bo = NULL; bo->preferred_domains = bp->preferred_domain ? bp->preferred_domain : bp->domain; diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.c index 0d54e70278ca..58ad2fecc9e3 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.c +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.c @@ -1841,10 +1841,6 @@ void amdgpu_ttm_fini(struct amdgpu_device *adev) amdgpu_bo_free_kernel(&adev->mman.discovery_memory, NULL, NULL); amdgpu_ttm_fw_reserve_vram_fini(adev); - if (adev->mman.aper_base_kaddr) - iounmap(adev->mman.aper_base_kaddr); - adev->mman.aper_base_kaddr = NULL; - amdgpu_vram_mgr_fini(adev); amdgpu_gtt_mgr_fini(adev); ttm_range_man_fini(&adev->mman.bdev, AMDGPU_PL_GDS);