From patchwork Tue Dec 11 10:34:40 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Chunming Zhou X-Patchwork-Id: 10723427 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id CBFF11751 for ; Tue, 11 Dec 2018 10:35:12 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id BA8122A0AA for ; Tue, 11 Dec 2018 10:35:12 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id AE6282A0DC; Tue, 11 Dec 2018 10:35:12 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-5.2 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_MED autolearn=unavailable version=3.3.1 Received: from gabe.freedesktop.org (gabe.freedesktop.org [131.252.210.177]) (using TLSv1.2 with cipher DHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.wl.linuxfoundation.org (Postfix) with ESMTPS id 15B162A0AA for ; Tue, 11 Dec 2018 10:35:12 +0000 (UTC) Received: from gabe.freedesktop.org (localhost [127.0.0.1]) by gabe.freedesktop.org (Postfix) with ESMTP id E251E89CAD; Tue, 11 Dec 2018 10:35:07 +0000 (UTC) X-Original-To: dri-devel@lists.freedesktop.org Delivered-To: dri-devel@lists.freedesktop.org Received: from NAM04-CO1-obe.outbound.protection.outlook.com (mail-eopbgr690052.outbound.protection.outlook.com [40.107.69.52]) by gabe.freedesktop.org (Postfix) with ESMTPS id B582589CAD; Tue, 11 Dec 2018 10:35:06 +0000 (UTC) Received: from MWHPR1201CA0011.namprd12.prod.outlook.com (2603:10b6:301:4a::21) by SN6PR12MB2640.namprd12.prod.outlook.com (2603:10b6:805:6f::17) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.1404.21; Tue, 11 Dec 2018 10:35:04 +0000 Received: from BY2NAM03FT037.eop-NAM03.prod.protection.outlook.com (2a01:111:f400:7e4a::208) by MWHPR1201CA0011.outlook.office365.com (2603:10b6:301:4a::21) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384) id 15.20.1425.18 via Frontend Transport; Tue, 11 Dec 2018 10:35:04 +0000 Received-SPF: None (protection.outlook.com: amd.com does not designate permitted sender hosts) Received: from SATLEXCHOV01.amd.com (165.204.84.17) by BY2NAM03FT037.mail.protection.outlook.com (10.152.84.188) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384) id 15.20.1425.16 via Frontend Transport; Tue, 11 Dec 2018 10:35:03 +0000 Received: from zhoucm1.amd.com (10.34.1.3) by SATLEXCHOV01.amd.com (10.181.40.71) with Microsoft SMTP Server id 14.3.389.1; Tue, 11 Dec 2018 04:35:00 -0600 From: Chunming Zhou To: , , , Subject: [PATCH 01/10] dma-buf: add new dma_fence_chain container v4 Date: Tue, 11 Dec 2018 18:34:40 +0800 Message-ID: <20181211103449.25899-1-david1.zhou@amd.com> X-Mailer: git-send-email 2.17.1 MIME-Version: 1.0 X-EOPAttributedMessage: 0 X-MS-Office365-Filtering-HT: Tenant X-Forefront-Antispam-Report: CIP:165.204.84.17; IPV:NLI; CTRY:US; EFV:NLI; SFV:NSPM; SFS:(10009020)(346002)(136003)(39860400002)(396003)(376002)(2980300002)(428003)(199004)(189003)(66574011)(5660300001)(336012)(26005)(186003)(72206003)(1076002)(77096007)(426003)(50466002)(478600001)(53416004)(36756003)(105586002)(14444005)(68736007)(53936002)(106466001)(54906003)(316002)(6666004)(97736004)(39060400002)(5820100001)(4744004)(305945005)(356004)(110136005)(2616005)(86362001)(4326008)(2201001)(23676004)(50226002)(8676002)(81156014)(81166006)(2906002)(7696005)(2870700001)(126002)(486006)(476003)(104016004)(47776003)(8936002)(2004002); DIR:OUT; SFP:1101; SCL:1; SRVR:SN6PR12MB2640; H:SATLEXCHOV01.amd.com; FPR:; SPF:None; LANG:en; PTR:InfoDomainNonexistent; A:1; MX:1; X-Microsoft-Exchange-Diagnostics: 1; BY2NAM03FT037; 1:XF4nwbDbvzN1VVGdQPXIxf7PnhdzWZ5PyWTpb2eVZMZh+kQ2vfXXfY/bXadS/JB/bid1nOVfnkz4POF5l8IjOEBemFgF24a18VsbhXU025I14PZhyZOlz6H8Zsxmq7/S X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: aefa2424-88d9-4c8c-f327-08d65f544fcb X-Microsoft-Antispam: BCL:0; PCL:0; RULEID:(2390098)(7020095)(4652040)(8989299)(4534185)(4627221)(201703031133081)(201702281549075)(8990200)(5600074)(711020)(2017052603328)(7153060); SRVR:SN6PR12MB2640; X-Microsoft-Exchange-Diagnostics: 1; SN6PR12MB2640; 3:ZsIEHxpsXPMzYWnbe7lkLT0BcTEt+sj3dr1/Ljpxu+ngCvNIAkBTlC4hmf4rwyY00TunxOB49x99x+S4giBIdSIVpVAv9fB7s1U5b4JurmZYrYCvJwdP3GWPHK43wLY15o6/ugeXQL0rNYYqYklwou7m0mTiGtigkMQF0EAt0BH+bo2oH45zTz+AKdUaQOw63KAbSTi53EG6pQdBi9l+Ldt+3GN8k72xriGKjlJmCptF1h9voWQqImKgpMjfiY4s534xwacpGwse8gmPIb8tVitR3o5gh4T8c1y6KrYsJJviBplJc1gsI4W9hdvcc/wyWfFodRoOcdoOUsSkQZy7j/2K+sDYMjG52M62C4CI97k=; 25:JsxwpPDc+d4XoF6w12bMS7GyohJhYtgnJTkTxxT25MZhTRLvNrvcIZ6l/m2pH0dBACrc71AASix8XmZ6VALp4Nm51PGRTkjN1wNH3lNXkrklMbD9O1M/U+r61B32AEcU6AQWBjaOjdlyR9cdaqb+ZKv2AqTCWBVuCkMOR96a8GjnYhOl6QHAnlQcpylfsYwlu08NAGZmR/8FKM80NNpdyOyRuFspXgKOko3enU3Y/ky8WH0t0F8hxSYioQ1HV809J1Qp91IX4aAUbFX9iJVzaMFXQsML+fMVbKSmRFcGRhozhm+hRV8LHA7+GMo/tdw4SKpuIlVyocDkP0Wn3sZS6w== X-MS-TrafficTypeDiagnostic: SN6PR12MB2640: X-Microsoft-Exchange-Diagnostics: 1; SN6PR12MB2640; 31:NXcduoElv2daCJRENACdx05UpaECMpbDQKMQTn8R1MrfMJnYbV1d2Y8OOXSEG7kb4Y50iH73vo6NTUTnHni2hWGFRKGV1uFHhn2l28B//dY2AYfcd8+mD+GvQ7ujBgSCBoS/WYiYRC3nstwt4Y6oRVs+JiE1PHCOYCs+AuKYCrhsVZ+F/qu4F1/qXCyrp8xS6BVQL3Gm/v2k50v9g1bnUbnn7OIKTvwzBC4VIM5O89M=; 20:n79aHqrJnAMksmB5zyZ+lpjmWu/yK6Ss5DvuIioO5TPdg2/nbfndGjPPznzWCZsYEtVOFfJRJ+tokEFhjPYBU0TSYMdNgNgyP0/aH8nAZ+NudadZ28czMM8VpKOwOKrWXvZSc9cr17Ku4zY/tjFSm7Rj+GXVe7aK8W8ae/qjj96t3SrazWnZSlIHbPR4JqkKrOtBoagLi61WgHUJWVYN1FycPerqhIK9C229VPGnR+e2iKc1VbtVFLQXoYlGrz4ubzMbuYRpBrjhmDrjoAwt93oN69oWshCiyP3Nhl7KA3Tmp6LWLn6v5MQLFuFdJFPo53yqC1qEFJda6FhoGBHPP1R+cWR615keSBF2QtD2xWXDKgPmGunFkhtRri+5nDYMBuLd2Guit9VgMn0vLua3vRaEC9UpPGn00sANZmleBH7CPJ+CfHtPUEqAfFkMvSH1ZnNu0qsPZ9zHIH4XLF4Ejc4JWCOK7ZSDrgDj1jQGNdRjjEZSYX8ee98yzrpF3D3a X-Microsoft-Antispam-PRVS: X-MS-Exchange-SenderADCheck: 1 X-Exchange-Antispam-Report-CFA-Test: BCL:0; PCL:0; RULEID:(8211001083)(6040522)(2401047)(5005006)(8121501046)(10201501046)(3231455)(999002)(944501520)(52105112)(3002001)(93006095)(93003095)(6055026)(148016)(149066)(150057)(6041310)(20161123562045)(201703131423095)(201702281528075)(20161123555045)(201703061421075)(201703061406153)(20161123558120)(20161123564045)(20161123560045)(201708071742011)(7699051)(76991095); SRVR:SN6PR12MB2640; BCL:0; PCL:0; RULEID:; SRVR:SN6PR12MB2640; X-Microsoft-Exchange-Diagnostics: 1; SN6PR12MB2640; 4:yzEpjJgd2NoWgW712rRo7X570h+6LD2QHIg5ddfevZHIOQneB8IdgkXFakmG2lG8xJuYCsQtqH6d82gOmoxBwFg/VtjtWvLuqHml04F/CkyTreMDK5vkiWYyG5HnScrNiwK34tKLCB448Bt6tHNsm/SoADz3bJq1JnTlv1tMEoErGa9m3YMfT874G0ayOkeB34D29emWIZMn+/uSu5I1vY40cFhRehx1v1VkdD5fQkgDY9L4KGCr4Pon8+Ka6mRREMfR/pDE+Yvyk3P/WvCSVA== X-Forefront-PRVS: 08831F51DC X-Microsoft-Exchange-Diagnostics: =?utf-8?q?1=3BSN6PR12MB2640=3B23=3A+S15+Fe?= =?utf-8?q?S40REHTU522yS3K0snu0qkgEf3xU3xJLyTnWdlauUBq6MhyumSv/ShVAtLY00pZmr?= =?utf-8?q?hZs7HvjOrJ6g2VacY4PKKNDH+vl6vPyN5UVF5e0tH/r0uO/NPR1sBZwjzeZxkZr2H?= =?utf-8?q?aYWrtCsZ0i0G4W6KBNZPV5CREXagIBkvQXlCB7GurNS6gw2Y6BIx6yZfzI2t4UzPv?= =?utf-8?q?oR+J4gWkSTp/gi1/XBOS5YhwTAKTew55y0vtgPI/ZFGjJFdMZxM7x5hYtI1PTH4DZ?= =?utf-8?q?GMCiHc2d7TK36aLDq2seBioiesurGWjOnSn3MMF2s94RcR4npWEP1lD/H2r6biTn+?= =?utf-8?q?NjYKKTf7AR7KCuZ8BaGrIoQzzreo+rkvk5FmuADzGaF1MbQ1zAiD+5iJI4OtGbD8Y?= =?utf-8?q?g9AM1rpwSMJNri4Yr4lKwGwzhB6oNtGdaXLdrce3QoeCuDo5YSxiaS+cKwn8EMkjz?= =?utf-8?q?yRRwSeeFwae9c0fuUhxN/JuZF9Mk1Vfx1K5MT76f5oCdI2cJPvbYFMgw+Lnnr+HdW?= =?utf-8?q?jGzGimUPaen2Jp2aynulzmWoIza691RDPBDgmOCFt8Z67nFpv1J3XsGbm07Q88QBw?= =?utf-8?q?RaS0Y/kYzuiiV8Ilb0A1ogvV6Z8synRg2O5+wEG+eLPG+bEj8cJae0AYAn1CVnoim?= =?utf-8?q?5e+qxZYzdXIuVePvlNobRRssRevnIbQM6pHxTeKnOSySCkZpVVJMfhvy3yohAAkwj?= =?utf-8?q?QPgEB83u0eEAK2lJ4pC0nLf/PmFDNzxFWBCqKoaEtcfuFijamvT/cuIm6pXRpOfPV?= =?utf-8?q?hp85+oui/fgZEi0UwJAsDCvor5RCDmvJ7uloBRji0oD3Pb2/4WTKndxPx6t9xHTeV?= =?utf-8?q?JrxUELXFp8Uw/J04g1/ne6XzxPsOlz1KoXorwxcU0NNgfy4gEAALhvGS8AeoKtp8p?= =?utf-8?q?wG9ZK/S5BNnhsrnkRH2C2wmh9qHC4A41ouFUDgFvKQ4wnSHdRdirkW6bdpixbPcgO?= =?utf-8?q?bgD3jIiu3+iFH+M3W0jHDnW8cApuS8qcOcFUSt1jHVLbfq08cgsbvUfKi3jqPbafa?= =?utf-8?q?C56LM8KSS4xkO4UPMh9Tv8rawLSLcNv8Ch7RgGFQlpyfyoVLL74CytD5SpNG7WeH0?= =?utf-8?q?vfaiT4GkekTgE/C+mPoMDG2oFRh0NAAjKSZLBGF9NVSZeJQeJKl7uvsqHwwoG0Hxr?= =?utf-8?q?/8PhfookQ7xo4BNqS4s6F86mHx96mdzD1yn1wFnczcDewb52ijb8pTqXEv+eFQycH?= =?utf-8?q?u6rEUQYGpVRRnFApc=3D?= X-Microsoft-Antispam-Message-Info: 8DD7S+3RdESomvMh3OVjhtDn0ta27gGd5uj4rTa9FYps6WmZE0BG1h54/dpkuakunD1O+tBGIgIB/Pker8f5Cdi9d3osaLwmk88hCO+wSthZ0IILmOBZXR2t7FEoXdSGy8ftILnTkLvYDu1Wu3vkLFoI9OAO2TpD7s4I0iOg9y58LJZndmjNRxlvfz2z84Hlj8y0PHLFOgsHjVgV8psHl+MCSdwI9dvoVSnAjhCrqfIE6Lfsow8jsLZbwuK+cOo5HAn4xJfuK3czFLvww3O+lzqOmg6zJNFpNkI334nd4xnGyj3+ILqwFdcMjTrmIUVV X-Microsoft-Exchange-Diagnostics: 1; SN6PR12MB2640; 6:C0hKJT/piqHULRwp/2OLZb9tCVE1CMThBrHY/VG2FnYFKOcUdoN60ZU7tH0pHOhb6CX+d+MyzYNTHG0Z53cnmRdzJVEMIzn/aDJcpNQOG5CxaUjGeYojeLze4Wb4pfGccNjsS+2pH7zfXkcZYUQdtbXGmvnaUUmdQ93LbfINYJPHlyw8aDcVBHlpMt98d8owP+S+9FXoSVjsPsE+v4xiKzRy2lLm7vaX3YVjPrRkXUhKOiMdp9QLS6B0uPD5eCaKmiqSAa8JN7uUrYIcz5Q2KXd84bPiu0iHoxVi06edG8K7TUL2bYEdIxlsEbLKBENDQExtIjCAG9/pbvStr2QIVj7b8W0i5x+hR50flSJvSMq9ji+SmIXJmrf9JJang5MU30GgA1H3V0V8L2DGutcsdcOZtNbdFgJvUGZJQBBBe6bKw4LcIxSngq8zHklo+qjk+j1AxIzSHi3RoQNjXdBm0g==; 5:wUfR9C0zV/8rgJ+YA7axhrYrOJJ48En0MSMKJ3PF0sn7EQbhUbKnYBac8JJOv0TMvgdoBW2kUwAfpyq+EdXn7s6rr8Z5gMrq7cw7+0aoVwPSEzJvOy48Ycs5/Qf1ZOG4SeRYLZWKJ3i9qU/npjCikIoapxRNVIKbnBavokwgDH4=; 7:Sb7e6LCO5kUcGdr3lq2x0lcEchvNJDMyANkhKk025s6brMSqIwBGuoDfrClO18pizGV96JJO+Y5H8UlVCaY3c29+Ekjy0bPG2dR+PnZDt0lFfv5VjF9NzSmeYfTkD8S3+2JoPEbE0JwOxJaBJO7r7w== SpamDiagnosticOutput: 1:99 SpamDiagnosticMetadata: NSPM X-Microsoft-Exchange-Diagnostics: 1; SN6PR12MB2640; 20:LfoWanmwSqKlD595r6IKVHUikG4ydI2ELAjKALXdRhK6vKHtKjTfvMiskvxmq+pFNhyJdrhmj76+HDoqhnLBfV4/Rp2GxgxF2tQx8c/oA4GCMseWa/UHaaeVqJxchyZ/xJSpbby/h7QS4c+cGSGg5XO6rDKByM/xmXq/1+i65KgC/rxH1VNoioyj41DPE8jXAJvIO5L+98uwZjCxzmRVW9AC57d2DhHF8nEfLyOsgpYdLsD9PRTdHN1zmcgK1W8G X-OriginatorOrg: amd.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 11 Dec 2018 10:35:03.7974 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: aefa2424-88d9-4c8c-f327-08d65f544fcb X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d; Ip=[165.204.84.17]; Helo=[SATLEXCHOV01.amd.com] X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: SN6PR12MB2640 X-BeenThere: dri-devel@lists.freedesktop.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Direct Rendering Infrastructure - Development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: =?utf-8?q?Christian_K=C3=B6nig?= , =?utf-8?q?Christian_K=C3=B6nig?= Errors-To: dri-devel-bounces@lists.freedesktop.org Sender: "dri-devel" X-Virus-Scanned: ClamAV using ClamSMTP From: Christian König Lockless container implementation similar to a dma_fence_array, but with only two elements per node and automatic garbage collection. v2: properly document dma_fence_chain_for_each, add dma_fence_chain_find_seqno, drop prev reference during garbage collection if it's not a chain fence. v3: use head and iterator for dma_fence_chain_for_each v4: fix reference count in dma_fence_chain_enable_signaling Signed-off-by: Christian König --- drivers/dma-buf/Makefile | 3 +- drivers/dma-buf/dma-fence-chain.c | 241 ++++++++++++++++++++++++++++++ include/linux/dma-fence-chain.h | 81 ++++++++++ 3 files changed, 324 insertions(+), 1 deletion(-) create mode 100644 drivers/dma-buf/dma-fence-chain.c create mode 100644 include/linux/dma-fence-chain.h diff --git a/drivers/dma-buf/Makefile b/drivers/dma-buf/Makefile index 0913a6ccab5a..1f006e083eb9 100644 --- a/drivers/dma-buf/Makefile +++ b/drivers/dma-buf/Makefile @@ -1,4 +1,5 @@ -obj-y := dma-buf.o dma-fence.o dma-fence-array.o reservation.o seqno-fence.o +obj-y := dma-buf.o dma-fence.o dma-fence-array.o dma-fence-chain.o \ + reservation.o seqno-fence.o obj-$(CONFIG_SYNC_FILE) += sync_file.o obj-$(CONFIG_SW_SYNC) += sw_sync.o sync_debug.o obj-$(CONFIG_UDMABUF) += udmabuf.o diff --git a/drivers/dma-buf/dma-fence-chain.c b/drivers/dma-buf/dma-fence-chain.c new file mode 100644 index 000000000000..0c5e3c902fa0 --- /dev/null +++ b/drivers/dma-buf/dma-fence-chain.c @@ -0,0 +1,241 @@ +/* + * fence-chain: chain fences together in a timeline + * + * Copyright (C) 2018 Advanced Micro Devices, Inc. + * Authors: + * Christian König + * + * This program is free software; you can redistribute it and/or modify it + * under the terms of the GNU General Public License version 2 as published by + * the Free Software Foundation. + * + * This program is distributed in the hope that it will be useful, but WITHOUT + * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or + * FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for + * more details. + */ + +#include + +static bool dma_fence_chain_enable_signaling(struct dma_fence *fence); + +/** + * dma_fence_chain_get_prev - use RCU to get a reference to the previous fence + * @chain: chain node to get the previous node from + * + * Use dma_fence_get_rcu_safe to get a reference to the previous fence of the + * chain node. + */ +static struct dma_fence *dma_fence_chain_get_prev(struct dma_fence_chain *chain) +{ + struct dma_fence *prev; + + rcu_read_lock(); + prev = dma_fence_get_rcu_safe(&chain->prev); + rcu_read_unlock(); + return prev; +} + +/** + * dma_fence_chain_walk - chain walking function + * @fence: current chain node + * + * Walk the chain to the next node. Returns the next fence or NULL if we are at + * the end of the chain. Garbage collects chain nodes which are already + * signaled. + */ +struct dma_fence *dma_fence_chain_walk(struct dma_fence *fence) +{ + struct dma_fence_chain *chain, *prev_chain; + struct dma_fence *prev, *replacement, *tmp; + + chain = to_dma_fence_chain(fence); + if (!chain) { + dma_fence_put(fence); + return NULL; + } + + while ((prev = dma_fence_chain_get_prev(chain))) { + + prev_chain = to_dma_fence_chain(prev); + if (prev_chain) { + if (!dma_fence_is_signaled(prev_chain->fence)) + break; + + replacement = dma_fence_chain_get_prev(prev_chain); + } else { + if (!dma_fence_is_signaled(prev)) + break; + + replacement = NULL; + } + + tmp = cmpxchg(&chain->prev, prev, replacement); + if (tmp == prev) + dma_fence_put(tmp); + else + dma_fence_put(replacement); + dma_fence_put(prev); + } + + dma_fence_put(fence); + return prev; +} +EXPORT_SYMBOL(dma_fence_chain_walk); + +/** + * dma_fence_chain_find_seqno - find fence chain node by seqno + * @pfence: pointer to the chain node where to start + * @seqno: the sequence number to search for + * + * Advance the fence pointer to the chain node which will signal this sequence + * number. If no sequence number is provided then this is a no-op. + * + * Returns EINVAL if the fence is not a chain node or the sequence number has + * not yet advanced far enough. + */ +int dma_fence_chain_find_seqno(struct dma_fence **pfence, uint64_t seqno) +{ + struct dma_fence_chain *chain; + + if (!seqno) + return 0; + + chain = to_dma_fence_chain(*pfence); + if (!chain || chain->base.seqno < seqno) + return -EINVAL; + + dma_fence_chain_for_each(*pfence, &chain->base) { + if ((*pfence)->context != chain->base.context || + to_dma_fence_chain(*pfence)->prev_seqno < seqno) + break; + } + dma_fence_put(&chain->base); + + return 0; +} +EXPORT_SYMBOL(dma_fence_chain_find_seqno); + +static const char *dma_fence_chain_get_driver_name(struct dma_fence *fence) +{ + return "dma_fence_chain"; +} + +static const char *dma_fence_chain_get_timeline_name(struct dma_fence *fence) +{ + return "unbound"; +} + +static void dma_fence_chain_irq_work(struct irq_work *work) +{ + struct dma_fence_chain *chain; + + chain = container_of(work, typeof(*chain), work); + + /* Try to rearm the callback */ + if (!dma_fence_chain_enable_signaling(&chain->base)) + /* Ok, we are done. No more unsignaled fences left */ + dma_fence_signal(&chain->base); + dma_fence_put(&chain->base); +} + +static void dma_fence_chain_cb(struct dma_fence *f, struct dma_fence_cb *cb) +{ + struct dma_fence_chain *chain; + + chain = container_of(cb, typeof(*chain), cb); + irq_work_queue(&chain->work); + dma_fence_put(f); +} + +static bool dma_fence_chain_enable_signaling(struct dma_fence *fence) +{ + struct dma_fence_chain *head = to_dma_fence_chain(fence); + + dma_fence_get(&head->base); + dma_fence_chain_for_each(fence, &head->base) { + struct dma_fence_chain *chain = to_dma_fence_chain(fence); + struct dma_fence *f = chain ? chain->fence : fence; + + dma_fence_get(f); + if (!dma_fence_add_callback(f, &head->cb, dma_fence_chain_cb)) { + dma_fence_put(fence); + return true; + } + dma_fence_put(f); + } + dma_fence_put(&head->base); + return false; +} + +static bool dma_fence_chain_signaled(struct dma_fence *fence) +{ + dma_fence_chain_for_each(fence, fence) { + struct dma_fence_chain *chain = to_dma_fence_chain(fence); + struct dma_fence *f = chain ? chain->fence : fence; + + if (!dma_fence_is_signaled(f)) { + dma_fence_put(fence); + return false; + } + } + + return true; +} + +static void dma_fence_chain_release(struct dma_fence *fence) +{ + struct dma_fence_chain *chain = to_dma_fence_chain(fence); + + dma_fence_put(chain->prev); + dma_fence_put(chain->fence); + dma_fence_free(fence); +} + +const struct dma_fence_ops dma_fence_chain_ops = { + .get_driver_name = dma_fence_chain_get_driver_name, + .get_timeline_name = dma_fence_chain_get_timeline_name, + .enable_signaling = dma_fence_chain_enable_signaling, + .signaled = dma_fence_chain_signaled, + .release = dma_fence_chain_release, +}; +EXPORT_SYMBOL(dma_fence_chain_ops); + +/** + * dma_fence_chain_init - initialize a fence chain + * @chain: the chain node to initialize + * @prev: the previous fence + * @fence: the current fence + * + * Initialize a new chain node and either start a new chain or add the node to + * the existing chain of the previous fence. + */ +void dma_fence_chain_init(struct dma_fence_chain *chain, + struct dma_fence *prev, + struct dma_fence *fence, + uint64_t seqno) +{ + struct dma_fence_chain *prev_chain = to_dma_fence_chain(prev); + uint64_t context; + + spin_lock_init(&chain->lock); + chain->prev = prev; + chain->fence = fence; + chain->prev_seqno = 0; + init_irq_work(&chain->work, dma_fence_chain_irq_work); + + /* Try to reuse the context of the previous chain node. */ + if (prev_chain && __dma_fence_is_later(seqno, prev->seqno)) { + context = prev->context; + chain->prev_seqno = prev->seqno; + } else { + context = dma_fence_context_alloc(1); + /* Make sure that we always have a valid sequence number. */ + if (prev_chain) + seqno = max(prev->seqno, seqno); + } + + dma_fence_init(&chain->base, &dma_fence_chain_ops, + &chain->lock, context, seqno); +} +EXPORT_SYMBOL(dma_fence_chain_init); diff --git a/include/linux/dma-fence-chain.h b/include/linux/dma-fence-chain.h new file mode 100644 index 000000000000..a5c2e8c6915c --- /dev/null +++ b/include/linux/dma-fence-chain.h @@ -0,0 +1,81 @@ +/* + * fence-chain: chain fences together in a timeline + * + * Copyright (C) 2018 Advanced Micro Devices, Inc. + * Authors: + * Christian König + * + * This program is free software; you can redistribute it and/or modify it + * under the terms of the GNU General Public License version 2 as published by + * the Free Software Foundation. + * + * This program is distributed in the hope that it will be useful, but WITHOUT + * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or + * FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for + * more details. + */ + +#ifndef __LINUX_DMA_FENCE_CHAIN_H +#define __LINUX_DMA_FENCE_CHAIN_H + +#include +#include + +/** + * struct dma_fence_chain - fence to represent an node of a fence chain + * @base: fence base class + * @lock: spinlock for fence handling + * @prev: previous fence of the chain + * @prev_seqno: original previous seqno before garbage collection + * @fence: encapsulated fence + * @cb: callback structure for signaling + * @work: irq work item for signaling + */ +struct dma_fence_chain { + struct dma_fence base; + spinlock_t lock; + struct dma_fence *prev; + u64 prev_seqno; + struct dma_fence *fence; + struct dma_fence_cb cb; + struct irq_work work; +}; + +extern const struct dma_fence_ops dma_fence_chain_ops; + +/** + * to_dma_fence_chain - cast a fence to a dma_fence_chain + * @fence: fence to cast to a dma_fence_array + * + * Returns NULL if the fence is not a dma_fence_chain, + * or the dma_fence_chain otherwise. + */ +static inline struct dma_fence_chain * +to_dma_fence_chain(struct dma_fence *fence) +{ + if (!fence || fence->ops != &dma_fence_chain_ops) + return NULL; + + return container_of(fence, struct dma_fence_chain, base); +} + +/** + * dma_fence_chain_for_each - iterate over all fences in chain + * @iter: current fence + * @head: starting point + * + * Iterate over all fences in the chain. We keep a reference to the current + * fence while inside the loop which must be dropped when breaking out. + */ +#define dma_fence_chain_for_each(iter, head) \ + for (iter = dma_fence_get(head); iter; \ + iter = dma_fence_chain_walk(head)) + +struct dma_fence *dma_fence_chain_walk(struct dma_fence *fence); +int dma_fence_chain_find_seqno(struct dma_fence **pfence, uint64_t seqno); +void dma_fence_chain_init(struct dma_fence_chain *chain, + struct dma_fence *prev, + struct dma_fence *fence, + uint64_t seqno); + +#endif /* __LINUX_DMA_FENCE_CHAIN_H */