From patchwork Tue Mar 12 03:10:41 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Chunming Zhou X-Patchwork-Id: 10848547 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 062F71850 for ; Tue, 12 Mar 2019 03:11:13 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id D7A7D294BD for ; Tue, 12 Mar 2019 03:11:12 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id CBA60294BF; Tue, 12 Mar 2019 03:11:12 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-5.2 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_MED autolearn=ham version=3.3.1 Received: from gabe.freedesktop.org (gabe.freedesktop.org [131.252.210.177]) (using TLSv1.2 with cipher DHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.wl.linuxfoundation.org (Postfix) with ESMTPS id C88A3294BD for ; Tue, 12 Mar 2019 03:11:11 +0000 (UTC) Received: from gabe.freedesktop.org (localhost [127.0.0.1]) by gabe.freedesktop.org (Postfix) with ESMTP id 2E8E089856; Tue, 12 Mar 2019 03:11:09 +0000 (UTC) X-Original-To: dri-devel@lists.freedesktop.org Delivered-To: dri-devel@lists.freedesktop.org Received: from NAM05-DM3-obe.outbound.protection.outlook.com (mail-eopbgr730080.outbound.protection.outlook.com [40.107.73.80]) by gabe.freedesktop.org (Postfix) with ESMTPS id 45CBC89854; Tue, 12 Mar 2019 03:11:07 +0000 (UTC) Received: from DM3PR12CA0126.namprd12.prod.outlook.com (2603:10b6:0:51::22) by BN8PR12MB2963.namprd12.prod.outlook.com (2603:10b6:408:61::21) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.1686.20; Tue, 12 Mar 2019 03:11:04 +0000 Received: from DM3NAM03FT041.eop-NAM03.prod.protection.outlook.com (2a01:111:f400:7e49::203) by DM3PR12CA0126.outlook.office365.com (2603:10b6:0:51::22) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.1686.18 via Frontend Transport; Tue, 12 Mar 2019 03:11:04 +0000 Received-SPF: None (protection.outlook.com: amd.com does not designate permitted sender hosts) Received: from SATLEXCHOV01.amd.com (165.204.84.17) by DM3NAM03FT041.mail.protection.outlook.com (10.152.83.207) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384) id 15.20.1686.19 via Frontend Transport; Tue, 12 Mar 2019 03:11:03 +0000 Received: from zhoucm1.amd.com (10.34.1.3) by SATLEXCHOV01.amd.com (10.181.40.71) with Microsoft SMTP Server id 14.3.389.1; Mon, 11 Mar 2019 22:11:02 -0500 From: Chunming Zhou To: , , , , Subject: [PATCH 1/9] dma-buf: add new dma_fence_chain container v5 Date: Tue, 12 Mar 2019 11:10:41 +0800 Message-ID: <20190312031049.23647-1-david1.zhou@amd.com> X-Mailer: git-send-email 2.17.1 MIME-Version: 1.0 X-EOPAttributedMessage: 0 X-MS-Office365-Filtering-HT: Tenant X-Forefront-Antispam-Report: CIP:165.204.84.17; IPV:NLI; CTRY:US; EFV:NLI; SFV:NSPM; SFS:(10009020)(376002)(396003)(136003)(39860400002)(346002)(2980300002)(428003)(189003)(199004)(72206003)(5820100001)(86362001)(8936002)(2616005)(110136005)(2906002)(4326008)(54906003)(77096007)(2870700001)(36756003)(186003)(7696005)(486006)(26005)(2201001)(478600001)(336012)(5660300002)(68736007)(97736004)(66574012)(6666004)(356004)(426003)(81156014)(81166006)(8676002)(1076003)(53416004)(47776003)(30864003)(104016004)(6636002)(105586002)(316002)(106466001)(476003)(50466002)(126002)(50226002)(23676004)(14444005)(305945005)(53936002)(2004002); DIR:OUT; SFP:1101; SCL:1; SRVR:BN8PR12MB2963; H:SATLEXCHOV01.amd.com; FPR:; SPF:None; LANG:en; PTR:InfoDomainNonexistent; MX:1; A:1; X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: de70122e-938f-4df0-d89a-08d6a6985cd2 X-Microsoft-Antispam: BCL:0; PCL:0; RULEID:(2390118)(7020095)(4652040)(8989299)(4534185)(4627221)(201703031133081)(201702281549075)(8990200)(5600127)(711020)(4605104)(2017052603328)(7153060); SRVR:BN8PR12MB2963; X-MS-TrafficTypeDiagnostic: BN8PR12MB2963: X-Microsoft-Exchange-Diagnostics: 1; BN8PR12MB2963; 20:Oc02L4GjkQFpqKKJ0JMRxge0fwnKvM8XOa8Y8lVD+t4x5d9c34qimdPAQpjVtGIgD/uZ+xL16QgHMYaajKK8weFpEi4jwJY2Lytz91IqBGSW+tgQrlzb+Z7e+6A3KTjx2phLhog228gqrfSLtYjL0yyDgMayOdByH3enOWIJr7G4W7IoKvmHuNLrtyen8wQBfDrhvs495vxG44S62Ft7dT009i1GkHkEFhs55LY7m2zHpoOCjIy0uzTy2KHsFXo+TchPkzKufdCujj4ZQnK0o4/SU6lNWv99A2sRindndCkAaEzq//Br/tUyNEun5cazPKYBok/5aD6RwT6mSKychfi/ON5HP4epgo5ArhlAsf1zR9/FZ3Ri5YS+p2J3piZMGNuggvcmkBY7Zw/Ih3Q7HCZWCHIPwLIYWb9Q9/OS73KfJoTCajqnf9/Iw5sVo3ClKSqr3QwwY/1bdhqUS5rfhiXyVrNgNDccDOe15m/NN1rE2EpOxciw3npXi3Lm0Uv8 X-Microsoft-Antispam-PRVS: X-Forefront-PRVS: 09749A275C X-Microsoft-Exchange-Diagnostics: =?utf-8?q?1=3BBN8PR12MB2963=3B23=3ACH3dmMv?= =?utf-8?q?VZPcKM4FAcjFLSWnyOYyh7kpW43+4nEw13NP/vhxOxjj/HFmaJ3pVQeTtSesDhku4?= =?utf-8?q?FnCNj0xQZLIsAQAT4DhjCZCB+byZoSc9v5T05RoKMnyCa4uz9ONA5Cv/RHbWgMSGt?= =?utf-8?q?7OpbTsHg8sWhbqaf/1RY7weqwiaKTlat1A8uq0dooeGnSGUBn9KSTiDw9s55ErJtn?= =?utf-8?q?u3Yj3tlntYchpabB844tHqmC1jq42g2mo/X04J1yV9KUN/AIstIrN6rNcaIOVOAGL?= =?utf-8?q?4Hp2AX1FzKOZaan1OFestR/Lr/Tm3GyKuIRZ5mKnG/KerYHXgOQK6ATkugw9pFT44?= =?utf-8?q?iMDSlGWoVezXId/DZQ3xaOJW5PYh/jqQ8E8XLdoG4eg15m400khw58XHh8brgCsVe?= =?utf-8?q?O+btFDMd++VNi4uL5drpGl9R+nVMgC4RVHSMDgUdvNXL5isl26vO0N4NzAj32c63G?= =?utf-8?q?KrjPqQWvYb9yavNk9OZ0z6uerU69CJyrvaw6gAodC96JnhQGCTPpIocbAk1bAr4gL?= =?utf-8?q?jI1ZkjmWWz0q/vRiV+Tpq/7x8Md5g6g1SznlLCjGjY7Oh1hKpMdrnOtrSInOwoQre?= =?utf-8?q?nv8SFtqPgeV19blMMCwpEE53S1lpmCk+omzdRQsVzjKqwpUUZGky8sBmlmt3USZyT?= =?utf-8?q?rkrHRtTjkD2kYYt+B7j2jRoAD0STvcKI2iPidLAKfcQUKg0icQwmeiV8Q1Ot+3Qei?= =?utf-8?q?CgUZvNafSH90Kv8L6y4sehqij22+6ZzL/Y3FDSb7MKg+jsM1PtygFsJ5LNcFqreAt?= =?utf-8?q?TO5rCzX4QOVd3x3fsZZjfchPbJAKLps55NxhM9ZXnyyagvnRdwrjLyEEAWGDzfESv?= =?utf-8?q?2dOmKoo2a3UxvibGeaMnUfLC/yaTLDC3vrx8EfC+VdZaRhIb8S8upMNSqX4cnNU/f?= =?utf-8?q?Yq1Y1M9k3xXmlOkmWsRz2rmJhTAr12ojIkeBX6wJeeYsu+Nyz/LErIWLKFRewtadL?= =?utf-8?q?yjae+rELD9AovxhgFd7AT5HmYCmHDmCqrypv8JzN5WeFAX09fK3yAIkkpLreZUrOq?= =?utf-8?q?mtSC22B0gmKxtfODJxbk1Z0sIefmvecdfwlL9Tz1hhy/yyl/VLd5ZldGniT7963e2?= =?utf-8?q?p5pVI1xe8jtB7j1YwC8z3EL34FZwrsFE30P3AydoR6p/qxUxJOUwi8PtJSqht1vFY?= =?utf-8?q?qKW9rhd/5bWhBtg2zfdx9aPZy87XZOov3m0D6qVW35W5isEXCUCaVJQIvMxEMJHOG?= =?utf-8?q?3O7MiKSg5s9QVoKOo=3D?= X-MS-Exchange-SenderADCheck: 1 X-Microsoft-Antispam-Message-Info: pbIoMAXTun+oU4QhOK1wYxPRYUgIre3nqUD0IIwcxbYeNgvblHI+waZXPHdy4HMIFLP4dQ5SXV1VxplTujtCmvwVC1rONmjU0ougZDsI1B0w6/j8bEa+dHm42m/Wnlwzv9joBOcJjYqClvE6njqVPhbhlvqWCZWUujN+3qlMQ8mDVJvYMzXoN12PEaJdP4VcidetV+x6NLAqvd3ISbtWZeD8xANw+LvRamTIFYdF42Cc5iFP/55P+4wzg8FSHtcS7zD5GUkH/rtfDG/4A6AFt6GvK12jb/ceUYS1KkPtqhf6BCi2GG/jVIfMlfsljmFdJbjjicCs6+C/Dpm1Vp/eIxaw/XctP7EasgfBtdpt42UUhfGzm6Ce3y+PGmB05p9eaCPkHfhieru1SwvX4dTgqJoBwDJuIs/HcHnV/2t2zgg= X-Microsoft-Exchange-Diagnostics: 1; BN8PR12MB2963; 20:eMBtOcwrYA0DvAFIqnGLjc5fc4uVW4SPAY2md7yfNTp26USNZQ3XX4m8p5uSlI5r7xa7/eSrudwWaSMjVar8LEQrgMQFlKgAAZI1y1MvXGbW6WKyMZmc2wFgJWEV2teHdG58mTa3MfdimkCsCgwLngL9YM4dkRw6+oxCm7sX6RRq7xOvtUrfG+fCydMZnfvbO94GiPMOFMjlJbK+W9A/zvGkH1dKnlY1RaAbVvqmMsBTqCFdmPxreUkU7KbluxKj X-OriginatorOrg: amd.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 12 Mar 2019 03:11:03.9093 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: de70122e-938f-4df0-d89a-08d6a6985cd2 X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d; Ip=[165.204.84.17]; Helo=[SATLEXCHOV01.amd.com] X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: BN8PR12MB2963 X-Mailman-Original-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amdcloud.onmicrosoft.com; s=selector1-amd-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=8/thAMSLcad1SuGRwHuIQD2QE7El5jiNqSuiGi5g3FU=; b=A9UArpYVTPeKFbdcKuH+wDQ5igmeVsjG6HnViyyn7wJOg0IuUshTQHBErFg/uDBtf7GQm1BuR+Q3Yop+PCV3wkfAr+xFgqb1mf6UuAUPVwB52yHFm6SIJ0Aocgfv9boFPPXzUej6TjQwxFUGwxLMmWGen81ZqhYCgWhk3ZaRgYs= X-Mailman-Original-Authentication-Results: spf=none (sender IP is 165.204.84.17) smtp.mailfrom=amd.com; jlekstrand.net; dkim=none (message not signed) header.d=none;jlekstrand.net; dmarc=permerror action=none header.from=amd.com; X-BeenThere: dri-devel@lists.freedesktop.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Direct Rendering Infrastructure - Development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: =?utf-8?q?Christian_K=C3=B6nig?= , =?utf-8?q?Christian_K=C3=B6nig?= Errors-To: dri-devel-bounces@lists.freedesktop.org Sender: "dri-devel" X-Virus-Scanned: ClamAV using ClamSMTP From: Christian König Lockless container implementation similar to a dma_fence_array, but with only two elements per node and automatic garbage collection. v2: properly document dma_fence_chain_for_each, add dma_fence_chain_find_seqno, drop prev reference during garbage collection if it's not a chain fence. v3: use head and iterator for dma_fence_chain_for_each v4: fix reference count in dma_fence_chain_enable_signaling v5: fix iteration when walking each chain node Signed-off-by: Christian König --- drivers/dma-buf/Makefile | 3 +- drivers/dma-buf/dma-fence-chain.c | 241 ++++++++++++++++++++++++++++++ include/linux/dma-fence-chain.h | 81 ++++++++++ 3 files changed, 324 insertions(+), 1 deletion(-) create mode 100644 drivers/dma-buf/dma-fence-chain.c create mode 100644 include/linux/dma-fence-chain.h diff --git a/drivers/dma-buf/Makefile b/drivers/dma-buf/Makefile index 0913a6ccab5a..1f006e083eb9 100644 --- a/drivers/dma-buf/Makefile +++ b/drivers/dma-buf/Makefile @@ -1,4 +1,5 @@ -obj-y := dma-buf.o dma-fence.o dma-fence-array.o reservation.o seqno-fence.o +obj-y := dma-buf.o dma-fence.o dma-fence-array.o dma-fence-chain.o \ + reservation.o seqno-fence.o obj-$(CONFIG_SYNC_FILE) += sync_file.o obj-$(CONFIG_SW_SYNC) += sw_sync.o sync_debug.o obj-$(CONFIG_UDMABUF) += udmabuf.o diff --git a/drivers/dma-buf/dma-fence-chain.c b/drivers/dma-buf/dma-fence-chain.c new file mode 100644 index 000000000000..0c5e3c902fa0 --- /dev/null +++ b/drivers/dma-buf/dma-fence-chain.c @@ -0,0 +1,241 @@ +/* + * fence-chain: chain fences together in a timeline + * + * Copyright (C) 2018 Advanced Micro Devices, Inc. + * Authors: + * Christian König + * + * This program is free software; you can redistribute it and/or modify it + * under the terms of the GNU General Public License version 2 as published by + * the Free Software Foundation. + * + * This program is distributed in the hope that it will be useful, but WITHOUT + * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or + * FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for + * more details. + */ + +#include + +static bool dma_fence_chain_enable_signaling(struct dma_fence *fence); + +/** + * dma_fence_chain_get_prev - use RCU to get a reference to the previous fence + * @chain: chain node to get the previous node from + * + * Use dma_fence_get_rcu_safe to get a reference to the previous fence of the + * chain node. + */ +static struct dma_fence *dma_fence_chain_get_prev(struct dma_fence_chain *chain) +{ + struct dma_fence *prev; + + rcu_read_lock(); + prev = dma_fence_get_rcu_safe(&chain->prev); + rcu_read_unlock(); + return prev; +} + +/** + * dma_fence_chain_walk - chain walking function + * @fence: current chain node + * + * Walk the chain to the next node. Returns the next fence or NULL if we are at + * the end of the chain. Garbage collects chain nodes which are already + * signaled. + */ +struct dma_fence *dma_fence_chain_walk(struct dma_fence *fence) +{ + struct dma_fence_chain *chain, *prev_chain; + struct dma_fence *prev, *replacement, *tmp; + + chain = to_dma_fence_chain(fence); + if (!chain) { + dma_fence_put(fence); + return NULL; + } + + while ((prev = dma_fence_chain_get_prev(chain))) { + + prev_chain = to_dma_fence_chain(prev); + if (prev_chain) { + if (!dma_fence_is_signaled(prev_chain->fence)) + break; + + replacement = dma_fence_chain_get_prev(prev_chain); + } else { + if (!dma_fence_is_signaled(prev)) + break; + + replacement = NULL; + } + + tmp = cmpxchg(&chain->prev, prev, replacement); + if (tmp == prev) + dma_fence_put(tmp); + else + dma_fence_put(replacement); + dma_fence_put(prev); + } + + dma_fence_put(fence); + return prev; +} +EXPORT_SYMBOL(dma_fence_chain_walk); + +/** + * dma_fence_chain_find_seqno - find fence chain node by seqno + * @pfence: pointer to the chain node where to start + * @seqno: the sequence number to search for + * + * Advance the fence pointer to the chain node which will signal this sequence + * number. If no sequence number is provided then this is a no-op. + * + * Returns EINVAL if the fence is not a chain node or the sequence number has + * not yet advanced far enough. + */ +int dma_fence_chain_find_seqno(struct dma_fence **pfence, uint64_t seqno) +{ + struct dma_fence_chain *chain; + + if (!seqno) + return 0; + + chain = to_dma_fence_chain(*pfence); + if (!chain || chain->base.seqno < seqno) + return -EINVAL; + + dma_fence_chain_for_each(*pfence, &chain->base) { + if ((*pfence)->context != chain->base.context || + to_dma_fence_chain(*pfence)->prev_seqno < seqno) + break; + } + dma_fence_put(&chain->base); + + return 0; +} +EXPORT_SYMBOL(dma_fence_chain_find_seqno); + +static const char *dma_fence_chain_get_driver_name(struct dma_fence *fence) +{ + return "dma_fence_chain"; +} + +static const char *dma_fence_chain_get_timeline_name(struct dma_fence *fence) +{ + return "unbound"; +} + +static void dma_fence_chain_irq_work(struct irq_work *work) +{ + struct dma_fence_chain *chain; + + chain = container_of(work, typeof(*chain), work); + + /* Try to rearm the callback */ + if (!dma_fence_chain_enable_signaling(&chain->base)) + /* Ok, we are done. No more unsignaled fences left */ + dma_fence_signal(&chain->base); + dma_fence_put(&chain->base); +} + +static void dma_fence_chain_cb(struct dma_fence *f, struct dma_fence_cb *cb) +{ + struct dma_fence_chain *chain; + + chain = container_of(cb, typeof(*chain), cb); + irq_work_queue(&chain->work); + dma_fence_put(f); +} + +static bool dma_fence_chain_enable_signaling(struct dma_fence *fence) +{ + struct dma_fence_chain *head = to_dma_fence_chain(fence); + + dma_fence_get(&head->base); + dma_fence_chain_for_each(fence, &head->base) { + struct dma_fence_chain *chain = to_dma_fence_chain(fence); + struct dma_fence *f = chain ? chain->fence : fence; + + dma_fence_get(f); + if (!dma_fence_add_callback(f, &head->cb, dma_fence_chain_cb)) { + dma_fence_put(fence); + return true; + } + dma_fence_put(f); + } + dma_fence_put(&head->base); + return false; +} + +static bool dma_fence_chain_signaled(struct dma_fence *fence) +{ + dma_fence_chain_for_each(fence, fence) { + struct dma_fence_chain *chain = to_dma_fence_chain(fence); + struct dma_fence *f = chain ? chain->fence : fence; + + if (!dma_fence_is_signaled(f)) { + dma_fence_put(fence); + return false; + } + } + + return true; +} + +static void dma_fence_chain_release(struct dma_fence *fence) +{ + struct dma_fence_chain *chain = to_dma_fence_chain(fence); + + dma_fence_put(chain->prev); + dma_fence_put(chain->fence); + dma_fence_free(fence); +} + +const struct dma_fence_ops dma_fence_chain_ops = { + .get_driver_name = dma_fence_chain_get_driver_name, + .get_timeline_name = dma_fence_chain_get_timeline_name, + .enable_signaling = dma_fence_chain_enable_signaling, + .signaled = dma_fence_chain_signaled, + .release = dma_fence_chain_release, +}; +EXPORT_SYMBOL(dma_fence_chain_ops); + +/** + * dma_fence_chain_init - initialize a fence chain + * @chain: the chain node to initialize + * @prev: the previous fence + * @fence: the current fence + * + * Initialize a new chain node and either start a new chain or add the node to + * the existing chain of the previous fence. + */ +void dma_fence_chain_init(struct dma_fence_chain *chain, + struct dma_fence *prev, + struct dma_fence *fence, + uint64_t seqno) +{ + struct dma_fence_chain *prev_chain = to_dma_fence_chain(prev); + uint64_t context; + + spin_lock_init(&chain->lock); + chain->prev = prev; + chain->fence = fence; + chain->prev_seqno = 0; + init_irq_work(&chain->work, dma_fence_chain_irq_work); + + /* Try to reuse the context of the previous chain node. */ + if (prev_chain && __dma_fence_is_later(seqno, prev->seqno)) { + context = prev->context; + chain->prev_seqno = prev->seqno; + } else { + context = dma_fence_context_alloc(1); + /* Make sure that we always have a valid sequence number. */ + if (prev_chain) + seqno = max(prev->seqno, seqno); + } + + dma_fence_init(&chain->base, &dma_fence_chain_ops, + &chain->lock, context, seqno); +} +EXPORT_SYMBOL(dma_fence_chain_init); diff --git a/include/linux/dma-fence-chain.h b/include/linux/dma-fence-chain.h new file mode 100644 index 000000000000..09b038d3f5ef --- /dev/null +++ b/include/linux/dma-fence-chain.h @@ -0,0 +1,81 @@ +/* + * fence-chain: chain fences together in a timeline + * + * Copyright (C) 2018 Advanced Micro Devices, Inc. + * Authors: + * Christian König + * + * This program is free software; you can redistribute it and/or modify it + * under the terms of the GNU General Public License version 2 as published by + * the Free Software Foundation. + * + * This program is distributed in the hope that it will be useful, but WITHOUT + * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or + * FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for + * more details. + */ + +#ifndef __LINUX_DMA_FENCE_CHAIN_H +#define __LINUX_DMA_FENCE_CHAIN_H + +#include +#include + +/** + * struct dma_fence_chain - fence to represent an node of a fence chain + * @base: fence base class + * @lock: spinlock for fence handling + * @prev: previous fence of the chain + * @prev_seqno: original previous seqno before garbage collection + * @fence: encapsulated fence + * @cb: callback structure for signaling + * @work: irq work item for signaling + */ +struct dma_fence_chain { + struct dma_fence base; + spinlock_t lock; + struct dma_fence *prev; + u64 prev_seqno; + struct dma_fence *fence; + struct dma_fence_cb cb; + struct irq_work work; +}; + +extern const struct dma_fence_ops dma_fence_chain_ops; + +/** + * to_dma_fence_chain - cast a fence to a dma_fence_chain + * @fence: fence to cast to a dma_fence_array + * + * Returns NULL if the fence is not a dma_fence_chain, + * or the dma_fence_chain otherwise. + */ +static inline struct dma_fence_chain * +to_dma_fence_chain(struct dma_fence *fence) +{ + if (!fence || fence->ops != &dma_fence_chain_ops) + return NULL; + + return container_of(fence, struct dma_fence_chain, base); +} + +/** + * dma_fence_chain_for_each - iterate over all fences in chain + * @iter: current fence + * @head: starting point + * + * Iterate over all fences in the chain. We keep a reference to the current + * fence while inside the loop which must be dropped when breaking out. + */ +#define dma_fence_chain_for_each(iter, head) \ + for (iter = dma_fence_get(head); iter; \ + iter = dma_fence_chain_walk(iter)) + +struct dma_fence *dma_fence_chain_walk(struct dma_fence *fence); +int dma_fence_chain_find_seqno(struct dma_fence **pfence, uint64_t seqno); +void dma_fence_chain_init(struct dma_fence_chain *chain, + struct dma_fence *prev, + struct dma_fence *fence, + uint64_t seqno); + +#endif /* __LINUX_DMA_FENCE_CHAIN_H */