From patchwork Fri Dec 7 15:54:13 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Chunming Zhou X-Patchwork-Id: 10718395 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 4B23E18A7 for ; Fri, 7 Dec 2018 15:54:43 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 1ED712DDBF for ; Fri, 7 Dec 2018 15:54:43 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 113002DF43; Fri, 7 Dec 2018 15:54:43 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-5.2 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_MED autolearn=ham version=3.3.1 Received: from gabe.freedesktop.org (gabe.freedesktop.org [131.252.210.177]) (using TLSv1.2 with cipher DHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.wl.linuxfoundation.org (Postfix) with ESMTPS id 464872DDBF for ; Fri, 7 Dec 2018 15:54:42 +0000 (UTC) Received: from gabe.freedesktop.org (localhost [127.0.0.1]) by gabe.freedesktop.org (Postfix) with ESMTP id A85816E7C0; Fri, 7 Dec 2018 15:54:41 +0000 (UTC) X-Original-To: intel-gfx@lists.freedesktop.org Delivered-To: intel-gfx@lists.freedesktop.org Received: from NAM03-DM3-obe.outbound.protection.outlook.com (mail-dm3nam03on0609.outbound.protection.outlook.com [IPv6:2a01:111:f400:fe49::609]) by gabe.freedesktop.org (Postfix) with ESMTPS id EF1BF6E7B9; Fri, 7 Dec 2018 15:54:39 +0000 (UTC) Received: from DM5PR12CA0068.namprd12.prod.outlook.com (2603:10b6:3:103::30) by BYAPR12MB2632.namprd12.prod.outlook.com (2603:10b6:a03:69::17) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.1404.19; Fri, 7 Dec 2018 15:54:36 +0000 Received: from BY2NAM03FT017.eop-NAM03.prod.protection.outlook.com (2a01:111:f400:7e4a::200) by DM5PR12CA0068.outlook.office365.com (2603:10b6:3:103::30) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384) id 15.20.1404.18 via Frontend Transport; Fri, 7 Dec 2018 15:54:36 +0000 Received-SPF: None (protection.outlook.com: amd.com does not designate permitted sender hosts) Received: from SATLEXCHOV02.amd.com (165.204.84.17) by BY2NAM03FT017.mail.protection.outlook.com (10.152.84.217) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384) id 15.20.1404.17 via Frontend Transport; Fri, 7 Dec 2018 15:54:35 +0000 Received: from zhoucm1.amd.com (10.34.1.3) by SATLEXCHOV02.amd.com (10.181.40.72) with Microsoft SMTP Server id 14.3.389.1; Fri, 7 Dec 2018 09:54:34 -0600 From: Chunming Zhou To: , , , Date: Fri, 7 Dec 2018 23:54:13 +0800 Message-ID: <20181207155422.15967-1-david1.zhou@amd.com> X-Mailer: git-send-email 2.17.1 MIME-Version: 1.0 X-EOPAttributedMessage: 0 X-MS-Office365-Filtering-HT: Tenant X-Forefront-Antispam-Report: CIP:165.204.84.17; IPV:NLI; CTRY:US; EFV:NLI; SFV:NSPM; SFS:(10009020)(346002)(376002)(39860400002)(136003)(396003)(2980300002)(428003)(199004)(189003)(126002)(476003)(2616005)(426003)(68736007)(5660300001)(47776003)(23676004)(81166006)(8936002)(81156014)(7696005)(53936002)(8676002)(39060400002)(50226002)(66574011)(305945005)(54906003)(316002)(486006)(110136005)(4326008)(36756003)(105586002)(106466001)(2201001)(97736004)(86362001)(5820100001)(77096007)(478600001)(50466002)(1076002)(53416004)(2870700001)(104016004)(6666004)(356004)(2906002)(4744004)(336012)(186003)(14444005)(72206003)(26005)(2004002); DIR:OUT; SFP:1101; SCL:1; SRVR:BYAPR12MB2632; H:SATLEXCHOV02.amd.com; FPR:; SPF:None; LANG:en; PTR:InfoDomainNonexistent; A:1; MX:1; X-Microsoft-Exchange-Diagnostics: 1; BY2NAM03FT017; 1:E9nc8/UJvX6ae7G9ezuUO27Ga0F7NlcWX5ta5Ad+i2QWncN+fgq18KCnNGKFnBa34t4Mm7Qq5tsCCL3ccNekf/qT+Gu2+gT5mGGV6oo5mcZ9jW4S0wqvZE++eCE6CcDb X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 076a7dea-835d-45a8-1efc-08d65c5c49a4 X-Microsoft-Antispam: BCL:0; PCL:0; RULEID:(2390098)(7020095)(4652040)(8989299)(4534185)(4627221)(201703031133081)(201702281549075)(8990200)(5600074)(711020)(2017052603328)(7153060); SRVR:BYAPR12MB2632; X-Microsoft-Exchange-Diagnostics: 1; BYAPR12MB2632; 3:t3cwSi2qZUbcJq4koiiTitmdZxfJSBHHg8hfEJS52WhqZZgosE4uN8I95nWB/xgj03k2wyby18T9sdq1Z5+rMeRC0csvQTv+25vn3V+KQfPR1XmhvW3jfyTo6bceNpO5oSKPkmZuRK5A4UQq+zIKL6u1ML8U1tlqjuoMwLLQgIwWDUF6DcEmCDSx6DfOeKRz7XyIceSfy+3nUxJnnqicD9BS9TJuQPUQZNUlsR4t2kk6H2gXNm008U3QN84AaDC+LjQ3xefDvU0uk5ifgb4eV1QmGomOHIGgDu8ZFIWjMYGLFo5KBIt7YLRHirD+GZMQb2sJJoiYPIb49ntsL8792iOYT5v5eV2hEpBX/kF6wgE=; 25:lzWOj3PDjmzm/uI6IXswPHMCvvqZ7pezHjpKbPnm3GdQmYFz4WBrR9DBlGo9xOVTk8/V8bWeVtbLM9iHKGHXEjVei9CDSstie2B3nUWQ3JiqhPs0DqNU60RaFKDxnYtrmERNP8Y1o/tNzTqP4ECSh5uwaEwB00TxZTIyIBzqHyMfcvdNKD5zXrSO+MKRPo4ACAjasQGG2D1Eb+l3RL+UNzyl3eXfRL+LJzCxx9c5tPVJDfL8qkWoEq8eHms0DTn8Ugp/OOzIibMA/KP2SoyCAeHAOjC9yAzoYQIDtPZmt8hloXXKvo3+6+QJGDJqCKCQ+bpa8YVltbPCryA47RtrCg== X-MS-TrafficTypeDiagnostic: BYAPR12MB2632: X-Microsoft-Exchange-Diagnostics: 1; BYAPR12MB2632; 31:+h0aDxaLbFx6hR0puadx/3hpmQRSV54hP+LjSVvlX8eGfK7JpdlL/ta5JKEPFV0x4ooTyzYFVVynfqqC4oeH6brsUxMDrYXiABSA+m4e9FPjGBEcX54N0uJD9h3/0w3t3oCsKXjKVIs2XMzPZolQgh+96cuo8LJv+XTOPukkifh9dm2uTH9qBWDPCziampz9oKoDrtCUOhoGC7RbokkJwYJ8Ssc9WUarvaHy4nKtDrk=; 20:IG7e6mjW8mSzeGhONHvQ1uesJHu0Cd0X/4MnZ+GOzgIBUKo9RUhEfi6vhEyP0GRcqZXl0WwkmI0fLffyIpVXwk2tNqRQsI3VbqWTlKznxJVlfFYrnfwAnCvC95HBJ+FRbc5Fcko8sR4916W9vOPWV5/2bECYUKFfPyoIC+RWQJA4OE2dkk2rTZsAX77IMmaHZQnIUhbRWkB50V0OakJteIZrHV4G+gk9u2zoEa7gX0McuUok5mk+6Q2T4cBUlM5n+a1RZYBimtxQhM+BxNAQCel1xqG+FAvso+iY0zI8bFEPWpqQKjzSMXdcqadMaQ+pA3JzVhbHbeYtE1xSujoz8pSs+0zwQpvFJv0agG1/30pPbSQHBD4PCiSjVGnryuQLYuUxQh4AQR5HnjkmLljSe0cUn1m16rouqKWm1UNOnTEAMrLT2s8nciZ8cpMjngK2+bPVbn7+UQOE21GkgO0kVgUy7ltvJCTd3fB4bwa+0eqHRXqistaxKOJodNQDePO7 X-Microsoft-Antispam-PRVS: X-MS-Exchange-SenderADCheck: 1 X-Exchange-Antispam-Report-CFA-Test: BCL:0; PCL:0; RULEID:(8211001083)(6040522)(2401047)(8121501046)(5005006)(10201501046)(3002001)(93006095)(93003095)(3231455)(999002)(944501520)(52105112)(6055026)(148016)(149066)(150057)(6041310)(20161123562045)(20161123558120)(20161123560045)(201703131423095)(201702281528075)(20161123555045)(201703061421075)(201703061406153)(20161123564045)(201708071742011)(7699051)(76991095); SRVR:BYAPR12MB2632; BCL:0; PCL:0; RULEID:; SRVR:BYAPR12MB2632; X-Microsoft-Exchange-Diagnostics: 1; BYAPR12MB2632; 4:E2aeOPrSceth90C0rfC7YHokDFFTlS5ZwbtjONmBS1/54ygifS0bElRyBBG0sdHVuv5vPQgGYBxnlJ8KmPjLgDJ0ZJw1gQKwRq6wvsso9sMrgwF29381LXvnF/xc8jyQNEbe6qfPPqyqjUa37vyWqhXxEfoHtzQCoh3eAkfSjdRd+RdqW/VApR9fdUIlId8bXoUsBlOfweEPsA5slJ31TUJ8r9+Bqg6N2Dvw92pq8Oro2ndSaxT+gCUfzX6zh5sb3LYWZ6uEG5BQNo+PUmgP3w== X-Forefront-PRVS: 0879599414 X-Microsoft-Exchange-Diagnostics: =?utf-8?q?1=3BBYAPR12MB2632=3B23=3A9b4r70C?= =?utf-8?q?Saxih39CY6A5rdgqLRPa8QPLirsaz+mMi4EIsvQyGhDBVisl/PwsZM1la7E10AunZ?= =?utf-8?q?Necqh7D5bvLnbrfJTpSQ2MShhvWJ2jtzBRglrZBwmA5Uom8pzJRZWWGt8KyKFQoDi?= =?utf-8?q?WvESawZsJFKNm8Q53wDClGfks0PmfnVE2sKVHl4eq8rmbYQerNDcfkUu1fypJZN8g?= =?utf-8?q?wMrqSbJyo3Q3uOmmsxPEe72u1UhE1zHewNuszRtdLrVrdaHIpbjPcXl2i+MI/9hpE?= =?utf-8?q?d4Vymfpe+aQJxcVrPh96rx+BKpN6S+tUpZof8g1ZYgJ2JncKiLXacO9VX/NAfJEzH?= =?utf-8?q?NwB+XOnnWW6G/UV4NPa7E4MtFQHHG9RSC4eBDfQ/dDBfC3T1cb3BxC3LrInR+yzDH?= =?utf-8?q?9yLeUZdCjTzy/sggiwNPdmnLnR6os2y20LlbrFxdQ7fj/g5glVbv8ya81pMGmglz2?= =?utf-8?q?eOGC1NcMOlJPVfwMOJd3VQVSGxgIfQYwqdxyYrgq/hYunzS7CjQqX744B53A88beJ?= =?utf-8?q?SOJ6iFzwoTZpXCyWQLIV7/FQ5/SAsuBexx8kOQfPckFMSJm01hVHKsCmcdHavygpH?= =?utf-8?q?LvW2jXrEBeJ6DjRtKLd50GNqUJYYXcckXOB7Qzykpt74YMk1aJ0C2OuLI/VdZt73I?= =?utf-8?q?UHiUQFE9+bsU8p5YWvK+r28Qy35P7Fsw2HFxgjE/tm7ls0kWhn3mnhi3fwiuGjgYU?= =?utf-8?q?sSntfsEC1TT/coNf7Ojr9QfmRxmGtvQYqPbLKWO5fBUg5iJN9ph8HlQeX7kpzmvmb?= =?utf-8?q?CrGGXV399+l4L61wukRetYca9ngcWstkrYbiouTPv3EPsv5Q4J3JIFHs0RZVWo4WX?= =?utf-8?q?lEehpyunV327ORbCENytCEjbHHt/EQi9zWIv/ykWcMwjUxTBuQQyGF7wFAW/14P/8?= =?utf-8?q?hfZU0A9K+cOGcHf3PI8w6xPfTqUSdwfnWRa/zpO6tJEqceZf4D2dpmVdDRNRknTQL?= =?utf-8?q?fpDF+RVJPFEOO34Bs8n9JVRVYYtDmDRkzJlFT2AvbhoHIRiYGSwYaK1BTWbt3hwRl?= =?utf-8?q?+tDxd5qqxXw9s2wJFLB/PZuguw1jXAYF98iitpHrgjORY4ZEo+6K1BHpskTRoJgXP?= =?utf-8?q?k/h9StYfzSuF+0AU6g9irWQ5psLrJ/m/KzQOpBMB4AAiPTCTXsTo+OsXFo51kmbON?= =?utf-8?q?kDtQZoybGMLMExyZwPgFnq0YwCBALnGhEkCVHj8MkjXTyqNv4fcRJuwu8NSfvgO7e?= =?utf-8?q?U+J5GJeFOfOh1stfA=3D?= X-Microsoft-Antispam-Message-Info: PelnTpDEESmeFolzK2IEfd8MNwe8DV4Gevk/67lSTW2yj+wBxtbkPoNokBDgggp9j0CA8O4r5EqH2d6Q/I4dMQFxaHPNv0K3WGJTl1CqY9nyM9YYXF4Vw6PbybHOXfHnxRHidk921QCocCHbnHItIYz6Od5knJCWaniwD3/xNundqJQyD9Netctpj/Pzc18nWuYKY7e8AWuFUKsQbjsNLdxYjPsu8IEXpH24Gg5p8qnZ+cK5InKFfxFRXMsOr24r2Cv/RPC/iZ9yd4TDNySmo2ROOLauKe1pryODmT43EhFeCYgoUzSZT2mZ6knvavi29Yu+gutrjBBNf31KWlgys5B7dwhqVaoKyebOc9Ue1DA= X-Microsoft-Exchange-Diagnostics: 1; BYAPR12MB2632; 6:v9Sd1jBN/PZGI7V8WzRn5/6KfL/d/42PGHxu4JtVZS1MJhi7YTyzyn7KQTqfiUMKwKWLYQ8eo6t5AqTWCaU24wHV1JWB7OL4NMSYjQzvvSVPtzQoUaSrsH5AO6TQcnaVqmTEWoRAC4w9zMEZKfgs8ETc+S6DmhY/elubNRd4mAdVB6L54hYywiAoJ4LlPgKaTcfJTUyr/sw/Fx057dtW9PPqYzVCes46e2xpHZghbQeBq+gt8z3dsZaNMJ8omywmWjh26qiYJGFm2eErFtZp2wTrLYePhs3bwbo50QoNSMTnJJDYfwKvOkW2CbhsrGBQrcbif1ofs2GtPTVBKT924tXpL9OfbxkYrsg2/9rxVpfbtoPBMG5/5+ZEh0DDVa+opfgltyIqtetdC1O0pwwwfiwn1sNJVkYyU4E8c244ypJQoYWMDXgsNjkI01KYrbK3Pr1csWj32/2v3/y8MEUaHw==; 5:8u1jZUUA6J7UB8G5wiRWtPCvR+yCKfbqfOAyUYcBXPXCcQlRpJmtw3lBzzCP5GQ4KKJp7H6OwDulTswefg5UAjMDoMWcFavFLgSbmnhKrnYwDYA/4+AYt+VIxaQgP8XNjlhl4akjKguFK0ZNc+E6vDzGvriVNfQ7k8bHElxkiB0=; 7:uWJoiqZFUrruWx9m8pHiaq/N0rY0Gm0frg0wpbS6+opRzTmX12oOYg/yoyo9dvKMxuvCF70v6u8LVSKyfN2GnycBoiK21qdOK+PbfiUfY9fy4j6LEXsMF6/Jt8B2nip713DSiwoWsNN0PqEEPzY+WA== SpamDiagnosticOutput: 1:99 SpamDiagnosticMetadata: NSPM X-Microsoft-Exchange-Diagnostics: 1; BYAPR12MB2632; 20:B/ryeYahv2RVthU7idEtVXSXZXaTenZ8twfMRYOWT8AohubLPCW5mbemrUgF7tHlpNMqR+qfi5EfQYxXydm03aEyYH0HcjS9WSXK55DvCD2HZAm4t2lAHN2dnGXqFRRFU1fuXkdQ6xPnFgxWrU4HJ9bX7Ek1+kpWvxembGwlzzvoQpLE+7dTVoxbxON9gLRpCY+rfDfSNzeHW7WWhuo8lwqRji+VsgkcVLj2BfM/pCCWSIoN6VQmlQYtntMzdYXf X-OriginatorOrg: amd.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 07 Dec 2018 15:54:35.9243 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 076a7dea-835d-45a8-1efc-08d65c5c49a4 X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d; Ip=[165.204.84.17]; Helo=[SATLEXCHOV02.amd.com] X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: BYAPR12MB2632 Subject: [Intel-gfx] [PATCH 01/10] dma-buf: add new dma_fence_chain container v4 X-BeenThere: intel-gfx@lists.freedesktop.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Intel graphics driver community testing & development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: =?utf-8?q?Christian_K=C3=B6nig?= , =?utf-8?q?Christian_K=C3=B6nig?= Errors-To: intel-gfx-bounces@lists.freedesktop.org Sender: "Intel-gfx" X-Virus-Scanned: ClamAV using ClamSMTP From: Christian König Lockless container implementation similar to a dma_fence_array, but with only two elements per node and automatic garbage collection. v2: properly document dma_fence_chain_for_each, add dma_fence_chain_find_seqno, drop prev reference during garbage collection if it's not a chain fence. v3: use head and iterator for dma_fence_chain_for_each v4: fix reference count in dma_fence_chain_enable_signaling Signed-off-by: Christian König --- drivers/dma-buf/Makefile | 3 +- drivers/dma-buf/dma-fence-chain.c | 241 ++++++++++++++++++++++++++++++ include/linux/dma-fence-chain.h | 81 ++++++++++ 3 files changed, 324 insertions(+), 1 deletion(-) create mode 100644 drivers/dma-buf/dma-fence-chain.c create mode 100644 include/linux/dma-fence-chain.h diff --git a/drivers/dma-buf/Makefile b/drivers/dma-buf/Makefile index 0913a6ccab5a..1f006e083eb9 100644 --- a/drivers/dma-buf/Makefile +++ b/drivers/dma-buf/Makefile @@ -1,4 +1,5 @@ -obj-y := dma-buf.o dma-fence.o dma-fence-array.o reservation.o seqno-fence.o +obj-y := dma-buf.o dma-fence.o dma-fence-array.o dma-fence-chain.o \ + reservation.o seqno-fence.o obj-$(CONFIG_SYNC_FILE) += sync_file.o obj-$(CONFIG_SW_SYNC) += sw_sync.o sync_debug.o obj-$(CONFIG_UDMABUF) += udmabuf.o diff --git a/drivers/dma-buf/dma-fence-chain.c b/drivers/dma-buf/dma-fence-chain.c new file mode 100644 index 000000000000..0c5e3c902fa0 --- /dev/null +++ b/drivers/dma-buf/dma-fence-chain.c @@ -0,0 +1,241 @@ +/* + * fence-chain: chain fences together in a timeline + * + * Copyright (C) 2018 Advanced Micro Devices, Inc. + * Authors: + * Christian König + * + * This program is free software; you can redistribute it and/or modify it + * under the terms of the GNU General Public License version 2 as published by + * the Free Software Foundation. + * + * This program is distributed in the hope that it will be useful, but WITHOUT + * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or + * FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for + * more details. + */ + +#include + +static bool dma_fence_chain_enable_signaling(struct dma_fence *fence); + +/** + * dma_fence_chain_get_prev - use RCU to get a reference to the previous fence + * @chain: chain node to get the previous node from + * + * Use dma_fence_get_rcu_safe to get a reference to the previous fence of the + * chain node. + */ +static struct dma_fence *dma_fence_chain_get_prev(struct dma_fence_chain *chain) +{ + struct dma_fence *prev; + + rcu_read_lock(); + prev = dma_fence_get_rcu_safe(&chain->prev); + rcu_read_unlock(); + return prev; +} + +/** + * dma_fence_chain_walk - chain walking function + * @fence: current chain node + * + * Walk the chain to the next node. Returns the next fence or NULL if we are at + * the end of the chain. Garbage collects chain nodes which are already + * signaled. + */ +struct dma_fence *dma_fence_chain_walk(struct dma_fence *fence) +{ + struct dma_fence_chain *chain, *prev_chain; + struct dma_fence *prev, *replacement, *tmp; + + chain = to_dma_fence_chain(fence); + if (!chain) { + dma_fence_put(fence); + return NULL; + } + + while ((prev = dma_fence_chain_get_prev(chain))) { + + prev_chain = to_dma_fence_chain(prev); + if (prev_chain) { + if (!dma_fence_is_signaled(prev_chain->fence)) + break; + + replacement = dma_fence_chain_get_prev(prev_chain); + } else { + if (!dma_fence_is_signaled(prev)) + break; + + replacement = NULL; + } + + tmp = cmpxchg(&chain->prev, prev, replacement); + if (tmp == prev) + dma_fence_put(tmp); + else + dma_fence_put(replacement); + dma_fence_put(prev); + } + + dma_fence_put(fence); + return prev; +} +EXPORT_SYMBOL(dma_fence_chain_walk); + +/** + * dma_fence_chain_find_seqno - find fence chain node by seqno + * @pfence: pointer to the chain node where to start + * @seqno: the sequence number to search for + * + * Advance the fence pointer to the chain node which will signal this sequence + * number. If no sequence number is provided then this is a no-op. + * + * Returns EINVAL if the fence is not a chain node or the sequence number has + * not yet advanced far enough. + */ +int dma_fence_chain_find_seqno(struct dma_fence **pfence, uint64_t seqno) +{ + struct dma_fence_chain *chain; + + if (!seqno) + return 0; + + chain = to_dma_fence_chain(*pfence); + if (!chain || chain->base.seqno < seqno) + return -EINVAL; + + dma_fence_chain_for_each(*pfence, &chain->base) { + if ((*pfence)->context != chain->base.context || + to_dma_fence_chain(*pfence)->prev_seqno < seqno) + break; + } + dma_fence_put(&chain->base); + + return 0; +} +EXPORT_SYMBOL(dma_fence_chain_find_seqno); + +static const char *dma_fence_chain_get_driver_name(struct dma_fence *fence) +{ + return "dma_fence_chain"; +} + +static const char *dma_fence_chain_get_timeline_name(struct dma_fence *fence) +{ + return "unbound"; +} + +static void dma_fence_chain_irq_work(struct irq_work *work) +{ + struct dma_fence_chain *chain; + + chain = container_of(work, typeof(*chain), work); + + /* Try to rearm the callback */ + if (!dma_fence_chain_enable_signaling(&chain->base)) + /* Ok, we are done. No more unsignaled fences left */ + dma_fence_signal(&chain->base); + dma_fence_put(&chain->base); +} + +static void dma_fence_chain_cb(struct dma_fence *f, struct dma_fence_cb *cb) +{ + struct dma_fence_chain *chain; + + chain = container_of(cb, typeof(*chain), cb); + irq_work_queue(&chain->work); + dma_fence_put(f); +} + +static bool dma_fence_chain_enable_signaling(struct dma_fence *fence) +{ + struct dma_fence_chain *head = to_dma_fence_chain(fence); + + dma_fence_get(&head->base); + dma_fence_chain_for_each(fence, &head->base) { + struct dma_fence_chain *chain = to_dma_fence_chain(fence); + struct dma_fence *f = chain ? chain->fence : fence; + + dma_fence_get(f); + if (!dma_fence_add_callback(f, &head->cb, dma_fence_chain_cb)) { + dma_fence_put(fence); + return true; + } + dma_fence_put(f); + } + dma_fence_put(&head->base); + return false; +} + +static bool dma_fence_chain_signaled(struct dma_fence *fence) +{ + dma_fence_chain_for_each(fence, fence) { + struct dma_fence_chain *chain = to_dma_fence_chain(fence); + struct dma_fence *f = chain ? chain->fence : fence; + + if (!dma_fence_is_signaled(f)) { + dma_fence_put(fence); + return false; + } + } + + return true; +} + +static void dma_fence_chain_release(struct dma_fence *fence) +{ + struct dma_fence_chain *chain = to_dma_fence_chain(fence); + + dma_fence_put(chain->prev); + dma_fence_put(chain->fence); + dma_fence_free(fence); +} + +const struct dma_fence_ops dma_fence_chain_ops = { + .get_driver_name = dma_fence_chain_get_driver_name, + .get_timeline_name = dma_fence_chain_get_timeline_name, + .enable_signaling = dma_fence_chain_enable_signaling, + .signaled = dma_fence_chain_signaled, + .release = dma_fence_chain_release, +}; +EXPORT_SYMBOL(dma_fence_chain_ops); + +/** + * dma_fence_chain_init - initialize a fence chain + * @chain: the chain node to initialize + * @prev: the previous fence + * @fence: the current fence + * + * Initialize a new chain node and either start a new chain or add the node to + * the existing chain of the previous fence. + */ +void dma_fence_chain_init(struct dma_fence_chain *chain, + struct dma_fence *prev, + struct dma_fence *fence, + uint64_t seqno) +{ + struct dma_fence_chain *prev_chain = to_dma_fence_chain(prev); + uint64_t context; + + spin_lock_init(&chain->lock); + chain->prev = prev; + chain->fence = fence; + chain->prev_seqno = 0; + init_irq_work(&chain->work, dma_fence_chain_irq_work); + + /* Try to reuse the context of the previous chain node. */ + if (prev_chain && __dma_fence_is_later(seqno, prev->seqno)) { + context = prev->context; + chain->prev_seqno = prev->seqno; + } else { + context = dma_fence_context_alloc(1); + /* Make sure that we always have a valid sequence number. */ + if (prev_chain) + seqno = max(prev->seqno, seqno); + } + + dma_fence_init(&chain->base, &dma_fence_chain_ops, + &chain->lock, context, seqno); +} +EXPORT_SYMBOL(dma_fence_chain_init); diff --git a/include/linux/dma-fence-chain.h b/include/linux/dma-fence-chain.h new file mode 100644 index 000000000000..a5c2e8c6915c --- /dev/null +++ b/include/linux/dma-fence-chain.h @@ -0,0 +1,81 @@ +/* + * fence-chain: chain fences together in a timeline + * + * Copyright (C) 2018 Advanced Micro Devices, Inc. + * Authors: + * Christian König + * + * This program is free software; you can redistribute it and/or modify it + * under the terms of the GNU General Public License version 2 as published by + * the Free Software Foundation. + * + * This program is distributed in the hope that it will be useful, but WITHOUT + * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or + * FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for + * more details. + */ + +#ifndef __LINUX_DMA_FENCE_CHAIN_H +#define __LINUX_DMA_FENCE_CHAIN_H + +#include +#include + +/** + * struct dma_fence_chain - fence to represent an node of a fence chain + * @base: fence base class + * @lock: spinlock for fence handling + * @prev: previous fence of the chain + * @prev_seqno: original previous seqno before garbage collection + * @fence: encapsulated fence + * @cb: callback structure for signaling + * @work: irq work item for signaling + */ +struct dma_fence_chain { + struct dma_fence base; + spinlock_t lock; + struct dma_fence *prev; + u64 prev_seqno; + struct dma_fence *fence; + struct dma_fence_cb cb; + struct irq_work work; +}; + +extern const struct dma_fence_ops dma_fence_chain_ops; + +/** + * to_dma_fence_chain - cast a fence to a dma_fence_chain + * @fence: fence to cast to a dma_fence_array + * + * Returns NULL if the fence is not a dma_fence_chain, + * or the dma_fence_chain otherwise. + */ +static inline struct dma_fence_chain * +to_dma_fence_chain(struct dma_fence *fence) +{ + if (!fence || fence->ops != &dma_fence_chain_ops) + return NULL; + + return container_of(fence, struct dma_fence_chain, base); +} + +/** + * dma_fence_chain_for_each - iterate over all fences in chain + * @iter: current fence + * @head: starting point + * + * Iterate over all fences in the chain. We keep a reference to the current + * fence while inside the loop which must be dropped when breaking out. + */ +#define dma_fence_chain_for_each(iter, head) \ + for (iter = dma_fence_get(head); iter; \ + iter = dma_fence_chain_walk(head)) + +struct dma_fence *dma_fence_chain_walk(struct dma_fence *fence); +int dma_fence_chain_find_seqno(struct dma_fence **pfence, uint64_t seqno); +void dma_fence_chain_init(struct dma_fence_chain *chain, + struct dma_fence *prev, + struct dma_fence *fence, + uint64_t seqno); + +#endif /* __LINUX_DMA_FENCE_CHAIN_H */