From patchwork Fri Nov 24 00:46:56 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Ma=C3=ADra_Canal?= X-Patchwork-Id: 13466852 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from gabe.freedesktop.org (gabe.freedesktop.org [131.252.210.177]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 9300DC61D97 for ; Fri, 24 Nov 2023 01:27:01 +0000 (UTC) Received: from gabe.freedesktop.org (localhost [127.0.0.1]) by gabe.freedesktop.org (Postfix) with ESMTP id 537B010E777; Fri, 24 Nov 2023 01:26:57 +0000 (UTC) Received: from fanzine2.igalia.com (fanzine.igalia.com [178.60.130.6]) by gabe.freedesktop.org (Postfix) with ESMTPS id EDB7B10E780 for ; Fri, 24 Nov 2023 01:26:52 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=igalia.com; s=20170329; h=Content-Transfer-Encoding:Content-Type:MIME-Version:Message-ID: Date:Subject:Cc:To:From:Sender:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: In-Reply-To:References:List-Id:List-Help:List-Unsubscribe:List-Subscribe: List-Post:List-Owner:List-Archive; bh=CLTL/hePkHm5kRRgl95sjwe7dOb+PLE8LsUR6yqOw60=; b=SP8GpHrynDPVeGWSLX88T+FpH3 KbQGh9oJ1A0Oqit6MrsIn+EYNeFt5FwVDfzGx96oWSbR8ALlB25jvj7Y8wt7esH74V4LE/iTMEFdF JjHSjZtm9mReGMAvPztK+21BccMum/okd9W/5GZTqIWL+Gmn6aLbQFzteqeNp7YelFLPH7PubvBFK 24JEL4mVP8OJTWhsItf+VdO3tQpQCLPKpc7tXhbp+7H1g29vxWNQNuikJYlOFpMV1V/I3E02Z1G7K n1pEU1oiO0SOuwN63KmQ9QRuoe3Jk/ZgDVzqoYPxrpBwItgztzcnXHKcaR0c5P/vzpMjIYUdC8Wj8 vur66Bag==; Received: from [177.34.168.16] (helo=morissey..) by fanzine2.igalia.com with esmtpsa (Cipher TLS1.3:ECDHE_X25519__RSA_PSS_RSAE_SHA256__AES_256_GCM:256) (Exim) id 1r6Kxo-006Zga-TD; Fri, 24 Nov 2023 02:26:37 +0100 From: =?utf-8?q?Ma=C3=ADra_Canal?= To: Melissa Wen , Iago Toral , David Airlie , Daniel Vetter , Maarten Lankhorst , Maxime Ripard , Thomas Zimmermann Subject: [PATCH v2 00/17] drm/v3d: Introduce CPU jobs Date: Thu, 23 Nov 2023 21:46:56 -0300 Message-ID: <20231124012548.772095-1-mcanal@igalia.com> X-Mailer: git-send-email 2.41.0 MIME-Version: 1.0 X-BeenThere: dri-devel@lists.freedesktop.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Direct Rendering Infrastructure - Development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: =?utf-8?q?Ma=C3=ADra_Canal?= , kernel-dev@igalia.com, Boris Brezillon , dri-devel@lists.freedesktop.org, Faith Ekstrand Errors-To: dri-devel-bounces@lists.freedesktop.org Sender: "dri-devel" This patchset implements the basic infrastructure for a new type of V3D job, a CPU job. A CPU job is a job that requires CPU intervention. It would be nice to perform this operations on the kernel space as we can attach multiple in/out syncobjs to it. Why we want a CPU job on the kernel? ==================================== There are some Vulkan commands that cannot be performed by the GPU, so we implement those as CPU jobs on Mesa. But to synchronize a CPU job in the user space, we need to hold part of the command submission flow in order to correctly synchronize their execution. By moving the CPU job to the kernel, we can make use of the DRM schedule queues and all the advantages it brings with it. This way, instead of stalling the submission thread, we can use syncobjs to synchronize the job, providing a more effective management. About the implementation ======================== After we decided that we would like to have a CPU job implementation in the kernel, we could think about two possible implementations for this job: creating an IOCTL for each type of CPU job or using an user extension to provide a polymorphic behavior to a single CPU job IOCTL. We decided for the latter one. We have different types of CPU jobs (indirect CSD jobs, timestamp query jobs, copy query results jobs...) and each of them have a common infrastructure, but perform different operations. Therefore, by using a single IOCTL that is extended by an user extension, we can reuse the common infrastructure - avoiding code repetition - and yet use the user extension ID to identify the type of job and depending on the type of job, perform a certain operation. About the patchset ================== This patchset introduces the basic infrastructure of a CPU job with a new V3D queue (V3D_CPU) e new tracers. Moreover, it introduces six types of CPU jobs: an indirect CSD job, a timestamp query job, a reset timestamp queries job, a copy timestamp query results job, a reset performance queries job, and a copy performance query results job. An indirect CSD job is a job that, when executed in the queue, will map the indirect buffer, read the dispatch parameters, and submit a regular dispatch. So, the CSD job depends on the CPU job execution. We attach the wait dependencies to the CPU job and once they are satisfied, we read the dispatch parameters, rewrite the uniforms (if needed) and enable the CSD job execution, which depends on the completion of the CPU job. A timestamp query job is a job that calculates the value of the timestamp query and updates the availability of the query. In order to implement this job, we had to change the Mesa implementation of the timestamp. Now, the timestamp query value is tracked in a BO, instead of using a memory address. Moreover, the timestamp query availability is tracked with a syncobj, which is signaled when the query is available. A reset timestamp queries job is a job that resets the timestamp queries by zeroing the timestamp BO in the right positions. The right position on the timestamp BO is found through the offset of the first query. A reset performance queries job is a job that zeros the values of the performance monitors associated to that query. Moreover, it resets the availability syncobj related to that query. A copy query results job is a job that copy the results of a query to a BO in a given offset with a given stride. The patchset is divided as such: * #1 - #4: refactoring operations to prepare for the introduction of the CPU job * #5: addressing a vulnerability in the multisync extension * #6: decouple job allocation from job initiation * #7 - #9: introduction of the CPU job * #10 - #11: refactoring operations to prepare for the introduction of the indirect CSD job * #12: introduction of the indirect CSD job * #13: introduction of the timestamp query job * #14: introduction of the reset timestamp queries job * #15: introduction of the copy timestamp query results job * #16: introduction of the reset performance queries job * #17: introduction of the copy performance query results job This patchset has its Mesa counterpart, which is available on [1]. Both the kernel and Mesa implementation were tested with * `dEQP-VK.compute.pipeline.indirect_dispatch.*`, * `dEQP-VK.pipeline.monolithic.timestamp.*`, * `dEQP-VK.synchronization.*`, * `dEQP-VK.query_pool.*` * and `dEQP-VK.multiview.*`. [1] https://gitlab.freedesktop.org/mairacanal/mesa/-/tree/v3dv/v5/cpu-job Changelog ========= v1 -> v2: https://lore.kernel.org/dri-devel/20230904175019.1172713-1-mcanal@igalia.com/ * Rebase on top of drm-misc-next. * Add GPU stats to the CPU queue. Best Regards, - Maíra Maíra Canal (11): drm/v3d: Don't allow two multisync extensions in the same job drm/v3d: Decouple job allocation from job initiation drm/v3d: Use v3d_get_extensions() to parse CPU job data drm/v3d: Create tracepoints to track the CPU job drm/v3d: Enable BO mapping drm/v3d: Create a CPU job extension for a indirect CSD job drm/v3d: Create a CPU job extension for the timestamp query job drm/v3d: Create a CPU job extension for the reset timestamp job drm/v3d: Create a CPU job extension to copy timestamp query to a buffer drm/v3d: Create a CPU job extension for the reset performance query job drm/v3d: Create a CPU job extension for the copy performance query job Melissa Wen (6): drm/v3d: Remove unused function header drm/v3d: Move wait BO ioctl to the v3d_bo file drm/v3d: Detach job submissions IOCTLs to a new specific file drm/v3d: Simplify job refcount handling drm/v3d: Add a CPU job submission drm/v3d: Detach the CSD job BO setup drivers/gpu/drm/v3d/Makefile | 3 +- drivers/gpu/drm/v3d/v3d_bo.c | 51 ++ drivers/gpu/drm/v3d/v3d_drv.c | 4 + drivers/gpu/drm/v3d/v3d_drv.h | 132 ++- drivers/gpu/drm/v3d/v3d_gem.c | 768 ------------------ drivers/gpu/drm/v3d/v3d_sched.c | 315 ++++++++ drivers/gpu/drm/v3d/v3d_submit.c | 1293 ++++++++++++++++++++++++++++++ drivers/gpu/drm/v3d/v3d_trace.h | 57 ++ include/uapi/drm/v3d_drm.h | 220 ++++- 9 files changed, 2063 insertions(+), 780 deletions(-) create mode 100644 drivers/gpu/drm/v3d/v3d_submit.c --- 2.41.0