From patchwork Wed May 15 21:42:51 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Lucas De Marchi X-Patchwork-Id: 13665563 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from gabe.freedesktop.org (gabe.freedesktop.org [131.252.210.177]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id C9C82C25B78 for ; Wed, 15 May 2024 21:43:09 +0000 (UTC) Received: from gabe.freedesktop.org (localhost [127.0.0.1]) by gabe.freedesktop.org (Postfix) with ESMTP id 6708610E1D6; Wed, 15 May 2024 21:43:00 +0000 (UTC) Authentication-Results: gabe.freedesktop.org; dkim=pass (2048-bit key; unprotected) header.d=intel.com header.i=@intel.com header.b="IXpZqpdC"; dkim-atps=neutral Received: from mgamail.intel.com (mgamail.intel.com [192.198.163.19]) by gabe.freedesktop.org (Postfix) with ESMTPS id 24B9A10E1D6; Wed, 15 May 2024 21:42:59 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1715809379; x=1747345379; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=VWumwO4JTUn0RMZg/K/sOdW7Z8QtJmS2dmW4AjvMp4M=; b=IXpZqpdCpa9J5UANVip2KMsDKUplV81DOTLXh2sxRs8Ph7dKEEHI/1fF pqBqHINWwvWZBhTPkYjQjpoc0Nnq5ftLvBWvijFf8r9RmCvDbZV5aFCAE ermnB7maG+o65EDbPxmPfr9M2pWqPvHi4IAfgeqHhsur+p7AMRb80maQy g6CIeWorDyKyyjaT/2H74l7P+1Q5i4eU5frAocoS8L/LNjZCHrU6YRBbE J25W45ZfDmJBqkecyxtIgl3ezp4M6QXzrDT2BUjwg61cD0w4RFedtWhqb 1sbnb5uZQb4y0NX8pFqwj0de5gSJlIOc4zQz3JBYyWjneyQRmwszObcZD g==; X-CSE-ConnectionGUID: ZPcjNUT6R+qLzMvF/p9cAQ== X-CSE-MsgGUID: feurSZCHTVSStPFv9kXL6w== X-IronPort-AV: E=McAfee;i="6600,9927,11074"; a="11739159" X-IronPort-AV: E=Sophos;i="6.08,162,1712646000"; d="scan'208";a="11739159" Received: from fmviesa005.fm.intel.com ([10.60.135.145]) by fmvoesa113.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 15 May 2024 14:42:59 -0700 X-CSE-ConnectionGUID: rT02rY+CRHK+vsmTNclmZQ== X-CSE-MsgGUID: HvABzK7zT5iMR8xmn8e/Qw== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.08,162,1712646000"; d="scan'208";a="35651125" Received: from lucas-s2600cw.jf.intel.com ([10.165.21.196]) by fmviesa005-auth.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 15 May 2024 14:42:58 -0700 From: Lucas De Marchi To: Cc: Tvrtko Ursulin , Umesh Nerlige Ramappa , dri-devel@lists.freedesktop.org, Lucas De Marchi , Nirmoy Das , Oak Zeng Subject: [PATCH v4 1/8] drm/xe: Promote xe_hw_engine_class_to_str() Date: Wed, 15 May 2024 14:42:51 -0700 Message-ID: <20240515214258.59209-2-lucas.demarchi@intel.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20240515214258.59209-1-lucas.demarchi@intel.com> References: <20240515214258.59209-1-lucas.demarchi@intel.com> MIME-Version: 1.0 X-BeenThere: dri-devel@lists.freedesktop.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Direct Rendering Infrastructure - Development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dri-devel-bounces@lists.freedesktop.org Sender: "dri-devel" Move it out of the sysfs compilation unit so it can be re-used in other places. Reviewed-by: Nirmoy Das Reviewed-by: Oak Zeng Signed-off-by: Lucas De Marchi --- drivers/gpu/drm/xe/xe_hw_engine.c | 18 ++++++++++++++++++ drivers/gpu/drm/xe/xe_hw_engine.h | 2 ++ drivers/gpu/drm/xe/xe_hw_engine_class_sysfs.c | 18 ------------------ 3 files changed, 20 insertions(+), 18 deletions(-) diff --git a/drivers/gpu/drm/xe/xe_hw_engine.c b/drivers/gpu/drm/xe/xe_hw_engine.c index 45f582a7caaa..520db5d518c3 100644 --- a/drivers/gpu/drm/xe/xe_hw_engine.c +++ b/drivers/gpu/drm/xe/xe_hw_engine.c @@ -1099,3 +1099,21 @@ bool xe_hw_engine_is_reserved(struct xe_hw_engine *hwe) return xe->info.has_usm && hwe->class == XE_ENGINE_CLASS_COPY && hwe->instance == gt->usm.reserved_bcs_instance; } + +const char *xe_hw_engine_class_to_str(enum xe_engine_class class) +{ + switch (class) { + case XE_ENGINE_CLASS_RENDER: + return "rcs"; + case XE_ENGINE_CLASS_VIDEO_DECODE: + return "vcs"; + case XE_ENGINE_CLASS_VIDEO_ENHANCE: + return "vecs"; + case XE_ENGINE_CLASS_COPY: + return "bcs"; + case XE_ENGINE_CLASS_COMPUTE: + return "ccs"; + default: + return NULL; + } +} diff --git a/drivers/gpu/drm/xe/xe_hw_engine.h b/drivers/gpu/drm/xe/xe_hw_engine.h index 71968ee2f600..843de159e47c 100644 --- a/drivers/gpu/drm/xe/xe_hw_engine.h +++ b/drivers/gpu/drm/xe/xe_hw_engine.h @@ -67,4 +67,6 @@ static inline bool xe_hw_engine_is_valid(struct xe_hw_engine *hwe) return hwe->name; } +const char *xe_hw_engine_class_to_str(enum xe_engine_class class); + #endif diff --git a/drivers/gpu/drm/xe/xe_hw_engine_class_sysfs.c b/drivers/gpu/drm/xe/xe_hw_engine_class_sysfs.c index 844ec68cbbb8..efce6c7dd2a2 100644 --- a/drivers/gpu/drm/xe/xe_hw_engine_class_sysfs.c +++ b/drivers/gpu/drm/xe/xe_hw_engine_class_sysfs.c @@ -618,24 +618,6 @@ static void hw_engine_class_sysfs_fini(struct drm_device *drm, void *arg) kobject_put(kobj); } -static const char *xe_hw_engine_class_to_str(enum xe_engine_class class) -{ - switch (class) { - case XE_ENGINE_CLASS_RENDER: - return "rcs"; - case XE_ENGINE_CLASS_VIDEO_DECODE: - return "vcs"; - case XE_ENGINE_CLASS_VIDEO_ENHANCE: - return "vecs"; - case XE_ENGINE_CLASS_COPY: - return "bcs"; - case XE_ENGINE_CLASS_COMPUTE: - return "ccs"; - default: - return NULL; - } -} - /** * xe_hw_engine_class_sysfs_init - Init HW engine classes on GT. * @gt: Xe GT. From patchwork Wed May 15 21:42:52 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Lucas De Marchi X-Patchwork-Id: 13665561 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from gabe.freedesktop.org (gabe.freedesktop.org [131.252.210.177]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 3AC04C25B79 for ; Wed, 15 May 2024 21:43:05 +0000 (UTC) Received: from gabe.freedesktop.org (localhost [127.0.0.1]) by gabe.freedesktop.org (Postfix) with ESMTP id 08BF410E340; Wed, 15 May 2024 21:43:00 +0000 (UTC) Authentication-Results: gabe.freedesktop.org; dkim=pass (2048-bit key; unprotected) header.d=intel.com header.i=@intel.com header.b="Mi9gPg0t"; dkim-atps=neutral Received: from mgamail.intel.com (mgamail.intel.com [192.198.163.19]) by gabe.freedesktop.org (Postfix) with ESMTPS id 4F67610E141; Wed, 15 May 2024 21:42:59 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1715809379; x=1747345379; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=+hyy5fOLL+nFN2OmZK8DUdooDecWrfu4D+Sq0mYZVl0=; b=Mi9gPg0tME85fncjisLfAYWQKlExbAlw/P9ly9F/gdOZDQoMkN6Mue/y EHaXpxHccEMQz1b+F0mfe9NVHpm+D3XKCaGb2MJr8L46OT4xmUXL1Qjxr LTUWdwxxyp/2fLzv1YiGWdvSNJ3Prv1uEDi8Hgn0U3DU/NMdFiKPM8dHj 44KN+4dQOLiuSGD3Why3agUEyjdDTMrvhZitx0aJ9xc7srpj01SIhp09Y dKzoa/w4H1d6N/O0ddJvsrQAqFwXh9cxelSiB2BNm4UszFjRyhkp1nZSf 8Pn9z9CvyZJIbsvqNigvaIiLhdMyPe776RKK1cmmRxjynSwbD8m0rxRb7 g==; X-CSE-ConnectionGUID: y71j+vqmTNi7eYcCUR+v7w== X-CSE-MsgGUID: MHuv2cAQQqimcVSyMlrJog== X-IronPort-AV: E=McAfee;i="6600,9927,11074"; a="11739161" X-IronPort-AV: E=Sophos;i="6.08,162,1712646000"; d="scan'208";a="11739161" Received: from fmviesa005.fm.intel.com ([10.60.135.145]) by fmvoesa113.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 15 May 2024 14:42:59 -0700 X-CSE-ConnectionGUID: k1Bmi3lZRZakgLt3LgSCtg== X-CSE-MsgGUID: Ruafe2N2QT2d93/MzJaRxA== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.08,162,1712646000"; d="scan'208";a="35651129" Received: from lucas-s2600cw.jf.intel.com ([10.165.21.196]) by fmviesa005-auth.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 15 May 2024 14:42:58 -0700 From: Lucas De Marchi To: Cc: Tvrtko Ursulin , Umesh Nerlige Ramappa , dri-devel@lists.freedesktop.org, Lucas De Marchi , Nirmoy Das Subject: [PATCH v4 2/8] drm/xe: Add XE_ENGINE_CLASS_OTHER to str conversion Date: Wed, 15 May 2024 14:42:52 -0700 Message-ID: <20240515214258.59209-3-lucas.demarchi@intel.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20240515214258.59209-1-lucas.demarchi@intel.com> References: <20240515214258.59209-1-lucas.demarchi@intel.com> MIME-Version: 1.0 X-BeenThere: dri-devel@lists.freedesktop.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Direct Rendering Infrastructure - Development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dri-devel-bounces@lists.freedesktop.org Sender: "dri-devel" XE_ENGINE_CLASS_OTHER was missing from the str conversion. Add it and remove the default handling so it's protected by -Wswitch. Currently the only user is xe_hw_engine_class_sysfs_init(), which already skips XE_ENGINE_CLASS_OTHER, so there's no change in behavior. Reviewed-by: Nirmoy Das Signed-off-by: Lucas De Marchi --- drivers/gpu/drm/xe/xe_hw_engine.c | 8 ++++++-- 1 file changed, 6 insertions(+), 2 deletions(-) diff --git a/drivers/gpu/drm/xe/xe_hw_engine.c b/drivers/gpu/drm/xe/xe_hw_engine.c index 520db5d518c3..f76bb507e1a5 100644 --- a/drivers/gpu/drm/xe/xe_hw_engine.c +++ b/drivers/gpu/drm/xe/xe_hw_engine.c @@ -1111,9 +1111,13 @@ const char *xe_hw_engine_class_to_str(enum xe_engine_class class) return "vecs"; case XE_ENGINE_CLASS_COPY: return "bcs"; + case XE_ENGINE_CLASS_OTHER: + return "other"; case XE_ENGINE_CLASS_COMPUTE: return "ccs"; - default: - return NULL; + case XE_ENGINE_CLASS_MAX: + break; } + + return NULL; } From patchwork Wed May 15 21:42:53 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Lucas De Marchi X-Patchwork-Id: 13665564 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from gabe.freedesktop.org (gabe.freedesktop.org [131.252.210.177]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id AE84CC25B75 for ; Wed, 15 May 2024 21:43:11 +0000 (UTC) Received: from gabe.freedesktop.org (localhost [127.0.0.1]) by gabe.freedesktop.org (Postfix) with ESMTP id 489D110EC17; Wed, 15 May 2024 21:43:01 +0000 (UTC) Authentication-Results: gabe.freedesktop.org; dkim=pass (2048-bit key; unprotected) header.d=intel.com header.i=@intel.com header.b="m2nbgnCT"; dkim-atps=neutral Received: from mgamail.intel.com (mgamail.intel.com [192.198.163.19]) by gabe.freedesktop.org (Postfix) with ESMTPS id AF78C10E267; Wed, 15 May 2024 21:42:59 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1715809380; x=1747345380; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=pHdJ8DpE5J2snIv+ddH+lWv1Pah7z5MpQ7nqg4bsPRU=; b=m2nbgnCTTN64mDrw1lQrybxUp+iQPr2pTYTD2eLJBsxKDwBLoGAuwpew qEl+1IpynWkQ7aUsxdkCKq7lxSoRUtnuWzN1PesTH51mq8sR0Cb2b2TeF aowo+PLAxW5kUIUej8pmNrrMwWZ4f3yLnS0BbilbgMwNPbnWr7Xk8kt+v SsPKluODin1BlP3jH56/9CKbRBX+pY/a+jJSR/kgSYlXXj8qRxzBMLKb/ efuL/W+5+aqI21+M8zIRok1J3w9gQB5UC0o7hxbgtKKwuM1/OkVDV7bKB ZixiITpAS3onlKd3xq8C3BPWQLy+y5KSRcc04Dv3Mv7xzmlaD95LQktds g==; X-CSE-ConnectionGUID: 3Gje12NDTuSi8pduzjJYFw== X-CSE-MsgGUID: hciz58YQRJ+03lxw20gb2g== X-IronPort-AV: E=McAfee;i="6600,9927,11074"; a="11739164" X-IronPort-AV: E=Sophos;i="6.08,162,1712646000"; d="scan'208";a="11739164" Received: from fmviesa005.fm.intel.com ([10.60.135.145]) by fmvoesa113.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 15 May 2024 14:42:59 -0700 X-CSE-ConnectionGUID: AIYkkAJtQny1Mf5CxnmwJg== X-CSE-MsgGUID: 42B23p8fStKn0k1CAgoBIg== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.08,162,1712646000"; d="scan'208";a="35651132" Received: from lucas-s2600cw.jf.intel.com ([10.165.21.196]) by fmviesa005-auth.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 15 May 2024 14:42:59 -0700 From: Lucas De Marchi To: Cc: Tvrtko Ursulin , Umesh Nerlige Ramappa , dri-devel@lists.freedesktop.org, Lucas De Marchi Subject: [PATCH v4 3/8] drm/xe/lrc: Add helper to capture context timestamp Date: Wed, 15 May 2024 14:42:53 -0700 Message-ID: <20240515214258.59209-4-lucas.demarchi@intel.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20240515214258.59209-1-lucas.demarchi@intel.com> References: <20240515214258.59209-1-lucas.demarchi@intel.com> MIME-Version: 1.0 X-BeenThere: dri-devel@lists.freedesktop.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Direct Rendering Infrastructure - Development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dri-devel-bounces@lists.freedesktop.org Sender: "dri-devel" From: Umesh Nerlige Ramappa Add a helper to capture CTX_TIMESTAMP from the context image so it can be used to calculate the runtime. v2: Add kernel-doc to clarify expectation from caller Signed-off-by: Umesh Nerlige Ramappa Signed-off-by: Lucas De Marchi Reviewed-by: Francois Dugast --- drivers/gpu/drm/xe/regs/xe_lrc_layout.h | 1 + drivers/gpu/drm/xe/xe_lrc.c | 12 ++++++++++++ drivers/gpu/drm/xe/xe_lrc.h | 14 ++++++++++++++ drivers/gpu/drm/xe/xe_lrc_types.h | 3 +++ 4 files changed, 30 insertions(+) diff --git a/drivers/gpu/drm/xe/regs/xe_lrc_layout.h b/drivers/gpu/drm/xe/regs/xe_lrc_layout.h index e6ca8bbda8f4..045dfd09db99 100644 --- a/drivers/gpu/drm/xe/regs/xe_lrc_layout.h +++ b/drivers/gpu/drm/xe/regs/xe_lrc_layout.h @@ -11,6 +11,7 @@ #define CTX_RING_TAIL (0x06 + 1) #define CTX_RING_START (0x08 + 1) #define CTX_RING_CTL (0x0a + 1) +#define CTX_TIMESTAMP (0x22 + 1) #define CTX_INDIRECT_RING_STATE (0x26 + 1) #define CTX_PDP0_UDW (0x30 + 1) #define CTX_PDP0_LDW (0x32 + 1) diff --git a/drivers/gpu/drm/xe/xe_lrc.c b/drivers/gpu/drm/xe/xe_lrc.c index 9b0a4078add3..f679cb9aaea7 100644 --- a/drivers/gpu/drm/xe/xe_lrc.c +++ b/drivers/gpu/drm/xe/xe_lrc.c @@ -844,6 +844,7 @@ int xe_lrc_init(struct xe_lrc *lrc, struct xe_hw_engine *hwe, lrc->tile = gt_to_tile(hwe->gt); lrc->ring.size = ring_size; lrc->ring.tail = 0; + lrc->ctx_timestamp = 0; xe_hw_fence_ctx_init(&lrc->fence_ctx, hwe->gt, hwe->fence_irq, hwe->name); @@ -898,6 +899,8 @@ int xe_lrc_init(struct xe_lrc *lrc, struct xe_hw_engine *hwe, RING_CTL_SIZE(lrc->ring.size) | RING_VALID); } + xe_lrc_write_ctx_reg(lrc, CTX_TIMESTAMP, 0); + if (xe->info.has_asid && vm) xe_lrc_write_ctx_reg(lrc, PVC_CTX_ASID, vm->usm.asid); @@ -1576,3 +1579,12 @@ void xe_lrc_snapshot_free(struct xe_lrc_snapshot *snapshot) xe_bo_put(snapshot->lrc_bo); kfree(snapshot); } + +u32 xe_lrc_update_timestamp(struct xe_lrc *lrc, u32 *old_ts) +{ + *old_ts = lrc->ctx_timestamp; + + lrc->ctx_timestamp = xe_lrc_read_ctx_reg(lrc, CTX_TIMESTAMP); + + return lrc->ctx_timestamp; +} diff --git a/drivers/gpu/drm/xe/xe_lrc.h b/drivers/gpu/drm/xe/xe_lrc.h index e0e841963c23..b9da1031083b 100644 --- a/drivers/gpu/drm/xe/xe_lrc.h +++ b/drivers/gpu/drm/xe/xe_lrc.h @@ -66,4 +66,18 @@ void xe_lrc_snapshot_capture_delayed(struct xe_lrc_snapshot *snapshot); void xe_lrc_snapshot_print(struct xe_lrc_snapshot *snapshot, struct drm_printer *p); void xe_lrc_snapshot_free(struct xe_lrc_snapshot *snapshot); +/** + * xe_lrc_update_timestamp - readout LRC timestamp and update cached value + * @lrc: logical ring context for this exec queue + * @old_ts: pointer where to save the previous timestamp + * + * Read the current timestamp for this LRC and update the cached value. The + * previous cached value is also returned in @old_ts so the caller can calculate + * the delta between 2 updates. Note that this is not intended to be called from + * any place, but just by the paths updating the drm client utilization. + * + * Returns the current LRC timestamp + */ +u32 xe_lrc_update_timestamp(struct xe_lrc *lrc, u32 *old_ts); + #endif diff --git a/drivers/gpu/drm/xe/xe_lrc_types.h b/drivers/gpu/drm/xe/xe_lrc_types.h index cdbf03faef15..0fa055da6b27 100644 --- a/drivers/gpu/drm/xe/xe_lrc_types.h +++ b/drivers/gpu/drm/xe/xe_lrc_types.h @@ -45,6 +45,9 @@ struct xe_lrc { /** @fence_ctx: context for hw fence */ struct xe_hw_fence_ctx fence_ctx; + + /** @ctx_timestamp: readout value of CTX_TIMESTAMP on last update */ + u32 ctx_timestamp; }; struct xe_lrc_snapshot; From patchwork Wed May 15 21:42:54 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Lucas De Marchi X-Patchwork-Id: 13665568 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from gabe.freedesktop.org (gabe.freedesktop.org [131.252.210.177]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id BC1F2C25B79 for ; Wed, 15 May 2024 21:43:16 +0000 (UTC) Received: from gabe.freedesktop.org (localhost [127.0.0.1]) by gabe.freedesktop.org (Postfix) with ESMTP id B982E10EC2B; Wed, 15 May 2024 21:43:02 +0000 (UTC) Authentication-Results: gabe.freedesktop.org; dkim=pass (2048-bit key; unprotected) header.d=intel.com header.i=@intel.com header.b="hySy1CGz"; dkim-atps=neutral Received: from mgamail.intel.com (mgamail.intel.com [192.198.163.19]) by gabe.freedesktop.org (Postfix) with ESMTPS id 88CAC10E1D6; Wed, 15 May 2024 21:42:59 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1715809380; x=1747345380; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=oNKLKwPHNRHrP3RgDBGmYROV3j8XFi8065ZP9okRlcE=; b=hySy1CGz04KRpdDRc9GqbKHe8TYCPqsP6A+240morRfsm1Ci78ihBecR AqrvDVEO+Js8opsMO5ksWZeJlUdvbOlQZ9/HWjBLpJkwZ0uYTF/ZyKEX1 3PgRPJKlRRBPAlgkW8cDSwakjGd0CjUra4ppeujuSihideBGPsuq2Z8+1 8UOA8Pm0mH6DiNKSH/UbFeq2vErfX7JzDhYI8x+MHUnZHyyVM9zvHw1fL /Uu6vJH2IwnhGU6kL38mSKcBkJt9EBRdycURcwlEOMHy4Bdz+kAMZPJvg /q7zu3beDsho21NkJVzHDjzM/C9BAvplymJTVa9qOif49VlEebnJ9uywe A==; X-CSE-ConnectionGUID: taoItlukRQuKdN+64FDZHA== X-CSE-MsgGUID: 8vXhGOEHRwWWCthtRrD+wQ== X-IronPort-AV: E=McAfee;i="6600,9927,11074"; a="11739166" X-IronPort-AV: E=Sophos;i="6.08,162,1712646000"; d="scan'208";a="11739166" Received: from fmviesa005.fm.intel.com ([10.60.135.145]) by fmvoesa113.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 15 May 2024 14:42:59 -0700 X-CSE-ConnectionGUID: C5oo3/AaQxmV4/wnFYSYNQ== X-CSE-MsgGUID: EruH1MH2Rs+3IPbrrNftwA== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.08,162,1712646000"; d="scan'208";a="35651135" Received: from lucas-s2600cw.jf.intel.com ([10.165.21.196]) by fmviesa005-auth.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 15 May 2024 14:42:59 -0700 From: Lucas De Marchi To: Cc: Tvrtko Ursulin , Umesh Nerlige Ramappa , dri-devel@lists.freedesktop.org, Lucas De Marchi Subject: [PATCH v4 4/8] drm/xe: Add helper to capture engine timestamp Date: Wed, 15 May 2024 14:42:54 -0700 Message-ID: <20240515214258.59209-5-lucas.demarchi@intel.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20240515214258.59209-1-lucas.demarchi@intel.com> References: <20240515214258.59209-1-lucas.demarchi@intel.com> MIME-Version: 1.0 X-BeenThere: dri-devel@lists.freedesktop.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Direct Rendering Infrastructure - Development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dri-devel-bounces@lists.freedesktop.org Sender: "dri-devel" Just like CTX_TIMESTAMP is used to calculate runtime, add a helper to get the timestamp for the engine so it can be used to calculate the "engine time" with the same unit as the runtime is recorded. Reviewed-by: Umesh Nerlige Ramappa Signed-off-by: Lucas De Marchi --- drivers/gpu/drm/xe/xe_hw_engine.c | 5 +++++ drivers/gpu/drm/xe/xe_hw_engine.h | 1 + 2 files changed, 6 insertions(+) diff --git a/drivers/gpu/drm/xe/xe_hw_engine.c b/drivers/gpu/drm/xe/xe_hw_engine.c index f76bb507e1a5..51536d05ad86 100644 --- a/drivers/gpu/drm/xe/xe_hw_engine.c +++ b/drivers/gpu/drm/xe/xe_hw_engine.c @@ -1121,3 +1121,8 @@ const char *xe_hw_engine_class_to_str(enum xe_engine_class class) return NULL; } + +u64 xe_hw_engine_read_timestamp(struct xe_hw_engine *hwe) +{ + return xe_mmio_read64_2x32(hwe->gt, RING_TIMESTAMP(hwe->mmio_base)); +} diff --git a/drivers/gpu/drm/xe/xe_hw_engine.h b/drivers/gpu/drm/xe/xe_hw_engine.h index 843de159e47c..7f2d27c0ba1a 100644 --- a/drivers/gpu/drm/xe/xe_hw_engine.h +++ b/drivers/gpu/drm/xe/xe_hw_engine.h @@ -68,5 +68,6 @@ static inline bool xe_hw_engine_is_valid(struct xe_hw_engine *hwe) } const char *xe_hw_engine_class_to_str(enum xe_engine_class class); +u64 xe_hw_engine_read_timestamp(struct xe_hw_engine *hwe); #endif From patchwork Wed May 15 21:42:55 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Lucas De Marchi X-Patchwork-Id: 13665562 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from gabe.freedesktop.org (gabe.freedesktop.org [131.252.210.177]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id D1AFAC25B75 for ; Wed, 15 May 2024 21:43:07 +0000 (UTC) Received: from gabe.freedesktop.org (localhost [127.0.0.1]) by gabe.freedesktop.org (Postfix) with ESMTP id 8E8EB10E3CA; Wed, 15 May 2024 21:43:00 +0000 (UTC) Authentication-Results: gabe.freedesktop.org; dkim=pass (2048-bit key; unprotected) header.d=intel.com header.i=@intel.com header.b="B48Jl9lx"; dkim-atps=neutral Received: from mgamail.intel.com (mgamail.intel.com [192.198.163.19]) by gabe.freedesktop.org (Postfix) with ESMTPS id 0509610E267; Wed, 15 May 2024 21:42:59 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1715809380; x=1747345380; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=II4IT1+MpR8Glvw/FHTDM6nfFh7W8K8VNJfz9XaJt2Q=; b=B48Jl9lxl5y1vw+YuO6cfBQW6DAlc2vN/ry3ymij7D3HIzCaG80bM3kN ftfU+K8/HjL5ZeNDEAIHiYxe084sOJi/8s3dBzs8tj5GFAhyVQxWD9Jve s5CY173J2Gl8o9PtGoMdEEOH52sSJA66TlPhWBplltwXeg7g4YaXFot58 mhNzON1TFpTDyE4/YHo6Gw9VeLvFqf0K2M6wXlJIXnW920njE4E6j5/O2 a2da5oMXfiROvh4oEvi293qMMZ4HYYIWSbZuf/u5dctCNccz1/upbHk6E cUKmTxV1RwHnvQiYMbfmorKp7EQCe1Co4KBwl4AaqMirjRWvizXm9srXY w==; X-CSE-ConnectionGUID: zka37fQ4R4ezbFXzg3Ih7g== X-CSE-MsgGUID: Z41JTPS6QVWS7VNfSZBn+A== X-IronPort-AV: E=McAfee;i="6600,9927,11074"; a="11739168" X-IronPort-AV: E=Sophos;i="6.08,162,1712646000"; d="scan'208";a="11739168" Received: from fmviesa005.fm.intel.com ([10.60.135.145]) by fmvoesa113.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 15 May 2024 14:43:00 -0700 X-CSE-ConnectionGUID: vqtSIWB+Q8uv8OvsUAVqNQ== X-CSE-MsgGUID: PNpb1aPlRpuenRqvyQLTiA== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.08,162,1712646000"; d="scan'208";a="35651138" Received: from lucas-s2600cw.jf.intel.com ([10.165.21.196]) by fmviesa005-auth.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 15 May 2024 14:42:59 -0700 From: Lucas De Marchi To: Cc: Tvrtko Ursulin , Umesh Nerlige Ramappa , dri-devel@lists.freedesktop.org, Lucas De Marchi Subject: [PATCH v4 5/8] drm/xe: Add helper to accumulate exec queue runtime Date: Wed, 15 May 2024 14:42:55 -0700 Message-ID: <20240515214258.59209-6-lucas.demarchi@intel.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20240515214258.59209-1-lucas.demarchi@intel.com> References: <20240515214258.59209-1-lucas.demarchi@intel.com> MIME-Version: 1.0 X-BeenThere: dri-devel@lists.freedesktop.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Direct Rendering Infrastructure - Development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dri-devel-bounces@lists.freedesktop.org Sender: "dri-devel" From: Umesh Nerlige Ramappa Add a helper to accumulate per-client runtime of all its exec queues. This is called every time a sched job is finished. v2: - Use guc_exec_queue_free_job() and execlist_job_free() to accumulate runtime when job is finished since xe_sched_job_completed() is not a notification that job finished. - Stop trying to update runtime from xe_exec_queue_fini() - that is redundant and may happen after xef is closed, leading to a use-after-free - Do not special case the first timestamp read: the default LRC sets CTX_TIMESTAMP to zero, so even the first sample should be a valid one. - Handle the parallel submission case by multiplying the runtime by width. v3: Update comments Signed-off-by: Umesh Nerlige Ramappa Signed-off-by: Lucas De Marchi --- drivers/gpu/drm/xe/xe_device_types.h | 3 +++ drivers/gpu/drm/xe/xe_exec_queue.c | 37 ++++++++++++++++++++++++++++ drivers/gpu/drm/xe/xe_exec_queue.h | 1 + drivers/gpu/drm/xe/xe_execlist.c | 1 + drivers/gpu/drm/xe/xe_guc_submit.c | 2 ++ 5 files changed, 44 insertions(+) diff --git a/drivers/gpu/drm/xe/xe_device_types.h b/drivers/gpu/drm/xe/xe_device_types.h index 5c5e36de452a..bc97990fd032 100644 --- a/drivers/gpu/drm/xe/xe_device_types.h +++ b/drivers/gpu/drm/xe/xe_device_types.h @@ -555,6 +555,9 @@ struct xe_file { struct mutex lock; } exec_queue; + /** @runtime: hw engine class runtime in ticks for this drm client */ + u64 runtime[XE_ENGINE_CLASS_MAX]; + /** @client: drm client */ struct xe_drm_client *client; }; diff --git a/drivers/gpu/drm/xe/xe_exec_queue.c b/drivers/gpu/drm/xe/xe_exec_queue.c index 395de93579fa..fa6dc996eca8 100644 --- a/drivers/gpu/drm/xe/xe_exec_queue.c +++ b/drivers/gpu/drm/xe/xe_exec_queue.c @@ -769,6 +769,43 @@ bool xe_exec_queue_is_idle(struct xe_exec_queue *q) q->lrc[0].fence_ctx.next_seqno - 1; } +/** + * xe_exec_queue_update_runtime() - Update runtime for this exec queue from hw + * @q: The exec queue + * + * Update the timestamp saved by HW for this exec queue and save runtime + * calculated by using the delta from last update. On multi-lrc case, only the + * first is considered. + */ +void xe_exec_queue_update_runtime(struct xe_exec_queue *q) +{ + struct xe_file *xef; + struct xe_lrc *lrc; + u32 old_ts, new_ts; + + /* + * Jobs that are run during driver load may use an exec_queue, but are + * not associated with a user xe file, so avoid accumulating busyness + * for kernel specific work. + */ + if (!q->vm || !q->vm->xef) + return; + + xef = q->vm->xef; + + /* + * Only sample the first LRC. For parallel submission, all of them are + * scheduled together and we compensate that below by multiplying by + * width - this may introduce errors if that premise is not true and + * they don't exit 100% aligned. On the other hand, looping through + * the LRCs and reading them in different time could also introduce + * errors. + */ + lrc = &q->lrc[0]; + new_ts = xe_lrc_update_timestamp(lrc, &old_ts); + xef->runtime[q->class] += (new_ts - old_ts) * q->width; +} + void xe_exec_queue_kill(struct xe_exec_queue *q) { struct xe_exec_queue *eq = q, *next; diff --git a/drivers/gpu/drm/xe/xe_exec_queue.h b/drivers/gpu/drm/xe/xe_exec_queue.h index 48f6da53a292..e0f07d28ee1a 100644 --- a/drivers/gpu/drm/xe/xe_exec_queue.h +++ b/drivers/gpu/drm/xe/xe_exec_queue.h @@ -75,5 +75,6 @@ struct dma_fence *xe_exec_queue_last_fence_get(struct xe_exec_queue *e, struct xe_vm *vm); void xe_exec_queue_last_fence_set(struct xe_exec_queue *e, struct xe_vm *vm, struct dma_fence *fence); +void xe_exec_queue_update_runtime(struct xe_exec_queue *q); #endif diff --git a/drivers/gpu/drm/xe/xe_execlist.c b/drivers/gpu/drm/xe/xe_execlist.c index e9dee1e14fef..bd7f27efe0e0 100644 --- a/drivers/gpu/drm/xe/xe_execlist.c +++ b/drivers/gpu/drm/xe/xe_execlist.c @@ -306,6 +306,7 @@ static void execlist_job_free(struct drm_sched_job *drm_job) { struct xe_sched_job *job = to_xe_sched_job(drm_job); + xe_exec_queue_update_runtime(job->q); xe_sched_job_put(job); } diff --git a/drivers/gpu/drm/xe/xe_guc_submit.c b/drivers/gpu/drm/xe/xe_guc_submit.c index 4efb88e3e056..ad2b8067d071 100644 --- a/drivers/gpu/drm/xe/xe_guc_submit.c +++ b/drivers/gpu/drm/xe/xe_guc_submit.c @@ -749,6 +749,8 @@ static void guc_exec_queue_free_job(struct drm_sched_job *drm_job) { struct xe_sched_job *job = to_xe_sched_job(drm_job); + xe_exec_queue_update_runtime(job->q); + trace_xe_sched_job_free(job); xe_sched_job_put(job); } From patchwork Wed May 15 21:42:56 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Lucas De Marchi X-Patchwork-Id: 13665565 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from gabe.freedesktop.org (gabe.freedesktop.org [131.252.210.177]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 2A991C25B78 for ; Wed, 15 May 2024 21:43:13 +0000 (UTC) Received: from gabe.freedesktop.org (localhost [127.0.0.1]) by gabe.freedesktop.org (Postfix) with ESMTP id 8FF6010EC1D; Wed, 15 May 2024 21:43:01 +0000 (UTC) Authentication-Results: gabe.freedesktop.org; dkim=pass (2048-bit key; unprotected) header.d=intel.com header.i=@intel.com header.b="U8JEOucQ"; dkim-atps=neutral Received: from mgamail.intel.com (mgamail.intel.com [192.198.163.19]) by gabe.freedesktop.org (Postfix) with ESMTPS id DD4F310E1D6; Wed, 15 May 2024 21:42:59 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1715809380; x=1747345380; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=GuECcZprAYcfXmWg6j7ESIi6mK+dOHDhpYaf0SgHn74=; b=U8JEOucQ3auVoQMClo0i0ebHReWFr4yOn5TnSVKSPVTlox7zdGP8T54v pBftRWB3eIEfU00/IwLmsicIG4HknK8faC0EHWG1LaXUIP14SgLq+cFb7 LHOC1Y+xUvULg1T/dsnAMZUQsohMDqTxYWZBlPwlGOXqm5LUj82OVEBWZ h6ORFJ7tCVpV+x4iFxGBUIP93ERwqK5gEF0zhdW8T5bt8RbXElPgGhkRW jasVkXRnJ2IMajnW/U+tBdggos0kcL92BfYS9DNovaT98qlPeX6v3+jHh og/eogqec9vNjbZ1JD4sUp7od+RmVFSn2q2FbOTrrti6Ovi4I1x+HVBGN g==; X-CSE-ConnectionGUID: WXcG1EeiSRynAQ8D4gL4CQ== X-CSE-MsgGUID: VbEpLwU3S3+tWInK2PfKqA== X-IronPort-AV: E=McAfee;i="6600,9927,11074"; a="11739170" X-IronPort-AV: E=Sophos;i="6.08,162,1712646000"; d="scan'208";a="11739170" Received: from fmviesa005.fm.intel.com ([10.60.135.145]) by fmvoesa113.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 15 May 2024 14:43:00 -0700 X-CSE-ConnectionGUID: yIWIzUU9SGWjxNH/AOqTJA== X-CSE-MsgGUID: X3OMBsEKSDSvjrMUSSHWVQ== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.08,162,1712646000"; d="scan'208";a="35651142" Received: from lucas-s2600cw.jf.intel.com ([10.165.21.196]) by fmviesa005-auth.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 15 May 2024 14:42:59 -0700 From: Lucas De Marchi To: Cc: Tvrtko Ursulin , Umesh Nerlige Ramappa , dri-devel@lists.freedesktop.org, Lucas De Marchi Subject: [PATCH v4 6/8] drm/xe: Cache data about user-visible engines Date: Wed, 15 May 2024 14:42:56 -0700 Message-ID: <20240515214258.59209-7-lucas.demarchi@intel.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20240515214258.59209-1-lucas.demarchi@intel.com> References: <20240515214258.59209-1-lucas.demarchi@intel.com> MIME-Version: 1.0 X-BeenThere: dri-devel@lists.freedesktop.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Direct Rendering Infrastructure - Development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dri-devel-bounces@lists.freedesktop.org Sender: "dri-devel" gt->info.engine_mask used to indicate the available engines, but that is not always true anymore: some engines are reserved to kernel and some may be exposed as a single engine (e.g. with ccs_mode). Runtime changes only happen when no clients exist, so it's safe to cache the list of engines in the gt and update that when it's needed. This will help implementing per client engine utilization so this (mostly constant) information doesn't need to be re-calculated on every query. Signed-off-by: Lucas De Marchi Reviewed-by: Jonathan Cavitt Reviewed-by: Umesh Nerlige Ramappa --- drivers/gpu/drm/xe/xe_gt.c | 23 +++++++++++++++++++++++ drivers/gpu/drm/xe/xe_gt.h | 13 +++++++++++++ drivers/gpu/drm/xe/xe_gt_ccs_mode.c | 1 + drivers/gpu/drm/xe/xe_gt_types.h | 21 ++++++++++++++++++++- 4 files changed, 57 insertions(+), 1 deletion(-) diff --git a/drivers/gpu/drm/xe/xe_gt.c b/drivers/gpu/drm/xe/xe_gt.c index e69a03ddd255..5194a3d38e76 100644 --- a/drivers/gpu/drm/xe/xe_gt.c +++ b/drivers/gpu/drm/xe/xe_gt.c @@ -560,9 +560,32 @@ int xe_gt_init(struct xe_gt *gt) if (err) return err; + xe_gt_record_user_engines(gt); + return drmm_add_action_or_reset(>_to_xe(gt)->drm, gt_fini, gt); } +void xe_gt_record_user_engines(struct xe_gt *gt) +{ + struct xe_hw_engine *hwe; + enum xe_hw_engine_id id; + + gt->user_engines.mask = 0; + memset(gt->user_engines.instances_per_class, 0, + sizeof(gt->user_engines.instances_per_class)); + + for_each_hw_engine(hwe, gt, id) { + if (xe_hw_engine_is_reserved(hwe)) + continue; + + gt->user_engines.mask |= BIT_ULL(id); + gt->user_engines.instances_per_class[hwe->class]++; + } + + xe_gt_assert(gt, (gt->user_engines.mask | gt->info.engine_mask) + == gt->info.engine_mask); +} + static int do_gt_reset(struct xe_gt *gt) { int err; diff --git a/drivers/gpu/drm/xe/xe_gt.h b/drivers/gpu/drm/xe/xe_gt.h index 8474c50b1b30..ad3fd31e0a41 100644 --- a/drivers/gpu/drm/xe/xe_gt.h +++ b/drivers/gpu/drm/xe/xe_gt.h @@ -38,6 +38,19 @@ int xe_gt_init_hwconfig(struct xe_gt *gt); int xe_gt_init_early(struct xe_gt *gt); int xe_gt_init(struct xe_gt *gt); int xe_gt_record_default_lrcs(struct xe_gt *gt); + +/** + * @xe_gt_record_user_engines - save data related to engines available to + * usersapce + * @gt: GT structure + * + * Walk the available HW engines from gt->info.engine_mask and calculate data + * related to those engines that may be used by userspace. To be used whenever + * available engines change in runtime (e.g. with ccs_mode) or during + * initialization + */ +void xe_gt_record_user_engines(struct xe_gt *gt); + void xe_gt_suspend_prepare(struct xe_gt *gt); int xe_gt_suspend(struct xe_gt *gt); int xe_gt_resume(struct xe_gt *gt); diff --git a/drivers/gpu/drm/xe/xe_gt_ccs_mode.c b/drivers/gpu/drm/xe/xe_gt_ccs_mode.c index a34c9a24dafc..c36218f4f6c8 100644 --- a/drivers/gpu/drm/xe/xe_gt_ccs_mode.c +++ b/drivers/gpu/drm/xe/xe_gt_ccs_mode.c @@ -134,6 +134,7 @@ ccs_mode_store(struct device *kdev, struct device_attribute *attr, if (gt->ccs_mode != num_engines) { xe_gt_info(gt, "Setting compute mode to %d\n", num_engines); gt->ccs_mode = num_engines; + xe_gt_record_user_engines(gt); xe_gt_reset_async(gt); } diff --git a/drivers/gpu/drm/xe/xe_gt_types.h b/drivers/gpu/drm/xe/xe_gt_types.h index 5a114fc9dde7..aaf2951749a6 100644 --- a/drivers/gpu/drm/xe/xe_gt_types.h +++ b/drivers/gpu/drm/xe/xe_gt_types.h @@ -112,7 +112,11 @@ struct xe_gt { enum xe_gt_type type; /** @info.reference_clock: clock frequency */ u32 reference_clock; - /** @info.engine_mask: mask of engines present on GT */ + /** + * @info.engine_mask: mask of engines present on GT. Some of + * them may be reserved in runtime and not available for user. + * See @user_engines.mask + */ u64 engine_mask; /** @info.gmdid: raw GMD_ID value from hardware */ u32 gmdid; @@ -365,6 +369,21 @@ struct xe_gt { /** @wa_active.oob: bitmap with active OOB workaroudns */ unsigned long *oob; } wa_active; + + /** @user_engines: engines present in GT and available to userspace */ + struct { + /** + * @mask: like @info->engine_mask, but take in consideration + * only engines available to userspace + */ + u64 mask; + + /** + * @instances_per_class: aggregate per class the number of + * engines available to userspace + */ + u8 instances_per_class[XE_ENGINE_CLASS_MAX]; + } user_engines; }; #endif From patchwork Wed May 15 21:42:57 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Lucas De Marchi X-Patchwork-Id: 13665567 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from gabe.freedesktop.org (gabe.freedesktop.org [131.252.210.177]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id ADD1DC25B78 for ; Wed, 15 May 2024 21:43:15 +0000 (UTC) Received: from gabe.freedesktop.org (localhost [127.0.0.1]) by gabe.freedesktop.org (Postfix) with ESMTP id 2C47B10EC27; Wed, 15 May 2024 21:43:02 +0000 (UTC) Authentication-Results: gabe.freedesktop.org; dkim=pass (2048-bit key; unprotected) header.d=intel.com header.i=@intel.com header.b="G+yjjgXW"; dkim-atps=neutral Received: from mgamail.intel.com (mgamail.intel.com [192.198.163.19]) by gabe.freedesktop.org (Postfix) with ESMTPS id 1243510E3CA; Wed, 15 May 2024 21:43:00 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1715809380; x=1747345380; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=q4BKRUonUrFhgjMaw6tHgpgzacmRW8CrOqMwqtPHYyY=; b=G+yjjgXWb/E/GsWod/UB0e61XWXTUwCDRiHvXDXkeW2/tdHptk/XyMIz rg3Aj9kAlbESKYdrcc6/evnxh1nt2OwyEjcv3v6fPSMO0ZsxZ+N/F4UTb f96YoT8tB/IA3itQRSLUzfoxqdR+HUkTCn3WW9OyBYE69yOEOVDaQu7vV tpybgii3gmfPDD3WD8WqFnQ4HAoztNFGm8Z1lY/5WmQAqtHiz+Q9JfUrD OgNkCOVCcaxyWKFWX+EHyv9F3LSPGcH2tWRidQayrS/s3wxaPRyprI6Rw /2Yy91NP161bRn5L2vgCsV1XHFwFEoLOQmRPp5lmgm8sM2dokZw8oOMXl g==; X-CSE-ConnectionGUID: UCEEsVeuTaCOH7m6vJOPyg== X-CSE-MsgGUID: NvK2ElagRDuUze/AbdvmDg== X-IronPort-AV: E=McAfee;i="6600,9927,11074"; a="11739172" X-IronPort-AV: E=Sophos;i="6.08,162,1712646000"; d="scan'208";a="11739172" Received: from fmviesa005.fm.intel.com ([10.60.135.145]) by fmvoesa113.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 15 May 2024 14:43:00 -0700 X-CSE-ConnectionGUID: 5qAc8ja5QGK79U4dWZGqSA== X-CSE-MsgGUID: JmEGZmPCS96gRCrnrwGDEw== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.08,162,1712646000"; d="scan'208";a="35651145" Received: from lucas-s2600cw.jf.intel.com ([10.165.21.196]) by fmviesa005-auth.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 15 May 2024 14:42:59 -0700 From: Lucas De Marchi To: Cc: Tvrtko Ursulin , Umesh Nerlige Ramappa , dri-devel@lists.freedesktop.org, Lucas De Marchi Subject: [PATCH v4 7/8] drm/xe: Add helper to return any available hw engine Date: Wed, 15 May 2024 14:42:57 -0700 Message-ID: <20240515214258.59209-8-lucas.demarchi@intel.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20240515214258.59209-1-lucas.demarchi@intel.com> References: <20240515214258.59209-1-lucas.demarchi@intel.com> MIME-Version: 1.0 X-BeenThere: dri-devel@lists.freedesktop.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Direct Rendering Infrastructure - Development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dri-devel-bounces@lists.freedesktop.org Sender: "dri-devel" Get the first available engine from a gt, which helps in the case any engine serves as a context, like when reading RING_TIMESTAMP. Signed-off-by: Lucas De Marchi Reviewed-by: Umesh Nerlige Ramappa --- drivers/gpu/drm/xe/xe_gt.c | 11 +++++++++++ drivers/gpu/drm/xe/xe_gt.h | 7 +++++++ 2 files changed, 18 insertions(+) diff --git a/drivers/gpu/drm/xe/xe_gt.c b/drivers/gpu/drm/xe/xe_gt.c index 5194a3d38e76..3432fef56486 100644 --- a/drivers/gpu/drm/xe/xe_gt.c +++ b/drivers/gpu/drm/xe/xe_gt.c @@ -833,3 +833,14 @@ struct xe_hw_engine *xe_gt_any_hw_engine_by_reset_domain(struct xe_gt *gt, return NULL; } + +struct xe_hw_engine *xe_gt_any_hw_engine(struct xe_gt *gt) +{ + struct xe_hw_engine *hwe; + enum xe_hw_engine_id id; + + for_each_hw_engine(hwe, gt, id) + return hwe; + + return NULL; +} diff --git a/drivers/gpu/drm/xe/xe_gt.h b/drivers/gpu/drm/xe/xe_gt.h index ad3fd31e0a41..a53f01362d94 100644 --- a/drivers/gpu/drm/xe/xe_gt.h +++ b/drivers/gpu/drm/xe/xe_gt.h @@ -67,6 +67,13 @@ void xe_gt_remove(struct xe_gt *gt); struct xe_hw_engine * xe_gt_any_hw_engine_by_reset_domain(struct xe_gt *gt, enum xe_engine_class class); +/** + * xe_gt_any_hw_engine - scan the list of engines and return the + * first available + * @gt: GT structure + */ +struct xe_hw_engine *xe_gt_any_hw_engine(struct xe_gt *gt); + struct xe_hw_engine *xe_gt_hw_engine(struct xe_gt *gt, enum xe_engine_class class, u16 instance, From patchwork Wed May 15 21:42:58 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Lucas De Marchi X-Patchwork-Id: 13665566 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from gabe.freedesktop.org (gabe.freedesktop.org [131.252.210.177]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 7E90CC25B75 for ; Wed, 15 May 2024 21:43:14 +0000 (UTC) Received: from gabe.freedesktop.org (localhost [127.0.0.1]) by gabe.freedesktop.org (Postfix) with ESMTP id 15F0510EC26; Wed, 15 May 2024 21:43:02 +0000 (UTC) Authentication-Results: gabe.freedesktop.org; dkim=pass (2048-bit key; unprotected) header.d=intel.com header.i=@intel.com header.b="Ib04uVL0"; dkim-atps=neutral Received: from mgamail.intel.com (mgamail.intel.com [192.198.163.19]) by gabe.freedesktop.org (Postfix) with ESMTPS id 3FC0F10E5FA; Wed, 15 May 2024 21:43:00 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1715809380; x=1747345380; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=R8TGBAf6rYvtQ6MUu1oPU6n9HbqRYJfnV2otXAU+A/o=; b=Ib04uVL0MdHXAo/KJSMffuUdpAbektgIu8PssxnCgvTd/JMQPISZhYam dekKsKCNkRdZeCaUpQBG211GJu2TDqQgzsbawgZsPNxtvOwISfwqJn0Q2 6mfkscp4kiGyosIFDenC47+Yg6Q78kZjsk9yuW4zAU2ozTKwcL8LgwL8X jWg95Q0/zdJHgRfnZzt4i4adipFz2kU5ZsHJZ3k3JyANtD4K0uo/RKE8w f652liXsklLqlHiromR+VJQN79WpamlPRSrv39UC4Fwmizp5nwiQuwo99 GScsRk3WjPi1QhFANDS6sGIsFWe9IRdLRQkeki1UIpecS3fXL2FUj2hp9 g==; X-CSE-ConnectionGUID: wYUyBdumSWORVASSC9rckQ== X-CSE-MsgGUID: VeGvjB0qSAqpPhtGke0Ujg== X-IronPort-AV: E=McAfee;i="6600,9927,11074"; a="11739174" X-IronPort-AV: E=Sophos;i="6.08,162,1712646000"; d="scan'208";a="11739174" Received: from fmviesa005.fm.intel.com ([10.60.135.145]) by fmvoesa113.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 15 May 2024 14:43:00 -0700 X-CSE-ConnectionGUID: hcovAz3VTUKWDZQ9eprTjw== X-CSE-MsgGUID: R30FVeSWQr2TNS/OdUQeMQ== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.08,162,1712646000"; d="scan'208";a="35651150" Received: from lucas-s2600cw.jf.intel.com ([10.165.21.196]) by fmviesa005-auth.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 15 May 2024 14:43:00 -0700 From: Lucas De Marchi To: Cc: Tvrtko Ursulin , Umesh Nerlige Ramappa , dri-devel@lists.freedesktop.org, Lucas De Marchi Subject: [PATCH v4 8/8] drm/xe/client: Print runtime to fdinfo Date: Wed, 15 May 2024 14:42:58 -0700 Message-ID: <20240515214258.59209-9-lucas.demarchi@intel.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20240515214258.59209-1-lucas.demarchi@intel.com> References: <20240515214258.59209-1-lucas.demarchi@intel.com> MIME-Version: 1.0 X-BeenThere: dri-devel@lists.freedesktop.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Direct Rendering Infrastructure - Development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dri-devel-bounces@lists.freedesktop.org Sender: "dri-devel" Print the accumulated runtime for client when printing fdinfo. Each time a query is done it first does 2 things: 1) loop through all the exec queues for the current client and accumulate the runtime, per engine class. CTX_TIMESTAMP is used for that, being read from the context image. 2) Read a "GPU timestamp" that can be used for considering "how much GPU time has passed" and that has the same unit/refclock as the one recording the runtime. RING_TIMESTAMP is used for that via MMIO. Since for all current platforms RING_TIMESTAMP follows the same refclock, just read it once, using any first engine available. This is exported to userspace as 2 numbers in fdinfo: drm-cycles-: drm-total-cycles-: Userspace is expected to collect at least 2 samples, which allows to know the client engine busyness as per: RUNTIME1 - RUNTIME0 busyness = --------------------- T1 - T0 Since drm-cycles- always starts at 0, it's also possible to know if and engine was ever used by a client. It's expected that userspace will read any 2 samples every few seconds. Given the update frequency of the counters involved and that CTX_TIMESTAMP is 32-bits, the counter for each exec_queue can wrap around (assuming 100% utilization) after ~200s. The wraparound is not perceived by userspace since it's just accumulated for all the exec_queues in a 64-bit counter) but the measurement will not be accurate if the samples are too far apart. This could be mitigated by adding a workqueue to accumulate the counters every so often, but it's additional complexity for something that is done already by userspace every few seconds in tools like gputop (from igt), htop, nvtop, etc, with none of them really defaulting to 1 sample per minute or more. Signed-off-by: Lucas De Marchi Acked-by: Tvrtko Ursulin Reviewed-by: Umesh Nerlige Ramappa --- Documentation/gpu/drm-usage-stats.rst | 21 +++- Documentation/gpu/xe/index.rst | 1 + Documentation/gpu/xe/xe-drm-usage-stats.rst | 10 ++ drivers/gpu/drm/xe/xe_drm_client.c | 121 +++++++++++++++++++- 4 files changed, 150 insertions(+), 3 deletions(-) create mode 100644 Documentation/gpu/xe/xe-drm-usage-stats.rst diff --git a/Documentation/gpu/drm-usage-stats.rst b/Documentation/gpu/drm-usage-stats.rst index 6dc299343b48..a80f95ca1b2f 100644 --- a/Documentation/gpu/drm-usage-stats.rst +++ b/Documentation/gpu/drm-usage-stats.rst @@ -112,6 +112,19 @@ larger value within a reasonable period. Upon observing a value lower than what was previously read, userspace is expected to stay with that larger previous value until a monotonic update is seen. +- drm-total-cycles-: + +Engine identifier string must be the same as the one specified in the +drm-cycles- tag and shall contain the total number cycles for the given +engine. + +This is a timestamp in GPU unspecified unit that matches the update rate +of drm-cycles-. For drivers that implement this interface, the engine +utilization can be calculated entirely on the GPU clock domain, without +considering the CPU sleep time between 2 samples. + +A driver may implement either this key or drm-maxfreq-, but not both. + - drm-maxfreq-: [Hz|MHz|KHz] Engine identifier string must be the same as the one specified in the @@ -121,6 +134,9 @@ percentage utilization of the engine, whereas drm-engine- only reflects time active without considering what frequency the engine is operating as a percentage of its maximum frequency. +A driver may implement either this key or drm-total-cycles-, but not +both. + Memory ^^^^^^ @@ -168,5 +184,6 @@ be documented above and where possible, aligned with other drivers. Driver specific implementations ------------------------------- -:ref:`i915-usage-stats` -:ref:`panfrost-usage-stats` +* :ref:`i915-usage-stats` +* :ref:`panfrost-usage-stats` +* :ref:`xe-usage-stats` diff --git a/Documentation/gpu/xe/index.rst b/Documentation/gpu/xe/index.rst index c224ecaee81e..3f07aa3b5432 100644 --- a/Documentation/gpu/xe/index.rst +++ b/Documentation/gpu/xe/index.rst @@ -23,3 +23,4 @@ DG2, etc is provided to prototype the driver. xe_firmware xe_tile xe_debugging + xe-drm-usage-stats.rst diff --git a/Documentation/gpu/xe/xe-drm-usage-stats.rst b/Documentation/gpu/xe/xe-drm-usage-stats.rst new file mode 100644 index 000000000000..482d503ae68a --- /dev/null +++ b/Documentation/gpu/xe/xe-drm-usage-stats.rst @@ -0,0 +1,10 @@ +.. SPDX-License-Identifier: GPL-2.0+ + +.. _xe-usage-stats: + +======================================== +Xe DRM client usage stats implementation +======================================== + +.. kernel-doc:: drivers/gpu/drm/xe/xe_drm_client.c + :doc: DRM Client usage stats diff --git a/drivers/gpu/drm/xe/xe_drm_client.c b/drivers/gpu/drm/xe/xe_drm_client.c index 08f0b7c95901..952b0cc87708 100644 --- a/drivers/gpu/drm/xe/xe_drm_client.c +++ b/drivers/gpu/drm/xe/xe_drm_client.c @@ -2,6 +2,7 @@ /* * Copyright © 2023 Intel Corporation */ +#include "xe_drm_client.h" #include #include @@ -12,9 +13,66 @@ #include "xe_bo.h" #include "xe_bo_types.h" #include "xe_device_types.h" -#include "xe_drm_client.h" +#include "xe_exec_queue.h" +#include "xe_force_wake.h" +#include "xe_gt.h" +#include "xe_hw_engine.h" +#include "xe_pm.h" #include "xe_trace.h" +/** + * DOC: DRM Client usage stats + * + * The drm/xe driver implements the DRM client usage stats specification as + * documented in :ref:`drm-client-usage-stats`. + * + * Example of the output showing the implemented key value pairs and entirety of + * the currently possible format options: + * + * :: + * + * pos: 0 + * flags: 0100002 + * mnt_id: 26 + * ino: 685 + * drm-driver: xe + * drm-client-id: 3 + * drm-pdev: 0000:03:00.0 + * drm-total-system: 0 + * drm-shared-system: 0 + * drm-active-system: 0 + * drm-resident-system: 0 + * drm-purgeable-system: 0 + * drm-total-gtt: 192 KiB + * drm-shared-gtt: 0 + * drm-active-gtt: 0 + * drm-resident-gtt: 192 KiB + * drm-total-vram0: 23992 KiB + * drm-shared-vram0: 16 MiB + * drm-active-vram0: 0 + * drm-resident-vram0: 23992 KiB + * drm-total-stolen: 0 + * drm-shared-stolen: 0 + * drm-active-stolen: 0 + * drm-resident-stolen: 0 + * drm-cycles-rcs: 28257900 + * drm-total-cycles-rcs: 7655183225 + * drm-cycles-bcs: 0 + * drm-total-cycles-bcs: 7655183225 + * drm-cycles-vcs: 0 + * drm-total-cycles-vcs: 7655183225 + * drm-engine-capacity-vcs: 2 + * drm-cycles-vecs: 0 + * drm-total-cycles-vecs: 7655183225 + * drm-engine-capacity-vecs: 2 + * drm-cycles-ccs: 0 + * drm-total-cycles-ccs: 7655183225 + * drm-engine-capacity-ccs: 4 + * + * Possible `drm-cycles-` key names are: `rcs`, `ccs`, `bcs`, `vcs`, `vecs` and + * "other". + */ + /** * xe_drm_client_alloc() - Allocate drm client * @void: No arg @@ -179,6 +237,66 @@ static void show_meminfo(struct drm_printer *p, struct drm_file *file) } } +static void show_runtime(struct drm_printer *p, struct drm_file *file) +{ + unsigned long class, i, gt_id, capacity[XE_ENGINE_CLASS_MAX] = { }; + struct xe_file *xef = file->driver_priv; + struct xe_device *xe = xef->xe; + struct xe_gt *gt; + struct xe_hw_engine *hwe; + struct xe_exec_queue *q; + u64 gpu_timestamp; + + xe_pm_runtime_get(xe); + + /* Accumulate all the exec queues from this client */ + mutex_lock(&xef->exec_queue.lock); + xa_for_each(&xef->exec_queue.xa, i, q) + xe_exec_queue_update_runtime(q); + mutex_unlock(&xef->exec_queue.lock); + + /* Get the total GPU cycles */ + for_each_gt(gt, xe, gt_id) { + hwe = xe_gt_any_hw_engine(gt); + if (!hwe) + continue; + + xe_force_wake_get(gt_to_fw(gt), XE_FW_GT); + gpu_timestamp = xe_hw_engine_read_timestamp(hwe); + xe_force_wake_put(gt_to_fw(gt), XE_FW_GT); + break; + } + + if (unlikely(!hwe)) + return; + + for (class = 0; class < XE_ENGINE_CLASS_MAX; class++) { + const char *class_name; + + for_each_gt(gt, xe, gt_id) + capacity[class] += gt->user_engines.instances_per_class[class]; + + /* + * Engines may be fused off or not exposed to userspace. Don't + * return anything if this entire class is not available + */ + if (!capacity[class]) + continue; + + class_name = xe_hw_engine_class_to_str(class); + drm_printf(p, "drm-cycles-%s:\t%llu\n", + class_name, xef->runtime[class]); + drm_printf(p, "drm-total-cycles-%s:\t%llu\n", + class_name, gpu_timestamp); + + if (capacity[class] > 1) + drm_printf(p, "drm-engine-capacity-%s:\t%lu\n", + class_name, capacity[class]); + } + + xe_pm_runtime_put(xe); +} + /** * xe_drm_client_fdinfo() - Callback for fdinfo interface * @p: The drm_printer ptr @@ -192,5 +310,6 @@ static void show_meminfo(struct drm_printer *p, struct drm_file *file) void xe_drm_client_fdinfo(struct drm_printer *p, struct drm_file *file) { show_meminfo(p, file); + show_runtime(p, file); } #endif