From patchwork Fri May 17 20:43:03 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Lucas De Marchi X-Patchwork-Id: 13667379 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from gabe.freedesktop.org (gabe.freedesktop.org [131.252.210.177]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 4825FC25B7A for ; Fri, 17 May 2024 20:43:20 +0000 (UTC) Received: from gabe.freedesktop.org (localhost [127.0.0.1]) by gabe.freedesktop.org (Postfix) with ESMTP id F2B5510EF78; Fri, 17 May 2024 20:43:11 +0000 (UTC) Authentication-Results: gabe.freedesktop.org; dkim=pass (2048-bit key; unprotected) header.d=intel.com header.i=@intel.com header.b="NqrSrOR3"; dkim-atps=neutral Received: from mgamail.intel.com (mgamail.intel.com [198.175.65.19]) by gabe.freedesktop.org (Postfix) with ESMTPS id 7FFC510E18D; Fri, 17 May 2024 20:43:10 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1715978590; x=1747514590; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=GTfAKeDB2dopS7Vzx9gfOKKkLX5ydQj9w0nD3eGwWvQ=; b=NqrSrOR3NJ0wGI39UMma5w75LhMDv0XE95kCovX/lbInVF3w3BhssTpd vaF/NkqZFzzcHPXHjjM5XWsr/+TX0bDBaNepvVpPZbLo5ru9/8NhSWseZ UxY43ujOa8wTjlsekFK9VMpQLsiQ45CLnZKw7Mys3EOxLoEcpr9OJJpuS XI6Tm6JOfoC3XHa3VsiMaLfnFe2fDsTe3Xs6d3sppXFRFSPN709hgSBMo +U6JTy8+zhZNGvTnL5bIq+ojlrMiZkNkg2y+SyxrKiID9HX7j47CTqD3q FXJH4elFr3am9Bzt2fHmcoDmmVkpmhx6oTAZEInBglthWK8mfzd++u5zS w==; X-CSE-ConnectionGUID: 5oGl+eMaQ662v6TpI5NZ8g== X-CSE-MsgGUID: vjomsRtfRjOdjyfTHYorqA== X-IronPort-AV: E=McAfee;i="6600,9927,11075"; a="12027911" X-IronPort-AV: E=Sophos;i="6.08,168,1712646000"; d="scan'208";a="12027911" Received: from fmviesa002.fm.intel.com ([10.60.135.142]) by orvoesa111.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 17 May 2024 13:43:09 -0700 X-CSE-ConnectionGUID: DaExsfUvSDKBPpxbrieECA== X-CSE-MsgGUID: CAQI1GcrTHaU7I764xr/3w== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.08,168,1712646000"; d="scan'208";a="55125236" Received: from lucas-s2600cw.jf.intel.com ([10.165.21.196]) by fmviesa002-auth.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 17 May 2024 13:43:09 -0700 From: Lucas De Marchi To: Cc: Tvrtko Ursulin , Umesh Nerlige Ramappa , dri-devel@lists.freedesktop.org, Lucas De Marchi , Nirmoy Das , Oak Zeng Subject: [PATCH v5 1/8] drm/xe: Promote xe_hw_engine_class_to_str() Date: Fri, 17 May 2024 13:43:03 -0700 Message-ID: <20240517204310.88854-2-lucas.demarchi@intel.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20240517204310.88854-1-lucas.demarchi@intel.com> References: <20240517204310.88854-1-lucas.demarchi@intel.com> MIME-Version: 1.0 X-BeenThere: dri-devel@lists.freedesktop.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Direct Rendering Infrastructure - Development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dri-devel-bounces@lists.freedesktop.org Sender: "dri-devel" Move it out of the sysfs compilation unit so it can be re-used in other places. Reviewed-by: Nirmoy Das Reviewed-by: Oak Zeng Signed-off-by: Lucas De Marchi --- drivers/gpu/drm/xe/xe_hw_engine.c | 18 ++++++++++++++++++ drivers/gpu/drm/xe/xe_hw_engine.h | 2 ++ drivers/gpu/drm/xe/xe_hw_engine_class_sysfs.c | 18 ------------------ 3 files changed, 20 insertions(+), 18 deletions(-) diff --git a/drivers/gpu/drm/xe/xe_hw_engine.c b/drivers/gpu/drm/xe/xe_hw_engine.c index e19af179af33..b71e90c555fa 100644 --- a/drivers/gpu/drm/xe/xe_hw_engine.c +++ b/drivers/gpu/drm/xe/xe_hw_engine.c @@ -1099,3 +1099,21 @@ bool xe_hw_engine_is_reserved(struct xe_hw_engine *hwe) return xe->info.has_usm && hwe->class == XE_ENGINE_CLASS_COPY && hwe->instance == gt->usm.reserved_bcs_instance; } + +const char *xe_hw_engine_class_to_str(enum xe_engine_class class) +{ + switch (class) { + case XE_ENGINE_CLASS_RENDER: + return "rcs"; + case XE_ENGINE_CLASS_VIDEO_DECODE: + return "vcs"; + case XE_ENGINE_CLASS_VIDEO_ENHANCE: + return "vecs"; + case XE_ENGINE_CLASS_COPY: + return "bcs"; + case XE_ENGINE_CLASS_COMPUTE: + return "ccs"; + default: + return NULL; + } +} diff --git a/drivers/gpu/drm/xe/xe_hw_engine.h b/drivers/gpu/drm/xe/xe_hw_engine.h index 71968ee2f600..843de159e47c 100644 --- a/drivers/gpu/drm/xe/xe_hw_engine.h +++ b/drivers/gpu/drm/xe/xe_hw_engine.h @@ -67,4 +67,6 @@ static inline bool xe_hw_engine_is_valid(struct xe_hw_engine *hwe) return hwe->name; } +const char *xe_hw_engine_class_to_str(enum xe_engine_class class); + #endif diff --git a/drivers/gpu/drm/xe/xe_hw_engine_class_sysfs.c b/drivers/gpu/drm/xe/xe_hw_engine_class_sysfs.c index 844ec68cbbb8..efce6c7dd2a2 100644 --- a/drivers/gpu/drm/xe/xe_hw_engine_class_sysfs.c +++ b/drivers/gpu/drm/xe/xe_hw_engine_class_sysfs.c @@ -618,24 +618,6 @@ static void hw_engine_class_sysfs_fini(struct drm_device *drm, void *arg) kobject_put(kobj); } -static const char *xe_hw_engine_class_to_str(enum xe_engine_class class) -{ - switch (class) { - case XE_ENGINE_CLASS_RENDER: - return "rcs"; - case XE_ENGINE_CLASS_VIDEO_DECODE: - return "vcs"; - case XE_ENGINE_CLASS_VIDEO_ENHANCE: - return "vecs"; - case XE_ENGINE_CLASS_COPY: - return "bcs"; - case XE_ENGINE_CLASS_COMPUTE: - return "ccs"; - default: - return NULL; - } -} - /** * xe_hw_engine_class_sysfs_init - Init HW engine classes on GT. * @gt: Xe GT. From patchwork Fri May 17 20:43:04 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Lucas De Marchi X-Patchwork-Id: 13667380 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from gabe.freedesktop.org (gabe.freedesktop.org [131.252.210.177]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 4491BC25B77 for ; Fri, 17 May 2024 20:43:22 +0000 (UTC) Received: from gabe.freedesktop.org (localhost [127.0.0.1]) by gabe.freedesktop.org (Postfix) with ESMTP id 198C910EF81; Fri, 17 May 2024 20:43:13 +0000 (UTC) Authentication-Results: gabe.freedesktop.org; dkim=pass (2048-bit key; unprotected) header.d=intel.com header.i=@intel.com header.b="YhFtrV5L"; dkim-atps=neutral Received: from mgamail.intel.com (mgamail.intel.com [198.175.65.19]) by gabe.freedesktop.org (Postfix) with ESMTPS id 98E2610EF6C; Fri, 17 May 2024 20:43:10 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1715978591; x=1747514591; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=SMHWCly7W2RkMZdXU4UyXZ1uNidTSyTrfZkvJs2smdY=; b=YhFtrV5LSE9miDndNzJcmHkjbTh/8YpvoUSCR6iqq1bXu1VdlDvQqMhv QovMdU4RbpjvPaj8xNN6rWVbUt0w778pdpDTjePjof2mUndNzYXQ/NA32 Xebn7MJ6CKOz4SFgB6s/Hrfs1bcFnvkozzUhzZyH3RSzMQIweiiQZWkmE exwHRad9F7fXlyAUVO5jhSKxf8A9DYb4Tf+TSrJHymCVWhuvMsDoOolGu v2ZBWNStpuwUquL+a5TCzNc5IWDsGBLFtZ6Zb51D5y3Enqy8oYclcC6aQ K33Ch2qwLiJ7EmMzZqGn34CpjCSza+Z55rDnWoorCACfBXkOAolt/vBC5 g==; X-CSE-ConnectionGUID: 4xBNNNLYTom3e2LknGyvKg== X-CSE-MsgGUID: 0znQZ7I0QB2piBo26pZ12Q== X-IronPort-AV: E=McAfee;i="6600,9927,11075"; a="12027914" X-IronPort-AV: E=Sophos;i="6.08,168,1712646000"; d="scan'208";a="12027914" Received: from fmviesa002.fm.intel.com ([10.60.135.142]) by orvoesa111.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 17 May 2024 13:43:10 -0700 X-CSE-ConnectionGUID: OjqRe3ApTKGVzJLmLk40Uw== X-CSE-MsgGUID: 1ZdkEI7rTrywmBDI8ocu5w== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.08,168,1712646000"; d="scan'208";a="55125239" Received: from lucas-s2600cw.jf.intel.com ([10.165.21.196]) by fmviesa002-auth.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 17 May 2024 13:43:09 -0700 From: Lucas De Marchi To: Cc: Tvrtko Ursulin , Umesh Nerlige Ramappa , dri-devel@lists.freedesktop.org, Lucas De Marchi , Nirmoy Das Subject: [PATCH v5 2/8] drm/xe: Add XE_ENGINE_CLASS_OTHER to str conversion Date: Fri, 17 May 2024 13:43:04 -0700 Message-ID: <20240517204310.88854-3-lucas.demarchi@intel.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20240517204310.88854-1-lucas.demarchi@intel.com> References: <20240517204310.88854-1-lucas.demarchi@intel.com> MIME-Version: 1.0 X-BeenThere: dri-devel@lists.freedesktop.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Direct Rendering Infrastructure - Development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dri-devel-bounces@lists.freedesktop.org Sender: "dri-devel" XE_ENGINE_CLASS_OTHER was missing from the str conversion. Add it and remove the default handling so it's protected by -Wswitch. Currently the only user is xe_hw_engine_class_sysfs_init(), which already skips XE_ENGINE_CLASS_OTHER, so there's no change in behavior. Reviewed-by: Nirmoy Das Signed-off-by: Lucas De Marchi --- drivers/gpu/drm/xe/xe_hw_engine.c | 8 ++++++-- 1 file changed, 6 insertions(+), 2 deletions(-) diff --git a/drivers/gpu/drm/xe/xe_hw_engine.c b/drivers/gpu/drm/xe/xe_hw_engine.c index b71e90c555fa..942fca8f1eb9 100644 --- a/drivers/gpu/drm/xe/xe_hw_engine.c +++ b/drivers/gpu/drm/xe/xe_hw_engine.c @@ -1111,9 +1111,13 @@ const char *xe_hw_engine_class_to_str(enum xe_engine_class class) return "vecs"; case XE_ENGINE_CLASS_COPY: return "bcs"; + case XE_ENGINE_CLASS_OTHER: + return "other"; case XE_ENGINE_CLASS_COMPUTE: return "ccs"; - default: - return NULL; + case XE_ENGINE_CLASS_MAX: + break; } + + return NULL; } From patchwork Fri May 17 20:43:05 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Lucas De Marchi X-Patchwork-Id: 13667381 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from gabe.freedesktop.org (gabe.freedesktop.org [131.252.210.177]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id CC350C04FFE for ; Fri, 17 May 2024 20:43:23 +0000 (UTC) Received: from gabe.freedesktop.org (localhost [127.0.0.1]) by gabe.freedesktop.org (Postfix) with ESMTP id 1312210EF83; Fri, 17 May 2024 20:43:15 +0000 (UTC) Authentication-Results: gabe.freedesktop.org; dkim=pass (2048-bit key; unprotected) header.d=intel.com header.i=@intel.com header.b="RnEv3WJO"; dkim-atps=neutral Received: from mgamail.intel.com (mgamail.intel.com [198.175.65.19]) by gabe.freedesktop.org (Postfix) with ESMTPS id A8CCB10EF70; Fri, 17 May 2024 20:43:10 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1715978591; x=1747514591; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=jAAH7nca/REOs/C6Nd0bQTUOEodfvlodNhpWGM+TVfo=; b=RnEv3WJOSl71dEEGAEAoZHHZ5kVtBIRmgq3nO3a5ej/2mEO3A303rhcs EMTRmvHwDayoPYh9l+Ribe+0PuYeWhOTZYhGLqqhPuhsoefsAolZdboYx XqEQgfVrtH8m99JDWfDamRz23IgIMRhEpYDaE7HRtiNDwVWiUHLy98BtP 0daQga2L/akzlP4ITGSpiR5X+0jhEYW6fb5Yef8lPpcf9pUeTeyNmSJAe mra7m1q+ggBealDwXfps1fc5DByZgKiarpITDIRCp9mP8z8oQXP1OySCH qDe5cjFNAX8RiE7OA0UnbOvtzdD5UTdwW+UvWzPBe5vg/P1U75hXTsQfU Q==; X-CSE-ConnectionGUID: Ssqit79XQ8SifQMZ2Hx3bQ== X-CSE-MsgGUID: lQSA/knhSYac33LURs6tEg== X-IronPort-AV: E=McAfee;i="6600,9927,11075"; a="12027916" X-IronPort-AV: E=Sophos;i="6.08,168,1712646000"; d="scan'208";a="12027916" Received: from fmviesa002.fm.intel.com ([10.60.135.142]) by orvoesa111.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 17 May 2024 13:43:10 -0700 X-CSE-ConnectionGUID: lzXoQudATgO5H+OM7e5OeQ== X-CSE-MsgGUID: YrgdTVU+Qbmw1F5oUeJ4IQ== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.08,168,1712646000"; d="scan'208";a="55125242" Received: from lucas-s2600cw.jf.intel.com ([10.165.21.196]) by fmviesa002-auth.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 17 May 2024 13:43:09 -0700 From: Lucas De Marchi To: Cc: Tvrtko Ursulin , Umesh Nerlige Ramappa , dri-devel@lists.freedesktop.org, Francois Dugast , Lucas De Marchi Subject: [PATCH v5 3/8] drm/xe/lrc: Add helper to capture context timestamp Date: Fri, 17 May 2024 13:43:05 -0700 Message-ID: <20240517204310.88854-4-lucas.demarchi@intel.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20240517204310.88854-1-lucas.demarchi@intel.com> References: <20240517204310.88854-1-lucas.demarchi@intel.com> MIME-Version: 1.0 X-BeenThere: dri-devel@lists.freedesktop.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Direct Rendering Infrastructure - Development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dri-devel-bounces@lists.freedesktop.org Sender: "dri-devel" From: Umesh Nerlige Ramappa Add a helper to capture CTX_TIMESTAMP from the context image so it can be used to calculate the runtime. v2: Add kernel-doc to clarify expectation from caller Signed-off-by: Umesh Nerlige Ramappa Reviewed-by: Francois Dugast Signed-off-by: Lucas De Marchi --- drivers/gpu/drm/xe/regs/xe_lrc_layout.h | 1 + drivers/gpu/drm/xe/xe_lrc.c | 12 ++++++++++++ drivers/gpu/drm/xe/xe_lrc.h | 14 ++++++++++++++ drivers/gpu/drm/xe/xe_lrc_types.h | 3 +++ 4 files changed, 30 insertions(+) diff --git a/drivers/gpu/drm/xe/regs/xe_lrc_layout.h b/drivers/gpu/drm/xe/regs/xe_lrc_layout.h index e6ca8bbda8f4..045dfd09db99 100644 --- a/drivers/gpu/drm/xe/regs/xe_lrc_layout.h +++ b/drivers/gpu/drm/xe/regs/xe_lrc_layout.h @@ -11,6 +11,7 @@ #define CTX_RING_TAIL (0x06 + 1) #define CTX_RING_START (0x08 + 1) #define CTX_RING_CTL (0x0a + 1) +#define CTX_TIMESTAMP (0x22 + 1) #define CTX_INDIRECT_RING_STATE (0x26 + 1) #define CTX_PDP0_UDW (0x30 + 1) #define CTX_PDP0_LDW (0x32 + 1) diff --git a/drivers/gpu/drm/xe/xe_lrc.c b/drivers/gpu/drm/xe/xe_lrc.c index 9b0a4078add3..f679cb9aaea7 100644 --- a/drivers/gpu/drm/xe/xe_lrc.c +++ b/drivers/gpu/drm/xe/xe_lrc.c @@ -844,6 +844,7 @@ int xe_lrc_init(struct xe_lrc *lrc, struct xe_hw_engine *hwe, lrc->tile = gt_to_tile(hwe->gt); lrc->ring.size = ring_size; lrc->ring.tail = 0; + lrc->ctx_timestamp = 0; xe_hw_fence_ctx_init(&lrc->fence_ctx, hwe->gt, hwe->fence_irq, hwe->name); @@ -898,6 +899,8 @@ int xe_lrc_init(struct xe_lrc *lrc, struct xe_hw_engine *hwe, RING_CTL_SIZE(lrc->ring.size) | RING_VALID); } + xe_lrc_write_ctx_reg(lrc, CTX_TIMESTAMP, 0); + if (xe->info.has_asid && vm) xe_lrc_write_ctx_reg(lrc, PVC_CTX_ASID, vm->usm.asid); @@ -1576,3 +1579,12 @@ void xe_lrc_snapshot_free(struct xe_lrc_snapshot *snapshot) xe_bo_put(snapshot->lrc_bo); kfree(snapshot); } + +u32 xe_lrc_update_timestamp(struct xe_lrc *lrc, u32 *old_ts) +{ + *old_ts = lrc->ctx_timestamp; + + lrc->ctx_timestamp = xe_lrc_read_ctx_reg(lrc, CTX_TIMESTAMP); + + return lrc->ctx_timestamp; +} diff --git a/drivers/gpu/drm/xe/xe_lrc.h b/drivers/gpu/drm/xe/xe_lrc.h index e0e841963c23..b9da1031083b 100644 --- a/drivers/gpu/drm/xe/xe_lrc.h +++ b/drivers/gpu/drm/xe/xe_lrc.h @@ -66,4 +66,18 @@ void xe_lrc_snapshot_capture_delayed(struct xe_lrc_snapshot *snapshot); void xe_lrc_snapshot_print(struct xe_lrc_snapshot *snapshot, struct drm_printer *p); void xe_lrc_snapshot_free(struct xe_lrc_snapshot *snapshot); +/** + * xe_lrc_update_timestamp - readout LRC timestamp and update cached value + * @lrc: logical ring context for this exec queue + * @old_ts: pointer where to save the previous timestamp + * + * Read the current timestamp for this LRC and update the cached value. The + * previous cached value is also returned in @old_ts so the caller can calculate + * the delta between 2 updates. Note that this is not intended to be called from + * any place, but just by the paths updating the drm client utilization. + * + * Returns the current LRC timestamp + */ +u32 xe_lrc_update_timestamp(struct xe_lrc *lrc, u32 *old_ts); + #endif diff --git a/drivers/gpu/drm/xe/xe_lrc_types.h b/drivers/gpu/drm/xe/xe_lrc_types.h index cdbf03faef15..0fa055da6b27 100644 --- a/drivers/gpu/drm/xe/xe_lrc_types.h +++ b/drivers/gpu/drm/xe/xe_lrc_types.h @@ -45,6 +45,9 @@ struct xe_lrc { /** @fence_ctx: context for hw fence */ struct xe_hw_fence_ctx fence_ctx; + + /** @ctx_timestamp: readout value of CTX_TIMESTAMP on last update */ + u32 ctx_timestamp; }; struct xe_lrc_snapshot; From patchwork Fri May 17 20:43:06 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Lucas De Marchi X-Patchwork-Id: 13667385 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from gabe.freedesktop.org (gabe.freedesktop.org [131.252.210.177]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id D610FC04FFE for ; Fri, 17 May 2024 20:43:28 +0000 (UTC) Received: from gabe.freedesktop.org (localhost [127.0.0.1]) by gabe.freedesktop.org (Postfix) with ESMTP id C716910EF8B; Fri, 17 May 2024 20:43:16 +0000 (UTC) Authentication-Results: gabe.freedesktop.org; dkim=pass (2048-bit key; unprotected) header.d=intel.com header.i=@intel.com header.b="XSpvcwo3"; dkim-atps=neutral Received: from mgamail.intel.com (mgamail.intel.com [198.175.65.19]) by gabe.freedesktop.org (Postfix) with ESMTPS id A33DE10E028; Fri, 17 May 2024 20:43:10 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1715978591; x=1747514591; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=SBCJXVoLkG3jD8wIu4oX82y81ZysAQwO6ftNb8XsKz0=; b=XSpvcwo3Mi7HiXFCfHDLOpIx/sejDssNMjez4UC1J758icYFBYYrwK17 krhF+UoGf8ZUT/pijgDMDceOepBNwI/Qh4W7JAh/Ksl9xtnW9Zm50+KT4 5Lgfj4lN1PPY50tcUPc5/+LvG8dwldsVSxj8nTBgmVgzqg4PAc7m1mUy8 SADL6P1x/fF03qnby192WyPQsyZ3rcNufl+OciJGJUcNJabPg3nExQ/SL xMpkDErYHw4S8Waxx4TI1eUNQWiH9YLU2wbzWvS85sOuOUeto9++Hk2ny 2YMUPoweBHjG1zf1po0u/WOs5niA9o496qFNZyON8AKpTz3e6U4wSt++Q Q==; X-CSE-ConnectionGUID: 7DkYpHQKTr+csS61oOlbrw== X-CSE-MsgGUID: 5SFtuwoKRkW248/T2FpAdA== X-IronPort-AV: E=McAfee;i="6600,9927,11075"; a="12027918" X-IronPort-AV: E=Sophos;i="6.08,168,1712646000"; d="scan'208";a="12027918" Received: from fmviesa002.fm.intel.com ([10.60.135.142]) by orvoesa111.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 17 May 2024 13:43:10 -0700 X-CSE-ConnectionGUID: bQJNXfblRoWSNj744WB1aQ== X-CSE-MsgGUID: an713P3jRNicg88zCdiybA== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.08,168,1712646000"; d="scan'208";a="55125245" Received: from lucas-s2600cw.jf.intel.com ([10.165.21.196]) by fmviesa002-auth.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 17 May 2024 13:43:10 -0700 From: Lucas De Marchi To: Cc: Tvrtko Ursulin , Umesh Nerlige Ramappa , dri-devel@lists.freedesktop.org, Lucas De Marchi Subject: [PATCH v5 4/8] drm/xe: Add helper to capture engine timestamp Date: Fri, 17 May 2024 13:43:06 -0700 Message-ID: <20240517204310.88854-5-lucas.demarchi@intel.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20240517204310.88854-1-lucas.demarchi@intel.com> References: <20240517204310.88854-1-lucas.demarchi@intel.com> MIME-Version: 1.0 X-BeenThere: dri-devel@lists.freedesktop.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Direct Rendering Infrastructure - Development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dri-devel-bounces@lists.freedesktop.org Sender: "dri-devel" Just like CTX_TIMESTAMP is used to calculate runtime, add a helper to get the timestamp for the engine so it can be used to calculate the "engine time" with the same unit as the runtime is recorded. Reviewed-by: Umesh Nerlige Ramappa Signed-off-by: Lucas De Marchi --- drivers/gpu/drm/xe/xe_hw_engine.c | 5 +++++ drivers/gpu/drm/xe/xe_hw_engine.h | 1 + 2 files changed, 6 insertions(+) diff --git a/drivers/gpu/drm/xe/xe_hw_engine.c b/drivers/gpu/drm/xe/xe_hw_engine.c index 942fca8f1eb9..de1aefaa2335 100644 --- a/drivers/gpu/drm/xe/xe_hw_engine.c +++ b/drivers/gpu/drm/xe/xe_hw_engine.c @@ -1121,3 +1121,8 @@ const char *xe_hw_engine_class_to_str(enum xe_engine_class class) return NULL; } + +u64 xe_hw_engine_read_timestamp(struct xe_hw_engine *hwe) +{ + return xe_mmio_read64_2x32(hwe->gt, RING_TIMESTAMP(hwe->mmio_base)); +} diff --git a/drivers/gpu/drm/xe/xe_hw_engine.h b/drivers/gpu/drm/xe/xe_hw_engine.h index 843de159e47c..7f2d27c0ba1a 100644 --- a/drivers/gpu/drm/xe/xe_hw_engine.h +++ b/drivers/gpu/drm/xe/xe_hw_engine.h @@ -68,5 +68,6 @@ static inline bool xe_hw_engine_is_valid(struct xe_hw_engine *hwe) } const char *xe_hw_engine_class_to_str(enum xe_engine_class class); +u64 xe_hw_engine_read_timestamp(struct xe_hw_engine *hwe); #endif From patchwork Fri May 17 20:43:07 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Lucas De Marchi X-Patchwork-Id: 13667382 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from gabe.freedesktop.org (gabe.freedesktop.org [131.252.210.177]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 4A832C25B77 for ; Fri, 17 May 2024 20:43:25 +0000 (UTC) Received: from gabe.freedesktop.org (localhost [127.0.0.1]) by gabe.freedesktop.org (Postfix) with ESMTP id 8FAD410EF85; Fri, 17 May 2024 20:43:15 +0000 (UTC) Authentication-Results: gabe.freedesktop.org; dkim=pass (2048-bit key; unprotected) header.d=intel.com header.i=@intel.com header.b="anojYahG"; dkim-atps=neutral Received: from mgamail.intel.com (mgamail.intel.com [198.175.65.19]) by gabe.freedesktop.org (Postfix) with ESMTPS id C2DCE10EF73; Fri, 17 May 2024 20:43:10 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1715978591; x=1747514591; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=II4IT1+MpR8Glvw/FHTDM6nfFh7W8K8VNJfz9XaJt2Q=; b=anojYahGoz2q1e6h2pUIGgPGRuu4r8WCCn1n6D/XHfWXAxx9BMZeR2aj EQ1e0VN/zk381aA6ifSes3aUTJO+06NMgbILhnMMI0Ru0MGP7o0tVEd5Y wnxStF/F08NuzVnoslNWnCkpSaB/+IcizqWso13oCbPkcIHb0ESPJy6Jc U4k9fiGoRcBsSTE6HZEuq4jfNFZah5TUNTFhkxTOM+gojGocfc8WIHtIC HMmqHrIBEp2eHtAqwISoRtm0Psptg8+8zSyY5wKbXoaus6u8C2+6beSNy L6jOtEe6MLGApxu4GcE/j0ycmjNZ/ekliGnr7Pnm6tkJpJg7Du1mgrIwG Q==; X-CSE-ConnectionGUID: /95bSOFtT4SC3opcqZ9syw== X-CSE-MsgGUID: GNV+J1hjTOW7bJvJfvbQMQ== X-IronPort-AV: E=McAfee;i="6600,9927,11075"; a="12027920" X-IronPort-AV: E=Sophos;i="6.08,168,1712646000"; d="scan'208";a="12027920" Received: from fmviesa002.fm.intel.com ([10.60.135.142]) by orvoesa111.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 17 May 2024 13:43:10 -0700 X-CSE-ConnectionGUID: rZ9XL2JfSeG0SM3bNqeEyA== X-CSE-MsgGUID: Tku6cNvIQa21r0qHvJXs6w== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.08,168,1712646000"; d="scan'208";a="55125248" Received: from lucas-s2600cw.jf.intel.com ([10.165.21.196]) by fmviesa002-auth.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 17 May 2024 13:43:10 -0700 From: Lucas De Marchi To: Cc: Tvrtko Ursulin , Umesh Nerlige Ramappa , dri-devel@lists.freedesktop.org, Lucas De Marchi Subject: [PATCH v5 5/8] drm/xe: Add helper to accumulate exec queue runtime Date: Fri, 17 May 2024 13:43:07 -0700 Message-ID: <20240517204310.88854-6-lucas.demarchi@intel.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20240517204310.88854-1-lucas.demarchi@intel.com> References: <20240517204310.88854-1-lucas.demarchi@intel.com> MIME-Version: 1.0 X-BeenThere: dri-devel@lists.freedesktop.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Direct Rendering Infrastructure - Development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dri-devel-bounces@lists.freedesktop.org Sender: "dri-devel" From: Umesh Nerlige Ramappa Add a helper to accumulate per-client runtime of all its exec queues. This is called every time a sched job is finished. v2: - Use guc_exec_queue_free_job() and execlist_job_free() to accumulate runtime when job is finished since xe_sched_job_completed() is not a notification that job finished. - Stop trying to update runtime from xe_exec_queue_fini() - that is redundant and may happen after xef is closed, leading to a use-after-free - Do not special case the first timestamp read: the default LRC sets CTX_TIMESTAMP to zero, so even the first sample should be a valid one. - Handle the parallel submission case by multiplying the runtime by width. v3: Update comments Signed-off-by: Umesh Nerlige Ramappa Signed-off-by: Lucas De Marchi Reviewed-by: Matt Roper --- drivers/gpu/drm/xe/xe_device_types.h | 3 +++ drivers/gpu/drm/xe/xe_exec_queue.c | 37 ++++++++++++++++++++++++++++ drivers/gpu/drm/xe/xe_exec_queue.h | 1 + drivers/gpu/drm/xe/xe_execlist.c | 1 + drivers/gpu/drm/xe/xe_guc_submit.c | 2 ++ 5 files changed, 44 insertions(+) diff --git a/drivers/gpu/drm/xe/xe_device_types.h b/drivers/gpu/drm/xe/xe_device_types.h index 5c5e36de452a..bc97990fd032 100644 --- a/drivers/gpu/drm/xe/xe_device_types.h +++ b/drivers/gpu/drm/xe/xe_device_types.h @@ -555,6 +555,9 @@ struct xe_file { struct mutex lock; } exec_queue; + /** @runtime: hw engine class runtime in ticks for this drm client */ + u64 runtime[XE_ENGINE_CLASS_MAX]; + /** @client: drm client */ struct xe_drm_client *client; }; diff --git a/drivers/gpu/drm/xe/xe_exec_queue.c b/drivers/gpu/drm/xe/xe_exec_queue.c index 395de93579fa..fa6dc996eca8 100644 --- a/drivers/gpu/drm/xe/xe_exec_queue.c +++ b/drivers/gpu/drm/xe/xe_exec_queue.c @@ -769,6 +769,43 @@ bool xe_exec_queue_is_idle(struct xe_exec_queue *q) q->lrc[0].fence_ctx.next_seqno - 1; } +/** + * xe_exec_queue_update_runtime() - Update runtime for this exec queue from hw + * @q: The exec queue + * + * Update the timestamp saved by HW for this exec queue and save runtime + * calculated by using the delta from last update. On multi-lrc case, only the + * first is considered. + */ +void xe_exec_queue_update_runtime(struct xe_exec_queue *q) +{ + struct xe_file *xef; + struct xe_lrc *lrc; + u32 old_ts, new_ts; + + /* + * Jobs that are run during driver load may use an exec_queue, but are + * not associated with a user xe file, so avoid accumulating busyness + * for kernel specific work. + */ + if (!q->vm || !q->vm->xef) + return; + + xef = q->vm->xef; + + /* + * Only sample the first LRC. For parallel submission, all of them are + * scheduled together and we compensate that below by multiplying by + * width - this may introduce errors if that premise is not true and + * they don't exit 100% aligned. On the other hand, looping through + * the LRCs and reading them in different time could also introduce + * errors. + */ + lrc = &q->lrc[0]; + new_ts = xe_lrc_update_timestamp(lrc, &old_ts); + xef->runtime[q->class] += (new_ts - old_ts) * q->width; +} + void xe_exec_queue_kill(struct xe_exec_queue *q) { struct xe_exec_queue *eq = q, *next; diff --git a/drivers/gpu/drm/xe/xe_exec_queue.h b/drivers/gpu/drm/xe/xe_exec_queue.h index 48f6da53a292..e0f07d28ee1a 100644 --- a/drivers/gpu/drm/xe/xe_exec_queue.h +++ b/drivers/gpu/drm/xe/xe_exec_queue.h @@ -75,5 +75,6 @@ struct dma_fence *xe_exec_queue_last_fence_get(struct xe_exec_queue *e, struct xe_vm *vm); void xe_exec_queue_last_fence_set(struct xe_exec_queue *e, struct xe_vm *vm, struct dma_fence *fence); +void xe_exec_queue_update_runtime(struct xe_exec_queue *q); #endif diff --git a/drivers/gpu/drm/xe/xe_execlist.c b/drivers/gpu/drm/xe/xe_execlist.c index e9dee1e14fef..bd7f27efe0e0 100644 --- a/drivers/gpu/drm/xe/xe_execlist.c +++ b/drivers/gpu/drm/xe/xe_execlist.c @@ -306,6 +306,7 @@ static void execlist_job_free(struct drm_sched_job *drm_job) { struct xe_sched_job *job = to_xe_sched_job(drm_job); + xe_exec_queue_update_runtime(job->q); xe_sched_job_put(job); } diff --git a/drivers/gpu/drm/xe/xe_guc_submit.c b/drivers/gpu/drm/xe/xe_guc_submit.c index 4efb88e3e056..ad2b8067d071 100644 --- a/drivers/gpu/drm/xe/xe_guc_submit.c +++ b/drivers/gpu/drm/xe/xe_guc_submit.c @@ -749,6 +749,8 @@ static void guc_exec_queue_free_job(struct drm_sched_job *drm_job) { struct xe_sched_job *job = to_xe_sched_job(drm_job); + xe_exec_queue_update_runtime(job->q); + trace_xe_sched_job_free(job); xe_sched_job_put(job); } From patchwork Fri May 17 20:43:08 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Lucas De Marchi X-Patchwork-Id: 13667384 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from gabe.freedesktop.org (gabe.freedesktop.org [131.252.210.177]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id BB92CC25B77 for ; Fri, 17 May 2024 20:43:27 +0000 (UTC) Received: from gabe.freedesktop.org (localhost [127.0.0.1]) by gabe.freedesktop.org (Postfix) with ESMTP id E192010EF88; Fri, 17 May 2024 20:43:15 +0000 (UTC) Authentication-Results: gabe.freedesktop.org; dkim=pass (2048-bit key; unprotected) header.d=intel.com header.i=@intel.com header.b="M9l/AWhs"; dkim-atps=neutral Received: from mgamail.intel.com (mgamail.intel.com [198.175.65.19]) by gabe.freedesktop.org (Postfix) with ESMTPS id C160310E18D; Fri, 17 May 2024 20:43:10 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1715978591; x=1747514591; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=p2v94YvHLIxrjFd61LO/tdkS4Vn1i7ANhCJ/DrIoVqs=; b=M9l/AWhsPIA+siv6m5yexryes4F59jico2Av+NMHKSRtaFjBpabBXtzT djuHfBU1kVd3xEPRZHPjCjSs4HcIybRHHai9xZvOVOnHleF7WWrmE0h+X XMkRulMlywmjJmmUDLVfuCHDEXfgY6aiz5LENAgUO6B6/l3IT42UwuWmu Zz6N8I24QkDo/f9KfC/XrwnwuLSEMOyhA2yc63pkhSNKSINBIrgmcsxQs liu5e73Gh5uPrr4cqy9MSFfjD9oFHNdA7j+FOBuCEeSpvcPCjHYRVoPqM hcXEPK5xV44dl2ozEGm2iAB/DHBy8F2Q2SnurKFjzXpG311LMZWIZnbQL w==; X-CSE-ConnectionGUID: B5bK333SRaa/oBvDbvixmA== X-CSE-MsgGUID: Sg0h1j5gTy66gMbnfC9+GA== X-IronPort-AV: E=McAfee;i="6600,9927,11075"; a="12027922" X-IronPort-AV: E=Sophos;i="6.08,168,1712646000"; d="scan'208";a="12027922" Received: from fmviesa002.fm.intel.com ([10.60.135.142]) by orvoesa111.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 17 May 2024 13:43:10 -0700 X-CSE-ConnectionGUID: Nyhen6ACRPWx8VGMPWLuPA== X-CSE-MsgGUID: 9tu+V2BgRku5P9/d1pWjng== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.08,168,1712646000"; d="scan'208";a="55125251" Received: from lucas-s2600cw.jf.intel.com ([10.165.21.196]) by fmviesa002-auth.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 17 May 2024 13:43:10 -0700 From: Lucas De Marchi To: Cc: Tvrtko Ursulin , Umesh Nerlige Ramappa , dri-devel@lists.freedesktop.org, Lucas De Marchi , Jonathan Cavitt Subject: [PATCH v5 6/8] drm/xe: Cache data about user-visible engines Date: Fri, 17 May 2024 13:43:08 -0700 Message-ID: <20240517204310.88854-7-lucas.demarchi@intel.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20240517204310.88854-1-lucas.demarchi@intel.com> References: <20240517204310.88854-1-lucas.demarchi@intel.com> MIME-Version: 1.0 X-BeenThere: dri-devel@lists.freedesktop.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Direct Rendering Infrastructure - Development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dri-devel-bounces@lists.freedesktop.org Sender: "dri-devel" gt->info.engine_mask used to indicate the available engines, but that is not always true anymore: some engines are reserved to kernel and some may be exposed as a single engine (e.g. with ccs_mode). Runtime changes only happen when no clients exist, so it's safe to cache the list of engines in the gt and update that when it's needed. This will help implementing per client engine utilization so this (mostly constant) information doesn't need to be re-calculated on every query. Reviewed-by: Jonathan Cavitt Reviewed-by: Umesh Nerlige Ramappa Signed-off-by: Lucas De Marchi --- drivers/gpu/drm/xe/xe_gt.c | 23 +++++++++++++++++++++++ drivers/gpu/drm/xe/xe_gt.h | 13 +++++++++++++ drivers/gpu/drm/xe/xe_gt_ccs_mode.c | 1 + drivers/gpu/drm/xe/xe_gt_types.h | 21 ++++++++++++++++++++- 4 files changed, 57 insertions(+), 1 deletion(-) diff --git a/drivers/gpu/drm/xe/xe_gt.c b/drivers/gpu/drm/xe/xe_gt.c index e69a03ddd255..5194a3d38e76 100644 --- a/drivers/gpu/drm/xe/xe_gt.c +++ b/drivers/gpu/drm/xe/xe_gt.c @@ -560,9 +560,32 @@ int xe_gt_init(struct xe_gt *gt) if (err) return err; + xe_gt_record_user_engines(gt); + return drmm_add_action_or_reset(>_to_xe(gt)->drm, gt_fini, gt); } +void xe_gt_record_user_engines(struct xe_gt *gt) +{ + struct xe_hw_engine *hwe; + enum xe_hw_engine_id id; + + gt->user_engines.mask = 0; + memset(gt->user_engines.instances_per_class, 0, + sizeof(gt->user_engines.instances_per_class)); + + for_each_hw_engine(hwe, gt, id) { + if (xe_hw_engine_is_reserved(hwe)) + continue; + + gt->user_engines.mask |= BIT_ULL(id); + gt->user_engines.instances_per_class[hwe->class]++; + } + + xe_gt_assert(gt, (gt->user_engines.mask | gt->info.engine_mask) + == gt->info.engine_mask); +} + static int do_gt_reset(struct xe_gt *gt) { int err; diff --git a/drivers/gpu/drm/xe/xe_gt.h b/drivers/gpu/drm/xe/xe_gt.h index 8474c50b1b30..1d010bf4a756 100644 --- a/drivers/gpu/drm/xe/xe_gt.h +++ b/drivers/gpu/drm/xe/xe_gt.h @@ -38,6 +38,19 @@ int xe_gt_init_hwconfig(struct xe_gt *gt); int xe_gt_init_early(struct xe_gt *gt); int xe_gt_init(struct xe_gt *gt); int xe_gt_record_default_lrcs(struct xe_gt *gt); + +/** + * xe_gt_record_user_engines - save data related to engines available to + * usersapce + * @gt: GT structure + * + * Walk the available HW engines from gt->info.engine_mask and calculate data + * related to those engines that may be used by userspace. To be used whenever + * available engines change in runtime (e.g. with ccs_mode) or during + * initialization + */ +void xe_gt_record_user_engines(struct xe_gt *gt); + void xe_gt_suspend_prepare(struct xe_gt *gt); int xe_gt_suspend(struct xe_gt *gt); int xe_gt_resume(struct xe_gt *gt); diff --git a/drivers/gpu/drm/xe/xe_gt_ccs_mode.c b/drivers/gpu/drm/xe/xe_gt_ccs_mode.c index a34c9a24dafc..c36218f4f6c8 100644 --- a/drivers/gpu/drm/xe/xe_gt_ccs_mode.c +++ b/drivers/gpu/drm/xe/xe_gt_ccs_mode.c @@ -134,6 +134,7 @@ ccs_mode_store(struct device *kdev, struct device_attribute *attr, if (gt->ccs_mode != num_engines) { xe_gt_info(gt, "Setting compute mode to %d\n", num_engines); gt->ccs_mode = num_engines; + xe_gt_record_user_engines(gt); xe_gt_reset_async(gt); } diff --git a/drivers/gpu/drm/xe/xe_gt_types.h b/drivers/gpu/drm/xe/xe_gt_types.h index 475fb58882f1..10a9a9529377 100644 --- a/drivers/gpu/drm/xe/xe_gt_types.h +++ b/drivers/gpu/drm/xe/xe_gt_types.h @@ -113,7 +113,11 @@ struct xe_gt { enum xe_gt_type type; /** @info.reference_clock: clock frequency */ u32 reference_clock; - /** @info.engine_mask: mask of engines present on GT */ + /** + * @info.engine_mask: mask of engines present on GT. Some of + * them may be reserved in runtime and not available for user. + * See @user_engines.mask + */ u64 engine_mask; /** @info.gmdid: raw GMD_ID value from hardware */ u32 gmdid; @@ -368,6 +372,21 @@ struct xe_gt { /** @wa_active.oob: bitmap with active OOB workaroudns */ unsigned long *oob; } wa_active; + + /** @user_engines: engines present in GT and available to userspace */ + struct { + /** + * @user_engines.mask: like @info->engine_mask, but take in + * consideration only engines available to userspace + */ + u64 mask; + + /** + * @user_engines.instances_per_class: aggregate per class the + * number of engines available to userspace + */ + u8 instances_per_class[XE_ENGINE_CLASS_MAX]; + } user_engines; }; #endif From patchwork Fri May 17 20:43:09 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Lucas De Marchi X-Patchwork-Id: 13667383 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from gabe.freedesktop.org (gabe.freedesktop.org [131.252.210.177]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 8545AC04FFE for ; Fri, 17 May 2024 20:43:26 +0000 (UTC) Received: from gabe.freedesktop.org (localhost [127.0.0.1]) by gabe.freedesktop.org (Postfix) with ESMTP id DBD6610EF87; Fri, 17 May 2024 20:43:15 +0000 (UTC) Authentication-Results: gabe.freedesktop.org; dkim=pass (2048-bit key; unprotected) header.d=intel.com header.i=@intel.com header.b="AXJbcA0D"; dkim-atps=neutral Received: from mgamail.intel.com (mgamail.intel.com [198.175.65.19]) by gabe.freedesktop.org (Postfix) with ESMTPS id D7DB910EF6C; Fri, 17 May 2024 20:43:10 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1715978591; x=1747514591; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=1cQ3w5hvk+fE/vadDvzkaOVMczVV7aLAy6G+Ufu7nDY=; b=AXJbcA0DcIGzCCE3nkO1KtZsDAYFlQj2KlgYuz1GS0ZIYkh66HudAZYY VzTe602hZqKrnyb1uYWY4X+Q/gTDfEiXnqclCjZAIj2FnRXGe7n1E1ItL eTv45lHcitU45Ppa3TAwD20iT6Ia+ny6zusAjGshqxF0OstzI2WPECeJP j1TnpPPQzFNk7Qv3dvB+8Bu2sJnHXFLCas+FZXecG4dOlDH1UITHLUecR nXyC/SDPR46Q4r0rfmNZSYKyfv2D2inJEIeGY3EGC5LDpNc2qBZrjRX7s rUsQiAsacwXtIrcMeWHng6RmfLs8GJrXacIcn9I2qrq0Fx3XPy0rbOtpg A==; X-CSE-ConnectionGUID: k7nScczWQGGVDVtSvgDWZA== X-CSE-MsgGUID: cmGv2u3/TtqoB6qp2ZTK0Q== X-IronPort-AV: E=McAfee;i="6600,9927,11075"; a="12027924" X-IronPort-AV: E=Sophos;i="6.08,168,1712646000"; d="scan'208";a="12027924" Received: from fmviesa002.fm.intel.com ([10.60.135.142]) by orvoesa111.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 17 May 2024 13:43:10 -0700 X-CSE-ConnectionGUID: TwRtlKULQcaoVpHjZE2+Og== X-CSE-MsgGUID: Q5ETi9cKTnK0Zm1VToWjqQ== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.08,168,1712646000"; d="scan'208";a="55125254" Received: from lucas-s2600cw.jf.intel.com ([10.165.21.196]) by fmviesa002-auth.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 17 May 2024 13:43:10 -0700 From: Lucas De Marchi To: Cc: Tvrtko Ursulin , Umesh Nerlige Ramappa , dri-devel@lists.freedesktop.org, Lucas De Marchi Subject: [PATCH v5 7/8] drm/xe: Add helper to return any available hw engine Date: Fri, 17 May 2024 13:43:09 -0700 Message-ID: <20240517204310.88854-8-lucas.demarchi@intel.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20240517204310.88854-1-lucas.demarchi@intel.com> References: <20240517204310.88854-1-lucas.demarchi@intel.com> MIME-Version: 1.0 X-BeenThere: dri-devel@lists.freedesktop.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Direct Rendering Infrastructure - Development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dri-devel-bounces@lists.freedesktop.org Sender: "dri-devel" Get the first available engine from a gt, which helps in the case any engine serves as a context, like when reading RING_TIMESTAMP. Reviewed-by: Umesh Nerlige Ramappa Signed-off-by: Lucas De Marchi --- drivers/gpu/drm/xe/xe_gt.c | 11 +++++++++++ drivers/gpu/drm/xe/xe_gt.h | 7 +++++++ 2 files changed, 18 insertions(+) diff --git a/drivers/gpu/drm/xe/xe_gt.c b/drivers/gpu/drm/xe/xe_gt.c index 5194a3d38e76..3432fef56486 100644 --- a/drivers/gpu/drm/xe/xe_gt.c +++ b/drivers/gpu/drm/xe/xe_gt.c @@ -833,3 +833,14 @@ struct xe_hw_engine *xe_gt_any_hw_engine_by_reset_domain(struct xe_gt *gt, return NULL; } + +struct xe_hw_engine *xe_gt_any_hw_engine(struct xe_gt *gt) +{ + struct xe_hw_engine *hwe; + enum xe_hw_engine_id id; + + for_each_hw_engine(hwe, gt, id) + return hwe; + + return NULL; +} diff --git a/drivers/gpu/drm/xe/xe_gt.h b/drivers/gpu/drm/xe/xe_gt.h index 1d010bf4a756..9073ac68a777 100644 --- a/drivers/gpu/drm/xe/xe_gt.h +++ b/drivers/gpu/drm/xe/xe_gt.h @@ -67,6 +67,13 @@ void xe_gt_remove(struct xe_gt *gt); struct xe_hw_engine * xe_gt_any_hw_engine_by_reset_domain(struct xe_gt *gt, enum xe_engine_class class); +/** + * xe_gt_any_hw_engine - scan the list of engines and return the + * first available + * @gt: GT structure + */ +struct xe_hw_engine *xe_gt_any_hw_engine(struct xe_gt *gt); + struct xe_hw_engine *xe_gt_hw_engine(struct xe_gt *gt, enum xe_engine_class class, u16 instance, From patchwork Fri May 17 20:43:10 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Lucas De Marchi X-Patchwork-Id: 13667377 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from gabe.freedesktop.org (gabe.freedesktop.org [131.252.210.177]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id A0DBCC04FFE for ; Fri, 17 May 2024 20:43:13 +0000 (UTC) Received: from gabe.freedesktop.org (localhost [127.0.0.1]) by gabe.freedesktop.org (Postfix) with ESMTP id CC4D810EF73; Fri, 17 May 2024 20:43:11 +0000 (UTC) Authentication-Results: gabe.freedesktop.org; dkim=pass (2048-bit key; unprotected) header.d=intel.com header.i=@intel.com header.b="e4/pViMM"; dkim-atps=neutral Received: from mgamail.intel.com (mgamail.intel.com [198.175.65.19]) by gabe.freedesktop.org (Postfix) with ESMTPS id E047A10EF78; Fri, 17 May 2024 20:43:10 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1715978591; x=1747514591; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=SHJhXNfRn7NbiLW5x8OUnDzeYVq9Sg95c5H6fxSPSgk=; b=e4/pViMMdBnYdSmmwq0jzkuabH0Ze0LILkrycZS1FpymDCVryMyyIZh9 5n7OFy5FI8k4K29jlctl+MrY8WQsyyi4axsoQ32DCr5t7KtzzCuKYzFa7 XBRcmnrBQ6f/JpMzynQPwUyb8Hn322bMBB05jyQ5VcGvJ3jJwmiOyXTlD aoYDewOjkrf8LYZoSMLDZGj/UyPG+AzmgSIjWpd32rS9Ps2zHbbOkElJw 9RYtykTsZGfS7ipoJA09/Vp1uk4cL7HmuiJs0M6VRAT612SJ8vZiaT5pH qr9vaOecKddtYFN5LiFhhJvHjiLzwwPFEy9d2mE9vF2VoygI+ovdY8y1/ A==; X-CSE-ConnectionGUID: nQRC8DqVQROTlSqoB58ZLQ== X-CSE-MsgGUID: 7S+v5RYPRWi73o47Pv1jiA== X-IronPort-AV: E=McAfee;i="6600,9927,11075"; a="12027925" X-IronPort-AV: E=Sophos;i="6.08,168,1712646000"; d="scan'208";a="12027925" Received: from fmviesa002.fm.intel.com ([10.60.135.142]) by orvoesa111.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 17 May 2024 13:43:11 -0700 X-CSE-ConnectionGUID: 5rSPBvgqQrKjCyhe8PeDSg== X-CSE-MsgGUID: DcAvD4SwTxqfm2JISqErrA== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.08,168,1712646000"; d="scan'208";a="55125257" Received: from lucas-s2600cw.jf.intel.com ([10.165.21.196]) by fmviesa002-auth.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 17 May 2024 13:43:10 -0700 From: Lucas De Marchi To: Cc: Tvrtko Ursulin , Umesh Nerlige Ramappa , dri-devel@lists.freedesktop.org, Lucas De Marchi , Tvrtko Ursulin Subject: [PATCH v5 8/8] drm/xe/client: Print runtime to fdinfo Date: Fri, 17 May 2024 13:43:10 -0700 Message-ID: <20240517204310.88854-9-lucas.demarchi@intel.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20240517204310.88854-1-lucas.demarchi@intel.com> References: <20240517204310.88854-1-lucas.demarchi@intel.com> MIME-Version: 1.0 X-BeenThere: dri-devel@lists.freedesktop.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Direct Rendering Infrastructure - Development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dri-devel-bounces@lists.freedesktop.org Sender: "dri-devel" Print the accumulated runtime for client when printing fdinfo. Each time a query is done it first does 2 things: 1) loop through all the exec queues for the current client and accumulate the runtime, per engine class. CTX_TIMESTAMP is used for that, being read from the context image. 2) Read a "GPU timestamp" that can be used for considering "how much GPU time has passed" and that has the same unit/refclock as the one recording the runtime. RING_TIMESTAMP is used for that via MMIO. Since for all current platforms RING_TIMESTAMP follows the same refclock, just read it once, using any first engine available. This is exported to userspace as 2 numbers in fdinfo: drm-cycles-: drm-total-cycles-: Userspace is expected to collect at least 2 samples, which allows to know the client engine busyness as per: RUNTIME1 - RUNTIME0 busyness = --------------------- T1 - T0 Since drm-cycles- always starts at 0, it's also possible to know if and engine was ever used by a client. It's expected that userspace will read any 2 samples every few seconds. Given the update frequency of the counters involved and that CTX_TIMESTAMP is 32-bits, the counter for each exec_queue can wrap around (assuming 100% utilization) after ~200s. The wraparound is not perceived by userspace since it's just accumulated for all the exec_queues in a 64-bit counter) but the measurement will not be accurate if the samples are too far apart. This could be mitigated by adding a workqueue to accumulate the counters every so often, but it's additional complexity for something that is done already by userspace every few seconds in tools like gputop (from igt), htop, nvtop, etc, with none of them really defaulting to 1 sample per minute or more. Reviewed-by: Umesh Nerlige Ramappa Acked-by: Tvrtko Ursulin Signed-off-by: Lucas De Marchi --- Documentation/gpu/drm-usage-stats.rst | 21 +++- Documentation/gpu/xe/index.rst | 1 + Documentation/gpu/xe/xe-drm-usage-stats.rst | 10 ++ drivers/gpu/drm/xe/xe_drm_client.c | 121 +++++++++++++++++++- 4 files changed, 150 insertions(+), 3 deletions(-) create mode 100644 Documentation/gpu/xe/xe-drm-usage-stats.rst diff --git a/Documentation/gpu/drm-usage-stats.rst b/Documentation/gpu/drm-usage-stats.rst index 6dc299343b48..a80f95ca1b2f 100644 --- a/Documentation/gpu/drm-usage-stats.rst +++ b/Documentation/gpu/drm-usage-stats.rst @@ -112,6 +112,19 @@ larger value within a reasonable period. Upon observing a value lower than what was previously read, userspace is expected to stay with that larger previous value until a monotonic update is seen. +- drm-total-cycles-: + +Engine identifier string must be the same as the one specified in the +drm-cycles- tag and shall contain the total number cycles for the given +engine. + +This is a timestamp in GPU unspecified unit that matches the update rate +of drm-cycles-. For drivers that implement this interface, the engine +utilization can be calculated entirely on the GPU clock domain, without +considering the CPU sleep time between 2 samples. + +A driver may implement either this key or drm-maxfreq-, but not both. + - drm-maxfreq-: [Hz|MHz|KHz] Engine identifier string must be the same as the one specified in the @@ -121,6 +134,9 @@ percentage utilization of the engine, whereas drm-engine- only reflects time active without considering what frequency the engine is operating as a percentage of its maximum frequency. +A driver may implement either this key or drm-total-cycles-, but not +both. + Memory ^^^^^^ @@ -168,5 +184,6 @@ be documented above and where possible, aligned with other drivers. Driver specific implementations ------------------------------- -:ref:`i915-usage-stats` -:ref:`panfrost-usage-stats` +* :ref:`i915-usage-stats` +* :ref:`panfrost-usage-stats` +* :ref:`xe-usage-stats` diff --git a/Documentation/gpu/xe/index.rst b/Documentation/gpu/xe/index.rst index c224ecaee81e..3f07aa3b5432 100644 --- a/Documentation/gpu/xe/index.rst +++ b/Documentation/gpu/xe/index.rst @@ -23,3 +23,4 @@ DG2, etc is provided to prototype the driver. xe_firmware xe_tile xe_debugging + xe-drm-usage-stats.rst diff --git a/Documentation/gpu/xe/xe-drm-usage-stats.rst b/Documentation/gpu/xe/xe-drm-usage-stats.rst new file mode 100644 index 000000000000..482d503ae68a --- /dev/null +++ b/Documentation/gpu/xe/xe-drm-usage-stats.rst @@ -0,0 +1,10 @@ +.. SPDX-License-Identifier: GPL-2.0+ + +.. _xe-usage-stats: + +======================================== +Xe DRM client usage stats implementation +======================================== + +.. kernel-doc:: drivers/gpu/drm/xe/xe_drm_client.c + :doc: DRM Client usage stats diff --git a/drivers/gpu/drm/xe/xe_drm_client.c b/drivers/gpu/drm/xe/xe_drm_client.c index 08f0b7c95901..af404c9e5cc0 100644 --- a/drivers/gpu/drm/xe/xe_drm_client.c +++ b/drivers/gpu/drm/xe/xe_drm_client.c @@ -2,6 +2,7 @@ /* * Copyright © 2023 Intel Corporation */ +#include "xe_drm_client.h" #include #include @@ -12,9 +13,66 @@ #include "xe_bo.h" #include "xe_bo_types.h" #include "xe_device_types.h" -#include "xe_drm_client.h" +#include "xe_exec_queue.h" +#include "xe_force_wake.h" +#include "xe_gt.h" +#include "xe_hw_engine.h" +#include "xe_pm.h" #include "xe_trace.h" +/** + * DOC: DRM Client usage stats + * + * The drm/xe driver implements the DRM client usage stats specification as + * documented in :ref:`drm-client-usage-stats`. + * + * Example of the output showing the implemented key value pairs and entirety of + * the currently possible format options: + * + * :: + * + * pos: 0 + * flags: 0100002 + * mnt_id: 26 + * ino: 685 + * drm-driver: xe + * drm-client-id: 3 + * drm-pdev: 0000:03:00.0 + * drm-total-system: 0 + * drm-shared-system: 0 + * drm-active-system: 0 + * drm-resident-system: 0 + * drm-purgeable-system: 0 + * drm-total-gtt: 192 KiB + * drm-shared-gtt: 0 + * drm-active-gtt: 0 + * drm-resident-gtt: 192 KiB + * drm-total-vram0: 23992 KiB + * drm-shared-vram0: 16 MiB + * drm-active-vram0: 0 + * drm-resident-vram0: 23992 KiB + * drm-total-stolen: 0 + * drm-shared-stolen: 0 + * drm-active-stolen: 0 + * drm-resident-stolen: 0 + * drm-cycles-rcs: 28257900 + * drm-total-cycles-rcs: 7655183225 + * drm-cycles-bcs: 0 + * drm-total-cycles-bcs: 7655183225 + * drm-cycles-vcs: 0 + * drm-total-cycles-vcs: 7655183225 + * drm-engine-capacity-vcs: 2 + * drm-cycles-vecs: 0 + * drm-total-cycles-vecs: 7655183225 + * drm-engine-capacity-vecs: 2 + * drm-cycles-ccs: 0 + * drm-total-cycles-ccs: 7655183225 + * drm-engine-capacity-ccs: 4 + * + * Possible `drm-cycles-` key names are: `rcs`, `ccs`, `bcs`, `vcs`, `vecs` and + * "other". + */ + /** * xe_drm_client_alloc() - Allocate drm client * @void: No arg @@ -179,6 +237,66 @@ static void show_meminfo(struct drm_printer *p, struct drm_file *file) } } +static void show_runtime(struct drm_printer *p, struct drm_file *file) +{ + unsigned long class, i, gt_id, capacity[XE_ENGINE_CLASS_MAX] = { }; + struct xe_file *xef = file->driver_priv; + struct xe_device *xe = xef->xe; + struct xe_gt *gt; + struct xe_hw_engine *hwe; + struct xe_exec_queue *q; + u64 gpu_timestamp; + + xe_pm_runtime_get(xe); + + /* Accumulate all the exec queues from this client */ + mutex_lock(&xef->exec_queue.lock); + xa_for_each(&xef->exec_queue.xa, i, q) + xe_exec_queue_update_runtime(q); + mutex_unlock(&xef->exec_queue.lock); + + /* Get the total GPU cycles */ + for_each_gt(gt, xe, gt_id) { + hwe = xe_gt_any_hw_engine(gt); + if (!hwe) + continue; + + xe_force_wake_get(gt_to_fw(gt), XE_FW_GT); + gpu_timestamp = xe_hw_engine_read_timestamp(hwe); + xe_force_wake_put(gt_to_fw(gt), XE_FW_GT); + break; + } + + xe_pm_runtime_put(xe); + + if (unlikely(!hwe)) + return; + + for (class = 0; class < XE_ENGINE_CLASS_MAX; class++) { + const char *class_name; + + for_each_gt(gt, xe, gt_id) + capacity[class] += gt->user_engines.instances_per_class[class]; + + /* + * Engines may be fused off or not exposed to userspace. Don't + * return anything if this entire class is not available + */ + if (!capacity[class]) + continue; + + class_name = xe_hw_engine_class_to_str(class); + drm_printf(p, "drm-cycles-%s:\t%llu\n", + class_name, xef->runtime[class]); + drm_printf(p, "drm-total-cycles-%s:\t%llu\n", + class_name, gpu_timestamp); + + if (capacity[class] > 1) + drm_printf(p, "drm-engine-capacity-%s:\t%lu\n", + class_name, capacity[class]); + } +} + /** * xe_drm_client_fdinfo() - Callback for fdinfo interface * @p: The drm_printer ptr @@ -192,5 +310,6 @@ static void show_meminfo(struct drm_printer *p, struct drm_file *file) void xe_drm_client_fdinfo(struct drm_printer *p, struct drm_file *file) { show_meminfo(p, file); + show_runtime(p, file); } #endif