From patchwork Tue Nov 29 00:48:30 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alan Previn X-Patchwork-Id: 13058046 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from gabe.freedesktop.org (gabe.freedesktop.org [131.252.210.177]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 83E1AC43217 for ; Tue, 29 Nov 2022 00:46:12 +0000 (UTC) Received: from gabe.freedesktop.org (localhost [127.0.0.1]) by gabe.freedesktop.org (Postfix) with ESMTP id 055DC10E35C; Tue, 29 Nov 2022 00:46:08 +0000 (UTC) Received: from mga05.intel.com (mga05.intel.com [192.55.52.43]) by gabe.freedesktop.org (Postfix) with ESMTPS id C52E010E35C for ; Tue, 29 Nov 2022 00:46:03 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1669682763; x=1701218763; h=from:to:subject:date:message-id:in-reply-to:references: mime-version:content-transfer-encoding; bh=QpuV7S1ih7yw7O0jZ9GCaRAMQuzrUTj0CvuBkxNHYJQ=; b=Y/Pfo7crYf1xE/FODXaNJGqVLNdjcjU2lJCqNoW68bGrfilZaRMShXZx a+SRNvNt8UC0q4HGsFmLQOJ70KPfWxyvkQpGwD+Tf8zSx7FJ+EuRX3XE5 iVp4aTnZ/6nVwHger8CERaAzMirk9zD3OZCJ1pqeRMjd9njfo07MpYFep bGSJHWIhQSMXzUnUctiB4GFgxtpMhrJJ0VsIx2PtbNjn4hCMqGjujNyAL PAB/c0sRuEqDZEPpAenmPlbUbAsvvQjs+x21C69cHZ48LagVIdleYouzt sE03LmZq9Eoi9+wmiUC19fiAR19FTgBSg8nyvp1IpGaIIc3YvOjIQCjqf w==; X-IronPort-AV: E=McAfee;i="6500,9779,10545"; a="401271778" X-IronPort-AV: E=Sophos;i="5.96,201,1665471600"; d="scan'208";a="401271778" Received: from orsmga001.jf.intel.com ([10.7.209.18]) by fmsmga105.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 28 Nov 2022 16:46:02 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6500,9779,10545"; a="676243287" X-IronPort-AV: E=Sophos;i="5.96,201,1665471600"; d="scan'208";a="676243287" Received: from aalteres-desk.fm.intel.com ([10.80.57.53]) by orsmga001.jf.intel.com with ESMTP; 28 Nov 2022 16:46:02 -0800 From: Alan Previn To: intel-gfx@lists.freedesktop.org Date: Mon, 28 Nov 2022 16:48:30 -0800 Message-Id: <20221129004832.3735764-2-alan.previn.teres.alexis@intel.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20221129004832.3735764-1-alan.previn.teres.alexis@intel.com> References: <20221129004832.3735764-1-alan.previn.teres.alexis@intel.com> MIME-Version: 1.0 Subject: [Intel-gfx] [PATCH v2 1/3] drm/i915/pxp: Invalidate all PXP fw sessions during teardown X-BeenThere: intel-gfx@lists.freedesktop.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Intel graphics driver community testing & development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: intel-gfx-bounces@lists.freedesktop.org Sender: "Intel-gfx" A gap was recently discovered where if an application did not invalidate all of the stream keys (intentionally or not), and the driver did a full PXP global teardown on the GT subsystem, we find that future session creation would fail on the security firmware's side of the equation. i915 is the entity that needs ensure the sessions' state across both iGT and security firmware are at a known clean point when performing a full global teardown. Architecturally speaking, i915 should inspect all active sessions and submit the invalidate-stream-key PXP command to the security firmware for each of them. However, for the upstream i915 driver we only support the arbitration session that can be created so that will be the only session we will cleanup. Signed-off-by: Alan Previn Reviewed-by: Juston Li --- drivers/gpu/drm/i915/pxp/intel_pxp.h | 1 + .../drm/i915/pxp/intel_pxp_cmd_interface_42.h | 15 ++++++++ .../i915/pxp/intel_pxp_cmd_interface_cmn.h | 3 ++ drivers/gpu/drm/i915/pxp/intel_pxp_session.c | 5 +++ drivers/gpu/drm/i915/pxp/intel_pxp_tee.c | 35 +++++++++++++++++++ drivers/gpu/drm/i915/pxp/intel_pxp_types.h | 2 ++ 6 files changed, 61 insertions(+) diff --git a/drivers/gpu/drm/i915/pxp/intel_pxp.h b/drivers/gpu/drm/i915/pxp/intel_pxp.h index 2da309088c6d..bbeb8ed8e211 100644 --- a/drivers/gpu/drm/i915/pxp/intel_pxp.h +++ b/drivers/gpu/drm/i915/pxp/intel_pxp.h @@ -23,6 +23,7 @@ void intel_pxp_init_hw(struct intel_pxp *pxp); void intel_pxp_fini_hw(struct intel_pxp *pxp); void intel_pxp_mark_termination_in_progress(struct intel_pxp *pxp); +void intel_pxp_tee_end_arb_fw_session(struct intel_pxp *pxp, u32 arb_session_id); int intel_pxp_start(struct intel_pxp *pxp); diff --git a/drivers/gpu/drm/i915/pxp/intel_pxp_cmd_interface_42.h b/drivers/gpu/drm/i915/pxp/intel_pxp_cmd_interface_42.h index 739f9072fa5f..26f7d9f01bf3 100644 --- a/drivers/gpu/drm/i915/pxp/intel_pxp_cmd_interface_42.h +++ b/drivers/gpu/drm/i915/pxp/intel_pxp_cmd_interface_42.h @@ -12,6 +12,9 @@ /* PXP-Opcode for Init Session */ #define PXP42_CMDID_INIT_SESSION 0x1e +/* PXP-Opcode for Invalidate Stream Key */ +#define PXP42_CMDID_INVALIDATE_STREAM_KEY 0x00000007 + /* PXP-Input-Packet: Init Session (Arb-Session) */ struct pxp42_create_arb_in { struct pxp_cmd_header header; @@ -25,4 +28,16 @@ struct pxp42_create_arb_out { struct pxp_cmd_header header; } __packed; +/* PXP-Input-Packet: Invalidate Stream Key */ +struct pxp42_inv_stream_key_in { + struct pxp_cmd_header header; + u32 rsvd[3]; +} __packed; + +/* PXP-Output-Packet: Invalidate Stream Key */ +struct pxp42_inv_stream_key_out { + struct pxp_cmd_header header; + u32 rsvd; +} __packed; + #endif /* __INTEL_PXP_FW_INTERFACE_42_H__ */ diff --git a/drivers/gpu/drm/i915/pxp/intel_pxp_cmd_interface_cmn.h b/drivers/gpu/drm/i915/pxp/intel_pxp_cmd_interface_cmn.h index c2f23394f9b8..69e34ec49e78 100644 --- a/drivers/gpu/drm/i915/pxp/intel_pxp_cmd_interface_cmn.h +++ b/drivers/gpu/drm/i915/pxp/intel_pxp_cmd_interface_cmn.h @@ -27,6 +27,9 @@ struct pxp_cmd_header { union { u32 status; /* out */ u32 stream_id; /* in */ +#define PXP_CMDHDR_EXTDATA_SESSION_VALID GENMASK(0, 0) +#define PXP_CMDHDR_EXTDATA_APP_TYPE GENMASK(1, 1) +#define PXP_CMDHDR_EXTDATA_SESSION_ID GENMASK(17, 2) }; /* Length of the message (excluding the header) */ u32 buffer_len; diff --git a/drivers/gpu/drm/i915/pxp/intel_pxp_session.c b/drivers/gpu/drm/i915/pxp/intel_pxp_session.c index 85572360c71a..8eb886d3f2a0 100644 --- a/drivers/gpu/drm/i915/pxp/intel_pxp_session.c +++ b/drivers/gpu/drm/i915/pxp/intel_pxp_session.c @@ -91,10 +91,13 @@ static int pxp_terminate_arb_session_and_global(struct intel_pxp *pxp) { int ret; struct intel_gt *gt = pxp_to_gt(pxp); + u32 active_sip_slots; /* must mark termination in progress calling this function */ GEM_WARN_ON(pxp->arb_is_valid); + active_sip_slots = intel_uncore_read(gt->uncore, GEN12_KCR_SIP); + /* terminate the hw sessions */ ret = intel_pxp_terminate_session(pxp, ARB_SESSION); if (ret) { @@ -110,6 +113,8 @@ static int pxp_terminate_arb_session_and_global(struct intel_pxp *pxp) intel_uncore_write(gt->uncore, PXP_GLOBAL_TERMINATE, 1); + intel_pxp_tee_end_arb_fw_session(pxp, ARB_SESSION); + return ret; } diff --git a/drivers/gpu/drm/i915/pxp/intel_pxp_tee.c b/drivers/gpu/drm/i915/pxp/intel_pxp_tee.c index b0c9170b1395..202ea01cbb88 100644 --- a/drivers/gpu/drm/i915/pxp/intel_pxp_tee.c +++ b/drivers/gpu/drm/i915/pxp/intel_pxp_tee.c @@ -309,3 +309,38 @@ int intel_pxp_tee_cmd_create_arb_session(struct intel_pxp *pxp, return ret; } + +void intel_pxp_tee_end_arb_fw_session(struct intel_pxp *pxp, u32 session_id) +{ + struct drm_i915_private *i915 = pxp_to_gt(pxp)->i915; + struct pxp42_inv_stream_key_in msg_in = {0}; + struct pxp42_inv_stream_key_out msg_out = {0}; + int ret, trials = 0; + +try_again: + memset(&msg_in, 0, sizeof(msg_in)); + memset(&msg_out, 0, sizeof(msg_out)); + msg_in.header.api_version = PXP_APIVER(4, 2); + msg_in.header.command_id = PXP42_CMDID_INVALIDATE_STREAM_KEY; + msg_in.header.buffer_len = sizeof(msg_in) - sizeof(msg_in.header); + + msg_in.header.stream_id = FIELD_PREP(PXP_CMDHDR_EXTDATA_SESSION_VALID, 1); + msg_in.header.stream_id |= FIELD_PREP(PXP_CMDHDR_EXTDATA_APP_TYPE, 0); + msg_in.header.stream_id |= FIELD_PREP(PXP_CMDHDR_EXTDATA_SESSION_ID, session_id); + + ret = intel_pxp_tee_io_message(pxp, + &msg_in, sizeof(msg_in), + &msg_out, sizeof(msg_out), + NULL); + + /* Cleanup coherency between GT and Firmware is critical, so try again if it fails */ + if ((ret || msg_out.header.status != 0x0) && ++trials < 3) + goto try_again; + + if (ret) + drm_err(&i915->drm, "Failed to send tee msg for inv-stream-key-%d, ret=[%d]\n", + session_id, ret); + else if (msg_out.header.status != 0x0) + drm_warn(&i915->drm, "PXP firmware failed inv-stream-key-%d with status 0x%08x\n", + session_id, msg_out.header.status); +} diff --git a/drivers/gpu/drm/i915/pxp/intel_pxp_types.h b/drivers/gpu/drm/i915/pxp/intel_pxp_types.h index f74b1e11a505..4d75b06ea4a0 100644 --- a/drivers/gpu/drm/i915/pxp/intel_pxp_types.h +++ b/drivers/gpu/drm/i915/pxp/intel_pxp_types.h @@ -14,6 +14,8 @@ struct intel_context; struct i915_pxp_component; +#define INTEL_PXP_MAX_HWDRM_SESSIONS 16 + /** * struct intel_pxp - pxp state */ From patchwork Tue Nov 29 00:48:31 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alan Previn X-Patchwork-Id: 13058045 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from gabe.freedesktop.org (gabe.freedesktop.org [131.252.210.177]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 550E3C43217 for ; Tue, 29 Nov 2022 00:46:08 +0000 (UTC) Received: from gabe.freedesktop.org (localhost [127.0.0.1]) by gabe.freedesktop.org (Postfix) with ESMTP id 3472310E358; Tue, 29 Nov 2022 00:46:07 +0000 (UTC) Received: from mga05.intel.com (mga05.intel.com [192.55.52.43]) by gabe.freedesktop.org (Postfix) with ESMTPS id DFA5810E35D for ; Tue, 29 Nov 2022 00:46:03 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1669682763; x=1701218763; h=from:to:subject:date:message-id:in-reply-to:references: mime-version:content-transfer-encoding; bh=x+cR3kBwV+fQo/WkGJGu4ikbProgHxMOdBp/eB57NDM=; b=WADxg5gssSJu8w+u6eUlsIIsIkL53PXUMSDwvsviKMRgmMK9Q8jualik 4sD5krY4jhJHxEMPuMf0ukjmwp9Mhh7qGMTZbKAjfSekqLHnfm16uKSd1 uE2qQuGPYiIJ8mIiu65ii5VE0nzzimTXDokgw4492gNbFs+Jvt6B9+gaP leu8rMM5sVwEzoD/Ez+VP9WwxRg12lgAAru2Fko6q6B1JUQhShFhm2Wy+ SIR61w2/GbUx0yB7w6n6C3ZdWM2wnnKvLN8gVWdiShDoq5cqcZWRZJ+8q 5yVu5sO4e91fpxcq4sDndwavIfv3N2OHViwEPB1xO4NI9+9uxHWX8bpEs A==; X-IronPort-AV: E=McAfee;i="6500,9779,10545"; a="401271780" X-IronPort-AV: E=Sophos;i="5.96,201,1665471600"; d="scan'208";a="401271780" Received: from orsmga001.jf.intel.com ([10.7.209.18]) by fmsmga105.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 28 Nov 2022 16:46:03 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6500,9779,10545"; a="676243288" X-IronPort-AV: E=Sophos;i="5.96,201,1665471600"; d="scan'208";a="676243288" Received: from aalteres-desk.fm.intel.com ([10.80.57.53]) by orsmga001.jf.intel.com with ESMTP; 28 Nov 2022 16:46:02 -0800 From: Alan Previn To: intel-gfx@lists.freedesktop.org Date: Mon, 28 Nov 2022 16:48:31 -0800 Message-Id: <20221129004832.3735764-3-alan.previn.teres.alexis@intel.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20221129004832.3735764-1-alan.previn.teres.alexis@intel.com> References: <20221129004832.3735764-1-alan.previn.teres.alexis@intel.com> MIME-Version: 1.0 Subject: [Intel-gfx] [PATCH v2 2/3] drm/i915/pxp: Trigger the global teardown for before suspending X-BeenThere: intel-gfx@lists.freedesktop.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Intel graphics driver community testing & development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: intel-gfx-bounces@lists.freedesktop.org Sender: "Intel-gfx" A driver bug was recently discovered where the security firmware was receiving internal HW signals indicating that session key expirations had occurred. Architecturally, the firmware was expecting a response from the GuC to acknowledge the event with the firmware side. However the OS was in a suspended state and GuC had been reset. Internal specifications actually required the driver to ensure that all active sessions be properly cleaned up in such cases where the system is suspended and the GuC potentially unable to respond. This patch adds the global teardown code in i915's suspend_prepare code path. Signed-off-by: Alan Previn Reviewed-by: Juston Li --- drivers/gpu/drm/i915/pxp/intel_pxp.c | 60 +++++++++++++++++--- drivers/gpu/drm/i915/pxp/intel_pxp.h | 1 + drivers/gpu/drm/i915/pxp/intel_pxp_pm.c | 2 +- drivers/gpu/drm/i915/pxp/intel_pxp_session.c | 9 ++- drivers/gpu/drm/i915/pxp/intel_pxp_session.h | 5 ++ 5 files changed, 64 insertions(+), 13 deletions(-) diff --git a/drivers/gpu/drm/i915/pxp/intel_pxp.c b/drivers/gpu/drm/i915/pxp/intel_pxp.c index 5efe61f67546..659410ae1b89 100644 --- a/drivers/gpu/drm/i915/pxp/intel_pxp.c +++ b/drivers/gpu/drm/i915/pxp/intel_pxp.c @@ -198,6 +198,55 @@ static bool pxp_component_bound(struct intel_pxp *pxp) return bound; } +static int __pxp_global_teardown_locked(struct intel_pxp *pxp, bool terminate_for_cleanup) +{ + if (terminate_for_cleanup) { + if (!pxp->arb_is_valid) + return 0; + /* + * To ensure synchronous and coherent session teardown completion + * in response to suspend or shutdown triggers, don't user a worker. + */ + intel_pxp_mark_termination_in_progress(pxp); + intel_pxp_terminate(pxp, false); + } else { + if (pxp->arb_is_valid) + return 0; + /* + * If we are not in final termination, and the arb-session is currently + * inactive, we are doing a reset and restart due to some runtime event. + * Use the worker that was designed for this. + */ + pxp_queue_termination(pxp); + } + + if (!wait_for_completion_timeout(&pxp->termination, msecs_to_jiffies(250))) + return -ETIMEDOUT; + + return 0; +} + +void intel_pxp_end(struct intel_pxp *pxp) +{ + struct drm_i915_private *i915 = pxp_to_gt(pxp)->i915; + intel_wakeref_t wakeref; + + if (!intel_pxp_is_enabled(pxp)) + return; + + wakeref = intel_runtime_pm_get(&i915->runtime_pm); + + mutex_lock(&pxp->arb_mutex); + + if (__pxp_global_teardown_locked(pxp, true)) + drm_dbg(&(pxp_to_gt(pxp))->i915->drm, "PXP end timed out\n"); + + mutex_unlock(&pxp->arb_mutex); + + intel_pxp_fini_hw(pxp); + intel_runtime_pm_put(&i915->runtime_pm, wakeref); +} + /* * the arb session is restarted from the irq work when we receive the * termination completion interrupt @@ -214,16 +263,9 @@ int intel_pxp_start(struct intel_pxp *pxp) mutex_lock(&pxp->arb_mutex); - if (pxp->arb_is_valid) - goto unlock; - - pxp_queue_termination(pxp); - - if (!wait_for_completion_timeout(&pxp->termination, - msecs_to_jiffies(250))) { - ret = -ETIMEDOUT; + ret = __pxp_global_teardown_locked(pxp, false); + if (ret) goto unlock; - } /* make sure the compiler doesn't optimize the double access */ barrier(); diff --git a/drivers/gpu/drm/i915/pxp/intel_pxp.h b/drivers/gpu/drm/i915/pxp/intel_pxp.h index bbeb8ed8e211..a06b65850246 100644 --- a/drivers/gpu/drm/i915/pxp/intel_pxp.h +++ b/drivers/gpu/drm/i915/pxp/intel_pxp.h @@ -26,6 +26,7 @@ void intel_pxp_mark_termination_in_progress(struct intel_pxp *pxp); void intel_pxp_tee_end_arb_fw_session(struct intel_pxp *pxp, u32 arb_session_id); int intel_pxp_start(struct intel_pxp *pxp); +void intel_pxp_end(struct intel_pxp *pxp); int intel_pxp_key_check(struct intel_pxp *pxp, struct drm_i915_gem_object *obj, diff --git a/drivers/gpu/drm/i915/pxp/intel_pxp_pm.c b/drivers/gpu/drm/i915/pxp/intel_pxp_pm.c index 6a7d4e2ee138..36af52c28e63 100644 --- a/drivers/gpu/drm/i915/pxp/intel_pxp_pm.c +++ b/drivers/gpu/drm/i915/pxp/intel_pxp_pm.c @@ -14,7 +14,7 @@ void intel_pxp_suspend_prepare(struct intel_pxp *pxp) if (!intel_pxp_is_enabled(pxp)) return; - pxp->arb_is_valid = false; + intel_pxp_end(pxp); intel_pxp_invalidate(pxp); } diff --git a/drivers/gpu/drm/i915/pxp/intel_pxp_session.c b/drivers/gpu/drm/i915/pxp/intel_pxp_session.c index 8eb886d3f2a0..794ccdd00bb9 100644 --- a/drivers/gpu/drm/i915/pxp/intel_pxp_session.c +++ b/drivers/gpu/drm/i915/pxp/intel_pxp_session.c @@ -118,11 +118,14 @@ static int pxp_terminate_arb_session_and_global(struct intel_pxp *pxp) return ret; } -static void pxp_terminate(struct intel_pxp *pxp) +void intel_pxp_terminate(struct intel_pxp *pxp, bool restart_arb) { int ret; - pxp->hw_state_invalidated = true; + if (restart_arb) + pxp->hw_state_invalidated = true; + else + pxp->hw_state_invalidated = false; /* * if we fail to submit the termination there is no point in waiting for @@ -170,7 +173,7 @@ static void pxp_session_work(struct work_struct *work) if (events & PXP_TERMINATION_REQUEST) { events &= ~PXP_TERMINATION_COMPLETE; - pxp_terminate(pxp); + intel_pxp_terminate(pxp, true); } if (events & PXP_TERMINATION_COMPLETE) diff --git a/drivers/gpu/drm/i915/pxp/intel_pxp_session.h b/drivers/gpu/drm/i915/pxp/intel_pxp_session.h index 903ac52cffa1..4f944b63b5b6 100644 --- a/drivers/gpu/drm/i915/pxp/intel_pxp_session.h +++ b/drivers/gpu/drm/i915/pxp/intel_pxp_session.h @@ -12,9 +12,14 @@ struct intel_pxp; #ifdef CONFIG_DRM_I915_PXP void intel_pxp_session_management_init(struct intel_pxp *pxp); +void intel_pxp_terminate(struct intel_pxp *pxp, bool restart_arb); #else static inline void intel_pxp_session_management_init(struct intel_pxp *pxp) { } + +static inline void intel_pxp_terminate(struct intel_pxp *pxp, bool restart_arb) +{ +} #endif #endif /* __INTEL_PXP_SESSION_H__ */ From patchwork Tue Nov 29 00:48:32 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alan Previn X-Patchwork-Id: 13058047 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from gabe.freedesktop.org (gabe.freedesktop.org [131.252.210.177]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id BD8B5C4321E for ; Tue, 29 Nov 2022 00:46:15 +0000 (UTC) Received: from gabe.freedesktop.org (localhost [127.0.0.1]) by gabe.freedesktop.org (Postfix) with ESMTP id 5222B10E35D; Tue, 29 Nov 2022 00:46:08 +0000 (UTC) Received: from mga05.intel.com (mga05.intel.com [192.55.52.43]) by gabe.freedesktop.org (Postfix) with ESMTPS id 0A0E610E35E for ; Tue, 29 Nov 2022 00:46:04 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1669682764; x=1701218764; h=from:to:subject:date:message-id:in-reply-to:references: mime-version:content-transfer-encoding; bh=2/0/UB2F424o917YDrtU0m+TCouJ4FWte4aITf2qwHA=; b=IcHiuOj99nOkMvuEROUeiDcwonVDtyKBbP34zgJ/j2GeWOeEbxIsJGPz IIpf8pDE9ioh+3v4lZLkb3H6QFwmY8cw41nV24/3YHC8YUWaduq0qdijY 3S2cf6q0ml2UVWkizCoOXEQJjmMdS8T8ItEmpu9vcpAyCUhmsDFKgXArj mobxQ4K6lLFfslaoOBIKSaSqoWDjeAfGc1AwP80vLeguNSTdgltd8XMKl FwF0rgPa6AE7LiO9reqIA3vCarRcGmEP7W1QCRnFGDmAME/6fIwJsF9si czsPjt52SmjjQQVwlAQIwPLsx97zrBmU2fGCCArdOAOTR3mFuKbic4FDn g==; X-IronPort-AV: E=McAfee;i="6500,9779,10545"; a="401271781" X-IronPort-AV: E=Sophos;i="5.96,201,1665471600"; d="scan'208";a="401271781" Received: from orsmga001.jf.intel.com ([10.7.209.18]) by fmsmga105.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 28 Nov 2022 16:46:03 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6500,9779,10545"; a="676243289" X-IronPort-AV: E=Sophos;i="5.96,201,1665471600"; d="scan'208";a="676243289" Received: from aalteres-desk.fm.intel.com ([10.80.57.53]) by orsmga001.jf.intel.com with ESMTP; 28 Nov 2022 16:46:03 -0800 From: Alan Previn To: intel-gfx@lists.freedesktop.org Date: Mon, 28 Nov 2022 16:48:32 -0800 Message-Id: <20221129004832.3735764-4-alan.previn.teres.alexis@intel.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20221129004832.3735764-1-alan.previn.teres.alexis@intel.com> References: <20221129004832.3735764-1-alan.previn.teres.alexis@intel.com> MIME-Version: 1.0 Subject: [Intel-gfx] [PATCH v2 3/3] drm/i915/pxp: Pxp hw init should be in resume_complete X-BeenThere: intel-gfx@lists.freedesktop.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Intel graphics driver community testing & development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: intel-gfx-bounces@lists.freedesktop.org Sender: "Intel-gfx" During suspend flow, i915 currently achors' on the pm_suspend_prepare callback as the location where we quiesce the entire GPU and perform all necessary cleanup in order to go into suspend. PXP is also called during this time to perform the arbitration session teardown (with the assurance no additional GEM IOCTLs will come after that could restart the session). However, if other devices or drivers fail their suspend_prepare, the system will not go into suspend and i915 will be expected to resume operation. In this case, we need to re-initialize the PXP hardware and this really should be done within the pm_resume_complete callback which is the correct opposing function in the resume sequence to match pm_suspend_prepare of the suspend sequence. Because this callback is the last thing at the end of resuming we expect little to no impact to the rest of the i915 resume sequence with this change. Signed-off-by: Alan Previn --- drivers/gpu/drm/i915/gem/i915_gem_pm.c | 11 +++++++++++ drivers/gpu/drm/i915/gem/i915_gem_pm.h | 2 ++ drivers/gpu/drm/i915/gt/intel_gt_pm.c | 7 +++++-- drivers/gpu/drm/i915/gt/intel_gt_pm.h | 2 ++ drivers/gpu/drm/i915/i915_driver.c | 16 ++++++++++++++++ drivers/gpu/drm/i915/pxp/intel_pxp_pm.c | 2 +- drivers/gpu/drm/i915/pxp/intel_pxp_pm.h | 6 +++--- 7 files changed, 40 insertions(+), 6 deletions(-) diff --git a/drivers/gpu/drm/i915/gem/i915_gem_pm.c b/drivers/gpu/drm/i915/gem/i915_gem_pm.c index 0d812f4d787d..774a3fcd192a 100644 --- a/drivers/gpu/drm/i915/gem/i915_gem_pm.c +++ b/drivers/gpu/drm/i915/gem/i915_gem_pm.c @@ -87,6 +87,17 @@ static void lmem_recover(struct drm_i915_private *i915) i915_ttm_recover_region(mr); } +void i915_gem_resume_complete(struct drm_i915_private *i915) +{ + struct intel_gt *gt; + unsigned int i; + + GEM_TRACE("%s\n", dev_name(i915->drm.dev)); + + for_each_gt(gt, i915, i) + intel_gt_resume_complete(gt); +} + int i915_gem_backup_suspend(struct drm_i915_private *i915) { int ret; diff --git a/drivers/gpu/drm/i915/gem/i915_gem_pm.h b/drivers/gpu/drm/i915/gem/i915_gem_pm.h index bedf1e95941a..8e791a06392a 100644 --- a/drivers/gpu/drm/i915/gem/i915_gem_pm.h +++ b/drivers/gpu/drm/i915/gem/i915_gem_pm.h @@ -20,6 +20,8 @@ void i915_gem_suspend(struct drm_i915_private *i915); void i915_gem_suspend_late(struct drm_i915_private *i915); int i915_gem_backup_suspend(struct drm_i915_private *i915); +void i915_gem_resume_complete(struct drm_i915_private *i915); + int i915_gem_freeze(struct drm_i915_private *i915); int i915_gem_freeze_late(struct drm_i915_private *i915); diff --git a/drivers/gpu/drm/i915/gt/intel_gt_pm.c b/drivers/gpu/drm/i915/gt/intel_gt_pm.c index 16db85fab0b1..e1941bd36336 100644 --- a/drivers/gpu/drm/i915/gt/intel_gt_pm.c +++ b/drivers/gpu/drm/i915/gt/intel_gt_pm.c @@ -304,8 +304,6 @@ int intel_gt_resume(struct intel_gt *gt) intel_uc_resume(>->uc); - intel_pxp_resume(>->pxp); - user_forcewake(gt, false); out_fw: @@ -335,6 +333,11 @@ static void wait_for_suspend(struct intel_gt *gt) intel_gt_pm_wait_for_idle(gt); } +void intel_gt_resume_complete(struct intel_gt *gt) +{ + intel_pxp_resume_complete(>->pxp); +} + void intel_gt_suspend_prepare(struct intel_gt *gt) { user_forcewake(gt, true); diff --git a/drivers/gpu/drm/i915/gt/intel_gt_pm.h b/drivers/gpu/drm/i915/gt/intel_gt_pm.h index 6c9a46452364..f23388805332 100644 --- a/drivers/gpu/drm/i915/gt/intel_gt_pm.h +++ b/drivers/gpu/drm/i915/gt/intel_gt_pm.h @@ -77,7 +77,9 @@ void intel_gt_pm_fini(struct intel_gt *gt); void intel_gt_suspend_prepare(struct intel_gt *gt); void intel_gt_suspend_late(struct intel_gt *gt); + int intel_gt_resume(struct intel_gt *gt); +void intel_gt_resume_complete(struct intel_gt *gt); void intel_gt_runtime_suspend(struct intel_gt *gt); int intel_gt_runtime_resume(struct intel_gt *gt); diff --git a/drivers/gpu/drm/i915/i915_driver.c b/drivers/gpu/drm/i915/i915_driver.c index 69103ae37779..e0b736bd0b06 100644 --- a/drivers/gpu/drm/i915/i915_driver.c +++ b/drivers/gpu/drm/i915/i915_driver.c @@ -1168,6 +1168,11 @@ static bool suspend_to_idle(struct drm_i915_private *dev_priv) return false; } +static void i915_drm_complete(struct drm_device *dev) +{ + i915_gem_resume_complete(to_i915(dev)); +} + static int i915_drm_prepare(struct drm_device *dev) { struct drm_i915_private *i915 = to_i915(dev); @@ -1470,6 +1475,16 @@ int i915_driver_resume_switcheroo(struct drm_i915_private *i915) return i915_drm_resume(&i915->drm); } +static void i915_pm_complete(struct device *kdev) +{ + struct drm_i915_private *i915 = kdev_to_i915(kdev); + + if (!i915) + dev_err(kdev, "DRM not initialized, aborting suspend.\n"); + + i915_drm_complete(&i915->drm); +} + static int i915_pm_prepare(struct device *kdev) { struct drm_i915_private *i915 = kdev_to_i915(kdev); @@ -1774,6 +1789,7 @@ const struct dev_pm_ops i915_pm_ops = { * PMSG_RESUME] */ .prepare = i915_pm_prepare, + .complete = i915_pm_complete, .suspend = i915_pm_suspend, .suspend_late = i915_pm_suspend_late, .resume_early = i915_pm_resume_early, diff --git a/drivers/gpu/drm/i915/pxp/intel_pxp_pm.c b/drivers/gpu/drm/i915/pxp/intel_pxp_pm.c index 36af52c28e63..bdd984fc4298 100644 --- a/drivers/gpu/drm/i915/pxp/intel_pxp_pm.c +++ b/drivers/gpu/drm/i915/pxp/intel_pxp_pm.c @@ -32,7 +32,7 @@ void intel_pxp_suspend(struct intel_pxp *pxp) } } -void intel_pxp_resume(struct intel_pxp *pxp) +void intel_pxp_resume_complete(struct intel_pxp *pxp) { if (!intel_pxp_is_enabled(pxp)) return; diff --git a/drivers/gpu/drm/i915/pxp/intel_pxp_pm.h b/drivers/gpu/drm/i915/pxp/intel_pxp_pm.h index 586be769104f..06b46f535b42 100644 --- a/drivers/gpu/drm/i915/pxp/intel_pxp_pm.h +++ b/drivers/gpu/drm/i915/pxp/intel_pxp_pm.h @@ -11,7 +11,7 @@ struct intel_pxp; #ifdef CONFIG_DRM_I915_PXP void intel_pxp_suspend_prepare(struct intel_pxp *pxp); void intel_pxp_suspend(struct intel_pxp *pxp); -void intel_pxp_resume(struct intel_pxp *pxp); +void intel_pxp_resume_complete(struct intel_pxp *pxp); void intel_pxp_runtime_suspend(struct intel_pxp *pxp); #else static inline void intel_pxp_suspend_prepare(struct intel_pxp *pxp) @@ -22,7 +22,7 @@ static inline void intel_pxp_suspend(struct intel_pxp *pxp) { } -static inline void intel_pxp_resume(struct intel_pxp *pxp) +static inline void intel_pxp_resume_complete(struct intel_pxp *pxp) { } @@ -32,6 +32,6 @@ static inline void intel_pxp_runtime_suspend(struct intel_pxp *pxp) #endif static inline void intel_pxp_runtime_resume(struct intel_pxp *pxp) { - intel_pxp_resume(pxp); + intel_pxp_resume_complete(pxp); } #endif /* __INTEL_PXP_PM_H__ */