From patchwork Thu Dec 15 15:10:57 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Akhil P Oommen X-Patchwork-Id: 13074282 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id B8EBDC46467 for ; Thu, 15 Dec 2022 15:12:07 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230031AbiLOPMF (ORCPT ); Thu, 15 Dec 2022 10:12:05 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:41202 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230082AbiLOPLv (ORCPT ); Thu, 15 Dec 2022 10:11:51 -0500 Received: from mx0b-0031df01.pphosted.com (mx0b-0031df01.pphosted.com [205.220.180.131]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 6A9F430575; Thu, 15 Dec 2022 07:11:46 -0800 (PST) Received: from pps.filterd (m0279872.ppops.net [127.0.0.1]) by mx0a-0031df01.pphosted.com (8.17.1.19/8.17.1.19) with ESMTP id 2BFAjkI2018902; Thu, 15 Dec 2022 15:11:30 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=quicinc.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-type; s=qcppdkim1; bh=NQjuWdRJhlzvQflQiWmSjH+nZpjlOK16M6kbc7ZMIv4=; b=MjYlOQcCGoJhiESaeavfM6gxfW7Xn+jK53Jip20RGxsTMbT2NB6HQJ7rZ/YSWWVW3ymW 0tOj1sRiZIs/gBWRB2H8bow0imZ3sNiCN7s0zMosGe9LO26sIog4JRoDESqNvYc4Mzap Z9SfllnVXZxM5IVCLQ6kE/azQFIX2VU+DfmU6CSMRPkdpy4sA4qLpgOTorfQQ6jMotWy yDfjtFmaE5afTbGwNgDbJgVK7wemFS4+d7+n6YSvyphYjxazxb9lx+KeRWvYqysd0GO9 R2pTak/MfSb1ba0qN0Jn99NcGq3VXwfiyZDLjnfFcVcohJatE0kydL6KrrKxl7aOVz18 tw== Received: from nalasppmta02.qualcomm.com (Global_NAT1.qualcomm.com [129.46.96.20]) by mx0a-0031df01.pphosted.com (PPS) with ESMTPS id 3mfukhstj5-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Thu, 15 Dec 2022 15:11:29 +0000 Received: from nalasex01a.na.qualcomm.com (nalasex01a.na.qualcomm.com [10.47.209.196]) by NALASPPMTA02.qualcomm.com (8.17.1.5/8.17.1.5) with ESMTPS id 2BFFBSss012483 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Thu, 15 Dec 2022 15:11:28 GMT Received: from hyd-lnxbld559.qualcomm.com (10.80.80.8) by nalasex01a.na.qualcomm.com (10.47.209.196) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.986.36; Thu, 15 Dec 2022 07:11:24 -0800 From: Akhil P Oommen To: freedreno , , , Rob Clark , Ulf Hansson , Bjorn Andersson CC: Akhil P Oommen , Greg Kroah-Hartman , Kevin Hilman , Len Brown , Pavel Machek , "Rafael J. Wysocki" , , Subject: [PATCH 1/5] PM: domains: Allow a genpd consumer to require a synced power off Date: Thu, 15 Dec 2022 20:40:57 +0530 Message-ID: <20221215203751.1.I3e6b1f078ad0f1ca9358c573daa7b70ec132cdbe@changeid> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1671117062-26276-1-git-send-email-quic_akhilpo@quicinc.com> References: <1671117062-26276-1-git-send-email-quic_akhilpo@quicinc.com> MIME-Version: 1.0 X-Originating-IP: [10.80.80.8] X-ClientProxiedBy: nasanex01b.na.qualcomm.com (10.46.141.250) To nalasex01a.na.qualcomm.com (10.47.209.196) X-QCInternal: smtphost X-Proofpoint-Virus-Version: vendor=nai engine=6200 definitions=5800 signatures=585085 X-Proofpoint-GUID: 3ozT3s2LygexvKrtlfkWG3Is6S-kalNQ X-Proofpoint-ORIG-GUID: 3ozT3s2LygexvKrtlfkWG3Is6S-kalNQ X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.205,Aquarius:18.0.923,Hydra:6.0.545,FMLib:17.11.122.1 definitions=2022-12-15_08,2022-12-15_02,2022-06-22_01 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 phishscore=0 clxscore=1015 mlxscore=0 malwarescore=0 impostorscore=0 lowpriorityscore=0 bulkscore=0 mlxlogscore=602 spamscore=0 priorityscore=1501 adultscore=0 suspectscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2212070000 definitions=main-2212150123 Precedence: bulk List-ID: X-Mailing-List: linux-arm-msm@vger.kernel.org From: Ulf Hansson Some genpd providers doesn't ensure that it has turned off at hardware. This is fine until the consumer really requires during some special scenarios that the power domain collapse at hardware before it is turned ON again. An example is the reset sequence of Adreno GPU which requires that the 'gpucc cx gdsc' power domain should move to OFF state in hardware at least once before turning in ON again to clear the internal state. Signed-off-by: Ulf Hansson Signed-off-by: Akhil P Oommen --- @Ulf, I took the liberty to cleanup and post your patch. drivers/base/power/domain.c | 23 +++++++++++++++++++++++ include/linux/pm_domain.h | 5 +++++ 2 files changed, 28 insertions(+) diff --git a/drivers/base/power/domain.c b/drivers/base/power/domain.c index 967bcf9d415e..4fa624218967 100644 --- a/drivers/base/power/domain.c +++ b/drivers/base/power/domain.c @@ -519,6 +519,28 @@ ktime_t dev_pm_genpd_get_next_hrtimer(struct device *dev) } EXPORT_SYMBOL_GPL(dev_pm_genpd_get_next_hrtimer); +/* + * dev_pm_genpd_synced_poweroff - Next power off should be synchronous + * + * @dev: A device that is attached to the genpd. + * + * Allows a consumer of the genpd to notify the provider that the next power off + * should be synchronous. + */ +void dev_pm_genpd_synced_poweroff(struct device *dev) +{ + struct generic_pm_domain *genpd; + + genpd = dev_to_genpd_safe(dev); + if (!genpd) + return; + + genpd_lock(genpd); + genpd->synced_poweroff = true; + genpd_unlock(genpd); +} +EXPORT_SYMBOL_GPL(dev_pm_genpd_synced_poweroff); + static int _genpd_power_on(struct generic_pm_domain *genpd, bool timed) { unsigned int state_idx = genpd->state_idx; @@ -562,6 +584,7 @@ static int _genpd_power_on(struct generic_pm_domain *genpd, bool timed) out: raw_notifier_call_chain(&genpd->power_notifiers, GENPD_NOTIFY_ON, NULL); + genpd->synced_poweroff = false; return 0; err: raw_notifier_call_chain(&genpd->power_notifiers, GENPD_NOTIFY_OFF, diff --git a/include/linux/pm_domain.h b/include/linux/pm_domain.h index 1cd41bdf73cf..f776fb93eaa0 100644 --- a/include/linux/pm_domain.h +++ b/include/linux/pm_domain.h @@ -136,6 +136,7 @@ struct generic_pm_domain { unsigned int prepared_count; /* Suspend counter of prepared devices */ unsigned int performance_state; /* Aggregated max performance state */ cpumask_var_t cpus; /* A cpumask of the attached CPUs */ + bool synced_poweroff; /* A consumer needs a synced poweroff */ int (*power_off)(struct generic_pm_domain *domain); int (*power_on)(struct generic_pm_domain *domain); struct raw_notifier_head power_notifiers; /* Power on/off notifiers */ @@ -235,6 +236,7 @@ int dev_pm_genpd_add_notifier(struct device *dev, struct notifier_block *nb); int dev_pm_genpd_remove_notifier(struct device *dev); void dev_pm_genpd_set_next_wakeup(struct device *dev, ktime_t next); ktime_t dev_pm_genpd_get_next_hrtimer(struct device *dev); +void dev_pm_genpd_synced_poweroff(struct device *dev); extern struct dev_power_governor simple_qos_governor; extern struct dev_power_governor pm_domain_always_on_gov; @@ -300,6 +302,9 @@ static inline ktime_t dev_pm_genpd_get_next_hrtimer(struct device *dev) { return KTIME_MAX; } +static inline void dev_pm_genpd_synced_poweroff(struct device *dev) +{ } + #define simple_qos_governor (*(struct dev_power_governor *)(NULL)) #define pm_domain_always_on_gov (*(struct dev_power_governor *)(NULL)) #endif From patchwork Thu Dec 15 15:10:58 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Akhil P Oommen X-Patchwork-Id: 13074281 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 237EFC4167B for ; Thu, 15 Dec 2022 15:12:07 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229624AbiLOPME (ORCPT ); Thu, 15 Dec 2022 10:12:04 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:41204 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230080AbiLOPLv (ORCPT ); Thu, 15 Dec 2022 10:11:51 -0500 Received: from mx0b-0031df01.pphosted.com (mx0b-0031df01.pphosted.com [205.220.180.131]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 6955A3055D; Thu, 15 Dec 2022 07:11:45 -0800 (PST) Received: from pps.filterd (m0279873.ppops.net [127.0.0.1]) by mx0a-0031df01.pphosted.com (8.17.1.19/8.17.1.19) with ESMTP id 2BFE6fwu017300; Thu, 15 Dec 2022 15:11:34 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=quicinc.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-type; s=qcppdkim1; bh=8O+QTXU6K0VLHiPY0M+KwUb71ZkMEdSLJ8ngjTRFyuo=; b=I5yY8PHDhp6PlVPwEEWoyvQl7hS1jG+txYlyq4LuTf2ijPA9/gif5a2qjiJGX6ljGt7Q JlFLXQ7lIAUFV0vD+QUFtwiksvWdCEej5Dh6QsczTsjUhtbxaSMdWWxuw+787XVtEgxo Ggft1LJ6cVcQ+BbPsOfdOXoIztP0bH763CTBQtIBi/TMp4j2tbkFPh/j7gXcttZyPO7x O2qzw2c7hf459rr0kC0O7VosEkJrCW3kwUNIqHu11hyVdyoNLo9uJZiVR1ByIHLBSpEw zQWH20FGnUFV4EwGuloRViAabjkuKiZ2yQMPiLifznDuWwKDPmIS/XXgcdfz1Xm2xN71 Xw== Received: from nalasppmta04.qualcomm.com (Global_NAT1.qualcomm.com [129.46.96.20]) by mx0a-0031df01.pphosted.com (PPS) with ESMTPS id 3mfxse9dp7-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Thu, 15 Dec 2022 15:11:34 +0000 Received: from nalasex01a.na.qualcomm.com (nalasex01a.na.qualcomm.com [10.47.209.196]) by NALASPPMTA04.qualcomm.com (8.17.1.5/8.17.1.5) with ESMTPS id 2BFFBX3l024897 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Thu, 15 Dec 2022 15:11:33 GMT Received: from hyd-lnxbld559.qualcomm.com (10.80.80.8) by nalasex01a.na.qualcomm.com (10.47.209.196) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.986.36; Thu, 15 Dec 2022 07:11:29 -0800 From: Akhil P Oommen To: freedreno , , , Rob Clark , Ulf Hansson , Bjorn Andersson CC: Akhil P Oommen , Andy Gross , Konrad Dybcio , Michael Turquette , Stephen Boyd , , Subject: [PATCH 2/5] clk: qcom: gdsc: Support 'synced_poweroff' genpd flag Date: Thu, 15 Dec 2022 20:40:58 +0530 Message-ID: <20221215203751.2.Ic128c1df50b7fc9a6b919932a3b41a799b5ed5e8@changeid> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1671117062-26276-1-git-send-email-quic_akhilpo@quicinc.com> References: <1671117062-26276-1-git-send-email-quic_akhilpo@quicinc.com> MIME-Version: 1.0 X-Originating-IP: [10.80.80.8] X-ClientProxiedBy: nasanex01b.na.qualcomm.com (10.46.141.250) To nalasex01a.na.qualcomm.com (10.47.209.196) X-QCInternal: smtphost X-Proofpoint-Virus-Version: vendor=nai engine=6200 definitions=5800 signatures=585085 X-Proofpoint-ORIG-GUID: vs2Lql3VQhP6jb_phHc5QEzo2fnlkxVT X-Proofpoint-GUID: vs2Lql3VQhP6jb_phHc5QEzo2fnlkxVT X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.205,Aquarius:18.0.923,Hydra:6.0.545,FMLib:17.11.122.1 definitions=2022-12-15_08,2022-12-15_02,2022-06-22_01 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 impostorscore=0 adultscore=0 clxscore=1015 spamscore=0 phishscore=0 priorityscore=1501 malwarescore=0 suspectscore=0 mlxscore=0 bulkscore=0 lowpriorityscore=0 mlxlogscore=999 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2212070000 definitions=main-2212150123 Precedence: bulk List-ID: X-Mailing-List: linux-arm-msm@vger.kernel.org Add support for the newly added 'synced_poweroff' genpd flag. This allows some clients (like adreno gpu driver) to request gdsc driver to ensure a votable gdsc (like gpucc cx gdsc) has collapsed at hardware. Signed-off-by: Akhil P Oommen --- drivers/clk/qcom/gdsc.c | 11 ++++++----- 1 file changed, 6 insertions(+), 5 deletions(-) diff --git a/drivers/clk/qcom/gdsc.c b/drivers/clk/qcom/gdsc.c index 9e4d6ce891aa..575019ba4768 100644 --- a/drivers/clk/qcom/gdsc.c +++ b/drivers/clk/qcom/gdsc.c @@ -136,7 +136,8 @@ static int gdsc_update_collapse_bit(struct gdsc *sc, bool val) return 0; } -static int gdsc_toggle_logic(struct gdsc *sc, enum gdsc_status status) +static int gdsc_toggle_logic(struct gdsc *sc, enum gdsc_status status, + bool force_sync) { int ret; @@ -149,7 +150,7 @@ static int gdsc_toggle_logic(struct gdsc *sc, enum gdsc_status status) ret = gdsc_update_collapse_bit(sc, status == GDSC_OFF); /* If disabling votable gdscs, don't poll on status */ - if ((sc->flags & VOTABLE) && status == GDSC_OFF) { + if ((sc->flags & VOTABLE) && status == GDSC_OFF && !force_sync) { /* * Add a short delay here to ensure that an enable * right after it was disabled does not put it in an @@ -275,7 +276,7 @@ static int gdsc_enable(struct generic_pm_domain *domain) gdsc_deassert_clamp_io(sc); } - ret = gdsc_toggle_logic(sc, GDSC_ON); + ret = gdsc_toggle_logic(sc, GDSC_ON, false); if (ret) return ret; @@ -352,7 +353,7 @@ static int gdsc_disable(struct generic_pm_domain *domain) if (sc->pwrsts == PWRSTS_RET_ON) return 0; - ret = gdsc_toggle_logic(sc, GDSC_OFF); + ret = gdsc_toggle_logic(sc, GDSC_OFF, domain->synced_poweroff); if (ret) return ret; @@ -392,7 +393,7 @@ static int gdsc_init(struct gdsc *sc) /* Force gdsc ON if only ON state is supported */ if (sc->pwrsts == PWRSTS_ON) { - ret = gdsc_toggle_logic(sc, GDSC_ON); + ret = gdsc_toggle_logic(sc, GDSC_ON, false); if (ret) return ret; } From patchwork Thu Dec 15 15:10:59 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Akhil P Oommen X-Patchwork-Id: 13074285 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id C8850C2D0CC for ; Thu, 15 Dec 2022 15:12:18 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230053AbiLOPMP (ORCPT ); Thu, 15 Dec 2022 10:12:15 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:41356 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230049AbiLOPML (ORCPT ); Thu, 15 Dec 2022 10:12:11 -0500 Received: from mx0a-0031df01.pphosted.com (mx0a-0031df01.pphosted.com [205.220.168.131]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 7FA3F21830; Thu, 15 Dec 2022 07:12:10 -0800 (PST) Received: from pps.filterd (m0279862.ppops.net [127.0.0.1]) by mx0a-0031df01.pphosted.com (8.17.1.19/8.17.1.19) with ESMTP id 2BFCKBTo030153; Thu, 15 Dec 2022 15:11:39 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=quicinc.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-type; s=qcppdkim1; bh=GCL9P5RwqHQG6cimmUsVySkZPb6XMDtNy9pxn8Zyaxg=; b=Q15VTjWpdqSrVN+yXCwWXX5hfE9Zp0JZ5tNiIvqeXbS63nf9SvCrNZhz7Q0K0UFj5K2v jkjEHok3/7WqhPKZriyMjy/NXLZLCwqN1Y3HF6rfFlTUJiG69mHNqOk1OqzCEAwK5H9E VDAjIuUkdCCE36OXNIF+Riy9oDupzMOBHxnvsPsfrlv9cXnSZnd0MymJbqO2oYprhw2k njjIo+FZF6vQ/BaCR23hliqlZ1Jc8adzgLOH8PrPkdFAFWHhxU8V+PgJa/VW2WxqUR/D hC3iUIeak/lwPitQhuAi8lJ7p8H3cUmF1uVts7HAE2cSmnKGJSId2PlIdJCQ4wIs70FG yA== Received: from nalasppmta04.qualcomm.com (Global_NAT1.qualcomm.com [129.46.96.20]) by mx0a-0031df01.pphosted.com (PPS) with ESMTPS id 3mf6rfcyt3-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Thu, 15 Dec 2022 15:11:39 +0000 Received: from nalasex01a.na.qualcomm.com (nalasex01a.na.qualcomm.com [10.47.209.196]) by NALASPPMTA04.qualcomm.com (8.17.1.5/8.17.1.5) with ESMTPS id 2BFFBcPF024955 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Thu, 15 Dec 2022 15:11:38 GMT Received: from hyd-lnxbld559.qualcomm.com (10.80.80.8) by nalasex01a.na.qualcomm.com (10.47.209.196) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.986.36; Thu, 15 Dec 2022 07:11:33 -0800 From: Akhil P Oommen To: freedreno , , , Rob Clark , Ulf Hansson , Bjorn Andersson CC: Akhil P Oommen , Abhinav Kumar , Daniel Vetter , David Airlie , Dmitry Baryshkov , "Douglas Anderson" , Geert Uytterhoeven , Guenter Roeck , Konrad Dybcio , Sean Paul , Subject: [PATCH 3/5] drm/msm/a6xx: Vote for cx gdsc from gpu driver Date: Thu, 15 Dec 2022 20:40:59 +0530 Message-ID: <20221215203751.3.I7f545d8494dcdbe6e96a15fbe8aaf5bb0c003d50@changeid> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1671117062-26276-1-git-send-email-quic_akhilpo@quicinc.com> References: <1671117062-26276-1-git-send-email-quic_akhilpo@quicinc.com> MIME-Version: 1.0 X-Originating-IP: [10.80.80.8] X-ClientProxiedBy: nasanex01b.na.qualcomm.com (10.46.141.250) To nalasex01a.na.qualcomm.com (10.47.209.196) X-QCInternal: smtphost X-Proofpoint-Virus-Version: vendor=nai engine=6200 definitions=5800 signatures=585085 X-Proofpoint-GUID: rjRpz4j6zSOaJvpH6ajO2ZgX2ocoPTBV X-Proofpoint-ORIG-GUID: rjRpz4j6zSOaJvpH6ajO2ZgX2ocoPTBV X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.205,Aquarius:18.0.923,Hydra:6.0.545,FMLib:17.11.122.1 definitions=2022-12-15_08,2022-12-15_02,2022-06-22_01 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 priorityscore=1501 clxscore=1015 mlxscore=0 impostorscore=0 adultscore=0 spamscore=0 phishscore=0 mlxlogscore=999 malwarescore=0 suspectscore=0 bulkscore=0 lowpriorityscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2212070000 definitions=main-2212150123 Precedence: bulk List-ID: X-Mailing-List: linux-arm-msm@vger.kernel.org When a device has multiple power domains, dev->power_domain is left empty during probe. That didn't cause any issue so far because we are freeloading on smmu driver's vote on cx gdsc. Instead of that, create a device_link between cx genpd device and gmu device to keep a vote from gpu driver. Before this patch: localhost ~ # cat /sys/kernel/debug/pm_genpd/pm_genpd_summary gx_gdsc on 0 /devices/genpd:1:3d6a000.gmu active 0 cx_gdsc on 0 /devices/platform/soc@0/3da0000.iommu active 0 After this patch: localhost ~ # cat /sys/kernel/debug/pm_genpd/pm_genpd_summary gx_gdsc on 0 /devices/genpd:1:3d6a000.gmu active 0 cx_gdsc on 0 /devices/platform/soc@0/3da0000.iommu active 0 /devices/genpd:0:3d6a000.gmu active 0 Signed-off-by: Akhil P Oommen --- drivers/gpu/drm/msm/adreno/a6xx_gmu.c | 31 +++++++++++++++++++++++++++---- drivers/gpu/drm/msm/adreno/a6xx_gmu.h | 1 + 2 files changed, 28 insertions(+), 4 deletions(-) diff --git a/drivers/gpu/drm/msm/adreno/a6xx_gmu.c b/drivers/gpu/drm/msm/adreno/a6xx_gmu.c index 6484b97c5344..1580d0090f35 100644 --- a/drivers/gpu/drm/msm/adreno/a6xx_gmu.c +++ b/drivers/gpu/drm/msm/adreno/a6xx_gmu.c @@ -1479,6 +1479,12 @@ void a6xx_gmu_remove(struct a6xx_gpu *a6xx_gpu) pm_runtime_force_suspend(gmu->dev); + /* + * Since cxpd is a virt device, the devlink with gmu-dev will be removed + * automatically when we do detach + */ + dev_pm_domain_detach(gmu->cxpd, false); + if (!IS_ERR_OR_NULL(gmu->gxpd)) { pm_runtime_disable(gmu->gxpd); dev_pm_domain_detach(gmu->gxpd, false); @@ -1605,8 +1611,10 @@ int a6xx_gmu_init(struct a6xx_gpu *a6xx_gpu, struct device_node *node) if (adreno_is_a650_family(adreno_gpu)) { gmu->rscc = a6xx_gmu_get_mmio(pdev, "rscc"); - if (IS_ERR(gmu->rscc)) + if (IS_ERR(gmu->rscc)) { + ret = -ENODEV; goto err_mmio; + } } else { gmu->rscc = gmu->mmio + 0x23000; } @@ -1615,8 +1623,22 @@ int a6xx_gmu_init(struct a6xx_gpu *a6xx_gpu, struct device_node *node) gmu->hfi_irq = a6xx_gmu_get_irq(gmu, pdev, "hfi", a6xx_hfi_irq); gmu->gmu_irq = a6xx_gmu_get_irq(gmu, pdev, "gmu", a6xx_gmu_irq); - if (gmu->hfi_irq < 0 || gmu->gmu_irq < 0) + if (gmu->hfi_irq < 0 || gmu->gmu_irq < 0) { + ret = -ENODEV; + goto err_mmio; + } + + gmu->cxpd = dev_pm_domain_attach_by_name(gmu->dev, "cx"); + if (IS_ERR(gmu->cxpd)) { + ret = PTR_ERR(gmu->cxpd); goto err_mmio; + } + + if (!device_link_add(gmu->dev, gmu->cxpd, + DL_FLAG_PM_RUNTIME)) { + ret = -ENODEV; + goto detach_cxpd; + } /* * Get a link to the GX power domain to reset the GPU in case of GMU @@ -1634,6 +1656,9 @@ int a6xx_gmu_init(struct a6xx_gpu *a6xx_gpu, struct device_node *node) return 0; +detach_cxpd: + dev_pm_domain_detach(gmu->cxpd, false); + err_mmio: iounmap(gmu->mmio); if (platform_get_resource_byname(pdev, IORESOURCE_MEM, "rscc")) @@ -1641,8 +1666,6 @@ int a6xx_gmu_init(struct a6xx_gpu *a6xx_gpu, struct device_node *node) free_irq(gmu->gmu_irq, gmu); free_irq(gmu->hfi_irq, gmu); - ret = -ENODEV; - err_memory: a6xx_gmu_memory_free(gmu); err_put_device: diff --git a/drivers/gpu/drm/msm/adreno/a6xx_gmu.h b/drivers/gpu/drm/msm/adreno/a6xx_gmu.h index e034935b3986..5a42dd4dd31f 100644 --- a/drivers/gpu/drm/msm/adreno/a6xx_gmu.h +++ b/drivers/gpu/drm/msm/adreno/a6xx_gmu.h @@ -56,6 +56,7 @@ struct a6xx_gmu { int gmu_irq; struct device *gxpd; + struct device *cxpd; int idle_level; From patchwork Thu Dec 15 15:11:00 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Akhil P Oommen X-Patchwork-Id: 13074283 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id B196CC4332F for ; Thu, 15 Dec 2022 15:12:09 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230089AbiLOPMH (ORCPT ); Thu, 15 Dec 2022 10:12:07 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:40962 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229911AbiLOPL6 (ORCPT ); Thu, 15 Dec 2022 10:11:58 -0500 Received: from mx0b-0031df01.pphosted.com (mx0b-0031df01.pphosted.com [205.220.180.131]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 7C80321830; Thu, 15 Dec 2022 07:11:56 -0800 (PST) Received: from pps.filterd (m0279871.ppops.net [127.0.0.1]) by mx0a-0031df01.pphosted.com (8.17.1.19/8.17.1.19) with ESMTP id 2BFCRATg029930; Thu, 15 Dec 2022 15:11:45 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=quicinc.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-type; s=qcppdkim1; bh=+QSJRhiAUn3DfW9c6T76XAzS/vJbi4cRpllQq+m6h3k=; b=hRTZr/GazXWlrQslyMd8/V8gnaUy5C+Tt/DOW2os/t+nXmVTOYf4PGVkeiTMtY5gmjKA IMaS8HYXdvr5fwK2uZh+aKsZ22kheBT9i3HFcrKtrPlbB0YQeRPB7jnIhvuk7ig3kjZb om9SoIMsJuz+yDQa9ag4QPq2OgwAZv/uwukjQEwI4GL0xIA0DhhXvMFeAN3Dy0V2g1WP 0XEOUuneDWAZ3TwUtf/bui2MZDmmMyY6mf0MqkGWUsHnDn5GB8IReIkWznMDLd2wZVvz esJVtaa9MyZkH87LqOsxvi1xzoTU7vl9Nk4EzXxDEJUQ8hi5gPrV6x3eJMp6VeQ694PQ Dw== Received: from nalasppmta04.qualcomm.com (Global_NAT1.qualcomm.com [129.46.96.20]) by mx0a-0031df01.pphosted.com (PPS) with ESMTPS id 3mg2890w25-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Thu, 15 Dec 2022 15:11:45 +0000 Received: from nalasex01a.na.qualcomm.com (nalasex01a.na.qualcomm.com [10.47.209.196]) by NALASPPMTA04.qualcomm.com (8.17.1.5/8.17.1.5) with ESMTPS id 2BFFBiOJ025388 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Thu, 15 Dec 2022 15:11:44 GMT Received: from hyd-lnxbld559.qualcomm.com (10.80.80.8) by nalasex01a.na.qualcomm.com (10.47.209.196) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.986.36; Thu, 15 Dec 2022 07:11:39 -0800 From: Akhil P Oommen To: freedreno , , , Rob Clark , Ulf Hansson , Bjorn Andersson CC: Akhil P Oommen , Abhinav Kumar , Chia-I Wu , Daniel Vetter , David Airlie , Dmitry Baryshkov , Douglas Anderson , Konrad Dybcio , Philipp Zabel , Sean Paul , Subject: [PATCH 4/5] drm/msm/a6xx: Remove cx gdsc polling using 'reset' Date: Thu, 15 Dec 2022 20:41:00 +0530 Message-ID: <20221215203751.4.I96e0bf9eaf96dd866111c1eec8a4c9b70fd7cbcb@changeid> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1671117062-26276-1-git-send-email-quic_akhilpo@quicinc.com> References: <1671117062-26276-1-git-send-email-quic_akhilpo@quicinc.com> MIME-Version: 1.0 X-Originating-IP: [10.80.80.8] X-ClientProxiedBy: nasanex01b.na.qualcomm.com (10.46.141.250) To nalasex01a.na.qualcomm.com (10.47.209.196) X-QCInternal: smtphost X-Proofpoint-Virus-Version: vendor=nai engine=6200 definitions=5800 signatures=585085 X-Proofpoint-GUID: cjk_DWjdX7c_hIFxIRTJ1eA2dL_b1Hpy X-Proofpoint-ORIG-GUID: cjk_DWjdX7c_hIFxIRTJ1eA2dL_b1Hpy X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.205,Aquarius:18.0.923,Hydra:6.0.545,FMLib:17.11.122.1 definitions=2022-12-15_08,2022-12-15_02,2022-06-22_01 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 clxscore=1015 mlxscore=0 lowpriorityscore=0 bulkscore=0 spamscore=0 priorityscore=1501 phishscore=0 impostorscore=0 suspectscore=0 malwarescore=0 adultscore=0 mlxlogscore=999 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2212070000 definitions=main-2212150123 Precedence: bulk List-ID: X-Mailing-List: linux-arm-msm@vger.kernel.org Remove the unused 'reset' interface which was supposed to help to ensure that cx gdsc has collapsed during gpu recovery. This is was not enabled so far due to missing gpucc driver support. Similar functionality using genpd framework will be implemented in the upcoming patch. Signed-off-by: Akhil P Oommen --- drivers/gpu/drm/msm/adreno/a6xx_gpu.c | 4 ---- drivers/gpu/drm/msm/msm_gpu.c | 4 ---- drivers/gpu/drm/msm/msm_gpu.h | 4 ---- 3 files changed, 12 deletions(-) diff --git a/drivers/gpu/drm/msm/adreno/a6xx_gpu.c b/drivers/gpu/drm/msm/adreno/a6xx_gpu.c index 36c8fb699b56..4b16e75dfa50 100644 --- a/drivers/gpu/drm/msm/adreno/a6xx_gpu.c +++ b/drivers/gpu/drm/msm/adreno/a6xx_gpu.c @@ -10,7 +10,6 @@ #include #include -#include #include #define GPU_PAS_ID 13 @@ -1298,9 +1297,6 @@ static void a6xx_recover(struct msm_gpu *gpu) /* And the final one from recover worker */ pm_runtime_put_sync(&gpu->pdev->dev); - /* Call into gpucc driver to poll for cx gdsc collapse */ - reset_control_reset(gpu->cx_collapse); - pm_runtime_use_autosuspend(&gpu->pdev->dev); if (active_submits) diff --git a/drivers/gpu/drm/msm/msm_gpu.c b/drivers/gpu/drm/msm/msm_gpu.c index 30ed45af76ad..97e1319d4577 100644 --- a/drivers/gpu/drm/msm/msm_gpu.c +++ b/drivers/gpu/drm/msm/msm_gpu.c @@ -16,7 +16,6 @@ #include #include #include -#include #include /* @@ -933,9 +932,6 @@ int msm_gpu_init(struct drm_device *drm, struct platform_device *pdev, if (IS_ERR(gpu->gpu_cx)) gpu->gpu_cx = NULL; - gpu->cx_collapse = devm_reset_control_get_optional_exclusive(&pdev->dev, - "cx_collapse"); - gpu->pdev = pdev; platform_set_drvdata(pdev, &gpu->adreno_smmu); diff --git a/drivers/gpu/drm/msm/msm_gpu.h b/drivers/gpu/drm/msm/msm_gpu.h index 651786bc55e5..fa9e34d02c91 100644 --- a/drivers/gpu/drm/msm/msm_gpu.h +++ b/drivers/gpu/drm/msm/msm_gpu.h @@ -13,7 +13,6 @@ #include #include #include -#include #include "msm_drv.h" #include "msm_fence.h" @@ -282,9 +281,6 @@ struct msm_gpu { bool hw_apriv; struct thermal_cooling_device *cooling; - - /* To poll for cx gdsc collapse during gpu recovery */ - struct reset_control *cx_collapse; }; static inline struct msm_gpu *dev_to_gpu(struct device *dev) From patchwork Thu Dec 15 15:11:01 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Akhil P Oommen X-Patchwork-Id: 13074286 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 84AC0C2D0CC for ; Thu, 15 Dec 2022 15:12:47 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230143AbiLOPMq (ORCPT ); Thu, 15 Dec 2022 10:12:46 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:41466 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230099AbiLOPMR (ORCPT ); Thu, 15 Dec 2022 10:12:17 -0500 Received: from mx0b-0031df01.pphosted.com (mx0b-0031df01.pphosted.com [205.220.180.131]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id F35C02FBFF; Thu, 15 Dec 2022 07:12:13 -0800 (PST) Received: from pps.filterd (m0279870.ppops.net [127.0.0.1]) by mx0a-0031df01.pphosted.com (8.17.1.19/8.17.1.19) with ESMTP id 2BFE2fkI022795; Thu, 15 Dec 2022 15:11:53 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=quicinc.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-type; s=qcppdkim1; bh=+TNCVzAQc98dTTmqznrkrdANu0/8YBru4BYnBEttDw4=; b=hdst9ulbSI8EKLG2AfSiiebuGOmV7B8u+j6qYdk6JZ5HXf3T5cVgrO/CKd0E2NFlZFaj DhtUG9py5pTS/jwmFUVi8QM/vATFOJWEMMTcnitvA2at54nYXUFZyI3i2JzpdI1U6gjA EWMjPml/UmAfleqzWYMh31dJcCQhOuBWLbMZuNY1ZVvJ0K+E7iE5a5RGUC3ovhnYyXqS zu2S+JwY1/Sgke6i9/eOCwoQz//xBO4Qd7zUO/quTrD+7bu7SQqh57JP+rg6cwcQ/3QD Lln2C1tfW3h4XZ52ugcemPyVqv7YRXFC2crZ1A7MvZJl+SyeICAizDvEHu0h70MR68Bu RA== Received: from nalasppmta01.qualcomm.com (Global_NAT1.qualcomm.com [129.46.96.20]) by mx0a-0031df01.pphosted.com (PPS) with ESMTPS id 3mf6rrmxk0-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Thu, 15 Dec 2022 15:11:53 +0000 Received: from nalasex01a.na.qualcomm.com (nalasex01a.na.qualcomm.com [10.47.209.196]) by NALASPPMTA01.qualcomm.com (8.17.1.5/8.17.1.5) with ESMTPS id 2BFFBoSB003106 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Thu, 15 Dec 2022 15:11:51 GMT Received: from hyd-lnxbld559.qualcomm.com (10.80.80.8) by nalasex01a.na.qualcomm.com (10.47.209.196) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.986.36; Thu, 15 Dec 2022 07:11:44 -0800 From: Akhil P Oommen To: freedreno , , , Rob Clark , Ulf Hansson , Bjorn Andersson CC: Akhil P Oommen , Abhinav Kumar , Chia-I Wu , Daniel Vetter , David Airlie , Dmitry Baryshkov , Douglas Anderson , Geert Uytterhoeven , Guenter Roeck , Konrad Dybcio , Sean Paul , Subject: [PATCH 5/5] drm/msm/a6xx: Use genpd notifier to ensure cx-gdsc collapse Date: Thu, 15 Dec 2022 20:41:01 +0530 Message-ID: <20221215203751.5.I9e10545c6a448d5eb1b734839b871d1b3146dac3@changeid> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1671117062-26276-1-git-send-email-quic_akhilpo@quicinc.com> References: <1671117062-26276-1-git-send-email-quic_akhilpo@quicinc.com> MIME-Version: 1.0 X-Originating-IP: [10.80.80.8] X-ClientProxiedBy: nasanex01b.na.qualcomm.com (10.46.141.250) To nalasex01a.na.qualcomm.com (10.47.209.196) X-QCInternal: smtphost X-Proofpoint-Virus-Version: vendor=nai engine=6200 definitions=5800 signatures=585085 X-Proofpoint-GUID: zeDPXzdPU7fKN7rVHZn3xom3pZpOjTQV X-Proofpoint-ORIG-GUID: zeDPXzdPU7fKN7rVHZn3xom3pZpOjTQV X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.205,Aquarius:18.0.923,Hydra:6.0.545,FMLib:17.11.122.1 definitions=2022-12-15_08,2022-12-15_02,2022-06-22_01 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 mlxlogscore=999 suspectscore=0 phishscore=0 malwarescore=0 impostorscore=0 adultscore=0 clxscore=1015 priorityscore=1501 lowpriorityscore=0 bulkscore=0 mlxscore=0 spamscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2212070000 definitions=main-2212150123 Precedence: bulk List-ID: X-Mailing-List: linux-arm-msm@vger.kernel.org As per the recommended recovery sequence of adreno gpu, cx gdsc should collapse at hardware before it is turned back ON. This helps to clear out the stale states in hardware before it is reinitialized. Use the genpd notifier along with the newly introduced dev_pm_genpd_synced_poweroff() api to ensure that cx gdsc has collapsed before we turn it back ON. Signed-off-by: Akhil P Oommen --- drivers/gpu/drm/msm/adreno/a6xx_gmu.c | 15 +++++++++++++++ drivers/gpu/drm/msm/adreno/a6xx_gmu.h | 6 ++++++ drivers/gpu/drm/msm/adreno/a6xx_gpu.c | 11 +++++++++++ 3 files changed, 32 insertions(+) diff --git a/drivers/gpu/drm/msm/adreno/a6xx_gmu.c b/drivers/gpu/drm/msm/adreno/a6xx_gmu.c index 1580d0090f35..c03830957c26 100644 --- a/drivers/gpu/drm/msm/adreno/a6xx_gmu.c +++ b/drivers/gpu/drm/msm/adreno/a6xx_gmu.c @@ -1507,6 +1507,17 @@ void a6xx_gmu_remove(struct a6xx_gpu *a6xx_gpu) gmu->initialized = false; } +static int cxpd_notifier_cb(struct notifier_block *nb, + unsigned long action, void *data) +{ + struct a6xx_gmu *gmu = container_of(nb, struct a6xx_gmu, pd_nb); + + if (action == GENPD_NOTIFY_OFF) + complete_all(&gmu->pd_gate); + + return 0; +} + int a6xx_gmu_init(struct a6xx_gpu *a6xx_gpu, struct device_node *node) { struct adreno_gpu *adreno_gpu = &a6xx_gpu->base; @@ -1640,6 +1651,10 @@ int a6xx_gmu_init(struct a6xx_gpu *a6xx_gpu, struct device_node *node) goto detach_cxpd; } + init_completion(&gmu->pd_gate); + complete_all(&gmu->pd_gate); + gmu->pd_nb.notifier_call = cxpd_notifier_cb; + /* * Get a link to the GX power domain to reset the GPU in case of GMU * crash diff --git a/drivers/gpu/drm/msm/adreno/a6xx_gmu.h b/drivers/gpu/drm/msm/adreno/a6xx_gmu.h index 5a42dd4dd31f..0bc3eb443fec 100644 --- a/drivers/gpu/drm/msm/adreno/a6xx_gmu.h +++ b/drivers/gpu/drm/msm/adreno/a6xx_gmu.h @@ -4,8 +4,10 @@ #ifndef _A6XX_GMU_H_ #define _A6XX_GMU_H_ +#include #include #include +#include #include "msm_drv.h" #include "a6xx_hfi.h" @@ -90,6 +92,10 @@ struct a6xx_gmu { bool initialized; bool hung; bool legacy; /* a618 or a630 */ + + /* For power domain callback */ + struct notifier_block pd_nb; + struct completion pd_gate; }; static inline u32 gmu_read(struct a6xx_gmu *gmu, u32 offset) diff --git a/drivers/gpu/drm/msm/adreno/a6xx_gpu.c b/drivers/gpu/drm/msm/adreno/a6xx_gpu.c index 4b16e75dfa50..dd618b099110 100644 --- a/drivers/gpu/drm/msm/adreno/a6xx_gpu.c +++ b/drivers/gpu/drm/msm/adreno/a6xx_gpu.c @@ -10,6 +10,7 @@ #include #include +#include #include #define GPU_PAS_ID 13 @@ -1258,6 +1259,7 @@ static void a6xx_recover(struct msm_gpu *gpu) { struct adreno_gpu *adreno_gpu = to_adreno_gpu(gpu); struct a6xx_gpu *a6xx_gpu = to_a6xx_gpu(adreno_gpu); + struct a6xx_gmu *gmu = &a6xx_gpu->gmu; int i, active_submits; adreno_dump_info(gpu); @@ -1290,6 +1292,10 @@ static void a6xx_recover(struct msm_gpu *gpu) */ gpu->active_submits = 0; + reinit_completion(&gmu->pd_gate); + dev_pm_genpd_add_notifier(gmu->cxpd, &gmu->pd_nb); + dev_pm_genpd_synced_poweroff(gmu->cxpd); + /* Drop the rpm refcount from active submits */ if (active_submits) pm_runtime_put(&gpu->pdev->dev); @@ -1297,6 +1303,11 @@ static void a6xx_recover(struct msm_gpu *gpu) /* And the final one from recover worker */ pm_runtime_put_sync(&gpu->pdev->dev); + if (!wait_for_completion_timeout(&gmu->pd_gate, msecs_to_jiffies(1000))) + DRM_DEV_ERROR(&gpu->pdev->dev, "cx gdsc didn't collapse\n"); + + dev_pm_genpd_remove_notifier(gmu->cxpd); + pm_runtime_use_autosuspend(&gpu->pdev->dev); if (active_submits)