From patchwork Wed Jul 5 16:38:16 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Gautham R Shenoy X-Patchwork-Id: 9826807 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork.web.codeaurora.org (Postfix) with ESMTP id B7B25602F0 for ; Wed, 5 Jul 2017 16:39:28 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id A94B928547 for ; Wed, 5 Jul 2017 16:39:28 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 9D7242856F; Wed, 5 Jul 2017 16:39:28 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-6.9 required=2.0 tests=BAYES_00,RCVD_IN_DNSWL_HI autolearn=unavailable version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 2461128547 for ; Wed, 5 Jul 2017 16:39:28 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1752133AbdGEQiq (ORCPT ); Wed, 5 Jul 2017 12:38:46 -0400 Received: from mx0a-001b2d01.pphosted.com ([148.163.156.1]:37780 "EHLO mx0a-001b2d01.pphosted.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1752123AbdGEQin (ORCPT ); Wed, 5 Jul 2017 12:38:43 -0400 Received: from pps.filterd (m0098409.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.16.0.20/8.16.0.20) with SMTP id v65GcgvJ019079 for ; Wed, 5 Jul 2017 12:38:43 -0400 Received: from e37.co.us.ibm.com (e37.co.us.ibm.com [32.97.110.158]) by mx0a-001b2d01.pphosted.com with ESMTP id 2bgmdvcq10-1 (version=TLSv1.2 cipher=AES256-SHA bits=256 verify=NOT) for ; Wed, 05 Jul 2017 12:38:42 -0400 Received: from localhost by e37.co.us.ibm.com with IBM ESMTP SMTP Gateway: Authorized Use Only! Violators will be prosecuted for from ; Wed, 5 Jul 2017 10:38:41 -0600 Received: from b03cxnp08026.gho.boulder.ibm.com (9.17.130.18) by e37.co.us.ibm.com (192.168.1.137) with IBM ESMTP SMTP Gateway: Authorized Use Only! Violators will be prosecuted; Wed, 5 Jul 2017 10:38:38 -0600 Received: from b03ledav003.gho.boulder.ibm.com (b03ledav003.gho.boulder.ibm.com [9.17.130.234]) by b03cxnp08026.gho.boulder.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id v65GcXJE65536254; Wed, 5 Jul 2017 09:38:33 -0700 Received: from b03ledav003.gho.boulder.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id BBA506A03D; Wed, 5 Jul 2017 10:38:33 -0600 (MDT) Received: from sofia.ibm.com (unknown [9.77.120.36]) by b03ledav003.gho.boulder.ibm.com (Postfix) with ESMTP id 5A37E6A03C; Wed, 5 Jul 2017 10:38:33 -0600 (MDT) Received: by sofia.ibm.com (Postfix, from userid 1000) id 603502E2DF3; Wed, 5 Jul 2017 22:08:29 +0530 (IST) From: "Gautham R. Shenoy" To: Michael Ellerman , Michael Neuling , Nicholas Piggin , Vaidyanathan Srinivasan , Shilpasri G Bhat , "Rafael J. Wysocki" , Akshay Adiga Cc: linuxppc-dev@lists.ozlabs.org, linux-kernel@vger.kernel.org, linux-pm@vger.kernel.org, "Gautham R. Shenoy" Subject: [PATCH 5/5] powernv:idle: Disable LOSE_FULL_CONTEXT states when stop-api fails. Date: Wed, 5 Jul 2017 22:08:16 +0530 X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1499272696-28751-1-git-send-email-ego@linux.vnet.ibm.com> References: <1499272696-28751-1-git-send-email-ego@linux.vnet.ibm.com> X-TM-AS-GCONF: 00 x-cbid: 17070516-0024-0000-0000-000016CA2F3F X-IBM-SpamModules-Scores: X-IBM-SpamModules-Versions: BY=3.00007324; HX=3.00000241; KW=3.00000007; PH=3.00000004; SC=3.00000214; SDB=6.00883263; UDB=6.00440614; IPR=6.00663454; BA=6.00005454; NDR=6.00000001; ZLA=6.00000005; ZF=6.00000009; ZB=6.00000000; ZP=6.00000000; ZH=6.00000000; ZU=6.00000002; MB=3.00016094; XFM=3.00000015; UTC=2017-07-05 16:38:41 X-IBM-AV-DETECTION: SAVI=unused REMOTE=unused XFE=unused x-cbparentid: 17070516-0025-0000-0000-00004BB14A19 Message-Id: <1499272696-28751-6-git-send-email-ego@linux.vnet.ibm.com> X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10432:, , definitions=2017-07-05_10:, , signatures=0 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 spamscore=0 suspectscore=0 malwarescore=0 phishscore=0 adultscore=0 bulkscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1703280000 definitions=main-1707050280 Sender: linux-pm-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-pm@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP From: "Gautham R. Shenoy" Currently, we use the opal call opal_slw_set_reg() to inform the that the Sleep-Winkle Engine (SLW) to restore the contents of some of the Hypervisor state on wakeup from deep idle states that lose full hypervisor context (characterized by the flag OPAL_PM_LOSE_FULL_CONTEXT). However, the current code has a bug in that if opal_slw_set_reg() fails, we don't disable the use of these deep states (winkle on POWER8, stop4 onwards on POWER9). This patch fixes this bug by ensuring that if the the sleep winkle engine is unable to restore the hypervisor states in pnv_save_sprs_for_deep_states(), then we mark as invalid the states which lose full context. As a side-effect, since supported_cpuidle_states in pnv_probe_idle_states() consists of flags of only the valid states, this patch will ensure that no other subsystem in the kernel can use the states which lose full context on stop-api failures. Signed-off-by: Gautham R. Shenoy --- arch/powerpc/platforms/powernv/idle.c | 98 +++++++++++++++++++++++++++++++---- 1 file changed, 87 insertions(+), 11 deletions(-) diff --git a/arch/powerpc/platforms/powernv/idle.c b/arch/powerpc/platforms/powernv/idle.c index 254a0db8..8d07ce6 100644 --- a/arch/powerpc/platforms/powernv/idle.c +++ b/arch/powerpc/platforms/powernv/idle.c @@ -217,9 +217,6 @@ static void pnv_alloc_idle_core_states(void) } update_subcore_sibling_mask(); - - if (supported_cpuidle_states & OPAL_PM_LOSE_FULL_CONTEXT) - pnv_save_sprs_for_deep_states(); } u32 pnv_get_supported_cpuidle_states(void) @@ -518,6 +515,57 @@ static void __init pnv_power9_idle_init(void) u64 max_residency_ns = 0; int i; int dt_idle_states = pnv_idle.nr_states; + bool save_sprs_for_deep_stop = false; + bool disable_lose_full_context = false; + u64 psscr_rl, residency_ns, psscr_val, psscr_mask; + u32 flags; + + /* + * pnv_deepest_stop_{val,mask} should be set to values + * corresponding to the deepest stop state. + */ + for (i = 0; i < dt_idle_states; i++) { + psscr_val = pnv_idle.states[i].ctrl_reg_val; + psscr_mask = pnv_idle.states[i].ctrl_reg_mask; + psscr_rl = psscr_val & PSSCR_RL_MASK; + flags = pnv_idle.states[i].flags; + residency_ns = pnv_idle.states[i].residency_ns; + + if (flags & OPAL_PM_LOSE_FULL_CONTEXT) + save_sprs_for_deep_stop = true; + + if (max_residency_ns < residency_ns) { + max_residency_ns = residency_ns; + pnv_deepest_stop_psscr_val = psscr_val; + pnv_deepest_stop_psscr_mask = psscr_mask; + deepest_stop_found = true; + } + } + + /* + * pnv_save_sprs_for_deep_states() expects + * pnv_deepest_stop_psscr_val to be initialized. + */ + if (save_sprs_for_deep_stop) { + int rc; + + rc = pnv_save_sprs_for_deep_states(); + + /* + * If the Sleep-Winkle Engine is unable to restore the + * critical SPRs on wakeup from some of the deep stop + * states that lose full context, then we mark such + * deep states as invalid and recompute the + * pnv_deepest_stop_psscr_val/mask from among the + * valid states. + */ + if (unlikely(rc)) { + pr_warn("cpuidle-powernv:Disabling full-context loss states.SLW unable to restore SPRs\n"); + disable_lose_full_context = true; + max_residency_ns = 0; + deepest_stop_found = false; + } + } /* * Set pnv_first_deep_stop_state, pnv_deepest_stop_psscr_{val,mask}, @@ -526,16 +574,20 @@ static void __init pnv_power9_idle_init(void) * pnv_first_deep_stop_state should be set to the first stop * level to cause hypervisor state loss. * - * pnv_deepest_stop_{val,mask} should be set to values corresponding to - * the deepest stop state. * * pnv_default_stop_{val,mask} should be set to values corresponding to * the shallowest (OPAL_PM_STOP_INST_FAST) loss-less stop state. */ pnv_first_deep_stop_state = MAX_STOP_STATE; for (i = 0; i < dt_idle_states; i++) { - u64 psscr_rl, residency_ns, psscr_val, psscr_mask; - u32 flags; + flags = pnv_idle.states[i].flags; + + if ((flags & OPAL_PM_LOSE_FULL_CONTEXT) && + disable_lose_full_context) { + pnv_idle.states[i].valid = false; + pr_warn("cpuidle-powernv: Disabling full-context loss state :%s\n", + pnv_idle.states[i].name); + } if (!pnv_idle.states[i].valid) continue; @@ -543,15 +595,14 @@ static void __init pnv_power9_idle_init(void) psscr_val = pnv_idle.states[i].ctrl_reg_val; psscr_mask = pnv_idle.states[i].ctrl_reg_mask; psscr_rl = psscr_val & PSSCR_RL_MASK; - flags = pnv_idle.states[i].flags; + residency_ns = pnv_idle.states[i].residency_ns; if ((flags & OPAL_PM_LOSE_FULL_CONTEXT) && - (pnv_first_deep_stop_state > psscr_rl)) { + (pnv_first_deep_stop_state > psscr_rl)) pnv_first_deep_stop_state = psscr_rl; - } - if (max_residency_ns < residency_ns) { + if (unlikely(max_residency_ns < residency_ns)) { max_residency_ns = residency_ns; pnv_deepest_stop_psscr_val = psscr_val; pnv_deepest_stop_psscr_mask = psscr_mask; @@ -593,6 +644,8 @@ static void __init pnv_power8_idle_init(void) bool has_nap = false; bool has_sleep_er1 = false; int dt_idle_states = pnv_idle.nr_states; + bool disable_full_context_loss = false; + bool sprs_for_deep_state_saved = false; for (i = 0; i < dt_idle_states; i++) { struct pnv_idle_state *state = &pnv_idle.states[i]; @@ -601,6 +654,29 @@ static void __init pnv_power8_idle_init(void) has_nap = true; if (state->flags & OPAL_PM_SLEEP_ENABLED_ER1) has_sleep_er1 = true; + if (state->flags & OPAL_PM_LOSE_FULL_CONTEXT) { + int rc; + + if (sprs_for_deep_state_saved) + continue; + if (disable_full_context_loss) { + state->valid = false; + pr_warn("cpuidle-powernv: Disabling full-context loss state :%s\n", + pnv_idle.states[i].name); + continue; + } + + rc = pnv_save_sprs_for_deep_states(); + if (likely(!rc)) { + sprs_for_deep_state_saved = true; + } else { + pr_warn("cpuidle-powernv:Disabling full-context loss states.SLW unable to restore SPRs.\n"); + disable_full_context_loss = true; + state->valid = false; + pr_warn("cpuidle-powernv:Disabling full-context loss state:%s\n", + pnv_idle.states[i].name); + } + } } if (!has_sleep_er1) {