From patchwork Thu Jun 3 05:40:45 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Vaittinen, Matti" X-Patchwork-Id: 12296299 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-13.8 required=3.0 tests=BAYES_00, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 7B0F7C4709A for ; Thu, 3 Jun 2021 05:40:57 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 60429613DE for ; Thu, 3 Jun 2021 05:40:57 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229826AbhFCFmj (ORCPT ); Thu, 3 Jun 2021 01:42:39 -0400 Received: from mail-lf1-f52.google.com ([209.85.167.52]:33600 "EHLO mail-lf1-f52.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229738AbhFCFmi (ORCPT ); Thu, 3 Jun 2021 01:42:38 -0400 Received: by mail-lf1-f52.google.com with SMTP id a5so7045955lfm.0; Wed, 02 Jun 2021 22:40:53 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:from:to:cc:subject:message-id:references :mime-version:content-disposition:in-reply-to; bh=Ne1WWYaCz65VdAFxTqyE7mZ4II+d9O5bnHyG1KNO9ao=; b=rllPTt9yXlLbdYDVXD/IrGc4zDp1OkdGD3wnd6wW7gpKpucWcsEP1Q4Wb67qlmHNf2 bOUfIGdt5uTNnJbfd6wBmzwL4XV3mQ2lfp5E3tH8CBqMjxcbL+qVJFioTnsQWVdQpLYZ Dn2kZRBSP6K3ELLfieDxRviwbRFxg7FiSxvFtnDzTzt1VVf65JNI/N8JiSxSJDO9GzYJ raYA8JNd1DP/Da+HhbJ4b+w/wi1iFTl1RC/ZXsp61GQpBsCl1Ed0MHRdr8MjKsxQ8so7 6D+RdHoHktGkyKIuVf9p9p7CEsVa7Tk/wRoJYRmKf7H3yhC5TGDc7IPqpYXjCOLMIYAJ zaAA== X-Gm-Message-State: AOAM532o13FIv7O+Qq+lvVS6XEbMGWxu/1Qs/QAsCHWjmFQ+TS7K7OMv Dc+ZSjG8jF0kOPo44fcerqs= X-Google-Smtp-Source: ABdhPJxFQnRk+OrsSMmQSfI6rnJotjROrRNP5Eozx9NckB7ER95WTv+coYn8QFo/ngLkmUgT/n7qxg== X-Received: by 2002:a05:6512:68e:: with SMTP id t14mr24206418lfe.624.1622698852327; Wed, 02 Jun 2021 22:40:52 -0700 (PDT) Received: from localhost.localdomain (dc7vkhyyyyyyyyyyyyycy-3.rev.dnainternet.fi. [2001:14ba:16e2:8300::4]) by smtp.gmail.com with ESMTPSA id q127sm233306ljq.88.2021.06.02.22.40.50 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 02 Jun 2021 22:40:51 -0700 (PDT) Date: Thu, 3 Jun 2021 08:40:45 +0300 From: Matti Vaittinen To: Matti Vaittinen , Matti Vaittinen Cc: Mark Brown , Kees Cook , Andy Shevchenko , Zhang Rui , Guenter Roeck , "agross@kernel.org" , "devicetree@vger.kernel.org" , linux-power , "linux-kernel@vger.kernel.org" , "linux-renesas-soc@vger.kernel.org" , "linux-arm-msm@vger.kernel.org" , "bjorn.andersson@linaro.org" , "lgirdwood@gmail.com" , "robh+dt@kernel.org" , Daniel Lezcano , Amit Kucheria , Matteo Croce , Andrew Morton , Petr Mladek , "Rafael J. Wysocki" , Mike Rapoport , Josef Bacik , Kai-Heng Feng , linux-pm@vger.kernel.org Subject: [PATCH RESEND v10 02/11] reboot: Add hardware protection power-off Message-ID: References: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: Precedence: bulk List-ID: X-Mailing-List: linux-pm@vger.kernel.org There can be few cases when we need to shut-down the system in order to protect the hardware. Currently this is done at least by the thermal core when temperature raises over certain limit. Some PMICs can also generate interrupts for example for over-current or over-voltage, voltage drops, short-circuit, ... etc. On some systems these are a sign of hardware failure and only thing to do is try to protect the rest of the hardware by shutting down the system. Add shut-down logic which can be used by all subsystems instead of implementing the shutdown in each subsystem. The logic is stolen from thermal_core with difference of using atomic_t instead of a mutex in order to allow calls directly from IRQ context and changing the WARN() to pr_emerg() as discussed here: https://lore.kernel.org/lkml/YJuPwAZroVZ%2Fw633@alley/ and here: https://lore.kernel.org/linux-iommu/20210331093104.383705-4-geert+renesas@glider.be/ Signed-off-by: Matti Vaittinen Acked-by: Daniel Lezcano --- Changelog: v10: (changes suggested by Petr Mladek and Geert) - replace WARN() by pr_emerg() v8: (changes suggested by Daniel Lezcano) - replace a protection implemented by a flag + spin_lock_irqsave() with simple atomic_dec_and_test(). - Split thermal-core changes and adding the new API to separate patches v7: - New patch --- include/linux/reboot.h | 1 + kernel/reboot.c | 79 ++++++++++++++++++++++++++++++++++++++++++ 2 files changed, 80 insertions(+) diff --git a/include/linux/reboot.h b/include/linux/reboot.h index 3734cd8f38a8..af907a3d68d1 100644 --- a/include/linux/reboot.h +++ b/include/linux/reboot.h @@ -79,6 +79,7 @@ extern char poweroff_cmd[POWEROFF_CMD_PATH_LEN]; extern void orderly_poweroff(bool force); extern void orderly_reboot(void); +void hw_protection_shutdown(const char *reason, int ms_until_forced); /* * Emergency restart, callable from an interrupt handler. diff --git a/kernel/reboot.c b/kernel/reboot.c index a6ad5eb2fa73..f7440c0c7e43 100644 --- a/kernel/reboot.c +++ b/kernel/reboot.c @@ -7,6 +7,7 @@ #define pr_fmt(fmt) "reboot: " fmt +#include #include #include #include @@ -518,6 +519,84 @@ void orderly_reboot(void) } EXPORT_SYMBOL_GPL(orderly_reboot); +/** + * hw_failure_emergency_poweroff_func - emergency poweroff work after a known delay + * @work: work_struct associated with the emergency poweroff function + * + * This function is called in very critical situations to force + * a kernel poweroff after a configurable timeout value. + */ +static void hw_failure_emergency_poweroff_func(struct work_struct *work) +{ + /* + * We have reached here after the emergency shutdown waiting period has + * expired. This means orderly_poweroff has not been able to shut off + * the system for some reason. + * + * Try to shut down the system immediately using kernel_power_off + * if populated + */ + pr_emerg("Hardware protection timed-out. Trying forced poweroff\n"); + kernel_power_off(); + + /* + * Worst of the worst case trigger emergency restart + */ + pr_emerg("Hardware protection shutdown failed. Trying emergency restart\n"); + emergency_restart(); +} + +static DECLARE_DELAYED_WORK(hw_failure_emergency_poweroff_work, + hw_failure_emergency_poweroff_func); + +/** + * hw_failure_emergency_poweroff - Trigger an emergency system poweroff + * + * This may be called from any critical situation to trigger a system shutdown + * after a given period of time. If time is negative this is not scheduled. + */ +static void hw_failure_emergency_poweroff(int poweroff_delay_ms) +{ + if (poweroff_delay_ms <= 0) + return; + schedule_delayed_work(&hw_failure_emergency_poweroff_work, + msecs_to_jiffies(poweroff_delay_ms)); +} + +/** + * hw_protection_shutdown - Trigger an emergency system poweroff + * + * @reason: Reason of emergency shutdown to be printed. + * @ms_until_forced: Time to wait for orderly shutdown before tiggering a + * forced shudown. Negative value disables the forced + * shutdown. + * + * Initiate an emergency system shutdown in order to protect hardware from + * further damage. Usage examples include a thermal protection or a voltage or + * current regulator failures. + * NOTE: The request is ignored if protection shutdown is already pending even + * if the previous request has given a large timeout for forced shutdown. + * Can be called from any context. + */ +void hw_protection_shutdown(const char *reason, int ms_until_forced) +{ + static atomic_t allow_proceed = ATOMIC_INIT(1); + + pr_emerg("HARDWARE PROTECTION shutdown (%s)\n", reason); + + /* Shutdown should be initiated only once. */ + if (!atomic_dec_and_test(&allow_proceed)) + return; + + /* + * Queue a backup emergency shutdown in the event of + * orderly_poweroff failure + */ + hw_failure_emergency_poweroff(ms_until_forced); + orderly_poweroff(true); +} +EXPORT_SYMBOL_GPL(hw_protection_shutdown); + static int __init reboot_setup(char *str) { for (;;) { From patchwork Thu Jun 3 05:41:03 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Vaittinen, Matti" X-Patchwork-Id: 12296301 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-13.8 required=3.0 tests=BAYES_00, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id B5F2FC47082 for ; Thu, 3 Jun 2021 05:41:26 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 92152613E6 for ; Thu, 3 Jun 2021 05:41:26 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229762AbhFCFnJ (ORCPT ); Thu, 3 Jun 2021 01:43:09 -0400 Received: from mail-lj1-f171.google.com ([209.85.208.171]:38506 "EHLO mail-lj1-f171.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229738AbhFCFnI (ORCPT ); Thu, 3 Jun 2021 01:43:08 -0400 Received: by mail-lj1-f171.google.com with SMTP id a4so5604059ljd.5; Wed, 02 Jun 2021 22:41:11 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:from:to:cc:subject:message-id:references :mime-version:content-disposition:in-reply-to; bh=avHrM7JfdyvFeGpZaxuvTLRbbrJcYYr/B2Y+l11QZLI=; b=nFrZNxyRY5OUywUNdwV5tHOacmSwKk18b4c02satxPJdccmmE+97Kb/g9plgdkuQTB MYEPzvEEg+wk7wxcmqduzdT30Kf2v49myIALvO0LG93GWtKXJRiLUhIy+n3dxLDlZToo 5rv+bwiZyGYppdck+Upk4fQJpVO6MRl3Cqkj3iANhkzxHVExCqofBsaiXP1DI+aumkJJ erj7VOAkr8uw9S+SmoaMQJgMHHZ7cxPTshbwhTjT0ketIlVd8ujKeSfn5jE1wUmHoGLu 4nESdWt1z67hreX4b/BHKvbmOvQYbR1gWyWrlkRo/1hjwuGedxppZgyLWr3WFMulzsvB qHQA== X-Gm-Message-State: AOAM532iKzrjk3dHtQB0RZh6PxzjL3IyPy0TaGfW6Gykj7i4JPbM2DeZ A5Me/d9kl9cf58NQjAPgDbY= X-Google-Smtp-Source: ABdhPJx65Mb0U6+bOzykCIrwAiL5/c73SJ3yyHRMr8zu84Ol76ONfTzz1p4wnCmSVqOF8o2RdMJQaQ== X-Received: by 2002:a2e:979a:: with SMTP id y26mr4689887lji.130.1622698870591; Wed, 02 Jun 2021 22:41:10 -0700 (PDT) Received: from localhost.localdomain (dc7vkhyyyyyyyyyyyyycy-3.rev.dnainternet.fi. [2001:14ba:16e2:8300::4]) by smtp.gmail.com with ESMTPSA id w8sm227523ljj.61.2021.06.02.22.41.09 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 02 Jun 2021 22:41:10 -0700 (PDT) Date: Thu, 3 Jun 2021 08:41:03 +0300 From: Matti Vaittinen To: Matti Vaittinen , Matti Vaittinen Cc: Mark Brown , Kees Cook , Andy Shevchenko , Zhang Rui , Guenter Roeck , "agross@kernel.org" , "devicetree@vger.kernel.org" , linux-power , "linux-kernel@vger.kernel.org" , "linux-renesas-soc@vger.kernel.org" , "linux-arm-msm@vger.kernel.org" , "bjorn.andersson@linaro.org" , "lgirdwood@gmail.com" , "robh+dt@kernel.org" , Daniel Lezcano , Amit Kucheria , Matteo Croce , Andrew Morton , Petr Mladek , "Rafael J. Wysocki" , Mike Rapoport , Josef Bacik , Kai-Heng Feng , linux-pm@vger.kernel.org Subject: [PATCH RESEND v10 03/11] thermal: Use generic HW-protection shutdown API Message-ID: <5531e89d9e710f5d10e7cdce3ee58957335b9e03.1622628333.git.matti.vaittinen@fi.rohmeurope.com> References: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: Precedence: bulk List-ID: X-Mailing-List: linux-pm@vger.kernel.org The hardware shutdown function was exported from kernel/reboot for other subsystems to use. Logic is copied from the thermal_core. The protection mutex is replaced by an atomic_t to allow calls also from an IRQ context. Also the WARN() was replaced by pr_emerg() based on discussions here: https://lore.kernel.org/lkml/YJuPwAZroVZ%2Fw633@alley/ and here: https://lore.kernel.org/linux-iommu/20210331093104.383705-4-geert+renesas@glider.be/ Use the exported API instead of implementing own just for the thermal_core. Signed-off-by: Matti Vaittinen Acked-by: Daniel Lezcano --- Changelog: v10: - update commit message to mention changing WARN() to pr_emerg() v9: - Update the thermal documentation v8: - new patch (change added in v7, splitted in own patch at v8) Use the exported API instead --- .../driver-api/thermal/sysfs-api.rst | 24 +++---- drivers/thermal/thermal_core.c | 63 ++----------------- 2 files changed, 13 insertions(+), 74 deletions(-) diff --git a/Documentation/driver-api/thermal/sysfs-api.rst b/Documentation/driver-api/thermal/sysfs-api.rst index 4b638c14bc16..c93fa5e961a0 100644 --- a/Documentation/driver-api/thermal/sysfs-api.rst +++ b/Documentation/driver-api/thermal/sysfs-api.rst @@ -740,21 +740,15 @@ possible. 5. thermal_emergency_poweroff ============================= -On an event of critical trip temperature crossing. Thermal framework -allows the system to shutdown gracefully by calling orderly_poweroff(). -In the event of a failure of orderly_poweroff() to shut down the system -we are in danger of keeping the system alive at undesirably high -temperatures. To mitigate this high risk scenario we program a work -queue to fire after a pre-determined number of seconds to start -an emergency shutdown of the device using the kernel_power_off() -function. In case kernel_power_off() fails then finally -emergency_restart() is called in the worst case. +On an event of critical trip temperature crossing the thermal framework +shuts down the system by calling hw_protection_shutdown(). The +hw_protection_shutdown() first attempts to perform an orderly shutdown +but accepts a delay after which it proceeds doing a forced power-off +or as last resort an emergency_restart. The delay should be carefully profiled so as to give adequate time for -orderly_poweroff(). In case of failure of an orderly_poweroff() the -emergency poweroff kicks in after the delay has elapsed and shuts down -the system. +orderly poweroff. -If set to 0 emergency poweroff will not be supported. So a carefully -profiled non-zero positive value is a must for emergency poweroff to be -triggered. +If the delay is set to 0 emergency poweroff will not be supported. So a +carefully profiled non-zero positive value is a must for emergency +poweroff to be triggered. diff --git a/drivers/thermal/thermal_core.c b/drivers/thermal/thermal_core.c index d20b25f40d19..10a2d8e1cacf 100644 --- a/drivers/thermal/thermal_core.c +++ b/drivers/thermal/thermal_core.c @@ -36,10 +36,8 @@ static LIST_HEAD(thermal_governor_list); static DEFINE_MUTEX(thermal_list_lock); static DEFINE_MUTEX(thermal_governor_lock); -static DEFINE_MUTEX(poweroff_lock); static atomic_t in_suspend; -static bool power_off_triggered; static struct thermal_governor *def_governor; @@ -327,70 +325,18 @@ static void handle_non_critical_trips(struct thermal_zone_device *tz, int trip) def_governor->throttle(tz, trip); } -/** - * thermal_emergency_poweroff_func - emergency poweroff work after a known delay - * @work: work_struct associated with the emergency poweroff function - * - * This function is called in very critical situations to force - * a kernel poweroff after a configurable timeout value. - */ -static void thermal_emergency_poweroff_func(struct work_struct *work) -{ - /* - * We have reached here after the emergency thermal shutdown - * Waiting period has expired. This means orderly_poweroff has - * not been able to shut off the system for some reason. - * Try to shut down the system immediately using kernel_power_off - * if populated - */ - WARN(1, "Attempting kernel_power_off: Temperature too high\n"); - kernel_power_off(); - - /* - * Worst of the worst case trigger emergency restart - */ - WARN(1, "Attempting emergency_restart: Temperature too high\n"); - emergency_restart(); -} - -static DECLARE_DELAYED_WORK(thermal_emergency_poweroff_work, - thermal_emergency_poweroff_func); - -/** - * thermal_emergency_poweroff - Trigger an emergency system poweroff - * - * This may be called from any critical situation to trigger a system shutdown - * after a known period of time. By default this is not scheduled. - */ -static void thermal_emergency_poweroff(void) +void thermal_zone_device_critical(struct thermal_zone_device *tz) { - int poweroff_delay_ms = CONFIG_THERMAL_EMERGENCY_POWEROFF_DELAY_MS; /* * poweroff_delay_ms must be a carefully profiled positive value. - * Its a must for thermal_emergency_poweroff_work to be scheduled + * Its a must for forced_emergency_poweroff_work to be scheduled. */ - if (poweroff_delay_ms <= 0) - return; - schedule_delayed_work(&thermal_emergency_poweroff_work, - msecs_to_jiffies(poweroff_delay_ms)); -} + int poweroff_delay_ms = CONFIG_THERMAL_EMERGENCY_POWEROFF_DELAY_MS; -void thermal_zone_device_critical(struct thermal_zone_device *tz) -{ dev_emerg(&tz->device, "%s: critical temperature reached, " "shutting down\n", tz->type); - mutex_lock(&poweroff_lock); - if (!power_off_triggered) { - /* - * Queue a backup emergency shutdown in the event of - * orderly_poweroff failure - */ - thermal_emergency_poweroff(); - orderly_poweroff(true); - power_off_triggered = true; - } - mutex_unlock(&poweroff_lock); + hw_protection_shutdown("Temperature too high", poweroff_delay_ms); } EXPORT_SYMBOL(thermal_zone_device_critical); @@ -1538,7 +1484,6 @@ static int __init thermal_init(void) ida_destroy(&thermal_cdev_ida); mutex_destroy(&thermal_list_lock); mutex_destroy(&thermal_governor_lock); - mutex_destroy(&poweroff_lock); return result; } postcore_initcall(thermal_init);