From patchwork Thu Mar 15 13:14:40 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ulf Hansson X-Patchwork-Id: 10284453 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork.web.codeaurora.org (Postfix) with ESMTP id 51EE060291 for ; Thu, 15 Mar 2018 13:14:59 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 49CD127F82 for ; Thu, 15 Mar 2018 13:14:59 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 3D178288F4; Thu, 15 Mar 2018 13:14:59 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID, DKIM_VALID_AU, RCVD_IN_DNSWL_HI autolearn=unavailable version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id B69D327F82 for ; Thu, 15 Mar 2018 13:14:58 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1751887AbeCONOp (ORCPT ); Thu, 15 Mar 2018 09:14:45 -0400 Received: from mail-it0-f66.google.com ([209.85.214.66]:51187 "EHLO mail-it0-f66.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1751763AbeCONOm (ORCPT ); Thu, 15 Mar 2018 09:14:42 -0400 Received: by mail-it0-f66.google.com with SMTP id d13-v6so9054792itf.0 for ; Thu, 15 Mar 2018 06:14:42 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc; bh=8R3lfbHiiUFRj0sAgFaFXLtVnR7tJCYk6amhrZuBlmk=; b=TvnajAO713+NckTW8KMd3lJ+kD7STQ+HwglOIpvXOPDjoicjxSXEhcp9vh7XJqzTZI 7NOMTI0Dd+Fc8AWL0G5ue070ufxAtzT86wy1Iy/0TdptdVdH/A0R5AexY7qZMRsswUUq YLRvldYuCbYvbo7SJthCPL8sFNlIr/eEzAY/Y= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to:cc; bh=8R3lfbHiiUFRj0sAgFaFXLtVnR7tJCYk6amhrZuBlmk=; b=BbjMbmy/XcsRWRmIIaz9JLOr4tnnOKE5fy9GGwHTt+XvgC8yC4TBTA2o9Ob1o9hOaQ R29oQax72X9o/euXforvVfncHHs6fnRPqTTHAYlOTiRnmU1ZloBb2vVCfNwkXZDZcrH9 Ts7cOLVfmcUBMYhAXUP/ktER+3K6LxZmXGJ6c5PLdf9FTC4Nk04HAaqN7dc/4GS6q/8e 0owh1uPSO6oJXn1KalVvvRJ+COiqpEaoxmOIqaQVwzEJhSxp1mO3rxMgb+u9fCqTuQuA wEN6bejhBlstSnJQyJKTSlCloKdVevrElr35SXEQa8aTjrJHq8Zg1fwxskYL+w8qGQz5 iA5g== X-Gm-Message-State: AElRT7Fslb0dSooQJP0Xuk3N1YC6bPLkxbMQKOnhgMkm0ndFwuccENMV jJexBO108hsNJh/6vOqJ7lna9Za0daCVMoBUzoVARg== X-Google-Smtp-Source: AG47ELvkZPbeQpDKcZ48hOvXeriTX/ynTkbARnPyjVWlwEEtWc/uSdob9YSFE9oL9DUJVHGBgO1pdT1I2KPheGEH6bs= X-Received: by 2002:a24:2b89:: with SMTP id h131-v6mr6331359ita.97.1521119681622; Thu, 15 Mar 2018 06:14:41 -0700 (PDT) MIME-Version: 1.0 Received: by 10.2.101.23 with HTTP; Thu, 15 Mar 2018 06:14:40 -0700 (PDT) In-Reply-To: References: <1521046715-30683-1-git-send-email-ulf.hansson@linaro.org> From: Ulf Hansson Date: Thu, 15 Mar 2018 14:14:40 +0100 Message-ID: Subject: Re: [PATCH v6 00/25] PM / Domains: Support hierarchical CPU arrangement (PSCI/ARM) To: Geert Uytterhoeven Cc: "Rafael J . Wysocki" , Sudeep Holla , Lorenzo Pieralisi , Linux PM list , Kevin Hilman , Lina Iyer , Lina Iyer , Rob Herring , Daniel Lezcano , Thomas Gleixner , Vincent Guittot , Stephen Boyd , Juri Lelli , Geert Uytterhoeven , Linux ARM , linux-arm-msm@vger.kernel.org, Linux Kernel Mailing List Sender: linux-pm-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-pm@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP On 15 March 2018 at 12:00, Geert Uytterhoeven wrote: > Hi Ulf, > > On Wed, Mar 14, 2018 at 5:58 PM, Ulf Hansson wrote: >> This series is a re-worked version from Lina Iyer's two series [1] that got >> posted more than a year ago by now. I have picked up the series and done a >> significant re-work of it and here's the result. All patches have been changed, >> some have been dropped, some are entirely new. For this reason I decided to not >> include a version history, as I think people need a fresh start anyway. > > Thanks for your series! > > I gave it a try on a few Renesas boards and SoCs, without adding any new DT > descriptions. On all of them it triggers Thanks a lot for testing! > > BUG: sleeping function called from invalid context at > drivers/base/power/runtime.c:1057 > > during system suspend (s2ram). > > On R-Car Gen2 and SH-Mobile AG5 (arm32, no PSCI): > > Disabling non-boot CPUs ... > BUG: sleeping function called from invalid context at > drivers/base/power/runtime.c:1057 > in_atomic(): 0, irqs_disabled(): 128, pid: 1725, name: s2ram > CPU: 0 PID: 1725 Comm: s2ram Not tainted > 4.16.0-rc5-koelsch-00475-gaa69fc46cc44c3d7 #4001 > Hardware name: Generic R-Car Gen2 (Flattened Device Tree) > [] (unwind_backtrace) from [] (show_stack+0x10/0x14) > [] (show_stack) from [] (dump_stack+0x7c/0x9c) > [] (dump_stack) from [] (___might_sleep+0x128/0x164) > [] (___might_sleep) from [] (__pm_runtime_suspend+0x70/0xa8) > [] (__pm_runtime_suspend) from [] (cpu_pm_enter+0x78/0x9c) > [] (cpu_pm_enter) from [] (cpu_pm_suspend+0x8/0x18) > [] (cpu_pm_suspend) from [] (syscore_suspend+0x88/0x138) > [] (syscore_suspend) from [] > (suspend_devices_and_enter+0x21c/0x564) > [] (suspend_devices_and_enter) from [] > (pm_suspend+0x250/0x2c8) > [] (pm_suspend) from [] (state_store+0xac/0xcc) > [] (state_store) from [] (kernfs_fop_write+0x170/0x1b0) > [] (kernfs_fop_write) from [] (__vfs_write+0x2c/0x140) > [] (__vfs_write) from [] (vfs_write+0xb8/0x144) > [] (vfs_write) from [] (SyS_write+0x54/0xac) > [] (SyS_write) from [] (ret_fast_syscall+0x0/0x4c) > Exception stack(0xeabcbfa8 to 0xeabcbff0) > bfa0: 00000004 000ce408 00000001 000ce408 00000004 00000000 > bfc0: 00000004 000ce408 b6e80b50 00000004 00000004 00000000 000c5758 00000000 > bfe0: 00000000 be866754 b6de3c85 b6e1ef26 > > > On R-Car Gen3 (arm64, PSCI): > > Disabling non-boot CPUs ... > CPU1: shutdown > psci: CPU1 killed. > CPU2: shutdown > psci: CPU2 killed. > CPU3: shutdown > psci: CPU3 killed. > CPU4: shutdown > psci: CPU4 killed. > CPU5: shutdown > psci: CPU5 killed. > CPU6: shutdown > psci: CPU6 killed. > CPU7: shutdown > psci: CPU7 killed. > BUG: sleeping function called from invalid context at > drivers/base/power/runtime.c:1057 > in_atomic(): 0, irqs_disabled(): 128, pid: 2592, name: s2ram > 4 locks held by s2ram/2592: > #0: (sb_writers#7){.+.+}, at: [<00000000cae1f0e5>] vfs_write+0xb0/0x164 > #1: (&of->mutex){+.+.}, at: [<000000003002e527>] kernfs_fop_write+0x114/0x1bc > #2: (kn->count#71){.+.+}, at: [<000000008c0217e1>] > kernfs_fop_write+0x11c/0x1bc > #3: (pm_mutex){+.+.}, at: [<000000009a6c23e2>] pm_suspend+0x194/0xb10 > irq event stamp: 69308 > hardirqs last enabled at (69307): [<00000000e9d09767>] > _raw_spin_unlock_irq+0x2c/0x4c > hardirqs last disabled at (69308): [<00000000619169c4>] > arch_suspend_disable_irqs+0x10/0x18 > softirqs last enabled at (69246): [<00000000b8a7706e>] > hrtimers_dead_cpu+0x2b8/0x2f0 > softirqs last disabled at (69242): [<000000004dee0c40>] > hrtimers_dead_cpu+0x48/0x2f0 > CPU: 0 PID: 2592 Comm: s2ram Not tainted > 4.16.0-rc5-salvator-x-00470-g319cfb3643965f46 #1685 > Hardware name: Renesas Salvator-X 2nd version board based on r8a7795 ES2.0+ (DT) > Call trace: > dump_backtrace+0x0/0x140 > show_stack+0x14/0x1c > dump_stack+0xb4/0xf0 > ___might_sleep+0x1fc/0x218 > __might_sleep+0x70/0x80 > __pm_runtime_suspend+0x6c/0xac > cpu_pm_enter+0x74/0x9c > cpu_pm_suspend+0xc/0x1c > syscore_suspend+0x1b8/0x410 > suspend_devices_and_enter+0x210/0xd9c > pm_suspend+0x9a4/0xb10 > state_store+0xd4/0xf8 > kobj_attr_store+0x18/0x28 > sysfs_kf_write+0x50/0x5c > kernfs_fop_write+0x178/0x1bc > __vfs_write+0x38/0x140 > vfs_write+0xc4/0x164 > SyS_write+0x54/0xa4 > el0_svc_naked+0x30/0x34 > > > I've bisected this to "[PATCH v6 09/25] kernel/cpu_pm: Manage runtime PM > in the idle path for CPUs". Thanks for the report, very much appreciated! I realized that pm_runtime_irq_safe() won't be called for CPU devices that hasn't been hooked up to a genpd - as of_genpd_attach_cpu() hasn't been called for them. I figure something out for the next version on how to address this properly, until then you may test with the following change: diff --git a/kernel/cpu_pm.c b/kernel/cpu_pm.c index 71317ff..57250ee 100644 --- a/kernel/cpu_pm.c +++ b/kernel/cpu_pm.c @@ -103,7 +103,7 @@ int cpu_pm_enter(void) */ cpu_pm_notify(CPU_PM_ENTER_FAILED, nr_calls - 1, NULL); - if (!ret && dev) + if (!ret && dev && pm_runtime_enabled(dev)) pm_runtime_put_sync_suspend(dev); return ret; @@ -126,7 +126,7 @@ int cpu_pm_exit(void) { struct device *dev = get_cpu_device(smp_processor_id()); - if (dev) + if (dev && pm_runtime_enabled(dev)) pm_runtime_get_sync(dev); return cpu_pm_notify(CPU_PM_EXIT, -1, NULL);