From patchwork Thu Mar 15 13:14:40 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ulf Hansson X-Patchwork-Id: 10284461 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork.web.codeaurora.org (Postfix) with ESMTP id B2BC060291 for ; Thu, 15 Mar 2018 13:15:21 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id A6C9E27F82 for ; Thu, 15 Mar 2018 13:15:21 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 9AF52288F4; Thu, 15 Mar 2018 13:15:21 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-1.9 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID autolearn=unavailable version=3.3.1 Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.wl.linuxfoundation.org (Postfix) with ESMTPS id 875A427F82 for ; Thu, 15 Mar 2018 13:15:20 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20170209; h=Sender: Content-Transfer-Encoding:Content-Type:Cc:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:To:Subject:Message-ID:Date:From: References:In-Reply-To:MIME-Version:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=6uthBUbcBqbIe62Zg75BhFLBM1DZ7lfi5px3LeTJbZg=; b=IbJe6X+2AYPJS7 aR2qj59CAPwK49SZ/P9XF92Bt4k/v54XxS6DiKXIhijkQHWSr/TJb/xOVKHdOLP7gwdj5LO25m5AF GxU7cxZ0frKB0vRkEOzDuqJGJ+Gn9F9OnQ51zRkQJePMGYyFUXwgLHPU5yxV4EdW99Blo62bFO+pd UIS4Gkqn6RMEN7q1RYdx6/Tm9jkGy8iztwhk6c1VrCJhvNthK/ke4YdunK6ZUHo58llu+GUCPyjnb 3cmuURkPJw3OalXHaALivCI2rFWc0IvhAnl9E4jly1agcDL3SLi1b/M87gT8WIU3LCiyafDkBhCoC 2e2ki13gqViDmnpTZ8jw==; Received: from localhost ([127.0.0.1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.90_1 #2 (Red Hat Linux)) id 1ewSif-0005q0-Hh; Thu, 15 Mar 2018 13:14:57 +0000 Received: from mail-it0-x243.google.com ([2607:f8b0:4001:c0b::243]) by bombadil.infradead.org with esmtps (Exim 4.90_1 #2 (Red Hat Linux)) id 1ewSia-0005ma-Qb for linux-arm-kernel@lists.infradead.org; Thu, 15 Mar 2018 13:14:55 +0000 Received: by mail-it0-x243.google.com with SMTP id d13-v6so9054790itf.0 for ; Thu, 15 Mar 2018 06:14:42 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc; bh=8R3lfbHiiUFRj0sAgFaFXLtVnR7tJCYk6amhrZuBlmk=; b=TvnajAO713+NckTW8KMd3lJ+kD7STQ+HwglOIpvXOPDjoicjxSXEhcp9vh7XJqzTZI 7NOMTI0Dd+Fc8AWL0G5ue070ufxAtzT86wy1Iy/0TdptdVdH/A0R5AexY7qZMRsswUUq YLRvldYuCbYvbo7SJthCPL8sFNlIr/eEzAY/Y= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to:cc; bh=8R3lfbHiiUFRj0sAgFaFXLtVnR7tJCYk6amhrZuBlmk=; b=ej/ybw3Z9C1TkWZYrCzHqNdLC0hAi/+trbWJPXsszbxq0K1o7/5nF084omvUlSdndI 4NM34snKXJYK3dSMLtyEqbb+OikI9ZFgd1Ymtt/OEboDaVKqZEWnKEDhC3aniZVlAExQ rrNXsqFU/5vj/sxv42Hsc/ikriZ/rnbs6xJ/tM/+wB1XcVD7xuCyCOc4W3F0WCfZjTz7 Y7ofJTipmdMuSNDFCgETm2LDmNJvq9qloxqUQ8cwy6jqvV+fehmOxjlkQQ8/vzq5lk7E rdrlYNXlTcwl/77rmuO3UyJuMXiJPh2gv82Zn+9d4H/BWSadd5alhNdnz/Q+HhZI70XK 6yHg== X-Gm-Message-State: AElRT7ETyA/PbYnNLYgMJjyo4o+xbZd0ZVKkb7PF95EQekJj8Ch8g2uu oIZOOE1dnTvVU67e5rA1gSUKrRGBeQ5mEsDwzmcgsg== X-Google-Smtp-Source: AG47ELvkZPbeQpDKcZ48hOvXeriTX/ynTkbARnPyjVWlwEEtWc/uSdob9YSFE9oL9DUJVHGBgO1pdT1I2KPheGEH6bs= X-Received: by 2002:a24:2b89:: with SMTP id h131-v6mr6331359ita.97.1521119681622; Thu, 15 Mar 2018 06:14:41 -0700 (PDT) MIME-Version: 1.0 Received: by 10.2.101.23 with HTTP; Thu, 15 Mar 2018 06:14:40 -0700 (PDT) In-Reply-To: References: <1521046715-30683-1-git-send-email-ulf.hansson@linaro.org> From: Ulf Hansson Date: Thu, 15 Mar 2018 14:14:40 +0100 Message-ID: Subject: Re: [PATCH v6 00/25] PM / Domains: Support hierarchical CPU arrangement (PSCI/ARM) To: Geert Uytterhoeven X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20180315_061453_068719_1771C83D X-CRM114-Status: GOOD ( 21.63 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Lorenzo Pieralisi , Vincent Guittot , Daniel Lezcano , Geert Uytterhoeven , Kevin Hilman , Stephen Boyd , linux-arm-msm@vger.kernel.org, Linux PM list , "Rafael J . Wysocki" , Linux Kernel Mailing List , Lina Iyer , Rob Herring , Linux ARM , Sudeep Holla , Thomas Gleixner , Lina Iyer , Juri Lelli Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+patchwork-linux-arm=patchwork.kernel.org@lists.infradead.org X-Virus-Scanned: ClamAV using ClamSMTP On 15 March 2018 at 12:00, Geert Uytterhoeven wrote: > Hi Ulf, > > On Wed, Mar 14, 2018 at 5:58 PM, Ulf Hansson wrote: >> This series is a re-worked version from Lina Iyer's two series [1] that got >> posted more than a year ago by now. I have picked up the series and done a >> significant re-work of it and here's the result. All patches have been changed, >> some have been dropped, some are entirely new. For this reason I decided to not >> include a version history, as I think people need a fresh start anyway. > > Thanks for your series! > > I gave it a try on a few Renesas boards and SoCs, without adding any new DT > descriptions. On all of them it triggers Thanks a lot for testing! > > BUG: sleeping function called from invalid context at > drivers/base/power/runtime.c:1057 > > during system suspend (s2ram). > > On R-Car Gen2 and SH-Mobile AG5 (arm32, no PSCI): > > Disabling non-boot CPUs ... > BUG: sleeping function called from invalid context at > drivers/base/power/runtime.c:1057 > in_atomic(): 0, irqs_disabled(): 128, pid: 1725, name: s2ram > CPU: 0 PID: 1725 Comm: s2ram Not tainted > 4.16.0-rc5-koelsch-00475-gaa69fc46cc44c3d7 #4001 > Hardware name: Generic R-Car Gen2 (Flattened Device Tree) > [] (unwind_backtrace) from [] (show_stack+0x10/0x14) > [] (show_stack) from [] (dump_stack+0x7c/0x9c) > [] (dump_stack) from [] (___might_sleep+0x128/0x164) > [] (___might_sleep) from [] (__pm_runtime_suspend+0x70/0xa8) > [] (__pm_runtime_suspend) from [] (cpu_pm_enter+0x78/0x9c) > [] (cpu_pm_enter) from [] (cpu_pm_suspend+0x8/0x18) > [] (cpu_pm_suspend) from [] (syscore_suspend+0x88/0x138) > [] (syscore_suspend) from [] > (suspend_devices_and_enter+0x21c/0x564) > [] (suspend_devices_and_enter) from [] > (pm_suspend+0x250/0x2c8) > [] (pm_suspend) from [] (state_store+0xac/0xcc) > [] (state_store) from [] (kernfs_fop_write+0x170/0x1b0) > [] (kernfs_fop_write) from [] (__vfs_write+0x2c/0x140) > [] (__vfs_write) from [] (vfs_write+0xb8/0x144) > [] (vfs_write) from [] (SyS_write+0x54/0xac) > [] (SyS_write) from [] (ret_fast_syscall+0x0/0x4c) > Exception stack(0xeabcbfa8 to 0xeabcbff0) > bfa0: 00000004 000ce408 00000001 000ce408 00000004 00000000 > bfc0: 00000004 000ce408 b6e80b50 00000004 00000004 00000000 000c5758 00000000 > bfe0: 00000000 be866754 b6de3c85 b6e1ef26 > > > On R-Car Gen3 (arm64, PSCI): > > Disabling non-boot CPUs ... > CPU1: shutdown > psci: CPU1 killed. > CPU2: shutdown > psci: CPU2 killed. > CPU3: shutdown > psci: CPU3 killed. > CPU4: shutdown > psci: CPU4 killed. > CPU5: shutdown > psci: CPU5 killed. > CPU6: shutdown > psci: CPU6 killed. > CPU7: shutdown > psci: CPU7 killed. > BUG: sleeping function called from invalid context at > drivers/base/power/runtime.c:1057 > in_atomic(): 0, irqs_disabled(): 128, pid: 2592, name: s2ram > 4 locks held by s2ram/2592: > #0: (sb_writers#7){.+.+}, at: [<00000000cae1f0e5>] vfs_write+0xb0/0x164 > #1: (&of->mutex){+.+.}, at: [<000000003002e527>] kernfs_fop_write+0x114/0x1bc > #2: (kn->count#71){.+.+}, at: [<000000008c0217e1>] > kernfs_fop_write+0x11c/0x1bc > #3: (pm_mutex){+.+.}, at: [<000000009a6c23e2>] pm_suspend+0x194/0xb10 > irq event stamp: 69308 > hardirqs last enabled at (69307): [<00000000e9d09767>] > _raw_spin_unlock_irq+0x2c/0x4c > hardirqs last disabled at (69308): [<00000000619169c4>] > arch_suspend_disable_irqs+0x10/0x18 > softirqs last enabled at (69246): [<00000000b8a7706e>] > hrtimers_dead_cpu+0x2b8/0x2f0 > softirqs last disabled at (69242): [<000000004dee0c40>] > hrtimers_dead_cpu+0x48/0x2f0 > CPU: 0 PID: 2592 Comm: s2ram Not tainted > 4.16.0-rc5-salvator-x-00470-g319cfb3643965f46 #1685 > Hardware name: Renesas Salvator-X 2nd version board based on r8a7795 ES2.0+ (DT) > Call trace: > dump_backtrace+0x0/0x140 > show_stack+0x14/0x1c > dump_stack+0xb4/0xf0 > ___might_sleep+0x1fc/0x218 > __might_sleep+0x70/0x80 > __pm_runtime_suspend+0x6c/0xac > cpu_pm_enter+0x74/0x9c > cpu_pm_suspend+0xc/0x1c > syscore_suspend+0x1b8/0x410 > suspend_devices_and_enter+0x210/0xd9c > pm_suspend+0x9a4/0xb10 > state_store+0xd4/0xf8 > kobj_attr_store+0x18/0x28 > sysfs_kf_write+0x50/0x5c > kernfs_fop_write+0x178/0x1bc > __vfs_write+0x38/0x140 > vfs_write+0xc4/0x164 > SyS_write+0x54/0xa4 > el0_svc_naked+0x30/0x34 > > > I've bisected this to "[PATCH v6 09/25] kernel/cpu_pm: Manage runtime PM > in the idle path for CPUs". Thanks for the report, very much appreciated! I realized that pm_runtime_irq_safe() won't be called for CPU devices that hasn't been hooked up to a genpd - as of_genpd_attach_cpu() hasn't been called for them. I figure something out for the next version on how to address this properly, until then you may test with the following change: diff --git a/kernel/cpu_pm.c b/kernel/cpu_pm.c index 71317ff..57250ee 100644 --- a/kernel/cpu_pm.c +++ b/kernel/cpu_pm.c @@ -103,7 +103,7 @@ int cpu_pm_enter(void) */ cpu_pm_notify(CPU_PM_ENTER_FAILED, nr_calls - 1, NULL); - if (!ret && dev) + if (!ret && dev && pm_runtime_enabled(dev)) pm_runtime_put_sync_suspend(dev); return ret; @@ -126,7 +126,7 @@ int cpu_pm_exit(void) { struct device *dev = get_cpu_device(smp_processor_id()); - if (dev) + if (dev && pm_runtime_enabled(dev)) pm_runtime_get_sync(dev); return cpu_pm_notify(CPU_PM_EXIT, -1, NULL);