From patchwork Mon Dec 19 15:14:59 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Krzysztof Kozlowski X-Patchwork-Id: 13076708 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 57F87C4167B for ; Mon, 19 Dec 2022 15:15:18 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232832AbiLSPPQ (ORCPT ); Mon, 19 Dec 2022 10:15:16 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:43004 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232837AbiLSPPP (ORCPT ); Mon, 19 Dec 2022 10:15:15 -0500 Received: from mail-lf1-x134.google.com (mail-lf1-x134.google.com [IPv6:2a00:1450:4864:20::134]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 9AE5865DF for ; Mon, 19 Dec 2022 07:15:13 -0800 (PST) Received: by mail-lf1-x134.google.com with SMTP id b13so14137877lfo.3 for ; Mon, 19 Dec 2022 07:15:13 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=5GU2NjmgE9pvJfcbP7Wbuf1L4Bg6ALcOGzwayLNEdCc=; b=M96PQCEKu708zYSAKfnI3vYYuOYQPIDYBpF3ko4XOFDTuP8CChmqa/eBMxnob9Xvvu 13bJnQo4DxOWhMM/jR1CrFAF5M9LqPtdN/7YNIKVERNdhI1SDlDl3cc86hykawp9nDlW A8LaZVRmgg3mME0zszFlmghrDv2CYA9UQGulYue18w+tfHQ41K2i09gLezGgefdt2Xat yA3iZWLVPKyr3leBBG1RmA/uIihnVnrApGFWHFy9C1HuHrxB4Ntk1TXktxvLrCwRmM4l HKEKzORBxoGPpeRsegGr5/AeKG3CT40OE9h/5COkm0Ke5wzV7SIHlnUWUrpTu1NnSrTp 0XRg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=5GU2NjmgE9pvJfcbP7Wbuf1L4Bg6ALcOGzwayLNEdCc=; b=ImoJMo1y7hDUCzrgg/bM0H0KYmdQJ0CLs+QNM+dT1YjBGA6qTKm2ajqcEjEjK/A3F0 PyNguCoyoaXo/4jXCh6lN1Klqi2cgAmaBo4I8nMaIsjLEgQOjyQgISfbwsYCIN0FuQnx o3r2Nryk6V8fHVr2OJzW2LZOK5lviL8LhwgcM6tor0R/ksiosUuMVWKXWpXJBPeqxOkv F2Pxitij0Dll7VP6xTEzfHVU/vdldUKbJKdTIBX8MbwlkYGQKUfAwSjplGt59a1oXNoG ub87ulgJX3fS8JtWw657Dqnp1BV+MSC9ACcfegpaPvB7g3n45h3XhpXH40q0bIryFOcs WtEg== X-Gm-Message-State: ANoB5pmpZVF4JMQRvOdIoPbwF/W/i6Lrivl2QvFYunHzzuNfMrskrXnh v3on0mq06o7GtQ1WleDhdK+cKQ== X-Google-Smtp-Source: AA0mqf5P0EXzSYerNvMnE1tqLXFMHI3GyGO5br1pN4dVVzBmBdh/UJ2E1Tze6Cn2gf/kxMeSGKg5YA== X-Received: by 2002:ac2:560a:0:b0:4b6:f0ea:4f49 with SMTP id v10-20020ac2560a000000b004b6f0ea4f49mr7591911lfd.9.1671462912024; Mon, 19 Dec 2022 07:15:12 -0800 (PST) Received: from krzk-bin.NAT.warszawa.vectranet.pl (088156142067.dynamic-2-waw-k-3-2-0.vectranet.pl. [88.156.142.67]) by smtp.gmail.com with ESMTPSA id e1-20020a05651236c100b004bd8534ebbcsm1109894lfs.37.2022.12.19.07.15.10 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 19 Dec 2022 07:15:11 -0800 (PST) From: Krzysztof Kozlowski To: "Rafael J. Wysocki" , Len Brown , Pavel Machek , Greg Kroah-Hartman , Kevin Hilman , Ulf Hansson , Daniel Lezcano , Lorenzo Pieralisi , Sudeep Holla , linux-pm@vger.kernel.org, linux-kernel@vger.kernel.org, linux-arm-kernel@lists.infradead.org Cc: Krzysztof Kozlowski , Adrien Thierry , Brian Masney , linux-rt-users@vger.kernel.org Subject: [PATCH v2 1/5] PM: domains: Add GENPD_FLAG_RT_SAFE for PREEMPT_RT Date: Mon, 19 Dec 2022 16:14:59 +0100 Message-Id: <20221219151503.385816-2-krzysztof.kozlowski@linaro.org> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20221219151503.385816-1-krzysztof.kozlowski@linaro.org> References: <20221219151503.385816-1-krzysztof.kozlowski@linaro.org> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-pm@vger.kernel.org Realtime kernels with PREEMPT_RT must use raw_spinlock_t for domains which are invoked from CPU idle (thus from atomic section). Example is cpuidle PSCI domain driver which itself is PREEMPT_RT safe, but is being called as part of cpuidle. Add a flag allowing a power domain provider to indicate it is RT safe. The flag is supposed to be used with existing GENPD_FLAG_IRQ_SAFE. Cc: Adrien Thierry Cc: Brian Masney Cc: linux-rt-users@vger.kernel.org Signed-off-by: Krzysztof Kozlowski --- Independently from Adrien, I encountered the same problem around genpd when using PREEMPT_RT kernel. Previous patch by Adrien: https://lore.kernel.org/all/20220615203605.1068453-1-athierry@redhat.com/ --- drivers/base/power/domain.c | 59 +++++++++++++++++++++++++++++++++++-- include/linux/pm_domain.h | 13 ++++++++ 2 files changed, 70 insertions(+), 2 deletions(-) diff --git a/drivers/base/power/domain.c b/drivers/base/power/domain.c index 967bcf9d415e..4dfce1d476f4 100644 --- a/drivers/base/power/domain.c +++ b/drivers/base/power/domain.c @@ -119,6 +119,48 @@ static const struct genpd_lock_ops genpd_spin_ops = { .unlock = genpd_unlock_spin, }; +static void genpd_lock_rawspin(struct generic_pm_domain *genpd) + __acquires(&genpd->rslock) +{ + unsigned long flags; + + raw_spin_lock_irqsave(&genpd->rslock, flags); + genpd->rlock_flags = flags; +} + +static void genpd_lock_nested_rawspin(struct generic_pm_domain *genpd, + int depth) + __acquires(&genpd->rslock) +{ + unsigned long flags; + + raw_spin_lock_irqsave_nested(&genpd->rslock, flags, depth); + genpd->rlock_flags = flags; +} + +static int genpd_lock_interruptible_rawspin(struct generic_pm_domain *genpd) + __acquires(&genpd->rslock) +{ + unsigned long flags; + + raw_spin_lock_irqsave(&genpd->rslock, flags); + genpd->rlock_flags = flags; + return 0; +} + +static void genpd_unlock_rawspin(struct generic_pm_domain *genpd) + __releases(&genpd->rslock) +{ + raw_spin_unlock_irqrestore(&genpd->rslock, genpd->rlock_flags); +} + +static const struct genpd_lock_ops genpd_rawspin_ops = { + .lock = genpd_lock_rawspin, + .lock_nested = genpd_lock_nested_rawspin, + .lock_interruptible = genpd_lock_interruptible_rawspin, + .unlock = genpd_unlock_rawspin, +}; + #define genpd_lock(p) p->lock_ops->lock(p) #define genpd_lock_nested(p, d) p->lock_ops->lock_nested(p, d) #define genpd_lock_interruptible(p) p->lock_ops->lock_interruptible(p) @@ -126,6 +168,8 @@ static const struct genpd_lock_ops genpd_spin_ops = { #define genpd_status_on(genpd) (genpd->status == GENPD_STATE_ON) #define genpd_is_irq_safe(genpd) (genpd->flags & GENPD_FLAG_IRQ_SAFE) +#define genpd_is_rt_safe(genpd) (genpd_is_irq_safe(genpd) && \ + (genpd->flags & GENPD_FLAG_RT_SAFE)) #define genpd_is_always_on(genpd) (genpd->flags & GENPD_FLAG_ALWAYS_ON) #define genpd_is_active_wakeup(genpd) (genpd->flags & GENPD_FLAG_ACTIVE_WAKEUP) #define genpd_is_cpu_domain(genpd) (genpd->flags & GENPD_FLAG_CPU_DOMAIN) @@ -1838,6 +1882,12 @@ static int genpd_add_subdomain(struct generic_pm_domain *genpd, return -EINVAL; } + if (!genpd_is_rt_safe(genpd) && genpd_is_rt_safe(subdomain)) { + WARN(1, "Parent %s of subdomain %s must be RT safe\n", + genpd->name, subdomain->name); + return -EINVAL; + } + link = kzalloc(sizeof(*link), GFP_KERNEL); if (!link) return -ENOMEM; @@ -2008,8 +2058,13 @@ static void genpd_free_data(struct generic_pm_domain *genpd) static void genpd_lock_init(struct generic_pm_domain *genpd) { if (genpd->flags & GENPD_FLAG_IRQ_SAFE) { - spin_lock_init(&genpd->slock); - genpd->lock_ops = &genpd_spin_ops; + if (genpd->flags & GENPD_FLAG_RT_SAFE) { + raw_spin_lock_init(&genpd->rslock); + genpd->lock_ops = &genpd_rawspin_ops; + } else { + spin_lock_init(&genpd->slock); + genpd->lock_ops = &genpd_spin_ops; + } } else { mutex_init(&genpd->mlock); genpd->lock_ops = &genpd_mtx_ops; diff --git a/include/linux/pm_domain.h b/include/linux/pm_domain.h index 1cd41bdf73cf..0a1600244963 100644 --- a/include/linux/pm_domain.h +++ b/include/linux/pm_domain.h @@ -61,6 +61,14 @@ * GENPD_FLAG_MIN_RESIDENCY: Enable the genpd governor to consider its * components' next wakeup when determining the * optimal idle state. + * + * GENPD_FLAG_RT_SAFE: When used with GENPD_FLAG_IRQ_SAFE, this informs + * genpd that its backend callbacks, ->power_on|off(), + * do not use other spinlocks. They might use + * raw_spinlocks or other pre-emption-disable + * methods, all of which are PREEMPT_RT safe. Note + * that, a genpd having this flag set, requires its + * masterdomains to also have it set. */ #define GENPD_FLAG_PM_CLK (1U << 0) #define GENPD_FLAG_IRQ_SAFE (1U << 1) @@ -69,6 +77,7 @@ #define GENPD_FLAG_CPU_DOMAIN (1U << 4) #define GENPD_FLAG_RPM_ALWAYS_ON (1U << 5) #define GENPD_FLAG_MIN_RESIDENCY (1U << 6) +#define GENPD_FLAG_RT_SAFE (1U << 7) enum gpd_status { GENPD_STATE_ON = 0, /* PM domain is on */ @@ -164,6 +173,10 @@ struct generic_pm_domain { spinlock_t slock; unsigned long lock_flags; }; + struct { + raw_spinlock_t rslock; + unsigned long rlock_flags; + }; }; }; From patchwork Mon Dec 19 15:15:00 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Krzysztof Kozlowski X-Patchwork-Id: 13076709 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id ED74FC10F1B for ; Mon, 19 Dec 2022 15:15:19 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232869AbiLSPPS (ORCPT ); Mon, 19 Dec 2022 10:15:18 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:43052 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232849AbiLSPPQ (ORCPT ); Mon, 19 Dec 2022 10:15:16 -0500 Received: from mail-lf1-x12a.google.com (mail-lf1-x12a.google.com [IPv6:2a00:1450:4864:20::12a]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id C9146BF45 for ; Mon, 19 Dec 2022 07:15:14 -0800 (PST) Received: by mail-lf1-x12a.google.com with SMTP id b3so14153044lfv.2 for ; Mon, 19 Dec 2022 07:15:14 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=710npLST/xPzuXiih2c1d9pFv9jXZIz92buQVuG0oAE=; b=uEio8RqgLj7kCKnMrT/ndnvx/qtape0C21ameHDn5oxR1frx1ZVQ/UjjKEP/ibeWWh /FrAeFueaLx7BNbF218RzyfLQhs6ts56jbLBL2inGVZ8II/hhK38SdmB46mvrjRjuhWs LneaquS3DyNt64GkklXVgbkS2zL9bf6bEp74uvqATEfY07UyjJTqmZM+QFcOLM4uy8Bo CE/mOMKX0Cwc47pRYdikRdnVWrHoU5LoezIzgPZcUuo1I6KJTFkMPX9rvq5ht+fmfcMC ONvGwjkQVW0VtSI8uJhE12tdg/7q+CM2fWOMU6PmvL3H+i1ZDUpFhpWuX439KLEKV4MS LZ7Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=710npLST/xPzuXiih2c1d9pFv9jXZIz92buQVuG0oAE=; b=mO4DVsiHNwITC+tr2NLhp8Gkc6l90yAJ+p+ECr9/Od6+V9MfktABW+SHH6V1zgYXsR rzAxdltMwrU1pNgyuBxtQvdX7/8Ht0tcv3NiiPnyGU+SvTWrjF45hdawkTthIe1ry4Vm WsZ/MyTMoVFl9rmus1QYvqWDqLz7pYO1J+sZs5TK5rQ+7KdhStx3G1iHM1qY9vihLCLm gB51R+7NfxBKIEajRM8pOSJHQNUwd9NedtOw/ouCw55ytIHsZjpKz25bD7PdknYl50nz j7sDJozkHdTUhlpayUERmKZSqdfAaPrF9PA4TwqOvIaxE++ZE4I84mEkJtmfmdpizzDD DkNw== X-Gm-Message-State: ANoB5plVKMtTw8wIUAWUtEVDVF9stK9am+Fke01fhvBEjmvWt8BNFpNf 5n/k1gT99u50vqlrvjfjr4zm+w== X-Google-Smtp-Source: AA0mqf5O5lOshUMBMDUAarE8CUlxxQDab0aOfGYCt75u7yth94O0C4yTULqeDUE5eHe18yeibJ7oYw== X-Received: by 2002:a05:6512:261b:b0:4b4:dd3d:4310 with SMTP id bt27-20020a056512261b00b004b4dd3d4310mr16841834lfb.19.1671462913205; Mon, 19 Dec 2022 07:15:13 -0800 (PST) Received: from krzk-bin.NAT.warszawa.vectranet.pl (088156142067.dynamic-2-waw-k-3-2-0.vectranet.pl. [88.156.142.67]) by smtp.gmail.com with ESMTPSA id e1-20020a05651236c100b004bd8534ebbcsm1109894lfs.37.2022.12.19.07.15.12 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 19 Dec 2022 07:15:12 -0800 (PST) From: Krzysztof Kozlowski To: "Rafael J. Wysocki" , Len Brown , Pavel Machek , Greg Kroah-Hartman , Kevin Hilman , Ulf Hansson , Daniel Lezcano , Lorenzo Pieralisi , Sudeep Holla , linux-pm@vger.kernel.org, linux-kernel@vger.kernel.org, linux-arm-kernel@lists.infradead.org Cc: Krzysztof Kozlowski , Adrien Thierry , Brian Masney , linux-rt-users@vger.kernel.org Subject: [PATCH v2 2/5] cpuidle: psci: Mark as PREEMPT_RT safe Date: Mon, 19 Dec 2022 16:15:00 +0100 Message-Id: <20221219151503.385816-3-krzysztof.kozlowski@linaro.org> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20221219151503.385816-1-krzysztof.kozlowski@linaro.org> References: <20221219151503.385816-1-krzysztof.kozlowski@linaro.org> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-pm@vger.kernel.org The PSCI cpuidle power domain in power_off callback uses __this_cpu_write() so it is PREEMPT_RT safe. This allows to use it in Realtime kernels and solves errors like: BUG: scheduling while atomic: swapper/2/0/0x00000002 Hardware name: Qualcomm Technologies, Inc. Robotics RB5 (DT) Call trace: dump_backtrace.part.0+0xe0/0xf0 show_stack+0x18/0x40 dump_stack_lvl+0x68/0x84 dump_stack+0x18/0x34 __schedule_bug+0x60/0x80 __schedule+0x628/0x800 schedule_rtlock+0x28/0x5c rtlock_slowlock_locked+0x360/0xd30 rt_spin_lock+0x88/0xb0 genpd_lock_nested_spin+0x1c/0x30 genpd_power_off.part.0.isra.0+0x20c/0x2a0 genpd_runtime_suspend+0x150/0x2bc __rpm_callback+0x48/0x170 rpm_callback+0x6c/0x7c rpm_suspend+0x108/0x660 __pm_runtime_suspend+0x4c/0x8c __psci_enter_domain_idle_state.constprop.0+0x54/0xe0 psci_enter_domain_idle_state+0x18/0x2c cpuidle_enter_state+0x8c/0x4e0 cpuidle_enter+0x38/0x50 do_idle+0x248/0x2f0 cpu_startup_entry+0x24/0x30 secondary_start_kernel+0x130/0x154 __secondary_switched+0xb0/0xb4 Cc: Adrien Thierry Cc: Brian Masney Cc: linux-rt-users@vger.kernel.org Signed-off-by: Krzysztof Kozlowski --- drivers/cpuidle/cpuidle-psci-domain.c | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/drivers/cpuidle/cpuidle-psci-domain.c b/drivers/cpuidle/cpuidle-psci-domain.c index c80cf9ddabd8..d15a91fb7048 100644 --- a/drivers/cpuidle/cpuidle-psci-domain.c +++ b/drivers/cpuidle/cpuidle-psci-domain.c @@ -62,7 +62,8 @@ static int psci_pd_init(struct device_node *np, bool use_osi) if (!pd_provider) goto free_pd; - pd->flags |= GENPD_FLAG_IRQ_SAFE | GENPD_FLAG_CPU_DOMAIN; + pd->flags |= GENPD_FLAG_IRQ_SAFE | GENPD_FLAG_RT_SAFE | \ + GENPD_FLAG_CPU_DOMAIN; /* Allow power off when OSI has been successfully enabled. */ if (use_osi) From patchwork Mon Dec 19 15:15:01 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Krzysztof Kozlowski X-Patchwork-Id: 13076711 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 3A4C2C3DA78 for ; Mon, 19 Dec 2022 15:15:24 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232885AbiLSPPV (ORCPT ); Mon, 19 Dec 2022 10:15:21 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:43120 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232865AbiLSPPR (ORCPT ); Mon, 19 Dec 2022 10:15:17 -0500 Received: from mail-lf1-x133.google.com (mail-lf1-x133.google.com [IPv6:2a00:1450:4864:20::133]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 021B810560 for ; Mon, 19 Dec 2022 07:15:16 -0800 (PST) Received: by mail-lf1-x133.google.com with SMTP id o6so9234991lfi.5 for ; Mon, 19 Dec 2022 07:15:15 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=NLwTozcUqVuaex+an++z/NMr4xtzIfovpnza53ZqTDE=; b=Ph1MMT+VKo3WhiyGRG1FZKkXojebF95OOhoxbbAdyKuxvtcuJHNFcTz+GQryrGBEJH JgoEcPjmfRrTHRZ6gQrzc0a97FezxX/lSEA5WkJkWTxfOmkD7BWFipsPNylStoKPLclU 4juNW9JRwUbpvvXYTgXaUp4kKivBGFRoR05rQgeYzBZ5+4gdktxjA3Mp7NsfrUCuZiAn 271GvZZQJo0W+1njeKjlKWlG7nJIT4KFHgVz9W1wNIB5EsXZdM/BJwAYkkXEkioVughp VFbbXbXy939GC7TRbJxNCrsD7XBrkTuQQLbJyg0zNVx+iQq/cWbFGjEYLrgt05etQPnn gOkw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=NLwTozcUqVuaex+an++z/NMr4xtzIfovpnza53ZqTDE=; b=W4RNuxRi0DiypcE2JFqwgnzPqFzvyn+V/3oG5Fg1Wok9dR/vYkQAfW06e/kK0z/uRO UVdqgqf80lJbR1WcNbm7qcgiOfYAfIOohHorloGAK79hyGRto4ps5b97twWCUPAOUfp8 DkwI2C9U6I0Pcg8I4x+IfEHTiBMBm1QQL7SsXe3g8aQ1ooWNXOFMJy0VI55f+yrMTSKl NE1ihtHo2HlrOvI9GhZr895dwGHXnQbUBOFJaVJdyttT6GbDBUEb80LB94VgHVwINdXQ yUKAQTeRS8DfsRYODVv2tC/hx+HtD9vua03VKW+0ySwgSGVDB5SrQTpY88gbjRIefEyD FlrQ== X-Gm-Message-State: AFqh2kr6aC6KdramXZNojLmBQUC+88ZfS4v3TjMbSY/kHfW0mVBTyQUx na654YUH6EDB5RgIy8E2ohn3CQ== X-Google-Smtp-Source: AMrXdXuGYlf+vpB46IweTHtie63ev7zPtyxOiEOJ2TTi6R+ghe6q6e1/IXIJXq9YScrmKCSb2fxeRg== X-Received: by 2002:a05:6512:695:b0:4c3:3a78:eab6 with SMTP id t21-20020a056512069500b004c33a78eab6mr4004090lfe.65.1671462914357; Mon, 19 Dec 2022 07:15:14 -0800 (PST) Received: from krzk-bin.NAT.warszawa.vectranet.pl (088156142067.dynamic-2-waw-k-3-2-0.vectranet.pl. [88.156.142.67]) by smtp.gmail.com with ESMTPSA id e1-20020a05651236c100b004bd8534ebbcsm1109894lfs.37.2022.12.19.07.15.13 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 19 Dec 2022 07:15:14 -0800 (PST) From: Krzysztof Kozlowski To: "Rafael J. Wysocki" , Len Brown , Pavel Machek , Greg Kroah-Hartman , Kevin Hilman , Ulf Hansson , Daniel Lezcano , Lorenzo Pieralisi , Sudeep Holla , linux-pm@vger.kernel.org, linux-kernel@vger.kernel.org, linux-arm-kernel@lists.infradead.org Cc: Krzysztof Kozlowski , Adrien Thierry , Brian Masney , linux-rt-users@vger.kernel.org Subject: [PATCH v2 3/5] cpuidle: psci: Do not suspend topology CPUs on PREEMPT_RT Date: Mon, 19 Dec 2022 16:15:01 +0100 Message-Id: <20221219151503.385816-4-krzysztof.kozlowski@linaro.org> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20221219151503.385816-1-krzysztof.kozlowski@linaro.org> References: <20221219151503.385816-1-krzysztof.kozlowski@linaro.org> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-pm@vger.kernel.org Realtime kernels are not supposed to go into deep sleep modes (neither system nor CPUs) because the latencies might become unpredictable. Therefore actual power suspending of CPU topology is not that important. On the other hand, this runtime PM of CPU topology is not compatible with PREEMPT_RT: 1. Core cpuidle path disables IRQs. 2. Core cpuidle calls cpuidle-psci. 3. cpuidle-psci in __psci_enter_domain_idle_state() calls pm_runtime_put_sync_suspend() and pm_runtime_get_sync() which use spinlocks (which are sleeping on PREEMPT_RT): BUG: sleeping function called from invalid context at kernel/locking/spinlock_rt.c:46 in_atomic(): 1, irqs_disabled(): 128, non_block: 0, pid: 0, name: swapper/0 preempt_count: 1, expected: 0 RCU nest depth: 0, expected: 0 1 lock held by swapper/0/0: #0: ffff20524613c1a0 (&dev->power.lock){+.+.}-{3:3}, at: __pm_runtime_suspend+0x30/0xac irq event stamp: 18776 hardirqs last enabled at (18775): [] tick_nohz_idle_enter+0x7c/0x17c hardirqs last disabled at (18776): [] do_idle+0xe0/0x300 softirqs last enabled at (4310): [] __local_bh_enable_ip+0x98/0x280 softirqs last disabled at (4288): [] cgroup_idr_alloc.constprop.0+0x0/0xd0 Preemption disabled at: [] schedule_preempt_disabled+0x20/0x30 CPU: 0 PID: 0 Comm: swapper/0 Tainted: G W 6.1.0-rt5-00322-gb49b67f1d8dc #1 Hardware name: Qualcomm Technologies, Inc. Robotics RB5 (DT) Call trace: dump_backtrace.part.0+0xe0/0xf0 show_stack+0x18/0x40 dump_stack_lvl+0x8c/0xb8 dump_stack+0x18/0x34 __might_resched+0x17c/0x214 rt_spin_lock+0x5c/0x100 __pm_runtime_suspend+0x30/0xac __psci_enter_domain_idle_state.constprop.0+0x60/0x104 psci_enter_domain_idle_state+0x18/0x2c cpuidle_enter_state+0x220/0x37c cpuidle_enter+0x38/0x50 do_idle+0x258/0x300 cpu_startup_entry+0x28/0x30 rest_init+0x104/0x180 arch_post_acpi_subsys_init+0x0/0x18 start_kernel+0x6fc/0x73c __primary_switched+0xbc/0xc4 Cc: Adrien Thierry Cc: Brian Masney Cc: linux-rt-users@vger.kernel.org Signed-off-by: Krzysztof Kozlowski --- drivers/cpuidle/cpuidle-psci.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/drivers/cpuidle/cpuidle-psci.c b/drivers/cpuidle/cpuidle-psci.c index 57bc3e3ae391..9d971cc4b12b 100644 --- a/drivers/cpuidle/cpuidle-psci.c +++ b/drivers/cpuidle/cpuidle-psci.c @@ -72,7 +72,7 @@ static int __psci_enter_domain_idle_state(struct cpuidle_device *dev, ct_irq_enter_irqson(); if (s2idle) dev_pm_genpd_suspend(pd_dev); - else + else if (!IS_ENABLED(CONFIG_PREEMPT_RT)) pm_runtime_put_sync_suspend(pd_dev); ct_irq_exit_irqson(); @@ -85,7 +85,7 @@ static int __psci_enter_domain_idle_state(struct cpuidle_device *dev, ct_irq_enter_irqson(); if (s2idle) dev_pm_genpd_resume(pd_dev); - else + else if (!IS_ENABLED(CONFIG_PREEMPT_RT)) pm_runtime_get_sync(pd_dev); ct_irq_exit_irqson(); From patchwork Mon Dec 19 15:15:02 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Krzysztof Kozlowski X-Patchwork-Id: 13076710 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 49019C4332F for ; Mon, 19 Dec 2022 15:15:22 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232880AbiLSPPT (ORCPT ); Mon, 19 Dec 2022 10:15:19 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:43024 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232855AbiLSPPR (ORCPT ); Mon, 19 Dec 2022 10:15:17 -0500 Received: from mail-lf1-x12a.google.com (mail-lf1-x12a.google.com [IPv6:2a00:1450:4864:20::12a]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id EF356F016 for ; Mon, 19 Dec 2022 07:15:15 -0800 (PST) Received: by mail-lf1-x12a.google.com with SMTP id q6so14075879lfm.10 for ; Mon, 19 Dec 2022 07:15:15 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=y2D0MZ2hxPTHwXNnho8HOkobrXZQciZPG6h42YTOjRw=; b=ucUJppIgRBgtpDQOSNoFKkTpre/wj89djnkxbJXoXfrK0vOY/QJLX1iUpqT5cY5qV/ GF+Dl82w3xJ9mA3oFvYtRV0YhivF59Kq+GNvnC3Jqv4aY+ohv2cJJWZlvBeaEZEzK6ou HIlLEZWK5gd1rofobutljGLWfO3nULqGfRQ0ICl88S/eoknin4PISCvnL7lw7XSStvto AE4nkyesvYnJaizYJIHpktZOUcmnuiuncKvdsRPUYSWCKWWwaxvzUs4Jljp2D+ucASqX xQDAJUDjIsYIBgPxGqZ7U5LO8uF94J/koNWBdV2fQu+agiJ3nra/ype6SXUSompNmfk6 bBwA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=y2D0MZ2hxPTHwXNnho8HOkobrXZQciZPG6h42YTOjRw=; b=xuYRKK4SXcXIaZGI3UnzuVFJhNj9N5Znmbg4pcZVREwKL52DIUDxM/iwEiVyGeTXPj P+t4uaqtyTA7ZdQxxwUP1Kr7gCPtr7A1vP1j9M9gdYrKAfNK/JAj9RH4OOernPVworZx KF+e8t2KDMkiLOCfS+lcmcwajwE/0pO/Tna1AyvnxcBNK2KKaNniqi6/6vXShYnh7x+u BxgDYQOoU6r9j1qiy7jMn1Dx++tPQS7RgoRTw/ISc3LMmpUPjEtNpljplWSLjBgp+GSa X01+rtD8MizzyXzvwlqPHyDtmQzZK7ZTkCYC2+OHabUzV0+5HQ/jcicz3/ph2c75WNeK Cz3Q== X-Gm-Message-State: ANoB5pn9WsTOvK+to2gVyvo8DmCmi84m9ZtJmDOePrdpNbo8HKm4hjJo bEEugdfK76BHJikA0yhpxupwRQ== X-Google-Smtp-Source: AA0mqf7n/NP8HRaYuYDbxI6GABLuMF7vUSdWaQUoDWCU0hAjFOQml5a/3nwbg+N3+37H31i1a6Yv2A== X-Received: by 2002:a05:6512:a94:b0:4b7:113:9296 with SMTP id m20-20020a0565120a9400b004b701139296mr14233163lfu.14.1671462915529; Mon, 19 Dec 2022 07:15:15 -0800 (PST) Received: from krzk-bin.NAT.warszawa.vectranet.pl (088156142067.dynamic-2-waw-k-3-2-0.vectranet.pl. [88.156.142.67]) by smtp.gmail.com with ESMTPSA id e1-20020a05651236c100b004bd8534ebbcsm1109894lfs.37.2022.12.19.07.15.14 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 19 Dec 2022 07:15:15 -0800 (PST) From: Krzysztof Kozlowski To: "Rafael J. Wysocki" , Len Brown , Pavel Machek , Greg Kroah-Hartman , Kevin Hilman , Ulf Hansson , Daniel Lezcano , Lorenzo Pieralisi , Sudeep Holla , linux-pm@vger.kernel.org, linux-kernel@vger.kernel.org, linux-arm-kernel@lists.infradead.org Cc: Krzysztof Kozlowski , Adrien Thierry , Brian Masney , linux-rt-users@vger.kernel.org Subject: [PATCH v2 4/5] PM: Allow calling dev_pm_domain_set() with raw spinlock Date: Mon, 19 Dec 2022 16:15:02 +0100 Message-Id: <20221219151503.385816-5-krzysztof.kozlowski@linaro.org> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20221219151503.385816-1-krzysztof.kozlowski@linaro.org> References: <20221219151503.385816-1-krzysztof.kozlowski@linaro.org> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-pm@vger.kernel.org device_pm_check_callbacks() uses dev->power spinlock, which on PREEMPT_RT sleeps. However some PM domains on PREEMPT_RT might be using raw spinlocks as genpd_lock(), thus dev_pm_domain_set() must not call device_pm_check_callbacks(). In fact device_pm_check_callbacks() is not strictly related to dev_pm_domain_set() and calls for these two can be made separately. Add new helper dev_pm_domain_set_no_cb() which will only set PM domain but will not check the callbacks, leaving the checl to the caller. Cc: Adrien Thierry Cc: Brian Masney Cc: linux-rt-users@vger.kernel.org Signed-off-by: Krzysztof Kozlowski --- drivers/base/power/common.c | 27 +++++++++++++++++++++++++-- include/linux/pm_domain.h | 3 +++ 2 files changed, 28 insertions(+), 2 deletions(-) diff --git a/drivers/base/power/common.c b/drivers/base/power/common.c index 72115917e0bd..f81cab6990ad 100644 --- a/drivers/base/power/common.c +++ b/drivers/base/power/common.c @@ -218,6 +218,30 @@ EXPORT_SYMBOL_GPL(dev_pm_domain_start); * This function must be called with the device lock held. */ void dev_pm_domain_set(struct device *dev, struct dev_pm_domain *pd) +{ + if (dev->pm_domain == pd) + return; + + dev_pm_domain_set_no_cb(dev, pd); + device_pm_check_callbacks(dev); +} +EXPORT_SYMBOL_GPL(dev_pm_domain_set); + +/** + * dev_pm_domain_set_no_cb - Set PM domain of a device. + * @dev: Device whose PM domain is to be set. + * @pd: PM domain to be set, or NULL. + * + * Sets the PM domain the device belongs to. The PM domain of a device needs + * to be set before its probe finishes (it's bound to a driver). + * + * This is exactly like dev_pm_domain_set(), however device_pm_check_callbacks() + * is not called and the caller is responsible to invoke + * device_pm_check_callbacks() with device lock held. + * + * This function must be called with the device lock held. + */ +void dev_pm_domain_set_no_cb(struct device *dev, struct dev_pm_domain *pd) { if (dev->pm_domain == pd) return; @@ -225,6 +249,5 @@ void dev_pm_domain_set(struct device *dev, struct dev_pm_domain *pd) WARN(pd && device_is_bound(dev), "PM domains can only be changed for unbound devices\n"); dev->pm_domain = pd; - device_pm_check_callbacks(dev); } -EXPORT_SYMBOL_GPL(dev_pm_domain_set); +EXPORT_SYMBOL_GPL(dev_pm_domain_set_no_cb); diff --git a/include/linux/pm_domain.h b/include/linux/pm_domain.h index 0a1600244963..352d0c76bfec 100644 --- a/include/linux/pm_domain.h +++ b/include/linux/pm_domain.h @@ -438,6 +438,7 @@ struct device *dev_pm_domain_attach_by_name(struct device *dev, void dev_pm_domain_detach(struct device *dev, bool power_off); int dev_pm_domain_start(struct device *dev); void dev_pm_domain_set(struct device *dev, struct dev_pm_domain *pd); +void dev_pm_domain_set_no_cb(struct device *dev, struct dev_pm_domain *pd); #else static inline int dev_pm_domain_attach(struct device *dev, bool power_on) { @@ -460,6 +461,8 @@ static inline int dev_pm_domain_start(struct device *dev) } static inline void dev_pm_domain_set(struct device *dev, struct dev_pm_domain *pd) {} +static inline void dev_pm_domain_set_no_cb(struct device *dev, + struct dev_pm_domain *pd) {} #endif #endif /* _LINUX_PM_DOMAIN_H */ From patchwork Mon Dec 19 15:15:03 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Krzysztof Kozlowski X-Patchwork-Id: 13076712 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 3BC74C4167B for ; Mon, 19 Dec 2022 15:15:26 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232865AbiLSPPX (ORCPT ); Mon, 19 Dec 2022 10:15:23 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:43086 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232849AbiLSPPS (ORCPT ); Mon, 19 Dec 2022 10:15:18 -0500 Received: from mail-lf1-x12b.google.com (mail-lf1-x12b.google.com [IPv6:2a00:1450:4864:20::12b]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 26DD411815 for ; Mon, 19 Dec 2022 07:15:17 -0800 (PST) Received: by mail-lf1-x12b.google.com with SMTP id q6so14075951lfm.10 for ; Mon, 19 Dec 2022 07:15:16 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=m8Uj/f0x4H68pfp6EBuS49q8mUm/dd1ntZTx2c5gQRw=; b=joGdkJgxChVHrhqXxh4D8sJDaKpzxBgUcFgLtPczIcYGrPWnS0gYSfGGb/xwur1BZA HXUG1BhIHBctF7bFBpebP9+UCrd1/wX6sQ06q9+cIYwobrrc2Zzo3WsBKRD3558LWzFQ /N47j8fJwjN1I4eRjd0Qr07xS6EOzoIv8QhoPs4d+muMgyaWlNH3o7v1b1Xr6gxm8Bsn zsRaxMej9uybQ/82hjh0l+j0+akCnf8EYggcdPI5wQ4zTU9cGGyx+pH3DhxSdkmYX4n/ boQmP89KDmwLe8i18IoxJ121DOicW9mnYp04EIuXSuoqBPamMqwF9q353kDRDAlSKDOO ifvw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=m8Uj/f0x4H68pfp6EBuS49q8mUm/dd1ntZTx2c5gQRw=; b=Dk+zQA1GHGTTfMxINFA1S9iq2du2NgTa0ij9wtnKGsHifQ6m2PFiaCQYLWgrdvgB48 JmPLLivogGYhg2FLSZrwFRHOuVEZDoLYePzB56fl2oUGDp4bx1Oej1M8mT795W0oDHd8 F8PC+BR+weqJdLHKsgEfVc4CrUpLny2TJH1ebNJm2ZL0KMy+N0ZSq2o8NbcciD9cuWs+ /eLHnzimRn8lXYKXxtREgwRmVJgk9lhIbV7eBk+hPa0F8GW5B6yL8dRA34aVzz63yWaB YT1PytDB43Jc195nk3foVin2s0sk+DWP6H8k8qOkM/ZJtnCjHrGMdIBhoxk7HdfqQ1KN Vvnw== X-Gm-Message-State: ANoB5pmjA5+TbC+Z6XOsdi01JN7GPRnrrCzN+MDLFk/Xs6kYvQ8KPzIW QOSQ/+Sda6yoHxaoKCsrwhHg+Q== X-Google-Smtp-Source: AA0mqf6TIA7wSK78A1e7LxWcZY/ah0RMdr0IMbRzO6DwrSrVhPg3kDS+rfPE3kSDU9y+6k9YFRKeKw== X-Received: by 2002:a05:6512:2c85:b0:4a4:68b9:607f with SMTP id dw5-20020a0565122c8500b004a468b9607fmr11616697lfb.10.1671462916643; Mon, 19 Dec 2022 07:15:16 -0800 (PST) Received: from krzk-bin.NAT.warszawa.vectranet.pl (088156142067.dynamic-2-waw-k-3-2-0.vectranet.pl. [88.156.142.67]) by smtp.gmail.com with ESMTPSA id e1-20020a05651236c100b004bd8534ebbcsm1109894lfs.37.2022.12.19.07.15.15 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 19 Dec 2022 07:15:16 -0800 (PST) From: Krzysztof Kozlowski To: "Rafael J. Wysocki" , Len Brown , Pavel Machek , Greg Kroah-Hartman , Kevin Hilman , Ulf Hansson , Daniel Lezcano , Lorenzo Pieralisi , Sudeep Holla , linux-pm@vger.kernel.org, linux-kernel@vger.kernel.org, linux-arm-kernel@lists.infradead.org Cc: Krzysztof Kozlowski , Adrien Thierry , Brian Masney , linux-rt-users@vger.kernel.org Subject: [PATCH v2 5/5] PM: domains: Do not call device_pm_check_callbacks() when holding genpd_lock() Date: Mon, 19 Dec 2022 16:15:03 +0100 Message-Id: <20221219151503.385816-6-krzysztof.kozlowski@linaro.org> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20221219151503.385816-1-krzysztof.kozlowski@linaro.org> References: <20221219151503.385816-1-krzysztof.kozlowski@linaro.org> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-pm@vger.kernel.org If PM domain on PREEMPT_RT is marked as GENPD_FLAG_RT_SAFE(), the genpd_lock() will be a raw spin lock, thus device_pm_check_callbacks() must be called outside of the domain lock. This solves on PREEMPT_RT: [ BUG: Invalid wait context ] 6.1.0-rt5-00325-g8a5f56bcfcca #8 Tainted: G W ----------------------------- swapper/0/1 is trying to lock: ffff76e045dec9a0 (&dev->power.lock){+.+.}-{3:3}, at: device_pm_check_callbacks+0x20/0xf0 other info that might help us debug this: context-{5:5} 3 locks held by swapper/0/1: #0: ffff76e045deb8e8 (&dev->mutex){....}-{4:4}, at: __device_attach+0x38/0x1c0 #1: ffffa92b81f825e0 (gpd_list_lock){+.+.}-{4:4}, at: __genpd_dev_pm_attach+0x7c/0x250 #2: ffff76e04105c7a0 (&genpd->rslock){....}-{2:2}, at: genpd_lock_rawspin+0x1c/0x30 stack backtrace: CPU: 5 PID: 1 Comm: swapper/0 Tainted: G W 6.1.0-rt5-00325-g8a5f56bcfcca #8 Hardware name: Qualcomm Technologies, Inc. Robotics RB5 (DT) Call trace: dump_backtrace.part.0+0xe0/0xf0 show_stack+0x18/0x40 dump_stack_lvl+0x8c/0xb8 dump_stack+0x18/0x34 __lock_acquire+0x938/0x2100 lock_acquire.part.0+0x104/0x28c lock_acquire+0x68/0x84 rt_spin_lock+0x40/0x100 device_pm_check_callbacks+0x20/0xf0 dev_pm_domain_set+0x54/0x64 genpd_add_device+0x258/0x340 __genpd_dev_pm_attach+0xa8/0x250 genpd_dev_pm_attach_by_id+0xc4/0x190 genpd_dev_pm_attach_by_name+0x3c/0x60 dev_pm_domain_attach_by_name+0x20/0x30 dt_idle_attach_cpu+0x24/0x90 psci_cpuidle_probe+0x300/0x4b0 platform_probe+0x68/0xe0 really_probe+0xbc/0x2dc __driver_probe_device+0x78/0xe0 driver_probe_device+0x3c/0x160 __device_attach_driver+0xb8/0x140 bus_for_each_drv+0x78/0xd0 __device_attach+0xa8/0x1c0 device_initial_probe+0x14/0x20 bus_probe_device+0x9c/0xa4 device_add+0x3b4/0x8dc platform_device_add+0x114/0x234 platform_device_register_full+0x108/0x1a4 psci_idle_init+0x6c/0xb0 do_one_initcall+0x74/0x450 kernel_init_freeable+0x2e0/0x350 kernel_init+0x24/0x130 ret_from_fork+0x10/0x20 Cc: Adrien Thierry Cc: Brian Masney Cc: linux-rt-users@vger.kernel.org Signed-off-by: Krzysztof Kozlowski --- drivers/base/power/domain.c | 6 +++++- 1 file changed, 5 insertions(+), 1 deletion(-) diff --git a/drivers/base/power/domain.c b/drivers/base/power/domain.c index 4dfce1d476f4..db499ba40497 100644 --- a/drivers/base/power/domain.c +++ b/drivers/base/power/domain.c @@ -1666,10 +1666,14 @@ static int genpd_add_device(struct generic_pm_domain *genpd, struct device *dev, if (ret) goto out; + + /* PREEMPT_RT: Must be outside of genpd_lock */ + device_pm_check_callbacks(dev); + genpd_lock(genpd); genpd_set_cpumask(genpd, gpd_data->cpu); - dev_pm_domain_set(dev, &genpd->domain); + dev_pm_domain_set_no_cb(dev, &genpd->domain); genpd->device_count++; if (gd)