From patchwork Tue Jul 24 18:26:04 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Waiman Long X-Patchwork-Id: 10542919 X-Patchwork-Delegate: rjw@sisk.pl Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 22B5917FD for ; Tue, 24 Jul 2018 18:26:29 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 1199229072 for ; Tue, 24 Jul 2018 18:26:29 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 058A32908C; Tue, 24 Jul 2018 18:26:29 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.9 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 9CAE429072 for ; Tue, 24 Jul 2018 18:26:28 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S2388592AbeGXTeA (ORCPT ); Tue, 24 Jul 2018 15:34:00 -0400 Received: from mx3-rdu2.redhat.com ([66.187.233.73]:51334 "EHLO mx1.redhat.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S2388495AbeGXTd7 (ORCPT ); Tue, 24 Jul 2018 15:33:59 -0400 Received: from smtp.corp.redhat.com (int-mx03.intmail.prod.int.rdu2.redhat.com [10.11.54.3]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 44A5E7A7E5; Tue, 24 Jul 2018 18:26:15 +0000 (UTC) Received: from llong.com (dhcp-17-175.bos.redhat.com [10.18.17.175]) by smtp.corp.redhat.com (Postfix) with ESMTP id EBE84111AF19; Tue, 24 Jul 2018 18:26:14 +0000 (UTC) From: Waiman Long To: "Rafael J. Wysocki" , Viresh Kumar , Thomas Gleixner , Peter Zijlstra , Ingo Molnar Cc: linux-kernel@vger.kernel.org, linux-pm@vger.kernel.org, "Paul E. McKenney" , Greg Kroah-Hartman , Konrad Rzeszutek Wilk , Waiman Long Subject: [PATCH v2 1/2] cpu/hotplug: Add a cpus_read_trylock() function Date: Tue, 24 Jul 2018 14:26:04 -0400 Message-Id: <1532456765-17576-2-git-send-email-longman@redhat.com> In-Reply-To: <1532456765-17576-1-git-send-email-longman@redhat.com> References: <1532456765-17576-1-git-send-email-longman@redhat.com> X-Scanned-By: MIMEDefang 2.78 on 10.11.54.3 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.11.55.2]); Tue, 24 Jul 2018 18:26:15 +0000 (UTC) X-Greylist: inspected by milter-greylist-4.5.16 (mx1.redhat.com [10.11.55.2]); Tue, 24 Jul 2018 18:26:15 +0000 (UTC) for IP:'10.11.54.3' DOMAIN:'int-mx03.intmail.prod.int.rdu2.redhat.com' HELO:'smtp.corp.redhat.com' FROM:'longman@redhat.com' RCPT:'' Sender: linux-pm-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-pm@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP There are use cases where it can be useful to have a cpus_read_trylock() function to work around circular lock dependency problem involving the cpu_hotplug_lock. Signed-off-by: Waiman Long --- include/linux/cpu.h | 2 ++ kernel/cpu.c | 6 ++++++ 2 files changed, 8 insertions(+) diff --git a/include/linux/cpu.h b/include/linux/cpu.h index a97a63e..e850bfe 100644 --- a/include/linux/cpu.h +++ b/include/linux/cpu.h @@ -103,6 +103,7 @@ static inline void cpu_maps_update_done(void) extern void cpus_write_unlock(void); extern void cpus_read_lock(void); extern void cpus_read_unlock(void); +extern int cpus_read_trylock(void); extern void lockdep_assert_cpus_held(void); extern void cpu_hotplug_disable(void); extern void cpu_hotplug_enable(void); @@ -115,6 +116,7 @@ static inline void cpus_write_lock(void) { } static inline void cpus_write_unlock(void) { } static inline void cpus_read_lock(void) { } static inline void cpus_read_unlock(void) { } +static inline int cpus_read_trylock(void) { return true; } static inline void lockdep_assert_cpus_held(void) { } static inline void cpu_hotplug_disable(void) { } static inline void cpu_hotplug_enable(void) { } diff --git a/kernel/cpu.c b/kernel/cpu.c index 0db8938..307486b 100644 --- a/kernel/cpu.c +++ b/kernel/cpu.c @@ -290,6 +290,12 @@ void cpus_read_lock(void) } EXPORT_SYMBOL_GPL(cpus_read_lock); +int cpus_read_trylock(void) +{ + return percpu_down_read_trylock(&cpu_hotplug_lock); +} +EXPORT_SYMBOL_GPL(cpus_read_trylock); + void cpus_read_unlock(void) { percpu_up_read(&cpu_hotplug_lock); From patchwork Tue Jul 24 18:26:05 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Waiman Long X-Patchwork-Id: 10542917 X-Patchwork-Delegate: rjw@sisk.pl Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id B7A0917FD for ; Tue, 24 Jul 2018 18:26:25 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id A426629072 for ; Tue, 24 Jul 2018 18:26:25 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 975EA2908A; Tue, 24 Jul 2018 18:26:25 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.9 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 2694529072 for ; Tue, 24 Jul 2018 18:26:25 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S2388495AbeGXTeB (ORCPT ); Tue, 24 Jul 2018 15:34:01 -0400 Received: from mx3-rdu2.redhat.com ([66.187.233.73]:52076 "EHLO mx1.redhat.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S2388427AbeGXTeA (ORCPT ); Tue, 24 Jul 2018 15:34:00 -0400 Received: from smtp.corp.redhat.com (int-mx03.intmail.prod.int.rdu2.redhat.com [10.11.54.3]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 99998DC41; Tue, 24 Jul 2018 18:26:15 +0000 (UTC) Received: from llong.com (dhcp-17-175.bos.redhat.com [10.18.17.175]) by smtp.corp.redhat.com (Postfix) with ESMTP id 4B75D111AF18; Tue, 24 Jul 2018 18:26:15 +0000 (UTC) From: Waiman Long To: "Rafael J. Wysocki" , Viresh Kumar , Thomas Gleixner , Peter Zijlstra , Ingo Molnar Cc: linux-kernel@vger.kernel.org, linux-pm@vger.kernel.org, "Paul E. McKenney" , Greg Kroah-Hartman , Konrad Rzeszutek Wilk , Waiman Long Subject: [PATCH v2 2/2] cpufreq: Fix a circular lock dependency problem Date: Tue, 24 Jul 2018 14:26:05 -0400 Message-Id: <1532456765-17576-3-git-send-email-longman@redhat.com> In-Reply-To: <1532456765-17576-1-git-send-email-longman@redhat.com> References: <1532456765-17576-1-git-send-email-longman@redhat.com> X-Scanned-By: MIMEDefang 2.78 on 10.11.54.3 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.11.55.1]); Tue, 24 Jul 2018 18:26:15 +0000 (UTC) X-Greylist: inspected by milter-greylist-4.5.16 (mx1.redhat.com [10.11.55.1]); Tue, 24 Jul 2018 18:26:15 +0000 (UTC) for IP:'10.11.54.3' DOMAIN:'int-mx03.intmail.prod.int.rdu2.redhat.com' HELO:'smtp.corp.redhat.com' FROM:'longman@redhat.com' RCPT:'' Sender: linux-pm-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-pm@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP With lockdep turned on, the following circular lock dependency problem was reported: [ 57.470040] ====================================================== [ 57.502900] WARNING: possible circular locking dependency detected [ 57.535208] 4.18.0-0.rc3.1.el8+7.x86_64+debug #1 Tainted: G [ 57.577761] ------------------------------------------------------ [ 57.609714] tuned/1505 is trying to acquire lock: [ 57.633808] 00000000559deec5 (cpu_hotplug_lock.rw_sem){++++}, at: store+0x27/0x120 [ 57.672880] [ 57.672880] but task is already holding lock: [ 57.702184] 000000002136ca64 (kn->count#118){++++}, at: kernfs_fop_write+0x1d0/0x410 [ 57.742176] [ 57.742176] which lock already depends on the new lock. [ 57.742176] [ 57.785220] [ 57.785220] the existing dependency chain (in reverse order) is: : [ 58.932512] other info that might help us debug this: [ 58.932512] [ 58.973344] Chain exists of: [ 58.973344] cpu_hotplug_lock.rw_sem --> subsys mutex#5 --> kn->count#118 [ 58.973344] [ 59.030795] Possible unsafe locking scenario: [ 59.030795] [ 59.061248] CPU0 CPU1 [ 59.085377] ---- ---- [ 59.108160] lock(kn->count#118); [ 59.124935] lock(subsys mutex#5); [ 59.156330] lock(kn->count#118); [ 59.186088] lock(cpu_hotplug_lock.rw_sem); [ 59.208541] [ 59.208541] *** DEADLOCK *** In the cpufreq_register_driver() function, the lock sequence is: cpus_read_lock --> kn->count For the cpufreq sysfs store method, the lock sequence is: kn->count --> cpus_read_lock These sequences are actually safe as they are taking a share lock on cpu_hotplug_lock. However, the current lockdep code doesn't check for share locking when detecting circular lock dependency. Fixing that could be a substantial effort. Instead, we can work around this problem by using cpus_read_trylock() in the store method which is much simpler. The chance of not getting the read lock is very small. If that happens, the userspace application that writes the sysfs file will get an error. Signed-off-by: Waiman Long --- drivers/cpufreq/cpufreq.c | 7 ++++++- 1 file changed, 6 insertions(+), 1 deletion(-) diff --git a/drivers/cpufreq/cpufreq.c b/drivers/cpufreq/cpufreq.c index b0dfd32..1ddb5ce 100644 --- a/drivers/cpufreq/cpufreq.c +++ b/drivers/cpufreq/cpufreq.c @@ -923,7 +923,12 @@ static ssize_t store(struct kobject *kobj, struct attribute *attr, struct freq_attr *fattr = to_attr(attr); ssize_t ret = -EINVAL; - cpus_read_lock(); + /* + * cpus_read_trylock() is used here to work around a circular lock + * dependency problem with respect to the cpufreq_register_driver(). + */ + if (!cpus_read_trylock()) + return -EBUSY; if (cpu_online(policy->cpu)) { down_write(&policy->rwsem);