From patchwork Mon Feb 4 17:09:04 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Robin Murphy X-Patchwork-Id: 10796155 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 146B013A4 for ; Mon, 4 Feb 2019 17:09:57 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 002132B668 for ; Mon, 4 Feb 2019 17:09:56 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id E822A2B6A6; Mon, 4 Feb 2019 17:09:56 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-5.2 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,MAILING_LIST_MULTI,RCVD_IN_DNSWL_MED autolearn=ham version=3.3.1 Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.wl.linuxfoundation.org (Postfix) with ESMTPS id 594A32B668 for ; Mon, 4 Feb 2019 17:09:56 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20170209; h=Sender: Content-Transfer-Encoding:Content-Type:Cc:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Message-Id:Date:Subject:To:From:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=60gn5UzK++IGOMdQU7zZ4WIJ0D4SsEa8D5tYW9SKTmE=; b=k3M44oyaO7wgiJ gAf+u9NPYbG5RNNTuz8BL14MZ+SaDXlR5dYa40eh/rsfJOxGjGjFbWTXEZZF6Rvdr//ugKuC8CBzq 66OLuKBnLOULhuJ5qG0nc8iaS+FGErmm5UArkbHsDAHPXI3tFWTBr856gpK6RYSTGuSwii3Mp/7bQ uMNnjUXcqZ5lDfxTcYvrxIGsPy2Jc7xzzN/aWw9YbhNE0hvGZy2jnI9j3x41w9Qe4yJVYdy+1+6Nh rk6OJepisrdyUrOA12wVEmFA4qanVKjJ5UcC62sOiTGZLIlSFYbNfnqZE1BqXjDl3AcCZMyXtwwGc hPhcWWd18akikaQWquzQ==; Received: from localhost ([127.0.0.1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.90_1 #2 (Red Hat Linux)) id 1gqhkn-0003US-8V; Mon, 04 Feb 2019 17:09:53 +0000 Received: from foss.arm.com ([217.140.101.70]) by bombadil.infradead.org with esmtp (Exim 4.90_1 #2 (Red Hat Linux)) id 1gqhkc-0003HQ-CM for linux-arm-kernel@lists.infradead.org; Mon, 04 Feb 2019 17:09:46 +0000 Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.72.51.249]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id C17A515AB; Mon, 4 Feb 2019 09:09:41 -0800 (PST) Received: from e110467-lin.cambridge.arm.com (e110467-lin.cambridge.arm.com [10.1.196.75]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPA id 24BF13F589; Mon, 4 Feb 2019 09:09:40 -0800 (PST) From: Robin Murphy To: will.deacon@arm.com, mark.rutland@arm.com Subject: [PATCH 1/5] perf/arm-cci: Fix CPU hotplug race avoidance Date: Mon, 4 Feb 2019 17:09:04 +0000 Message-Id: <606ff8c3f7f35ccdcb4b52a49f692fb20e27359c.1549299188.git.robin.murphy@arm.com> X-Mailer: git-send-email 2.20.1.dirty In-Reply-To: References: MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20190204_090943_096904_0DB62D17 X-CRM114-Status: GOOD ( 17.98 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: suzuki.poulose@arm.com, peterz@infradead.org, bigeasy@linutronix.de, linux-kernel@vger.kernel.org, tglx@linutronix.de, "Li, Meng" , linux-arm-kernel@lists.infradead.org Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+patchwork-linux-arm=patchwork.kernel.org@lists.infradead.org X-Virus-Scanned: ClamAV using ClamSMTP The arm-cci probe logic faces a cyclic dependency wherein it has to pick a valid CPU to associate with before registering the PMU device, has to have the PMU state initialised before handling hotplug events in case it must be migrated, but has to have the hotplug notifier registered before the chosen CPU may go offline lest things get out of sync. The present code has tried to solve the races by using get_cpu() to pick the current CPU and prevent it from disappearing while the other two registrations are performed, but that results in taking mutexes with preemption disabled, which makes certain configurations very unhappy: [ 1.983337] BUG: sleeping function called from invalid context at kernel/locking/rtmutex.c:2004 [ 1.983340] in_atomic(): 1, irqs_disabled(): 0, pid: 1, name: swapper/0 [ 1.983342] Preemption disabled at: [ 1.983353] [] cci_pmu_probe+0x1dc/0x488 [ 1.983360] CPU: 0 PID: 1 Comm: swapper/0 Not tainted 4.18.20-rt8-yocto-preempt-rt #1 [ 1.983362] Hardware name: ZynqMP ZCU102 Rev1.0 (DT) [ 1.983364] Call trace: [ 1.983369] dump_backtrace+0x0/0x158 [ 1.983372] show_stack+0x24/0x30 [ 1.983378] dump_stack+0x80/0xa4 [ 1.983383] ___might_sleep+0x138/0x160 [ 1.983386] __might_sleep+0x58/0x90 [ 1.983391] __rt_mutex_lock_state+0x30/0xc0 [ 1.983395] _mutex_lock+0x24/0x30 [ 1.983400] perf_pmu_register+0x2c/0x388 [ 1.983404] cci_pmu_probe+0x2bc/0x488 [ 1.983409] platform_drv_probe+0x58/0xa8 However, we don't actually mind being preempted or migrated at this point; all that really matters is that whichever CPU we pick does not get offlined before we're done. Thus, do the robust thing and instead take the lock to inhibit CPU hotplug for the duration. This also revealed an additional race in assigning the global pointer too late relative to the hotplug notifier, so that gets fixed in the process. Reported-by: "Li, Meng" Signed-off-by: Robin Murphy Tested-by: Corentin Labbe Reviewed-by: Suzuki K Poulose --- drivers/perf/arm-cci.c | 22 ++++++++++++---------- 1 file changed, 12 insertions(+), 10 deletions(-) diff --git a/drivers/perf/arm-cci.c b/drivers/perf/arm-cci.c index 1bfeb160c5b1..f6d9df07ec9b 100644 --- a/drivers/perf/arm-cci.c +++ b/drivers/perf/arm-cci.c @@ -1692,21 +1692,23 @@ static int cci_pmu_probe(struct platform_device *pdev) raw_spin_lock_init(&cci_pmu->hw_events.pmu_lock); mutex_init(&cci_pmu->reserve_mutex); atomic_set(&cci_pmu->active_events, 0); - cci_pmu->cpu = get_cpu(); + + cpus_read_lock(); + cci_pmu->cpu = smp_processor_id(); ret = cci_pmu_init(cci_pmu, pdev); - if (ret) { - put_cpu(); - return ret; - } + if (ret) + goto out; - cpuhp_setup_state_nocalls(CPUHP_AP_PERF_ARM_CCI_ONLINE, - "perf/arm/cci:online", NULL, - cci_pmu_offline_cpu); - put_cpu(); g_cci_pmu = cci_pmu; + cpuhp_setup_state_nocalls_cpuslocked(CPUHP_AP_PERF_ARM_CCI_ONLINE, + "perf/arm/cci:online", NULL, + cci_pmu_offline_cpu); + pr_info("ARM %s PMU driver probed", cci_pmu->model->name); - return 0; +out: + cpus_read_unlock(); + return ret; } static int cci_pmu_remove(struct platform_device *pdev)