From patchwork Sun Apr 16 20:27:02 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Daniel Lezcano X-Patchwork-Id: 9683217 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork.web.codeaurora.org (Postfix) with ESMTP id 573B0602CB for ; Sun, 16 Apr 2017 20:42:41 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 429CB27CEA for ; Sun, 16 Apr 2017 20:42:41 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 3491327D0C; Sun, 16 Apr 2017 20:42:41 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-1.9 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID autolearn=ham version=3.3.1 Received: from bombadil.infradead.org (bombadil.infradead.org [65.50.211.133]) (using TLSv1.2 with cipher AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.wl.linuxfoundation.org (Postfix) with ESMTPS id BA89727CEA for ; Sun, 16 Apr 2017 20:42:40 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20170209; h=Sender: Content-Transfer-Encoding:Content-Type:MIME-Version:Cc:List-Subscribe: List-Help:List-Post:List-Archive:List-Unsubscribe:List-Id:References: In-Reply-To:Message-Id:Date:Subject:To:From:Reply-To:Content-ID: Content-Description:Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc :Resent-Message-ID:List-Owner; bh=GuSq+c0M6rvag6UWHzW9oPRi0T1QHTal3aFLynabizQ=; b=kBoDxFcN/qvJXd6iIcvc3MAGGX RHgvyq12LwdgcGbweO39KwlrLwBn6qTreW1fY0DVzhz3KNAw/yNdOMfMqF/l5VDSlH4m702JSSEMw uaarLs8YzVp+gnd5afp4vFDEwjNHlqcBYYD7sD/qhcPW5qBqVie6PSTA9yglZF21U0Qqj0jkALvSM dF7hTzQd0Apvh4qfDsql/ApNzkrU5+BI5gMNGwbMgOTo13S3/bxc30qXMhi38fykAm0FLMH3vOMPQ aofvSiqnTYx6xC7M89vxEjsjeeb4YeIwHBNPcb+gvoTmAxSVdd8dypvwL3Np5AwJ2ukMOY5cpvoDJ Ahhy+J/A==; Received: from localhost ([127.0.0.1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.87 #1 (Red Hat Linux)) id 1czr0J-00009b-4r; Sun, 16 Apr 2017 20:42:39 +0000 Received: from mail-wr0-x235.google.com ([2a00:1450:400c:c0c::235]) by bombadil.infradead.org with esmtps (Exim 4.87 #1 (Red Hat Linux)) id 1czqmg-00062h-NF for linux-arm-kernel@lists.infradead.org; Sun, 16 Apr 2017 20:28:44 +0000 Received: by mail-wr0-x235.google.com with SMTP id o21so74476744wrb.2 for ; Sun, 16 Apr 2017 13:28:16 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=8VhRqAkMimYWtWdUn3BrlxJA87Vw+h9uAQyt1LeVGAQ=; b=VwNNeRH+vx+I9PaDdbiQYk5Ig2Qa5Z55Vg7d2QA2T8lhVSqY31MoNb6A7YzqM1jN5C aCR2irt6IAeTHFrWPKjz2ua087HSCo6t0N2KdjUeBfE0LH5ZBIFvFmacvwuM9nP44WIp 56lynJPC2pThaUre5/NQtqQsvoxRBHLrTuuEg= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=8VhRqAkMimYWtWdUn3BrlxJA87Vw+h9uAQyt1LeVGAQ=; b=g0UMLU7icbgKFZMNMFXYjw7BvhQQPwAZMwy4qqxU1LzvksYvuQvaknLBOJihKRqOF1 JfIRqHK8ZrPg9WBtRbPy4F7SvM46RREs/erCOc6JWMkvampo5PkX2WL72B+ktVPTlE/6 ksIYhSAwtVF1KVNVnuI11jJxKBvd3P+fa2bP+LSstA2tyuhZSwREC+RRYn8h5LYPJaWK ysb9rvdx0dQYnzPxBOhXdqkMOAdqyqiGImS/v4Flg2QlyLvvrxVS8Yjw239x0V3KYvu9 CYhJv29LPNv+/hQ9xN1BUk0hiAmlR/KJvc3aEnH4P8mboHf1b9DnK8Wl08cR+RZiRtAj jFAQ== X-Gm-Message-State: AN3rC/628WsbeKj9htq0eOpVyVADl7/9i8I5dO4gvSwVnxDfD6oBSX3V fl9rUif6O3tK0WMu X-Received: by 10.223.178.68 with SMTP id y4mr16887515wra.88.1492374494999; Sun, 16 Apr 2017 13:28:14 -0700 (PDT) Received: from mai.lan ([2001:41d0:fe90:b800:20c0:6248:a385:db35]) by smtp.gmail.com with ESMTPSA id 81sm7732196wmj.9.2017.04.16.13.28.13 (version=TLS1_2 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Sun, 16 Apr 2017 13:28:14 -0700 (PDT) From: Daniel Lezcano To: tglx@linutronix.de Subject: [PATCH 12/29] arm64: arch_timer: Allows a CPU-specific erratum to only affect a subset of CPUs Date: Sun, 16 Apr 2017 22:27:02 +0200 Message-Id: <1492374441-23336-12-git-send-email-daniel.lezcano@linaro.org> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1492374441-23336-1-git-send-email-daniel.lezcano@linaro.org> References: <20170416202542.GV2078@mai> <1492374441-23336-1-git-send-email-daniel.lezcano@linaro.org> X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20170416_132835_615002_59DD5685 X-CRM114-Status: GOOD ( 16.02 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Mark Rutland , Marc Zyngier , Catalin Marinas , Will Deacon , linux-kernel@vger.kernel.org, "moderated list:ARM ARCHITECTED TIMER DRIVER" MIME-Version: 1.0 Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+patchwork-linux-arm=patchwork.kernel.org@lists.infradead.org X-Virus-Scanned: ClamAV using ClamSMTP From: Marc Zyngier Instead of applying a CPU-specific workaround to all CPUs in the system, allow it to only affect a subset of them (typical big-little case). This is done by turning the erratum pointer into a per-CPU variable. Acked-by: Thomas Gleixner Signed-off-by: Marc Zyngier --- arch/arm64/include/asm/arch_timer.h | 31 ++++++++++++++++---------- drivers/clocksource/arm_arch_timer.c | 42 +++++++++++++++++++++++++----------- 2 files changed, 50 insertions(+), 23 deletions(-) diff --git a/arch/arm64/include/asm/arch_timer.h b/arch/arm64/include/asm/arch_timer.h index 01917b4..6bd1a9a 100644 --- a/arch/arm64/include/asm/arch_timer.h +++ b/arch/arm64/include/asm/arch_timer.h @@ -25,6 +25,7 @@ #include #include #include +#include #include #include @@ -55,17 +56,25 @@ struct arch_timer_erratum_workaround { int (*set_next_event_virt)(unsigned long, struct clock_event_device *); }; -extern const struct arch_timer_erratum_workaround *timer_unstable_counter_workaround; - -#define arch_timer_reg_read_stable(reg) \ -({ \ - u64 _val; \ - if (needs_unstable_timer_counter_workaround() && \ - timer_unstable_counter_workaround->read_##reg) \ - _val = timer_unstable_counter_workaround->read_##reg(); \ - else \ - _val = read_sysreg(reg); \ - _val; \ +DECLARE_PER_CPU(const struct arch_timer_erratum_workaround *, + timer_unstable_counter_workaround); + +#define arch_timer_reg_read_stable(reg) \ +({ \ + u64 _val; \ + if (needs_unstable_timer_counter_workaround()) { \ + const struct arch_timer_erratum_workaround *wa; \ + preempt_disable(); \ + wa = __this_cpu_read(timer_unstable_counter_workaround); \ + if (wa && wa->read_##reg) \ + _val = wa->read_##reg(); \ + else \ + _val = read_sysreg(reg); \ + preempt_enable(); \ + } else { \ + _val = read_sysreg(reg); \ + } \ + _val; \ }) /* diff --git a/drivers/clocksource/arm_arch_timer.c b/drivers/clocksource/arm_arch_timer.c index a0c9ee8..4551587 100644 --- a/drivers/clocksource/arm_arch_timer.c +++ b/drivers/clocksource/arm_arch_timer.c @@ -235,7 +235,8 @@ static u64 notrace hisi_161010101_read_cntvct_el0(void) #endif #ifdef CONFIG_ARM_ARCH_TIMER_OOL_WORKAROUND -const struct arch_timer_erratum_workaround *timer_unstable_counter_workaround = NULL; +DEFINE_PER_CPU(const struct arch_timer_erratum_workaround *, + timer_unstable_counter_workaround); EXPORT_SYMBOL_GPL(timer_unstable_counter_workaround); DEFINE_STATIC_KEY_FALSE(arch_timer_read_ool_enabled); @@ -338,9 +339,18 @@ arch_timer_iterate_errata(enum arch_timer_erratum_match_type type, } static -void arch_timer_enable_workaround(const struct arch_timer_erratum_workaround *wa) +void arch_timer_enable_workaround(const struct arch_timer_erratum_workaround *wa, + bool local) { - timer_unstable_counter_workaround = wa; + int i; + + if (local) { + __this_cpu_write(timer_unstable_counter_workaround, wa); + } else { + for_each_possible_cpu(i) + per_cpu(timer_unstable_counter_workaround, i) = wa; + } + static_branch_enable(&arch_timer_read_ool_enabled); } @@ -369,14 +379,17 @@ static void arch_timer_check_ool_workaround(enum arch_timer_erratum_match_type t return; if (needs_unstable_timer_counter_workaround()) { - if (wa != timer_unstable_counter_workaround) + const struct arch_timer_erratum_workaround *__wa; + __wa = __this_cpu_read(timer_unstable_counter_workaround); + if (__wa && wa != __wa) pr_warn("Can't enable workaround for %s (clashes with %s\n)", - wa->desc, - timer_unstable_counter_workaround->desc); - return; + wa->desc, __wa->desc); + + if (__wa) + return; } - arch_timer_enable_workaround(wa); + arch_timer_enable_workaround(wa, local); pr_info("Enabling %s workaround for %s\n", local ? "local" : "global", wa->desc); } @@ -384,10 +397,15 @@ static void arch_timer_check_ool_workaround(enum arch_timer_erratum_match_type t #define erratum_handler(fn, r, ...) \ ({ \ bool __val; \ - if (needs_unstable_timer_counter_workaround() && \ - timer_unstable_counter_workaround->fn) { \ - r = timer_unstable_counter_workaround->fn(__VA_ARGS__); \ - __val = true; \ + if (needs_unstable_timer_counter_workaround()) { \ + const struct arch_timer_erratum_workaround *__wa; \ + __wa = __this_cpu_read(timer_unstable_counter_workaround); \ + if (__wa && __wa->fn) { \ + r = __wa->fn(__VA_ARGS__); \ + __val = true; \ + } else { \ + __val = false; \ + } \ } else { \ __val = false; \ } \