From patchwork Tue Apr 4 17:18:23 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Marc Zyngier X-Patchwork-Id: 9662263 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork.web.codeaurora.org (Postfix) with ESMTP id 97286602BA for ; Tue, 4 Apr 2017 17:38:09 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 873812851F for ; Tue, 4 Apr 2017 17:38:09 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 7BF6D28524; Tue, 4 Apr 2017 17:38:09 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-1.9 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID autolearn=unavailable version=3.3.1 Received: from bombadil.infradead.org (bombadil.infradead.org [65.50.211.133]) (using TLSv1.2 with cipher AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.wl.linuxfoundation.org (Postfix) with ESMTPS id 173CD2851F for ; Tue, 4 Apr 2017 17:38:09 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20170209; h=Sender: Content-Transfer-Encoding:Content-Type:MIME-Version:Cc:List-Subscribe: List-Help:List-Post:List-Archive:List-Unsubscribe:List-Id:References: In-Reply-To:Message-Id:Date:Subject:To:From:Reply-To:Content-ID: Content-Description:Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc :Resent-Message-ID:List-Owner; bh=dU6O1vbn700xQL3ugdX2i2ijJ9DtzDm7k5P1kiYymPI=; b=acXhRoY5HGO6SKy2iPdYju18yT KJkQfMpLyk0NOIGt9E2z6VimX82V+RMdZkCmE+qM2064HQCo/l326d+sjyVS5ZKRC5T4Wn1E4eqHl vaxiBD/h8UQ67JGT/xIx0aD7iho/y8IDw/kZSqUcN0Ates+1/SDPNEd3BpNshqR0qbfZXV7CaxhGb uFTGsjtx0zf2Xkpe7+REoVwZfimzksaypMaXjXsVYkcN0BIHD1GR49dhmMSXDFoPYoyH2q1Sn/D65 VnoKZ5K/fg7iwG4WZWgjRdjS7eP9eGTYvtHR10ysWfz+IrMCXumvA6R3USjDSVWkGtX7Xra5VTAxF 7Bhyux3w==; Received: from localhost ([127.0.0.1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.87 #1 (Red Hat Linux)) id 1cvSOy-0007Yb-Vo; Tue, 04 Apr 2017 17:37:57 +0000 Received: from foss.arm.com ([217.140.101.70]) by bombadil.infradead.org with esmtp (Exim 4.87 #1 (Red Hat Linux)) id 1cvS6v-00043z-MJ for linux-arm-kernel@lists.infradead.org; Tue, 04 Apr 2017 17:19:30 +0000 Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.72.51.249]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id A32D215A2; Tue, 4 Apr 2017 10:19:03 -0700 (PDT) Received: from approximate.cambridge.arm.com (approximate.cambridge.arm.com [10.1.207.16]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id E85EF3F3E1; Tue, 4 Apr 2017 10:19:01 -0700 (PDT) From: Marc Zyngier To: linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org Subject: [PATCH v3 15/18] arm64: arch_timer: Enable CNTVCT_EL0 trap if workaround is enabled Date: Tue, 4 Apr 2017 18:18:23 +0100 Message-Id: <20170404171826.25030-16-marc.zyngier@arm.com> X-Mailer: git-send-email 2.11.0 In-Reply-To: <20170404171826.25030-1-marc.zyngier@arm.com> References: <20170404171826.25030-1-marc.zyngier@arm.com> X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20170404_101919_208683_EDDA6762 X-CRM114-Status: GOOD ( 14.29 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Mark Rutland , Catalin Marinas , Daniel Lezcano , Will Deacon , Scott Wood , Hanjun Guo , Ding Tianhong , dann frazier , Thomas Gleixner MIME-Version: 1.0 Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+patchwork-linux-arm=patchwork.kernel.org@lists.infradead.org X-Virus-Scanned: ClamAV using ClamSMTP Userspace being allowed to use read CNTVCT_EL0 anytime (and not only in the VDSO), we need to enable trapping whenever a cntvct workaround is enabled on a given CPU. Signed-off-by: Marc Zyngier --- drivers/clocksource/arm_arch_timer.c | 45 +++++++++++++++++++++++++----------- 1 file changed, 32 insertions(+), 13 deletions(-) diff --git a/drivers/clocksource/arm_arch_timer.c b/drivers/clocksource/arm_arch_timer.c index 5c114da0ed71..f8adea258cf0 100644 --- a/drivers/clocksource/arm_arch_timer.c +++ b/drivers/clocksource/arm_arch_timer.c @@ -83,6 +83,7 @@ static enum ppi_nr arch_timer_uses_ppi = VIRT_PPI; static bool arch_timer_c3stop; static bool arch_timer_mem_use_virtual; static bool arch_counter_suspend_stop; +static bool vdso_default = true; static bool evtstrm_enable = IS_ENABLED(CONFIG_ARM_ARCH_TIMER_EVTSTREAM); @@ -383,6 +384,17 @@ void arch_timer_enable_workaround(const struct arch_timer_erratum_workaround *wa } static_branch_enable(&arch_timer_read_ool_enabled); + + /* + * Don't use the vdso fastpath if errata require using the + * out-of-line counter accessor. We may change our mind pretty + * late in the game (with a per-CPU erratum, for example), so + * change both the default value and the vdso itself. + */ + if (wa->read_cntvct_el0) { + clocksource_counter.archdata.vdso_direct = false; + vdso_default = false; + } } static void arch_timer_check_ool_workaround(enum arch_timer_erratum_match_type type, @@ -443,11 +455,19 @@ static void arch_timer_check_ool_workaround(enum arch_timer_erratum_match_type t __val; \ }) +static bool arch_timer_this_cpu_has_cntvct_wa(void) +{ + const struct arch_timer_erratum_workaround *wa; + + wa = __this_cpu_read(timer_unstable_counter_workaround); + return wa && wa->read_cntvct_el0; +} #else #define arch_timer_check_ool_workaround(t,a) do { } while(0) #define erratum_set_next_event_tval_virt(...) ({BUG(); 0;}) #define erratum_set_next_event_tval_phys(...) ({BUG(); 0;}) #define erratum_handler(fn, r, ...) ({false;}) +#define arch_timer_this_cpu_has_cntvct_wa() ({false;}) #endif /* CONFIG_ARM_ARCH_TIMER_OOL_WORKAROUND */ static __always_inline irqreturn_t timer_handler(const int access, @@ -660,15 +680,23 @@ static void arch_counter_set_user_access(void) { u32 cntkctl = arch_timer_get_cntkctl(); - /* Disable user access to the timers and the physical counter */ + /* Disable user access to the timers and both counters */ /* Also disable virtual event stream */ cntkctl &= ~(ARCH_TIMER_USR_PT_ACCESS_EN | ARCH_TIMER_USR_VT_ACCESS_EN + | ARCH_TIMER_USR_VCT_ACCESS_EN | ARCH_TIMER_VIRT_EVT_EN | ARCH_TIMER_USR_PCT_ACCESS_EN); - /* Enable user access to the virtual counter */ - cntkctl |= ARCH_TIMER_USR_VCT_ACCESS_EN; + /* + * Enable user access to the virtual counter if it doesn't + * need to be workaround. The vdso may have been already + * disabled though. + */ + if (arch_timer_this_cpu_has_cntvct_wa()) + pr_info("CPU%d: Trapping CNTVCT access\n", smp_processor_id()); + else + cntkctl |= ARCH_TIMER_USR_VCT_ACCESS_EN; arch_timer_set_cntkctl(cntkctl); } @@ -791,16 +819,7 @@ static void __init arch_counter_register(unsigned type) else arch_timer_read_counter = arch_counter_get_cntpct; - clocksource_counter.archdata.vdso_direct = true; - -#ifdef CONFIG_ARM_ARCH_TIMER_OOL_WORKAROUND - /* - * Don't use the vdso fastpath if errata require using - * the out-of-line counter accessor. - */ - if (static_branch_unlikely(&arch_timer_read_ool_enabled)) - clocksource_counter.archdata.vdso_direct = false; -#endif + clocksource_counter.archdata.vdso_direct = vdso_default; } else { arch_timer_read_counter = arch_counter_get_cntvct_mem; }