From patchwork Tue Jun 25 16:18:02 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Vincenzo Frascino X-Patchwork-Id: 11015957 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 144B71398 for ; Tue, 25 Jun 2019 16:18:48 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 0717F27F92 for ; Tue, 25 Jun 2019 16:18:48 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id EF1D527F8F; Tue, 25 Jun 2019 16:18:47 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.9 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_HI autolearn=unavailable version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id AF96327F8C for ; Tue, 25 Jun 2019 16:18:47 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726422AbfFYQSf (ORCPT ); Tue, 25 Jun 2019 12:18:35 -0400 Received: from foss.arm.com ([217.140.110.172]:44760 "EHLO foss.arm.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726420AbfFYQSf (ORCPT ); Tue, 25 Jun 2019 12:18:35 -0400 Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 916D1360; Tue, 25 Jun 2019 09:18:34 -0700 (PDT) Received: from e119884-lin.cambridge.arm.com (e119884-lin.cambridge.arm.com [10.1.196.72]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id F12313F718; Tue, 25 Jun 2019 09:18:31 -0700 (PDT) From: Vincenzo Frascino To: linux-arch@vger.kernel.org, linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, linux-mips@vger.kernel.org, linux-kselftest@vger.kernel.org Cc: catalin.marinas@arm.com, will.deacon@arm.com, arnd@arndb.de, linux@armlinux.org.uk, ralf@linux-mips.org, paul.burton@mips.com, daniel.lezcano@linaro.org, tglx@linutronix.de, salyzyn@android.com, pcc@google.com, shuah@kernel.org, 0x7f454c46@gmail.com, linux@rasmusvillemoes.dk, huw@codeweavers.com, sthotton@marvell.com, andre.przywara@arm.com Subject: [PATCH 1/3] lib/vdso: Delay mask application in do_hres() Date: Tue, 25 Jun 2019 17:18:02 +0100 Message-Id: <20190625161804.38713-1-vincenzo.frascino@arm.com> X-Mailer: git-send-email 2.22.0 In-Reply-To: <20190624133607.GI29497@fuggles.cambridge.arm.com> References: <20190624133607.GI29497@fuggles.cambridge.arm.com> MIME-Version: 1.0 Sender: linux-mips-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-mips@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP do_hres() in the vDSO generic library masks the hw counter value immediately after reading it. Postpone the mask application after checking if the syscall fallback is enabled, in order to be able to detect a possible fallback for the architectures that have masks smaller than ULLONG_MAX. Signed-off-by: Vincenzo Frascino --- lib/vdso/gettimeofday.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/lib/vdso/gettimeofday.c b/lib/vdso/gettimeofday.c index ef28cc5d7bff..ee1221ba1d32 100644 --- a/lib/vdso/gettimeofday.c +++ b/lib/vdso/gettimeofday.c @@ -35,12 +35,12 @@ static int do_hres(const struct vdso_data *vd, clockid_t clk, do { seq = vdso_read_begin(vd); - cycles = __arch_get_hw_counter(vd->clock_mode) & - vd->mask; + cycles = __arch_get_hw_counter(vd->clock_mode); ns = vdso_ts->nsec; last = vd->cycle_last; if (unlikely((s64)cycles < 0)) return clock_gettime_fallback(clk, ts); + cycles &= vd->mask; if (cycles > last) ns += (cycles - last) * vd->mult; ns >>= vd->shift; From patchwork Tue Jun 25 16:18:03 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Vincenzo Frascino X-Patchwork-Id: 11015955 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 21CAF14BB for ; Tue, 25 Jun 2019 16:18:47 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 14D8427F89 for ; Tue, 25 Jun 2019 16:18:47 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 08C8F27F8F; Tue, 25 Jun 2019 16:18:47 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.9 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_HI autolearn=unavailable version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id AD22C27F8C for ; Tue, 25 Jun 2019 16:18:46 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727714AbfFYQSi (ORCPT ); Tue, 25 Jun 2019 12:18:38 -0400 Received: from foss.arm.com ([217.140.110.172]:44794 "EHLO foss.arm.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726420AbfFYQSh (ORCPT ); Tue, 25 Jun 2019 12:18:37 -0400 Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 5EEE8C0A; Tue, 25 Jun 2019 09:18:37 -0700 (PDT) Received: from e119884-lin.cambridge.arm.com (e119884-lin.cambridge.arm.com [10.1.196.72]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id C5DB03F718; Tue, 25 Jun 2019 09:18:34 -0700 (PDT) From: Vincenzo Frascino To: linux-arch@vger.kernel.org, linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, linux-mips@vger.kernel.org, linux-kselftest@vger.kernel.org Cc: catalin.marinas@arm.com, will.deacon@arm.com, arnd@arndb.de, linux@armlinux.org.uk, ralf@linux-mips.org, paul.burton@mips.com, daniel.lezcano@linaro.org, tglx@linutronix.de, salyzyn@android.com, pcc@google.com, shuah@kernel.org, 0x7f454c46@gmail.com, linux@rasmusvillemoes.dk, huw@codeweavers.com, sthotton@marvell.com, andre.przywara@arm.com Subject: [PATCH 2/3] arm64: Fix __arch_get_hw_counter() implementation Date: Tue, 25 Jun 2019 17:18:03 +0100 Message-Id: <20190625161804.38713-2-vincenzo.frascino@arm.com> X-Mailer: git-send-email 2.22.0 In-Reply-To: <20190625161804.38713-1-vincenzo.frascino@arm.com> References: <20190624133607.GI29497@fuggles.cambridge.arm.com> <20190625161804.38713-1-vincenzo.frascino@arm.com> MIME-Version: 1.0 Sender: linux-mips-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-mips@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP Provide the following fixes for the __arch_get_hw_counter() implementation on arm64: - Fallback on syscall when an unstable counter is detected. - Introduce isb()s before and after the counter read to avoid speculation of the counter value and of the seq lock respectively. The second isb() is a temporary solution that will be revisited in 5.3-rc1. These fixes restore the semantics that __arch_counter_get_cntvct() had on arm64. Cc: Catalin Marinas Cc: Will Deacon Signed-off-by: Vincenzo Frascino --- arch/arm64/include/asm/vdso/gettimeofday.h | 19 +++++++++++++++++++ 1 file changed, 19 insertions(+) diff --git a/arch/arm64/include/asm/vdso/gettimeofday.h b/arch/arm64/include/asm/vdso/gettimeofday.h index 447ef417de45..b08f476b72b4 100644 --- a/arch/arm64/include/asm/vdso/gettimeofday.h +++ b/arch/arm64/include/asm/vdso/gettimeofday.h @@ -10,6 +10,8 @@ #include #include +#define __VDSO_USE_SYSCALL ULLONG_MAX + #define VDSO_HAS_CLOCK_GETRES 1 static __always_inline @@ -68,7 +70,24 @@ static __always_inline u64 __arch_get_hw_counter(s32 clock_mode) { u64 res; + /* + * clock_mode == 0 implies that vDSO are enabled otherwise + * fallback on syscall. + */ + if (clock_mode) + return __VDSO_USE_SYSCALL; + + /* + * This isb() is required to prevent that the counter value + * is speculated. + */ + isb(); asm volatile("mrs %0, cntvct_el0" : "=r" (res) :: "memory"); + /* + * This isb() is required to prevent that the seq lock is + * speculated.# + */ + isb(); return res; } From patchwork Tue Jun 25 16:18:04 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Vincenzo Frascino X-Patchwork-Id: 11015951 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id BF2366C5 for ; Tue, 25 Jun 2019 16:18:45 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id B0E5127F89 for ; Tue, 25 Jun 2019 16:18:45 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id A3CB127F92; Tue, 25 Jun 2019 16:18:45 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.9 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 40E3327F89 for ; Tue, 25 Jun 2019 16:18:45 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1728339AbfFYQSl (ORCPT ); Tue, 25 Jun 2019 12:18:41 -0400 Received: from foss.arm.com ([217.140.110.172]:44832 "EHLO foss.arm.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726420AbfFYQSk (ORCPT ); Tue, 25 Jun 2019 12:18:40 -0400 Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 2D8ACED1; Tue, 25 Jun 2019 09:18:40 -0700 (PDT) Received: from e119884-lin.cambridge.arm.com (e119884-lin.cambridge.arm.com [10.1.196.72]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id 96EB73F718; Tue, 25 Jun 2019 09:18:37 -0700 (PDT) From: Vincenzo Frascino To: linux-arch@vger.kernel.org, linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, linux-mips@vger.kernel.org, linux-kselftest@vger.kernel.org Cc: catalin.marinas@arm.com, will.deacon@arm.com, arnd@arndb.de, linux@armlinux.org.uk, ralf@linux-mips.org, paul.burton@mips.com, daniel.lezcano@linaro.org, tglx@linutronix.de, salyzyn@android.com, pcc@google.com, shuah@kernel.org, 0x7f454c46@gmail.com, linux@rasmusvillemoes.dk, huw@codeweavers.com, sthotton@marvell.com, andre.przywara@arm.com Subject: [PATCH 3/3] arm64: compat: Fix __arch_get_hw_counter() implementation Date: Tue, 25 Jun 2019 17:18:04 +0100 Message-Id: <20190625161804.38713-3-vincenzo.frascino@arm.com> X-Mailer: git-send-email 2.22.0 In-Reply-To: <20190625161804.38713-1-vincenzo.frascino@arm.com> References: <20190624133607.GI29497@fuggles.cambridge.arm.com> <20190625161804.38713-1-vincenzo.frascino@arm.com> MIME-Version: 1.0 Sender: linux-mips-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-mips@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP Provide the following fixes for the __arch_get_hw_counter() implementation on arm64: - Fallback on syscall when an unstable counter is detected. - Introduce isb()s before and after the counter read to avoid speculation of the counter value and of the seq lock respectively. The second isb() is a temporary solution that will be revisited in 5.3-rc1. These fixes restore the semantics that __arch_counter_get_cntvct() had on arm64. Cc: Catalin Marinas Cc: Will Deacon Signed-off-by: Vincenzo Frascino --- .../include/asm/vdso/compat_gettimeofday.h | 18 ++++++++++++++++++ 1 file changed, 18 insertions(+) diff --git a/arch/arm64/include/asm/vdso/compat_gettimeofday.h b/arch/arm64/include/asm/vdso/compat_gettimeofday.h index 93dbd935b66d..f4812777f5c5 100644 --- a/arch/arm64/include/asm/vdso/compat_gettimeofday.h +++ b/arch/arm64/include/asm/vdso/compat_gettimeofday.h @@ -12,6 +12,8 @@ #include +#define __VDSO_USE_SYSCALL ULLONG_MAX + #define VDSO_HAS_CLOCK_GETRES 1 static __always_inline @@ -74,8 +76,24 @@ static __always_inline u64 __arch_get_hw_counter(s32 clock_mode) { u64 res; + /* + * clock_mode == 0 implies that vDSO are enabled otherwise + * fallback on syscall. + */ + if (clock_mode) + return __VDSO_USE_SYSCALL; + + /* + * This isb() is required to prevent that the counter value + * is speculated. + */ isb(); asm volatile("mrrc p15, 1, %Q0, %R0, c14" : "=r" (res)); + /* + * This isb() is required to prevent that the seq lock is + * speculated. + */ + isb(); return res; }