From patchwork Mon Jul 22 10:33:28 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Marc Zyngier X-Patchwork-Id: 11052117 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id D283E138D for ; Mon, 22 Jul 2019 10:34:03 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id BF6F828475 for ; Mon, 22 Jul 2019 10:34:03 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id B1C682850D; Mon, 22 Jul 2019 10:34:03 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-5.2 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,MAILING_LIST_MULTI,RCVD_IN_DNSWL_MED autolearn=ham version=3.3.1 Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.wl.linuxfoundation.org (Postfix) with ESMTPS id 5006428475 for ; Mon, 22 Jul 2019 10:34:03 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20170209; h=Sender: Content-Transfer-Encoding:Content-Type:Cc:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Message-Id:Date:Subject:To:From:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=tlsaau08Pg8g+j83sWJGjMuKoXH0KDBP6rqRiSNtP+c=; b=O1/EXR5EpYp7g0 iJ0OImUuqwfq0weJOS6CrkBO7ucgercjH1WESFmfGMcNCXgZ5oO4AvmfgwTsE88B+LMZ9iHVHhkf9 xuvYyqk+8m47YfeXE8ddyeH7lWPbtAG6DO7yNXbun9P0W9bWl7GXDiiemGhSsogGVc6EzyJfvCd0p 8m0ZA4MgIti9ECxW1ont5yeMkBCATfjMbPoujpOe8nLlfcLzOKJqzOCbRzukmhrSa4X8jrVoBswmi 137OrPVSbv/SYfin2t5VG8UfnuPWl3dmueH7Lgen5LNjhKULsB+aGE4wrLYfFwr/YRvqck4KAQsT2 EPguHU63Cg2kxMBebHWQ==; Received: from localhost ([127.0.0.1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.92 #3 (Red Hat Linux)) id 1hpVdj-0003Jo-W0; Mon, 22 Jul 2019 10:33:56 +0000 Received: from foss.arm.com ([217.140.110.172]) by bombadil.infradead.org with esmtp (Exim 4.92 #3 (Red Hat Linux)) id 1hpVdX-000380-8w for linux-arm-kernel@lists.infradead.org; Mon, 22 Jul 2019 10:33:44 +0000 Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 3E68D1509; Mon, 22 Jul 2019 03:33:42 -0700 (PDT) Received: from filthy-habits.cambridge.arm.com (filthy-habits.cambridge.arm.com [10.1.197.61]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id C32B53F71A; Mon, 22 Jul 2019 03:33:40 -0700 (PDT) From: Marc Zyngier To: Thomas Gleixner , John Stultz , Pavel Tatashin , Petr Mladek , Sergey Senozhatsky , Steven Rostedt , Will Deacon , Catalin Marinas , Mark Rutland Subject: [PATCH 1/3] printk: Allow architecture-specific timestamping function Date: Mon, 22 Jul 2019 11:33:28 +0100 Message-Id: <20190722103330.255312-2-marc.zyngier@arm.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20190722103330.255312-1-marc.zyngier@arm.com> References: <20190722103330.255312-1-marc.zyngier@arm.com> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20190722_033343_355668_0B5895BA X-CRM114-Status: GOOD ( 12.26 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: linux-kernel@vger.kernel.org, linux-arm-kernel@lists.infradead.org Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+patchwork-linux-arm=patchwork.kernel.org@lists.infradead.org X-Virus-Scanned: ClamAV using ClamSMTP printk currently relies on local_clock to time-stamp the kernel messages. In order to allow the timestamping (and only that) to be overridden by architecture-specific code, let's declare a new timestamp_clock() function, which gets used by the printk code. Architectures willing to make use of this facility will have to define CONFIG_ARCH_HAS_TIMESTAMP_CLOCK. The default is of course to return local_clock(), so that the existing behaviour stays unchanged. Signed-off-by: Marc Zyngier --- include/linux/sched/clock.h | 13 +++++++++++++ kernel/printk/printk.c | 4 ++-- 2 files changed, 15 insertions(+), 2 deletions(-) diff --git a/include/linux/sched/clock.h b/include/linux/sched/clock.h index 867d588314e0..3cf4b2a8ce18 100644 --- a/include/linux/sched/clock.h +++ b/include/linux/sched/clock.h @@ -98,4 +98,17 @@ static inline void enable_sched_clock_irqtime(void) {} static inline void disable_sched_clock_irqtime(void) {} #endif +#ifdef CONFIG_ARCH_HAS_TIMESTAMP_CLOCK +/* Special need architectures can provide their timestamping function */ +extern u64 timestamp_clock(void); + +#else + +static inline u64 timestamp_clock(void) +{ + return local_clock(); +} + +#endif + #endif /* _LINUX_SCHED_CLOCK_H */ diff --git a/kernel/printk/printk.c b/kernel/printk/printk.c index 1888f6a3b694..166702316714 100644 --- a/kernel/printk/printk.c +++ b/kernel/printk/printk.c @@ -638,7 +638,7 @@ static int log_store(u32 caller_id, int facility, int level, if (ts_nsec > 0) msg->ts_nsec = ts_nsec; else - msg->ts_nsec = local_clock(); + msg->ts_nsec = timestamp_clock(); #ifdef CONFIG_PRINTK_CALLER msg->caller_id = caller_id; #endif @@ -1841,7 +1841,7 @@ static bool cont_add(u32 caller_id, int facility, int level, cont.facility = facility; cont.level = level; cont.caller_id = caller_id; - cont.ts_nsec = local_clock(); + cont.ts_nsec = timestamp_clock(); cont.flags = flags; } From patchwork Mon Jul 22 10:33:29 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Marc Zyngier X-Patchwork-Id: 11052119 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 7B8E9112C for ; Mon, 22 Jul 2019 10:34:09 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 65D8B28475 for ; Mon, 22 Jul 2019 10:34:09 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 59B6B2850D; Mon, 22 Jul 2019 10:34:09 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-5.2 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,MAILING_LIST_MULTI,RCVD_IN_DNSWL_MED autolearn=ham version=3.3.1 Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.wl.linuxfoundation.org (Postfix) with ESMTPS id 0476328475 for ; Mon, 22 Jul 2019 10:34:09 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20170209; h=Sender: Content-Transfer-Encoding:Content-Type:Cc:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Message-Id:Date:Subject:To:From:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=9OV3xeTJIza/o/QnElCaSu5XFiHY4IDPE2lzUW+USYk=; b=H8QCCkn0s+b5Nl 9WokjVnI/H38juoPHwp/YdD1FZn7YxdAryL/hqwv3ZScNz8rt7qb1xDOLkjIhzCp00agS1wCVQ8kX f/aGCDRvg3q/b2BLVpm3KU/rfG2pFK/t0R7kiSkm+r8Ywd8Y5aqYpRt36+IEf9o4awHRQr41czePB abwrqn+G8amcacylb7tW8M8xO2UDvUeTv19z1+QuOSCM+xqisxkyGNWRl5sxK/7ecCV8xu5tdaWCl NEq8EypDjiHKOsTt/KjpzYl+OV40FuMbzFkRodQwHPmppo19qHiWf5poWg3ej2QkCc7tfuf1KIRgG DTONXONLlLGqcyrUD8yA==; Received: from localhost ([127.0.0.1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.92 #3 (Red Hat Linux)) id 1hpVdv-0003Wc-Id; Mon, 22 Jul 2019 10:34:07 +0000 Received: from foss.arm.com ([217.140.110.172]) by bombadil.infradead.org with esmtp (Exim 4.92 #3 (Red Hat Linux)) id 1hpVdY-00038K-PI for linux-arm-kernel@lists.infradead.org; Mon, 22 Jul 2019 10:33:46 +0000 Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id E27DD1576; Mon, 22 Jul 2019 03:33:43 -0700 (PDT) Received: from filthy-habits.cambridge.arm.com (filthy-habits.cambridge.arm.com [10.1.197.61]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id 7347E3F71A; Mon, 22 Jul 2019 03:33:42 -0700 (PDT) From: Marc Zyngier To: Thomas Gleixner , John Stultz , Pavel Tatashin , Petr Mladek , Sergey Senozhatsky , Steven Rostedt , Will Deacon , Catalin Marinas , Mark Rutland Subject: [PATCH 2/3] sched/clock: Allow sched_clock to inherit timestamp_clock epoch Date: Mon, 22 Jul 2019 11:33:29 +0100 Message-Id: <20190722103330.255312-3-marc.zyngier@arm.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20190722103330.255312-1-marc.zyngier@arm.com> References: <20190722103330.255312-1-marc.zyngier@arm.com> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20190722_033344_863178_F85529F1 X-CRM114-Status: GOOD ( 13.80 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: linux-kernel@vger.kernel.org, linux-arm-kernel@lists.infradead.org Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+patchwork-linux-arm=patchwork.kernel.org@lists.infradead.org X-Virus-Scanned: ClamAV using ClamSMTP Now that we can let an architecture override the timestamping function, it becomes desirable to ensure that, should the architecture code switch its timestamping code to sched_clock once it has been registered, the sched_clock inherits the timestamp value as its new epoch. This ensures that the time stamps are continuous and that there is no jitter other than that introduced by the lack of quality of the timestamping clock. Signed-off-by: Marc Zyngier --- kernel/time/sched_clock.c | 10 ++++++++++ 1 file changed, 10 insertions(+) diff --git a/kernel/time/sched_clock.c b/kernel/time/sched_clock.c index 142b07619918..ee1bd449ec81 100644 --- a/kernel/time/sched_clock.c +++ b/kernel/time/sched_clock.c @@ -192,6 +192,16 @@ sched_clock_register(u64 (*read)(void), int bits, unsigned long rate) new_epoch = read(); cyc = cd.actual_read_sched_clock(); ns = rd.epoch_ns + cyc_to_ns((cyc - rd.epoch_cyc) & rd.sched_clock_mask, rd.mult, rd.shift); + + /* + * If the architecture has a timestamp clock, and this is the + * first time we register a new sched_clock, use the timestamp + * clock as the epoch. + */ + if (IS_ENABLED(CONFIG_ARCH_HAS_TIMESTAMP_CLOCK) && + unlikely(cd.actual_read_sched_clock == jiffy_sched_clock_read)) + ns = timestamp_clock(); + cd.actual_read_sched_clock = read; rd.read_sched_clock = read; From patchwork Mon Jul 22 10:33:30 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Marc Zyngier X-Patchwork-Id: 11052121 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 44486138D for ; Mon, 22 Jul 2019 10:34:29 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 2D76828475 for ; Mon, 22 Jul 2019 10:34:29 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 1E7F82850D; Mon, 22 Jul 2019 10:34:29 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-5.2 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,MAILING_LIST_MULTI,RCVD_IN_DNSWL_MED autolearn=ham version=3.3.1 Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.wl.linuxfoundation.org (Postfix) with ESMTPS id B17C628475 for ; Mon, 22 Jul 2019 10:34:28 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20170209; h=Sender: Content-Transfer-Encoding:Content-Type:Cc:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Message-Id:Date:Subject:To:From:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=2kH0MGh6faBfKIDhIf4V8deKLKUYjTdhZQ9fcIUSfsg=; b=kV3zr9CgtF8d4j AmQHp72huYBb2EPFswr8jJytjCX/rNJwCL1bamCbGY/ujrcBML7/Na6EKAkCAMfx0IXgbwnyHKI7H sFNJgHcA8XZ15F+Do4txt3mVrMp7s1OG0pPCn27KfUrfAHCxKCs8gyJ8sTNjBHWxfNxZSK6/mobrM SfR5YUZ1q3YlJmTLUtDHl1avxwMc9frAKLRMmr8q1KSt/p7Qq0TKITD+PsGq2J1TQy4H9Qy77I+KD hNEVGbsDern3nc5+QwZRSjWTuzet080uMggcJmI7NRMCyORT7rSE2rPyBDG023NfngCPKLx3YJhkh 1+fjRzV6dI5LwopXIm9Q==; Received: from localhost ([127.0.0.1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.92 #3 (Red Hat Linux)) id 1hpVeA-0003r3-Ba; Mon, 22 Jul 2019 10:34:22 +0000 Received: from foss.arm.com ([217.140.110.172]) by bombadil.infradead.org with esmtp (Exim 4.92 #3 (Red Hat Linux)) id 1hpVda-00039n-R1 for linux-arm-kernel@lists.infradead.org; Mon, 22 Jul 2019 10:33:48 +0000 Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 926C11596; Mon, 22 Jul 2019 03:33:45 -0700 (PDT) Received: from filthy-habits.cambridge.arm.com (filthy-habits.cambridge.arm.com [10.1.197.61]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id 236233F71A; Mon, 22 Jul 2019 03:33:44 -0700 (PDT) From: Marc Zyngier To: Thomas Gleixner , John Stultz , Pavel Tatashin , Petr Mladek , Sergey Senozhatsky , Steven Rostedt , Will Deacon , Catalin Marinas , Mark Rutland Subject: [PATCH 3/3] arm64: Allow early time stamping Date: Mon, 22 Jul 2019 11:33:30 +0100 Message-Id: <20190722103330.255312-4-marc.zyngier@arm.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20190722103330.255312-1-marc.zyngier@arm.com> References: <20190722103330.255312-1-marc.zyngier@arm.com> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20190722_033347_026806_9AC93047 X-CRM114-Status: GOOD ( 12.77 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: linux-kernel@vger.kernel.org, linux-arm-kernel@lists.infradead.org Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+patchwork-linux-arm=patchwork.kernel.org@lists.infradead.org X-Virus-Scanned: ClamAV using ClamSMTP In order to provide early timestamping on arm64 systems, we provide a timestamp_clock() function that is available as early as possible. This function simply returns the current counter value scales in nanoseconds, and 0-based. In order to deal with the idiosyncrasies of some broken platforms, we condition this on the frequency being set in the CNTFRQ_EL0 register, and revert back to using local_clock() as soon as it starts returning non-zero values. Signed-off-by: Marc Zyngier --- arch/arm64/Kconfig | 3 +++ arch/arm64/kernel/setup.c | 44 +++++++++++++++++++++++++++++++++++++++ 2 files changed, 47 insertions(+) diff --git a/arch/arm64/Kconfig b/arch/arm64/Kconfig index 3adcec05b1f6..ac3882ddc1c2 100644 --- a/arch/arm64/Kconfig +++ b/arch/arm64/Kconfig @@ -911,6 +911,9 @@ config ARCH_WANT_HUGE_PMD_SHARE config ARCH_HAS_CACHE_LINE_SIZE def_bool y +config ARCH_HAS_TIMESTAMP_CLOCK + def_bool y + config ARCH_ENABLE_SPLIT_PMD_PTLOCK def_bool y if PGTABLE_LEVELS > 2 diff --git a/arch/arm64/kernel/setup.c b/arch/arm64/kernel/setup.c index 9c4bad7d7131..cf21e3df7165 100644 --- a/arch/arm64/kernel/setup.c +++ b/arch/arm64/kernel/setup.c @@ -15,6 +15,7 @@ #include #include #include +#include #include #include #include @@ -28,10 +29,12 @@ #include #include #include +#include #include #include #include +#include #include #include #include @@ -269,8 +272,49 @@ arch_initcall(reserve_memblock_reserved_regions); u64 __cpu_logical_map[NR_CPUS] = { [0 ... NR_CPUS-1] = INVALID_HWID }; +static u64 notrace ts_counter_read_cc(const struct cyclecounter *cc) +{ + return __arch_counter_get_cntvct(); +} + +static struct cyclecounter ts_cc __ro_after_init = { + .read = ts_counter_read_cc, + .mask = CLOCKSOURCE_MASK(56), +}; + +static struct timecounter ts_tc; + +static bool ts_cc_valid __ro_after_init = false; + +static __init void timestamp_clock_init(void) +{ + u64 frq = arch_timer_get_cntfrq(); + + if (!frq) + return; + + clocks_calc_mult_shift(&ts_cc.mult, &ts_cc.shift, + frq, NSEC_PER_SEC, 3600); + /* timestamp starts at 0 (local_clock is a good enough approximation) */ + timecounter_init(&ts_tc, &ts_cc, local_clock()); + ts_cc_valid = true; + pr_info("Using timestamp clock @%lluMHz\n", frq / 1000 / 1000); +} + +u64 timestamp_clock(void) +{ + u64 ns = local_clock(); + + if (likely(ns || !ts_cc_valid)) + return ns; + + return timecounter_read(&ts_tc); +} + void __init setup_arch(char **cmdline_p) { + timestamp_clock_init(); + init_mm.start_code = (unsigned long) _text; init_mm.end_code = (unsigned long) _etext; init_mm.end_data = (unsigned long) _edata;