From patchwork Mon Oct 1 17:58:31 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Mark Salyzyn X-Patchwork-Id: 10622555 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 4871E15E8 for ; Mon, 1 Oct 2018 18:20:09 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 386BF27F54 for ; Mon, 1 Oct 2018 18:20:05 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 2976E28390; Mon, 1 Oct 2018 18:20:05 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-2.9 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,MAILING_LIST_MULTI,RCVD_IN_DNSWL_NONE autolearn=ham version=3.3.1 Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.wl.linuxfoundation.org (Postfix) with ESMTPS id 3C10E27F54 for ; Mon, 1 Oct 2018 18:20:03 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20170209; h=Sender: Content-Transfer-Encoding:Content-Type:Cc:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Message-Id:Date:Subject:To:From:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=oKZf5QfHRrRp8dc/21AYYdWTlXP/WHLfBe+pg+zD1Sc=; b=JADKmLgqXVTmTL dinCS6WiVzeRDjQ6v0wziuRGSD1Vm64afcZyUKB6bJU17FdQaux+9a0aka+iXTbJg/36bl9Ei1NM1 6v1I3U7dwOeKNQPIUh1evEtYUu2gfbfNUXZ6OAWq0o8VmKoR9Mq3C8Qb4oU6X45vW3mhvAo3Z5Hes WJiNrT+ln4s8+VUf+kJsKBWnj1tdnmE0Prh509cvCWLwMH89zdGtleTyffnKL/hge3T/sbgs0FoCA hYT5xlKPHJOXxsQMULE8QuXn/eYNAHSxOFFiN/VJ3WU9k9m3lMNOlvl6JhkRs70OKPaeeJJ7ZO+uH KPUe6A1mOQfio8s5Dlsw==; Received: from localhost ([127.0.0.1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.90_1 #2 (Red Hat Linux)) id 1g72nO-0000ym-F1; Mon, 01 Oct 2018 18:19:50 +0000 Received: from casper.infradead.org ([2001:8b0:10b:1236::1]) by bombadil.infradead.org with esmtps (Exim 4.90_1 #2 (Red Hat Linux)) id 1g72jr-0007UJ-Lf for linux-arm-kernel@bombadil.infradead.org; Mon, 01 Oct 2018 18:16:11 +0000 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=casper.20170209; h=Content-Transfer-Encoding:MIME-Version: References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From:Sender:Reply-To: Content-Type:Content-ID:Content-Description:Resent-Date:Resent-From: Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Id:List-Help: List-Unsubscribe:List-Subscribe:List-Post:List-Owner:List-Archive; bh=jwGOhrlc4+Uhv+pULBUIRs6yd641ryVOFZ2DcBqGhXY=; b=TqZensr1RyI6/FLKli5wAdZO/4 +Pku1GLbxfu7XT38fj83SnPnnqcGrY+TLayjc/BCHg6ocpDA6kpQXss9jz+P6IeKjxOUVDQoaXqsj ZAM51qrFyDMxAJgE94vbpB350wTaOZrYCH9TrweykXqjkj4pQyJ9m3rMm5WdfqWE4Ts0cbtiIb4kV eX8GT0cAr3wUwebzkqW6B5L2ygrCj/eChqlgYdCCNnrDN1uCMaLfl6qjMTWYq5l3wrLq7FkF8s9d9 C1aeg5Ldoej8DZdozbn0xs22UeNTSpq5YmKhqdncXhO1mYC+YHF/U43EsL0PY20YEfnvTU56WSxEr tkmrr9wA==; Received: from mail-pg1-x542.google.com ([2607:f8b0:4864:20::542]) by casper.infradead.org with esmtps (Exim 4.90_1 #2 (Red Hat Linux)) id 1g72Wv-00062j-Ad for linux-arm-kernel@lists.infradead.org; Mon, 01 Oct 2018 18:02:51 +0000 Received: by mail-pg1-x542.google.com with SMTP id f18-v6so6501159pgv.3 for ; Mon, 01 Oct 2018 11:02:39 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=android.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=jwGOhrlc4+Uhv+pULBUIRs6yd641ryVOFZ2DcBqGhXY=; b=EFEXfEENbkuCZ5Z+M2gsso0kmFZKFWG/Jud/Ra2BfYvmafGiYHWHiOFFv20IzyDltu MfsuGQ6L6YAMfth9VgAp5dtffopzjzq43AhBBwARHCuIeo7relbaCadAfxaKboTNVys+ DMj5AD64TOlMQYThVxMQLPsXWcuAvLcV9vKtRo5GX6pQ9IDKGxlLzlsFj7CjKWZiTBBI NFDY9LpSnq9/B4Tys2iYB2Gp5Vu+g9ZmxxVvDq+OR0U7pbbGvkTlyeWQuj3gv3E3d1QU uq0TZ3biqupEIYePOGn+8jlvHgjnV667olChs3Z7dFe30sFUOpyF1UDX+X1FRCCDCm3L h5RA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=jwGOhrlc4+Uhv+pULBUIRs6yd641ryVOFZ2DcBqGhXY=; b=OZnX1Eq/7ElPVLGpln5+sSOujYN/XnUHOCWebm0k/zPSA8tnBK7vE/eCOvCdTxw6kK YB1nKBwXhXJmiZhjhH2VzrtfOalI3L9j1OQQRDSwBtIOw0aRaD+7D/RAF1ZTrN4jaSR/ hsOu62/2ZqsQORewiQDOlJLlSkDasr8w/yfp9GD7fOMqYxJyYgiaiYw+dZphHBa/3M2r xU740TpkNWaqUkYdx1INtP4uERD3UAuSK4FlpgP/S1FEZSSz4f9MOD47VXG5mdRFxH3M 9MGpI3Bk2/cvz6WgRQLDQYVj6V0y0SVSq6J3cf4co3MLtdwH9wYOMOZxQpMzeoMr8Uch 7RFg== X-Gm-Message-State: ABuFfoiVUqBSQO03BeZYXOkaDvxCyjBuexsn4XNEyco4zZrqaEewORNY ybniE4tQWEsKLulaWGN3jgEcIg== X-Google-Smtp-Source: ACcGV63SkxcP2PAm5mcf5EGdgiguWOejcJCgDQX/ZkVglv33qmnfX1uEl9pw4+if4ln6tMyOg2cYZA== X-Received: by 2002:a17:902:24e7:: with SMTP id l36-v6mr13060947plg.234.1538416957415; Mon, 01 Oct 2018 11:02:37 -0700 (PDT) Received: from nebulus.mtv.corp.google.com ([2620:0:1000:1612:b4fb:6752:f21f:3502]) by smtp.gmail.com with ESMTPSA id d186-v6sm24469261pfg.173.2018.10.01.11.02.36 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Mon, 01 Oct 2018 11:02:36 -0700 (PDT) From: Mark Salyzyn To: linux-kernel@vger.kernel.org Subject: RESEND [PATCH v6 11/12] lib: vdso: Add support for CLOCK_BOOTTIME Date: Mon, 1 Oct 2018 10:58:31 -0700 Message-Id: <20181001175845.168430-12-salyzyn@android.com> X-Mailer: git-send-email 2.19.0.605.g01d371f741-goog In-Reply-To: <20181001175845.168430-1-salyzyn@android.com> References: <20181001175845.168430-1-salyzyn@android.com> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20181001_190249_400279_AE02DECB X-CRM114-Status: GOOD ( 28.72 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Mark Rutland , Catalin Marinas , Will Deacon , Mark Salyzyn , Ingo Molnar , Russell King , Andy Gross , Laura Abbott , Kees Cook , Kevin Brodsky , Jinbum Park , John Stultz , Thomas Gleixner , linux-arm-kernel@lists.infradead.org, "Paul E. McKenney" , Ard Biesheuvel , Dmitry Safonov , android-kernel@android.com, Jeremy Linton , Andy Lutomirski , James Morse , Andrew Pinski Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+patchwork-linux-arm=patchwork.kernel.org@lists.infradead.org X-Virus-Scanned: ClamAV using ClamSMTP Take an effort to recode the arm64 vdso code from assembler to C previously submitted by Andrew Pinski , rework it for use in both arm and arm64, overlapping any optimizations for each architecture. But instead of landing it in arm64, land the result into lib/vdso and unify both implementations to simplify future maintenance. Add a case for CLOCK_BOOTTIME as it is popular for measuring relative time on systems expected to suspend() or hibernate(). Android uses CLOCK_BOOTTIME for all relative time measurements and timeouts. Switching to vdso reduced CPU utilization and improves accuracy. There is also a desire by some partners to switch all logging over to CLOCK_BOOTTIME, and thus this operation alone would contribute to a near percentile CPU load. Signed-off-by: Mark Salyzyn Tested-by: Mark Salyzyn Cc: James Morse Cc: Russell King Cc: Catalin Marinas Cc: Will Deacon Cc: Andy Lutomirski Cc: Dmitry Safonov Cc: John Stultz Cc: Mark Rutland Cc: Laura Abbott Cc: Kees Cook Cc: Ard Biesheuvel Cc: Andy Gross Cc: Kevin Brodsky Cc: Andrew Pinski Cc: Thomas Gleixner Cc: linux-kernel@vger.kernel.org Cc: linux-arm-kernel@lists.infradead.org Cc: Jeremy Linton Cc: android-kernel@android.com v2: - rebased and changed from 3/3 to 10/10, fortified commit message. v3: - move arch/arm/vdso/vgettimeofday.c to lib/vdso/vgettimeofday.c. v4: - update commit message to reflect specific, and overall reasoning of patch series. - drop forced inline operations. - switch typeof() with __kernel_time_t. v5: - added comment about open coded timepsec_add_ns() for clarity. v6: - fix issue with __iter_div_u64_rem scaling by splitting sec & nsec --- arch/arm/include/asm/vdso_datapage.h | 2 + arch/arm/kernel/vdso.c | 4 ++ arch/arm64/include/asm/vdso_datapage.h | 2 + arch/arm64/kernel/vdso.c | 4 ++ lib/vdso/vgettimeofday.c | 56 ++++++++++++++++++++++++++ 5 files changed, 68 insertions(+) diff --git a/arch/arm/include/asm/vdso_datapage.h b/arch/arm/include/asm/vdso_datapage.h index 1c6e6a5d5d9d..0120852b6b12 100644 --- a/arch/arm/include/asm/vdso_datapage.h +++ b/arch/arm/include/asm/vdso_datapage.h @@ -64,6 +64,8 @@ struct vdso_data { u32 tz_minuteswest; /* timezone info for gettimeofday(2) */ u32 tz_dsttime; + u32 btm_sec; /* monotonic to boot time */ + u32 btm_nsec; /* Raw clocksource multipler */ u32 cs_raw_mult; /* Raw time */ diff --git a/arch/arm/kernel/vdso.c b/arch/arm/kernel/vdso.c index c299967df63c..51d8dcbd9952 100644 --- a/arch/arm/kernel/vdso.c +++ b/arch/arm/kernel/vdso.c @@ -337,6 +337,8 @@ void update_vsyscall(struct timekeeper *tk) vdso_data->wtm_clock_nsec = wtm->tv_nsec; if (!vdso_data->use_syscall) { + struct timespec btm = ktime_to_timespec(tk->offs_boot); + vdso_data->cs_cycle_last = tk->tkr_mono.cycle_last; vdso_data->raw_time_sec = tk->raw_sec; vdso_data->raw_time_nsec = tk->tkr_raw.xtime_nsec; @@ -347,6 +349,8 @@ void update_vsyscall(struct timekeeper *tk) /* tkr_mono.shift == tkr_raw.shift */ vdso_data->cs_shift = tk->tkr_mono.shift; vdso_data->cs_mask = tk->tkr_mono.mask; + vdso_data->btm_sec = btm.tv_sec; + vdso_data->btm_nsec = btm.tv_nsec; } vdso_write_end(vdso_data); diff --git a/arch/arm64/include/asm/vdso_datapage.h b/arch/arm64/include/asm/vdso_datapage.h index 95f4a7abab80..348b9be9efe7 100644 --- a/arch/arm64/include/asm/vdso_datapage.h +++ b/arch/arm64/include/asm/vdso_datapage.h @@ -45,6 +45,8 @@ struct vdso_data { __u64 xtime_coarse_nsec; __u64 wtm_clock_sec; /* Wall to monotonic time */ vdso_wtm_clock_nsec_t wtm_clock_nsec; + __u32 btm_sec; /* monotonic to boot time */ + __u32 btm_nsec; __u32 tb_seq_count; /* Timebase sequence counter */ /* cs_* members must be adjacent and in this order (ldp accesses) */ __u32 cs_mono_mult; /* NTP-adjusted clocksource multiplier */ diff --git a/arch/arm64/kernel/vdso.c b/arch/arm64/kernel/vdso.c index 59f150c25889..8dd2ad220a0f 100644 --- a/arch/arm64/kernel/vdso.c +++ b/arch/arm64/kernel/vdso.c @@ -233,6 +233,8 @@ void update_vsyscall(struct timekeeper *tk) vdso_data->wtm_clock_nsec = tk->wall_to_monotonic.tv_nsec; if (!use_syscall) { + struct timespec btm = ktime_to_timespec(tk->offs_boot); + /* tkr_mono.cycle_last == tkr_raw.cycle_last */ vdso_data->cs_cycle_last = tk->tkr_mono.cycle_last; vdso_data->raw_time_sec = tk->raw_sec; @@ -243,6 +245,8 @@ void update_vsyscall(struct timekeeper *tk) vdso_data->cs_raw_mult = tk->tkr_raw.mult; /* tkr_mono.shift == tkr_raw.shift */ vdso_data->cs_shift = tk->tkr_mono.shift; + vdso_data->btm_sec = btm.tv_sec; + vdso_data->btm_nsec = btm.tv_nsec; } smp_wmb(); diff --git a/lib/vdso/vgettimeofday.c b/lib/vdso/vgettimeofday.c index 33c5917fe9f8..4c3af7bc6499 100644 --- a/lib/vdso/vgettimeofday.c +++ b/lib/vdso/vgettimeofday.c @@ -247,6 +247,51 @@ static notrace int do_monotonic_raw(const struct vdso_data *vd, return 0; } +static notrace int do_boottime(const struct vdso_data *vd, struct timespec *ts) +{ + u32 seq, mult, shift; + u64 nsec, cycle_last; + vdso_wtm_clock_nsec_t wtm_nsec; +#ifdef ARCH_CLOCK_FIXED_MASK + static const u64 mask = ARCH_CLOCK_FIXED_MASK; +#else + u64 mask; +#endif + __kernel_time_t sec; + + do { + seq = vdso_read_begin(vd); + + if (vd->use_syscall) + return -1; + + cycle_last = vd->cs_cycle_last; + + mult = vd->cs_mono_mult; + shift = vd->cs_shift; +#ifndef ARCH_CLOCK_FIXED_MASK + mask = vd->cs_mask; +#endif + + sec = vd->xtime_clock_sec; + nsec = vd->xtime_clock_snsec; + + sec += vd->wtm_clock_sec + vd->btm_sec; + wtm_nsec = vd->wtm_clock_nsec + vd->btm_nsec; + + } while (unlikely(vdso_read_retry(vd, seq))); + + nsec += get_clock_shifted_nsec(cycle_last, mult, mask); + nsec >>= shift; + nsec += wtm_nsec; + + /* open coding timespec_add_ns to save a ts->tv_nsec = 0 */ + ts->tv_sec = sec + __iter_div_u64_rem(nsec, NSEC_PER_SEC, &nsec); + ts->tv_nsec = nsec; + + return 0; +} + #else /* ARCH_PROVIDES_TIMER */ static notrace int do_realtime(const struct vdso_data *vd, struct timespec *ts) @@ -265,6 +310,12 @@ static notrace int do_monotonic_raw(const struct vdso_data *vd, return -1; } +static notrace int do_boottime(const struct vdso_data *vd, + struct timespec *ts) +{ + return -1; +} + #endif /* ARCH_PROVIDES_TIMER */ notrace int __vdso_clock_gettime(clockid_t clock, struct timespec *ts) @@ -290,6 +341,10 @@ notrace int __vdso_clock_gettime(clockid_t clock, struct timespec *ts) if (do_monotonic_raw(vd, ts)) goto fallback; break; + case CLOCK_BOOTTIME: + if (do_boottime(vd, ts)) + goto fallback; + break; default: goto fallback; } @@ -326,6 +381,7 @@ int __vdso_clock_getres(clockid_t clock, struct timespec *res) long nsec; if (clock == CLOCK_REALTIME || + clock == CLOCK_BOOTTIME || clock == CLOCK_MONOTONIC || clock == CLOCK_MONOTONIC_RAW) nsec = MONOTONIC_RES_NSEC;