From patchwork Wed Oct 9 08:28:54 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Anna-Maria Behnsen X-Patchwork-Id: 13827737 Received: from galois.linutronix.de (Galois.linutronix.de [193.142.43.55]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id AA11B18871D; Wed, 9 Oct 2024 08:29:16 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=193.142.43.55 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1728462558; cv=none; b=Zq0V5n1x+MJreQ7FSi+EUanhlSgubx6+doLKPrvNMV2VDOM5ghmoJBIDnrkZDEhSfz0OcPRIFElUSdUUJM+d19PDMyGNwwKuWQxtIcAsyGyaA/OfxYgQpGKNgnpFC2pHY6fh6F+fvjgw1aybNIwWO7e+n59nckt38MAQpP5qdy4= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1728462558; c=relaxed/simple; bh=Vmv3Ug/H9zP/x9oaSfMaN3Bjme40VaX6S9AQQdUuQbo=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=sjqKVfCMZGl0tNHUKV5eK4H95InHTvSS3jyno5wWR/WPqU5mhvcsjzC8CQA45dZMXniUpGqjKuGD8/ub0hqx3PLm2VpYrbHYn2fuzpLjdsSHpw8urxRBsd0eWXyXJKJZLDeWJDJAqEYMIRqY3FPTQ07K9VC72deM56HbQyFL5to= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linutronix.de; spf=pass smtp.mailfrom=linutronix.de; dkim=pass (2048-bit key) header.d=linutronix.de header.i=@linutronix.de header.b=GhIQdmaw; dkim=permerror (0-bit key) header.d=linutronix.de header.i=@linutronix.de header.b=/rUXg3ja; arc=none smtp.client-ip=193.142.43.55 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linutronix.de Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linutronix.de Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=linutronix.de header.i=@linutronix.de header.b="GhIQdmaw"; dkim=permerror (0-bit key) header.d=linutronix.de header.i=@linutronix.de header.b="/rUXg3ja" From: Anna-Maria Behnsen DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linutronix.de; s=2020; t=1728462554; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=4otxVIHP6NRuH18Zefj2/uhmVo7zXK2jePiVQ9Mnwyw=; b=GhIQdmawFkoScx06Bu4wJr0dVyZiPNG4DuhRF0NTSDM8uhW5+F7cTZwot4mkpgUzSYIH18 Y9GfLj3yio93VFda19xql3npI51uovymenMeNqQ/CuB6766IYtjGhwg7GbE1JZOUvBg7t0 iEzePBsdmb0t8HFC3XridC9cBPfEpHiCNICAgX3HTc7ALfN/7w2O4kwwHxBq7yw/4Z21/3 i97AqMDZbNAQJvXSvTrNst8FNsVf503Lf6h0iadGYOdGquG5oOGnah8LRDLicQE7PXn8+T s/5OE4mJ8L8Pa3bAHtKGd/BOA7pDyMrom7OveCzjoktWAQ6Xps1ikdf2Wb1bIQ== DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=linutronix.de; s=2020e; t=1728462554; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=4otxVIHP6NRuH18Zefj2/uhmVo7zXK2jePiVQ9Mnwyw=; b=/rUXg3jaDMmfAE3jz4AWfSmut6UDck7Tj3M+414u7khMv9jFBYEnpgcQC9oDk5iD+bAEiB l5uhsQ+yKYz62BBw== Date: Wed, 09 Oct 2024 10:28:54 +0200 Subject: [PATCH v2 01/25] timekeeping: Read NTP tick length only once Precedence: bulk X-Mailing-List: netdev@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Message-Id: <20241009-devel-anna-maria-b4-timers-ptp-timekeeping-v2-1-554456a44a15@linutronix.de> References: <20241009-devel-anna-maria-b4-timers-ptp-timekeeping-v2-0-554456a44a15@linutronix.de> In-Reply-To: <20241009-devel-anna-maria-b4-timers-ptp-timekeeping-v2-0-554456a44a15@linutronix.de> To: John Stultz , Frederic Weisbecker , Thomas Gleixner Cc: linux-kernel@vger.kernel.org, netdev@vger.kernel.org, Miroslav Lichvar , Richard Cochran , Christopher S Hall , Anna-Maria Behnsen From: Thomas Gleixner From: Thomas Gleixner No point in reading it a second time when the comparison fails. Signed-off-by: Thomas Gleixner Signed-off-by: Anna-Maria Behnsen Acked-by: John Stultz --- kernel/time/timekeeping.c | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) diff --git a/kernel/time/timekeeping.c b/kernel/time/timekeeping.c index 7e6f409bf311..6acff4cb7b1c 100644 --- a/kernel/time/timekeeping.c +++ b/kernel/time/timekeeping.c @@ -2130,16 +2130,17 @@ static __always_inline void timekeeping_apply_adjustment(struct timekeeper *tk, */ static void timekeeping_adjust(struct timekeeper *tk, s64 offset) { + u64 ntp_tl = ntp_tick_length(); u32 mult; /* * Determine the multiplier from the current NTP tick length. * Avoid expensive division when the tick length doesn't change. */ - if (likely(tk->ntp_tick == ntp_tick_length())) { + if (likely(tk->ntp_tick == ntp_tl)) { mult = tk->tkr_mono.mult - tk->ntp_err_mult; } else { - tk->ntp_tick = ntp_tick_length(); + tk->ntp_tick = ntp_tl; mult = div64_u64((tk->ntp_tick >> tk->ntp_error_shift) - tk->xtime_remainder, tk->cycle_interval); } From patchwork Wed Oct 9 08:28:55 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Anna-Maria Behnsen X-Patchwork-Id: 13827739 Received: from galois.linutronix.de (Galois.linutronix.de [193.142.43.55]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id AA0CC1885BF; Wed, 9 Oct 2024 08:29:16 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=193.142.43.55 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1728462558; cv=none; b=fHW6KjVfL1MA/Z2jiFT5HQBFOHhKAgpSYvmapefgI4cJ/ESOOv+qXySfW+G/Z7A2ZYPaRAACaOSgIfqUs77ja8FW4tCL9U1892JBYcvn9G0+59IGlp6UQqWKOWTRAu7CW40ge86IDU+UFvsyAB5aXi1E1wewGgCAQPutLIvk5oI= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1728462558; c=relaxed/simple; bh=6NsryiVtEAAtoUHSfTYQER7IlKdE4VbXYKGa6hiaALk=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=pN8ndaN2kE4ZmDU2AQRk8WDKf5chiYz7MLjmGtmEDjXl3TG/YZMDVmdK5ACtLHSN69kI+1F63qEwm9Mks8HV4hlY7xoj3Cp8GnWKAtx6cM72RNSn2WKUDt2Ot/WkOLO3xz1XEe7FZ8dd4Tub7TVcM2EXmTrk5br+jmGK1+u9jTI= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linutronix.de; spf=pass smtp.mailfrom=linutronix.de; dkim=pass (2048-bit key) header.d=linutronix.de header.i=@linutronix.de header.b=qshuilf2; dkim=permerror (0-bit key) header.d=linutronix.de header.i=@linutronix.de header.b=7feMh+FY; arc=none smtp.client-ip=193.142.43.55 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linutronix.de Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linutronix.de Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=linutronix.de header.i=@linutronix.de header.b="qshuilf2"; dkim=permerror (0-bit key) header.d=linutronix.de header.i=@linutronix.de header.b="7feMh+FY" From: Anna-Maria Behnsen DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linutronix.de; s=2020; t=1728462555; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=kj9PA1CgwjnhqQ8xTBpdTyX7M4shxiowM3IXmHkmtUY=; b=qshuilf2Hm1GrjlDTPIQe41ua/iADXA5rvntWoXADVfzi1GaOi3JVqnvonaCiHF84TG7hM 6WqLWQvHaYBGAkORg8f03kuElDqBh2XqCRM99vOaOKRv2sZ/w7znPWhDA+3tsh3DWQdPyW S8R4/H13RX7cgFuC1xvPxO7hy+DPZo/i3qZyJXtvfFVMoHTet9jW+IpksffEPXZ9MBxAvf x2wLDCWTia4R0ZzRs+TJce67diiIQTtj5DUx8S2xrExHeuOF8lw7iPIFu7I5IYlB1qzRL+ dkqcydeSwRfeh1G/zgOyDlJCGZ64n4AAZD33vRv1L192WIq/glj/6jEYl9Oxag== DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=linutronix.de; s=2020e; t=1728462555; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=kj9PA1CgwjnhqQ8xTBpdTyX7M4shxiowM3IXmHkmtUY=; b=7feMh+FY3I6dThEI0O+O6COLM0+O0x3V55Qk2usZZlD06lJpnGbW8iiwba9FkZtVkmIMKq wa9Wh3qxmP/TceBA== Date: Wed, 09 Oct 2024 10:28:55 +0200 Subject: [PATCH v2 02/25] timekeeping: Don't stop time readers across hard_pps() update Precedence: bulk X-Mailing-List: netdev@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Message-Id: <20241009-devel-anna-maria-b4-timers-ptp-timekeeping-v2-2-554456a44a15@linutronix.de> References: <20241009-devel-anna-maria-b4-timers-ptp-timekeeping-v2-0-554456a44a15@linutronix.de> In-Reply-To: <20241009-devel-anna-maria-b4-timers-ptp-timekeeping-v2-0-554456a44a15@linutronix.de> To: John Stultz , Frederic Weisbecker , Thomas Gleixner Cc: linux-kernel@vger.kernel.org, netdev@vger.kernel.org, Miroslav Lichvar , Richard Cochran , Christopher S Hall , Anna-Maria Behnsen From: Thomas Gleixner From: Thomas Gleixner hard_pps() update does not modify anything which might be required by time readers so forcing readers out of the way during the update is a pointless exercise. The interaction with adjtimex() and timekeeper updates which call into the NTP code is properly serialized by timekeeper_lock. Signed-off-by: Thomas Gleixner Signed-off-by: Anna-Maria Behnsen --- kernel/time/timekeeping.c | 4 ---- 1 file changed, 4 deletions(-) diff --git a/kernel/time/timekeeping.c b/kernel/time/timekeeping.c index 6acff4cb7b1c..4e3afe22d28c 100644 --- a/kernel/time/timekeeping.c +++ b/kernel/time/timekeeping.c @@ -2627,11 +2627,7 @@ void hardpps(const struct timespec64 *phase_ts, const struct timespec64 *raw_ts) unsigned long flags; raw_spin_lock_irqsave(&timekeeper_lock, flags); - write_seqcount_begin(&tk_core.seq); - __hardpps(phase_ts, raw_ts); - - write_seqcount_end(&tk_core.seq); raw_spin_unlock_irqrestore(&timekeeper_lock, flags); } EXPORT_SYMBOL(hardpps); From patchwork Wed Oct 9 08:28:56 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Anna-Maria Behnsen X-Patchwork-Id: 13827738 Received: from galois.linutronix.de (Galois.linutronix.de [193.142.43.55]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 17813188737; Wed, 9 Oct 2024 08:29:16 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=193.142.43.55 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1728462558; cv=none; b=Z4Ruw1lutDiCXFMriUck2WIqdUyBdi525WqRLIHGToeWhSaQVr858WEX7eGORdujO2IZJQTwumO2xUXqgR9zJMHaYxkixJvuVo7bjdyEg/Ae9H4ylMyaAhqeITFKdPfV9+N3ooOBWXz9FJ3f7whuh9Htld//1RShCgq3s8jOydo= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1728462558; c=relaxed/simple; bh=/uNkAm7s/oMcGkPl0Y/czgMjwjkuogvFlaage6etO5U=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=JH8vZL+lfrDuaW9N6HAyqXcLQzPQoMCQ9FXQQDKjVlAqO7bgrtirS4gmk3rwUUCiaVQt+t0vyf6ujh2hZyvHZ4BhKZL81exlc4CR+sS7SA6kVyZYs1KKVBbwBVjlVloBauvSM0sg23be71PYvOX/D2MMTEMuSd6+OJVpJTVqDDs= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linutronix.de; spf=pass smtp.mailfrom=linutronix.de; dkim=pass (2048-bit key) header.d=linutronix.de header.i=@linutronix.de header.b=nrKYwAtP; dkim=permerror (0-bit key) header.d=linutronix.de header.i=@linutronix.de header.b=lk2cDjxI; arc=none smtp.client-ip=193.142.43.55 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linutronix.de Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linutronix.de Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=linutronix.de header.i=@linutronix.de header.b="nrKYwAtP"; dkim=permerror (0-bit key) header.d=linutronix.de header.i=@linutronix.de header.b="lk2cDjxI" From: Anna-Maria Behnsen DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linutronix.de; s=2020; t=1728462555; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=PTWEOkMgCycRVxphDRFrN3uHtpIO2IbyhOjzG4V0kAg=; b=nrKYwAtPcGQvrzCBhXB5niUCUFZZVpON6wwhLnqLjnBPO2yiup4dEeuHWZGZUoshdbmX+F ylhfk4EM/5zhEIN92/gh9F4DLIdY/ccDkL7uIBQlqgXLMSzhYhX3Jh0WzzkPYseDZYc8wL +6B1Nc7AC0TRYkWrcNmqzVh/bHMJmWYbXn5tGMrwuCd4MJscta9t4RZ54DInp0B5HzZTpL W5Kgtq6TKhRifovadeTuyMnj+KhhrHW0AeXNk0LkS3066tDKAZtNmQqqR62Vo8kXQ+hB8W rr82WkC27s4oIiqMCUdvt9/aWBvCy391r+h44RXhzRWeFGWJLz4lDiHhecIMOw== DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=linutronix.de; s=2020e; t=1728462555; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=PTWEOkMgCycRVxphDRFrN3uHtpIO2IbyhOjzG4V0kAg=; b=lk2cDjxIFuUKsWR6IPIL7ObiQsMRSQSVEjnxsuUpX+vVxYSCdZBbDEefO10UYhSU3Znymk djIvhdqGczqugsBA== Date: Wed, 09 Oct 2024 10:28:56 +0200 Subject: [PATCH v2 03/25] timekeeping: Avoid duplicate leap state update Precedence: bulk X-Mailing-List: netdev@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Message-Id: <20241009-devel-anna-maria-b4-timers-ptp-timekeeping-v2-3-554456a44a15@linutronix.de> References: <20241009-devel-anna-maria-b4-timers-ptp-timekeeping-v2-0-554456a44a15@linutronix.de> In-Reply-To: <20241009-devel-anna-maria-b4-timers-ptp-timekeeping-v2-0-554456a44a15@linutronix.de> To: John Stultz , Frederic Weisbecker , Thomas Gleixner Cc: linux-kernel@vger.kernel.org, netdev@vger.kernel.org, Miroslav Lichvar , Richard Cochran , Christopher S Hall , Anna-Maria Behnsen From: Anna-Maria Behnsen do_adjtimex() invokes tk_update_leap_state() unconditionally even when a previous invocation of timekeeping_update() already did that update. Put it into the else path which is invoked when timekeeping_update() is not called. Signed-off-by: Anna-Maria Behnsen Acked-by: John Stultz --- kernel/time/timekeeping.c | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/kernel/time/timekeeping.c b/kernel/time/timekeeping.c index 4e3afe22d28c..c4dd460b6f2b 100644 --- a/kernel/time/timekeeping.c +++ b/kernel/time/timekeeping.c @@ -2596,8 +2596,9 @@ int do_adjtimex(struct __kernel_timex *txc) __timekeeping_set_tai_offset(tk, tai); timekeeping_update(tk, TK_MIRROR | TK_CLOCK_WAS_SET); clock_set = true; + } else { + tk_update_leap_state(tk); } - tk_update_leap_state(tk); write_seqcount_end(&tk_core.seq); raw_spin_unlock_irqrestore(&timekeeper_lock, flags); From patchwork Wed Oct 9 08:28:57 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Anna-Maria Behnsen X-Patchwork-Id: 13827740 Received: from galois.linutronix.de (Galois.linutronix.de [193.142.43.55]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 8228A188739; Wed, 9 Oct 2024 08:29:17 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=193.142.43.55 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1728462559; cv=none; b=D3rxKmCkc0xDzUuw731t1qIlwj0HlfTsTLuCL9i9f6emokJ7b03ewb6Rq1KEq9oFxT2VFGtVltYGLvqDvlDnw1yItfR2Ttk0XAsXJPI7Hi0sYMMzWwXt7ILEySaSscvFptHbXJjW0qvorRbVPjh1wmeOK9awtzP0IKlFdW8Wn3Q= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1728462559; c=relaxed/simple; bh=zlJbCDZ6B9SeUKQDoLwvls4pdXQE7EYEHEQq9ledjk4=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=AQnltbBGpcUPM4D0LWtcI2lH/HKrqhabBSSp3WEiEnJa+xVaMSR85paeTI/lMJQioE15kd7wmAU/Rjjs13WA+IOegEPjth67mHBqGqx64tQ0YeCZNSpXGl4IPoP5MuNv5bet1sv66MbvcKFp4j3Z4yXeZ0qwnQtz4wx3GOweo2E= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linutronix.de; spf=pass smtp.mailfrom=linutronix.de; dkim=pass (2048-bit key) header.d=linutronix.de header.i=@linutronix.de header.b=Tl8D/Rr4; dkim=permerror (0-bit key) header.d=linutronix.de header.i=@linutronix.de header.b=Q/jxvWJF; arc=none smtp.client-ip=193.142.43.55 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linutronix.de Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linutronix.de Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=linutronix.de header.i=@linutronix.de header.b="Tl8D/Rr4"; dkim=permerror (0-bit key) header.d=linutronix.de header.i=@linutronix.de header.b="Q/jxvWJF" From: Anna-Maria Behnsen DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linutronix.de; s=2020; t=1728462555; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=+wzvWtgEBSrKgOrWII19Tdc3v5jZTMrQEdlErVE1mMc=; b=Tl8D/Rr4dRkEQBUdDIEJQIy8B3ylOFTKqY+ynnnRegsIc2USTBxv1X3rhThZf3y4T/sptX oMmpfloa58p1T93bhifLCu4KoLk8CMbEmMT4sYG4g8qdl75PLJP56walDndjzsFbN5syzv ErSSnwcPKozTkKbmhVERas53tEDXG+evmgxczKLRpnr/xRa2NjPvzQEfOV/rPdrHFvJskt VOJjtlLquTSz9QeMOBbajhO/tTHGyxZez95c/J79RnHaZKN8r3B5xcbOOZolrRV/xcc2bf urdfJPfGTWox4x/wX3VxJDnkq9O4aEb63rE1R7u2n62pWe3PB9g/AenseG4jcw== DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=linutronix.de; s=2020e; t=1728462555; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=+wzvWtgEBSrKgOrWII19Tdc3v5jZTMrQEdlErVE1mMc=; b=Q/jxvWJFjHojB2i1lX6/MZ1fDbU5/1AKfPXakao1PdpAsHR9JeN6RVhTLv50sRgVRwVWXN a4JGbe1T/S+RIUCg== Date: Wed, 09 Oct 2024 10:28:57 +0200 Subject: [PATCH v2 04/25] timekeeping: Abort clocksource change in case of failure Precedence: bulk X-Mailing-List: netdev@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Message-Id: <20241009-devel-anna-maria-b4-timers-ptp-timekeeping-v2-4-554456a44a15@linutronix.de> References: <20241009-devel-anna-maria-b4-timers-ptp-timekeeping-v2-0-554456a44a15@linutronix.de> In-Reply-To: <20241009-devel-anna-maria-b4-timers-ptp-timekeeping-v2-0-554456a44a15@linutronix.de> To: John Stultz , Frederic Weisbecker , Thomas Gleixner Cc: linux-kernel@vger.kernel.org, netdev@vger.kernel.org, Miroslav Lichvar , Richard Cochran , Christopher S Hall , Anna-Maria Behnsen From: Thomas Gleixner From: Thomas Gleixner There is no point to go through a full timekeeping update when acquiring a module reference or enabling the new clocksource fails. Signed-off-by: Thomas Gleixner Signed-off-by: Anna-Maria Behnsen Acked-by: John Stultz --- kernel/time/timekeeping.c | 31 +++++++++++++------------------ 1 file changed, 13 insertions(+), 18 deletions(-) diff --git a/kernel/time/timekeeping.c b/kernel/time/timekeeping.c index c4dd460b6f2b..0ae35a71f8cc 100644 --- a/kernel/time/timekeeping.c +++ b/kernel/time/timekeeping.c @@ -1577,33 +1577,29 @@ static void __timekeeping_set_tai_offset(struct timekeeper *tk, s32 tai_offset) static int change_clocksource(void *data) { struct timekeeper *tk = &tk_core.timekeeper; - struct clocksource *new, *old = NULL; + struct clocksource *new = data, *old = NULL; unsigned long flags; - bool change = false; - - new = (struct clocksource *) data; /* - * If the cs is in module, get a module reference. Succeeds - * for built-in code (owner == NULL) as well. + * If the clocksource is in a module, get a module reference. + * Succeeds for built-in code (owner == NULL) as well. Abort if the + * reference can't be acquired. */ - if (try_module_get(new->owner)) { - if (!new->enable || new->enable(new) == 0) - change = true; - else - module_put(new->owner); + if (!try_module_get(new->owner)) + return 0; + + /* Abort if the device can't be enabled */ + if (new->enable && new->enable(new) != 0) { + module_put(new->owner); + return 0; } raw_spin_lock_irqsave(&timekeeper_lock, flags); write_seqcount_begin(&tk_core.seq); timekeeping_forward_now(tk); - - if (change) { - old = tk->tkr_mono.clock; - tk_setup_internals(tk, new); - } - + old = tk->tkr_mono.clock; + tk_setup_internals(tk, new); timekeeping_update(tk, TK_CLEAR_NTP | TK_MIRROR | TK_CLOCK_WAS_SET); write_seqcount_end(&tk_core.seq); @@ -1612,7 +1608,6 @@ static int change_clocksource(void *data) if (old) { if (old->disable) old->disable(old); - module_put(old->owner); } From patchwork Wed Oct 9 08:28:58 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Anna-Maria Behnsen X-Patchwork-Id: 13827750 Received: from galois.linutronix.de (Galois.linutronix.de [193.142.43.55]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id A638218C91B; Wed, 9 Oct 2024 08:29:19 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=193.142.43.55 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1728462563; cv=none; b=q3AmGggeewBlM60hyXjoLDxqUVQc7puBVxzbySnB1/JFxuczncr+drpiBlpsV7v91prhDQcmTGuiKjnPxZksi/uqhcAyFHV+CnuPSzQDmJzcAOMJ6DzTj6GTjdpEStrXmHQkPG4oeJJMNuhbD8/3zLlLmOZCrmYo4Wk3VNdWDLY= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1728462563; c=relaxed/simple; bh=bP6PT+BjsxnxKJYQnixZ0YZkzL5ONtzGBN692fn+CXU=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=pNupCQojJ9TsOzo+uh90q6VTgaIiF6EKT1eg3OkZ8cVQob9JMIy+MsyjWZl+p/EAev6hE4tdqEOgeEGnoMWqsoFbT2Nj3sDQUAiWLbQECu15sZ+dr+KeTqjPNrxtTh9w0n1HKZiH8OuUf5nXyj8g7M6ktPoq2uDEPtXCvTYQcV4= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linutronix.de; spf=pass smtp.mailfrom=linutronix.de; dkim=pass (2048-bit key) header.d=linutronix.de header.i=@linutronix.de header.b=HiS84X8s; dkim=permerror (0-bit key) header.d=linutronix.de header.i=@linutronix.de header.b=RmBQl7H9; arc=none smtp.client-ip=193.142.43.55 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linutronix.de Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linutronix.de Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=linutronix.de header.i=@linutronix.de header.b="HiS84X8s"; dkim=permerror (0-bit key) header.d=linutronix.de header.i=@linutronix.de header.b="RmBQl7H9" From: Anna-Maria Behnsen DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linutronix.de; s=2020; t=1728462556; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=W0QD9rffq9aVVw2zmPBZOEcSey8yBIu2G1fc+/IahHg=; b=HiS84X8slIUVYKRdo9+rhCw28+nWEGrsqoesfOi2JmDpLYHdB2hDqklemAiUy67IMV93/U 5oOCRCqZ5TR2qIN8Ypd1CBUA6hIULW1g8IuOQLhX3X51utHlRbhzwyh+2SQLoSLG9PZava a8UnVkQOMVOBw7uFu/IYe8hLQfFSHjP9ScqihdhUXSK9W0zzBGWsUkb4hcP6d2bhNusaAv D9RtQS2E8GYRHsgKewzVXWWlbpVVhl6h2a05MeUpyTjam4wP4R7Bj3fOny2gaHuovixvrI cj11KWtDGHH6ln0kiRxhHXeaTtQlOS0Dw05WLb2kim7vzPNI4wQ+PxJs5djZtw== DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=linutronix.de; s=2020e; t=1728462556; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=W0QD9rffq9aVVw2zmPBZOEcSey8yBIu2G1fc+/IahHg=; b=RmBQl7H9TEbo6aYmLgzBRkOYzDKdEqxc0j+nl/eSSp27sKklyp/OwQ0E8dw6Fq+Gk+Z4Np hOdjWT96g9cX9ECA== Date: Wed, 09 Oct 2024 10:28:58 +0200 Subject: [PATCH v2 05/25] timekeeping: Simplify code in timekeeping_advance() Precedence: bulk X-Mailing-List: netdev@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Message-Id: <20241009-devel-anna-maria-b4-timers-ptp-timekeeping-v2-5-554456a44a15@linutronix.de> References: <20241009-devel-anna-maria-b4-timers-ptp-timekeeping-v2-0-554456a44a15@linutronix.de> In-Reply-To: <20241009-devel-anna-maria-b4-timers-ptp-timekeeping-v2-0-554456a44a15@linutronix.de> To: John Stultz , Frederic Weisbecker , Thomas Gleixner Cc: linux-kernel@vger.kernel.org, netdev@vger.kernel.org, Miroslav Lichvar , Richard Cochran , Christopher S Hall , Anna-Maria Behnsen From: Thomas Gleixner From: Thomas Gleixner timekeeping_advance() takes the timekeeper_lock and releases it before returning. When an early return is required, goto statements are used to make sure the lock is realeased properly. When the code was written the locking guard() was not yet available. Use the guard() to simplify the code and while at it cleanup ordering of function variables. No functional change. Signed-off-by: Thomas Gleixner Signed-off-by: Anna-Maria Behnsen Acked-by: John Stultz --- kernel/time/timekeeping.c | 16 ++++++---------- 1 file changed, 6 insertions(+), 10 deletions(-) diff --git a/kernel/time/timekeeping.c b/kernel/time/timekeeping.c index 0ae35a71f8cc..e747e46a1a2d 100644 --- a/kernel/time/timekeeping.c +++ b/kernel/time/timekeeping.c @@ -2276,23 +2276,22 @@ static bool timekeeping_advance(enum timekeeping_adv_mode mode) { struct timekeeper *real_tk = &tk_core.timekeeper; struct timekeeper *tk = &shadow_timekeeper; - u64 offset; - int shift = 0, maxshift; unsigned int clock_set = 0; - unsigned long flags; + int shift = 0, maxshift; + u64 offset; - raw_spin_lock_irqsave(&timekeeper_lock, flags); + guard(raw_spinlock_irqsave)(&timekeeper_lock); /* Make sure we're fully resumed: */ if (unlikely(timekeeping_suspended)) - goto out; + return false; offset = clocksource_delta(tk_clock_read(&tk->tkr_mono), tk->tkr_mono.cycle_last, tk->tkr_mono.mask); /* Check if there's really nothing to do */ if (offset < real_tk->cycle_interval && mode == TK_ADV_TICK) - goto out; + return false; /* Do some additional sanity checking */ timekeeping_check_update(tk, offset); @@ -2311,8 +2310,7 @@ static bool timekeeping_advance(enum timekeeping_adv_mode mode) maxshift = (64 - (ilog2(ntp_tick_length())+1)) - 1; shift = min(shift, maxshift); while (offset >= tk->cycle_interval) { - offset = logarithmic_accumulation(tk, offset, shift, - &clock_set); + offset = logarithmic_accumulation(tk, offset, shift, &clock_set); if (offset < tk->cycle_interval< X-Patchwork-Id: 13827744 Received: from galois.linutronix.de (Galois.linutronix.de [193.142.43.55]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id A627618C900; Wed, 9 Oct 2024 08:29:19 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=193.142.43.55 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1728462561; cv=none; b=sgqfeAMD3QlJURM8Xlziai2XTRS0kyVFQTzIr4oDOe526t+pHzG2kk+A5N510ltOB9fiG5iVQUcEq2MfYFW1/uAqxa7j32jTyp1i+37p4T0Z+wU8epeGgbsdG1ildmwlwfatvaKp2iQDcY92GacMGkUTZQEB5JoXW7Rn+EnXDl0= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1728462561; c=relaxed/simple; bh=FGhVrq4Urqn/SzxPlaaZEIbxZHILDBxsBDX4EvG/mGM=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=ZvGsAlFSF+aLfsO6ttxJM2tKydNmoss8bMpM5HwV1OHVUEUqQLo3Q4pFRh30/mXUqLnyeIYOoym8SDGBt8cZcgpmt/DpO16/hDZFO3Ij02WJmN8qDpNs3BChFNuw1QQMXeZ9YqI645thT7nm4w71TQZfOvghWbiGM+CH748pvPw= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linutronix.de; spf=pass smtp.mailfrom=linutronix.de; dkim=pass (2048-bit key) header.d=linutronix.de header.i=@linutronix.de header.b=Jtt+1Nc9; dkim=permerror (0-bit key) header.d=linutronix.de header.i=@linutronix.de header.b=7AjJyT2l; arc=none smtp.client-ip=193.142.43.55 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linutronix.de Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linutronix.de Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=linutronix.de header.i=@linutronix.de header.b="Jtt+1Nc9"; dkim=permerror (0-bit key) header.d=linutronix.de header.i=@linutronix.de header.b="7AjJyT2l" From: Anna-Maria Behnsen DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linutronix.de; s=2020; t=1728462556; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=Rx0nNyoiuj3twfKhvVfpIoSTggUDAfol6D4phv7a204=; b=Jtt+1Nc9AIG7CxL59tYIz/305cfUxw3a0iF4fo7kxR0VyYsvFRBb4jfvd/VsvKgQOR7RTm yq08Itwg3Yp7s3e29NNLnJ0A7Xouz04596sehI71iiewO+uOlZuBHyFyM5NH4TLxzq4EYr fIGnlokQsXGhfkWkX43gr5VlK0WMoc9dHl6nyj2TRF8DokSiOQ1ZSEWLaUvBdULjhNNL7N 17MUn4/zder2ZXeZqIKqh1Smv/ABhVpT+q2ZI9lGVon5mGBbEojf25TWODce5Rj2gRvrsn hpcybTJWZI9e1SwBOv5XNDRGxfKIOhNb+2lLiXBQrK7Z40j8Fl6WtEgxRbb8Tg== DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=linutronix.de; s=2020e; t=1728462556; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=Rx0nNyoiuj3twfKhvVfpIoSTggUDAfol6D4phv7a204=; b=7AjJyT2l57INLvvjyZzHKXK6VWS7DuY3mJ7BJvqkjPPqznpMEEkt6vX5cbsaN3oiO8biJH mCw4C3/pPIB8MQCQ== Date: Wed, 09 Oct 2024 10:28:59 +0200 Subject: [PATCH v2 06/25] timekeeping: Reorder struct timekeeper Precedence: bulk X-Mailing-List: netdev@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Message-Id: <20241009-devel-anna-maria-b4-timers-ptp-timekeeping-v2-6-554456a44a15@linutronix.de> References: <20241009-devel-anna-maria-b4-timers-ptp-timekeeping-v2-0-554456a44a15@linutronix.de> In-Reply-To: <20241009-devel-anna-maria-b4-timers-ptp-timekeeping-v2-0-554456a44a15@linutronix.de> To: John Stultz , Frederic Weisbecker , Thomas Gleixner Cc: linux-kernel@vger.kernel.org, netdev@vger.kernel.org, Miroslav Lichvar , Richard Cochran , Christopher S Hall , Anna-Maria Behnsen From: Thomas Gleixner From: Thomas Gleixner struct timekeeper is ordered suboptimal vs. cachelines. The layout, including the preceding seqcount (see struct tk_core in timekeeper.c) is: cacheline 0: seqcount, tkr_mono cacheline 1: tkr_raw, xtime_sec cacheline 2: ktime_sec ... tai_offset, internal variables cacheline 3: next_leap_ktime, raw_sec, internal variables cacheline 4: internal variables So any access to via ktime_get*() except for access to CLOCK_MONOTONIC_RAW will use either cachelines 0 + 1 or cachelines 0 + 2. Access to CLOCK_MONOTONIC_RAW uses cachelines 0 + 1 + 3. Reorder the members so that the result is more efficient: cacheline 0: seqcount, tkr_mono cacheline 1: xtime_sec, ktime_sec ... tai_offset cacheline 2: tkr_raw, raw_sec cacheline 3: internal variables cacheline 4: internal variables That means ktime_get*() will access cacheline 0 + 1 and CLOCK_MONOTONIC_RAW access will use cachelines 0 + 2. Update kernel-doc and fix formatting issues while at it. Also fix a typo in struct tk_read_base kernel-doc. Signed-off-by: Thomas Gleixner Signed-off-by: Anna-Maria Behnsen Acked-by: John Stultz --- include/linux/timekeeper_internal.h | 102 +++++++++++++++++++++--------------- 1 file changed, 61 insertions(+), 41 deletions(-) diff --git a/include/linux/timekeeper_internal.h b/include/linux/timekeeper_internal.h index 902c20ef495a..430e40549136 100644 --- a/include/linux/timekeeper_internal.h +++ b/include/linux/timekeeper_internal.h @@ -26,7 +26,7 @@ * occupies a single 64byte cache line. * * The struct is separate from struct timekeeper as it is also used - * for a fast NMI safe accessors. + * for the fast NMI safe accessors. * * @base_real is for the fast NMI safe accessor to allow reading clock * realtime from any context. @@ -44,33 +44,41 @@ struct tk_read_base { /** * struct timekeeper - Structure holding internal timekeeping values. - * @tkr_mono: The readout base structure for CLOCK_MONOTONIC - * @tkr_raw: The readout base structure for CLOCK_MONOTONIC_RAW - * @xtime_sec: Current CLOCK_REALTIME time in seconds - * @ktime_sec: Current CLOCK_MONOTONIC time in seconds - * @wall_to_monotonic: CLOCK_REALTIME to CLOCK_MONOTONIC offset - * @offs_real: Offset clock monotonic -> clock realtime - * @offs_boot: Offset clock monotonic -> clock boottime - * @offs_tai: Offset clock monotonic -> clock tai - * @tai_offset: The current UTC to TAI offset in seconds - * @clock_was_set_seq: The sequence number of clock was set events - * @cs_was_changed_seq: The sequence number of clocksource change events - * @next_leap_ktime: CLOCK_MONOTONIC time value of a pending leap-second - * @raw_sec: CLOCK_MONOTONIC_RAW time in seconds - * @monotonic_to_boot: CLOCK_MONOTONIC to CLOCK_BOOTTIME offset - * @cycle_interval: Number of clock cycles in one NTP interval - * @xtime_interval: Number of clock shifted nano seconds in one NTP - * interval. - * @xtime_remainder: Shifted nano seconds left over when rounding - * @cycle_interval - * @raw_interval: Shifted raw nano seconds accumulated per NTP interval. - * @ntp_error: Difference between accumulated time and NTP time in ntp - * shifted nano seconds. - * @ntp_error_shift: Shift conversion between clock shifted nano seconds and - * ntp shifted nano seconds. - * @last_warning: Warning ratelimiter (DEBUG_TIMEKEEPING) - * @underflow_seen: Underflow warning flag (DEBUG_TIMEKEEPING) - * @overflow_seen: Overflow warning flag (DEBUG_TIMEKEEPING) + * @tkr_mono: The readout base structure for CLOCK_MONOTONIC + * @xtime_sec: Current CLOCK_REALTIME time in seconds + * @ktime_sec: Current CLOCK_MONOTONIC time in seconds + * @wall_to_monotonic: CLOCK_REALTIME to CLOCK_MONOTONIC offset + * @offs_real: Offset clock monotonic -> clock realtime + * @offs_boot: Offset clock monotonic -> clock boottime + * @offs_tai: Offset clock monotonic -> clock tai + * @tai_offset: The current UTC to TAI offset in seconds + * @tkr_raw: The readout base structure for CLOCK_MONOTONIC_RAW + * @raw_sec: CLOCK_MONOTONIC_RAW time in seconds + * @clock_was_set_seq: The sequence number of clock was set events + * @cs_was_changed_seq: The sequence number of clocksource change events + * @monotonic_to_boot: CLOCK_MONOTONIC to CLOCK_BOOTTIME offset + * @cycle_interval: Number of clock cycles in one NTP interval + * @xtime_interval: Number of clock shifted nano seconds in one NTP + * interval. + * @xtime_remainder: Shifted nano seconds left over when rounding + * @cycle_interval + * @raw_interval: Shifted raw nano seconds accumulated per NTP interval. + * @next_leap_ktime: CLOCK_MONOTONIC time value of a pending leap-second + * @ntp_tick: The ntp_tick_length() value currently being + * used. This cached copy ensures we consistently + * apply the tick length for an entire tick, as + * ntp_tick_length may change mid-tick, and we don't + * want to apply that new value to the tick in + * progress. + * @ntp_error: Difference between accumulated time and NTP time in ntp + * shifted nano seconds. + * @ntp_error_shift: Shift conversion between clock shifted nano seconds and + * ntp shifted nano seconds. + * @ntp_err_mult: Multiplication factor for scaled math conversion + * @skip_second_overflow: Flag used to avoid updating NTP twice with same second + * @last_warning: Warning ratelimiter (DEBUG_TIMEKEEPING) + * @underflow_seen: Underflow warning flag (DEBUG_TIMEKEEPING) + * @overflow_seen: Overflow warning flag (DEBUG_TIMEKEEPING) * * Note: For timespec(64) based interfaces wall_to_monotonic is what * we need to add to xtime (or xtime corrected for sub jiffy times) @@ -88,10 +96,25 @@ struct tk_read_base { * * @monotonic_to_boottime is a timespec64 representation of @offs_boot to * accelerate the VDSO update for CLOCK_BOOTTIME. + * + * The cacheline ordering of the structure is optimized for in kernel usage + * of the ktime_get() and ktime_get_ts64() family of time accessors. Struct + * timekeeper is prepended in the core timekeeeping code with a sequence + * count, which results in the following cacheline layout: + * + * 0: seqcount, tkr_mono + * 1: xtime_sec ... tai_offset + * 2: tkr_raw, raw_sec + * 3,4: Internal variables + * + * Cacheline 0,1 contain the data which is used for accessing + * CLOCK_MONOTONIC/REALTIME/BOOTTIME/TAI, while cacheline 2 contains the + * data for accessing CLOCK_MONOTONIC_RAW. Cacheline 3,4 are internal + * variables which are only accessed during timekeeper updates once per + * tick. */ struct timekeeper { struct tk_read_base tkr_mono; - struct tk_read_base tkr_raw; u64 xtime_sec; unsigned long ktime_sec; struct timespec64 wall_to_monotonic; @@ -99,31 +122,28 @@ struct timekeeper { ktime_t offs_boot; ktime_t offs_tai; s32 tai_offset; + + struct tk_read_base tkr_raw; + u64 raw_sec; + + /* The following members are for timekeeping internal use */ unsigned int clock_was_set_seq; u8 cs_was_changed_seq; - ktime_t next_leap_ktime; - u64 raw_sec; + struct timespec64 monotonic_to_boot; - /* The following members are for timekeeping internal use */ u64 cycle_interval; u64 xtime_interval; s64 xtime_remainder; u64 raw_interval; - /* The ntp_tick_length() value currently being used. - * This cached copy ensures we consistently apply the tick - * length for an entire tick, as ntp_tick_length may change - * mid-tick, and we don't want to apply that new value to - * the tick in progress. - */ + + ktime_t next_leap_ktime; u64 ntp_tick; - /* Difference between accumulated time and NTP time in ntp - * shifted nano seconds. */ s64 ntp_error; u32 ntp_error_shift; u32 ntp_err_mult; - /* Flag used to avoid updating NTP twice with same second */ u32 skip_second_overflow; + #ifdef CONFIG_DEBUG_TIMEKEEPING long last_warning; /* From patchwork Wed Oct 9 08:29:00 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Anna-Maria Behnsen X-Patchwork-Id: 13827742 Received: from galois.linutronix.de (Galois.linutronix.de [193.142.43.55]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id A631718C91A; Wed, 9 Oct 2024 08:29:19 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=193.142.43.55 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1728462561; cv=none; b=nx9LdiSEpu/n0o4ECOgCdm+vh2uu6/P1XPdV5DAMO9bGCSQ23f4xUmIqxhmg0q0LP7P6m+BL0Ibgy/XWdcL1th8GkOMkigUkLsGsZLfrDuZEwupT6jYtzeGG75l1igzXjiuOyHj7QpK2kr3PgJvid2yQLWCq5JDjPeEvAFuPHZw= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1728462561; c=relaxed/simple; bh=Lmnv1b2ucOFQSUA24R7e7ebxT8F4nIGS58fa4J9BytE=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=tbfaaLxS1xSARo4egNU5xtHm105rp72T7l2NsOrrRW/09EtAbGr1CuncGGKeXveUv/BnwPxv7vuTb5Z0oE/z+bTA4dq7TMjuS0dfWSU60yp6jQ9kZSkkpZHeFvaq4avDF59hxIRCPu/nLi4bpSfpKGTOfp9675em5Ti49zE8iE8= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linutronix.de; spf=pass smtp.mailfrom=linutronix.de; dkim=pass (2048-bit key) header.d=linutronix.de header.i=@linutronix.de header.b=QY2C6JuR; dkim=permerror (0-bit key) header.d=linutronix.de header.i=@linutronix.de header.b=BGrwc7zb; arc=none smtp.client-ip=193.142.43.55 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linutronix.de Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linutronix.de Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=linutronix.de header.i=@linutronix.de header.b="QY2C6JuR"; dkim=permerror (0-bit key) header.d=linutronix.de header.i=@linutronix.de header.b="BGrwc7zb" From: Anna-Maria Behnsen DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linutronix.de; s=2020; t=1728462556; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=+gKqhrrEtuUuIiu1c0v+h4KjnR2UQ6QcFIf5LjmxyqA=; b=QY2C6JuR4KT02bGdvaqaeEq8+eYbz7Phz512iuZBq1wlrQm0pZNc60ZLyyCqknXNfSn5Wb /bZttYmgJ96ry3XvJd/jhcrty4wZMDK+yBBD+wcwZkod30U4GSFVHFIB+SDMsB6mHUzX3x KqGMSYKUapxNsk+2DrUKBJu0SwBZ+nEfomXTKYww4rr95efxyd994/67rzqf/JAywkDhOl syeLuOgLsksTmxRURzE2eQchWlzhSh7IfzAT7H2nRnZWqA0Ci++JPGpFkORm2ol8VkoQgF eRbPbkgFk33WBSUMYknqU6PqgU9qWzL1ZC9cwn/auMLkj30WWN9VVSuQm57amw== DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=linutronix.de; s=2020e; t=1728462556; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=+gKqhrrEtuUuIiu1c0v+h4KjnR2UQ6QcFIf5LjmxyqA=; b=BGrwc7zbAeMl1Mei92Pf/PRBzcsi68gso2N2IljcxgYKXjZ6ewt+qOm5e5cXkeYIMshT/8 fQJ4SuWEOfTAh9BA== Date: Wed, 09 Oct 2024 10:29:00 +0200 Subject: [PATCH v2 07/25] timekeeping: Move shadow_timekeeper into tk_core Precedence: bulk X-Mailing-List: netdev@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Message-Id: <20241009-devel-anna-maria-b4-timers-ptp-timekeeping-v2-7-554456a44a15@linutronix.de> References: <20241009-devel-anna-maria-b4-timers-ptp-timekeeping-v2-0-554456a44a15@linutronix.de> In-Reply-To: <20241009-devel-anna-maria-b4-timers-ptp-timekeeping-v2-0-554456a44a15@linutronix.de> To: John Stultz , Frederic Weisbecker , Thomas Gleixner Cc: linux-kernel@vger.kernel.org, netdev@vger.kernel.org, Miroslav Lichvar , Richard Cochran , Christopher S Hall , Anna-Maria Behnsen From: Thomas Gleixner From: Thomas Gleixner tk_core requires shadow_timekeeper to allow timekeeping_advance() updating without holding the timekeeper sequence count write locked. This allows the readers to make progress up to the actual update where the shadow timekeeper is copied over to the real timekeeper. As long as there is only a single timekeeper, having them separate is fine. But when the timekeeper infrastructure will be reused for per ptp clock timekeepers, shadow_timekeeper needs to be part of tk_core. No functional change. Signed-off-by: Thomas Gleixner Signed-off-by: Anna-Maria Behnsen Acked-by: John Stultz --- kernel/time/timekeeping.c | 7 +++---- 1 file changed, 3 insertions(+), 4 deletions(-) diff --git a/kernel/time/timekeeping.c b/kernel/time/timekeeping.c index e747e46a1a2d..267b28cf2ab0 100644 --- a/kernel/time/timekeeping.c +++ b/kernel/time/timekeeping.c @@ -50,11 +50,11 @@ DEFINE_RAW_SPINLOCK(timekeeper_lock); static struct { seqcount_raw_spinlock_t seq; struct timekeeper timekeeper; + struct timekeeper shadow_timekeeper; } tk_core ____cacheline_aligned = { .seq = SEQCNT_RAW_SPINLOCK_ZERO(tk_core.seq, &timekeeper_lock), }; -static struct timekeeper shadow_timekeeper; /* flag for if timekeeping is suspended */ int __read_mostly timekeeping_suspended; @@ -776,8 +776,7 @@ static void timekeeping_update(struct timekeeper *tk, unsigned int action) * timekeeper structure on the next update with stale data */ if (action & TK_MIRROR) - memcpy(&shadow_timekeeper, &tk_core.timekeeper, - sizeof(tk_core.timekeeper)); + memcpy(&tk_core.shadow_timekeeper, &tk_core.timekeeper, sizeof(tk_core.timekeeper)); } /** @@ -2274,8 +2273,8 @@ static u64 logarithmic_accumulation(struct timekeeper *tk, u64 offset, */ static bool timekeeping_advance(enum timekeeping_adv_mode mode) { + struct timekeeper *tk = &tk_core.shadow_timekeeper; struct timekeeper *real_tk = &tk_core.timekeeper; - struct timekeeper *tk = &shadow_timekeeper; unsigned int clock_set = 0; int shift = 0, maxshift; u64 offset; From patchwork Wed Oct 9 08:29:01 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Anna-Maria Behnsen X-Patchwork-Id: 13827741 Received: from galois.linutronix.de (Galois.linutronix.de [193.142.43.55]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id A62C618C90C; Wed, 9 Oct 2024 08:29:19 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=193.142.43.55 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1728462561; cv=none; b=T5WaZlsegRAystIEAYRHw61AQDuAlyplfPYsh7xtMA7GbSwgZJDwtKnas2+06o8BiSlyOndCdczbS0DKJmEzqUIzJlV35R5BQOdiStV6MZ30zIyW+FnXPRoPMmg1WHEolyPRTbbiZS1wBe2X/q8vdOMrhEPxYpqdovayDSdKcu4= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1728462561; c=relaxed/simple; bh=thlEHaeMIi5clGAUANmNSg25A9UJhi9FpJHEZWgWQbU=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=dG9I2sGgmhiE1CFGsEKef/7MPtgPrMcGQ21Z8qOIyIhzxjGaL8feJqwFaBoJvQtujqv/haZ9kM+M6idfqjFLK3YTRI3c2oQuyTWn2WbyhGsma8Rhwd/kY4lmy0uf8PCm7kEiHhhEluQr9wnIeGEvIEWbst8lW3XBLjqLOExpHzs= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linutronix.de; spf=pass smtp.mailfrom=linutronix.de; dkim=pass (2048-bit key) header.d=linutronix.de header.i=@linutronix.de header.b=mp8UjpWc; dkim=permerror (0-bit key) header.d=linutronix.de header.i=@linutronix.de header.b=T6HYk4Jl; arc=none smtp.client-ip=193.142.43.55 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linutronix.de Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linutronix.de Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=linutronix.de header.i=@linutronix.de header.b="mp8UjpWc"; dkim=permerror (0-bit key) header.d=linutronix.de header.i=@linutronix.de header.b="T6HYk4Jl" From: Anna-Maria Behnsen DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linutronix.de; s=2020; t=1728462557; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=vJKTpvkSbF3tVHNy/ba8jZ7XL06i2bhWx2B1Ab4gGbI=; b=mp8UjpWcTIAyi7jbWYx7CCLGfPKpTKMVpNG71zx8Jakp4GHbL9N+fh/ahC1rmVaiGfpdu1 UYIwSwXo2FyvjMEZEPHjNbzpOdvV5YzL7BqAIaQyln9Qh2yVrxnd7QR72dUQQ9QKGONOX0 wtTCTft1fVqdTOkQTq387eykZ2/oVoKN2B8zRKGXJ2tTwbXrlHDRjvY4q9szLnkErlxKA+ D/U2DuogAgam3324OJ9HlbeKvrwemJIGcgJH3LmZcx98fYfaIXkEjLKHpq1Yvwu/cg0LV4 LwI3tl2WzR7fziHnMUehDGSt7NyK9NRgXBqjmCooyFN6458J2IcvCqBzOgiMBg== DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=linutronix.de; s=2020e; t=1728462557; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=vJKTpvkSbF3tVHNy/ba8jZ7XL06i2bhWx2B1Ab4gGbI=; b=T6HYk4JlzgwxomJBzk5u1UoZcw48pSeqj79aMq28abg32Ryy1sES93bHpY+d6GoLZc0uBV PTV3SSX3gI4bc4Aw== Date: Wed, 09 Oct 2024 10:29:01 +0200 Subject: [PATCH v2 08/25] timekeeping: Encapsulate locking/unlocking of timekeeper_lock Precedence: bulk X-Mailing-List: netdev@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Message-Id: <20241009-devel-anna-maria-b4-timers-ptp-timekeeping-v2-8-554456a44a15@linutronix.de> References: <20241009-devel-anna-maria-b4-timers-ptp-timekeeping-v2-0-554456a44a15@linutronix.de> In-Reply-To: <20241009-devel-anna-maria-b4-timers-ptp-timekeeping-v2-0-554456a44a15@linutronix.de> To: John Stultz , Frederic Weisbecker , Thomas Gleixner Cc: linux-kernel@vger.kernel.org, netdev@vger.kernel.org, Miroslav Lichvar , Richard Cochran , Christopher S Hall , Anna-Maria Behnsen From: Thomas Gleixner From: Thomas Gleixner timekeeper_lock protects updates of timekeeper (tk_core). It is also used by vdso_update_begin/end() and not only internally by the timekeeper code. As long as there is only a single timekeeper, this works fine. But when the timekeeper infrastructure will be reused for per ptp clock timekeepers, timekeeper_lock needs to be part of tk_core.. Therefore encapuslate locking/unlocking of timekeeper_lock and make the lock static. Signed-off-by: Thomas Gleixner Signed-off-by: Anna-Maria Behnsen Acked-by: John Stultz --- kernel/time/timekeeping.c | 15 ++++++++++++++- kernel/time/timekeeping_internal.h | 3 ++- kernel/time/vsyscall.c | 5 ++--- 3 files changed, 18 insertions(+), 5 deletions(-) diff --git a/kernel/time/timekeeping.c b/kernel/time/timekeeping.c index 267b28cf2ab0..c69004936b29 100644 --- a/kernel/time/timekeeping.c +++ b/kernel/time/timekeeping.c @@ -41,7 +41,7 @@ enum timekeeping_adv_mode { TK_ADV_FREQ }; -DEFINE_RAW_SPINLOCK(timekeeper_lock); +static DEFINE_RAW_SPINLOCK(timekeeper_lock); /* * The most important data for readout fits into a single 64 byte @@ -114,6 +114,19 @@ static struct tk_fast tk_fast_raw ____cacheline_aligned = { .base[1] = FAST_TK_INIT, }; +unsigned long timekeeper_lock_irqsave(void) +{ + unsigned long flags; + + raw_spin_lock_irqsave(&timekeeper_lock, flags); + return flags; +} + +void timekeeper_unlock_irqrestore(unsigned long flags) +{ + raw_spin_unlock_irqrestore(&timekeeper_lock, flags); +} + static inline void tk_normalize_xtime(struct timekeeper *tk) { while (tk->tkr_mono.xtime_nsec >= ((u64)NSEC_PER_SEC << tk->tkr_mono.shift)) { diff --git a/kernel/time/timekeeping_internal.h b/kernel/time/timekeeping_internal.h index 4ca2787d1642..3babb2bac579 100644 --- a/kernel/time/timekeeping_internal.h +++ b/kernel/time/timekeeping_internal.h @@ -34,6 +34,7 @@ static inline u64 clocksource_delta(u64 now, u64 last, u64 mask) #endif /* Semi public for serialization of non timekeeper VDSO updates. */ -extern raw_spinlock_t timekeeper_lock; +unsigned long timekeeper_lock_irqsave(void); +void timekeeper_unlock_irqrestore(unsigned long flags); #endif /* _TIMEKEEPING_INTERNAL_H */ diff --git a/kernel/time/vsyscall.c b/kernel/time/vsyscall.c index 9193d6133e5d..98488b20b594 100644 --- a/kernel/time/vsyscall.c +++ b/kernel/time/vsyscall.c @@ -151,9 +151,8 @@ void update_vsyscall_tz(void) unsigned long vdso_update_begin(void) { struct vdso_data *vdata = __arch_get_k_vdso_data(); - unsigned long flags; + unsigned long flags = timekeeper_lock_irqsave(); - raw_spin_lock_irqsave(&timekeeper_lock, flags); vdso_write_begin(vdata); return flags; } @@ -172,5 +171,5 @@ void vdso_update_end(unsigned long flags) vdso_write_end(vdata); __arch_sync_vdso_data(vdata); - raw_spin_unlock_irqrestore(&timekeeper_lock, flags); + timekeeper_unlock_irqrestore(flags); } From patchwork Wed Oct 9 08:29:02 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Anna-Maria Behnsen X-Patchwork-Id: 13827748 Received: from galois.linutronix.de (Galois.linutronix.de [193.142.43.55]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 0FF8B18CC12; Wed, 9 Oct 2024 08:29:19 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=193.142.43.55 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1728462563; cv=none; b=b2v92gOK8s8mtQ4OJIwMNDd5TMbHw+R1WB6ZHIvJ/oDg+bfJV/TrzzOq/Yz4FqNAdvUqnfqr0ZdiXKfIQZlguC6KjNUzCBcDlAap0Sz68R7DY3ODSufKgzNet89XA7sR5N+L9/mrENkXayPtkK8ToZTpF8JK8VwN2y/SxP/JjCI= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1728462563; c=relaxed/simple; bh=/wCUxTAAWXiNlHlzBVt+2w+bdH0nbWdZeI/9VMfEVuE=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=pNKVajbe82S+x0hxk2zNxUBcCzJK0PEWa7sm511pRJtoAnbxYTfF6/2b98W0DvNyTFFX42vaAfXRHt/DQiSmOKZ97Yi5fPxBYleJToCXO2wr4rAgosUSyFrRml2kY1T3Tz8xKdVYpcIbjBz6MLHjaKMexIsDjhRHBjFNbIhkDro= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linutronix.de; spf=pass smtp.mailfrom=linutronix.de; dkim=pass (2048-bit key) header.d=linutronix.de header.i=@linutronix.de header.b=Pk4FSoIc; dkim=permerror (0-bit key) header.d=linutronix.de header.i=@linutronix.de header.b=Hvf0W/3i; arc=none smtp.client-ip=193.142.43.55 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linutronix.de Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linutronix.de Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=linutronix.de header.i=@linutronix.de header.b="Pk4FSoIc"; dkim=permerror (0-bit key) header.d=linutronix.de header.i=@linutronix.de header.b="Hvf0W/3i" From: Anna-Maria Behnsen DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linutronix.de; s=2020; t=1728462557; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=ljxaAp+nO21hPqn0BhGelNMq77OIs2w6b9yghI2F5cI=; b=Pk4FSoIcEIN18KvlsB4ht6qwo6s66lebAYDqPh49r7SVat9vZJ9nIDZz+bXH6GP35h/D7J d0ZumKnuEJ5iTHBs/euB3XyrEHj2dsz5tRosLchzT91zQ6gyCyf4ei2N4miXdhlisDRcVr aAdV3YnINU/AiMeq8TIjYiag5eUBzlpQRrQ9WuzAmIOZlYWD4fWk2CSyQsEfJL3OhDSNqJ WUKo7eFZ0AIbdq93SzJsI1exqqPPd6hXG2C0uWWl5dlkB9EGNqxVtnZoaX1qBb0cDgD/sr 0WtqLpm0nPk8bpQbim/acqGQWM3bnCaWBXhAyDOrhUfUbqJopCGTeUju+XAgVw== DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=linutronix.de; s=2020e; t=1728462557; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=ljxaAp+nO21hPqn0BhGelNMq77OIs2w6b9yghI2F5cI=; b=Hvf0W/3iRMzzXYsTxwtNLa2q/n77HhKSS93VT8zoTc0nXyYa4ylh4pVrPlzHBTNFabfISl p3a5oSpSCpjBpCDQ== Date: Wed, 09 Oct 2024 10:29:02 +0200 Subject: [PATCH v2 09/25] timekeeping: Move timekeeper_lock into tk_core Precedence: bulk X-Mailing-List: netdev@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Message-Id: <20241009-devel-anna-maria-b4-timers-ptp-timekeeping-v2-9-554456a44a15@linutronix.de> References: <20241009-devel-anna-maria-b4-timers-ptp-timekeeping-v2-0-554456a44a15@linutronix.de> In-Reply-To: <20241009-devel-anna-maria-b4-timers-ptp-timekeeping-v2-0-554456a44a15@linutronix.de> To: John Stultz , Frederic Weisbecker , Thomas Gleixner Cc: linux-kernel@vger.kernel.org, netdev@vger.kernel.org, Miroslav Lichvar , Richard Cochran , Christopher S Hall , Anna-Maria Behnsen From: Anna-Maria Behnsen timekeeper_lock protects updates to struct tk_core but is not part of struct tk_core. As long as there is only a single timekeeper, this is not a problem. But when the timekeeper infrastructure will be reused for per ptp clock timekeepers, timekeeper_lock needs to be part of tk_core. Move the lock into tk_core, move initialisation of the lock and sequence counter into timekeeping_init() and update all users of timekeeper_lock. As this is touching all lock sites, convert them to use: guard(raw_spinlock_irqsave)(&tk_core.lock); instead of lock/unlock functions whenever possible. Suggested-by: Thomas Gleixner Signed-off-by: Anna-Maria Behnsen Acked-by: John Stultz --- kernel/time/timekeeping.c | 72 +++++++++++++++++++---------------------------- 1 file changed, 29 insertions(+), 43 deletions(-) diff --git a/kernel/time/timekeeping.c b/kernel/time/timekeeping.c index c69004936b29..8557d32e5e3d 100644 --- a/kernel/time/timekeeping.c +++ b/kernel/time/timekeeping.c @@ -41,8 +41,6 @@ enum timekeeping_adv_mode { TK_ADV_FREQ }; -static DEFINE_RAW_SPINLOCK(timekeeper_lock); - /* * The most important data for readout fits into a single 64 byte * cache line. @@ -51,10 +49,8 @@ static struct { seqcount_raw_spinlock_t seq; struct timekeeper timekeeper; struct timekeeper shadow_timekeeper; -} tk_core ____cacheline_aligned = { - .seq = SEQCNT_RAW_SPINLOCK_ZERO(tk_core.seq, &timekeeper_lock), -}; - + raw_spinlock_t lock; +} tk_core ____cacheline_aligned; /* flag for if timekeeping is suspended */ int __read_mostly timekeeping_suspended; @@ -118,13 +114,13 @@ unsigned long timekeeper_lock_irqsave(void) { unsigned long flags; - raw_spin_lock_irqsave(&timekeeper_lock, flags); + raw_spin_lock_irqsave(&tk_core.lock, flags); return flags; } void timekeeper_unlock_irqrestore(unsigned long flags) { - raw_spin_unlock_irqrestore(&timekeeper_lock, flags); + raw_spin_unlock_irqrestore(&tk_core.lock, flags); } static inline void tk_normalize_xtime(struct timekeeper *tk) @@ -197,7 +193,7 @@ static inline void tk_update_sleep_time(struct timekeeper *tk, ktime_t delta) * the tkr's clocksource may change between the read reference, and the * clock reference passed to the read function. This can cause crashes if * the wrong clocksource is passed to the wrong read function. - * This isn't necessary to use when holding the timekeeper_lock or doing + * This isn't necessary to use when holding the tk_core.lock or doing * a read of the fast-timekeeper tkrs (which is protected by its own locking * and update logic). */ @@ -689,13 +685,11 @@ static void update_pvclock_gtod(struct timekeeper *tk, bool was_set) int pvclock_gtod_register_notifier(struct notifier_block *nb) { struct timekeeper *tk = &tk_core.timekeeper; - unsigned long flags; int ret; - raw_spin_lock_irqsave(&timekeeper_lock, flags); + guard(raw_spinlock_irqsave)(&tk_core.lock); ret = raw_notifier_chain_register(&pvclock_gtod_chain, nb); update_pvclock_gtod(tk, true); - raw_spin_unlock_irqrestore(&timekeeper_lock, flags); return ret; } @@ -708,14 +702,8 @@ EXPORT_SYMBOL_GPL(pvclock_gtod_register_notifier); */ int pvclock_gtod_unregister_notifier(struct notifier_block *nb) { - unsigned long flags; - int ret; - - raw_spin_lock_irqsave(&timekeeper_lock, flags); - ret = raw_notifier_chain_unregister(&pvclock_gtod_chain, nb); - raw_spin_unlock_irqrestore(&timekeeper_lock, flags); - - return ret; + guard(raw_spinlock_irqsave)(&tk_core.lock); + return raw_notifier_chain_unregister(&pvclock_gtod_chain, nb); } EXPORT_SYMBOL_GPL(pvclock_gtod_unregister_notifier); @@ -763,7 +751,7 @@ static inline void tk_update_ktime_data(struct timekeeper *tk) tk->tkr_raw.base = ns_to_ktime(tk->raw_sec * NSEC_PER_SEC); } -/* must hold timekeeper_lock */ +/* must hold tk_core.lock */ static void timekeeping_update(struct timekeeper *tk, unsigned int action) { if (action & TK_CLEAR_NTP) { @@ -1460,7 +1448,7 @@ int do_settimeofday64(const struct timespec64 *ts) if (!timespec64_valid_settod(ts)) return -EINVAL; - raw_spin_lock_irqsave(&timekeeper_lock, flags); + raw_spin_lock_irqsave(&tk_core.lock, flags); write_seqcount_begin(&tk_core.seq); timekeeping_forward_now(tk); @@ -1480,7 +1468,7 @@ int do_settimeofday64(const struct timespec64 *ts) timekeeping_update(tk, TK_CLEAR_NTP | TK_MIRROR | TK_CLOCK_WAS_SET); write_seqcount_end(&tk_core.seq); - raw_spin_unlock_irqrestore(&timekeeper_lock, flags); + raw_spin_unlock_irqrestore(&tk_core.lock, flags); /* Signal hrtimers about time change */ clock_was_set(CLOCK_SET_WALL); @@ -1510,7 +1498,7 @@ static int timekeeping_inject_offset(const struct timespec64 *ts) if (ts->tv_nsec < 0 || ts->tv_nsec >= NSEC_PER_SEC) return -EINVAL; - raw_spin_lock_irqsave(&timekeeper_lock, flags); + raw_spin_lock_irqsave(&tk_core.lock, flags); write_seqcount_begin(&tk_core.seq); timekeeping_forward_now(tk); @@ -1530,7 +1518,7 @@ static int timekeeping_inject_offset(const struct timespec64 *ts) timekeeping_update(tk, TK_CLEAR_NTP | TK_MIRROR | TK_CLOCK_WAS_SET); write_seqcount_end(&tk_core.seq); - raw_spin_unlock_irqrestore(&timekeeper_lock, flags); + raw_spin_unlock_irqrestore(&tk_core.lock, flags); /* Signal hrtimers about time change */ clock_was_set(CLOCK_SET_WALL); @@ -1606,7 +1594,7 @@ static int change_clocksource(void *data) return 0; } - raw_spin_lock_irqsave(&timekeeper_lock, flags); + raw_spin_lock_irqsave(&tk_core.lock, flags); write_seqcount_begin(&tk_core.seq); timekeeping_forward_now(tk); @@ -1615,7 +1603,7 @@ static int change_clocksource(void *data) timekeeping_update(tk, TK_CLEAR_NTP | TK_MIRROR | TK_CLOCK_WAS_SET); write_seqcount_end(&tk_core.seq); - raw_spin_unlock_irqrestore(&timekeeper_lock, flags); + raw_spin_unlock_irqrestore(&tk_core.lock, flags); if (old) { if (old->disable) @@ -1770,7 +1758,9 @@ void __init timekeeping_init(void) struct timespec64 wall_time, boot_offset, wall_to_mono; struct timekeeper *tk = &tk_core.timekeeper; struct clocksource *clock; - unsigned long flags; + + raw_spin_lock_init(&tk_core.lock); + seqcount_raw_spinlock_init(&tk_core.seq, &tkd->lock); read_persistent_wall_and_boot_offset(&wall_time, &boot_offset); if (timespec64_valid_settod(&wall_time) && @@ -1790,7 +1780,7 @@ void __init timekeeping_init(void) */ wall_to_mono = timespec64_sub(boot_offset, wall_time); - raw_spin_lock_irqsave(&timekeeper_lock, flags); + guard(raw_spinlock_irqsave)(&tk_core.lock); write_seqcount_begin(&tk_core.seq); ntp_init(); @@ -1807,7 +1797,6 @@ void __init timekeeping_init(void) timekeeping_update(tk, TK_MIRROR | TK_CLOCK_WAS_SET); write_seqcount_end(&tk_core.seq); - raw_spin_unlock_irqrestore(&timekeeper_lock, flags); } /* time in seconds when suspend began for persistent clock */ @@ -1888,7 +1877,7 @@ void timekeeping_inject_sleeptime64(const struct timespec64 *delta) struct timekeeper *tk = &tk_core.timekeeper; unsigned long flags; - raw_spin_lock_irqsave(&timekeeper_lock, flags); + raw_spin_lock_irqsave(&tk_core.lock, flags); write_seqcount_begin(&tk_core.seq); suspend_timing_needed = false; @@ -1900,7 +1889,7 @@ void timekeeping_inject_sleeptime64(const struct timespec64 *delta) timekeeping_update(tk, TK_CLEAR_NTP | TK_MIRROR | TK_CLOCK_WAS_SET); write_seqcount_end(&tk_core.seq); - raw_spin_unlock_irqrestore(&timekeeper_lock, flags); + raw_spin_unlock_irqrestore(&tk_core.lock, flags); /* Signal hrtimers about time change */ clock_was_set(CLOCK_SET_WALL | CLOCK_SET_BOOT); @@ -1924,7 +1913,7 @@ void timekeeping_resume(void) clockevents_resume(); clocksource_resume(); - raw_spin_lock_irqsave(&timekeeper_lock, flags); + raw_spin_lock_irqsave(&tk_core.lock, flags); write_seqcount_begin(&tk_core.seq); /* @@ -1962,7 +1951,7 @@ void timekeeping_resume(void) timekeeping_suspended = 0; timekeeping_update(tk, TK_MIRROR | TK_CLOCK_WAS_SET); write_seqcount_end(&tk_core.seq); - raw_spin_unlock_irqrestore(&timekeeper_lock, flags); + raw_spin_unlock_irqrestore(&tk_core.lock, flags); touch_softlockup_watchdog(); @@ -1993,7 +1982,7 @@ int timekeeping_suspend(void) suspend_timing_needed = true; - raw_spin_lock_irqsave(&timekeeper_lock, flags); + raw_spin_lock_irqsave(&tk_core.lock, flags); write_seqcount_begin(&tk_core.seq); timekeeping_forward_now(tk); timekeeping_suspended = 1; @@ -2032,7 +2021,7 @@ int timekeeping_suspend(void) timekeeping_update(tk, TK_MIRROR); halt_fast_timekeeper(tk); write_seqcount_end(&tk_core.seq); - raw_spin_unlock_irqrestore(&timekeeper_lock, flags); + raw_spin_unlock_irqrestore(&tk_core.lock, flags); tick_suspend(); clocksource_suspend(); @@ -2292,7 +2281,7 @@ static bool timekeeping_advance(enum timekeeping_adv_mode mode) int shift = 0, maxshift; u64 offset; - guard(raw_spinlock_irqsave)(&timekeeper_lock); + guard(raw_spinlock_irqsave)(&tk_core.lock); /* Make sure we're fully resumed: */ if (unlikely(timekeeping_suspended)) @@ -2589,7 +2578,7 @@ int do_adjtimex(struct __kernel_timex *txc) ktime_get_real_ts64(&ts); add_device_randomness(&ts, sizeof(ts)); - raw_spin_lock_irqsave(&timekeeper_lock, flags); + raw_spin_lock_irqsave(&tk_core.lock, flags); write_seqcount_begin(&tk_core.seq); orig_tai = tai = tk->tai_offset; @@ -2604,7 +2593,7 @@ int do_adjtimex(struct __kernel_timex *txc) } write_seqcount_end(&tk_core.seq); - raw_spin_unlock_irqrestore(&timekeeper_lock, flags); + raw_spin_unlock_irqrestore(&tk_core.lock, flags); audit_ntp_log(&ad); @@ -2628,11 +2617,8 @@ int do_adjtimex(struct __kernel_timex *txc) */ void hardpps(const struct timespec64 *phase_ts, const struct timespec64 *raw_ts) { - unsigned long flags; - - raw_spin_lock_irqsave(&timekeeper_lock, flags); + guard(raw_spinlock_irqsave)(&tk_core.lock); __hardpps(phase_ts, raw_ts); - raw_spin_unlock_irqrestore(&timekeeper_lock, flags); } EXPORT_SYMBOL(hardpps); #endif /* CONFIG_NTP_PPS */ From patchwork Wed Oct 9 08:29:03 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Anna-Maria Behnsen X-Patchwork-Id: 13827743 Received: from galois.linutronix.de (Galois.linutronix.de [193.142.43.55]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 0F7BC18CC0D; Wed, 9 Oct 2024 08:29:19 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=193.142.43.55 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1728462562; cv=none; b=fG0Wup/X1VEUFn6IexfxJoiSXtRlDdcT+fjfZ0kUlfe3PMUNd0F0SYODd3wFyTbPVpACQx1ZyB9ziA9C+Jgb0c6pjENpSR/cKixWhu2W/AYL+Qzbo1C04GXf8YQ+1D/gz3g/NKBU0VqVXT6QH7TNZkNMOJGokdPFmBYbITh2DdM= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1728462562; c=relaxed/simple; bh=VmpacbXvFICUwe6HFWWyFXiddgpQsONmFUPY0mpNvc4=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=QTtc7Vm2A4W8k0i7GrLtvFlSAnSrUJfA1RKWe/UNqju7025WIlRDeZosV24TT3lFm6vWbQO3G7R2UAY5AbWSz5XAFNt9uqHlqLA2ezeOaA9rsKRJqZJdAn/YA28uVRBavaD2LCOL7QB9BacRuQp+37wo7W66c+8Lgu3Hw2BaqkI= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linutronix.de; spf=pass smtp.mailfrom=linutronix.de; dkim=pass (2048-bit key) header.d=linutronix.de header.i=@linutronix.de header.b=CsQiryeB; dkim=permerror (0-bit key) header.d=linutronix.de header.i=@linutronix.de header.b=YUihAkcy; arc=none smtp.client-ip=193.142.43.55 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linutronix.de Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linutronix.de Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=linutronix.de header.i=@linutronix.de header.b="CsQiryeB"; dkim=permerror (0-bit key) header.d=linutronix.de header.i=@linutronix.de header.b="YUihAkcy" From: Anna-Maria Behnsen DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linutronix.de; s=2020; t=1728462557; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=63USMTPRy4krudbQE7I3ZBsVQpkprmvzSEp3iWRhZlU=; b=CsQiryeBLJsNBz2jKFmcxvrfWox/mTlGnCvdLq4bvsn7YFFM9f03Yr8UaikBE0Rt0as8Vn tH+T7BvubJEgetj1SlM5tsR44ei68KK+2tMMGyeBJhKvnSTKylLyD9+OXjTw6QsqYuQO7h 2qW5RmXsd4aJwtkk1XGZubFV2g1otzsdIDbcqDGiPat6sJzjBnG1jYrdpyyZRi7k7Ik8sZ yLsEVhkhlSAUp5TWQPA5etJpP3Z/mfQpevVa9c2d1vkh9pkPLuhqQWqQ8kr/m3ab9zD2aj oLwWv84RjqpjPZdZAShRO6ZGedJCaiteZ6VK9imlvwDnD0To6lOv2KBemCS9/g== DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=linutronix.de; s=2020e; t=1728462557; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=63USMTPRy4krudbQE7I3ZBsVQpkprmvzSEp3iWRhZlU=; b=YUihAkcyZMaZghMDfclMsOmqBcSKiCPtyO+8XZLxjKW3sOI1yYvmzXbVBRuhBz70wcjnrm QvkZ+eD0AyT9K9BA== Date: Wed, 09 Oct 2024 10:29:03 +0200 Subject: [PATCH v2 10/25] timekeeping: Define a struct type for tk_core to make it reusable Precedence: bulk X-Mailing-List: netdev@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Message-Id: <20241009-devel-anna-maria-b4-timers-ptp-timekeeping-v2-10-554456a44a15@linutronix.de> References: <20241009-devel-anna-maria-b4-timers-ptp-timekeeping-v2-0-554456a44a15@linutronix.de> In-Reply-To: <20241009-devel-anna-maria-b4-timers-ptp-timekeeping-v2-0-554456a44a15@linutronix.de> To: John Stultz , Frederic Weisbecker , Thomas Gleixner Cc: linux-kernel@vger.kernel.org, netdev@vger.kernel.org, Miroslav Lichvar , Richard Cochran , Christopher S Hall , Anna-Maria Behnsen From: Anna-Maria Behnsen The struct tk_core uses is not reusable. As long as there is only a single timekeeper, this is not a problem. But when the timekeeper infrastructure will be reused for per ptp clock timekeepers, an explicit struct type is required. Define struct tk_data as explicit struct type for tk_core. Signed-off-by: Anna-Maria Behnsen Acked-by: John Stultz --- kernel/time/timekeeping.c | 6 ++++-- 1 file changed, 4 insertions(+), 2 deletions(-) diff --git a/kernel/time/timekeeping.c b/kernel/time/timekeeping.c index 8557d32e5e3d..40c60bb88416 100644 --- a/kernel/time/timekeeping.c +++ b/kernel/time/timekeeping.c @@ -45,12 +45,14 @@ enum timekeeping_adv_mode { * The most important data for readout fits into a single 64 byte * cache line. */ -static struct { +struct tk_data { seqcount_raw_spinlock_t seq; struct timekeeper timekeeper; struct timekeeper shadow_timekeeper; raw_spinlock_t lock; -} tk_core ____cacheline_aligned; +} ____cacheline_aligned; + +static struct tk_data tk_core; /* flag for if timekeeping is suspended */ int __read_mostly timekeeping_suspended; From patchwork Wed Oct 9 08:29:04 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Anna-Maria Behnsen X-Patchwork-Id: 13827745 Received: from galois.linutronix.de (Galois.linutronix.de [193.142.43.55]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 4DA0318D643; Wed, 9 Oct 2024 08:29:20 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=193.142.43.55 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1728462562; cv=none; b=Zm4E/Lr+rG6F6xDW7wbx/Dj6OMOiWmME+NWM9f6mRMv1Qgp8IGuxB8tjA/2UcSyzq0NiAwb4N3Kv77mE0uHpV1ssMtpX41LcyWJYynA/E/8IH88Q3gCpE/1DAMFL2Hem/xzPMMally+/YXHQhT8xvfzn1IZyaH6dcD0XrjyAq4U= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1728462562; c=relaxed/simple; bh=3OUvtUjQqnQDfYli5HJIk26F6ond8cHnb8VAZNno3Hs=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=MVfcWv56K3QS9HRrO477jglE63B+h+mssM7j2Cer44plDol8qtftj9LPAlz9QqXfeMXxfXmzaJtDHU0LGFAMOx4xEMgLG+XIo8fL6YIFN48IgpwF4s1hO8q/DW7tcy41GK+YvXnJUdBjanR1ODxyVU3nKlWJV14j+Pw8vP1C4jg= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linutronix.de; spf=pass smtp.mailfrom=linutronix.de; dkim=pass (2048-bit key) header.d=linutronix.de header.i=@linutronix.de header.b=Gr75EYUI; dkim=permerror (0-bit key) header.d=linutronix.de header.i=@linutronix.de header.b=vi3oOdd+; arc=none smtp.client-ip=193.142.43.55 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linutronix.de Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linutronix.de Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=linutronix.de header.i=@linutronix.de header.b="Gr75EYUI"; dkim=permerror (0-bit key) header.d=linutronix.de header.i=@linutronix.de header.b="vi3oOdd+" From: Anna-Maria Behnsen DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linutronix.de; s=2020; t=1728462557; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=xLAnHBvTjt0WPGqjgr+Qtf2CpGbFQtqVrOAWERdILJk=; b=Gr75EYUIRfSAdOMM6xfthBAc7wxWG8pRUV9gAYL/MfG7W6zbHKgHUWpjBkdhipbuOR3wgd /aed6k3rhbxPipSb5beClkP6AJojaNmY0JKB8MkGKwIIaTNui2+jXbjp702TVlIEB4ESDY FG6Qb6+kvpitOKwq5dZnunMcjCr89csZpr80DDAbguhYD9a64rs79p0hBNzinQubj0n9j7 VwpC4OqTnCmkWYt8Rl4r8HQeUS1asF5Y3VaquF/XDkgNt0qvlsa2tfKhsiHM9IZHXpVlvi lWx/1s/mBWZlzULTzmZzDGBKl5imSs3s3Ke/mufPHs+/ct67VVtWlw5QKZi1rg== DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=linutronix.de; s=2020e; t=1728462557; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=xLAnHBvTjt0WPGqjgr+Qtf2CpGbFQtqVrOAWERdILJk=; b=vi3oOdd+CkD4lH2yijlP8toPAAtcFBsJu8SPmKKhNdQ4pfXTq0ur9DRWsAvxfHgcYvfQkG kQNKqWV7Fqql8ICg== Date: Wed, 09 Oct 2024 10:29:04 +0200 Subject: [PATCH v2 11/25] timekeeping: Introduce tkd_basic_setup() to make lock and seqcount init reusable Precedence: bulk X-Mailing-List: netdev@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Message-Id: <20241009-devel-anna-maria-b4-timers-ptp-timekeeping-v2-11-554456a44a15@linutronix.de> References: <20241009-devel-anna-maria-b4-timers-ptp-timekeeping-v2-0-554456a44a15@linutronix.de> In-Reply-To: <20241009-devel-anna-maria-b4-timers-ptp-timekeeping-v2-0-554456a44a15@linutronix.de> To: John Stultz , Frederic Weisbecker , Thomas Gleixner Cc: linux-kernel@vger.kernel.org, netdev@vger.kernel.org, Miroslav Lichvar , Richard Cochran , Christopher S Hall , Anna-Maria Behnsen Initialization of lock and seqcount needs to be done for every instance of timekeeper struct. To be able to easily reuse it, create a separate function for it. Signed-off-by: Anna-Maria Behnsen Acked-by: John Stultz --- v2: new in v2 - splitted from another patch --- kernel/time/timekeeping.c | 9 +++++++-- 1 file changed, 7 insertions(+), 2 deletions(-) diff --git a/kernel/time/timekeeping.c b/kernel/time/timekeeping.c index 40c60bb88416..a98f823be6db 100644 --- a/kernel/time/timekeeping.c +++ b/kernel/time/timekeeping.c @@ -1734,6 +1734,12 @@ read_persistent_wall_and_boot_offset(struct timespec64 *wall_time, *boot_offset = ns_to_timespec64(local_clock()); } +static __init void tkd_basic_setup(struct tk_data *tkd) +{ + raw_spin_lock_init(&tkd->lock); + seqcount_raw_spinlock_init(&tkd->seq, &tkd->lock); +} + /* * Flag reflecting whether timekeeping_resume() has injected sleeptime. * @@ -1761,8 +1767,7 @@ void __init timekeeping_init(void) struct timekeeper *tk = &tk_core.timekeeper; struct clocksource *clock; - raw_spin_lock_init(&tk_core.lock); - seqcount_raw_spinlock_init(&tk_core.seq, &tkd->lock); + tkd_basic_setup(&tk_core); read_persistent_wall_and_boot_offset(&wall_time, &boot_offset); if (timespec64_valid_settod(&wall_time) && From patchwork Wed Oct 9 08:29:05 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Anna-Maria Behnsen X-Patchwork-Id: 13827746 Received: from galois.linutronix.de (Galois.linutronix.de [193.142.43.55]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 4DABC18D645; Wed, 9 Oct 2024 08:29:20 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=193.142.43.55 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1728462562; cv=none; b=pRUVQN9KERpcUVhpkD3IBhpbnoZ4bn5gqmD7ogJcshQfSlD2BPeur/+jRhHMYp7MrfCwfGw4l0mErEhts7mUz2OmsJD/fyzxQxiV0p4uQ6CjNEMOlfaoqxtONmAQLBJ67XKTt4yq6c2yPag1mpRcrrZKzyw4jstZyzgAOnVgGf4= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1728462562; c=relaxed/simple; bh=obVFLu8V+n6TI61DhdW8w4lee9Lzx6rSm9FQxifHM5c=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=m0VjY8vaQqc3rg7R96FTPJzSO7ocVGOpankqE7QTgZ7J72sA1Wg70j6uGP8k+XAN5lAQYerOqw4IgpjoTNt4LeORMy1ipCI44ScAWxwfqLhBcHq0vv2al4fB88Tl66epcHxEwbjUaHn8Ue4osbIWhAPr1kfv33+I7eDu6F/R/lA= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linutronix.de; spf=pass smtp.mailfrom=linutronix.de; dkim=pass (2048-bit key) header.d=linutronix.de header.i=@linutronix.de header.b=FKGVTMeT; dkim=permerror (0-bit key) header.d=linutronix.de header.i=@linutronix.de header.b=grGEs+Q5; arc=none smtp.client-ip=193.142.43.55 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linutronix.de Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linutronix.de Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=linutronix.de header.i=@linutronix.de header.b="FKGVTMeT"; dkim=permerror (0-bit key) header.d=linutronix.de header.i=@linutronix.de header.b="grGEs+Q5" From: Anna-Maria Behnsen DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linutronix.de; s=2020; t=1728462558; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=pxYJUNMlmiuyRG4rAKJCLjd4R1BXiz2qhdWO0TNO0rk=; b=FKGVTMeT/JWtH423tps92Cton1G4pd/c6eUjer+kSnbprvaQgM9dVElUAhzRVAeG9xHxrX ieAUZ1QTftuyEVf9p/Cgc5FyezFsJpktjW0bLFE5h/XY5Bs/y+k4ep4C/6zlFV2pU/wRsj zPYtzmOJ/E6yQTRN6/pljN3NC7DbVzpW4ZwTCPOJK8NTmQJWqPwiJLc+h+an+VO5XC6TEg lbBiy7fJke15z1PlJxVR7Ss/ewjyEtBDzreCj6+i5SruyP9Lz4m+5Ft9ZCXE9djoqOc+Y3 iP02wH7P5rQ/VEoarvjn/RWxfkRNCwzg3dHsEie4JhCGmSuBBajBKwT75XpDfA== DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=linutronix.de; s=2020e; t=1728462558; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=pxYJUNMlmiuyRG4rAKJCLjd4R1BXiz2qhdWO0TNO0rk=; b=grGEs+Q5KR6ZEXc7gdjJIzu2sd6zgSeIuzxzffm9yyq5qlhcP9tMLomoHjLN/i0xeTyvlD +yq3Tld6EccN4FDg== Date: Wed, 09 Oct 2024 10:29:05 +0200 Subject: [PATCH v2 12/25] timekeeping: Add struct tk_data as argument to timekeeping_update() Precedence: bulk X-Mailing-List: netdev@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Message-Id: <20241009-devel-anna-maria-b4-timers-ptp-timekeeping-v2-12-554456a44a15@linutronix.de> References: <20241009-devel-anna-maria-b4-timers-ptp-timekeeping-v2-0-554456a44a15@linutronix.de> In-Reply-To: <20241009-devel-anna-maria-b4-timers-ptp-timekeeping-v2-0-554456a44a15@linutronix.de> To: John Stultz , Frederic Weisbecker , Thomas Gleixner Cc: linux-kernel@vger.kernel.org, netdev@vger.kernel.org, Miroslav Lichvar , Richard Cochran , Christopher S Hall , Anna-Maria Behnsen From: Anna-Maria Behnsen Updates of the timekeeper are done in two ways: 1. Updating timekeeper and afterwards memcpy()'ing the result into shadow_timekeeper using timekeeping_update(). Used everywhere for updates except in timekeeping_advance(); the sequence counter protected region starts before the first change to the timekeeper is done. 2. Updating shadow_timekeeper and then memcpy()'ing the result into timekeeper. Used only by in timekeeping_advance(); The seqence counter protected region is only around timekeeping_update() and the memcpy for copy from shadow to timekeeper. The second option is fast path optimized. The sequence counter protected region is as short as possible. As this behaviour is mainly documented by commit messages, but not in code, it makes the not easy timekeeping code more complicated to read. There is no reason why updates to the timekeeper can't use the optimized version everywhere. With this, the code will be cleaner, as code is reused instead of duplicated. To be able to access tk_data which contains all required information, add a pointer to tk_data as an argument to timekeeping_update(). With that convert the comment about holding the lock into a lockdep assert. No functional change. Signed-off-by: Anna-Maria Behnsen Acked-by: John Stultz --- kernel/time/timekeeping.c | 25 +++++++++++++------------ 1 file changed, 13 insertions(+), 12 deletions(-) diff --git a/kernel/time/timekeeping.c b/kernel/time/timekeeping.c index a98f823be6db..878f9606946d 100644 --- a/kernel/time/timekeeping.c +++ b/kernel/time/timekeeping.c @@ -528,7 +528,7 @@ EXPORT_SYMBOL_GPL(ktime_get_raw_fast_ns); * timekeeping_inject_sleeptime64() * __timekeeping_inject_sleeptime(tk, delta); * timestamp(); - * timekeeping_update(tk, TK_CLEAR_NTP...); + * timekeeping_update(tkd, tk, TK_CLEAR_NTP...); * * (2) On 32-bit systems, the 64-bit boot offset (tk->offs_boot) may be * partially updated. Since the tk->offs_boot update is a rare event, this @@ -753,9 +753,10 @@ static inline void tk_update_ktime_data(struct timekeeper *tk) tk->tkr_raw.base = ns_to_ktime(tk->raw_sec * NSEC_PER_SEC); } -/* must hold tk_core.lock */ -static void timekeeping_update(struct timekeeper *tk, unsigned int action) +static void timekeeping_update(struct tk_data *tkd, struct timekeeper *tk, unsigned int action) { + lockdep_assert_held(&tkd->lock); + if (action & TK_CLEAR_NTP) { tk->ntp_error = 0; ntp_clear(); @@ -1467,7 +1468,7 @@ int do_settimeofday64(const struct timespec64 *ts) tk_set_xtime(tk, ts); out: - timekeeping_update(tk, TK_CLEAR_NTP | TK_MIRROR | TK_CLOCK_WAS_SET); + timekeeping_update(&tk_core, tk, TK_CLEAR_NTP | TK_MIRROR | TK_CLOCK_WAS_SET); write_seqcount_end(&tk_core.seq); raw_spin_unlock_irqrestore(&tk_core.lock, flags); @@ -1517,7 +1518,7 @@ static int timekeeping_inject_offset(const struct timespec64 *ts) tk_set_wall_to_mono(tk, timespec64_sub(tk->wall_to_monotonic, *ts)); error: /* even if we error out, we forwarded the time, so call update */ - timekeeping_update(tk, TK_CLEAR_NTP | TK_MIRROR | TK_CLOCK_WAS_SET); + timekeeping_update(&tk_core, tk, TK_CLEAR_NTP | TK_MIRROR | TK_CLOCK_WAS_SET); write_seqcount_end(&tk_core.seq); raw_spin_unlock_irqrestore(&tk_core.lock, flags); @@ -1602,7 +1603,7 @@ static int change_clocksource(void *data) timekeeping_forward_now(tk); old = tk->tkr_mono.clock; tk_setup_internals(tk, new); - timekeeping_update(tk, TK_CLEAR_NTP | TK_MIRROR | TK_CLOCK_WAS_SET); + timekeeping_update(&tk_core, tk, TK_CLEAR_NTP | TK_MIRROR | TK_CLOCK_WAS_SET); write_seqcount_end(&tk_core.seq); raw_spin_unlock_irqrestore(&tk_core.lock, flags); @@ -1801,7 +1802,7 @@ void __init timekeeping_init(void) tk_set_wall_to_mono(tk, wall_to_mono); - timekeeping_update(tk, TK_MIRROR | TK_CLOCK_WAS_SET); + timekeeping_update(&tk_core, tk, TK_MIRROR | TK_CLOCK_WAS_SET); write_seqcount_end(&tk_core.seq); } @@ -1893,7 +1894,7 @@ void timekeeping_inject_sleeptime64(const struct timespec64 *delta) __timekeeping_inject_sleeptime(tk, delta); - timekeeping_update(tk, TK_CLEAR_NTP | TK_MIRROR | TK_CLOCK_WAS_SET); + timekeeping_update(&tk_core, tk, TK_CLEAR_NTP | TK_MIRROR | TK_CLOCK_WAS_SET); write_seqcount_end(&tk_core.seq); raw_spin_unlock_irqrestore(&tk_core.lock, flags); @@ -1956,7 +1957,7 @@ void timekeeping_resume(void) tk->ntp_error = 0; timekeeping_suspended = 0; - timekeeping_update(tk, TK_MIRROR | TK_CLOCK_WAS_SET); + timekeeping_update(&tk_core, tk, TK_MIRROR | TK_CLOCK_WAS_SET); write_seqcount_end(&tk_core.seq); raw_spin_unlock_irqrestore(&tk_core.lock, flags); @@ -2025,7 +2026,7 @@ int timekeeping_suspend(void) } } - timekeeping_update(tk, TK_MIRROR); + timekeeping_update(&tk_core, tk, TK_MIRROR); halt_fast_timekeeper(tk); write_seqcount_end(&tk_core.seq); raw_spin_unlock_irqrestore(&tk_core.lock, flags); @@ -2343,7 +2344,7 @@ static bool timekeeping_advance(enum timekeeping_adv_mode mode) * memcpy under the tk_core.seq against one before we start * updating. */ - timekeeping_update(tk, clock_set); + timekeeping_update(&tk_core, tk, clock_set); memcpy(real_tk, tk, sizeof(*tk)); /* The memcpy must come last. Do not put anything here! */ write_seqcount_end(&tk_core.seq); @@ -2593,7 +2594,7 @@ int do_adjtimex(struct __kernel_timex *txc) if (tai != orig_tai) { __timekeeping_set_tai_offset(tk, tai); - timekeeping_update(tk, TK_MIRROR | TK_CLOCK_WAS_SET); + timekeeping_update(&tk_core, tk, TK_MIRROR | TK_CLOCK_WAS_SET); clock_set = true; } else { tk_update_leap_state(tk); From patchwork Wed Oct 9 08:29:06 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Anna-Maria Behnsen X-Patchwork-Id: 13827749 Received: from galois.linutronix.de (Galois.linutronix.de [193.142.43.55]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 9348318DF68; Wed, 9 Oct 2024 08:29:20 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=193.142.43.55 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1728462563; cv=none; b=sluKOrwDyZLosmsplEwHqygqnKljYeVTeuw2KaXfcS6g/ZxPT5uNNAcqBY0hy8uuKr52wT+k97S/bRLmunBjXuBCuu9STHJ9nEw/NhPCSUUMcjYKbPzAo83VMEPVN4X/I0XbnIrnr04K9GB3xETZspfTgZMgTU7/HJbcEbki5X4= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1728462563; c=relaxed/simple; bh=XgL8cfM/IWJXb5ZekypyN3VDBOsxqkLZwkvFEHABaYY=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=XJpNE+kM/4bppETQVMmexSq+DDwR7fVjm954/dNARibBQU9507twsimVpW1/ciLRuWzzLKlBMmY43kQFYdl902G5cC0cq4O9CSEfoZd835VxCPlUiWkB6ahcJtKP5gOx5GchRl4GWJWAZHDbWAbFl8WkUk03rM+kCHPQxZLAM+w= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linutronix.de; spf=pass smtp.mailfrom=linutronix.de; dkim=pass (2048-bit key) header.d=linutronix.de header.i=@linutronix.de header.b=R1yQIBYc; dkim=permerror (0-bit key) header.d=linutronix.de header.i=@linutronix.de header.b=SzauA7G8; arc=none smtp.client-ip=193.142.43.55 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linutronix.de Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linutronix.de Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=linutronix.de header.i=@linutronix.de header.b="R1yQIBYc"; dkim=permerror (0-bit key) header.d=linutronix.de header.i=@linutronix.de header.b="SzauA7G8" From: Anna-Maria Behnsen DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linutronix.de; s=2020; t=1728462558; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=DT27laaYnC7O8oK5GmcQbcmPPnUUBCGdfOGBs/sIYMk=; b=R1yQIBYcbfDDAyar1DjECFO2S+WteoEXatodGI1OS26Zo5Sqj9g8IQbHAGWuZyh7XS7j1e gKGOgQ+w5Eizb4Fui1Dg3PR8xpcXzDp4x7HZ76PSLFES9yXkxxbqNnXiuCDTWTpYszueTe 4aWFNafU4HIMknVk3gJHX6nvYBR3IvGYQoEINh0Nw2t/xunw4W9v1m9+sTSOt+QhcD2Gf4 WCyvAOujP8gXHdPz9nrlR5LDzcceSf/MgJ3LMirg27NhvCuWOMou9zLAD2SzJWECUbR/5j yFnYBM6Q7YlJqTucVSitCkCXYDCULQW6L8NF4xUujp+d3r8Jj9K+cvKZXluuOw== DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=linutronix.de; s=2020e; t=1728462558; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=DT27laaYnC7O8oK5GmcQbcmPPnUUBCGdfOGBs/sIYMk=; b=SzauA7G8H1YFB+Es4FrwX+NDz0ceFKpUkDy4MgwtMutxcVP1aYuB/XhiAne9hKd4oTmyye cMvkbo3pDrzL7NCA== Date: Wed, 09 Oct 2024 10:29:06 +0200 Subject: [PATCH v2 13/25] timekeeping: Split out timekeeper update of timekeeping_advanced() Precedence: bulk X-Mailing-List: netdev@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Message-Id: <20241009-devel-anna-maria-b4-timers-ptp-timekeeping-v2-13-554456a44a15@linutronix.de> References: <20241009-devel-anna-maria-b4-timers-ptp-timekeeping-v2-0-554456a44a15@linutronix.de> In-Reply-To: <20241009-devel-anna-maria-b4-timers-ptp-timekeeping-v2-0-554456a44a15@linutronix.de> To: John Stultz , Frederic Weisbecker , Thomas Gleixner Cc: linux-kernel@vger.kernel.org, netdev@vger.kernel.org, Miroslav Lichvar , Richard Cochran , Christopher S Hall , Anna-Maria Behnsen From: Anna-Maria Behnsen timekeeping_advance() is the only optimized function which uses shadow_timekeeper for updating the real timekeeper to keep the sequence counter protected region as small as possible. To be able to transform timekeeper updates in other functions to use the same logic, split out functionality into a separate function timekeeper_update_staged(). While at it, document the reason why the sequence counter must be write held over the call to timekeeping_update() and the copying to the real timekeeper and why using a pointer based update is suboptimal. No functional change. Signed-off-by: Anna-Maria Behnsen Acked-by: John Stultz --- kernel/time/timekeeping.c | 43 +++++++++++++++++++++++++++---------------- 1 file changed, 27 insertions(+), 16 deletions(-) diff --git a/kernel/time/timekeeping.c b/kernel/time/timekeeping.c index 878f9606946d..fcb2b8b232d2 100644 --- a/kernel/time/timekeeping.c +++ b/kernel/time/timekeeping.c @@ -780,7 +780,32 @@ static void timekeeping_update(struct tk_data *tkd, struct timekeeper *tk, unsig * timekeeper structure on the next update with stale data */ if (action & TK_MIRROR) - memcpy(&tk_core.shadow_timekeeper, &tk_core.timekeeper, sizeof(tk_core.timekeeper)); + memcpy(&tkd->shadow_timekeeper, tk, sizeof(*tk)); +} + +static void timekeeping_update_staged(struct tk_data *tkd, unsigned int action) +{ + /* + * Block out readers before invoking timekeeping_update() because + * that updates VDSO and other time related infrastructure. Not + * blocking the readers might let a reader see time going backwards + * when reading from the VDSO after the VDSO update and then + * reading in the kernel from the timekeeper before that got updated. + */ + write_seqcount_begin(&tkd->seq); + + timekeeping_update(tkd, &tkd->shadow_timekeeper, action); + + /* + * Update the real timekeeper. + * + * We could avoid this memcpy() by switching pointers, but that has + * the downside that the reader side does not longer benefit from + * the cacheline optimized data layout of the timekeeper and requires + * another indirection. + */ + memcpy(&tkd->timekeeper, &tkd->shadow_timekeeper, sizeof(tkd->shadow_timekeeper)); + write_seqcount_end(&tkd->seq); } /** @@ -2333,21 +2358,7 @@ static bool timekeeping_advance(enum timekeeping_adv_mode mode) */ clock_set |= accumulate_nsecs_to_secs(tk); - write_seqcount_begin(&tk_core.seq); - /* - * Update the real timekeeper. - * - * We could avoid this memcpy by switching pointers, but that - * requires changes to all other timekeeper usage sites as - * well, i.e. move the timekeeper pointer getter into the - * spinlocked/seqcount protected sections. And we trade this - * memcpy under the tk_core.seq against one before we start - * updating. - */ - timekeeping_update(&tk_core, tk, clock_set); - memcpy(real_tk, tk, sizeof(*tk)); - /* The memcpy must come last. Do not put anything here! */ - write_seqcount_end(&tk_core.seq); + timekeeping_update_staged(&tk_core, clock_set); return !!clock_set; } From patchwork Wed Oct 9 08:29:07 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Anna-Maria Behnsen X-Patchwork-Id: 13827747 Received: from galois.linutronix.de (Galois.linutronix.de [193.142.43.55]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 8A4CC18DF64; Wed, 9 Oct 2024 08:29:20 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=193.142.43.55 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1728462562; cv=none; b=qwV/5U4uarxfq7MIr37xzGbi4BdmWSYc2yqA6deouEwXHEUWIlxRanYiM9TddWx9eCNbXrKDLDSdFAQcaiRzD6zAkfJv7NC07XSj7vZJoJMBqJrHAen/oVB3j7wkeAownhvh1jvAPNp8GpocUioGhCLsYPiyXjkBwGHWFll3YuM= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1728462562; c=relaxed/simple; bh=bRWBXGzdASFmvDLqh3OdneAcymX6b/eaXSsanouUKn8=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=GzrcO/i7SGVlXzEb01uBB2wWtOONGTMThPP3RPyHyuXj6zvhWeGA/WXPB2aW8P4X4eBSja3ESZvQTCMLMrKOT+D5DuTcuuNXV6/Eee9BlEyoo/3xjcqsbatYGH5TKeujLthYgBKeOG4qvqh8C83aeGNDVNaOhtePAcD1yCZlopQ= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linutronix.de; spf=pass smtp.mailfrom=linutronix.de; dkim=pass (2048-bit key) header.d=linutronix.de header.i=@linutronix.de header.b=MFnrhvZ4; dkim=permerror (0-bit key) header.d=linutronix.de header.i=@linutronix.de header.b=DuK3xOtn; arc=none smtp.client-ip=193.142.43.55 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linutronix.de Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linutronix.de Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=linutronix.de header.i=@linutronix.de header.b="MFnrhvZ4"; dkim=permerror (0-bit key) header.d=linutronix.de header.i=@linutronix.de header.b="DuK3xOtn" From: Anna-Maria Behnsen DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linutronix.de; s=2020; t=1728462558; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=T8iNDmVaqSsH9dpFWjGVt85012RSAhD8fVD7Om8YFQs=; b=MFnrhvZ4Sh5ipH9HHN/wAMmkfu8uoX0L0vO8sNogaYRkc9IXDkxUqifPjxXRgevdnXGnsy ztpR0syIwqSTZYsQyMnrNRjhB67q9eyeSzfdheEWnZzwE7S3px9dMUpWjWGbDfa93Hm5d2 3SpxYzvE2xVtvXMDsL0UoPMbP9rl89OF3cBk4BapHjd534Unc++FCAmGfFe9zBsn8kaVwC UkogcoTMmBlmDr9lb1v+O09FtV/NX6VBYJI2ddV/mGtJw9E3jqQgoFahaMRpCM3ggmP76h AIcDFiRavTORJ8k4GBorq7uTC0yfCrzD/HS+pedvOfPptYA+OCCr0YTK18NI5A== DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=linutronix.de; s=2020e; t=1728462558; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=T8iNDmVaqSsH9dpFWjGVt85012RSAhD8fVD7Om8YFQs=; b=DuK3xOtnPddfIE1Zqtm2ymatOqqB3LvijIF91YUNhVwt1ecebfrM7Vc8FVXDdcz088FPuO B2QQfg+KDXKmmRBQ== Date: Wed, 09 Oct 2024 10:29:07 +0200 Subject: [PATCH v2 14/25] timekeeping: Introduce combined timekeeping action flag Precedence: bulk X-Mailing-List: netdev@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Message-Id: <20241009-devel-anna-maria-b4-timers-ptp-timekeeping-v2-14-554456a44a15@linutronix.de> References: <20241009-devel-anna-maria-b4-timers-ptp-timekeeping-v2-0-554456a44a15@linutronix.de> In-Reply-To: <20241009-devel-anna-maria-b4-timers-ptp-timekeeping-v2-0-554456a44a15@linutronix.de> To: John Stultz , Frederic Weisbecker , Thomas Gleixner Cc: linux-kernel@vger.kernel.org, netdev@vger.kernel.org, Miroslav Lichvar , Richard Cochran , Christopher S Hall , Anna-Maria Behnsen Instead of explicitly listing all the separate timekeeping actions flags, introduce a new one which covers all actions except TK_MIRROR action. No functional change. Signed-off-by: Anna-Maria Behnsen --- kernel/time/timekeeping.c | 10 ++++++---- 1 file changed, 6 insertions(+), 4 deletions(-) diff --git a/kernel/time/timekeeping.c b/kernel/time/timekeeping.c index fcb2b8b232d2..5a747afe64b4 100644 --- a/kernel/time/timekeeping.c +++ b/kernel/time/timekeeping.c @@ -33,6 +33,8 @@ #define TK_MIRROR (1 << 1) #define TK_CLOCK_WAS_SET (1 << 2) +#define TK_UPDATE_ALL (TK_CLEAR_NTP | TK_CLOCK_WAS_SET) + enum timekeeping_adv_mode { /* Update timekeeper when a tick has passed */ TK_ADV_TICK, @@ -1493,7 +1495,7 @@ int do_settimeofday64(const struct timespec64 *ts) tk_set_xtime(tk, ts); out: - timekeeping_update(&tk_core, tk, TK_CLEAR_NTP | TK_MIRROR | TK_CLOCK_WAS_SET); + timekeeping_update(&tk_core, tk, TK_UPDATE_ALL | TK_MIRROR); write_seqcount_end(&tk_core.seq); raw_spin_unlock_irqrestore(&tk_core.lock, flags); @@ -1543,7 +1545,7 @@ static int timekeeping_inject_offset(const struct timespec64 *ts) tk_set_wall_to_mono(tk, timespec64_sub(tk->wall_to_monotonic, *ts)); error: /* even if we error out, we forwarded the time, so call update */ - timekeeping_update(&tk_core, tk, TK_CLEAR_NTP | TK_MIRROR | TK_CLOCK_WAS_SET); + timekeeping_update(&tk_core, tk, TK_UPDATE_ALL | TK_MIRROR); write_seqcount_end(&tk_core.seq); raw_spin_unlock_irqrestore(&tk_core.lock, flags); @@ -1628,7 +1630,7 @@ static int change_clocksource(void *data) timekeeping_forward_now(tk); old = tk->tkr_mono.clock; tk_setup_internals(tk, new); - timekeeping_update(&tk_core, tk, TK_CLEAR_NTP | TK_MIRROR | TK_CLOCK_WAS_SET); + timekeeping_update(&tk_core, tk, TK_UPDATE_ALL | TK_MIRROR); write_seqcount_end(&tk_core.seq); raw_spin_unlock_irqrestore(&tk_core.lock, flags); @@ -1919,7 +1921,7 @@ void timekeeping_inject_sleeptime64(const struct timespec64 *delta) __timekeeping_inject_sleeptime(tk, delta); - timekeeping_update(&tk_core, tk, TK_CLEAR_NTP | TK_MIRROR | TK_CLOCK_WAS_SET); + timekeeping_update(&tk_core, tk, TK_UPDATE_ALL | TK_MIRROR); write_seqcount_end(&tk_core.seq); raw_spin_unlock_irqrestore(&tk_core.lock, flags); From patchwork Wed Oct 9 08:29:08 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Anna-Maria Behnsen X-Patchwork-Id: 13827752 Received: from galois.linutronix.de (Galois.linutronix.de [193.142.43.55]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 26E6018EFEE; Wed, 9 Oct 2024 08:29:22 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=193.142.43.55 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1728462565; cv=none; b=hhyQBHn4VUHY2Mt00SvbFtUbyDe5gDSelxeAKO342W/cUcBiErwtwqplbZCe1fLDv2xDbS+5TfM/LKsY1UF5eYmUb9uYZHcp7paBYvGTl8um+fCKNcksSd8qqNqkyAWz8S0bjIbAf1jKpANWGCAXiHA2WQWo0oBRCjpTkskVrXU= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1728462565; c=relaxed/simple; bh=XoVZht3tq9m+uvk+V3rIk/CgzvWibAz49VRlt28j31k=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=oleKk4uigzqTDjjzPPN5Hgdq72vr/RxKC5XiwXviAEhsl5xgRLZ/+2LG4JlYU+9mhvWTM+IpBBWydxGnT47DpiErZ/MpBwgj44jTjw83F833fUlu6YV+HvQjDp92xPlo3ECDqhmcwqpZADSlxjIpKPaRDUXH+5epwYq1OLv8zvw= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linutronix.de; spf=pass smtp.mailfrom=linutronix.de; dkim=pass (2048-bit key) header.d=linutronix.de header.i=@linutronix.de header.b=BbFCoVzy; dkim=permerror (0-bit key) header.d=linutronix.de header.i=@linutronix.de header.b=GxM2Z8Xy; arc=none smtp.client-ip=193.142.43.55 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linutronix.de Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linutronix.de Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=linutronix.de header.i=@linutronix.de header.b="BbFCoVzy"; dkim=permerror (0-bit key) header.d=linutronix.de header.i=@linutronix.de header.b="GxM2Z8Xy" From: Anna-Maria Behnsen DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linutronix.de; s=2020; t=1728462559; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=8IqW3zmJXjELGAb9g+xJ2OjtrPdNRM2dHq+mEGEowas=; b=BbFCoVzyV/TqIrjEIvLd6bnJyqYUJmfWc8s/wtVTFPUrqdW6iLbkj44VATGbsRYjXuhGbE WKJedPjPoz+gLIUbMQMo4egwaa36rv/cy7VS9e/vBXFxqt3nqnVxEldCxizqJLYBfMCJpH IC4PMSYSLIPJTGakNnG3jWLMWajoOE83xgel+pHXa2wgvkXay6UNsVEv7+bCC8eWAEYUpq FNkGaEu3hAGOa6qxUvzii/Sac/UCHWvdlM73o8zMphoEG5EtiRCh6cgS9oV2drjhfZvlAU qaqNHs+sx2oa091ETME/m2jYWl0+auRXks8y1CLpNeC8E4PPJg7ZjobxjFfkzA== DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=linutronix.de; s=2020e; t=1728462559; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=8IqW3zmJXjELGAb9g+xJ2OjtrPdNRM2dHq+mEGEowas=; b=GxM2Z8XyhHzI7/Dxom3uy6qboysuzOYBEYBt6EbmLC2VvdgOZ/sHrGPvgHJAD3BBkzA7uB cWyDPseFg/9TQoDg== Date: Wed, 09 Oct 2024 10:29:08 +0200 Subject: [PATCH v2 15/25] timekeeping: Provide timekeeping_restore_shadow() Precedence: bulk X-Mailing-List: netdev@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Message-Id: <20241009-devel-anna-maria-b4-timers-ptp-timekeeping-v2-15-554456a44a15@linutronix.de> References: <20241009-devel-anna-maria-b4-timers-ptp-timekeeping-v2-0-554456a44a15@linutronix.de> In-Reply-To: <20241009-devel-anna-maria-b4-timers-ptp-timekeeping-v2-0-554456a44a15@linutronix.de> To: John Stultz , Frederic Weisbecker , Thomas Gleixner Cc: linux-kernel@vger.kernel.org, netdev@vger.kernel.org, Miroslav Lichvar , Richard Cochran , Christopher S Hall , Anna-Maria Behnsen From: Thomas Gleixner From: Thomas Gleixner Functions which operate on the real timekeeper, e.g. do_settimeofday(), have error conditions. If they are hit a full timekeeping update is still required because the already committed operations modified the timekeeper. When switching these functions to operate on the shadow timekeeper then the full update can be avoided in the error case, but the modified shadow timekeeper has to be restored. Provide a helper function for that. Signed-off-by: Thomas Gleixner Signed-off-by: Anna-Maria Behnsen Acked-by: John Stultz --- kernel/time/timekeeping.c | 11 ++++++++++- 1 file changed, 10 insertions(+), 1 deletion(-) diff --git a/kernel/time/timekeeping.c b/kernel/time/timekeeping.c index 5a747afe64b4..41d88f645868 100644 --- a/kernel/time/timekeeping.c +++ b/kernel/time/timekeeping.c @@ -755,6 +755,15 @@ static inline void tk_update_ktime_data(struct timekeeper *tk) tk->tkr_raw.base = ns_to_ktime(tk->raw_sec * NSEC_PER_SEC); } +/* + * Restore the shadow timekeeper from the real timekeeper. + */ +static void timekeeping_restore_shadow(struct tk_data *tkd) +{ + lockdep_assert_held(&tkd->lock); + memcpy(&tkd->shadow_timekeeper, &tkd->timekeeper, sizeof(tkd->timekeeper)); +} + static void timekeeping_update(struct tk_data *tkd, struct timekeeper *tk, unsigned int action) { lockdep_assert_held(&tkd->lock); @@ -782,7 +791,7 @@ static void timekeeping_update(struct tk_data *tkd, struct timekeeper *tk, unsig * timekeeper structure on the next update with stale data */ if (action & TK_MIRROR) - memcpy(&tkd->shadow_timekeeper, tk, sizeof(*tk)); + timekeeping_restore_shadow(tkd); } static void timekeeping_update_staged(struct tk_data *tkd, unsigned int action) From patchwork Wed Oct 9 08:29:09 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Anna-Maria Behnsen X-Patchwork-Id: 13827751 Received: from galois.linutronix.de (Galois.linutronix.de [193.142.43.55]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id DFE5718E779; Wed, 9 Oct 2024 08:29:22 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=193.142.43.55 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1728462564; cv=none; b=JbTOR80bENSgw3lp5WSnd33vv3sbEA0qGSJBNxTcyYFjqlXZBIuq/AVBpVmF2+0cst9iVcNpxquea2qfcwFDVGTu65Dz9CVqsrZwgissxUpFxT3OluXYwT79yAmIoLKS/VDvWNbWQ4fTCMSth1hI/yfjA/4yit63EvPGTTDnyUI= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1728462564; c=relaxed/simple; bh=2b3V099iyYcM42QJJZx9RFoxOEkUlAcOJ9jaApElm5I=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=V7JV1snv1ai26easA6Qt/mGI1oWMLF8B68/uBIrt9/wQljs8mz3pL9eOwUFT6NrCIuJW6HP2lcT2PUBOIWMHgFbjVT1Kes8QsPK0mBNLh1yVkTBb7jsxX7l27bpAJ/B4EsnR9IKlzMJJDGctSqSCvp0JxJ6GAAT393ec5BEzXBo= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linutronix.de; spf=pass smtp.mailfrom=linutronix.de; dkim=pass (2048-bit key) header.d=linutronix.de header.i=@linutronix.de header.b=S2IZTJVh; dkim=permerror (0-bit key) header.d=linutronix.de header.i=@linutronix.de header.b=AaqATKZP; arc=none smtp.client-ip=193.142.43.55 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linutronix.de Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linutronix.de Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=linutronix.de header.i=@linutronix.de header.b="S2IZTJVh"; dkim=permerror (0-bit key) header.d=linutronix.de header.i=@linutronix.de header.b="AaqATKZP" From: Anna-Maria Behnsen DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linutronix.de; s=2020; t=1728462559; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=VWVP22ee0N3hgHNE8ntTO7usiiv5V20D1YtJX5nx4VY=; b=S2IZTJVhScM9Fgm0PTu+PI9ANaNuZG4BGdeuwTpN4sWhd0IgZWLTUHW4X1JvNgoTK4nEwm KNKOqOSwIS7ylLVOLrCv6pTvNk33M+kZW0IBIrG4wKmI8J2+QcuQWbRrw6OBpKl3i0cUrR VHmhXVm2miIfaMyn2UyIrM6wzm72UiD5xM7nl4In4tGl0BJIC+5cYWsHXRD6kr+c/tWzXe VNZtmSsWnGop5taboVpP17rV2x6GaCK5s1KGJ+dNwFyofkXQ7/54x1TdDRZgM8qGrhLovF LwwwlulqzOQI9w0oGh4/bNYR1LqwY9SBSolbybMXOQArxgByP58hBb0sRM1Dtw== DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=linutronix.de; s=2020e; t=1728462559; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=VWVP22ee0N3hgHNE8ntTO7usiiv5V20D1YtJX5nx4VY=; b=AaqATKZPnYtnZd5BwO0ZDj3RjxUgoOt0x2i8T9sCwocCOLHEBHMNBvjL/jgm5aN+d2whl7 MXySXMfV5eatESBA== Date: Wed, 09 Oct 2024 10:29:09 +0200 Subject: [PATCH v2 16/25] timekeeping: Rework do_settimeofday64() to use shadow_timekeeper Precedence: bulk X-Mailing-List: netdev@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Message-Id: <20241009-devel-anna-maria-b4-timers-ptp-timekeeping-v2-16-554456a44a15@linutronix.de> References: <20241009-devel-anna-maria-b4-timers-ptp-timekeeping-v2-0-554456a44a15@linutronix.de> In-Reply-To: <20241009-devel-anna-maria-b4-timers-ptp-timekeeping-v2-0-554456a44a15@linutronix.de> To: John Stultz , Frederic Weisbecker , Thomas Gleixner Cc: linux-kernel@vger.kernel.org, netdev@vger.kernel.org, Miroslav Lichvar , Richard Cochran , Christopher S Hall , Anna-Maria Behnsen From: Anna-Maria Behnsen Updates of the timekeeper can be done by operating on the shadow timekeeper and afterwards copying the result into the real timekeeper. This has the advantage, that the sequence count write protected region is kept as small as possible. Convert do_settimeofday64() to use this scheme. That allows to use a scoped_guard() for locking the timekeeper lock as the usage of the shadow timekeeper allows a rollback in the error case instead of the full timekeeper update of the original code. Signed-off-by: Anna-Maria Behnsen Acked-by: John Stultz --- kernel/time/timekeeping.c | 42 ++++++++++++++++-------------------------- 1 file changed, 16 insertions(+), 26 deletions(-) diff --git a/kernel/time/timekeeping.c b/kernel/time/timekeeping.c index 41d88f645868..cc01ad53d96d 100644 --- a/kernel/time/timekeeping.c +++ b/kernel/time/timekeeping.c @@ -1479,45 +1479,35 @@ EXPORT_SYMBOL_GPL(timekeeping_clocksource_has_base); */ int do_settimeofday64(const struct timespec64 *ts) { - struct timekeeper *tk = &tk_core.timekeeper; struct timespec64 ts_delta, xt; - unsigned long flags; - int ret = 0; if (!timespec64_valid_settod(ts)) return -EINVAL; - raw_spin_lock_irqsave(&tk_core.lock, flags); - write_seqcount_begin(&tk_core.seq); - - timekeeping_forward_now(tk); - - xt = tk_xtime(tk); - ts_delta = timespec64_sub(*ts, xt); + scoped_guard (raw_spinlock_irqsave, &tk_core.lock) { + struct timekeeper *tk = &tk_core.shadow_timekeeper; - if (timespec64_compare(&tk->wall_to_monotonic, &ts_delta) > 0) { - ret = -EINVAL; - goto out; - } + timekeeping_forward_now(tk); - tk_set_wall_to_mono(tk, timespec64_sub(tk->wall_to_monotonic, ts_delta)); + xt = tk_xtime(tk); + ts_delta = timespec64_sub(*ts, xt); - tk_set_xtime(tk, ts); -out: - timekeeping_update(&tk_core, tk, TK_UPDATE_ALL | TK_MIRROR); + if (timespec64_compare(&tk->wall_to_monotonic, &ts_delta) > 0) { + timekeeping_restore_shadow(&tk_core); + return -EINVAL; + } - write_seqcount_end(&tk_core.seq); - raw_spin_unlock_irqrestore(&tk_core.lock, flags); + tk_set_wall_to_mono(tk, timespec64_sub(tk->wall_to_monotonic, ts_delta)); + tk_set_xtime(tk, ts); + timekeeping_update_staged(&tk_core, TK_UPDATE_ALL); + } /* Signal hrtimers about time change */ clock_was_set(CLOCK_SET_WALL); - if (!ret) { - audit_tk_injoffset(ts_delta); - add_device_randomness(ts, sizeof(*ts)); - } - - return ret; + audit_tk_injoffset(ts_delta); + add_device_randomness(ts, sizeof(*ts)); + return 0; } EXPORT_SYMBOL(do_settimeofday64); From patchwork Wed Oct 9 08:29:10 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Anna-Maria Behnsen X-Patchwork-Id: 13827757 Received: from galois.linutronix.de (Galois.linutronix.de [193.142.43.55]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 47567188CCA; Wed, 9 Oct 2024 08:29:23 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=193.142.43.55 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1728462565; cv=none; b=TglUmN25J3TsIpiBSrDmt5UQGbFFu6DkiSKYkBf72kJ54CiLM55GrIhhiF9VVckH56NP08nCjFStbFTJ1IMY9DnQE3w9ci3ycnNOzrg6OGMYFTQ13cRja/oOez0HgTp0tyOONJ0km8S77JphTkHJad9ZAl4oXHfRR2+4o7xoF1E= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1728462565; c=relaxed/simple; bh=0pwo+UQw7f3hUgtvEqKtPPSPkIVOdx13yLlbdyuf5fo=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=GAs1DleukshDnWHQVAryOcYQ4RMEGfBJk6v5uxRCFgfqnUnbzA7ETt+Cq3DrziX85kVSiIp6Okx0zJPWnlQK0YJtr//720QRNdv0VnBQ0abH44KD2edFgr8gA4or/ANdT6AvU2jIdqBlkfTqxa8vyuoPcxooBCbH/5gftTswMoU= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linutronix.de; spf=pass smtp.mailfrom=linutronix.de; dkim=pass (2048-bit key) header.d=linutronix.de header.i=@linutronix.de header.b=LBtJJSX7; dkim=permerror (0-bit key) header.d=linutronix.de header.i=@linutronix.de header.b=n4A3Pqvl; arc=none smtp.client-ip=193.142.43.55 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linutronix.de Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linutronix.de Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=linutronix.de header.i=@linutronix.de header.b="LBtJJSX7"; dkim=permerror (0-bit key) header.d=linutronix.de header.i=@linutronix.de header.b="n4A3Pqvl" From: Anna-Maria Behnsen DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linutronix.de; s=2020; t=1728462559; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=jCa3cuYkyhSihHTRaUKx3sdAbT7LIH+0SPU7UmLweK0=; b=LBtJJSX7qBWFutAbGz5ng81Ro8g25UiHGAPwPYKGwPOXE0CeICoaLSzOIgAvGWhGHBGqQX iHE9P4iGD51Y8tsnHbca9VDPBPKKAiP17v9a/3LaXlbqy9EQLVotwSOriI+8h91cfsOr3Z dHzABIHQ/RYJ8Bj/YmtkH0YCi0SE0LJM/DuwYSURujjZrhqXWL2VNCujkJgw4MUN1AvXDb VvK/g/q67TTf3HQtdrlPJpMipt3kOg/bjnU17Xmi9CpYLz9JtEzLkx/TIZrAo2pcQ0L/Ve c8U+8FrIRe+3FAaEMEJ2/FzkVu1xfsgHw85+1hed6KrecgB/dgaIwOvePGNkIg== DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=linutronix.de; s=2020e; t=1728462559; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=jCa3cuYkyhSihHTRaUKx3sdAbT7LIH+0SPU7UmLweK0=; b=n4A3PqvlXcP9xvO/vFupQ1a7ql9I4dH3o16K4xHnK+KPchkafF8UYulkovuHCgjeeauOAq toeza730WucZwyBw== Date: Wed, 09 Oct 2024 10:29:10 +0200 Subject: [PATCH v2 17/25] timekeeping: Rework timekeeping_inject_offset() to use shadow_timekeeper Precedence: bulk X-Mailing-List: netdev@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Message-Id: <20241009-devel-anna-maria-b4-timers-ptp-timekeeping-v2-17-554456a44a15@linutronix.de> References: <20241009-devel-anna-maria-b4-timers-ptp-timekeeping-v2-0-554456a44a15@linutronix.de> In-Reply-To: <20241009-devel-anna-maria-b4-timers-ptp-timekeeping-v2-0-554456a44a15@linutronix.de> To: John Stultz , Frederic Weisbecker , Thomas Gleixner Cc: linux-kernel@vger.kernel.org, netdev@vger.kernel.org, Miroslav Lichvar , Richard Cochran , Christopher S Hall , Anna-Maria Behnsen From: Anna-Maria Behnsen Updates of the timekeeper can be done by operating on the shadow timekeeper and afterwards copying the result into the real timekeeper. This has the advantage, that the sequence count write protected region is kept as small as possible. Convert timekeeping_inject_offset() to use this scheme. That allows to use a scoped_guard() for locking the timekeeper lock as the usage of the shadow timekeeper allows a rollback in the error case instead of the full timekeeper update of the original code. Signed-off-by: Anna-Maria Behnsen Acked-by: John Stultz --- kernel/time/timekeeping.c | 41 ++++++++++++++++------------------------- 1 file changed, 16 insertions(+), 25 deletions(-) diff --git a/kernel/time/timekeeping.c b/kernel/time/timekeeping.c index cc01ad53d96d..051041e92e54 100644 --- a/kernel/time/timekeeping.c +++ b/kernel/time/timekeeping.c @@ -1519,40 +1519,31 @@ EXPORT_SYMBOL(do_settimeofday64); */ static int timekeeping_inject_offset(const struct timespec64 *ts) { - struct timekeeper *tk = &tk_core.timekeeper; - unsigned long flags; - struct timespec64 tmp; - int ret = 0; - if (ts->tv_nsec < 0 || ts->tv_nsec >= NSEC_PER_SEC) return -EINVAL; - raw_spin_lock_irqsave(&tk_core.lock, flags); - write_seqcount_begin(&tk_core.seq); - - timekeeping_forward_now(tk); - - /* Make sure the proposed value is valid */ - tmp = timespec64_add(tk_xtime(tk), *ts); - if (timespec64_compare(&tk->wall_to_monotonic, ts) > 0 || - !timespec64_valid_settod(&tmp)) { - ret = -EINVAL; - goto error; - } + scoped_guard (raw_spinlock_irqsave, &tk_core.lock) { + struct timekeeper *tk = &tk_core.shadow_timekeeper; + struct timespec64 tmp; - tk_xtime_add(tk, ts); - tk_set_wall_to_mono(tk, timespec64_sub(tk->wall_to_monotonic, *ts)); + timekeeping_forward_now(tk); -error: /* even if we error out, we forwarded the time, so call update */ - timekeeping_update(&tk_core, tk, TK_UPDATE_ALL | TK_MIRROR); + /* Make sure the proposed value is valid */ + tmp = timespec64_add(tk_xtime(tk), *ts); + if (timespec64_compare(&tk->wall_to_monotonic, ts) > 0 || + !timespec64_valid_settod(&tmp)) { + timekeeping_restore_shadow(&tk_core); + return -EINVAL; + } - write_seqcount_end(&tk_core.seq); - raw_spin_unlock_irqrestore(&tk_core.lock, flags); + tk_xtime_add(tk, ts); + tk_set_wall_to_mono(tk, timespec64_sub(tk->wall_to_monotonic, *ts)); + timekeeping_update_staged(&tk_core, TK_UPDATE_ALL); + } /* Signal hrtimers about time change */ clock_was_set(CLOCK_SET_WALL); - - return ret; + return 0; } /* From patchwork Wed Oct 9 08:29:11 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Anna-Maria Behnsen X-Patchwork-Id: 13827753 Received: from galois.linutronix.de (Galois.linutronix.de [193.142.43.55]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 475E518F2C5; Wed, 9 Oct 2024 08:29:23 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=193.142.43.55 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1728462565; cv=none; b=a4vxjwRSQZmS7+HnQwy+zhduIjyRlyfD0Pgw8Yqp3QpRVGoaGaOp2OLbVCM0/CMHJdijE6Qj1o3kOZ/j3v2I3FICpHjPMF8WgJgYy9VTdwxkzAl6nJ8kZO1R5tazoQyw5jN36S04GXLcNgdnwXhhX27u7W03RxkVsPJA3xVa0zY= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1728462565; c=relaxed/simple; bh=WDO3Ytc4nrYwYDexC/hZyP3VzuTayhrvdBH51HROiRM=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=XwKe96EevdIeaoFv7XHuJ2HRDBfTW1vRlS+PB670E6tz7qmzpj6a+Kn/LgZFVEVs7qV9MSqzz94oYK4cSusGrq4tqCSuD63ArmxriaVD4oqG+qgjq2IwtsXf5gnM5EZ5kepAyFLndZuarQgMI850WyvyhPZkZ76FKGoWNthaJ+Y= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linutronix.de; spf=pass smtp.mailfrom=linutronix.de; dkim=pass (2048-bit key) header.d=linutronix.de header.i=@linutronix.de header.b=ILGBnHgS; dkim=permerror (0-bit key) header.d=linutronix.de header.i=@linutronix.de header.b=5yrMPzp0; arc=none smtp.client-ip=193.142.43.55 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linutronix.de Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linutronix.de Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=linutronix.de header.i=@linutronix.de header.b="ILGBnHgS"; dkim=permerror (0-bit key) header.d=linutronix.de header.i=@linutronix.de header.b="5yrMPzp0" From: Anna-Maria Behnsen DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linutronix.de; s=2020; t=1728462559; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=tO9lJ4hSftezkW1I9W2pEnXRBh82ke3Ir/FYPjmdFQA=; b=ILGBnHgSrrLIhi25h9e7ZIZIi4daXPW80PgxWgepOKxU2d0QXUjUS4Z0OeAWmlFS6jLOtM FFlERkiPKafxhBHsOtMH5uOymkmSSgiADQq0h/Fdji/4CSjFcYiYXwIi4YSwbkNkhHh4BF hQqTGPlngjHHqIeiYWuWCPdpqxNGF2opRCN2kxFB4YCLboOrR4I/hxkloW+ukY2Al9Hpft 3RA7NaGRzDUdJ83Ijroinxi4qhxbSZY8K2pppLUlx5TBaDxh9W70ArvKNq3xROg5dac93e 0uFoh+G/LC2yCqdYjSel3LAZZqakmoKoBmFEK++oOQoAltuBsV0zg3jNXSIPUQ== DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=linutronix.de; s=2020e; t=1728462559; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=tO9lJ4hSftezkW1I9W2pEnXRBh82ke3Ir/FYPjmdFQA=; b=5yrMPzp0wLf7oBogJPnocNULhLeWTDZDmUixeWyUuS0pMK2ggsYjrkD0P4LcidGwl9k2sN 32UIBLIJtfKl2bBg== Date: Wed, 09 Oct 2024 10:29:11 +0200 Subject: [PATCH v2 18/25] timekeeping: Rework change_clocksource() to use shadow_timekeeper Precedence: bulk X-Mailing-List: netdev@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Message-Id: <20241009-devel-anna-maria-b4-timers-ptp-timekeeping-v2-18-554456a44a15@linutronix.de> References: <20241009-devel-anna-maria-b4-timers-ptp-timekeeping-v2-0-554456a44a15@linutronix.de> In-Reply-To: <20241009-devel-anna-maria-b4-timers-ptp-timekeeping-v2-0-554456a44a15@linutronix.de> To: John Stultz , Frederic Weisbecker , Thomas Gleixner Cc: linux-kernel@vger.kernel.org, netdev@vger.kernel.org, Miroslav Lichvar , Richard Cochran , Christopher S Hall , Anna-Maria Behnsen From: Anna-Maria Behnsen Updates of the timekeeper can be done by operating on the shadow timekeeper and afterwards copying the result into the real timekeeper. This has the advantage, that the sequence count write protected region is kept as small as possible. Convert change_clocksource() to use this scheme. Signed-off-by: Anna-Maria Behnsen Acked-by: John Stultz --- kernel/time/timekeeping.c | 18 +++++++----------- 1 file changed, 7 insertions(+), 11 deletions(-) diff --git a/kernel/time/timekeeping.c b/kernel/time/timekeeping.c index 051041e92e54..6d21e9bf5c35 100644 --- a/kernel/time/timekeeping.c +++ b/kernel/time/timekeeping.c @@ -1596,9 +1596,7 @@ static void __timekeeping_set_tai_offset(struct timekeeper *tk, s32 tai_offset) */ static int change_clocksource(void *data) { - struct timekeeper *tk = &tk_core.timekeeper; struct clocksource *new = data, *old = NULL; - unsigned long flags; /* * If the clocksource is in a module, get a module reference. @@ -1614,16 +1612,14 @@ static int change_clocksource(void *data) return 0; } - raw_spin_lock_irqsave(&tk_core.lock, flags); - write_seqcount_begin(&tk_core.seq); - - timekeeping_forward_now(tk); - old = tk->tkr_mono.clock; - tk_setup_internals(tk, new); - timekeeping_update(&tk_core, tk, TK_UPDATE_ALL | TK_MIRROR); + scoped_guard (raw_spinlock_irqsave, &tk_core.lock) { + struct timekeeper *tk = &tk_core.shadow_timekeeper; - write_seqcount_end(&tk_core.seq); - raw_spin_unlock_irqrestore(&tk_core.lock, flags); + timekeeping_forward_now(tk); + old = tk->tkr_mono.clock; + tk_setup_internals(tk, new); + timekeeping_update_staged(&tk_core, TK_UPDATE_ALL); + } if (old) { if (old->disable) From patchwork Wed Oct 9 08:29:12 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Anna-Maria Behnsen X-Patchwork-Id: 13827755 Received: from galois.linutronix.de (Galois.linutronix.de [193.142.43.55]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 66EED18F2FD; Wed, 9 Oct 2024 08:29:23 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=193.142.43.55 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1728462565; cv=none; b=mCgROfNJbr+7g9mO/sUcQZGJ6aXkN0oLfto3t4lhwOrxEYZF9maKvdOZq8Qx8BCgOCnvtF08OLFGymW+dPCuoQZ6gxooYwlcP0f2migLsxYNJOQbFgderQ5QiPC6ie7/Olg2ePjsJhFBOL7+AkYzRFmoJBa2AJvvJgvRLwV5L84= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1728462565; c=relaxed/simple; bh=OV2RbWL3lOHjlv0YduQhmOvwG+FZWlmHlXI0HnvVwFE=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=u+7e/R/TisshshP2CeCJOz46Uyd2DDLunO4MV9oH7N73tllGdH1A3rCZ6kGpaFoCpXDurIbu3HSpj0nLo2NhHEBsZeNIH2edpOVBQVx3mk16Inmi/IRePjAXKPEB0jGrdfDo+MTGh2LCEQDnla0DgsYVYVSh9a1ggT0sR8r7kh0= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linutronix.de; spf=pass smtp.mailfrom=linutronix.de; dkim=pass (2048-bit key) header.d=linutronix.de header.i=@linutronix.de header.b=o/+xvFBg; dkim=permerror (0-bit key) header.d=linutronix.de header.i=@linutronix.de header.b=ymrGE37r; arc=none smtp.client-ip=193.142.43.55 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linutronix.de Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linutronix.de Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=linutronix.de header.i=@linutronix.de header.b="o/+xvFBg"; dkim=permerror (0-bit key) header.d=linutronix.de header.i=@linutronix.de header.b="ymrGE37r" From: Anna-Maria Behnsen DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linutronix.de; s=2020; t=1728462560; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=Tdm7e41dOSBQbrRJct9Hds0vyLBx4funvYTF+LlLDVI=; b=o/+xvFBgQoIfomFZbTU/LtvQKz6jAg8xWVov+tlMaCaj3MNyLHUpE/ULwLuSErOK2eKFHF 74RyDXnj4lf1kwHiGGzOnX99AJ3WwZXoNY7INQcU1phBMvGa5n8RY+NouUj1KIUKGDaM/P U/Y7WwtqoyDaNa+EjLwO3kr9OEIfzQZ+ovqEn+UjPyDfehzE7lnrgaKlHl8m+cVVeGjHV4 kP8ZAe3GNWkKP0XCrZF6LLeIJUtz08rtqqg84TuQH12Ay3quhBlp9c1GzMR820JhICU4Vi Jqfqm7fV4NlrHvNnhAAWkyWEA86PIuFfyOIaoddGrAKZ+OmQsQDZKjKXXeWArA== DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=linutronix.de; s=2020e; t=1728462560; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=Tdm7e41dOSBQbrRJct9Hds0vyLBx4funvYTF+LlLDVI=; b=ymrGE37rt5+5av0iME01J140p7PuSkX1xQeH/lgQpRljwmE6tsaDlRcPYPy/qML4T8N5Yw JR0RBE7ogiwAJdDQ== Date: Wed, 09 Oct 2024 10:29:12 +0200 Subject: [PATCH v2 19/25] timekeeping: Rework timekeeping_init() to use shadow_timekeeper Precedence: bulk X-Mailing-List: netdev@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Message-Id: <20241009-devel-anna-maria-b4-timers-ptp-timekeeping-v2-19-554456a44a15@linutronix.de> References: <20241009-devel-anna-maria-b4-timers-ptp-timekeeping-v2-0-554456a44a15@linutronix.de> In-Reply-To: <20241009-devel-anna-maria-b4-timers-ptp-timekeeping-v2-0-554456a44a15@linutronix.de> To: John Stultz , Frederic Weisbecker , Thomas Gleixner Cc: linux-kernel@vger.kernel.org, netdev@vger.kernel.org, Miroslav Lichvar , Richard Cochran , Christopher S Hall , Anna-Maria Behnsen From: Anna-Maria Behnsen For timekeeping_init() the sequence count write held time is not relevant and it could keep working on the real timekeeper, but there is no reason to make it different from other timekeeper updates. Convert it to operate on the shadow timekeeper. Signed-off-by: Anna-Maria Behnsen Acked-by: John Stultz --- kernel/time/timekeeping.c | 8 +++----- 1 file changed, 3 insertions(+), 5 deletions(-) diff --git a/kernel/time/timekeeping.c b/kernel/time/timekeeping.c index 6d21e9bf5c35..bb225534fee1 100644 --- a/kernel/time/timekeeping.c +++ b/kernel/time/timekeeping.c @@ -1778,7 +1778,7 @@ static bool persistent_clock_exists; void __init timekeeping_init(void) { struct timespec64 wall_time, boot_offset, wall_to_mono; - struct timekeeper *tk = &tk_core.timekeeper; + struct timekeeper *tk = &tk_core.shadow_timekeeper; struct clocksource *clock; tkd_basic_setup(&tk_core); @@ -1802,7 +1802,7 @@ void __init timekeeping_init(void) wall_to_mono = timespec64_sub(boot_offset, wall_time); guard(raw_spinlock_irqsave)(&tk_core.lock); - write_seqcount_begin(&tk_core.seq); + ntp_init(); clock = clocksource_default_clock(); @@ -1815,9 +1815,7 @@ void __init timekeeping_init(void) tk_set_wall_to_mono(tk, wall_to_mono); - timekeeping_update(&tk_core, tk, TK_MIRROR | TK_CLOCK_WAS_SET); - - write_seqcount_end(&tk_core.seq); + timekeeping_update_staged(&tk_core, TK_CLOCK_WAS_SET); } /* time in seconds when suspend began for persistent clock */ From patchwork Wed Oct 9 08:29:13 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Anna-Maria Behnsen X-Patchwork-Id: 13827756 Received: from galois.linutronix.de (Galois.linutronix.de [193.142.43.55]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 6A17718E35F; Wed, 9 Oct 2024 08:29:23 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=193.142.43.55 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1728462565; cv=none; b=CEC3zCnN3fl09sLxcWIUrfoD5ElA6CV9BtWe2cjVRU1y0kweOk3EC0DFEwO8HSOJiaZLYPdeyJMGfbNPkQVJBusRZ2xtgJbb/wiJYT3G6HW6sDWXAl54ZkcnaV2v9+opMjPLIT0QcFsLyqlSotbAYEFokK7o5Eq9OenWW4U3EFk= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1728462565; c=relaxed/simple; bh=Ro9wXRsRW3pwaaOyyuMDj0Rd4VJ5BecOwketZ6eYHgE=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=bKIC4GnQ7AZPCxtUlzB1/ENq/x7duw5d5BNPpz4h62dmc7u+MvqE86kD6ayDIfA7WCgE7alnB/xAKy4Tlp7/sVfKti31C45tnRQwpqu6vez5XJooUkQwZp3Czxvsf6NGBNhkiJlbBGomELjWIG64Z7YYlNS5asXpSWlcFvqPJHs= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linutronix.de; spf=pass smtp.mailfrom=linutronix.de; dkim=pass (2048-bit key) header.d=linutronix.de header.i=@linutronix.de header.b=R7HyOkfJ; dkim=permerror (0-bit key) header.d=linutronix.de header.i=@linutronix.de header.b=bwKQ/GXi; arc=none smtp.client-ip=193.142.43.55 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linutronix.de Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linutronix.de Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=linutronix.de header.i=@linutronix.de header.b="R7HyOkfJ"; dkim=permerror (0-bit key) header.d=linutronix.de header.i=@linutronix.de header.b="bwKQ/GXi" From: Anna-Maria Behnsen DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linutronix.de; s=2020; t=1728462560; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=LVHw94u6ZhSsv2Z/b/AuuBVF799Bd06TmChz9mXDUYQ=; b=R7HyOkfJsSj27VWd7JVkmZTveh52E2mLCUT+NcKoqMyZ0zOHNMUMtrmPqzONE0GrqI3e0g ahGiN1gkTSg8kA9jGmurVlm7ERUpC0xn0vsYvz80s5Aq2qEOQ5wxDe3LDQdY2k7fQkbv8X 4Mmkwn1+Kt8YVzCd7wH9OWmOmBodv+2qGmTh3fCYmP65LmmSnlg5Finw4l0COBZ0muWPoR SvHuT+aOsDfQiltHBPi9zXTikNHCqYjY5Xv9L/xh9uOSVHZ7sAnv6u2CGwZjZDWehqJwC7 8MVXfc9Pou71ldc5LuFFXoWMFXHqNk4Gt9iUXZu5cppovFgk2D3Tghls9SIZMg== DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=linutronix.de; s=2020e; t=1728462560; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=LVHw94u6ZhSsv2Z/b/AuuBVF799Bd06TmChz9mXDUYQ=; b=bwKQ/GXiRn2Ilil/lb3rTpDYr3n8BE983xjzB4sjHrzYhP966XfDljsaBDdT7ArLlq48/o SZBXom/vnk/+SXBQ== Date: Wed, 09 Oct 2024 10:29:13 +0200 Subject: [PATCH v2 20/25] timekeeping: Rework timekeeping_inject_sleeptime64() to use shadow_timekeeper Precedence: bulk X-Mailing-List: netdev@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Message-Id: <20241009-devel-anna-maria-b4-timers-ptp-timekeeping-v2-20-554456a44a15@linutronix.de> References: <20241009-devel-anna-maria-b4-timers-ptp-timekeeping-v2-0-554456a44a15@linutronix.de> In-Reply-To: <20241009-devel-anna-maria-b4-timers-ptp-timekeeping-v2-0-554456a44a15@linutronix.de> To: John Stultz , Frederic Weisbecker , Thomas Gleixner Cc: linux-kernel@vger.kernel.org, netdev@vger.kernel.org, Miroslav Lichvar , Richard Cochran , Christopher S Hall , Anna-Maria Behnsen From: Anna-Maria Behnsen Updates of the timekeeper can be done by operating on the shadow timekeeper and afterwards copying the result into the real timekeeper. This has the advantage, that the sequence count write protected region is kept as small as possible. Convert timekeeping_inject_sleeptime64() to use this scheme. Signed-off-by: Anna-Maria Behnsen Acked-by: John Stultz --- kernel/time/timekeeping.c | 22 +++++++--------------- 1 file changed, 7 insertions(+), 15 deletions(-) diff --git a/kernel/time/timekeeping.c b/kernel/time/timekeeping.c index bb225534fee1..c1a2726a0d41 100644 --- a/kernel/time/timekeeping.c +++ b/kernel/time/timekeeping.c @@ -1893,22 +1893,14 @@ bool timekeeping_rtc_skipsuspend(void) */ void timekeeping_inject_sleeptime64(const struct timespec64 *delta) { - struct timekeeper *tk = &tk_core.timekeeper; - unsigned long flags; - - raw_spin_lock_irqsave(&tk_core.lock, flags); - write_seqcount_begin(&tk_core.seq); - - suspend_timing_needed = false; - - timekeeping_forward_now(tk); - - __timekeeping_inject_sleeptime(tk, delta); - - timekeeping_update(&tk_core, tk, TK_UPDATE_ALL | TK_MIRROR); + scoped_guard(raw_spinlock_irqsave, &tk_core.lock) { + struct timekeeper *tk = &tk_core.shadow_timekeeper; - write_seqcount_end(&tk_core.seq); - raw_spin_unlock_irqrestore(&tk_core.lock, flags); + suspend_timing_needed = false; + timekeeping_forward_now(tk); + __timekeeping_inject_sleeptime(tk, delta); + timekeeping_update_staged(&tk_core, TK_UPDATE_ALL); + } /* Signal hrtimers about time change */ clock_was_set(CLOCK_SET_WALL | CLOCK_SET_BOOT); From patchwork Wed Oct 9 08:29:14 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Anna-Maria Behnsen X-Patchwork-Id: 13827759 Received: from galois.linutronix.de (Galois.linutronix.de [193.142.43.55]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 636A318F2FB; Wed, 9 Oct 2024 08:29:23 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=193.142.43.55 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1728462566; cv=none; b=GC02LrGLOHqxiTpwoZcvbLi716p15D5JoT6WkdVHfIfDCb4QsJ+xQ30T5JvwtOagxV4W0Y0G+Tmy1a1ORxhvw6rQuxfiYZX0/LNG/UGnI7d12VdTzEpMDApO8rc2UtCBpOmstRNBFINdlDSeVXSRHY0CMGlzHnvcTSXBhEFMvb4= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1728462566; c=relaxed/simple; bh=tM7sTcZmZRY3Zvk3KpyPvf+k6AIMdSK74Z/iy7bhhdE=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=hH+Ln5Gll6b4AHOKekCYi5PWnOPuUXhXKR8YR08pHzN4f8puyL9CBBgz2XiIfTUEMa/9QqvjzVkJxd+TYyh9QCtmxSVSzgvcGbXJorYESY5B2oDQGnhOa8EF1oDDMEsSgh7+K2oAwoXrxFiosAk+Nq/wQgw4FuEanWghD3Ujrr4= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linutronix.de; spf=pass smtp.mailfrom=linutronix.de; dkim=pass (2048-bit key) header.d=linutronix.de header.i=@linutronix.de header.b=DxC0b7wF; dkim=permerror (0-bit key) header.d=linutronix.de header.i=@linutronix.de header.b=PctOmlZU; arc=none smtp.client-ip=193.142.43.55 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linutronix.de Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linutronix.de Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=linutronix.de header.i=@linutronix.de header.b="DxC0b7wF"; dkim=permerror (0-bit key) header.d=linutronix.de header.i=@linutronix.de header.b="PctOmlZU" From: Anna-Maria Behnsen DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linutronix.de; s=2020; t=1728462560; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=V1hjixVzWyEI6kbexIge7cPNj3aTuExc8IijWH/Vafg=; b=DxC0b7wFueSrXRrGbIHhxxFlAo9IUiTFKBVCSlS/ONckolGkJtHYaKxDIs3b6nMD5jR+94 /x6PFkKsta2Mj4n6IZBnO3mGBaTW5VYIRQDXpL1fsVWWQWiiEqf8VySLhKBYiSYpcHJtJk 6MzQvqLr93UDvZzGhl03e4B54P7SbLxfjpxPyUXT2J4/mTaTyYB4yPjPALTYuuxctVR5Xl MNYNFfZ9U0g3R4fhfpPYwesrc+fofHc2qPg5+A2zasIy3Uc8ne3OBFlY6eDOtpCsV6rqMV Tzk+G7bivpK4o6RSFtxUpSvQyzgUxJcqViTuV2arO8ON0cIGLVevfZp82F7nLg== DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=linutronix.de; s=2020e; t=1728462560; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=V1hjixVzWyEI6kbexIge7cPNj3aTuExc8IijWH/Vafg=; b=PctOmlZUHkreDdQWJIxOTK3Xnf8MuPkvmEBpyEB5T8yGj7DMUjhxAcVcrUfv5u61aDaqHv aZSwpmBamYkQJ3DA== Date: Wed, 09 Oct 2024 10:29:14 +0200 Subject: [PATCH v2 21/25] timekeeping: Rework timekeeping_resume() to use shadow_timekeeper Precedence: bulk X-Mailing-List: netdev@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Message-Id: <20241009-devel-anna-maria-b4-timers-ptp-timekeeping-v2-21-554456a44a15@linutronix.de> References: <20241009-devel-anna-maria-b4-timers-ptp-timekeeping-v2-0-554456a44a15@linutronix.de> In-Reply-To: <20241009-devel-anna-maria-b4-timers-ptp-timekeeping-v2-0-554456a44a15@linutronix.de> To: John Stultz , Frederic Weisbecker , Thomas Gleixner Cc: linux-kernel@vger.kernel.org, netdev@vger.kernel.org, Miroslav Lichvar , Richard Cochran , Christopher S Hall , Anna-Maria Behnsen From: Anna-Maria Behnsen Updates of the timekeeper can be done by operating on the shadow timekeeper and afterwards copying the result into the real timekeeper. This has the advantage, that the sequence count write protected region is kept as small as possible. While the sequence count held time is not relevant for the resume path as there is no concurrency, there is no reason to have this function different than all the other update sites. Convert timekeeping_inject_offset() to use this scheme and cleanup the variable declaration while at it. Signed-off-by: Anna-Maria Behnsen Acked-by: John Stultz --- kernel/time/timekeeping.c | 10 ++++------ 1 file changed, 4 insertions(+), 6 deletions(-) diff --git a/kernel/time/timekeeping.c b/kernel/time/timekeeping.c index c1a2726a0d41..63b7a1379ae8 100644 --- a/kernel/time/timekeeping.c +++ b/kernel/time/timekeeping.c @@ -1912,12 +1912,12 @@ void timekeeping_inject_sleeptime64(const struct timespec64 *delta) */ void timekeeping_resume(void) { - struct timekeeper *tk = &tk_core.timekeeper; + struct timekeeper *tk = &tk_core.shadow_timekeeper; struct clocksource *clock = tk->tkr_mono.clock; - unsigned long flags; struct timespec64 ts_new, ts_delta; - u64 cycle_now, nsec; bool inject_sleeptime = false; + u64 cycle_now, nsec; + unsigned long flags; read_persistent_clock64(&ts_new); @@ -1925,7 +1925,6 @@ void timekeeping_resume(void) clocksource_resume(); raw_spin_lock_irqsave(&tk_core.lock, flags); - write_seqcount_begin(&tk_core.seq); /* * After system resumes, we need to calculate the suspended time and @@ -1960,8 +1959,7 @@ void timekeeping_resume(void) tk->ntp_error = 0; timekeeping_suspended = 0; - timekeeping_update(&tk_core, tk, TK_MIRROR | TK_CLOCK_WAS_SET); - write_seqcount_end(&tk_core.seq); + timekeeping_update_staged(&tk_core, TK_CLOCK_WAS_SET); raw_spin_unlock_irqrestore(&tk_core.lock, flags); touch_softlockup_watchdog(); From patchwork Wed Oct 9 08:29:15 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Anna-Maria Behnsen X-Patchwork-Id: 13827754 Received: from galois.linutronix.de (Galois.linutronix.de [193.142.43.55]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id A4B6618FC83; Wed, 9 Oct 2024 08:29:23 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=193.142.43.55 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1728462565; cv=none; b=MJZFC0ynAz2uGzJbTLgZP5lV4yRPMjBjk4lL8Pt0nTms/VatmZjLEgj/Qxt7QYRl3w8ieEQhkz+bwXw6NGPpOxktqSLiggyJiKgHvoQhx6pGlZs0ILCdlHlAu5QMkqqwGK9uxq6vylc9OKW6Y5b9uE1M04ao0umQ2mLSG5low7o= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1728462565; c=relaxed/simple; bh=JPb37VDuKMoRNJ9vp81Du4hr25S86cQJYo5RCttwRE0=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=mGmVACGNlZdqr2GAontVda/787G1/2QMTjDgPJHmE2hvxJhvFEUBW6zZ0lUzfs8jzDmyh7Ejz/iiuq8+3yPP6Fuc84lS4Lp1Tk5PDqWZabd4EU/c4rYMWTVM9vabUs7jwIfO0rYWsu+0GyOFSSTQ1ts2P2ffXlVNT0NNkrNysPw= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linutronix.de; spf=pass smtp.mailfrom=linutronix.de; dkim=pass (2048-bit key) header.d=linutronix.de header.i=@linutronix.de header.b=loBhx1E+; dkim=permerror (0-bit key) header.d=linutronix.de header.i=@linutronix.de header.b=GoBj7OTt; arc=none smtp.client-ip=193.142.43.55 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linutronix.de Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linutronix.de Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=linutronix.de header.i=@linutronix.de header.b="loBhx1E+"; dkim=permerror (0-bit key) header.d=linutronix.de header.i=@linutronix.de header.b="GoBj7OTt" From: Anna-Maria Behnsen DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linutronix.de; s=2020; t=1728462561; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=QXKFPtRu4Sx6Ya+F6FaITcYNeyub0Jgc6BL9e6PKQT8=; b=loBhx1E+vYnrxsr3U8V1ED4dJ0+WiD8wUa7ZlWp5tOHE7T7epQMBepx1ZqfpGodvrbE+MZ FTsmwawfoxtUt7bJhaLANkcT8pjyRBdaV7P7uq9yc30BraPuoBKeSnbOyM/E5RFIs2Darc 4fMHa8JFwMVz/OG9gELy5DGNdNvbZ4BiGtCnZwa95x30b7qA3JbdTTXKKnxarRPTrluRDI WUPd4y/GXXNv4KAtW3yOe2LwmQsxdj07L8RbcS5DC2oXBQJUnQuRTseeDBgaQnJSfLCQw4 CQ/7XvRifOQ+57EozhG6m9zP0JueJuRROyf7LIqMvLitf2HMW5C8Ku70fVG0Fg== DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=linutronix.de; s=2020e; t=1728462561; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=QXKFPtRu4Sx6Ya+F6FaITcYNeyub0Jgc6BL9e6PKQT8=; b=GoBj7OTtmqRlNlZCMMFa2AL8ftdeBqxfxnd/wtyir8pLMhX5AEJIqGQSAekXOyqiou14OB 4a6olzOLA2zxl5Cg== Date: Wed, 09 Oct 2024 10:29:15 +0200 Subject: [PATCH v2 22/25] timekeeping: Rework timekeeping_suspend() to use shadow_timekeeper Precedence: bulk X-Mailing-List: netdev@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Message-Id: <20241009-devel-anna-maria-b4-timers-ptp-timekeeping-v2-22-554456a44a15@linutronix.de> References: <20241009-devel-anna-maria-b4-timers-ptp-timekeeping-v2-0-554456a44a15@linutronix.de> In-Reply-To: <20241009-devel-anna-maria-b4-timers-ptp-timekeeping-v2-0-554456a44a15@linutronix.de> To: John Stultz , Frederic Weisbecker , Thomas Gleixner Cc: linux-kernel@vger.kernel.org, netdev@vger.kernel.org, Miroslav Lichvar , Richard Cochran , Christopher S Hall , Anna-Maria Behnsen From: Anna-Maria Behnsen Updates of the timekeeper can be done by operating on the shadow timekeeper and afterwards copying the result into the real timekeeper. This has the advantage, that the sequence count write protected region is kept as small as possible. While the sequence count held time is not relevant for the resume path as there is no concurrency, there is no reason to have this function different than all the other update sites. Convert timekeeping_inject_offset() to use this scheme and cleanup the variable declarations while at it. As halt_fast_timekeeper() does not need protection sequence counter, it is no problem to move it with this change outside of the sequence counter protected area. But it still needs to be executed while holding the lock. Signed-off-by: Anna-Maria Behnsen Acked-by: John Stultz --- kernel/time/timekeeping.c | 12 +++++------- 1 file changed, 5 insertions(+), 7 deletions(-) diff --git a/kernel/time/timekeeping.c b/kernel/time/timekeeping.c index 63b7a1379ae8..e15e843ba2b8 100644 --- a/kernel/time/timekeeping.c +++ b/kernel/time/timekeeping.c @@ -1972,11 +1972,11 @@ void timekeeping_resume(void) int timekeeping_suspend(void) { - struct timekeeper *tk = &tk_core.timekeeper; - unsigned long flags; - struct timespec64 delta, delta_delta; - static struct timespec64 old_delta; + struct timekeeper *tk = &tk_core.shadow_timekeeper; + struct timespec64 delta, delta_delta; + static struct timespec64 old_delta; struct clocksource *curr_clock; + unsigned long flags; u64 cycle_now; read_persistent_clock64(&timekeeping_suspend_time); @@ -1992,7 +1992,6 @@ int timekeeping_suspend(void) suspend_timing_needed = true; raw_spin_lock_irqsave(&tk_core.lock, flags); - write_seqcount_begin(&tk_core.seq); timekeeping_forward_now(tk); timekeeping_suspended = 1; @@ -2027,9 +2026,8 @@ int timekeeping_suspend(void) } } - timekeeping_update(&tk_core, tk, TK_MIRROR); + timekeeping_update_staged(&tk_core, 0); halt_fast_timekeeper(tk); - write_seqcount_end(&tk_core.seq); raw_spin_unlock_irqrestore(&tk_core.lock, flags); tick_suspend(); From patchwork Wed Oct 9 08:29:16 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Anna-Maria Behnsen X-Patchwork-Id: 13827761 Received: from galois.linutronix.de (Galois.linutronix.de [193.142.43.55]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id C2AEC18FC89; Wed, 9 Oct 2024 08:29:23 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=193.142.43.55 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1728462567; cv=none; b=rOZ7UmczqhDibwlN52LOstUGziCFUeNVfu/SivfZ3Tqiodpi67IUzHRMZNfDVdG+xoW4q1SccqXNxOeN/R4mXWOoJKLJDVyziDyxETts2bmQqmBFqBf/1Op//EobS+MxFIxInpBzeIJ0lY5G6CoWO7xjC+pf4V37kJA3Pw6lAYM= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1728462567; c=relaxed/simple; bh=zYy7jptOBH6pwHU/1OD0oFfh3Z3m29vNja2soSof/NM=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=a7ci2bqeo4N9ew1BvYRkrNO/uFVAXJWiOWKjhqiEDRmgl/xDtfmKSW/iO3AnAZEXAogcKPTwL2nc32nZ29vfORrC2GyZAHnE08mqwSE/alVHtS8Nkba11RvU5aAB0PVABiZH/t7H8NNagVSQX9XjgAa35aa6VQQqGgWXG8n+6Ec= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linutronix.de; spf=pass smtp.mailfrom=linutronix.de; dkim=pass (2048-bit key) header.d=linutronix.de header.i=@linutronix.de header.b=yI9twtJf; dkim=permerror (0-bit key) header.d=linutronix.de header.i=@linutronix.de header.b=GNXTF5ia; arc=none smtp.client-ip=193.142.43.55 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linutronix.de Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linutronix.de Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=linutronix.de header.i=@linutronix.de header.b="yI9twtJf"; dkim=permerror (0-bit key) header.d=linutronix.de header.i=@linutronix.de header.b="GNXTF5ia" From: Anna-Maria Behnsen DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linutronix.de; s=2020; t=1728462561; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=ZvIhrKxr/EBu+615Dq6VhsxlBFePOPxcYzgUF7RjckE=; b=yI9twtJfjevAR9uwqAxgaY2P1wm+7Qqx2lfs8yZy4aq/YqhY4CjEongoQEZV4etCV5rWSz vjf+XJCLXzTZaIVoQUvpmyiZ9+gHCysUOkVgtvXC/42RTcnwSCI0KI7YfwqjbHnWRRJa5t kdCPIk/klqK3qA9v4GZKYFQr9xPBLJDKDMtEfqt3DCDOlKpImP2oRPcTcjx2RV6ofyHGQC OPlB70UechcBt7869bcjYvuTms08dVNJDNpBbBVLpbY+RAnfRTrohMmoIg+yLdkPta7imW o7kwghbK9MmRXVy8P9TsVuH20vNHGbo6b6vIeeIAmJnaalmav99MunkJxMj1sw== DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=linutronix.de; s=2020e; t=1728462561; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=ZvIhrKxr/EBu+615Dq6VhsxlBFePOPxcYzgUF7RjckE=; b=GNXTF5iaHL+6h7QUImKiW8UsaqAtt2UtULu0GR4hZcNXdI/9Z3ooNRa1umSq3kcBc9mFTm 6DDtG5oGBMiAkwBA== Date: Wed, 09 Oct 2024 10:29:16 +0200 Subject: [PATCH v2 23/25] timekeeping: Rework do_adjtimex() to use shadow_timekeeper Precedence: bulk X-Mailing-List: netdev@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Message-Id: <20241009-devel-anna-maria-b4-timers-ptp-timekeeping-v2-23-554456a44a15@linutronix.de> References: <20241009-devel-anna-maria-b4-timers-ptp-timekeeping-v2-0-554456a44a15@linutronix.de> In-Reply-To: <20241009-devel-anna-maria-b4-timers-ptp-timekeeping-v2-0-554456a44a15@linutronix.de> To: John Stultz , Frederic Weisbecker , Thomas Gleixner Cc: linux-kernel@vger.kernel.org, netdev@vger.kernel.org, Miroslav Lichvar , Richard Cochran , Christopher S Hall , Anna-Maria Behnsen From: Anna-Maria Behnsen Updates of the timekeeper can be done by operating on the shadow timekeeper and afterwards copying the result into the real timekeeper. This has the advantage, that the sequence count write protected region is kept as small as possible. Convert do_adjtimex() to use this scheme and take the opportunity to use a scoped_guard() for locking. That requires to have a separate function for updating the leap state so that the update is protected by the sequence count. This also brings the timekeeper and the shadow timekeeper in sync for this state, which was not the case so far. That's not a correctness problem as the state is only used at the read sides which use the real timekeeper, but it's inconsistent nevertheless. Signed-off-by: Anna-Maria Behnsen Acked-by: John Stultz --- kernel/time/timekeeping.c | 40 ++++++++++++++++++++++++---------------- 1 file changed, 24 insertions(+), 16 deletions(-) diff --git a/kernel/time/timekeeping.c b/kernel/time/timekeeping.c index e15e843ba2b8..b2683a589470 100644 --- a/kernel/time/timekeeping.c +++ b/kernel/time/timekeeping.c @@ -722,6 +722,18 @@ static inline void tk_update_leap_state(struct timekeeper *tk) tk->next_leap_ktime = ktime_sub(tk->next_leap_ktime, tk->offs_real); } +/* + * Leap state update for both shadow and the real timekeeper + * Separate to spare a full memcpy() of the timekeeper. + */ +static void tk_update_leap_state_all(struct tk_data *tkd) +{ + write_seqcount_begin(&tkd->seq); + tk_update_leap_state(&tkd->shadow_timekeeper); + tkd->timekeeper.next_leap_ktime = tkd->shadow_timekeeper.next_leap_ktime; + write_seqcount_end(&tkd->seq); +} + /* * Update the ktime_t based scalar nsec members of the timekeeper */ @@ -2537,13 +2549,11 @@ EXPORT_SYMBOL_GPL(random_get_entropy_fallback); */ int do_adjtimex(struct __kernel_timex *txc) { - struct timekeeper *tk = &tk_core.timekeeper; + struct timekeeper *tk = &tk_core.shadow_timekeeper; struct audit_ntp_data ad; bool offset_set = false; bool clock_set = false; struct timespec64 ts; - unsigned long flags; - s32 orig_tai, tai; int ret; /* Validate the data before disabling interrupts */ @@ -2571,23 +2581,21 @@ int do_adjtimex(struct __kernel_timex *txc) ktime_get_real_ts64(&ts); add_device_randomness(&ts, sizeof(ts)); - raw_spin_lock_irqsave(&tk_core.lock, flags); - write_seqcount_begin(&tk_core.seq); + scoped_guard (raw_spinlock_irqsave, &tk_core.lock) { + s32 orig_tai, tai; - orig_tai = tai = tk->tai_offset; - ret = __do_adjtimex(txc, &ts, &tai, &ad); + orig_tai = tai = tk->tai_offset; + ret = __do_adjtimex(txc, &ts, &tai, &ad); - if (tai != orig_tai) { - __timekeeping_set_tai_offset(tk, tai); - timekeeping_update(&tk_core, tk, TK_MIRROR | TK_CLOCK_WAS_SET); - clock_set = true; - } else { - tk_update_leap_state(tk); + if (tai != orig_tai) { + __timekeeping_set_tai_offset(tk, tai); + timekeeping_update_staged(&tk_core, TK_CLOCK_WAS_SET); + clock_set = true; + } else { + tk_update_leap_state_all(&tk_core); + } } - write_seqcount_end(&tk_core.seq); - raw_spin_unlock_irqrestore(&tk_core.lock, flags); - audit_ntp_log(&ad); /* Update the multiplier immediately if frequency was set directly */ From patchwork Wed Oct 9 08:29:17 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Anna-Maria Behnsen X-Patchwork-Id: 13827758 Received: from galois.linutronix.de (Galois.linutronix.de [193.142.43.55]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id C289618FC86; Wed, 9 Oct 2024 08:29:23 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=193.142.43.55 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1728462566; cv=none; b=VR+Hek9FOdB8+voyCo05D9cHJiQ65cGE+/voc8clvtF77yOScdG34m9LRxffm9nwzDEJMviINDsnd7ZUGNtqKMfo1Q0nMteqbbzKy+IqIhcGy1HA0nq83iCn5QatHZnwseb0WwcaKCsUQNZNqhoC7CiXBjs2buw4XYLZZRRBI6Y= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1728462566; c=relaxed/simple; bh=yFxmegOrxfY1kdAuzvtcvTabjp1FL3Sf++7UBkCOy74=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=jhc7cCcwya0pncefzNXpMAKfWKKQhtvnCZPH3/U6xLf+MLYRNXhwSL1qOQ68hsyhfZTb3jD2LuqB/3gtFMwfYw3DJPGMLQGvzN6ZwVT+5/gqgzLqA3xDtRCX8ZQH0zT3OI2mr9Yn7kls1f6F7rMy40V/lXx3SHMLc0a2e1BiErY= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linutronix.de; spf=pass smtp.mailfrom=linutronix.de; dkim=pass (2048-bit key) header.d=linutronix.de header.i=@linutronix.de header.b=aWgT2Gn1; dkim=permerror (0-bit key) header.d=linutronix.de header.i=@linutronix.de header.b=9h2b0jNg; arc=none smtp.client-ip=193.142.43.55 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linutronix.de Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linutronix.de Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=linutronix.de header.i=@linutronix.de header.b="aWgT2Gn1"; dkim=permerror (0-bit key) header.d=linutronix.de header.i=@linutronix.de header.b="9h2b0jNg" From: Anna-Maria Behnsen DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linutronix.de; s=2020; t=1728462561; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=eoUlvD35oQHkwv1qwQ5WsrLRNLxDye46TStt/7cTBQ0=; b=aWgT2Gn1kwkB9vpNC5asC5vrvcckmA1e/PVN18k8Bap6QlNnYtuPy55rlXrvzq35IjYUCG Pjs9yJ//wrN+d7JeeBjCruYObcU3hZQ3+vIV36rkBIeMrAq/IPOYWvsXKAXlSBftjP06UT /C4qLJGqpPZsD35tgYi2+VjHycPwENu1hAnRXMzjBdqazJaPE7p9v/7XcxOoGfqwzpBOii C50FIILW094PIj48CISkumqYSWGTsMIUlu3DF/RoMULRMLa6AX3RBPPhOTkqMtZMQgllyc qzDIRC6yAdiLQ6nU9rk4AU60QWRmLngQPPXQ72/8GuQW8TVK6bBVKZJwzh2PhA== DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=linutronix.de; s=2020e; t=1728462561; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=eoUlvD35oQHkwv1qwQ5WsrLRNLxDye46TStt/7cTBQ0=; b=9h2b0jNg4VFKJcvnAZRb/vs7QkqWXE/03Pr5h+kW31TrbI8FE/QAOhfrf++aoxNAsHhteU QKCxOHQ77CBZPMDg== Date: Wed, 09 Oct 2024 10:29:17 +0200 Subject: [PATCH v2 24/25] timekeeping: Remove TK_MIRROR timekeeping_update() action Precedence: bulk X-Mailing-List: netdev@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Message-Id: <20241009-devel-anna-maria-b4-timers-ptp-timekeeping-v2-24-554456a44a15@linutronix.de> References: <20241009-devel-anna-maria-b4-timers-ptp-timekeeping-v2-0-554456a44a15@linutronix.de> In-Reply-To: <20241009-devel-anna-maria-b4-timers-ptp-timekeeping-v2-0-554456a44a15@linutronix.de> To: John Stultz , Frederic Weisbecker , Thomas Gleixner Cc: linux-kernel@vger.kernel.org, netdev@vger.kernel.org, Miroslav Lichvar , Richard Cochran , Christopher S Hall , Anna-Maria Behnsen From: Anna-Maria Behnsen All call sites of using TK_MIRROR flag in timekeeping_update() are gone. The TK_MIRROR dependent code path is therefore dead code. Remove it along with the TK_MIRROR define. Signed-off-by: Anna-Maria Behnsen Acked-by: John Stultz --- kernel/time/timekeeping.c | 10 +--------- 1 file changed, 1 insertion(+), 9 deletions(-) diff --git a/kernel/time/timekeeping.c b/kernel/time/timekeeping.c index b2683a589470..67d7be2e02fb 100644 --- a/kernel/time/timekeeping.c +++ b/kernel/time/timekeeping.c @@ -30,8 +30,7 @@ #include "timekeeping_internal.h" #define TK_CLEAR_NTP (1 << 0) -#define TK_MIRROR (1 << 1) -#define TK_CLOCK_WAS_SET (1 << 2) +#define TK_CLOCK_WAS_SET (1 << 1) #define TK_UPDATE_ALL (TK_CLEAR_NTP | TK_CLOCK_WAS_SET) @@ -797,13 +796,6 @@ static void timekeeping_update(struct tk_data *tkd, struct timekeeper *tk, unsig if (action & TK_CLOCK_WAS_SET) tk->clock_was_set_seq++; - /* - * The mirroring of the data to the shadow-timekeeper needs - * to happen last here to ensure we don't over-write the - * timekeeper structure on the next update with stale data - */ - if (action & TK_MIRROR) - timekeeping_restore_shadow(tkd); } static void timekeeping_update_staged(struct tk_data *tkd, unsigned int action) From patchwork Wed Oct 9 08:29:18 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Anna-Maria Behnsen X-Patchwork-Id: 13827760 Received: from galois.linutronix.de (Galois.linutronix.de [193.142.43.55]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 579DF18FDD5; Wed, 9 Oct 2024 08:29:24 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=193.142.43.55 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1728462566; cv=none; b=WaLlBkuqM0EwZ9nJs5T5kA6sltcVwiFn3htNdndh0xRGc/29zUgYO2uxaO1bAz5hxcuIg27coN9sVe4x6cgU9YH88tbEqOLZiaaU8GGTMtcDv7sTbICUfG+E82x0p09fFW7Yc8HzE2qjcTWOfChFI6Z5U4jBFtYegUi72SpnKbI= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1728462566; c=relaxed/simple; bh=L3svMwNfFnD1IGZWVUvu4iABd7NGGztlzGb8Neh4W3k=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=cOAt95t+dT1oyIGkBGTOLtABIM73UNvroZT/omOmgliiUSW2lIK4G7AD9Pr4qpOybskQKvl47qwvvcKM88jMRM8CXYQyEYC94r5MP2m3Zz3ygc0JKUAPoGCqcUlGfpWQ3kocPqFAKRsuzr5mBC9lQ4Vj8Aqs76BOhaXg4d0x+zU= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linutronix.de; spf=pass smtp.mailfrom=linutronix.de; dkim=pass (2048-bit key) header.d=linutronix.de header.i=@linutronix.de header.b=FcOqk1Wp; dkim=permerror (0-bit key) header.d=linutronix.de header.i=@linutronix.de header.b=PQX6gGdz; arc=none smtp.client-ip=193.142.43.55 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linutronix.de Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linutronix.de Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=linutronix.de header.i=@linutronix.de header.b="FcOqk1Wp"; dkim=permerror (0-bit key) header.d=linutronix.de header.i=@linutronix.de header.b="PQX6gGdz" From: Anna-Maria Behnsen DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linutronix.de; s=2020; t=1728462561; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=NoPilZctoG6Xm/IewLUEVe8c/+zKsk2WG02YLl9fzmk=; b=FcOqk1WpSdSk/Hm5ralNVoLRh3OjL8kItRcUbfOFdpaRG/8FPctXGqfXXYeznkDezYvmMs 8YRzxini+77nLDyrZU0/ejuDJ+zgTqVDYJN0DINc6oLUlFhSg+1P/Zi6JQi2O2+lr12QMJ HFruOVXNqdfUUvllESfZexTUQAepa8sT/LsBU3aHxX4RM41LKGPEuYN9tuYbDUUtOQKX7A B0iFaWmEtEgvUTMkSlUJSzJYJu0yBloRLWFmEJ/5s+yk5rM7e1OhsDWRA/MjR9WTZKBIJd dwqIUhGxQw/EJ4Qn/Qgj+rO6apxQ5HXdrL2It/v2RCpwWUUXR/PpptLqYbPpeQ== DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=linutronix.de; s=2020e; t=1728462561; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=NoPilZctoG6Xm/IewLUEVe8c/+zKsk2WG02YLl9fzmk=; b=PQX6gGdzlYhcFJz/OcTuPbG7p1ZKn6wCk+8eHaElrh5qPC513Ba0thrTGbwFvIdLHQ+TuS Eqc/f+Qn3NU7mKCw== Date: Wed, 09 Oct 2024 10:29:18 +0200 Subject: [PATCH v2 25/25] timekeeping: Merge timekeeping_update_staged() and timekeeping_update() Precedence: bulk X-Mailing-List: netdev@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Message-Id: <20241009-devel-anna-maria-b4-timers-ptp-timekeeping-v2-25-554456a44a15@linutronix.de> References: <20241009-devel-anna-maria-b4-timers-ptp-timekeeping-v2-0-554456a44a15@linutronix.de> In-Reply-To: <20241009-devel-anna-maria-b4-timers-ptp-timekeeping-v2-0-554456a44a15@linutronix.de> To: John Stultz , Frederic Weisbecker , Thomas Gleixner Cc: linux-kernel@vger.kernel.org, netdev@vger.kernel.org, Miroslav Lichvar , Richard Cochran , Christopher S Hall , Anna-Maria Behnsen From: Anna-Maria Behnsen timekeeping_update_staged() is the only call site of timekeeping_update(). Merge those functions. No functional change. Signed-off-by: Anna-Maria Behnsen Acked-by: John Stultz --- kernel/time/timekeeping.c | 31 ++++++++++++++----------------- 1 file changed, 14 insertions(+), 17 deletions(-) diff --git a/kernel/time/timekeeping.c b/kernel/time/timekeeping.c index 67d7be2e02fb..d07eb1946ff1 100644 --- a/kernel/time/timekeeping.c +++ b/kernel/time/timekeeping.c @@ -529,7 +529,7 @@ EXPORT_SYMBOL_GPL(ktime_get_raw_fast_ns); * timekeeping_inject_sleeptime64() * __timekeeping_inject_sleeptime(tk, delta); * timestamp(); - * timekeeping_update(tkd, tk, TK_CLEAR_NTP...); + * timekeeping_update_staged(tkd, TK_CLEAR_NTP...); * * (2) On 32-bit systems, the 64-bit boot offset (tk->offs_boot) may be * partially updated. Since the tk->offs_boot update is a rare event, this @@ -775,10 +775,21 @@ static void timekeeping_restore_shadow(struct tk_data *tkd) memcpy(&tkd->shadow_timekeeper, &tkd->timekeeper, sizeof(tkd->timekeeper)); } -static void timekeeping_update(struct tk_data *tkd, struct timekeeper *tk, unsigned int action) +static void timekeeping_update_staged(struct tk_data *tkd, unsigned int action) { + struct timekeeper *tk = &tk_core.shadow_timekeeper; + lockdep_assert_held(&tkd->lock); + /* + * Block out readers before running the updates below because that + * updates VDSO and other time related infrastructure. Not blocking + * the readers might let a reader see time going backwards when + * reading from the VDSO after the VDSO update and then reading in + * the kernel from the timekeeper before that got updated. + */ + write_seqcount_begin(&tkd->seq); + if (action & TK_CLEAR_NTP) { tk->ntp_error = 0; ntp_clear(); @@ -796,20 +807,6 @@ static void timekeeping_update(struct tk_data *tkd, struct timekeeper *tk, unsig if (action & TK_CLOCK_WAS_SET) tk->clock_was_set_seq++; -} - -static void timekeeping_update_staged(struct tk_data *tkd, unsigned int action) -{ - /* - * Block out readers before invoking timekeeping_update() because - * that updates VDSO and other time related infrastructure. Not - * blocking the readers might let a reader see time going backwards - * when reading from the VDSO after the VDSO update and then - * reading in the kernel from the timekeeper before that got updated. - */ - write_seqcount_begin(&tkd->seq); - - timekeeping_update(tkd, &tkd->shadow_timekeeper, action); /* * Update the real timekeeper. @@ -819,7 +816,7 @@ static void timekeeping_update_staged(struct tk_data *tkd, unsigned int action) * the cacheline optimized data layout of the timekeeper and requires * another indirection. */ - memcpy(&tkd->timekeeper, &tkd->shadow_timekeeper, sizeof(tkd->shadow_timekeeper)); + memcpy(&tkd->timekeeper, tk, sizeof(*tk)); write_seqcount_end(&tkd->seq); }