| Message ID | 20230630171052.985577-5-peter.hilber@opensynergy.com (mailing list archive) |
|---|---|
| State | New, archived |
| Headers | show |
| Series | Add virtio_rtc module and related changes | expand |
On Fri, 30 Jun 2023 18:10:47 +0100, Peter Hilber <peter.hilber@opensynergy.com> wrote: > > Export helper functions to allow other code to > > - determine the counter type in use (virtual or physical, CP15 or memory), > > - get a pointer to the arm_arch_timer clocksource, which can be compared > with the current clocksource. > > The virtio_rtc driver will require the clocksource pointer when using > get_device_system_crosststamp(), and should communicate the actual Arm > counter type to the Virtio RTC device (cf. spec draft [1]). I really don't see why you should poke at the clocksource backend: - the MMIO clocksource is only used in PM situations, which a virtio driver has no business being involved with - only the virtual counter is relevant -- it is always at a 0-offset from the physical one when userspace has an opportunity to run So it really looks that out of the four combinations, only one is relevant. I'm not Cc'd on the rest of the series, so I can't even see in which context this is used. But as it is, the approach looks wrong. Thanks, M.
On 03.07.23 10:13, Marc Zyngier wrote: > On Fri, 30 Jun 2023 18:10:47 +0100, > Peter Hilber <peter.hilber@opensynergy.com> wrote: >> >> Export helper functions to allow other code to >> >> - determine the counter type in use (virtual or physical, CP15 or memory), >> >> - get a pointer to the arm_arch_timer clocksource, which can be compared >> with the current clocksource. >> >> The virtio_rtc driver will require the clocksource pointer when using >> get_device_system_crosststamp(), and should communicate the actual Arm >> counter type to the Virtio RTC device (cf. spec draft [1]). > > I really don't see why you should poke at the clocksource backend: > > - the MMIO clocksource is only used in PM situations, which a virtio > driver has no business being involved with > > - only the virtual counter is relevant -- it is always at a 0-offset > from the physical one when userspace has an opportunity to run > > So it really looks that out of the four combinations, only one is > relevant. Thanks for the explanation. Dropping arch_timer_counter_get_type() and assuming that the CP15 virtual counter is in use should also work for now. With the physical/virtual counter distinction, I tried to be future-proof, as per the following considerations: The intended consumer of arch_timer_counter_get_type() is the Virtio RTC device (draft spec [2], patch series [1]). If the Virtio device has optional cross-timestamp support, it must know the current Linux kernel view of the current clocksource counter. The Virtio driver tells the Virtio device the current counter type (in the Arm case, CNTVCT_EL0 or CNTPCT_EL0). It is intentionally left unspecified how the Virtio device would know the counter value. AFAIU, if the Linux kernel were a virtualization host itself, it would be better for the Virtio device to look at the physical counter, since the virtual counter could be set for a guest. And in other cases, the guest OSes use a virtual counter with an offset. This was the rationale to come up with the physical/virtual counter distinction for the Virtio RTC device. Looking at extensions such as FEAT_ECV, where the CNTPCT_EL0 value can depend on the EL, or FEAT_NV*, it might be a bit simplistic. Does this physical/virtual counter distinction sound like a good idea? Otherwise I would drop the arch_timer_counter_get_type() in the next iteration. > > I'm not Cc'd on the rest of the series, so I can't even see in which > context this is used. But as it is, the approach looks wrong. > Sorry, I will Cc you on all relevant patches in the next iteration, which I will send out soon. The first patch series can be found at [1]. I think the second helper function in this patch, arch_timer_get_cs(), would still be needed, in order to supply the clocksource to get_device_system_crosststamp(). Thanks for the comments, Peter [1] https://lore.kernel.org/lkml/20230630171052.985577-1-peter.hilber@opensynergy.com/T/#me7df2d4db4fe1119d821dc9c4054b9404c15b02d [2] https://lists.oasis-open.org/archives/virtio-comment/202306/msg00592.html
On Thu, 27 Jul 2023 11:22:11 +0100, Peter Hilber <peter.hilber@opensynergy.com> wrote: > > On 03.07.23 10:13, Marc Zyngier wrote: > > On Fri, 30 Jun 2023 18:10:47 +0100, > > Peter Hilber <peter.hilber@opensynergy.com> wrote: > >> > >> Export helper functions to allow other code to > >> > >> - determine the counter type in use (virtual or physical, CP15 or memory), > >> > >> - get a pointer to the arm_arch_timer clocksource, which can be compared > >> with the current clocksource. > >> > >> The virtio_rtc driver will require the clocksource pointer when using > >> get_device_system_crosststamp(), and should communicate the actual Arm > >> counter type to the Virtio RTC device (cf. spec draft [1]). > > > > I really don't see why you should poke at the clocksource backend: > > > > - the MMIO clocksource is only used in PM situations, which a virtio > > driver has no business being involved with > > > > - only the virtual counter is relevant -- it is always at a 0-offset > > from the physical one when userspace has an opportunity to run > > > > So it really looks that out of the four combinations, only one is > > relevant. > > Thanks for the explanation. Dropping arch_timer_counter_get_type() and > assuming that the CP15 virtual counter is in use should also work for > now. With the physical/virtual counter distinction, I tried to be > future-proof, as per the following considerations: > > The intended consumer of arch_timer_counter_get_type() is the Virtio RTC > device (draft spec [2], patch series [1]). If the Virtio device has > optional cross-timestamp support, it must know the current Linux kernel > view of the current clocksource counter. The Virtio driver tells the > Virtio device the current counter type (in the Arm case, CNTVCT_EL0 or > CNTPCT_EL0). It is intentionally left unspecified how the Virtio device > would know the counter value. AFAIU, if the Linux kernel were a > virtualization host itself, it would be better for the Virtio device to > look at the physical counter, since the virtual counter could be set for > a guest. And in other cases, the guest OSes use a virtual counter with > an offset. The physical counter can equally be offset (and KVM does offset it), just like the virtual counter. With NV, the offsets themselves are partially under control of the guest itself. So either counters *as seen from the guest* are absolutely pointless to an observer on the host. That observer sees a virtual counter that is strictly equal to the physical counter. > This was the rationale to come up with the physical/virtual counter > distinction for the Virtio RTC device. Looking at extensions such as > FEAT_ECV, where the CNTPCT_EL0 value can depend on the EL, or FEAT_NV*, > it might be a bit simplistic. Not just simplistic. It doesn't make sense. For this to work, you'd need to know the global offset that KVM applies to the global counter, plus the *virtualised* CNTPOFF/CNTVOFF values that the guest can change at any time on a *per-CPU* basis. None of that is available outside of KVM, nor would it make any sense anyway. > Does this physical/virtual counter distinction sound like a good idea? > Otherwise I would drop the arch_timer_counter_get_type() in the next > iteration. My take on this is that only the global counter value makes any sense. That value is already available from the host as the virtual counter, because we guarantee that CNTVOFF is 0 when outside of the guest (well, technically, outside of the vcpu_load/vcpu_put section). > > > > > I'm not Cc'd on the rest of the series, so I can't even see in which > > context this is used. But as it is, the approach looks wrong. > > > > Sorry, I will Cc you on all relevant patches in the next iteration, > which I will send out soon. > > The first patch series can be found at [1]. I think the second helper > function in this patch, arch_timer_get_cs(), would still be needed, in > order to supply the clocksource to get_device_system_crosststamp(). We already have to deal with the kvm_arch_ptp_get_crosststamp() monstrosity (which I will forever regret having merged). Surely you can reuse some of that? Thanks, M.
On 28.07.23 10:11, Marc Zyngier wrote: > On Thu, 27 Jul 2023 11:22:11 +0100, > Peter Hilber <peter.hilber@opensynergy.com> wrote: >> >> On 03.07.23 10:13, Marc Zyngier wrote: >>> On Fri, 30 Jun 2023 18:10:47 +0100, >>> Peter Hilber <peter.hilber@opensynergy.com> wrote: >>>> >>>> Export helper functions to allow other code to >>>> >>>> - determine the counter type in use (virtual or physical, CP15 or memory), >>>> >>>> - get a pointer to the arm_arch_timer clocksource, which can be compared >>>> with the current clocksource. >>>> >>>> The virtio_rtc driver will require the clocksource pointer when using >>>> get_device_system_crosststamp(), and should communicate the actual Arm >>>> counter type to the Virtio RTC device (cf. spec draft [1]). >>> >>> I really don't see why you should poke at the clocksource backend: >>> >>> - the MMIO clocksource is only used in PM situations, which a virtio >>> driver has no business being involved with >>> >>> - only the virtual counter is relevant -- it is always at a 0-offset >>> from the physical one when userspace has an opportunity to run >>> >>> So it really looks that out of the four combinations, only one is >>> relevant. >> >> Thanks for the explanation. Dropping arch_timer_counter_get_type() and >> assuming that the CP15 virtual counter is in use should also work for >> now. With the physical/virtual counter distinction, I tried to be >> future-proof, as per the following considerations: >> >> The intended consumer of arch_timer_counter_get_type() is the Virtio RTC >> device (draft spec [2], patch series [1]). If the Virtio device has >> optional cross-timestamp support, it must know the current Linux kernel >> view of the current clocksource counter. The Virtio driver tells the >> Virtio device the current counter type (in the Arm case, CNTVCT_EL0 or >> CNTPCT_EL0). It is intentionally left unspecified how the Virtio device >> would know the counter value. AFAIU, if the Linux kernel were a >> virtualization host itself, it would be better for the Virtio device to >> look at the physical counter, since the virtual counter could be set for >> a guest. And in other cases, the guest OSes use a virtual counter with >> an offset. > > The physical counter can equally be offset (and KVM does offset it), > just like the virtual counter. With NV, the offsets themselves are > partially under control of the guest itself. > > So either counters *as seen from the guest* are absolutely pointless > to an observer on the host. That observer sees a virtual counter that > is strictly equal to the physical counter. > >> This was the rationale to come up with the physical/virtual counter >> distinction for the Virtio RTC device. Looking at extensions such as >> FEAT_ECV, where the CNTPCT_EL0 value can depend on the EL, or FEAT_NV*, >> it might be a bit simplistic. > > Not just simplistic. It doesn't make sense. For this to work, you'd > need to know the global offset that KVM applies to the global counter, > plus the *virtualised* CNTPOFF/CNTVOFF values that the guest can > change at any time on a *per-CPU* basis. None of that is available > outside of KVM, nor would it make any sense anyway. > >> Does this physical/virtual counter distinction sound like a good idea? >> Otherwise I would drop the arch_timer_counter_get_type() in the next >> iteration. > > My take on this is that only the global counter value makes any sense. > That value is already available from the host as the virtual counter, > because we guarantee that CNTVOFF is 0 when outside of the guest > (well, technically, outside of the vcpu_load/vcpu_put section). > OK, I agree that a virtual/physical counter distinction doesn't make sense on Linux (unless one wants to abuse it to distinguish very special use cases with and without Linux). Probably I'll also remove the distinction from the spec draft. >> >>> >>> I'm not Cc'd on the rest of the series, so I can't even see in which >>> context this is used. But as it is, the approach looks wrong. >>> >> >> Sorry, I will Cc you on all relevant patches in the next iteration, >> which I will send out soon. >> >> The first patch series can be found at [1]. I think the second helper >> function in this patch, arch_timer_get_cs(), would still be needed, in >> order to supply the clocksource to get_device_system_crosststamp(). > > We already have to deal with the kvm_arch_ptp_get_crosststamp() > monstrosity (which I will forever regret having merged). Surely you > can reuse some of that? > I'm not sure what this is hinting at. From the virtio_rtc perspective, the only behavior shared with kvm_arch_ptp_get_crosststamp() would be exposing &clocksource_counter (and distinguishing virtual/physical counter, but virtio_rtc should stop doing this). Exposing &clocksource_counter is also the only thing arch_timer_get_cs() does. If &clocksource_counter should not be exposed, then I can see two alternatives: Alternative 1: Put a function of type int (*get_time_fn) (ktime_t *device_time, struct system_counterval_t *sys_counterval, void *ctx) into arm_arch_timer.c, as required by get_device_system_crosststamp() (and include a virtio_rtc header). Alternative 2: Change get_device_system_crosststamp(), resp. struct system_counterval_t, to use enum clocksource_ids to identify a clocksource, instead of using struct clocksource *. Then there would be no need to change arm_arch_timer. Thanks for the feedback, Peter
diff --git a/drivers/clocksource/arm_arch_timer.c b/drivers/clocksource/arm_arch_timer.c index e733a2a1927a..cebdc1b2db4c 100644 --- a/drivers/clocksource/arm_arch_timer.c +++ b/drivers/clocksource/arm_arch_timer.c @@ -92,6 +92,7 @@ static enum vdso_clock_mode vdso_default = VDSO_CLOCKMODE_ARCHTIMER; #else static enum vdso_clock_mode vdso_default = VDSO_CLOCKMODE_NONE; #endif /* CONFIG_GENERIC_GETTIMEOFDAY */ +static enum arch_timer_counter_type arch_counter_type __ro_after_init = ARCH_COUNTER_CP15_VIRT; static cpumask_t evtstrm_available = CPU_MASK_NONE; static bool evtstrm_enable __ro_after_init = IS_ENABLED(CONFIG_ARM_ARCH_TIMER_EVTSTREAM); @@ -1109,6 +1110,7 @@ static void __init arch_counter_register(unsigned type) rd = arch_counter_get_cntvct; scr = arch_counter_get_cntvct; } + arch_counter_type = ARCH_COUNTER_CP15_VIRT; } else { if (arch_timer_counter_has_wa()) { rd = arch_counter_get_cntpct_stable; @@ -1117,6 +1119,7 @@ static void __init arch_counter_register(unsigned type) rd = arch_counter_get_cntpct; scr = arch_counter_get_cntpct; } + arch_counter_type = ARCH_COUNTER_CP15_PHYS; } arch_timer_read_counter = rd; @@ -1124,6 +1127,7 @@ static void __init arch_counter_register(unsigned type) } else { arch_timer_read_counter = arch_counter_get_cntvct_mem; scr = arch_counter_get_cntvct_mem; + arch_counter_type = ARCH_COUNTER_MEM_VIRT; } width = arch_counter_get_width(); @@ -1777,6 +1781,18 @@ static int __init arch_timer_acpi_init(struct acpi_table_header *table) TIMER_ACPI_DECLARE(arch_timer, ACPI_SIG_GTDT, arch_timer_acpi_init); #endif +enum arch_timer_counter_type arch_timer_counter_get_type(void) +{ + return arch_counter_type; +} +EXPORT_SYMBOL_GPL(arch_timer_counter_get_type); + +struct clocksource *arch_timer_get_cs(void) +{ + return &clocksource_counter; +} +EXPORT_SYMBOL_GPL(arch_timer_get_cs); + int kvm_arch_ptp_get_crosststamp(u64 *cycle, struct timespec64 *ts, struct clocksource **cs) { diff --git a/include/clocksource/arm_arch_timer.h b/include/clocksource/arm_arch_timer.h index cbbc9a6dc571..b442db0b5ca0 100644 --- a/include/clocksource/arm_arch_timer.h +++ b/include/clocksource/arm_arch_timer.h @@ -43,6 +43,13 @@ enum arch_timer_spi_nr { ARCH_TIMER_MAX_TIMER_SPI }; +enum arch_timer_counter_type { + ARCH_COUNTER_CP15_VIRT, + ARCH_COUNTER_CP15_PHYS, + ARCH_COUNTER_MEM_VIRT, + ARCH_COUNTER_MEM_PHYS, +}; + #define ARCH_TIMER_PHYS_ACCESS 0 #define ARCH_TIMER_VIRT_ACCESS 1 #define ARCH_TIMER_MEM_PHYS_ACCESS 2 @@ -89,6 +96,8 @@ extern u32 arch_timer_get_rate(void); extern u64 (*arch_timer_read_counter)(void); extern struct arch_timer_kvm_info *arch_timer_get_kvm_info(void); extern bool arch_timer_evtstrm_available(void); +extern enum arch_timer_counter_type arch_timer_counter_get_type(void); +extern struct clocksource *arch_timer_get_cs(void); #else @@ -107,6 +116,16 @@ static inline bool arch_timer_evtstrm_available(void) return false; } +static inline enum arch_timer_counter_type arch_timer_counter_get_type(void) +{ + return ARCH_COUNTER_CP15_VIRT; +} + +static inline struct clocksource *arch_timer_get_cs(void) +{ + return NULL; +} + #endif #endif
Export helper functions to allow other code to - determine the counter type in use (virtual or physical, CP15 or memory), - get a pointer to the arm_arch_timer clocksource, which can be compared with the current clocksource. The virtio_rtc driver will require the clocksource pointer when using get_device_system_crosststamp(), and should communicate the actual Arm counter type to the Virtio RTC device (cf. spec draft [1]). [1] https://lists.oasis-open.org/archives/virtio-comment/202306/msg00592.html Signed-off-by: Peter Hilber <peter.hilber@opensynergy.com> --- drivers/clocksource/arm_arch_timer.c | 16 ++++++++++++++++ include/clocksource/arm_arch_timer.h | 19 +++++++++++++++++++ 2 files changed, 35 insertions(+)