| Message ID | 20220308142410.3193729-4-vincent.whitchurch@axis.com (mailing list archive) |
|---|---|
| State | New, archived |
| Headers | show |
| Series | clocksource: Add MCT support for ARTPEC-8 | expand |
On 08/03/2022 15:24, Vincent Whitchurch wrote: > Support the documented semantics of the local-timer-index property: Use > it as the first index of the local timer, ensure that global timer clock > events device is not registered, and don't write to the global FRC if it > is already started. > > Signed-off-by: Vincent Whitchurch <vincent.whitchurch@axis.com> > --- > > Notes: > v2: Use devicetree property instead of module parameter. > > drivers/clocksource/exynos_mct.c | 25 ++++++++++++++++++++----- > 1 file changed, 20 insertions(+), 5 deletions(-) > > diff --git a/drivers/clocksource/exynos_mct.c b/drivers/clocksource/exynos_mct.c > index f29c812b70c9..5f8b516614eb 100644 > --- a/drivers/clocksource/exynos_mct.c > +++ b/drivers/clocksource/exynos_mct.c > @@ -33,7 +33,7 @@ > #define EXYNOS4_MCT_G_INT_ENB EXYNOS4_MCTREG(0x248) > #define EXYNOS4_MCT_G_WSTAT EXYNOS4_MCTREG(0x24C) > #define _EXYNOS4_MCT_L_BASE EXYNOS4_MCTREG(0x300) > -#define EXYNOS4_MCT_L_BASE(x) (_EXYNOS4_MCT_L_BASE + (0x100 * x)) > +#define EXYNOS4_MCT_L_BASE(x) (_EXYNOS4_MCT_L_BASE + (0x100 * (x))) > #define EXYNOS4_MCT_L_MASK (0xffffff00) > > #define MCT_L_TCNTB_OFFSET (0x00) > @@ -75,6 +75,7 @@ enum { > static void __iomem *reg_base; > static unsigned long clk_rate; > static unsigned int mct_int_type; > +static unsigned int mct_local_idx; No more static variables. This was wrong design, happens, but let's not grow the list. I propose to conditionally (depending on property samsung,frc-shared) assign .resume callback to NULL or exynos4_frc_resume. The init can receive an argument whether to call frc_start(). > static int mct_irqs[MCT_NR_IRQS]; > > struct mct_clock_event_device { > @@ -157,6 +158,17 @@ static void exynos4_mct_frc_start(void) > u32 reg; > > reg = readl_relaxed(reg_base + EXYNOS4_MCT_G_TCON); > + > + /* > + * If the FRC is already running, we don't need to start it again. We > + * could probably just do this on all systems, but, to avoid any risk > + * for regressions, we only do it on systems where it's absolutely > + * necessary (i.e., on systems where writes to the global registers > + * need to be avoided). > + */ > + if (mct_local_idx && (reg & MCT_G_TCON_START)) This contradicts your intentions in commit #2 msg, where you described that A53 will be started first. 1. If A53 is always started first, is it possible to be here from A5? 2. If above is possible, how do you handle locking? For example: a. A53 started with some delay, entered exynos4_mct_frc_start() pass this check; b. A5 gets to exynos4_mct_frc_start(), check is still false, so A5 enables the FRC, c. A53 also enables the FRC. Having here relaxed reads and writes makes it even worse, unfortunately. Best regards, Krzysztof
On Tue, Mar 08, 2022 at 03:57:55PM +0100, Krzysztof Kozlowski wrote: > On 08/03/2022 15:24, Vincent Whitchurch wrote: > > diff --git a/drivers/clocksource/exynos_mct.c b/drivers/clocksource/exynos_mct.c > > index f29c812b70c9..5f8b516614eb 100644 > > --- a/drivers/clocksource/exynos_mct.c > > +++ b/drivers/clocksource/exynos_mct.c > > @@ -33,7 +33,7 @@ > > #define EXYNOS4_MCT_G_INT_ENB EXYNOS4_MCTREG(0x248) > > #define EXYNOS4_MCT_G_WSTAT EXYNOS4_MCTREG(0x24C) > > #define _EXYNOS4_MCT_L_BASE EXYNOS4_MCTREG(0x300) > > -#define EXYNOS4_MCT_L_BASE(x) (_EXYNOS4_MCT_L_BASE + (0x100 * x)) > > +#define EXYNOS4_MCT_L_BASE(x) (_EXYNOS4_MCT_L_BASE + (0x100 * (x))) > > #define EXYNOS4_MCT_L_MASK (0xffffff00) > > > > #define MCT_L_TCNTB_OFFSET (0x00) > > @@ -75,6 +75,7 @@ enum { > > static void __iomem *reg_base; > > static unsigned long clk_rate; > > static unsigned int mct_int_type; > > +static unsigned int mct_local_idx; > > No more static variables. This was wrong design, happens, but let's not > grow the list. > > I propose to conditionally (depending on property samsung,frc-shared) > assign .resume callback to NULL or exynos4_frc_resume. The init can > receive an argument whether to call frc_start(). Could we just add the skip-write-register-if-already-started change in exynos4_mct_frc_start() uncondtionally? Perhaps it could be in a separate patch too? I was probably being over-cautious when I did it conditionally on mct_local_idx. Doing it uncondtionally would make it easier to remove the global variable. On my system the FRC is actually started long before Linux, and I assume it's similar on other chips. > > > static int mct_irqs[MCT_NR_IRQS]; > > > > struct mct_clock_event_device { > > @@ -157,6 +158,17 @@ static void exynos4_mct_frc_start(void) > > u32 reg; > > > > reg = readl_relaxed(reg_base + EXYNOS4_MCT_G_TCON); > > + > > + /* > > + * If the FRC is already running, we don't need to start it again. We > > + * could probably just do this on all systems, but, to avoid any risk > > + * for regressions, we only do it on systems where it's absolutely > > + * necessary (i.e., on systems where writes to the global registers > > + * need to be avoided). > > + */ > > + if (mct_local_idx && (reg & MCT_G_TCON_START)) > > This contradicts your intentions in commit #2 msg, where you described > that A53 will be started first. Yes, you're right. The case of the FRC not being running when the A5 starts up is only ever hit in our simulation environment where we are able to start Linux on the A5 directly, without having to go via the A53. > 1. If A53 is always started first, is it possible to be here from A5? > 2. If above is possible, how do you handle locking? For example: > a. A53 started with some delay, entered exynos4_mct_frc_start() pass > this check; > b. A5 gets to exynos4_mct_frc_start(), check is still false, so A5 > enables the FRC, > c. A53 also enables the FRC. The A5 is normally started from Linux on the A53 (using the remoteproc framework). This is long after exynos4_mct_frc_start() has been called on the A53.
On 11/03/2022 12:35, Vincent Whitchurch wrote: > On Tue, Mar 08, 2022 at 03:57:55PM +0100, Krzysztof Kozlowski wrote: >> On 08/03/2022 15:24, Vincent Whitchurch wrote: >>> diff --git a/drivers/clocksource/exynos_mct.c b/drivers/clocksource/exynos_mct.c >>> index f29c812b70c9..5f8b516614eb 100644 >>> --- a/drivers/clocksource/exynos_mct.c >>> +++ b/drivers/clocksource/exynos_mct.c >>> @@ -33,7 +33,7 @@ >>> #define EXYNOS4_MCT_G_INT_ENB EXYNOS4_MCTREG(0x248) >>> #define EXYNOS4_MCT_G_WSTAT EXYNOS4_MCTREG(0x24C) >>> #define _EXYNOS4_MCT_L_BASE EXYNOS4_MCTREG(0x300) >>> -#define EXYNOS4_MCT_L_BASE(x) (_EXYNOS4_MCT_L_BASE + (0x100 * x)) >>> +#define EXYNOS4_MCT_L_BASE(x) (_EXYNOS4_MCT_L_BASE + (0x100 * (x))) >>> #define EXYNOS4_MCT_L_MASK (0xffffff00) >>> >>> #define MCT_L_TCNTB_OFFSET (0x00) >>> @@ -75,6 +75,7 @@ enum { >>> static void __iomem *reg_base; >>> static unsigned long clk_rate; >>> static unsigned int mct_int_type; >>> +static unsigned int mct_local_idx; >> >> No more static variables. This was wrong design, happens, but let's not >> grow the list. >> >> I propose to conditionally (depending on property samsung,frc-shared) >> assign .resume callback to NULL or exynos4_frc_resume. The init can >> receive an argument whether to call frc_start(). > > Could we just add the skip-write-register-if-already-started change in > exynos4_mct_frc_start() uncondtionally? Perhaps it could be in a > separate patch too? I was probably being over-cautious when I did it > conditionally on mct_local_idx. Doing it uncondtionally would make it > easier to remove the global variable. > > On my system the FRC is actually started long before Linux, and I assume > it's similar on other chips. +Cc Marek, Maybe we could skip it, I don't know. It could be enabled by early boot code or by trusted firmware. This would require more testing, on few different platforms. On my Exynos5422 HC1 board the MCT is not running upon boot. The EXYNOS4_MCT_G_TCON starts with a reset value (0x0). > >> >>> static int mct_irqs[MCT_NR_IRQS]; >>> >>> struct mct_clock_event_device { >>> @@ -157,6 +158,17 @@ static void exynos4_mct_frc_start(void) >>> u32 reg; >>> >>> reg = readl_relaxed(reg_base + EXYNOS4_MCT_G_TCON); >>> + >>> + /* >>> + * If the FRC is already running, we don't need to start it again. We >>> + * could probably just do this on all systems, but, to avoid any risk >>> + * for regressions, we only do it on systems where it's absolutely >>> + * necessary (i.e., on systems where writes to the global registers >>> + * need to be avoided). >>> + */ >>> + if (mct_local_idx && (reg & MCT_G_TCON_START)) >> >> This contradicts your intentions in commit #2 msg, where you described >> that A53 will be started first. > > Yes, you're right. The case of the FRC not being running when the A5 > starts up is only ever hit in our simulation environment where we are > able to start Linux on the A5 directly, without having to go via the > A53. > >> 1. If A53 is always started first, is it possible to be here from A5? >> 2. If above is possible, how do you handle locking? For example: >> a. A53 started with some delay, entered exynos4_mct_frc_start() pass >> this check; >> b. A5 gets to exynos4_mct_frc_start(), check is still false, so A5 >> enables the FRC, >> c. A53 also enables the FRC. > > The A5 is normally started from Linux on the A53 (using the remoteproc > framework). This is long after exynos4_mct_frc_start() has been called > on the A53. If it is 100% like this, let's make it explicit - if it is A53 (lack of dedicated property), let's start it. If it A5 (property present), skip it. Let's wait for Marek thoughts, he was digging the MCT a lot. Best regards, Krzysztof
Hi Krzysztof, On 11.03.2022 13:51, Krzysztof Kozlowski wrote: > On 11/03/2022 12:35, Vincent Whitchurch wrote: >> On Tue, Mar 08, 2022 at 03:57:55PM +0100, Krzysztof Kozlowski wrote: >>> On 08/03/2022 15:24, Vincent Whitchurch wrote: >>>> diff --git a/drivers/clocksource/exynos_mct.c b/drivers/clocksource/exynos_mct.c >>>> index f29c812b70c9..5f8b516614eb 100644 >>>> --- a/drivers/clocksource/exynos_mct.c >>>> +++ b/drivers/clocksource/exynos_mct.c >>>> @@ -33,7 +33,7 @@ >>>> #define EXYNOS4_MCT_G_INT_ENB EXYNOS4_MCTREG(0x248) >>>> #define EXYNOS4_MCT_G_WSTAT EXYNOS4_MCTREG(0x24C) >>>> #define _EXYNOS4_MCT_L_BASE EXYNOS4_MCTREG(0x300) >>>> -#define EXYNOS4_MCT_L_BASE(x) (_EXYNOS4_MCT_L_BASE + (0x100 * x)) >>>> +#define EXYNOS4_MCT_L_BASE(x) (_EXYNOS4_MCT_L_BASE + (0x100 * (x))) >>>> #define EXYNOS4_MCT_L_MASK (0xffffff00) >>>> >>>> #define MCT_L_TCNTB_OFFSET (0x00) >>>> @@ -75,6 +75,7 @@ enum { >>>> static void __iomem *reg_base; >>>> static unsigned long clk_rate; >>>> static unsigned int mct_int_type; >>>> +static unsigned int mct_local_idx; >>> No more static variables. This was wrong design, happens, but let's not >>> grow the list. >>> >>> I propose to conditionally (depending on property samsung,frc-shared) >>> assign .resume callback to NULL or exynos4_frc_resume. The init can >>> receive an argument whether to call frc_start(). >> Could we just add the skip-write-register-if-already-started change in >> exynos4_mct_frc_start() uncondtionally? Perhaps it could be in a >> separate patch too? I was probably being over-cautious when I did it >> conditionally on mct_local_idx. Doing it uncondtionally would make it >> easier to remove the global variable. >> >> On my system the FRC is actually started long before Linux, and I assume >> it's similar on other chips. > +Cc Marek, > > Maybe we could skip it, I don't know. It could be enabled by early boot > code or by trusted firmware. This would require more testing, on few > different platforms. > > On my Exynos5422 HC1 board the MCT is not running upon boot. The > EXYNOS4_MCT_G_TCON starts with a reset value (0x0). > >>>> static int mct_irqs[MCT_NR_IRQS]; >>>> >>>> struct mct_clock_event_device { >>>> @@ -157,6 +158,17 @@ static void exynos4_mct_frc_start(void) >>>> u32 reg; >>>> >>>> reg = readl_relaxed(reg_base + EXYNOS4_MCT_G_TCON); >>>> + >>>> + /* >>>> + * If the FRC is already running, we don't need to start it again. We >>>> + * could probably just do this on all systems, but, to avoid any risk >>>> + * for regressions, we only do it on systems where it's absolutely >>>> + * necessary (i.e., on systems where writes to the global registers >>>> + * need to be avoided). >>>> + */ >>>> + if (mct_local_idx && (reg & MCT_G_TCON_START)) >>> This contradicts your intentions in commit #2 msg, where you described >>> that A53 will be started first. >> Yes, you're right. The case of the FRC not being running when the A5 >> starts up is only ever hit in our simulation environment where we are >> able to start Linux on the A5 directly, without having to go via the >> A53. >> >>> 1. If A53 is always started first, is it possible to be here from A5? >>> 2. If above is possible, how do you handle locking? For example: >>> a. A53 started with some delay, entered exynos4_mct_frc_start() pass >>> this check; >>> b. A5 gets to exynos4_mct_frc_start(), check is still false, so A5 >>> enables the FRC, >>> c. A53 also enables the FRC. >> The A5 is normally started from Linux on the A53 (using the remoteproc >> framework). This is long after exynos4_mct_frc_start() has been called >> on the A53. > If it is 100% like this, let's make it explicit - if it is A53 (lack of > dedicated property), let's start it. If it A5 (property present), skip it. > > Let's wait for Marek thoughts, he was digging the MCT a lot. Right, I've played a bit with MCT on some older Exynos SoCs (ARM 32bit based and even Exynos5433) and it looked that none of it enabled MCT FRC timer in their proprietary firmware. I've even proposed a patch for this once ([1]), but such approach has been rejected. I think that calling exynos4_mct_frc_start() unconditionally won't hurt. [1] https://lore.kernel.org/all/20181018095708.1527-5-m.szyprowski@samsung.com/ Best regards
On Mon, Mar 21, 2022 at 09:00:08AM +0100, Marek Szyprowski wrote: > Right, I've played a bit with MCT on some older Exynos SoCs (ARM 32bit > based and even Exynos5433) and it looked that none of it enabled MCT FRC > timer in their proprietary firmware. I've even proposed a patch for this > once ([1]), but such approach has been rejected. I think that calling > exynos4_mct_frc_start() unconditionally won't hurt. Thank you for looking into this. The proposal was however not to avoid changing when exynos4_mct_frc_start() is called, but to instead skip the write to the Timer Enable bit of the G_TCON register if it is already set, like in the below patch. (This is needed to avoid races when the FRC is shared between CPUs in an AMP configuration, since TCON can be modified for other reasons from the CPU which is using the global comparator.) If I understand your comment correctly, such a change should not cause any difference at least on the platforms you looked at since there MCT_G_TCON_START will not have been set at startup. diff --git a/drivers/clocksource/exynos_mct.c b/drivers/clocksource/exynos_mct.c index 6db3d5511b0f..ed462e0a77ff 100644 --- a/drivers/clocksource/exynos_mct.c +++ b/drivers/clocksource/exynos_mct.c @@ -162,6 +162,9 @@ static void exynos4_mct_frc_start(void) u32 reg; reg = readl_relaxed(reg_base + EXYNOS4_MCT_G_TCON); + if (reg & MCT_G_TCON_START) + return; + reg |= MCT_G_TCON_START; exynos4_mct_write(reg, EXYNOS4_MCT_G_TCON); }
On Wed, Mar 30, 2022 at 10:21:37AM +0200, Vincent Whitchurch wrote: > On Mon, Mar 21, 2022 at 09:00:08AM +0100, Marek Szyprowski wrote: > > Right, I've played a bit with MCT on some older Exynos SoCs (ARM 32bit > > based and even Exynos5433) and it looked that none of it enabled MCT FRC > > timer in their proprietary firmware. I've even proposed a patch for this > > once ([1]), but such approach has been rejected. I think that calling > > exynos4_mct_frc_start() unconditionally won't hurt. > > Thank you for looking into this. The proposal was however not to avoid > changing when exynos4_mct_frc_start() is called, but to instead skip the > write to the Timer Enable bit of the G_TCON register if it is already > set, like in the below patch. (This is needed to avoid races when the > FRC is shared between CPUs in an AMP configuration, since TCON can be > modified for other reasons from the CPU which is using the global > comparator.) > > If I understand your comment correctly, such a change should not cause > any difference at least on the platforms you looked at since there > MCT_G_TCON_START will not have been set at startup. I needed the frc-shared property anyway to prevent registration of the clock events so I followed Krzysztof's suggestion of doing this conditionally and also clearing the resume callback.
diff --git a/drivers/clocksource/exynos_mct.c b/drivers/clocksource/exynos_mct.c index f29c812b70c9..5f8b516614eb 100644 --- a/drivers/clocksource/exynos_mct.c +++ b/drivers/clocksource/exynos_mct.c @@ -33,7 +33,7 @@ #define EXYNOS4_MCT_G_INT_ENB EXYNOS4_MCTREG(0x248) #define EXYNOS4_MCT_G_WSTAT EXYNOS4_MCTREG(0x24C) #define _EXYNOS4_MCT_L_BASE EXYNOS4_MCTREG(0x300) -#define EXYNOS4_MCT_L_BASE(x) (_EXYNOS4_MCT_L_BASE + (0x100 * x)) +#define EXYNOS4_MCT_L_BASE(x) (_EXYNOS4_MCT_L_BASE + (0x100 * (x))) #define EXYNOS4_MCT_L_MASK (0xffffff00) #define MCT_L_TCNTB_OFFSET (0x00) @@ -75,6 +75,7 @@ enum { static void __iomem *reg_base; static unsigned long clk_rate; static unsigned int mct_int_type; +static unsigned int mct_local_idx; static int mct_irqs[MCT_NR_IRQS]; struct mct_clock_event_device { @@ -157,6 +158,17 @@ static void exynos4_mct_frc_start(void) u32 reg; reg = readl_relaxed(reg_base + EXYNOS4_MCT_G_TCON); + + /* + * If the FRC is already running, we don't need to start it again. We + * could probably just do this on all systems, but, to avoid any risk + * for regressions, we only do it on systems where it's absolutely + * necessary (i.e., on systems where writes to the global registers + * need to be avoided). + */ + if (mct_local_idx && (reg & MCT_G_TCON_START)) + return; + reg |= MCT_G_TCON_START; exynos4_mct_write(reg, EXYNOS4_MCT_G_TCON); } @@ -449,7 +461,7 @@ static int exynos4_mct_starting_cpu(unsigned int cpu) per_cpu_ptr(&percpu_mct_tick, cpu); struct clock_event_device *evt = &mevt->evt; - mevt->base = EXYNOS4_MCT_L_BASE(cpu); + mevt->base = EXYNOS4_MCT_L_BASE(mct_local_idx + cpu); snprintf(mevt->name, sizeof(mevt->name), "mct_tick%d", cpu); evt->name = mevt->name; @@ -554,13 +566,14 @@ static int __init exynos4_timer_interrupts(struct device_node *np, } else { for_each_possible_cpu(cpu) { int mct_irq; + unsigned int irqidx = MCT_L0_IRQ + mct_local_idx + cpu; struct mct_clock_event_device *pcpu_mevt = per_cpu_ptr(&percpu_mct_tick, cpu); pcpu_mevt->evt.irq = -1; - if (MCT_L0_IRQ + cpu >= ARRAY_SIZE(mct_irqs)) + if (irqidx >= ARRAY_SIZE(mct_irqs)) break; - mct_irq = mct_irqs[MCT_L0_IRQ + cpu]; + mct_irq = mct_irqs[irqidx]; irq_set_status_flags(mct_irq, IRQ_NOAUTOEN); if (request_irq(mct_irq, @@ -607,6 +620,8 @@ static int __init mct_init_dt(struct device_node *np, unsigned int int_type) { int ret; + of_property_read_u32(np, "local-timer-index", &mct_local_idx); + ret = exynos4_timer_resources(np); if (ret) return ret; @@ -619,7 +634,7 @@ static int __init mct_init_dt(struct device_node *np, unsigned int int_type) if (ret) return ret; - return exynos4_clockevent_init(); + return (mct_local_idx == 0) ? exynos4_clockevent_init() : ret; }
Support the documented semantics of the local-timer-index property: Use it as the first index of the local timer, ensure that global timer clock events device is not registered, and don't write to the global FRC if it is already started. Signed-off-by: Vincent Whitchurch <vincent.whitchurch@axis.com> --- Notes: v2: Use devicetree property instead of module parameter. drivers/clocksource/exynos_mct.c | 25 ++++++++++++++++++++----- 1 file changed, 20 insertions(+), 5 deletions(-)