| Message ID | 570c473134426b8e11bb043aa59e3cde3dab8af3.1723229941.git.robin.murphy@arm.com (mailing list archive) |
|---|---|
| State | New, archived |
| Headers | show |
| Series | perf/arm-cmn: Fixes and updates | expand |
On Fri, Aug 09, 2024 at 08:15:44PM +0100, Robin Murphy wrote: > By default, CMN has automatic clock-gating with the implication that a > DTC's cycle counter may not increment while the domain is sufficiently > idle. Similar is true for the cycles event on the CPU side, so this has some precedent. > Given that we may have up to 4 DTCs to choose from when scheduling > a cycles event, this may potentially lead to surprising results if > trying to measure metrics based on activity in a different DTC domain > from where cycles end up being counted. Make the reasonable assumption > that if the user wants to count cycles, they almost certainly want to > count all of the cycles, and disable clock gating while a DTC's cycle > counter is in use. As above, the default does match the CPU side behaviour, and a user might be trying to determine how much clock gating occurs over some period, so it's not necessarily right to always disable clock gating. That might need to be an explicit option on the cycles event. Do we always have the ability to disable clock gating, or can that be locked down by system integration or FW? Mark. > > Signed-off-by: Robin Murphy <robin.murphy@arm.com> > --- > drivers/perf/arm-cmn.c | 16 +++++++++++----- > 1 file changed, 11 insertions(+), 5 deletions(-) > > diff --git a/drivers/perf/arm-cmn.c b/drivers/perf/arm-cmn.c > index 8f7a1a6f8ab7..4d128db2040c 100644 > --- a/drivers/perf/arm-cmn.c > +++ b/drivers/perf/arm-cmn.c > @@ -115,6 +115,7 @@ > /* The DTC node is where the magic happens */ > #define CMN_DT_DTC_CTL 0x0a00 > #define CMN_DT_DTC_CTL_DT_EN BIT(0) > +#define CMN_DT_DTC_CTL_CG_DISABLE BIT(10) > > /* DTC counters are paired in 64-bit registers on a 16-byte stride. Yuck */ > #define _CMN_DT_CNT_REG(n) ((((n) / 2) * 4 + (n) % 2) * 4) > @@ -1544,9 +1545,12 @@ static void arm_cmn_event_start(struct perf_event *event, int flags) > int i; > > if (type == CMN_TYPE_DTC) { > - i = hw->dtc_idx[0]; > - writeq_relaxed(CMN_CC_INIT, cmn->dtc[i].base + CMN_DT_PMCCNTR); > - cmn->dtc[i].cc_active = true; > + struct arm_cmn_dtc *dtc = cmn->dtc + hw->dtc_idx[0]; > + > + writel_relaxed(CMN_DT_DTC_CTL_DT_EN | CMN_DT_DTC_CTL_CG_DISABLE, > + dtc->base + CMN_DT_DTC_CTL); > + writeq_relaxed(CMN_CC_INIT, dtc->base + CMN_DT_PMCCNTR); > + dtc->cc_active = true; > } else if (type == CMN_TYPE_WP) { > u64 val = CMN_EVENT_WP_VAL(event); > u64 mask = CMN_EVENT_WP_MASK(event); > @@ -1575,8 +1579,10 @@ static void arm_cmn_event_stop(struct perf_event *event, int flags) > int i; > > if (type == CMN_TYPE_DTC) { > - i = hw->dtc_idx[0]; > - cmn->dtc[i].cc_active = false; > + struct arm_cmn_dtc *dtc = cmn->dtc + hw->dtc_idx[0]; > + > + dtc->cc_active = false; > + writel_relaxed(CMN_DT_DTC_CTL_DT_EN, dtc->base + CMN_DT_DTC_CTL); > } else if (type == CMN_TYPE_WP) { > for_each_hw_dn(hw, dn, i) { > void __iomem *base = dn->pmu_base + CMN_DTM_OFFSET(hw->dtm_offset); > -- > 2.39.2.101.g768bb238c484.dirty >
On 16/08/2024 11:25 am, Mark Rutland wrote: > On Fri, Aug 09, 2024 at 08:15:44PM +0100, Robin Murphy wrote: >> By default, CMN has automatic clock-gating with the implication that a >> DTC's cycle counter may not increment while the domain is sufficiently >> idle. > > Similar is true for the cycles event on the CPU side, so this has some > precedent. > >> Given that we may have up to 4 DTCs to choose from when scheduling >> a cycles event, this may potentially lead to surprising results if >> trying to measure metrics based on activity in a different DTC domain >> from where cycles end up being counted. Make the reasonable assumption >> that if the user wants to count cycles, they almost certainly want to >> count all of the cycles, and disable clock gating while a DTC's cycle >> counter is in use. > > As above, the default does match the CPU side behaviour, It may end up looking equivalent with only a single DTC domain, but the effect with multiple DTCs is as if any cpu_cycles event always counted on CPU0 (or more recently, any random CPU) - that's the real issue here. > and a user > might be trying to determine how much clock gating occurs over some > period, so it's not necessarily right to always disable clock gating. > That might need to be an explicit option on the cycles event. I have also been wondering whether there would be any use-case for a "dtc_active_cycles" event, but it would would have to be targeted at a specific DTC node ID or domain number to be meaningful, so would still be a new thing either way, but even then it seems it would be incredibly niche - you couldn't get an actual measure of idleness without a second DTC to count ungated cycles over the same period, and it could only be correlated with targeted single-node events, since we don't have support or a syntax for aggregating only within a specific DTC domain. Yes, some of that could be achieved, but it would represent significant further complication to the driver and interface, in aid of something which to the best of my knowledge nobody's actually asking for. What users *have* asked me, though, is why they were seeing inconsistent dtc_cycles counts, hence this patch. > Do we always have the ability to disable clock gating, or can that be > locked down by system integration or FW? Yes, the entire PMU may be locked down by a secure override, but otherwise if we do have use of the DTC then there is no limitation on cg_disable itself. Thanks, Robin. > > Mark. > >> >> Signed-off-by: Robin Murphy <robin.murphy@arm.com> >> --- >> drivers/perf/arm-cmn.c | 16 +++++++++++----- >> 1 file changed, 11 insertions(+), 5 deletions(-) >> >> diff --git a/drivers/perf/arm-cmn.c b/drivers/perf/arm-cmn.c >> index 8f7a1a6f8ab7..4d128db2040c 100644 >> --- a/drivers/perf/arm-cmn.c >> +++ b/drivers/perf/arm-cmn.c >> @@ -115,6 +115,7 @@ >> /* The DTC node is where the magic happens */ >> #define CMN_DT_DTC_CTL 0x0a00 >> #define CMN_DT_DTC_CTL_DT_EN BIT(0) >> +#define CMN_DT_DTC_CTL_CG_DISABLE BIT(10) >> >> /* DTC counters are paired in 64-bit registers on a 16-byte stride. Yuck */ >> #define _CMN_DT_CNT_REG(n) ((((n) / 2) * 4 + (n) % 2) * 4) >> @@ -1544,9 +1545,12 @@ static void arm_cmn_event_start(struct perf_event *event, int flags) >> int i; >> >> if (type == CMN_TYPE_DTC) { >> - i = hw->dtc_idx[0]; >> - writeq_relaxed(CMN_CC_INIT, cmn->dtc[i].base + CMN_DT_PMCCNTR); >> - cmn->dtc[i].cc_active = true; >> + struct arm_cmn_dtc *dtc = cmn->dtc + hw->dtc_idx[0]; >> + >> + writel_relaxed(CMN_DT_DTC_CTL_DT_EN | CMN_DT_DTC_CTL_CG_DISABLE, >> + dtc->base + CMN_DT_DTC_CTL); >> + writeq_relaxed(CMN_CC_INIT, dtc->base + CMN_DT_PMCCNTR); >> + dtc->cc_active = true; >> } else if (type == CMN_TYPE_WP) { >> u64 val = CMN_EVENT_WP_VAL(event); >> u64 mask = CMN_EVENT_WP_MASK(event); >> @@ -1575,8 +1579,10 @@ static void arm_cmn_event_stop(struct perf_event *event, int flags) >> int i; >> >> if (type == CMN_TYPE_DTC) { >> - i = hw->dtc_idx[0]; >> - cmn->dtc[i].cc_active = false; >> + struct arm_cmn_dtc *dtc = cmn->dtc + hw->dtc_idx[0]; >> + >> + dtc->cc_active = false; >> + writel_relaxed(CMN_DT_DTC_CTL_DT_EN, dtc->base + CMN_DT_DTC_CTL); >> } else if (type == CMN_TYPE_WP) { >> for_each_hw_dn(hw, dn, i) { >> void __iomem *base = dn->pmu_base + CMN_DTM_OFFSET(hw->dtm_offset); >> -- >> 2.39.2.101.g768bb238c484.dirty >>
diff --git a/drivers/perf/arm-cmn.c b/drivers/perf/arm-cmn.c index 8f7a1a6f8ab7..4d128db2040c 100644 --- a/drivers/perf/arm-cmn.c +++ b/drivers/perf/arm-cmn.c @@ -115,6 +115,7 @@ /* The DTC node is where the magic happens */ #define CMN_DT_DTC_CTL 0x0a00 #define CMN_DT_DTC_CTL_DT_EN BIT(0) +#define CMN_DT_DTC_CTL_CG_DISABLE BIT(10) /* DTC counters are paired in 64-bit registers on a 16-byte stride. Yuck */ #define _CMN_DT_CNT_REG(n) ((((n) / 2) * 4 + (n) % 2) * 4) @@ -1544,9 +1545,12 @@ static void arm_cmn_event_start(struct perf_event *event, int flags) int i; if (type == CMN_TYPE_DTC) { - i = hw->dtc_idx[0]; - writeq_relaxed(CMN_CC_INIT, cmn->dtc[i].base + CMN_DT_PMCCNTR); - cmn->dtc[i].cc_active = true; + struct arm_cmn_dtc *dtc = cmn->dtc + hw->dtc_idx[0]; + + writel_relaxed(CMN_DT_DTC_CTL_DT_EN | CMN_DT_DTC_CTL_CG_DISABLE, + dtc->base + CMN_DT_DTC_CTL); + writeq_relaxed(CMN_CC_INIT, dtc->base + CMN_DT_PMCCNTR); + dtc->cc_active = true; } else if (type == CMN_TYPE_WP) { u64 val = CMN_EVENT_WP_VAL(event); u64 mask = CMN_EVENT_WP_MASK(event); @@ -1575,8 +1579,10 @@ static void arm_cmn_event_stop(struct perf_event *event, int flags) int i; if (type == CMN_TYPE_DTC) { - i = hw->dtc_idx[0]; - cmn->dtc[i].cc_active = false; + struct arm_cmn_dtc *dtc = cmn->dtc + hw->dtc_idx[0]; + + dtc->cc_active = false; + writel_relaxed(CMN_DT_DTC_CTL_DT_EN, dtc->base + CMN_DT_DTC_CTL); } else if (type == CMN_TYPE_WP) { for_each_hw_dn(hw, dn, i) { void __iomem *base = dn->pmu_base + CMN_DTM_OFFSET(hw->dtm_offset);
By default, CMN has automatic clock-gating with the implication that a DTC's cycle counter may not increment while the domain is sufficiently idle. Given that we may have up to 4 DTCs to choose from when scheduling a cycles event, this may potentially lead to surprising results if trying to measure metrics based on activity in a different DTC domain from where cycles end up being counted. Make the reasonable assumption that if the user wants to count cycles, they almost certainly want to count all of the cycles, and disable clock gating while a DTC's cycle counter is in use. Signed-off-by: Robin Murphy <robin.murphy@arm.com> --- drivers/perf/arm-cmn.c | 16 +++++++++++----- 1 file changed, 11 insertions(+), 5 deletions(-)