| Message ID | 1477687813-11412-1-git-send-email-nleeder@codeaurora.org (mailing list archive) |
|---|---|
| State | Not Applicable, archived |
| Delegated to: | Andy Gross |
| Headers | show |
Hi Neil, Apologies for the delay in replying to this. This is looking good. I have a few specific comments and a couple of general concerns below. Will, please see the bit below about cluster/socket aggregation (grep for your name). On Fri, Oct 28, 2016 at 04:50:13PM -0400, Neil Leeder wrote: > +#include <linux/acpi.h> > +#include <linux/interrupt.h> > +#include <linux/perf_event.h> > +#include <linux/platform_device.h> While scanning the driver. I noticed a number of other headers that are required for things the driver explicitly references. Please add the following so that we're not relying on (fragile) transitive includes: #include <linux/bitops.h> #include <linux/bug.h> #include <linux/cpuhotplug.h> #include <linux/cpumask.h> #include <linux/device.h> #include <linux/errno.h> #include <linux/kernel.h> #include <linux/list.h> #include <linux/percpu.h> #include <linux/smp.h> #include <linux/spinlock.h> #include <linux/sysfs.h> #include <linux/types.h> #include <asm/barrier.h> #include <asm/local64.h> #include <asm/sysreg.h> [...] > +/* > + * Aggregate PMU. Implements the core pmu functions and manages > + * the hardware PMUs. > + */ > +struct l2cache_pmu { > + struct hlist_node node; > + u32 num_pmus; > + struct pmu pmu; > + int num_counters; > + cpumask_t cpumask; > + struct platform_device *pdev; > +}; > + > +/* > + * The cache is made up of one or more clusters, each cluster has its own PMU. > + * Each cluster is associated with one or more CPUs. > + * This structure represents one of the hardware PMUs. > + * > + * Events can be envisioned as a 2-dimensional array. Each column represents > + * a group of events. There are 8 groups. Only one entry from each > + * group can be in use at a time. When an event is assigned a counter > + * by *_event_add(), the counter index is assigned to group_to_counter[group]. If I've followed correctly, this means each group has a dedicated counter? I take it groups are not symmetric (i.e. each column has different events)? Or does each column contain the same events? Is there any overlap? > + * This allows *filter_match() to detect and reject conflicting events in > + * the same group. > + * Events are specified as 0xCCG, where CC is 2 hex digits specifying > + * the code (array row) and G specifies the group (column). > + * > + * In addition there is a cycle counter event specified by L2CYCLE_CTR_RAW_CODE > + * which is outside the above scheme. > + */ > +struct hml2_pmu { It's not clear to me what "hml2" is, and now we seem to use "cluster" and "hml2" interchangeably in comments and code. Can you please define what "hml2" is in a comment, mentioning that we refer to this (or its container?) as a cluster? Can we also please s/hml2/cluster/ in the code, for consistency? > + struct perf_event *events[MAX_L2_CTRS]; > + struct l2cache_pmu *l2cache_pmu; > + DECLARE_BITMAP(used_counters, MAX_L2_CTRS); > + DECLARE_BITMAP(used_groups, L2_EVT_GROUP_MAX + 1); > + int group_to_counter[L2_EVT_GROUP_MAX + 1]; > + int irq; > + /* The CPU that is used for collecting events on this cluster */ > + int on_cpu; > + /* All the CPUs associated with this cluster */ > + cpumask_t cluster_cpus; I'm still uncertain about aggregating all cluster PMUs into a larger PMU, given the cluster PMUs are logically independent (at least in terms of the programming model). However, from what I understand the x86 uncore PMU drivers aggregate symmetric instances of uncore PMUs (and also aggregate across packages to the same logical PMU). Whatever we do, it would be nice for the uncore drivers to align on a common behaviour (and I think we're currently going the oppposite route with Cavium's uncore PMU). Will, thoughts? > + spinlock_t pmu_lock; > +}; [...] > +static void hml2_pmu__set_resr(struct hml2_pmu *cluster, > + u32 event_group, u32 event_cc) > +{ > + u64 field; > + u64 resr_val; > + u32 shift; > + unsigned long flags; > + > + shift = L2PMRESR_GROUP_BITS * event_group; > + field = ((u64)(event_cc & L2PMRESR_GROUP_MASK) << shift) | L2PMRESR_EN; The L2PMRESR_EN isn't part of the shifted field... > + > + spin_lock_irqsave(&cluster->pmu_lock, flags); > + > + resr_val = get_l2_indirect_reg(L2PMRESR); > + resr_val &= ~(L2PMRESR_GROUP_MASK << shift); > + resr_val |= field; ... so can we please or that in on a separate line here, e.g. resr_val |= L2PMRESR_EN; That said, do we need to set this bit each time we program an event? > + set_l2_indirect_reg(L2PMRESR, resr_val); > + > + spin_unlock_irqrestore(&cluster->pmu_lock, flags); > +} [...] > +static void l2_cache__event_update_from_cluster(struct perf_event *event, > + struct hml2_pmu *cluster) > +{ > + struct hw_perf_event *hwc = &event->hw; > + u64 delta64, prev, now; > + u32 delta; > + u32 idx = hwc->idx; > + > + do { > + prev = local64_read(&hwc->prev_count); > + now = hml2_pmu__counter_get_value(idx); > + } while (local64_cmpxchg(&hwc->prev_count, prev, now) != prev); > + > + if (idx == l2_cycle_ctr_idx) { > + /* > + * The cycle counter is 64-bit so needs separate handling > + * of 64-bit delta. > + */ > + delta64 = now - prev; > + local64_add(delta64, &event->count); > + } else { > + /* > + * 32-bit counters need the unsigned 32-bit math to handle > + * overflow and now < prev > + */ > + delta = now - prev; > + local64_add(delta, &event->count); > + } > +} I believe you can get rid of the delta/delta64 split with something like the below (with a 64-bit delta): /* * The cycle counter is 64-bit, but all other counters are * 32-bit, and we must handle 32-bit overflow explicitly. */ delta = now - prev; if (idx != l2_cycle_ctr_idx) delta &= 0xffffffff; local64_add(delta, &event->count); > + > +static void l2_cache__cluster_set_period(struct hml2_pmu *cluster, > + struct hw_perf_event *hwc) > +{ > + u64 base = L2_MAX_PERIOD - (L2_CNT_PERIOD - 1); Is L2_CNT_PERIOD a hw-derived value? Or just something that happened to be small enough to avoid overflow in testing? > + u32 idx = hwc->idx; > + u64 prev = local64_read(&hwc->prev_count); > + u64 value; > + > + /* > + * Limit the maximum period to prevent the counter value > + * from overtaking the one we are about to program. > + * Use a starting value which is high enough that after > + * an overflow, interrupt latency will not cause the count > + * to reach the base value. If the previous value > + * is below the base, increase it to be above the base > + * and update prev_count accordingly. Otherwise if > + * the previous value is already above the base > + * nothing needs to be done to prev_count. > + */ > + if (prev < base) { > + value = base + prev; > + local64_set(&hwc->prev_count, value); > + } else { > + value = prev; > + } > + > + hml2_pmu__counter_set_value(idx, value); > +} Given that we don't do sampling, and are only using the interrupt to prevent losing events in the case of overflow, I think we can use a constant value to handle all cases. Additionally, I think we need to handle the cycle counter separately, since it has a different width. Something like the below (perhaps with different constants): static void l2_cache__cluster_set_period(struct hml2_pmu *cluster, struct hw_perf_event *hwc) { u64 new; /* * We limit the max period to half of the max counter value so * that even in the case of extreme interrupt latency the * counter will (hopefully) not wrap past its initial value. */ if (hwc->idx == l2_cycle_ctr_idx) new = BIT_ULL(63); else new = BIT_ULL(31); local64_set(&hwc->prev_count, new); hml2_pmu__counter_set_value(idx, new); } > +static int l2_cache__get_event_idx(struct hml2_pmu *cluster, > + struct perf_event *event) > +{ > + struct hw_perf_event *hwc = &event->hw; > + int idx; > + > + if (hwc->config_base == L2CYCLE_CTR_RAW_CODE) { > + if (test_and_set_bit(l2_cycle_ctr_idx, cluster->used_counters)) > + return -EAGAIN; > + > + return l2_cycle_ctr_idx; > + } > + > + for (idx = 0; idx < cluster->l2cache_pmu->num_counters - 1; idx++) { > + if (!test_and_set_bit(idx, cluster->used_counters)) { > + set_bit(L2_EVT_GROUP(hwc->config_base), > + cluster->used_groups); > + return idx; > + } > + } > + /* The counters are all in use. */ > + return -EAGAIN; > +} Are we racing with another bitmap manipulation here? Or can we split the find and set of the bit, i.e. int num_ctrs = cluster->l2cache_pmu->num_counters; int idx = find_first_zero_bit(cluster->used_counters, num_ctrs); if (idx == num_ctrs) return -EAGAIN; set_bit(idx, cluster->used_counters) Otherwise, I think the outer loop can use for_each_clear_bit(). [...] > +static int l2_cache__event_init(struct perf_event *event) > +{ > + struct hw_perf_event *hwc = &event->hw; > + struct hml2_pmu *cluster; > + struct perf_event *sibling; > + struct l2cache_pmu *l2cache_pmu; > + > + if (event->attr.type != event->pmu->type) > + return -ENOENT; > + > + l2cache_pmu = to_l2cache_pmu(event->pmu); > + > + if (hwc->sample_period) { > + dev_warn(&l2cache_pmu->pdev->dev, "Sampling not supported\n"); > + return -EOPNOTSUPP; > + } Please remove these dev_warns, or at lease relegate them to ratelimited debug messages rather than warning messages. Users can easily trigger these by accident and/or when probing capabilities in the usual fashion, and there's no need for them. The CCN driver is an unfortunate bad example in this respect. [...] > + /* Don't allow groups with mixed PMUs, except for s/w events */ > + if (event->group_leader->pmu != event->pmu && > + !is_software_event(event->group_leader)) { > + dev_warn(&l2cache_pmu->pdev->dev, > + "Can't create mixed PMU group\n"); > + return -EINVAL; > + } > + > + list_for_each_entry(sibling, &event->group_leader->sibling_list, > + group_entry) > + if (sibling->pmu != event->pmu && > + !is_software_event(sibling)) { > + dev_warn(&l2cache_pmu->pdev->dev, > + "Can't create mixed PMU group\n"); > + return -EINVAL; > + } > + > + /* Ensure all events in a group are on the same cpu */ > + cluster = get_hml2_pmu(event->cpu); > + if ((event->group_leader != event) && > + (cluster->on_cpu != event->group_leader->cpu)) { > + dev_warn(&l2cache_pmu->pdev->dev, > + "Can't create group on CPUs %d and %d", > + event->cpu, event->group_leader->cpu); > + return -EINVAL; > + } It's probably worth also checking that the events are co-schedulable (e.g. they don't conflict column-wise). [...] > +static void l2_cache__event_start(struct perf_event *event, int flags) > +{ > + struct hml2_pmu *cluster; > + struct hw_perf_event *hwc = &event->hw; > + int idx = hwc->idx; > + u32 config; > + u32 event_cc, event_group; > + > + hwc->state = 0; > + > + cluster = get_hml2_pmu(event->cpu); > + l2_cache__cluster_set_period(cluster, hwc); > + > + if (hwc->config_base == L2CYCLE_CTR_RAW_CODE) { > + hml2_pmu__set_evccntcr(0x0); Nit: s/0x0/0/ -- there's no other hex usage to be consistent with, and a single digit is clearer. > + } else { > + config = hwc->config_base; > + event_cc = L2_EVT_CODE(config); > + event_group = L2_EVT_GROUP(config); > + > + hml2_pmu__set_evcntcr(idx, 0x0); Nit: s/0x0/0/ -- as above. > + hml2_pmu__set_evtyper(idx, event_group); > + hml2_pmu__set_resr(cluster, event_group, event_cc); > + hml2_pmu__set_evfilter_sys_mode(idx); > + } > + > + hml2_pmu__counter_enable_interrupt(idx); > + hml2_pmu__counter_enable(idx); > +} > + > +static void l2_cache__event_stop(struct perf_event *event, int flags) > +{ > + struct hml2_pmu *cluster; > + struct hw_perf_event *hwc = &event->hw; > + int idx = hwc->idx; > + > + if (!(hwc->state & PERF_HES_STOPPED)) { It's better to have an early return, e.g. /* If already stopped, we have nothing to do */ if (hwc->state & PERF_HES_STOPPED) return; ... remaining logic here, not in a block ... > + cluster = get_hml2_pmu(event->cpu); > + hml2_pmu__counter_disable_interrupt(idx); > + hml2_pmu__counter_disable(idx); > + > + if (flags & PERF_EF_UPDATE) > + l2_cache__event_update_from_cluster(event, cluster); > + hwc->state |= PERF_HES_STOPPED | PERF_HES_UPTODATE; > + } > +} > + > +static int l2_cache__event_add(struct perf_event *event, int flags) > +{ > + struct hw_perf_event *hwc = &event->hw; > + int idx; > + int err = 0; > + struct hml2_pmu *cluster; > + > + cluster = get_hml2_pmu(event->cpu); > + > + idx = l2_cache__get_event_idx(cluster, event); > + if (idx < 0) { > + err = idx; > + return err; > + } You can return idx directly and avoid the block here. > + hwc->idx = idx; > + hwc->state = PERF_HES_STOPPED | PERF_HES_UPTODATE; > + cluster->events[idx] = event; > + cluster->group_to_counter[L2_EVT_GROUP(hwc->config_base)] = idx; > + local64_set(&hwc->prev_count, 0ULL); Nit: s/0ULL/0/ [...] > +static int l2_cache_filter_match(struct perf_event *event) > +{ > + struct hw_perf_event *hwc = &event->hw; > + struct hml2_pmu *cluster = get_hml2_pmu(event->cpu); > + unsigned int group = L2_EVT_GROUP(hwc->config_base); > + > + /* check for column exclusion: group already in use by another event */ > + if (test_bit(group, cluster->used_groups) && > + cluster->events[cluster->group_to_counter[group]] != event) > + return 0; > + > + return 1; > +} I'm still not keen on this. I'm still hoping to kill off filter_match, as for many reasons it is not great, and I know that x86 have similar scheduling requirements, and somehow manage without this. It would be worth adding justification for this in a comment block above, such as why you need this, and why this won't result in sub-optimal behaviour. [...] > + if (acpi_bus_get_device(ACPI_HANDLE(dev), &device)) > + return -ENODEV; > + > + if (kstrtol(device->pnp.unique_id, 10, &fw_cluster_id) < 0) { > + dev_err(&pdev->dev, "unable to read ACPI uid\n"); > + return -ENODEV; > + } > + cluster->l2cache_pmu = l2cache_pmu; > + for_each_present_cpu(cpu) { > + if (topology_physical_package_id(cpu) == fw_cluster_id) { > + cpumask_set_cpu(cpu, &cluster->cluster_cpus); > + per_cpu(pmu_cluster, cpu) = cluster; > + } > + } I'm still uneasy about this. The topology_* API doesn't have a well-defined mapping to the MPIDR.Aff* levels, which itself also don't have a well-defined mapping to your hardware's clusters (and therefore fw_cluster_id). Thus, I'm rather worried that this is going to get broken easily, either by changes in the kernel, or in future HW revisions where the mapping of clusters to MPIDR.Aff* fields changes. [...] > + err = devm_request_irq(&pdev->dev, irq, l2_cache__handle_irq, > + IRQF_NOBALANCING, "l2-cache-pmu", cluster); I believe this also needs IRQF_NO_THREAD. [...] > + cluster->pmu_lock = __SPIN_LOCK_UNLOCKED(cluster->pmu_lock); Please use: spin_lock_init(&cluster->pmu_lock); [...] > + platform_set_drvdata(pdev, l2cache_pmu); > + l2cache_pmu->pmu = (struct pmu) { > + /* suffix is instance id for future use with multiple sockets */ > + .name = "l2cache_0", Sorry to go back and forth on this, but as above with the aggregation comments, we might not need/want the suffix after all, or we might want per-cluster PMUs. > + .task_ctx_nr = perf_invalid_context, > + .pmu_enable = l2_cache__pmu_enable, > + .pmu_disable = l2_cache__pmu_disable, > + .event_init = l2_cache__event_init, > + .add = l2_cache__event_add, > + .del = l2_cache__event_del, > + .start = l2_cache__event_start, > + .stop = l2_cache__event_stop, > + .read = l2_cache__event_read, > + .attr_groups = l2_cache_pmu_attr_grps, > + .filter_match = l2_cache_filter_match, > + }; As a general note, can we please s/__/_/ for function names? The double underscore gains us nothing. [...] > + l2_counter_present_mask = GENMASK(l2cache_pmu->num_counters - 2, 0) | > + L2PM_CC_ENABLE; This looks confusing, and it's the only use of L2PM_CC_ENABLE. Perhaps use BIT(L2CYCLE_CTR_BIT) instead, and get rid of L2PM_CC_ENABLE? [...] > + err = perf_pmu_register(&l2cache_pmu->pmu, l2cache_pmu->pmu.name, -1); > + if (err < 0) { > + dev_err(&pdev->dev, "Error %d registering L2 cache PMU\n", err); > + return err; > + } > + > + dev_info(&pdev->dev, "Registered L2 cache PMU using %d HW PMUs\n", > + l2cache_pmu->num_pmus); > + > + err = cpuhp_state_add_instance_nocalls(CPUHP_AP_PERF_ARM_QCOM_L2_ONLINE, > + &l2cache_pmu->node); > + > + return err; > +} Can we race with hotplug? I believe the hotplug callback registration should come before the perf_pmu_register() > +static int l2_cache_pmu_remove(struct platform_device *pdev) > +{ > + struct l2cache_pmu *l2cache_pmu = > + to_l2cache_pmu(platform_get_drvdata(pdev)); > + > + cpuhp_state_remove_instance_nocalls(CPUHP_AP_PERF_ARM_QCOM_L2_ONLINE, > + &l2cache_pmu->node); > + perf_pmu_unregister(&l2cache_pmu->pmu); > + return 0; > +} Mirroring the above, I think we should unregister the PMU before removing the hotplug callbacks. Thanks, Mark. -- To unsubscribe from this list: send the line "unsubscribe linux-arm-msm" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
On Wed, Nov 09, 2016 at 05:54:13PM +0000, Mark Rutland wrote: > On Fri, Oct 28, 2016 at 04:50:13PM -0400, Neil Leeder wrote: > > + struct perf_event *events[MAX_L2_CTRS]; > > + struct l2cache_pmu *l2cache_pmu; > > + DECLARE_BITMAP(used_counters, MAX_L2_CTRS); > > + DECLARE_BITMAP(used_groups, L2_EVT_GROUP_MAX + 1); > > + int group_to_counter[L2_EVT_GROUP_MAX + 1]; > > + int irq; > > + /* The CPU that is used for collecting events on this cluster */ > > + int on_cpu; > > + /* All the CPUs associated with this cluster */ > > + cpumask_t cluster_cpus; > > I'm still uncertain about aggregating all cluster PMUs into a larger > PMU, given the cluster PMUs are logically independent (at least in terms > of the programming model). > > However, from what I understand the x86 uncore PMU drivers aggregate > symmetric instances of uncore PMUs (and also aggregate across packages > to the same logical PMU). > > Whatever we do, it would be nice for the uncore drivers to align on a > common behaviour (and I think we're currently going the oppposite route > with Cavium's uncore PMU). Will, thoughts? I'm not a big fan of aggregating this stuff. Ultimately, the user in the driving seat of perf is going to need some knowledge about the toplogy of the system in order to perform sensible profiling using an uncore PMU. If the kernel tries to present a single, unified PMU then we paint ourselves into a corner when the hardware isn't as symmetric as we want it to be (big/little on the CPU side is the extreme example of this). If we want to be consistent, then exposing each uncore unit as a separate PMU is the way to go. That doesn't mean we can't aggregate the components of a distributed PMU (e.g. the CCN or the SMMU), but we don't want to aggregate at the programming interface/IP block level. We could consider exposing some topology information in sysfs if that's seen as an issue with the non-aggregated case. Will -- To unsubscribe from this list: send the line "unsubscribe linux-arm-msm" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Hi Mark, Thanks for the review. I'll handle all the syntactic comments, so I won't reply to them all individually here. For the aggregation, I'll reply separately to Will's post to keep all those comments together. On 11/9/2016 12:54 PM, Mark Rutland wrote: >> + >> +/* >> + * The cache is made up of one or more clusters, each cluster has its own PMU. >> + * Each cluster is associated with one or more CPUs. >> + * This structure represents one of the hardware PMUs. >> + * >> + * Events can be envisioned as a 2-dimensional array. Each column represents >> + * a group of events. There are 8 groups. Only one entry from each >> + * group can be in use at a time. When an event is assigned a counter >> + * by *_event_add(), the counter index is assigned to group_to_counter[group]. > > If I've followed correctly, this means each group has a dedicated > counter? > > I take it groups are not symmetric (i.e. each column has different > events)? Or does each column contain the same events? > > Is there any overlap? Each group will have at most one counter, but it's not dedicated. There's no requirement that an implementation have as many counters as there are groups, so an event can be assigned to any available counter. Every entry in the 2-dimensional array is unique, so no duplicates. Once you have used an event, you cannot use any other event from its column. As an aside, hardware designers put in some effort to ensure that events which may need to be collected at the same time are located in different columns. >> +static int l2_cache__event_init(struct perf_event *event) >> +{ [...] >> + /* Don't allow groups with mixed PMUs, except for s/w events */ >> + if (event->group_leader->pmu != event->pmu && >> + !is_software_event(event->group_leader)) { >> + dev_warn(&l2cache_pmu->pdev->dev, >> + "Can't create mixed PMU group\n"); >> + return -EINVAL; >> + } >> + >> + list_for_each_entry(sibling, &event->group_leader->sibling_list, >> + group_entry) >> + if (sibling->pmu != event->pmu && >> + !is_software_event(sibling)) { >> + dev_warn(&l2cache_pmu->pdev->dev, >> + "Can't create mixed PMU group\n"); >> + return -EINVAL; >> + } >> + >> + /* Ensure all events in a group are on the same cpu */ >> + cluster = get_hml2_pmu(event->cpu); >> + if ((event->group_leader != event) && >> + (cluster->on_cpu != event->group_leader->cpu)) { >> + dev_warn(&l2cache_pmu->pdev->dev, >> + "Can't create group on CPUs %d and %d", >> + event->cpu, event->group_leader->cpu); >> + return -EINVAL; >> + } > > It's probably worth also checking that the events are co-schedulable > (e.g. they don't conflict column-wise). That's what filter_match() is doing - stopping column-conflicting events from even getting to init(). In init() we don't have a record of which other events are being co-scheduled. We could keep a list of groups used by other events to compare against, but because there's no matching term() function there's no obvious way of removing them from the list. In my very first patchset I had the check inside event_add() because entries could be removed in event_del(). That was where you suggested that I use filter_match(). > >> + if (acpi_bus_get_device(ACPI_HANDLE(dev), &device)) >> + return -ENODEV; >> + >> + if (kstrtol(device->pnp.unique_id, 10, &fw_cluster_id) < 0) { >> + dev_err(&pdev->dev, "unable to read ACPI uid\n"); >> + return -ENODEV; >> + } > >> + cluster->l2cache_pmu = l2cache_pmu; >> + for_each_present_cpu(cpu) { >> + if (topology_physical_package_id(cpu) == fw_cluster_id) { >> + cpumask_set_cpu(cpu, &cluster->cluster_cpus); >> + per_cpu(pmu_cluster, cpu) = cluster; >> + } >> + } > > I'm still uneasy about this. > > The topology_* API doesn't have a well-defined mapping to the MPIDR.Aff* > levels, which itself also don't have a well-defined mapping to your > hardware's clusters (and therefore fw_cluster_id). > > Thus, I'm rather worried that this is going to get broken easily, either > by changes in the kernel, or in future HW revisions where the mapping of > clusters to MPIDR.Aff* fields changes. I'm not sure how else to get a mapping of CPU to cluster which doesn't eventually end with MPIDR. This is the definition of topology_physical_package_id() from asm/topology.h: #define topology_physical_package_id(cpu) (cpu_topology[cpu].cluster_id) It seems to be a pretty solid connection between cpu and cluster. I don't think this is an abuse of this function. Unless there is some other way of getting this mapping I'd suggest using this, and if some future chip should change MPIDR usage it can be addressed it then. > > Thanks, > Mark. > Thanks, Neil
On Thu, Nov 10, 2016 at 06:25:47PM -0500, Leeder, Neil wrote: > On 11/9/2016 12:54 PM, Mark Rutland wrote: > >>+ > >>+/* > >>+ * The cache is made up of one or more clusters, each cluster has its own PMU. > >>+ * Each cluster is associated with one or more CPUs. > >>+ * This structure represents one of the hardware PMUs. > >>+ * > >>+ * Events can be envisioned as a 2-dimensional array. Each column represents > >>+ * a group of events. There are 8 groups. Only one entry from each > >>+ * group can be in use at a time. When an event is assigned a counter > >>+ * by *_event_add(), the counter index is assigned to group_to_counter[group]. > > > >If I've followed correctly, this means each group has a dedicated > >counter? > > > >I take it groups are not symmetric (i.e. each column has different > >events)? Or does each column contain the same events? > > > >Is there any overlap? > > Each group will have at most one counter, but it's not dedicated. > There's no requirement that an implementation have as many counters > as there are groups, so an event can be assigned to any available > counter. > > Every entry in the 2-dimensional array is unique, so no duplicates. > Once you have used an event, you cannot use any other event from its > column. Ok; thanks for clarifying that! > >>+static int l2_cache__event_init(struct perf_event *event) > >>+ /* Don't allow groups with mixed PMUs, except for s/w events */ > >>+ if (event->group_leader->pmu != event->pmu && > >>+ !is_software_event(event->group_leader)) { > >>+ dev_warn(&l2cache_pmu->pdev->dev, > >>+ "Can't create mixed PMU group\n"); > >>+ return -EINVAL; > >>+ } > >>+ > >>+ list_for_each_entry(sibling, &event->group_leader->sibling_list, > >>+ group_entry) > >>+ if (sibling->pmu != event->pmu && > >>+ !is_software_event(sibling)) { > >>+ dev_warn(&l2cache_pmu->pdev->dev, > >>+ "Can't create mixed PMU group\n"); > >>+ return -EINVAL; > >>+ } > >>+ > >>+ /* Ensure all events in a group are on the same cpu */ > >>+ cluster = get_hml2_pmu(event->cpu); > >>+ if ((event->group_leader != event) && > >>+ (cluster->on_cpu != event->group_leader->cpu)) { > >>+ dev_warn(&l2cache_pmu->pdev->dev, > >>+ "Can't create group on CPUs %d and %d", > >>+ event->cpu, event->group_leader->cpu); > >>+ return -EINVAL; > >>+ } > > > >It's probably worth also checking that the events are co-schedulable > >(e.g. they don't conflict column-wise). > > That's what filter_match() is doing - stopping column-conflicting > events from even getting to init(). In init() we don't have a record > of which other events are being co-scheduled. We could keep a list > of groups used by other events to compare against, but because > there's no matching term() function there's no obvious way of > removing them from the list. I mean within the group, in addition to the filter_match() logic. When you event_init() an event, you can determine whether there is any column conflict within the group the new events is being placed in, and whether you have sufficient counters to ever be able to schedule that group. If not, the group should be rejected. Other PMUs have similar checks; see l2x0_pmu_group_is_valid() in arch/arm/mm/cache-l2x0-pmu.c, and valiate_group() in drivers/perf/arm_pmu.c. I don't believe that filer_match() can catch that, as it's called on each event in a group individually prior to add() time, and thus there's no visibility of the group as a whole. Regardless, we can and should catch that case far earlier. [...] > >>+ if (acpi_bus_get_device(ACPI_HANDLE(dev), &device)) > >>+ return -ENODEV; > >>+ > >>+ if (kstrtol(device->pnp.unique_id, 10, &fw_cluster_id) < 0) { > >>+ dev_err(&pdev->dev, "unable to read ACPI uid\n"); > >>+ return -ENODEV; > >>+ } > > > >>+ cluster->l2cache_pmu = l2cache_pmu; > >>+ for_each_present_cpu(cpu) { > >>+ if (topology_physical_package_id(cpu) == fw_cluster_id) { > >>+ cpumask_set_cpu(cpu, &cluster->cluster_cpus); > >>+ per_cpu(pmu_cluster, cpu) = cluster; > >>+ } > >>+ } > > > >I'm still uneasy about this. > > > >The topology_* API doesn't have a well-defined mapping to the MPIDR.Aff* > >levels, which itself also don't have a well-defined mapping to your > >hardware's clusters (and therefore fw_cluster_id). > > > >Thus, I'm rather worried that this is going to get broken easily, either > >by changes in the kernel, or in future HW revisions where the mapping of > >clusters to MPIDR.Aff* fields changes. > > I'm not sure how else to get a mapping of CPU to cluster which > doesn't eventually end with MPIDR. This is unfortunate. :( It would have been much nicer if the FW also provided the MPIDR.Aff<n> level to match up to, as that would be unambiguous. > This is the definition of topology_physical_package_id() from > asm/topology.h: > > #define topology_physical_package_id(cpu) > (cpu_topology[cpu].cluster_id) > > It seems to be a pretty solid connection between cpu and cluster. As I mentioned above, there's no well-defined mapping from MPIDR.Aff* to the topology API levels. The "cluster_id" here is a guess, and one that might change in future based on other heuristics. > I don't think this is an abuse of this function. Unless there is some > other way of getting this mapping I'd suggest using this, and if some > future chip should change MPIDR usage it can be addressed it then. I don't think it's an abuse, as such, but I don't think that it is reliable. That said, I don't see that we can do any better, as you say. It's probably worth adding a comment block regarding our expectations, i.e. that cluster_id means Aff1 for CPUs without multi-threading, Aff2 otherwise, and that we hope future systems don't choose another MPIDR.Aff* mapping scheme. Thanks, Mark. -- To unsubscribe from this list: send the line "unsubscribe linux-arm-msm" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Hi Will, On 11/9/2016 1:16 PM, Will Deacon wrote: > On Wed, Nov 09, 2016 at 05:54:13PM +0000, Mark Rutland wrote: >> On Fri, Oct 28, 2016 at 04:50:13PM -0400, Neil Leeder wrote: >>> + struct perf_event *events[MAX_L2_CTRS]; >>> + struct l2cache_pmu *l2cache_pmu; >>> + DECLARE_BITMAP(used_counters, MAX_L2_CTRS); >>> + DECLARE_BITMAP(used_groups, L2_EVT_GROUP_MAX + 1); >>> + int group_to_counter[L2_EVT_GROUP_MAX + 1]; >>> + int irq; >>> + /* The CPU that is used for collecting events on this cluster */ >>> + int on_cpu; >>> + /* All the CPUs associated with this cluster */ >>> + cpumask_t cluster_cpus; >> >> I'm still uncertain about aggregating all cluster PMUs into a larger >> PMU, given the cluster PMUs are logically independent (at least in terms >> of the programming model). >> >> However, from what I understand the x86 uncore PMU drivers aggregate >> symmetric instances of uncore PMUs (and also aggregate across packages >> to the same logical PMU). >> >> Whatever we do, it would be nice for the uncore drivers to align on a >> common behaviour (and I think we're currently going the oppposite route >> with Cavium's uncore PMU). Will, thoughts? > > I'm not a big fan of aggregating this stuff. Ultimately, the user in the > driving seat of perf is going to need some knowledge about the toplogy of > the system in order to perform sensible profiling using an uncore PMU. > If the kernel tries to present a single, unified PMU then we paint ourselves > into a corner when the hardware isn't as symmetric as we want it to be > (big/little on the CPU side is the extreme example of this). If we want > to be consistent, then exposing each uncore unit as a separate PMU is > the way to go. That doesn't mean we can't aggregate the components of a > distributed PMU (e.g. the CCN or the SMMU), but we don't want to aggregate > at the programming interface/IP block level. > > We could consider exposing some topology information in sysfs if that's > seen as an issue with the non-aggregated case. > > Will So is there a use-case for individual uncore PMUs when they can't be used in task mode or per-cpu? The main (only?) use will be in system mode, in which case surely it makes sense to provide a single aggregated count? With individual PMUs exposed there will be potentially dozens of nodes for userspace to collect from which would make perf command-line usage unwieldy at best. Neil
On Fri, Nov 11, 2016 at 04:52:35PM -0500, Leeder, Neil wrote: > So is there a use-case for individual uncore PMUs when they can't be > used in task mode or per-cpu? > > The main (only?) use will be in system mode, in which case surely it > makes sense to provide a single aggregated count? If you are aware of the system topology, the numbers may be more useful than the summed count. If you aren't, it's still possible to sum them in userspace. Having them summed by the kernel means that the kernel is implying it supports group semantics that it cannot, since it cannot start/stop all counters in a group atomically if they're split across several units. > With individual PMUs exposed there will be potentially dozens of > nodes for userspace to collect from which would make perf > command-line usage unwieldy at best. FWIW, for uncore/system PMUs, even on x86 there are a number of independent units. Some PMUs (those which are symmetric across the topology) get hidden behind the same struct PMU, but instances are still isolated (and their values not summed). Thanks, Mark. -- To unsubscribe from this list: send the line "unsubscribe linux-arm-msm" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
diff --git a/drivers/perf/Kconfig b/drivers/perf/Kconfig index 4d5c5f9..9365190 100644 --- a/drivers/perf/Kconfig +++ b/drivers/perf/Kconfig @@ -12,6 +12,15 @@ config ARM_PMU Say y if you want to use CPU performance monitors on ARM-based systems. +config QCOM_L2_PMU + bool "Qualcomm Technologies L2-cache PMU" + depends on ARCH_QCOM && ARM64 && PERF_EVENTS && ACPI + help + Provides support for the L2 cache performance monitor unit (PMU) + in Qualcomm Technologies processors. + Adds the L2 cache PMU into the perf events subsystem for + monitoring L2 cache events. + config XGENE_PMU depends on PERF_EVENTS && ARCH_XGENE bool "APM X-Gene SoC PMU" diff --git a/drivers/perf/Makefile b/drivers/perf/Makefile index b116e98..ef24833 100644 --- a/drivers/perf/Makefile +++ b/drivers/perf/Makefile @@ -1,2 +1,3 @@ obj-$(CONFIG_ARM_PMU) += arm_pmu.o +obj-$(CONFIG_QCOM_L2_PMU) += qcom_l2_pmu.o obj-$(CONFIG_XGENE_PMU) += xgene_pmu.o diff --git a/drivers/perf/qcom_l2_pmu.c b/drivers/perf/qcom_l2_pmu.c new file mode 100644 index 0000000..c7b159f --- /dev/null +++ b/drivers/perf/qcom_l2_pmu.c @@ -0,0 +1,947 @@ +/* Copyright (c) 2015,2016 The Linux Foundation. All rights reserved. + * + * This program is free software; you can redistribute it and/or modify + * it under the terms of the GNU General Public License version 2 and + * only version 2 as published by the Free Software Foundation. + * + * This program is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + */ +#include <linux/acpi.h> +#include <linux/interrupt.h> +#include <linux/perf_event.h> +#include <linux/platform_device.h> + +#define MAX_L2_CTRS 9 + +#define L2PMCR_NUM_EV_SHIFT 11 +#define L2PMCR_NUM_EV_MASK 0x1F + +#define L2PMCR 0x400 +#define L2PMCNTENCLR 0x403 +#define L2PMCNTENSET 0x404 +#define L2PMINTENCLR 0x405 +#define L2PMINTENSET 0x406 +#define L2PMOVSCLR 0x407 +#define L2PMOVSSET 0x408 +#define L2PMCCNTCR 0x409 +#define L2PMCCNTR 0x40A +#define L2PMCCNTSR 0x40C +#define L2PMRESR 0x410 +#define IA_L2PMXEVCNTCR_BASE 0x420 +#define IA_L2PMXEVCNTR_BASE 0x421 +#define IA_L2PMXEVFILTER_BASE 0x423 +#define IA_L2PMXEVTYPER_BASE 0x424 + +#define IA_L2_REG_OFFSET 0x10 + +#define L2PMXEVFILTER_SUFILTER_ALL 0x000E0000 +#define L2PMXEVFILTER_ORGFILTER_IDINDEP 0x00000004 +#define L2PMXEVFILTER_ORGFILTER_ALL 0x00000003 + +#define L2PM_CC_ENABLE 0x80000000 + +#define L2EVTYPER_REG_SHIFT 3 + +#define L2PMRESR_GROUP_BITS 8 +#define L2PMRESR_GROUP_MASK GENMASK(7, 0) + +#define L2CYCLE_CTR_BIT 31 +#define L2CYCLE_CTR_RAW_CODE 0xFE + +#define L2PMCR_RESET_ALL 0x6 +#define L2PMCR_COUNTERS_ENABLE 0x1 +#define L2PMCR_COUNTERS_DISABLE 0x0 + +#define L2PMRESR_EN ((u64)1 << 63) + +#define L2_EVT_MASK 0x00000FFF +#define L2_EVT_CODE_MASK 0x00000FF0 +#define L2_EVT_GRP_MASK 0x0000000F +#define L2_EVT_CODE_SHIFT 4 +#define L2_EVT_GRP_SHIFT 0 + +#define L2_EVT_CODE(event) (((event) & L2_EVT_CODE_MASK) >> L2_EVT_CODE_SHIFT) +#define L2_EVT_GROUP(event) (((event) & L2_EVT_GRP_MASK) >> L2_EVT_GRP_SHIFT) + +#define L2_EVT_GROUP_MAX 7 + +#define L2_MAX_PERIOD U32_MAX +#define L2_CNT_PERIOD (U32_MAX - GENMASK(26, 0)) + +#define L2CPUSRSELR_EL1 S3_3_c15_c0_6 +#define L2CPUSRDR_EL1 S3_3_c15_c0_7 + +static DEFINE_RAW_SPINLOCK(l2_access_lock); + +/** + * set_l2_indirect_reg: write value to an L2 register + * @reg: Address of L2 register. + * @value: Value to be written to register. + * + * Use architecturally required barriers for ordering between system register + * accesses + */ +static void set_l2_indirect_reg(u64 reg, u64 val) +{ + unsigned long flags; + + raw_spin_lock_irqsave(&l2_access_lock, flags); + write_sysreg(reg, L2CPUSRSELR_EL1); + isb(); + write_sysreg(val, L2CPUSRDR_EL1); + isb(); + raw_spin_unlock_irqrestore(&l2_access_lock, flags); +} + +/** + * get_l2_indirect_reg: read an L2 register value + * @reg: Address of L2 register. + * + * Use architecturally required barriers for ordering between system register + * accesses + */ +static u64 get_l2_indirect_reg(u64 reg) +{ + u64 val; + unsigned long flags; + + raw_spin_lock_irqsave(&l2_access_lock, flags); + write_sysreg(reg, L2CPUSRSELR_EL1); + isb(); + val = read_sysreg(L2CPUSRDR_EL1); + raw_spin_unlock_irqrestore(&l2_access_lock, flags); + + return val; +} + +/* + * Aggregate PMU. Implements the core pmu functions and manages + * the hardware PMUs. + */ +struct l2cache_pmu { + struct hlist_node node; + u32 num_pmus; + struct pmu pmu; + int num_counters; + cpumask_t cpumask; + struct platform_device *pdev; +}; + +/* + * The cache is made up of one or more clusters, each cluster has its own PMU. + * Each cluster is associated with one or more CPUs. + * This structure represents one of the hardware PMUs. + * + * Events can be envisioned as a 2-dimensional array. Each column represents + * a group of events. There are 8 groups. Only one entry from each + * group can be in use at a time. When an event is assigned a counter + * by *_event_add(), the counter index is assigned to group_to_counter[group]. + * This allows *filter_match() to detect and reject conflicting events in + * the same group. + * Events are specified as 0xCCG, where CC is 2 hex digits specifying + * the code (array row) and G specifies the group (column). + * + * In addition there is a cycle counter event specified by L2CYCLE_CTR_RAW_CODE + * which is outside the above scheme. + */ +struct hml2_pmu { + struct perf_event *events[MAX_L2_CTRS]; + struct l2cache_pmu *l2cache_pmu; + DECLARE_BITMAP(used_counters, MAX_L2_CTRS); + DECLARE_BITMAP(used_groups, L2_EVT_GROUP_MAX + 1); + int group_to_counter[L2_EVT_GROUP_MAX + 1]; + int irq; + /* The CPU that is used for collecting events on this cluster */ + int on_cpu; + /* All the CPUs associated with this cluster */ + cpumask_t cluster_cpus; + spinlock_t pmu_lock; +}; + +#define to_l2cache_pmu(p) (container_of(p, struct l2cache_pmu, pmu)) + +static DEFINE_PER_CPU(struct hml2_pmu *, pmu_cluster); +static u32 l2_cycle_ctr_idx; +static u32 l2_counter_present_mask; + +static inline u32 idx_to_reg_bit(u32 idx) +{ + if (idx == l2_cycle_ctr_idx) + return BIT(L2CYCLE_CTR_BIT); + + return BIT(idx); +} + +static inline struct hml2_pmu *get_hml2_pmu(int cpu) +{ + return per_cpu(pmu_cluster, cpu); +} + +static void hml2_pmu__reset_on_cluster(void *x) +{ + /* Reset all ctrs */ + set_l2_indirect_reg(L2PMCR, L2PMCR_RESET_ALL); + set_l2_indirect_reg(L2PMCNTENCLR, l2_counter_present_mask); + set_l2_indirect_reg(L2PMINTENCLR, l2_counter_present_mask); + set_l2_indirect_reg(L2PMOVSCLR, l2_counter_present_mask); +} + +static inline void hml2_pmu__reset(struct hml2_pmu *cluster) +{ + cpumask_t *mask = &cluster->cluster_cpus; + + if (smp_call_function_any(mask, hml2_pmu__reset_on_cluster, NULL, 1)) + dev_err(&cluster->l2cache_pmu->pdev->dev, + "Failed to reset on cluster with cpu %d\n", + cpumask_first(&cluster->cluster_cpus)); +} + +static inline void hml2_pmu__enable(void) +{ + set_l2_indirect_reg(L2PMCR, L2PMCR_COUNTERS_ENABLE); +} + +static inline void hml2_pmu__disable(void) +{ + set_l2_indirect_reg(L2PMCR, L2PMCR_COUNTERS_DISABLE); +} + +static inline void hml2_pmu__counter_set_value(u32 idx, u64 value) +{ + u32 counter_reg; + + if (idx == l2_cycle_ctr_idx) { + set_l2_indirect_reg(L2PMCCNTR, value); + } else { + counter_reg = (idx * IA_L2_REG_OFFSET) + IA_L2PMXEVCNTR_BASE; + set_l2_indirect_reg(counter_reg, value & GENMASK(31, 0)); + } +} + +static inline u64 hml2_pmu__counter_get_value(u32 idx) +{ + u64 value; + u32 counter_reg; + + if (idx == l2_cycle_ctr_idx) { + value = get_l2_indirect_reg(L2PMCCNTR); + } else { + counter_reg = (idx * IA_L2_REG_OFFSET) + IA_L2PMXEVCNTR_BASE; + value = get_l2_indirect_reg(counter_reg); + } + + return value; +} + +static inline void hml2_pmu__counter_enable(u32 idx) +{ + set_l2_indirect_reg(L2PMCNTENSET, idx_to_reg_bit(idx)); +} + +static inline void hml2_pmu__counter_disable(u32 idx) +{ + set_l2_indirect_reg(L2PMCNTENCLR, idx_to_reg_bit(idx)); +} + +static inline void hml2_pmu__counter_enable_interrupt(u32 idx) +{ + set_l2_indirect_reg(L2PMINTENSET, idx_to_reg_bit(idx)); +} + +static inline void hml2_pmu__counter_disable_interrupt(u32 idx) +{ + set_l2_indirect_reg(L2PMINTENCLR, idx_to_reg_bit(idx)); +} + +static inline void hml2_pmu__set_evccntcr(u32 val) +{ + set_l2_indirect_reg(L2PMCCNTCR, val); +} + +static inline void hml2_pmu__set_evcntcr(u32 ctr, u32 val) +{ + u32 evtcr_reg = (ctr * IA_L2_REG_OFFSET) + IA_L2PMXEVCNTCR_BASE; + + set_l2_indirect_reg(evtcr_reg, val); +} + +static inline void hml2_pmu__set_evtyper(u32 ctr, u32 val) +{ + u32 evtype_reg = (ctr * IA_L2_REG_OFFSET) + IA_L2PMXEVTYPER_BASE; + + set_l2_indirect_reg(evtype_reg, val); +} + +static void hml2_pmu__set_resr(struct hml2_pmu *cluster, + u32 event_group, u32 event_cc) +{ + u64 field; + u64 resr_val; + u32 shift; + unsigned long flags; + + shift = L2PMRESR_GROUP_BITS * event_group; + field = ((u64)(event_cc & L2PMRESR_GROUP_MASK) << shift) | L2PMRESR_EN; + + spin_lock_irqsave(&cluster->pmu_lock, flags); + + resr_val = get_l2_indirect_reg(L2PMRESR); + resr_val &= ~(L2PMRESR_GROUP_MASK << shift); + resr_val |= field; + set_l2_indirect_reg(L2PMRESR, resr_val); + + spin_unlock_irqrestore(&cluster->pmu_lock, flags); +} + +/* + * Hardware allows filtering of events based on the originating + * CPU. Turn this off by setting filter bits to allow events from + * all CPUS, subunits and ID independent events in this cluster. + */ +static inline void hml2_pmu__set_evfilter_sys_mode(u32 ctr) +{ + u32 reg = (ctr * IA_L2_REG_OFFSET) + IA_L2PMXEVFILTER_BASE; + u32 val = L2PMXEVFILTER_SUFILTER_ALL | + L2PMXEVFILTER_ORGFILTER_IDINDEP | + L2PMXEVFILTER_ORGFILTER_ALL; + + set_l2_indirect_reg(reg, val); +} + +static inline u32 hml2_pmu__getreset_ovsr(void) +{ + u32 result = get_l2_indirect_reg(L2PMOVSSET); + + set_l2_indirect_reg(L2PMOVSCLR, result); + return result; +} + +static inline bool hml2_pmu__has_overflowed(u32 ovsr) +{ + return !!(ovsr & l2_counter_present_mask); +} + +static inline bool hml2_pmu__counter_has_overflowed(u32 ovsr, u32 idx) +{ + return !!(ovsr & idx_to_reg_bit(idx)); +} + +static void l2_cache__event_update_from_cluster(struct perf_event *event, + struct hml2_pmu *cluster) +{ + struct hw_perf_event *hwc = &event->hw; + u64 delta64, prev, now; + u32 delta; + u32 idx = hwc->idx; + + do { + prev = local64_read(&hwc->prev_count); + now = hml2_pmu__counter_get_value(idx); + } while (local64_cmpxchg(&hwc->prev_count, prev, now) != prev); + + if (idx == l2_cycle_ctr_idx) { + /* + * The cycle counter is 64-bit so needs separate handling + * of 64-bit delta. + */ + delta64 = now - prev; + local64_add(delta64, &event->count); + } else { + /* + * 32-bit counters need the unsigned 32-bit math to handle + * overflow and now < prev + */ + delta = now - prev; + local64_add(delta, &event->count); + } +} + +static void l2_cache__cluster_set_period(struct hml2_pmu *cluster, + struct hw_perf_event *hwc) +{ + u64 base = L2_MAX_PERIOD - (L2_CNT_PERIOD - 1); + u32 idx = hwc->idx; + u64 prev = local64_read(&hwc->prev_count); + u64 value; + + /* + * Limit the maximum period to prevent the counter value + * from overtaking the one we are about to program. + * Use a starting value which is high enough that after + * an overflow, interrupt latency will not cause the count + * to reach the base value. If the previous value + * is below the base, increase it to be above the base + * and update prev_count accordingly. Otherwise if + * the previous value is already above the base + * nothing needs to be done to prev_count. + */ + if (prev < base) { + value = base + prev; + local64_set(&hwc->prev_count, value); + } else { + value = prev; + } + + hml2_pmu__counter_set_value(idx, value); +} + +static int l2_cache__get_event_idx(struct hml2_pmu *cluster, + struct perf_event *event) +{ + struct hw_perf_event *hwc = &event->hw; + int idx; + + if (hwc->config_base == L2CYCLE_CTR_RAW_CODE) { + if (test_and_set_bit(l2_cycle_ctr_idx, cluster->used_counters)) + return -EAGAIN; + + return l2_cycle_ctr_idx; + } + + for (idx = 0; idx < cluster->l2cache_pmu->num_counters - 1; idx++) { + if (!test_and_set_bit(idx, cluster->used_counters)) { + set_bit(L2_EVT_GROUP(hwc->config_base), + cluster->used_groups); + return idx; + } + } + + /* The counters are all in use. */ + return -EAGAIN; +} + +static void l2_cache__clear_event_idx(struct hml2_pmu *cluster, + struct perf_event *event) +{ + struct hw_perf_event *hwc = &event->hw; + int idx = hwc->idx; + + clear_bit(idx, cluster->used_counters); + if (hwc->config_base != L2CYCLE_CTR_RAW_CODE) + clear_bit(L2_EVT_GROUP(hwc->config_base), cluster->used_groups); +} + +static irqreturn_t l2_cache__handle_irq(int irq_num, void *data) +{ + struct hml2_pmu *cluster = data; + int num_counters = cluster->l2cache_pmu->num_counters; + u32 ovsr; + int idx; + + ovsr = hml2_pmu__getreset_ovsr(); + if (!hml2_pmu__has_overflowed(ovsr)) + return IRQ_NONE; + + for_each_set_bit(idx, cluster->used_counters, num_counters) { + struct perf_event *event = cluster->events[idx]; + struct hw_perf_event *hwc; + + if (!hml2_pmu__counter_has_overflowed(ovsr, idx)) + continue; + + l2_cache__event_update_from_cluster(event, cluster); + hwc = &event->hw; + + l2_cache__cluster_set_period(cluster, hwc); + } + + return IRQ_HANDLED; +} + +/* + * Implementation of abstract pmu functionality required by + * the core perf events code. + */ + +static void l2_cache__pmu_enable(struct pmu *pmu) +{ + /* + * Although there is only one PMU (per socket) controlling multiple + * physical PMUs (per cluster), because we do not support per-task mode + * each event is associated with a CPU. Each event has pmu_enable + * called on its CPU, so here it is only necessary to enable the + * counters for the current CPU. + */ + + hml2_pmu__enable(); +} + +static void l2_cache__pmu_disable(struct pmu *pmu) +{ + hml2_pmu__disable(); +} + +static int l2_cache__event_init(struct perf_event *event) +{ + struct hw_perf_event *hwc = &event->hw; + struct hml2_pmu *cluster; + struct perf_event *sibling; + struct l2cache_pmu *l2cache_pmu; + + if (event->attr.type != event->pmu->type) + return -ENOENT; + + l2cache_pmu = to_l2cache_pmu(event->pmu); + + if (hwc->sample_period) { + dev_warn(&l2cache_pmu->pdev->dev, "Sampling not supported\n"); + return -EOPNOTSUPP; + } + + if (event->cpu < 0) { + dev_warn(&l2cache_pmu->pdev->dev, "Per-task mode not supported\n"); + return -EOPNOTSUPP; + } + + /* We cannot filter accurately so we just don't allow it. */ + if (event->attr.exclude_user || event->attr.exclude_kernel || + event->attr.exclude_hv || event->attr.exclude_idle) { + dev_warn(&l2cache_pmu->pdev->dev, "Can't exclude execution levels\n"); + return -EOPNOTSUPP; + } + + if (((L2_EVT_GROUP(event->attr.config) > L2_EVT_GROUP_MAX) || + ((event->attr.config & ~L2_EVT_MASK) != 0)) && + (event->attr.config != L2CYCLE_CTR_RAW_CODE)) { + dev_warn(&l2cache_pmu->pdev->dev, "Invalid config %llx\n", + event->attr.config); + return -EINVAL; + } + + /* Don't allow groups with mixed PMUs, except for s/w events */ + if (event->group_leader->pmu != event->pmu && + !is_software_event(event->group_leader)) { + dev_warn(&l2cache_pmu->pdev->dev, + "Can't create mixed PMU group\n"); + return -EINVAL; + } + + list_for_each_entry(sibling, &event->group_leader->sibling_list, + group_entry) + if (sibling->pmu != event->pmu && + !is_software_event(sibling)) { + dev_warn(&l2cache_pmu->pdev->dev, + "Can't create mixed PMU group\n"); + return -EINVAL; + } + + /* Ensure all events in a group are on the same cpu */ + cluster = get_hml2_pmu(event->cpu); + if ((event->group_leader != event) && + (cluster->on_cpu != event->group_leader->cpu)) { + dev_warn(&l2cache_pmu->pdev->dev, + "Can't create group on CPUs %d and %d", + event->cpu, event->group_leader->cpu); + return -EINVAL; + } + + hwc->idx = -1; + hwc->config_base = event->attr.config; + + /* + * Ensure all events are on the same cpu so all events are in the + * same cpu context, to avoid races on pmu_enable etc. + */ + event->cpu = cluster->on_cpu; + + return 0; +} + +static void l2_cache__event_start(struct perf_event *event, int flags) +{ + struct hml2_pmu *cluster; + struct hw_perf_event *hwc = &event->hw; + int idx = hwc->idx; + u32 config; + u32 event_cc, event_group; + + hwc->state = 0; + + cluster = get_hml2_pmu(event->cpu); + l2_cache__cluster_set_period(cluster, hwc); + + if (hwc->config_base == L2CYCLE_CTR_RAW_CODE) { + hml2_pmu__set_evccntcr(0x0); + } else { + config = hwc->config_base; + event_cc = L2_EVT_CODE(config); + event_group = L2_EVT_GROUP(config); + + hml2_pmu__set_evcntcr(idx, 0x0); + hml2_pmu__set_evtyper(idx, event_group); + hml2_pmu__set_resr(cluster, event_group, event_cc); + hml2_pmu__set_evfilter_sys_mode(idx); + } + + hml2_pmu__counter_enable_interrupt(idx); + hml2_pmu__counter_enable(idx); +} + +static void l2_cache__event_stop(struct perf_event *event, int flags) +{ + struct hml2_pmu *cluster; + struct hw_perf_event *hwc = &event->hw; + int idx = hwc->idx; + + if (!(hwc->state & PERF_HES_STOPPED)) { + cluster = get_hml2_pmu(event->cpu); + hml2_pmu__counter_disable_interrupt(idx); + hml2_pmu__counter_disable(idx); + + if (flags & PERF_EF_UPDATE) + l2_cache__event_update_from_cluster(event, cluster); + hwc->state |= PERF_HES_STOPPED | PERF_HES_UPTODATE; + } +} + +static int l2_cache__event_add(struct perf_event *event, int flags) +{ + struct hw_perf_event *hwc = &event->hw; + int idx; + int err = 0; + struct hml2_pmu *cluster; + + cluster = get_hml2_pmu(event->cpu); + + idx = l2_cache__get_event_idx(cluster, event); + if (idx < 0) { + err = idx; + return err; + } + + hwc->idx = idx; + hwc->state = PERF_HES_STOPPED | PERF_HES_UPTODATE; + cluster->events[idx] = event; + cluster->group_to_counter[L2_EVT_GROUP(hwc->config_base)] = idx; + local64_set(&hwc->prev_count, 0ULL); + + if (flags & PERF_EF_START) + l2_cache__event_start(event, flags); + + /* Propagate changes to the userspace mapping. */ + perf_event_update_userpage(event); + + return err; +} + +static void l2_cache__event_del(struct perf_event *event, int flags) +{ + struct hw_perf_event *hwc = &event->hw; + struct hml2_pmu *cluster; + int idx = hwc->idx; + + cluster = get_hml2_pmu(event->cpu); + l2_cache__event_stop(event, flags | PERF_EF_UPDATE); + cluster->events[idx] = NULL; + l2_cache__clear_event_idx(cluster, event); + + perf_event_update_userpage(event); +} + +static void l2_cache__event_read(struct perf_event *event) +{ + l2_cache__event_update_from_cluster(event, get_hml2_pmu(event->cpu)); +} + +static int l2_cache_filter_match(struct perf_event *event) +{ + struct hw_perf_event *hwc = &event->hw; + struct hml2_pmu *cluster = get_hml2_pmu(event->cpu); + unsigned int group = L2_EVT_GROUP(hwc->config_base); + + /* check for column exclusion: group already in use by another event */ + if (test_bit(group, cluster->used_groups) && + cluster->events[cluster->group_to_counter[group]] != event) + return 0; + + return 1; +} + +static ssize_t l2_cache_pmu_cpumask_show(struct device *dev, + struct device_attribute *attr, + char *buf) +{ + struct l2cache_pmu *l2cache_pmu = to_l2cache_pmu(dev_get_drvdata(dev)); + + return cpumap_print_to_pagebuf(true, buf, &l2cache_pmu->cpumask); +} + +static DEVICE_ATTR(cpumask, S_IRUGO, l2_cache_pmu_cpumask_show, NULL); + +static struct attribute *l2_cache_pmu_cpumask_attrs[] = { + &dev_attr_cpumask.attr, + NULL, +}; + +static struct attribute_group l2_cache_pmu_cpumask_group = { + .attrs = l2_cache_pmu_cpumask_attrs, +}; + +/* CCG format for perf RAW codes. */ +PMU_FORMAT_ATTR(l2_code, "config:4-11"); +PMU_FORMAT_ATTR(l2_group, "config:0-3"); +static struct attribute *l2_cache_pmu_formats[] = { + &format_attr_l2_code.attr, + &format_attr_l2_group.attr, + NULL, +}; + +static struct attribute_group l2_cache_pmu_format_group = { + .name = "format", + .attrs = l2_cache_pmu_formats, +}; + +static const struct attribute_group *l2_cache_pmu_attr_grps[] = { + &l2_cache_pmu_format_group, + &l2_cache_pmu_cpumask_group, + NULL, +}; + +/* + * Generic device handlers + */ + +static const struct acpi_device_id l2_cache_pmu_acpi_match[] = { + { "QCOM8130", }, + { } +}; + +static int get_num_counters(void) +{ + int val; + + val = get_l2_indirect_reg(L2PMCR); + + /* + * Read number of counters from L2PMCR and add 1 + * for the cycle counter. + */ + return ((val >> L2PMCR_NUM_EV_SHIFT) & L2PMCR_NUM_EV_MASK) + 1; +} + +static int l2cache_pmu_online_cpu(unsigned int cpu, struct hlist_node *node) +{ + struct hml2_pmu *cluster; + cpumask_t cluster_online_cpus; + struct l2cache_pmu *l2cache_pmu; + + l2cache_pmu = hlist_entry_safe(node, struct l2cache_pmu, node); + cluster = get_hml2_pmu(cpu); + cpumask_and(&cluster_online_cpus, &cluster->cluster_cpus, + cpu_online_mask); + + if (cpumask_weight(&cluster_online_cpus) == 1) { + /* all CPUs on this cluster were down, use this one */ + cluster->on_cpu = cpu; + cpumask_set_cpu(cpu, &l2cache_pmu->cpumask); + WARN_ON(irq_set_affinity(cluster->irq, cpumask_of(cpu))); + } + + return 0; +} + +static int l2cache_pmu_offline_cpu(unsigned int cpu, struct hlist_node *node) +{ + struct hml2_pmu *cluster; + struct l2cache_pmu *l2cache_pmu; + cpumask_t cluster_online_cpus; + unsigned int target; + + l2cache_pmu = hlist_entry_safe(node, struct l2cache_pmu, node); + + if (!cpumask_test_and_clear_cpu(cpu, &l2cache_pmu->cpumask)) + return 0; + cluster = get_hml2_pmu(cpu); + cpumask_and(&cluster_online_cpus, &cluster->cluster_cpus, + cpu_online_mask); + + /* Any other CPU for this cluster which is still online */ + target = cpumask_any_but(&cluster_online_cpus, cpu); + if (target >= nr_cpu_ids) + return 0; + + perf_pmu_migrate_context(&l2cache_pmu->pmu, cpu, target); + cluster->on_cpu = target; + cpumask_set_cpu(target, &l2cache_pmu->cpumask); + WARN_ON(irq_set_affinity(cluster->irq, cpumask_of(target))); + + return 0; +} + +static int l2_cache_pmu_probe_cluster(struct device *dev, void *data) +{ + struct platform_device *pdev = to_platform_device(dev->parent); + struct platform_device *sdev = to_platform_device(dev); + struct l2cache_pmu *l2cache_pmu = data; + struct hml2_pmu *cluster; + struct acpi_device *device; + unsigned long fw_cluster_id; + int cpu; + int err; + int irq; + + if (acpi_bus_get_device(ACPI_HANDLE(dev), &device)) + return -ENODEV; + + if (kstrtol(device->pnp.unique_id, 10, &fw_cluster_id) < 0) { + dev_err(&pdev->dev, "unable to read ACPI uid\n"); + return -ENODEV; + } + + irq = platform_get_irq(sdev, 0); + if (irq < 0) { + dev_err(&pdev->dev, + "Failed to get valid irq for cluster %ld\n", + fw_cluster_id); + return irq; + } + + cluster = devm_kzalloc(&pdev->dev, sizeof(*cluster), GFP_KERNEL); + if (!cluster) + return -ENOMEM; + + cluster->l2cache_pmu = l2cache_pmu; + for_each_present_cpu(cpu) { + if (topology_physical_package_id(cpu) == fw_cluster_id) { + cpumask_set_cpu(cpu, &cluster->cluster_cpus); + per_cpu(pmu_cluster, cpu) = cluster; + } + } + cluster->irq = irq; + + if (cpumask_empty(&cluster->cluster_cpus)) { + dev_err(&pdev->dev, "No CPUs found for L2 cache instance %ld\n", + fw_cluster_id); + return -ENODEV; + } + + /* Pick one CPU to be the preferred one to use in the cluster */ + cluster->on_cpu = cpumask_first(&cluster->cluster_cpus); + + if (irq_set_affinity(irq, cpumask_of(cluster->on_cpu))) { + dev_err(&pdev->dev, + "Unable to set irq affinity (irq=%d, cpu=%d)\n", + irq, cluster->on_cpu); + return -ENODEV; + } + + err = devm_request_irq(&pdev->dev, irq, l2_cache__handle_irq, + IRQF_NOBALANCING, "l2-cache-pmu", cluster); + if (err) { + dev_err(&pdev->dev, + "Unable to request IRQ%d for L2 PMU counters\n", irq); + return err; + } + + dev_info(&pdev->dev, + "Registered L2 cache PMU instance %ld with %d CPUs\n", + fw_cluster_id, cpumask_weight(&cluster->cluster_cpus)); + + cluster->pmu_lock = __SPIN_LOCK_UNLOCKED(cluster->pmu_lock); + cpumask_set_cpu(cluster->on_cpu, &l2cache_pmu->cpumask); + + hml2_pmu__reset(cluster); + l2cache_pmu->num_pmus++; + + return 0; +} + +static int l2_cache_pmu_probe(struct platform_device *pdev) +{ + int err; + struct l2cache_pmu *l2cache_pmu; + + l2cache_pmu = + devm_kzalloc(&pdev->dev, sizeof(*l2cache_pmu), GFP_KERNEL); + if (!l2cache_pmu) + return -ENOMEM; + + platform_set_drvdata(pdev, l2cache_pmu); + l2cache_pmu->pmu = (struct pmu) { + /* suffix is instance id for future use with multiple sockets */ + .name = "l2cache_0", + .task_ctx_nr = perf_invalid_context, + .pmu_enable = l2_cache__pmu_enable, + .pmu_disable = l2_cache__pmu_disable, + .event_init = l2_cache__event_init, + .add = l2_cache__event_add, + .del = l2_cache__event_del, + .start = l2_cache__event_start, + .stop = l2_cache__event_stop, + .read = l2_cache__event_read, + .attr_groups = l2_cache_pmu_attr_grps, + .filter_match = l2_cache_filter_match, + }; + + l2cache_pmu->num_counters = get_num_counters(); + l2cache_pmu->pdev = pdev; + l2_cycle_ctr_idx = l2cache_pmu->num_counters - 1; + l2_counter_present_mask = GENMASK(l2cache_pmu->num_counters - 2, 0) | + L2PM_CC_ENABLE; + + cpumask_clear(&l2cache_pmu->cpumask); + + /* Read cluster info and initialize each cluster */ + err = device_for_each_child(&pdev->dev, l2cache_pmu, + l2_cache_pmu_probe_cluster); + if (err < 0) + return err; + + if (l2cache_pmu->num_pmus == 0) { + dev_err(&pdev->dev, "No hardware L2 cache PMUs found\n"); + return -ENODEV; + } + + err = perf_pmu_register(&l2cache_pmu->pmu, l2cache_pmu->pmu.name, -1); + if (err < 0) { + dev_err(&pdev->dev, "Error %d registering L2 cache PMU\n", err); + return err; + } + + dev_info(&pdev->dev, "Registered L2 cache PMU using %d HW PMUs\n", + l2cache_pmu->num_pmus); + + err = cpuhp_state_add_instance_nocalls(CPUHP_AP_PERF_ARM_QCOM_L2_ONLINE, + &l2cache_pmu->node); + + return err; +} + +static int l2_cache_pmu_remove(struct platform_device *pdev) +{ + struct l2cache_pmu *l2cache_pmu = + to_l2cache_pmu(platform_get_drvdata(pdev)); + + cpuhp_state_remove_instance_nocalls(CPUHP_AP_PERF_ARM_QCOM_L2_ONLINE, + &l2cache_pmu->node); + perf_pmu_unregister(&l2cache_pmu->pmu); + return 0; +} + +static struct platform_driver l2_cache_pmu_driver = { + .driver = { + .name = "qcom-l2cache-pmu", + .owner = THIS_MODULE, + .acpi_match_table = ACPI_PTR(l2_cache_pmu_acpi_match), + }, + .probe = l2_cache_pmu_probe, + .remove = l2_cache_pmu_remove, +}; + +static int __init register_l2_cache_pmu_driver(void) +{ + int err; + + err = cpuhp_setup_state_multi(CPUHP_AP_PERF_ARM_QCOM_L2_ONLINE, + "AP_PERF_ARM_QCOM_L2_ONLINE", + l2cache_pmu_online_cpu, + l2cache_pmu_offline_cpu); + if (err) + return err; + + return platform_driver_register(&l2_cache_pmu_driver); +} +device_initcall(register_l2_cache_pmu_driver); diff --git a/include/linux/cpuhotplug.h b/include/linux/cpuhotplug.h index 9b207a8..fbe33d2 100644 --- a/include/linux/cpuhotplug.h +++ b/include/linux/cpuhotplug.h @@ -117,6 +117,7 @@ enum cpuhp_state { CPUHP_AP_PERF_ARM_CCI_ONLINE, CPUHP_AP_PERF_ARM_CCN_ONLINE, CPUHP_AP_PERF_ARM_L2X0_ONLINE, + CPUHP_AP_PERF_ARM_QCOM_L2_ONLINE, CPUHP_AP_WORKQUEUE_ONLINE, CPUHP_AP_RCUTREE_ONLINE, CPUHP_AP_NOTIFY_ONLINE,
Adds perf events support for L2 cache PMU. The L2 cache PMU driver is named 'l2cache_0' and can be used with perf events to profile L2 events such as cache hits and misses on Qualcomm Technologies processors. Signed-off-by: Neil Leeder <nleeder@codeaurora.org> --- v7: Move to drivers/perf Rebased on 4.9-rc1 Successfully ran perf fuzzer against driver v6: restore accidentally dropped Kconfig dependencies v5: Fold the header and l2-accessors into .c file Use multi-instance framework for hotplug Change terminology from slice to cluster for clarity Remove unnecessary rmw sequence for enable registers Use prev_count in hwc rather than in slice Enforce all events in same group on same CPU Add comments, rename variables for clarity v4: Replace notifier with hotplug statemachine Allocate PMU struct dynamically v3: Remove exports from l2-accessors Change l2-accessors Kconfig to make it not user-selectable Reorder and remove unnecessary includes v2: Add the l2-accessors patch to this patchset, previously posted separately. Remove sampling and per-task functionality for this uncore PMU. Use cpumask to replace code which filtered events to one cpu per slice. Replace manual event filtering with filter_match callback. Use a separate used_mask for event groups. Add hotplug notifier for CPU and irq migration. Remove extraneous synchronisation instructions. Other miscellaneous cleanup. drivers/perf/Kconfig | 9 + drivers/perf/Makefile | 1 + drivers/perf/qcom_l2_pmu.c | 947 +++++++++++++++++++++++++++++++++++++++++++++ include/linux/cpuhotplug.h | 1 + 4 files changed, 958 insertions(+) create mode 100644 drivers/perf/qcom_l2_pmu.c