| Message ID | 20240710045117.3164577-1-zhao1.liu@intel.com (mailing list archive) |
|---|---|
| Headers | show |
| Series | accel/kvm: Support KVM PMU filter | expand |
On 7/10/2024 12:51 PM, Zhao Liu wrote: > Hi QEMU maintainers, arm and PMU folks, > > I picked up Shaoqing's previous work [1] on the KVM PMU filter for arm, > and now is trying to support this feature for x86 with a JSON-compatible > API. > > While arm and x86 use different KVM ioctls to configure the PMU filter, > considering they all have similar inputs (PMU event + action), it is > still possible to abstract a generic, cross-architecture kvm-pmu-filter > object and provide users with a sufficiently generic or near-consistent > QAPI interface. > > That's what I did in this series, a new kvm-pmu-filter object, with the > API like: > > -object '{"qom-type":"kvm-pmu-filter","id":"f0","events":[{"action":"allow","format":"raw","code":"0xc4"}]}' I'm not sure why the action is defined to event scope instead of the entire filter scope. At least for x86 arch, the action is defined to filter scope in KVM. Do other archs support event scope's action? If the action can be defined to filter scope, the code could be simplified much. > > For i386, this object is inserted into kvm accelerator and is extended > to support fixed-counter and more formats ("x86-default" and > "x86-masked-entry"): > > -accel kvm,pmu-filter=f0 \ > -object pmu='{"qom-type":"kvm-pmu-filter","id":"f0","x86-fixed-counter":{"action":"allow","bitmap":"0x0"},"events":[{"action":"allow","format":"x86-masked-entry","select":"0xc4","mask":"0xff","match":"0","exclude":true},{"action":"allow","format":"x86-masked-entry","select":"0xc5","mask":"0xff","match":"0","exclude":true}]}' > > This object can still be added as the property to the arch CPU if it is > desired as a per CPU feature (as Shaoqin did for arm before). > > Welcome your feedback and comments! > > > Introduction > ============ > > > Formats supported in kvm-pmu-filter > ----------------------------------- > > This series supports 3 formats: > > * raw format (general format). > > This format indicates the code that has been encoded to be able to > index the PMU events, and which can be delivered directly to the KVM > ioctl. For arm, this means the event code, and for i386, this means > the raw event with the layout like: > > select high bit | umask | select low bits > > * x86-default format (i386 specific) > > x86 commonly uses select&umask to identify PMU events, and this format > is used to support the select&umask. Then QEMU will encode select and > umask into a raw format code. > > * x86-masked-entry (i386 specific) > > This is a special format that x86's KVM_SET_PMU_EVENT_FILTER supports. > > > Hexadecimal value string > ------------------------ > > In practice, the values associated with PMU events (code for arm, select& > umask for x86) are often expressed in hexadecimal. Further, from linux > perf related information (tools/perf/pmu-events/arch/*/*/*.json), x86/ > arm64/riscv/nds32/powerpc all prefer the hexadecimal numbers and only > s390 uses decimal value. > > Therefore, it is necessary to support hexadecimal in order to honor PMU > conventions. > > However, unfortunately, standard JSON (RFC 8259) does not support > hexadecimal numbers. So I can only consider using the numeric string in > the QAPI and then parsing it to a number. > > To achieve this, I defined two versions of PMU-related structures in > kvm.json: > * a native version that accepts numeric values, which is used for > QEMU's internal code processing, > > * and a variant version that accepts numeric string, which is used to > receive user input. > > kvm-pmu-filter object will take care of converting the string version > of the event/counter information into the numeric version. > > The related implementation can be found in patch 1. > > > CPU property v.s. KVM property > ------------------------------ > > In Shaoqin's previous implementation [1], KVM PMU filter is made as a > arm CPU property. This is because arm uses a per CPU ioctl > (KVM_SET_DEVICE_ATTR) to configure KVM PMU filter. > > However, for x86, the dependent ioctl (KVM_SET_PMU_EVENT_FILTER) is per > VM. In the meantime, considering that for hybrid architecture, maybe in > the future there will be a new per vCPU ioctl, or there will be > practices to support filter fixed counter by configuring CPUIDs. > > Based on the above thoughts, for x86, it is not appropriate to make the > current per-VM ioctl-based PMU filter a CPU property. Instead, I make it > a kvm property and configure it via "-accel kvm,pmu-filter=obj_id". > > So in summary, it is feasible to use the KVM PMU filter as either a CPU > or a KVM property, depending on whether it is used as a CPU feature or a > VM feature. > > The kvm-pmu-filter object, as an abstraction, is general enough to > support filter configurations for different scopes (per-CPU or per-VM). Per my understanding, the cpus sharing same uarch should share same perf events filter. Not sure if there is real requirement to support different filters on the CPUs with same arch. > > > [1]: https://lore.kernel.org/qemu-devel/20240409024940.180107-1-shahuang@redhat.com/ > > Thanks and Best Regards, > Zhao > --- > Zhao Liu (5): > qapi/qom: Introduce kvm-pmu-filter object > i386/kvm: Support initial KVM PMU filter > i386/kvm: Support event with select&umask format in KVM PMU filter > i386/kvm: Support event with masked entry format in KVM PMU filter > i386/kvm: Support fixed counter in KVM PMU filter > > MAINTAINERS | 1 + > accel/kvm/kvm-pmu.c | 367 +++++++++++++++++++++++++++++++++++++ > accel/kvm/meson.build | 1 + > include/sysemu/kvm-pmu.h | 43 +++++ > include/sysemu/kvm_int.h | 2 + > qapi/kvm.json | 255 ++++++++++++++++++++++++++ > qapi/meson.build | 1 + > qapi/qapi-schema.json | 1 + > qapi/qom.json | 3 + > target/i386/kvm/kvm.c | 211 +++++++++++++++++++++ > target/i386/kvm/kvm_i386.h | 1 + > 11 files changed, 886 insertions(+) > create mode 100644 accel/kvm/kvm-pmu.c > create mode 100644 include/sysemu/kvm-pmu.h > create mode 100644 qapi/kvm.json >
Also ping Shaoqin and Eric, Hopefully I can get your input on interface design to align with the Arm requirements together! On Thu, Jul 18, 2024 at 01:27:48PM +0800, Mi, Dapeng wrote: > Date: Thu, 18 Jul 2024 13:27:48 +0800 > From: "Mi, Dapeng" <dapeng1.mi@linux.intel.com> > Subject: Re: [RFC 0/5] accel/kvm: Support KVM PMU filter > > > On 7/10/2024 12:51 PM, Zhao Liu wrote: > > Hi QEMU maintainers, arm and PMU folks, > > > > I picked up Shaoqing's previous work [1] on the KVM PMU filter for arm, > > and now is trying to support this feature for x86 with a JSON-compatible > > API. > > > > While arm and x86 use different KVM ioctls to configure the PMU filter, > > considering they all have similar inputs (PMU event + action), it is > > still possible to abstract a generic, cross-architecture kvm-pmu-filter > > object and provide users with a sufficiently generic or near-consistent > > QAPI interface. > > > > That's what I did in this series, a new kvm-pmu-filter object, with the > > API like: > > > > -object '{"qom-type":"kvm-pmu-filter","id":"f0","events":[{"action":"allow","format":"raw","code":"0xc4"}]}' > > I'm not sure why the action is defined to event scope instead of the entire > filter scope. At least for x86 arch, the action is defined to filter scope > in KVM. Do other archs support event scope's action? If the action can be > defined to filter scope, the code could be simplified much. Yeah, that's fine for x86. @Shaoqin, hi Shaoqin, if I use a global action for all events, is this Okay for Arm? I suppose this should work, since the interface here needs to enumerate events one by one, and doesn't support a continuous enumeration format (0x0-0x3) like the previous implementation. Right? > > > > For i386, this object is inserted into kvm accelerator and is extended > > to support fixed-counter and more formats ("x86-default" and > > "x86-masked-entry"): > > > > -accel kvm,pmu-filter=f0 \ > > -object pmu='{"qom-type":"kvm-pmu-filter","id":"f0","x86-fixed-counter":{"action":"allow","bitmap":"0x0"},"events":[{"action":"allow","format":"x86-masked-entry","select":"0xc4","mask":"0xff","match":"0","exclude":true},{"action":"allow","format":"x86-masked-entry","select":"0xc5","mask":"0xff","match":"0","exclude":true}]}' > > > > This object can still be added as the property to the arch CPU if it is > > desired as a per CPU feature (as Shaoqin did for arm before). > > > > Welcome your feedback and comments! > > > > > > Introduction > > ============ > > > > > > Formats supported in kvm-pmu-filter > > ----------------------------------- > > > > This series supports 3 formats: > > > > * raw format (general format). > > > > This format indicates the code that has been encoded to be able to > > index the PMU events, and which can be delivered directly to the KVM > > ioctl. For arm, this means the event code, and for i386, this means > > the raw event with the layout like: > > > > select high bit | umask | select low bits > > > > * x86-default format (i386 specific) > > > > x86 commonly uses select&umask to identify PMU events, and this format > > is used to support the select&umask. Then QEMU will encode select and > > umask into a raw format code. > > > > * x86-masked-entry (i386 specific) > > > > This is a special format that x86's KVM_SET_PMU_EVENT_FILTER supports. > > > > > > Hexadecimal value string > > ------------------------ > > > > In practice, the values associated with PMU events (code for arm, select& > > umask for x86) are often expressed in hexadecimal. Further, from linux > > perf related information (tools/perf/pmu-events/arch/*/*/*.json), x86/ > > arm64/riscv/nds32/powerpc all prefer the hexadecimal numbers and only > > s390 uses decimal value. > > > > Therefore, it is necessary to support hexadecimal in order to honor PMU > > conventions. > > > > However, unfortunately, standard JSON (RFC 8259) does not support > > hexadecimal numbers. So I can only consider using the numeric string in > > the QAPI and then parsing it to a number. > > > > To achieve this, I defined two versions of PMU-related structures in > > kvm.json: > > * a native version that accepts numeric values, which is used for > > QEMU's internal code processing, > > > > * and a variant version that accepts numeric string, which is used to > > receive user input. > > > > kvm-pmu-filter object will take care of converting the string version > > of the event/counter information into the numeric version. > > > > The related implementation can be found in patch 1. > > > > > > CPU property v.s. KVM property > > ------------------------------ > > > > In Shaoqin's previous implementation [1], KVM PMU filter is made as a > > arm CPU property. This is because arm uses a per CPU ioctl > > (KVM_SET_DEVICE_ATTR) to configure KVM PMU filter. > > > > However, for x86, the dependent ioctl (KVM_SET_PMU_EVENT_FILTER) is per > > VM. In the meantime, considering that for hybrid architecture, maybe in > > the future there will be a new per vCPU ioctl, or there will be > > practices to support filter fixed counter by configuring CPUIDs. > > > > Based on the above thoughts, for x86, it is not appropriate to make the > > current per-VM ioctl-based PMU filter a CPU property. Instead, I make it > > a kvm property and configure it via "-accel kvm,pmu-filter=obj_id". > > > > So in summary, it is feasible to use the KVM PMU filter as either a CPU > > or a KVM property, depending on whether it is used as a CPU feature or a > > VM feature. > > > > The kvm-pmu-filter object, as an abstraction, is general enough to > > support filter configurations for different scopes (per-CPU or per-VM). > > Per my understanding, the cpus sharing same uarch should share same perf > events filter. Not sure if there is real requirement to support different > filters on the CPUs with same arch. The main consideration here is also to align with the underlying ioctl of the PMU filter, Arm's ioctl is per vCPU and I'm not sure if there will be a difference between CPUs in the feature. But this I think could be an arch-specific implementation, x86 needs per VM then use “-accel kvm,pmu-filter=obj_id”, Arm if it does need per vCPU then it can be set for Arm CPUs via “-cpu host,pmu-filter=obj_id”. So the generic part is this kvm-pmu-filter object itself, where exactly it should be placed/inserted can be decided by the architecture itself. Hi @Eric and @Shaoqin, do you agree? Best Regards, Zhao
Hi Zhao, On 7/10/24 12:51, Zhao Liu wrote: > Hi QEMU maintainers, arm and PMU folks, > > I picked up Shaoqing's previous work [1] on the KVM PMU filter for arm, > and now is trying to support this feature for x86 with a JSON-compatible > API. > > While arm and x86 use different KVM ioctls to configure the PMU filter, > considering they all have similar inputs (PMU event + action), it is > still possible to abstract a generic, cross-architecture kvm-pmu-filter > object and provide users with a sufficiently generic or near-consistent > QAPI interface. > > That's what I did in this series, a new kvm-pmu-filter object, with the > API like: > > -object '{"qom-type":"kvm-pmu-filter","id":"f0","events":[{"action":"allow","format":"raw","code":"0xc4"}]}' > > For i386, this object is inserted into kvm accelerator and is extended > to support fixed-counter and more formats ("x86-default" and > "x86-masked-entry"): > > -accel kvm,pmu-filter=f0 \ > -object pmu='{"qom-type":"kvm-pmu-filter","id":"f0","x86-fixed-counter":{"action":"allow","bitmap":"0x0"},"events":[{"action":"allow","format":"x86-masked-entry","select":"0xc4","mask":"0xff","match":"0","exclude":true},{"action":"allow","format":"x86-masked-entry","select":"0xc5","mask":"0xff","match":"0","exclude":true}]}' What if I want to create the PMU Filter on ARM to deny the event range [0x5,0x10], and allow deny event 0x13, how should I write the json? Thanks, Shaoqin > > This object can still be added as the property to the arch CPU if it is > desired as a per CPU feature (as Shaoqin did for arm before). > > Welcome your feedback and comments! > > > Introduction > ============ > > > Formats supported in kvm-pmu-filter > ----------------------------------- > > This series supports 3 formats: > > * raw format (general format). > > This format indicates the code that has been encoded to be able to > index the PMU events, and which can be delivered directly to the KVM > ioctl. For arm, this means the event code, and for i386, this means > the raw event with the layout like: > > select high bit | umask | select low bits > > * x86-default format (i386 specific) > > x86 commonly uses select&umask to identify PMU events, and this format > is used to support the select&umask. Then QEMU will encode select and > umask into a raw format code. > > * x86-masked-entry (i386 specific) > > This is a special format that x86's KVM_SET_PMU_EVENT_FILTER supports. > > > Hexadecimal value string > ------------------------ > > In practice, the values associated with PMU events (code for arm, select& > umask for x86) are often expressed in hexadecimal. Further, from linux > perf related information (tools/perf/pmu-events/arch/*/*/*.json), x86/ > arm64/riscv/nds32/powerpc all prefer the hexadecimal numbers and only > s390 uses decimal value. > > Therefore, it is necessary to support hexadecimal in order to honor PMU > conventions. > > However, unfortunately, standard JSON (RFC 8259) does not support > hexadecimal numbers. So I can only consider using the numeric string in > the QAPI and then parsing it to a number. > > To achieve this, I defined two versions of PMU-related structures in > kvm.json: > * a native version that accepts numeric values, which is used for > QEMU's internal code processing, > > * and a variant version that accepts numeric string, which is used to > receive user input. > > kvm-pmu-filter object will take care of converting the string version > of the event/counter information into the numeric version. > > The related implementation can be found in patch 1. > > > CPU property v.s. KVM property > ------------------------------ > > In Shaoqin's previous implementation [1], KVM PMU filter is made as a > arm CPU property. This is because arm uses a per CPU ioctl > (KVM_SET_DEVICE_ATTR) to configure KVM PMU filter. > > However, for x86, the dependent ioctl (KVM_SET_PMU_EVENT_FILTER) is per > VM. In the meantime, considering that for hybrid architecture, maybe in > the future there will be a new per vCPU ioctl, or there will be > practices to support filter fixed counter by configuring CPUIDs. > > Based on the above thoughts, for x86, it is not appropriate to make the > current per-VM ioctl-based PMU filter a CPU property. Instead, I make it > a kvm property and configure it via "-accel kvm,pmu-filter=obj_id". > > So in summary, it is feasible to use the KVM PMU filter as either a CPU > or a KVM property, depending on whether it is used as a CPU feature or a > VM feature. > > The kvm-pmu-filter object, as an abstraction, is general enough to > support filter configurations for different scopes (per-CPU or per-VM). > > > [1]: https://lore.kernel.org/qemu-devel/20240409024940.180107-1-shahuang@redhat.com/ > > Thanks and Best Regards, > Zhao > --- > Zhao Liu (5): > qapi/qom: Introduce kvm-pmu-filter object > i386/kvm: Support initial KVM PMU filter > i386/kvm: Support event with select&umask format in KVM PMU filter > i386/kvm: Support event with masked entry format in KVM PMU filter > i386/kvm: Support fixed counter in KVM PMU filter > > MAINTAINERS | 1 + > accel/kvm/kvm-pmu.c | 367 +++++++++++++++++++++++++++++++++++++ > accel/kvm/meson.build | 1 + > include/sysemu/kvm-pmu.h | 43 +++++ > include/sysemu/kvm_int.h | 2 + > qapi/kvm.json | 255 ++++++++++++++++++++++++++ > qapi/meson.build | 1 + > qapi/qapi-schema.json | 1 + > qapi/qom.json | 3 + > target/i386/kvm/kvm.c | 211 +++++++++++++++++++++ > target/i386/kvm/kvm_i386.h | 1 + > 11 files changed, 886 insertions(+) > create mode 100644 accel/kvm/kvm-pmu.c > create mode 100644 include/sysemu/kvm-pmu.h > create mode 100644 qapi/kvm.json >
Hello Shaoqin, On Fri, Aug 02, 2024 at 05:01:47PM +0800, Shaoqin Huang wrote: > Date: Fri, 2 Aug 2024 17:01:47 +0800 > From: Shaoqin Huang <shahuang@redhat.com> > Subject: Re: [RFC 0/5] accel/kvm: Support KVM PMU filter > > Hi Zhao, > > On 7/10/24 12:51, Zhao Liu wrote: > > Hi QEMU maintainers, arm and PMU folks, > > > > I picked up Shaoqing's previous work [1] on the KVM PMU filter for arm, > > and now is trying to support this feature for x86 with a JSON-compatible > > API. > > > > While arm and x86 use different KVM ioctls to configure the PMU filter, > > considering they all have similar inputs (PMU event + action), it is > > still possible to abstract a generic, cross-architecture kvm-pmu-filter > > object and provide users with a sufficiently generic or near-consistent > > QAPI interface. > > > > That's what I did in this series, a new kvm-pmu-filter object, with the > > API like: > > > > -object '{"qom-type":"kvm-pmu-filter","id":"f0","events":[{"action":"allow","format":"raw","code":"0xc4"}]}' > > > > For i386, this object is inserted into kvm accelerator and is extended > > to support fixed-counter and more formats ("x86-default" and > > "x86-masked-entry"): > > > > -accel kvm,pmu-filter=f0 \ > > -object pmu='{"qom-type":"kvm-pmu-filter","id":"f0","x86-fixed-counter":{"action":"allow","bitmap":"0x0"},"events":[{"action":"allow","format":"x86-masked-entry","select":"0xc4","mask":"0xff","match":"0","exclude":true},{"action":"allow","format":"x86-masked-entry","select":"0xc5","mask":"0xff","match":"0","exclude":true}]}' > > What if I want to create the PMU Filter on ARM to deny the event range > [0x5,0x10], and allow deny event 0x13, how should I write the json? > Cuurently this doesn't support the event range (since the raw format of x86 event cannot be said to be continuous). So with the basic support, we need to configure events one by one: -object pmu='{"qom-type":"kvm-pmu-filter","id":"f0","events":[{"action":"allow","format":"raw","code":"0x5"},{"action":"allow","format":"raw","select":"0x6"},{"action":"allow","format":"raw","code":"0x7"},{"action":"allow","format":"raw","code":"0x8"},{"action":"allow","format":"raw","code":"0x9"},{"action":"allow","format":"raw","code":"0x10"},{"action":"deny","format":"raw","code":"0x13"}]}' This one looks a lot more complicated, but in the future, arm could further support event-range (maybe implement event-range via mask), but I think this could be arch-specific format since not all architectures' events are continuous. Additional, I'm a bit confused by your example, and I hope you can help me understand that: when configuring 0x5~0x10 to be allow, isn't it true that all other events are denied by default, so denying 0x13 again is a redundant operation? What is the default action for all other events except 0x5~0x10 and 0x13? If we specify action as allow for 0x5~0x10 and deny for the rest by default, then there is no need to set an action for each event but only a global one (as suggested by Dapeng), so the above command line can be simplified as: -object pmu='{"qom-type":"kvm-pmu-filter","id":"f0","action":"allow","events":[{"format":"raw","code":"0x5"},{"format":"raw","select":"0x6"},{"format":"raw","code":"0x7"},{"format":"raw","code":"0x8"},{"format":"raw","code":"0x9"},{"format":"raw","code":"0x10"}]}' Thanks, Zhao
Hi Zhao, On 8/2/24 17:37, Zhao Liu wrote: > Hello Shaoqin, > > On Fri, Aug 02, 2024 at 05:01:47PM +0800, Shaoqin Huang wrote: >> Date: Fri, 2 Aug 2024 17:01:47 +0800 >> From: Shaoqin Huang <shahuang@redhat.com> >> Subject: Re: [RFC 0/5] accel/kvm: Support KVM PMU filter >> >> Hi Zhao, >> >> On 7/10/24 12:51, Zhao Liu wrote: >>> Hi QEMU maintainers, arm and PMU folks, >>> >>> I picked up Shaoqing's previous work [1] on the KVM PMU filter for arm, >>> and now is trying to support this feature for x86 with a JSON-compatible >>> API. >>> >>> While arm and x86 use different KVM ioctls to configure the PMU filter, >>> considering they all have similar inputs (PMU event + action), it is >>> still possible to abstract a generic, cross-architecture kvm-pmu-filter >>> object and provide users with a sufficiently generic or near-consistent >>> QAPI interface. >>> >>> That's what I did in this series, a new kvm-pmu-filter object, with the >>> API like: >>> >>> -object '{"qom-type":"kvm-pmu-filter","id":"f0","events":[{"action":"allow","format":"raw","code":"0xc4"}]}' >>> >>> For i386, this object is inserted into kvm accelerator and is extended >>> to support fixed-counter and more formats ("x86-default" and >>> "x86-masked-entry"): >>> >>> -accel kvm,pmu-filter=f0 \ >>> -object pmu='{"qom-type":"kvm-pmu-filter","id":"f0","x86-fixed-counter":{"action":"allow","bitmap":"0x0"},"events":[{"action":"allow","format":"x86-masked-entry","select":"0xc4","mask":"0xff","match":"0","exclude":true},{"action":"allow","format":"x86-masked-entry","select":"0xc5","mask":"0xff","match":"0","exclude":true}]}' >> >> What if I want to create the PMU Filter on ARM to deny the event range >> [0x5,0x10], and allow deny event 0x13, how should I write the json? >> > > Cuurently this doesn't support the event range (since the raw format of > x86 event cannot be said to be continuous). > > So with the basic support, we need to configure events one by one: > > -object pmu='{"qom-type":"kvm-pmu-filter","id":"f0","events":[{"action":"allow","format":"raw","code":"0x5"},{"action":"allow","format":"raw","select":"0x6"},{"action":"allow","format":"raw","code":"0x7"},{"action":"allow","format":"raw","code":"0x8"},{"action":"allow","format":"raw","code":"0x9"},{"action":"allow","format":"raw","code":"0x10"},{"action":"deny","format":"raw","code":"0x13"}]}' > > This one looks a lot more complicated, but in the future, arm could > further support event-range (maybe implement event-range via mask), but > I think this could be arch-specific format since not all architectures' > events are continuous. > > Additional, I'm a bit confused by your example, and I hope you can help > me understand that: when configuring 0x5~0x10 to be allow, isn't it true > that all other events are denied by default, so denying 0x13 again is a > redundant operation? What is the default action for all other events > except 0x5~0x10 and 0x13? > > If we specify action as allow for 0x5~0x10 and deny for the rest by > default, then there is no need to set an action for each event but only > a global one (as suggested by Dapeng), so the above command line can be > simplified as: > > -object pmu='{"qom-type":"kvm-pmu-filter","id":"f0","action":"allow","events":[{"format":"raw","code":"0x5"},{"format":"raw","select":"0x6"},{"format":"raw","code":"0x7"},{"format":"raw","code":"0x8"},{"format":"raw","code":"0x9"},{"format":"raw","code":"0x10"}]}' > Yes you are right. On Arm when you first set the PMU Filter, if the first filter is allow, then all other event will be denied by default. The reverse is also the same, if the first filter is deny, then all other event will be allowed by default. On ARM the PMU Filter is much more simper than x86 I think. We only need to care about the special event with allow or deny action. If we don't support event range filter, I think that's fine. This can be added in the future. Thanks, Shaoqin > Thanks, > Zhao >
On Fri, Aug 02, 2024 at 05:41:57PM +0800, Shaoqin Huang wrote: > Date: Fri, 2 Aug 2024 17:41:57 +0800 > From: Shaoqin Huang <shahuang@redhat.com> > Subject: Re: [RFC 0/5] accel/kvm: Support KVM PMU filter > > Hi Zhao, > > On 8/2/24 17:37, Zhao Liu wrote: > > Hello Shaoqin, > > > > On Fri, Aug 02, 2024 at 05:01:47PM +0800, Shaoqin Huang wrote: > > > Date: Fri, 2 Aug 2024 17:01:47 +0800 > > > From: Shaoqin Huang <shahuang@redhat.com> > > > Subject: Re: [RFC 0/5] accel/kvm: Support KVM PMU filter > > > > > > Hi Zhao, > > > > > > On 7/10/24 12:51, Zhao Liu wrote: > > > > Hi QEMU maintainers, arm and PMU folks, > > > > > > > > I picked up Shaoqing's previous work [1] on the KVM PMU filter for arm, > > > > and now is trying to support this feature for x86 with a JSON-compatible > > > > API. > > > > > > > > While arm and x86 use different KVM ioctls to configure the PMU filter, > > > > considering they all have similar inputs (PMU event + action), it is > > > > still possible to abstract a generic, cross-architecture kvm-pmu-filter > > > > object and provide users with a sufficiently generic or near-consistent > > > > QAPI interface. > > > > > > > > That's what I did in this series, a new kvm-pmu-filter object, with the > > > > API like: > > > > > > > > -object '{"qom-type":"kvm-pmu-filter","id":"f0","events":[{"action":"allow","format":"raw","code":"0xc4"}]}' > > > > > > > > For i386, this object is inserted into kvm accelerator and is extended > > > > to support fixed-counter and more formats ("x86-default" and > > > > "x86-masked-entry"): > > > > > > > > -accel kvm,pmu-filter=f0 \ > > > > -object pmu='{"qom-type":"kvm-pmu-filter","id":"f0","x86-fixed-counter":{"action":"allow","bitmap":"0x0"},"events":[{"action":"allow","format":"x86-masked-entry","select":"0xc4","mask":"0xff","match":"0","exclude":true},{"action":"allow","format":"x86-masked-entry","select":"0xc5","mask":"0xff","match":"0","exclude":true}]}' > > > > > > What if I want to create the PMU Filter on ARM to deny the event range > > > [0x5,0x10], and allow deny event 0x13, how should I write the json? > > > > > > > Cuurently this doesn't support the event range (since the raw format of > > x86 event cannot be said to be continuous). > > > > So with the basic support, we need to configure events one by one: > > > > -object pmu='{"qom-type":"kvm-pmu-filter","id":"f0","events":[{"action":"allow","format":"raw","code":"0x5"},{"action":"allow","format":"raw","select":"0x6"},{"action":"allow","format":"raw","code":"0x7"},{"action":"allow","format":"raw","code":"0x8"},{"action":"allow","format":"raw","code":"0x9"},{"action":"allow","format":"raw","code":"0x10"},{"action":"deny","format":"raw","code":"0x13"}]}' > > > > This one looks a lot more complicated, but in the future, arm could > > further support event-range (maybe implement event-range via mask), but > > I think this could be arch-specific format since not all architectures' > > events are continuous. > > > > Additional, I'm a bit confused by your example, and I hope you can help > > me understand that: when configuring 0x5~0x10 to be allow, isn't it true > > that all other events are denied by default, so denying 0x13 again is a > > redundant operation? What is the default action for all other events > > except 0x5~0x10 and 0x13? > > > > If we specify action as allow for 0x5~0x10 and deny for the rest by > > default, then there is no need to set an action for each event but only > > a global one (as suggested by Dapeng), so the above command line can be > > simplified as: > > > > -object pmu='{"qom-type":"kvm-pmu-filter","id":"f0","action":"allow","events":[{"format":"raw","code":"0x5"},{"format":"raw","select":"0x6"},{"format":"raw","code":"0x7"},{"format":"raw","code":"0x8"},{"format":"raw","code":"0x9"},{"format":"raw","code":"0x10"}]}' > > > > Yes you are right. On Arm when you first set the PMU Filter, if the first > filter is allow, then all other event will be denied by default. The reverse > is also the same, if the first filter is deny, then all other event will be > allowed by default. > > On ARM the PMU Filter is much more simper than x86 I think. We only need to > care about the special event with allow or deny action. > > If we don't support event range filter, I think that's fine. This can be > added in the future. This is good news for me, I can implement global action in the next version and iterate further! Thank you for your confirmation! Regards, Zhao
Hi QEMU maintainers, arm and PMU folks, I picked up Shaoqing's previous work [1] on the KVM PMU filter for arm, and now is trying to support this feature for x86 with a JSON-compatible API. While arm and x86 use different KVM ioctls to configure the PMU filter, considering they all have similar inputs (PMU event + action), it is still possible to abstract a generic, cross-architecture kvm-pmu-filter object and provide users with a sufficiently generic or near-consistent QAPI interface. That's what I did in this series, a new kvm-pmu-filter object, with the API like: -object '{"qom-type":"kvm-pmu-filter","id":"f0","events":[{"action":"allow","format":"raw","code":"0xc4"}]}' For i386, this object is inserted into kvm accelerator and is extended to support fixed-counter and more formats ("x86-default" and "x86-masked-entry"): -accel kvm,pmu-filter=f0 \ -object pmu='{"qom-type":"kvm-pmu-filter","id":"f0","x86-fixed-counter":{"action":"allow","bitmap":"0x0"},"events":[{"action":"allow","format":"x86-masked-entry","select":"0xc4","mask":"0xff","match":"0","exclude":true},{"action":"allow","format":"x86-masked-entry","select":"0xc5","mask":"0xff","match":"0","exclude":true}]}' This object can still be added as the property to the arch CPU if it is desired as a per CPU feature (as Shaoqin did for arm before). Welcome your feedback and comments! Introduction ============ Formats supported in kvm-pmu-filter ----------------------------------- This series supports 3 formats: * raw format (general format). This format indicates the code that has been encoded to be able to index the PMU events, and which can be delivered directly to the KVM ioctl. For arm, this means the event code, and for i386, this means the raw event with the layout like: select high bit | umask | select low bits * x86-default format (i386 specific) x86 commonly uses select&umask to identify PMU events, and this format is used to support the select&umask. Then QEMU will encode select and umask into a raw format code. * x86-masked-entry (i386 specific) This is a special format that x86's KVM_SET_PMU_EVENT_FILTER supports. Hexadecimal value string ------------------------ In practice, the values associated with PMU events (code for arm, select& umask for x86) are often expressed in hexadecimal. Further, from linux perf related information (tools/perf/pmu-events/arch/*/*/*.json), x86/ arm64/riscv/nds32/powerpc all prefer the hexadecimal numbers and only s390 uses decimal value. Therefore, it is necessary to support hexadecimal in order to honor PMU conventions. However, unfortunately, standard JSON (RFC 8259) does not support hexadecimal numbers. So I can only consider using the numeric string in the QAPI and then parsing it to a number. To achieve this, I defined two versions of PMU-related structures in kvm.json: * a native version that accepts numeric values, which is used for QEMU's internal code processing, * and a variant version that accepts numeric string, which is used to receive user input. kvm-pmu-filter object will take care of converting the string version of the event/counter information into the numeric version. The related implementation can be found in patch 1. CPU property v.s. KVM property ------------------------------ In Shaoqin's previous implementation [1], KVM PMU filter is made as a arm CPU property. This is because arm uses a per CPU ioctl (KVM_SET_DEVICE_ATTR) to configure KVM PMU filter. However, for x86, the dependent ioctl (KVM_SET_PMU_EVENT_FILTER) is per VM. In the meantime, considering that for hybrid architecture, maybe in the future there will be a new per vCPU ioctl, or there will be practices to support filter fixed counter by configuring CPUIDs. Based on the above thoughts, for x86, it is not appropriate to make the current per-VM ioctl-based PMU filter a CPU property. Instead, I make it a kvm property and configure it via "-accel kvm,pmu-filter=obj_id". So in summary, it is feasible to use the KVM PMU filter as either a CPU or a KVM property, depending on whether it is used as a CPU feature or a VM feature. The kvm-pmu-filter object, as an abstraction, is general enough to support filter configurations for different scopes (per-CPU or per-VM). [1]: https://lore.kernel.org/qemu-devel/20240409024940.180107-1-shahuang@redhat.com/ Thanks and Best Regards, Zhao --- Zhao Liu (5): qapi/qom: Introduce kvm-pmu-filter object i386/kvm: Support initial KVM PMU filter i386/kvm: Support event with select&umask format in KVM PMU filter i386/kvm: Support event with masked entry format in KVM PMU filter i386/kvm: Support fixed counter in KVM PMU filter MAINTAINERS | 1 + accel/kvm/kvm-pmu.c | 367 +++++++++++++++++++++++++++++++++++++ accel/kvm/meson.build | 1 + include/sysemu/kvm-pmu.h | 43 +++++ include/sysemu/kvm_int.h | 2 + qapi/kvm.json | 255 ++++++++++++++++++++++++++ qapi/meson.build | 1 + qapi/qapi-schema.json | 1 + qapi/qom.json | 3 + target/i386/kvm/kvm.c | 211 +++++++++++++++++++++ target/i386/kvm/kvm_i386.h | 1 + 11 files changed, 886 insertions(+) create mode 100644 accel/kvm/kvm-pmu.c create mode 100644 include/sysemu/kvm-pmu.h create mode 100644 qapi/kvm.json