| Message ID | 20210512014759.55556-3-krish.sadhukhan@oracle.com (mailing list archive) |
|---|---|
| State | New, archived |
| Headers | show |
| Series | KVM: nVMX: Add more statistics to KVM debugfs | expand |
On Tue, May 11, 2021 at 7:37 PM Krish Sadhukhan <krish.sadhukhan@oracle.com> wrote: > > Add the following per-VCPU statistic to KVM debugfs to show if a given > VCPU is running a nested guest: > > nested_guest_running > > Also add this as a per-VM statistic to KVM debugfs to show the total number > of VCPUs running a nested guest in a given VM. > > Signed-off-by: Krish Sadhukhan <Krish.Sadhukhan@oracle.com> This is fine, but I don't really see its usefulness. OTOH, one statistic I would really like to see is how many vCPUs have *ever* run a nested guest.
On 5/12/21 9:01 AM, Jim Mattson wrote: > On Tue, May 11, 2021 at 7:37 PM Krish Sadhukhan > <krish.sadhukhan@oracle.com> wrote: >> Add the following per-VCPU statistic to KVM debugfs to show if a given >> VCPU is running a nested guest: >> >> nested_guest_running >> >> Also add this as a per-VM statistic to KVM debugfs to show the total number >> of VCPUs running a nested guest in a given VM. >> >> Signed-off-by: Krish Sadhukhan <Krish.Sadhukhan@oracle.com> > This is fine, but I don't really see its usefulness. OTOH, one Two potential uses: 1. If Live Migration of L2 guests is broken/buggy, this can be used to determine a safer time to trigger Live Migration of L1 guests. 2. This can be used to create a time-graph of the load of L1 and L2 in a given VM as well across the host. > statistic I would really like to see is how many vCPUs have *ever* run > a nested guest. 'nested_runs' statistic provides this data, though only till VCPUs are alive. We can convert 'nested_runs' to be persistent beyond VCPU destruction. But I am curious about the usage you are thinking of with this.
On Wed, May 12, 2021 at 10:56 AM Krish Sadhukhan
<krish.sadhukhan@oracle.com> wrote:
> But I am curious about the usage you are thinking of with this.
I'm interested in tracking usage of nested virtualization.
On Wed, May 12, 2021, Krish Sadhukhan wrote: > > On 5/12/21 9:01 AM, Jim Mattson wrote: > > On Tue, May 11, 2021 at 7:37 PM Krish Sadhukhan > > <krish.sadhukhan@oracle.com> wrote: > > > Add the following per-VCPU statistic to KVM debugfs to show if a given > > > VCPU is running a nested guest: > > > > > > nested_guest_running > > > > > > Also add this as a per-VM statistic to KVM debugfs to show the total number > > > of VCPUs running a nested guest in a given VM. > > > > > > Signed-off-by: Krish Sadhukhan <Krish.Sadhukhan@oracle.com> > > This is fine, but I don't really see its usefulness. OTOH, one > > Two potential uses: > > 1. If Live Migration of L2 guests is broken/buggy, this can be used to > determine a safer time to trigger Live Migration of L1 guests. This seems tenuous. The stats are inherently racy, so userspace would still need to check for "guest mode" after retrieving state. And wouldn't you want to wait until L1 turns VMX/SVM _off_? If migrating L2 is broken, simply waiting until L2 exits likely isn't going to help all that much. > 2. This can be used to create a time-graph of the load of L1 and L2 in a > given VM as well across the host. Hrm, I like the idea of being able to observe how much time a vCPU is spending in L1 vs. L2, but cross-referencing guest time with "in L2" seems difficult and error prone. I wonder if we can do better, i.e. explicitly track L1 vs. L2+ usage. I think that would also grant Jim's wish of being able to more precisely track nested virtualization utilization. > > statistic I would really like to see is how many vCPUs have *ever* run > > a nested guest.
diff --git a/arch/x86/include/asm/kvm_host.h b/arch/x86/include/asm/kvm_host.h index cf8557b2b90f..884f6e5ba669 100644 --- a/arch/x86/include/asm/kvm_host.h +++ b/arch/x86/include/asm/kvm_host.h @@ -1173,6 +1173,7 @@ struct kvm_vcpu_stat { u64 nested_runs; u64 directed_yield_attempted; u64 directed_yield_successful; + u64 nested_guest_running; }; struct x86_instruction_info; diff --git a/arch/x86/kvm/debugfs.c b/arch/x86/kvm/debugfs.c index 7e818d64bb4d..465d243afaac 100644 --- a/arch/x86/kvm/debugfs.c +++ b/arch/x86/kvm/debugfs.c @@ -17,6 +17,15 @@ static int vcpu_get_timer_advance_ns(void *data, u64 *val) DEFINE_SIMPLE_ATTRIBUTE(vcpu_timer_advance_ns_fops, vcpu_get_timer_advance_ns, NULL, "%llu\n"); +static int vcpu_get_guest_mode(void *data, u64 *val) +{ + struct kvm_vcpu *vcpu = (struct kvm_vcpu *) data; + *val = vcpu->stat.nested_guest_running; + return 0; +} + +DEFINE_SIMPLE_ATTRIBUTE(vcpu_guest_mode_fops, vcpu_get_guest_mode, NULL, "%lld\n"); + static int vcpu_get_tsc_offset(void *data, u64 *val) { struct kvm_vcpu *vcpu = (struct kvm_vcpu *) data; @@ -45,6 +54,8 @@ DEFINE_SIMPLE_ATTRIBUTE(vcpu_tsc_scaling_frac_fops, vcpu_get_tsc_scaling_frac_bi void kvm_arch_create_vcpu_debugfs(struct kvm_vcpu *vcpu, struct dentry *debugfs_dentry) { + debugfs_create_file("nested_guest_running", 0444, debugfs_dentry, vcpu, + &vcpu_guest_mode_fops); debugfs_create_file("tsc-offset", 0444, debugfs_dentry, vcpu, &vcpu_tsc_offset_fops); diff --git a/arch/x86/kvm/kvm_cache_regs.h b/arch/x86/kvm/kvm_cache_regs.h index cf52cbff18d3..348ef98c6199 100644 --- a/arch/x86/kvm/kvm_cache_regs.h +++ b/arch/x86/kvm/kvm_cache_regs.h @@ -163,6 +163,7 @@ static inline void enter_guest_mode(struct kvm_vcpu *vcpu) { vcpu->arch.hflags |= HF_GUEST_MASK; ++vcpu->stat.nested_runs; + vcpu->stat.nested_guest_running = 1; } static inline void leave_guest_mode(struct kvm_vcpu *vcpu) @@ -173,6 +174,8 @@ static inline void leave_guest_mode(struct kvm_vcpu *vcpu) vcpu->arch.load_eoi_exitmap_pending = false; kvm_make_request(KVM_REQ_LOAD_EOI_EXITMAP, vcpu); } + + vcpu->stat.nested_guest_running = 0; } static inline bool is_guest_mode(struct kvm_vcpu *vcpu) diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c index 6d1f51f6c344..01805b68dc9b 100644 --- a/arch/x86/kvm/x86.c +++ b/arch/x86/kvm/x86.c @@ -246,6 +246,7 @@ struct kvm_stats_debugfs_item debugfs_entries[] = { VCPU_STAT("nested_runs", nested_runs), VCPU_STAT("directed_yield_attempted", directed_yield_attempted), VCPU_STAT("directed_yield_successful", directed_yield_successful), + VCPU_STAT("nested_guest_running", nested_guest_running), VM_STAT("mmu_shadow_zapped", mmu_shadow_zapped), VM_STAT("mmu_pte_write", mmu_pte_write), VM_STAT("mmu_pde_zapped", mmu_pde_zapped),
Add the following per-VCPU statistic to KVM debugfs to show if a given VCPU is running a nested guest: nested_guest_running Also add this as a per-VM statistic to KVM debugfs to show the total number of VCPUs running a nested guest in a given VM. Signed-off-by: Krish Sadhukhan <Krish.Sadhukhan@oracle.com> --- arch/x86/include/asm/kvm_host.h | 1 + arch/x86/kvm/debugfs.c | 11 +++++++++++ arch/x86/kvm/kvm_cache_regs.h | 3 +++ arch/x86/kvm/x86.c | 1 + 4 files changed, 16 insertions(+)