[0/7] KVM: x86/mmu: Optimize TDP MMU huge page recovery during disable-dirty-log

Message

David Matlack Aug. 5, 2024, 11:31 p.m. UTC

Rework the TDP MMU disable-dirty-log path to batch TLB flushes and
recover huge page mappings, rather than zapping and flushing for every
potential huge page mapping.

With this series, dirty_log_perf_test shows a decrease in the time it
takes to disable dirty logging, as well as a decrease in the number of
vCPU faults:

 $ ./dirty_log_perf_test -s anonymous_hugetlb_2mb -v 64 -e -b 4g

 Before: Disabling dirty logging time: 14.334453428s (131072 flushes)
 After:  Disabling dirty logging time: 4.794969689s  (76 flushes)

 Before: 393,599      kvm:kvm_page_fault
 After:  262,575      kvm:kvm_page_fault

David Matlack (7):
  Revert "KVM: x86/mmu: Don't bottom out on leafs when zapping
    collapsible SPTEs"
  KVM: x86/mmu: Drop @max_level from kvm_mmu_max_mapping_level()
  KVM: x86/mmu: Batch TLB flushes when zapping collapsible TDP MMU SPTEs
  KVM: x86/mmu: Recover TDP MMU huge page mappings in-place instead of
    zapping
  KVM: x86/mmu: Rename make_huge_page_split_spte() to make_small_spte()
  KVM: x86/mmu: WARN if huge page recovery triggered during dirty
    logging
  KVM: x86/mmu: Recheck SPTE points to a PT during huge page recovery

 arch/x86/include/asm/kvm_host.h |   4 +-
 arch/x86/kvm/mmu/mmu.c          |  16 ++--
 arch/x86/kvm/mmu/mmu_internal.h |   3 +-
 arch/x86/kvm/mmu/spte.c         |  40 ++++++++--
 arch/x86/kvm/mmu/spte.h         |   5 +-
 arch/x86/kvm/mmu/tdp_iter.c     |   9 +++
 arch/x86/kvm/mmu/tdp_iter.h     |   1 +
 arch/x86/kvm/mmu/tdp_mmu.c      | 127 ++++++++++++++------------------
 arch/x86/kvm/mmu/tdp_mmu.h      |   4 +-
 arch/x86/kvm/x86.c              |  18 ++---
 10 files changed, 121 insertions(+), 106 deletions(-)


base-commit: 332d2c1d713e232e163386c35a3ba0c1b90df83f