| Message ID | 20230714064656.20147-1-yan.y.zhao@intel.com (mailing list archive) |
|---|---|
| Headers | show |
| Series | KVM: x86/mmu: refine memtype related mmu zap | expand |
On Fri, Jul 14, 2023, Yan Zhao wrote: > This series refines mmu zap caused by EPT memory type update when guest > MTRRs are honored. > > Patches 1-5 revolve around utilizing helper functions to check if > KVM TDP honors guest MTRRs, TDP zaps and page fault max_level reduction > are now only targeted to TDPs that honor guest MTRRs. > > -The 5th patch will trigger zapping of TDP leaf entries if non-coherent > DMA devices count goes from 0 to 1 or from 1 to 0. > > Patches 6-7 are fixes and patches 9-12 are optimizations for mmu zaps > when guest MTRRs are honored. > Those mmu zaps are intended to remove stale memtypes of TDP entries > caused by changes of guest MTRRs and CR0.CD and are usually triggered from > all vCPUs in bursts. Sorry for the delayed review, especially with respect to patches 1-5. I completely forgot there were cleanups at the beginning of this series. I'll make to grab 1-5 early in the 6.7 cycle, even if you haven't sent a new version before then.
On Fri, Aug 25, 2023 at 04:17:09PM -0700, Sean Christopherson wrote: > On Fri, Jul 14, 2023, Yan Zhao wrote: > > This series refines mmu zap caused by EPT memory type update when guest > > MTRRs are honored. > > > > Patches 1-5 revolve around utilizing helper functions to check if > > KVM TDP honors guest MTRRs, TDP zaps and page fault max_level reduction > > are now only targeted to TDPs that honor guest MTRRs. > > > > -The 5th patch will trigger zapping of TDP leaf entries if non-coherent > > DMA devices count goes from 0 to 1 or from 1 to 0. > > > > Patches 6-7 are fixes and patches 9-12 are optimizations for mmu zaps > > when guest MTRRs are honored. > > Those mmu zaps are intended to remove stale memtypes of TDP entries > > caused by changes of guest MTRRs and CR0.CD and are usually triggered from > > all vCPUs in bursts. > > Sorry for the delayed review, especially with respect to patches 1-5. I completely > forgot there were cleanups at the beginning of this series. I'll make to grab > 1-5 early in the 6.7 cycle, even if you haven't sent a new version before then. Never mind and thanks a lot regarding to patches 1-5! I may not be able to spin the next version soon as I got a high priority task and I need to finish the task first (I wish I can complete it in 1-1.5 months) Sorry and thanks again!
On Fri, 14 Jul 2023 14:46:56 +0800, Yan Zhao wrote: > This series refines mmu zap caused by EPT memory type update when guest > MTRRs are honored. > > Patches 1-5 revolve around utilizing helper functions to check if > KVM TDP honors guest MTRRs, TDP zaps and page fault max_level reduction > are now only targeted to TDPs that honor guest MTRRs. > > [...] Applied 1-5 and 7 to kvm-x86 mmu. I squashed 1 and 2 as introducing helpers to consolidate existing code without converting the existing code is wierd and makes it unnecessarily impossible to properly test the helpers when they're added. I skipped 6, "move TDP zaps from guest MTRRs update to CR0.CD toggling", for now as your performance numbers showed that it slowed down the guest even though the number of zaps went down. I'm definitely not against the patch, I just don't want to risk regressing guest performance, i.e. I don't wantt to take it without the rest of the series that takes advantage of the change. I massaged a few shortlogs and changelogs, but didn't touch any code. Holler if anything looks funky. Thanks much! [1/5] KVM: x86/mmu: Add helpers to return if KVM honors guest MTRRs https://github.com/kvm-x86/linux/commit/6590a37e7ec6 [2/5] KVM: x86/mmu: Zap SPTEs when CR0.CD is toggled iff guest MTRRs are honored https://github.com/kvm-x86/linux/commit/c0ad4a14c5af [3/5] KVM: x86/mmu: Zap SPTEs on MTRR update iff guest MTRRs are honored https://github.com/kvm-x86/linux/commit/a1596812cce1 [4/5] KVM: x86/mmu: Xap KVM TDP when noncoherent DMA assignment starts/stops https://github.com/kvm-x86/linux/commit/3c4955c04b95 [5/5] KVM: VMX: drop IPAT in memtype when CD=1 for KVM_X86_QUIRK_CD_NW_CLEARED https://github.com/kvm-x86/linux/commit/f7b4bcd501ef -- https://github.com/kvm-x86/linux/tree/next
On Wed, 2023-10-04 at 18:29 -0700, Sean Christopherson wrote: > [4/5] KVM: x86/mmu: Xap KVM TDP when noncoherent DMA assignment starts/stops > https://github.com/kvm-x86/linux/commit/3c4955c04b95 Xap -> Zap? :-) Apologize if I missed something.
On Thu, Oct 05, 2023, Kai Huang wrote: > On Wed, 2023-10-04 at 18:29 -0700, Sean Christopherson wrote: > > [4/5] KVM: x86/mmu: Xap KVM TDP when noncoherent DMA assignment starts/stops > > https://github.com/kvm-x86/linux/commit/3c4955c04b95 > > Xap -> Zap? :-) Dagnabbit, I tried to capitalize z => Z and hit the wrong key. I'll fixup. Thanks Kai!
On Wed, Oct 04, 2023, Sean Christopherson wrote: > On Thu, Oct 05, 2023, Kai Huang wrote: > > On Wed, 2023-10-04 at 18:29 -0700, Sean Christopherson wrote: > > > [4/5] KVM: x86/mmu: Xap KVM TDP when noncoherent DMA assignment starts/stops > > > https://github.com/kvm-x86/linux/commit/3c4955c04b95 > > > > Xap -> Zap? :-) > > Dagnabbit, I tried to capitalize z => Z and hit the wrong key. I'll fixup. LOL, the real irony is that this particular patch also has this in the changelog: [sean: fix misspelled words in comment and changelog] Anyways, fixed. New hashes are: [4/5] KVM: x86/mmu: Zap KVM TDP when noncoherent DMA assignment starts/stops https://github.com/kvm-x86/linux/commit/539c103e2a13 [5/5] KVM: VMX: drop IPAT in memtype when CD=1 for KVM_X86_QUIRK_CD_NW_CLEARED https://github.com/kvm-x86/linux/commit/10ed442fefdd
This series refines mmu zap caused by EPT memory type update when guest MTRRs are honored. Patches 1-5 revolve around utilizing helper functions to check if KVM TDP honors guest MTRRs, TDP zaps and page fault max_level reduction are now only targeted to TDPs that honor guest MTRRs. -The 5th patch will trigger zapping of TDP leaf entries if non-coherent DMA devices count goes from 0 to 1 or from 1 to 0. Patches 6-7 are fixes and patches 9-12 are optimizations for mmu zaps when guest MTRRs are honored. Those mmu zaps are intended to remove stale memtypes of TDP entries caused by changes of guest MTRRs and CR0.CD and are usually triggered from all vCPUs in bursts. - The 6th patch places TDP zap to when CR0.CD toggles and when guest MTRRs update under CR0.CD=0. - The 7th-8th patches refine KVM_X86_QUIRK_CD_NW_CLEARED by removing the IPAT bit in EPT memtype when CR0.CD=1 and guest MTRRs are honored. - The 9th-11th patches are optimizations of the mmu zap when guest MTRRs are honored by serializing vCPUs' gfn zap requests and calculating of precise fine-grained ranges to zap. They are put in mtrr.c because the optimizations are related to when guest MTRRs are honored and because it requires to read guest MTRRs for fine-grained ranges. Calls to kvm_unmap_gfn_range() are not included into the optimization, because they are not triggered from all vCPUs in bursts and not all of them are blockable. They usually happen at memslot removal and thus do not affect the mmu zaps when guest MTRRs are honored. Also, current performance data shows that there's no observable performance difference to mmu zaps by turning on/off auto numa balancing triggered kvm_unmap_gfn_range(). - The 12th patch further convert kvm_zap_gfn_range() to use shared mmu_lock in TDP MMU. It can visibly help to reduce cost in contentions along with vCPUs number increases. A reference performance data for last 7 patches as below: Base1: base code before patch 6 Base2: Base 1 + patches 6 + 7 + 8 patch 6: move TDP zaps from guest MTRRs update to CR0.CD toggling patch 7: drop IPAT in memtype when CD=1 for KVM_X86_QUIRK_CD_NW_CLEARED patch 8: entralize code to get CD=1 memtype when guest MTRRs are honored patch 9: serialize gfn zap patch 10: fine-grained gfn zap patch 11: split and zap in-slot gfn ranges only ** patch 12: convert gfn zap to use shared mmu_lock