| Message ID | 20210305011101.3597423-1-seanjc@google.com (mailing list archive) |
|---|---|
| Headers | show |
| Series | KVM: x86/mmu: Lots of bug fixes | expand |
On 05/03/21 02:10, Sean Christopherson wrote: > Fix nested NPT (nSVM) with 32-bit L1 and SME with shadow paging, which > are completely broken. Opportunistically fix theoretical bugs related to > prematurely reloading/unloading the MMU. > > If nNPT is enabled, L1 can crash the host simply by using 32-bit NPT to > trigger a null pointer dereference on pae_root. > > SME with shadow paging (including nNPT) fails to set the C-bit in the > shadow pages that don't go through standard MMU flows (PDPTPRs and the > PML4 used by nNPT to shadow legacy NPT). It also failes to account for > CR3[63:32], and thus the C-bit, being ignored outside of 64-bit mode. > > Patches 01 and 02 fix the null pointer bugs. > > Patches 03-09 fix mostly-benign related memory leaks. > > Patches 10-12 fix the SME shadow paging bugs, which are also what led me to > the nNPT null pointer bugs. > > Patches 13 and 14 fix theoretical bugs with PTP_SWITCH and INVPCID that > I found when auditing flows that touch the MMU context. > > Patches 14-17 do additional clean up to hopefully make it harder to > introduce bugs in the future. > > On the plus side, I finally understand why KVM supports shadowing 2-level > page tables with 4-level page tables... > > Based on kvm/queue, commit fe5f0041c026 ("KVM/SVM: Move vmenter.S exception > fixups out of line"). The null pointer fixes cherry-pick cleanly onto > kvm/master, haven't tried the other bug fixes (I doubt they're worth > backporting even though I tagged 'em with stable). > > v2: > - Collect a review from Ben (did not include his review of patch 03 > since the patch and its direct dependencies were changed). > - Move pae_root and lm_root allocation to a separate helper to avoid > sleeping via get_zeroed_page() while holding mmu_lock. > - Add a patch to grab 'mmu' in a local variable. > - Remove the BUILD_BUG_ON() in make_mmu_pages_available() since the > final check wouldn't actually guarnatee 4 pages were "available". > Instead, add a comment about the limit being soft. > > v1: > - https://lkml.kernel.org/r/20210302184540.2829328-1-seanjc@google.com > > Sean Christopherson (17): > KVM: nSVM: Set the shadow root level to the TDP level for nested NPT > KVM: x86/mmu: Alloc page for PDPTEs when shadowing 32-bit NPT with > 64-bit > KVM: x86/mmu: Capture 'mmu' in a local variable when allocating roots > KVM: x86/mmu: Allocate the lm_root before allocating PAE roots > KVM: x86/mmu: Allocate pae_root and lm_root pages in dedicated helper > KVM: x86/mmu: Ensure MMU pages are available when allocating roots > KVM: x86/mmu: Check PDPTRs before allocating PAE roots > KVM: x86/mmu: Fix and unconditionally enable WARNs to detect PAE leaks > KVM: x86/mmu: Use '0' as the one and only value for an invalid PAE > root > KVM: x86/mmu: Set the C-bit in the PDPTRs and LM pseudo-PDPTRs > KVM: x86/mmu: Mark the PAE roots as decrypted for shadow paging > KVM: SVM: Don't strip the C-bit from CR2 on #PF interception > KVM: nVMX: Defer the MMU reload to the normal path on an EPTP switch > KVM: x86: Defer the MMU unload to the normal path on an global INVPCID > KVM: x86/mmu: Unexport MMU load/unload functions > KVM: x86/mmu: Sync roots after MMU load iff load as successful > KVM: x86/mmu: WARN on NULL pae_root or lm_root, or bad shadow root > level > > arch/x86/include/asm/kvm_host.h | 3 - > arch/x86/kvm/mmu.h | 4 + > arch/x86/kvm/mmu/mmu.c | 273 ++++++++++++++++++++------------ > arch/x86/kvm/mmu/tdp_mmu.c | 23 +-- > arch/x86/kvm/svm/svm.c | 9 +- > arch/x86/kvm/vmx/nested.c | 9 +- > arch/x86/kvm/x86.c | 2 +- > 7 files changed, 192 insertions(+), 131 deletions(-) > Queued all except 9 and 11, thanks. Paolo
Fix nested NPT (nSVM) with 32-bit L1 and SME with shadow paging, which are completely broken. Opportunistically fix theoretical bugs related to prematurely reloading/unloading the MMU. If nNPT is enabled, L1 can crash the host simply by using 32-bit NPT to trigger a null pointer dereference on pae_root. SME with shadow paging (including nNPT) fails to set the C-bit in the shadow pages that don't go through standard MMU flows (PDPTPRs and the PML4 used by nNPT to shadow legacy NPT). It also failes to account for CR3[63:32], and thus the C-bit, being ignored outside of 64-bit mode. Patches 01 and 02 fix the null pointer bugs. Patches 03-09 fix mostly-benign related memory leaks. Patches 10-12 fix the SME shadow paging bugs, which are also what led me to the nNPT null pointer bugs. Patches 13 and 14 fix theoretical bugs with PTP_SWITCH and INVPCID that I found when auditing flows that touch the MMU context. Patches 14-17 do additional clean up to hopefully make it harder to introduce bugs in the future. On the plus side, I finally understand why KVM supports shadowing 2-level page tables with 4-level page tables... Based on kvm/queue, commit fe5f0041c026 ("KVM/SVM: Move vmenter.S exception fixups out of line"). The null pointer fixes cherry-pick cleanly onto kvm/master, haven't tried the other bug fixes (I doubt they're worth backporting even though I tagged 'em with stable). v2: - Collect a review from Ben (did not include his review of patch 03 since the patch and its direct dependencies were changed). - Move pae_root and lm_root allocation to a separate helper to avoid sleeping via get_zeroed_page() while holding mmu_lock. - Add a patch to grab 'mmu' in a local variable. - Remove the BUILD_BUG_ON() in make_mmu_pages_available() since the final check wouldn't actually guarnatee 4 pages were "available". Instead, add a comment about the limit being soft. v1: - https://lkml.kernel.org/r/20210302184540.2829328-1-seanjc@google.com Sean Christopherson (17): KVM: nSVM: Set the shadow root level to the TDP level for nested NPT KVM: x86/mmu: Alloc page for PDPTEs when shadowing 32-bit NPT with 64-bit KVM: x86/mmu: Capture 'mmu' in a local variable when allocating roots KVM: x86/mmu: Allocate the lm_root before allocating PAE roots KVM: x86/mmu: Allocate pae_root and lm_root pages in dedicated helper KVM: x86/mmu: Ensure MMU pages are available when allocating roots KVM: x86/mmu: Check PDPTRs before allocating PAE roots KVM: x86/mmu: Fix and unconditionally enable WARNs to detect PAE leaks KVM: x86/mmu: Use '0' as the one and only value for an invalid PAE root KVM: x86/mmu: Set the C-bit in the PDPTRs and LM pseudo-PDPTRs KVM: x86/mmu: Mark the PAE roots as decrypted for shadow paging KVM: SVM: Don't strip the C-bit from CR2 on #PF interception KVM: nVMX: Defer the MMU reload to the normal path on an EPTP switch KVM: x86: Defer the MMU unload to the normal path on an global INVPCID KVM: x86/mmu: Unexport MMU load/unload functions KVM: x86/mmu: Sync roots after MMU load iff load as successful KVM: x86/mmu: WARN on NULL pae_root or lm_root, or bad shadow root level arch/x86/include/asm/kvm_host.h | 3 - arch/x86/kvm/mmu.h | 4 + arch/x86/kvm/mmu/mmu.c | 273 ++++++++++++++++++++------------ arch/x86/kvm/mmu/tdp_mmu.c | 23 +-- arch/x86/kvm/svm/svm.c | 9 +- arch/x86/kvm/vmx/nested.c | 9 +- arch/x86/kvm/x86.c | 2 +- 7 files changed, 192 insertions(+), 131 deletions(-)