From patchwork Thu Jan 13 23:30:17 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Matlack X-Patchwork-Id: 12713165 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 4CED2C4332F for ; Thu, 13 Jan 2022 23:30:28 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S235515AbiAMXa1 (ORCPT ); Thu, 13 Jan 2022 18:30:27 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:45830 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231823AbiAMXa0 (ORCPT ); Thu, 13 Jan 2022 18:30:26 -0500 Received: from mail-pj1-x1049.google.com (mail-pj1-x1049.google.com [IPv6:2607:f8b0:4864:20::1049]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id CAB6BC06161C for ; Thu, 13 Jan 2022 15:30:26 -0800 (PST) Received: by mail-pj1-x1049.google.com with SMTP id u13-20020a17090a450d00b001b1e6726fccso11524599pjg.0 for ; Thu, 13 Jan 2022 15:30:26 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20210112; h=date:in-reply-to:message-id:mime-version:references:subject:from:to :cc; bh=8Xf/e2mla+a2kQPDivDH7OGq00ewbcGCMFDMPzdpwvE=; b=IdvTdJ//RODLIZYbXCAcblDVDPO3Qrho5vqIdrnRfdxEhkSqgPN+fMau0HFBAy49gk Bs4Yngkk0ivwQ7BgEHMHEl1VOVBQGJorB5K00PGmeXVZNh6cQ2ipzXeS1grRMSGFncyu KhmJN5zNkYe42jy68MUxT9XI6uh2+FWRO4TkyU2g0XS791szPZv8TCFdahx7DpvZAdog 5truukzvpsmiUlR9N6AIGtoPiGHBCxA8ezKDWHBbNvTpWANhmkzAn28KRp/opSVeXE9g cUGogyN077teOVBnK5p06NRrOLBA6zz0Cf7BIwo2wdlp+J0iCwhNkCS+ZylZSmi8DpIk y/yw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=8Xf/e2mla+a2kQPDivDH7OGq00ewbcGCMFDMPzdpwvE=; b=OkSQw5Wu/AgrH1Nhk9sztvJyNuovYcndz8q3Uvh8nKltP0KCOhU/ms3k1Rsm6DyGbT JQhbBD844RlmtCbA+p+e9DeN7wBi+Z6rSB6y5sBWlDyH4jV6Pk0ntedszBB/YlA4oMAq w87bQ3m8BLuj9Mh/fO5opQQTvGoODVSGPTMKohBV2ulWB2OGu+f51T6ssH3KXpx6O375 IGomWt1U94Q5Vh5rmRUES5SUlAZXX7hcIz52mdH+TCEslYhquhwYQ2goOINgMArDVIrR QzDyEQkkl1hqJjy8jrv093I2CngMkTMNfBJOlCw6ZztBbXzUPraOlHFphWwm89rcbgkF Kksw== X-Gm-Message-State: AOAM531s0inm7Pz59HRZtXDvP8fxuxF3kZTSIftVctiweLXdXxVKtQix dWng03KeQKH93fdy6csJ7YumKGb68M8dZg== X-Google-Smtp-Source: ABdhPJzZN7Dkcmwgib6lrraxadhYhDPs7lxuG3B6gpkIj2hheVI1CS7IDjBgKzZz6GD2xb8Q9IsjsLAZhY6BEw== X-Received: from dmatlack-heavy.c.googlers.com ([fda3:e722:ac3:cc00:7f:e700:c0a8:19cd]) (user=dmatlack job=sendgmr) by 2002:a63:af07:: with SMTP id w7mr5876042pge.209.1642116626187; Thu, 13 Jan 2022 15:30:26 -0800 (PST) Date: Thu, 13 Jan 2022 23:30:17 +0000 In-Reply-To: <20220113233020.3986005-1-dmatlack@google.com> Message-Id: <20220113233020.3986005-2-dmatlack@google.com> Mime-Version: 1.0 References: <20220113233020.3986005-1-dmatlack@google.com> X-Mailer: git-send-email 2.34.1.703.g22d0c6ccf7-goog Subject: [PATCH v2 1/4] KVM: x86/mmu: Fix write-protection of PTs mapped by the TDP MMU From: David Matlack To: Paolo Bonzini Cc: Sean Christopherson , Vitaly Kuznetsov , Wanpeng Li , Jim Mattson , Joerg Roedel , Ben Gardon , kvm@vger.kernel.org, David Matlack , stable@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org When the TDP MMU is write-protection GFNs for page table protection (as opposed to for dirty logging, or due to the HVA not being writable), it checks if the SPTE is already write-protected and if so skips modifying the SPTE and the TLB flush. This behavior is incorrect because the SPTE may be write-protected for dirty logging. This implies that the SPTE could be locklessly be made writable on the next write access, and that vCPUs could still be running with writable SPTEs cached in their TLB. Fix this by only skipping setting the SPTE if the SPTE is already write-protected *and* MMU-writable is already clear. Fixes: 46044f72c382 ("kvm: x86/mmu: Support write protection for nesting in tdp MMU") Cc: stable@vger.kernel.org Signed-off-by: David Matlack Reviewed-by: Sean Christopherson --- arch/x86/kvm/mmu/tdp_mmu.c | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) base-commit: fea31d1690945e6dd6c3e89ec5591490857bc3d4 diff --git a/arch/x86/kvm/mmu/tdp_mmu.c b/arch/x86/kvm/mmu/tdp_mmu.c index 7b1bc816b7c3..bc9e3553fba2 100644 --- a/arch/x86/kvm/mmu/tdp_mmu.c +++ b/arch/x86/kvm/mmu/tdp_mmu.c @@ -1442,12 +1442,12 @@ static bool write_protect_gfn(struct kvm *kvm, struct kvm_mmu_page *root, !is_last_spte(iter.old_spte, iter.level)) continue; - if (!is_writable_pte(iter.old_spte)) - break; - new_spte = iter.old_spte & ~(PT_WRITABLE_MASK | shadow_mmu_writable_mask); + if (new_spte == iter.old_spte) + break; + tdp_mmu_set_spte(kvm, &iter, new_spte); spte_set = true; } From patchwork Thu Jan 13 23:30:18 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Matlack X-Patchwork-Id: 12713166 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id BAA58C433EF for ; Thu, 13 Jan 2022 23:30:29 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S238225AbiAMXa3 (ORCPT ); Thu, 13 Jan 2022 18:30:29 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:45834 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231823AbiAMXa2 (ORCPT ); Thu, 13 Jan 2022 18:30:28 -0500 Received: from mail-pj1-x1049.google.com (mail-pj1-x1049.google.com [IPv6:2607:f8b0:4864:20::1049]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 56C78C061574 for ; Thu, 13 Jan 2022 15:30:28 -0800 (PST) Received: by mail-pj1-x1049.google.com with SMTP id b11-20020a17090a6e0b00b001b34cd8941bso7373098pjk.1 for ; Thu, 13 Jan 2022 15:30:28 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20210112; h=date:in-reply-to:message-id:mime-version:references:subject:from:to :cc; bh=AODmhi+VFVOG+MvvbisKfeB6Iw9Su49eHa71dUbmlmA=; b=bxeFdK9WJLI66HSyP4lpHPAtIHAYec8lVTpn/S7/MaOF2WsxLgxtfnQbYV677PXhMu urawvuv3svgJdAzIyGU8nCDNfUXvjEGWrGGc3cJfvItDRfUCRaU8KlHB7qgz+gIQJC4f 4iiA4HecR57+G0i799Aci9KYlm60dqibXQ5mQq1ygAwRtF6UWaQzHrBt3/XGMBOjDLe6 acePpfhA7FGAZliB20S8B7d1IlaD8Tp908JE5LiPnkVW4zHo4kPP42kTCPjqi7KKSFWN xG6gSsvouTddqbjPZwDz3968UDCmK6mn7f/4yJhByqGlbDZq5XJ7AawvEU/vG68Hg1jw w96g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=AODmhi+VFVOG+MvvbisKfeB6Iw9Su49eHa71dUbmlmA=; b=4K+UEDePvHiB4p9UavN/wuClEteBz7lmDzIB+NuMWV+XlBIItCHQ8irM6iC6deHiJw LM/f/Lz8afPjYZqgt+ZdP6vIb2GTj9/wJmSkq/rBJpYpogf+1tSy6cdUdKnf9TrlMz+n qpwoSOY4L2zmqrZM0klSHXGmyxEFugJFhwXFX6jcpVz3in1rk6Ns84LCNUUppunKMcwA 2whNC1puzQrbWDNb+8LVdaHhxoo/5lmYFeD42XHiVaGT2iD9VViqeXiURv/y6cWjm8/D A4qH7RyJhavqAeCxHdInVdZNKOxr8OMyqQagnnc1neI5LInGAesEK3Zm2KKXokSM+xJe UxTw== X-Gm-Message-State: AOAM533pRX2lovvWKmSbS6URcH0Tg8tionZP5i/6Cvcajk5Z0Gd4aquR 5ErWhPsyB1IC/iA3PS6qn5QU5vYSe1orGw== X-Google-Smtp-Source: ABdhPJxriZlwjXtF3bvJFUTSejcCogC/4nB+r5mqRBy8EOIMxpDaUBXxu+Ojo7pkTbJUC3yDhA9I4vhV86og9w== X-Received: from dmatlack-heavy.c.googlers.com ([fda3:e722:ac3:cc00:7f:e700:c0a8:19cd]) (user=dmatlack job=sendgmr) by 2002:a17:90b:4012:: with SMTP id ie18mr7620806pjb.43.1642116627813; Thu, 13 Jan 2022 15:30:27 -0800 (PST) Date: Thu, 13 Jan 2022 23:30:18 +0000 In-Reply-To: <20220113233020.3986005-1-dmatlack@google.com> Message-Id: <20220113233020.3986005-3-dmatlack@google.com> Mime-Version: 1.0 References: <20220113233020.3986005-1-dmatlack@google.com> X-Mailer: git-send-email 2.34.1.703.g22d0c6ccf7-goog Subject: [PATCH v2 2/4] KVM: x86/mmu: Clear MMU-writable during changed_pte notifier From: David Matlack To: Paolo Bonzini Cc: Sean Christopherson , Vitaly Kuznetsov , Wanpeng Li , Jim Mattson , Joerg Roedel , Ben Gardon , kvm@vger.kernel.org, David Matlack Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org When handling the changed_pte notifier and the new PTE is read-only, clear both the Host-writable and MMU-writable bits in the SPTE. This preserves the invariant that MMU-writable is set if-and-only-if Host-writable is set. No functional change intended. Nothing currently relies on the afformentioned invariant and technically the changed_pte notifier is dead code. Signed-off-by: David Matlack Reviewed-by: Sean Christopherson --- arch/x86/kvm/mmu/spte.c | 1 + 1 file changed, 1 insertion(+) diff --git a/arch/x86/kvm/mmu/spte.c b/arch/x86/kvm/mmu/spte.c index 8a7b03207762..f8677404c93c 100644 --- a/arch/x86/kvm/mmu/spte.c +++ b/arch/x86/kvm/mmu/spte.c @@ -215,6 +215,7 @@ u64 kvm_mmu_changed_pte_notifier_make_spte(u64 old_spte, kvm_pfn_t new_pfn) new_spte &= ~PT_WRITABLE_MASK; new_spte &= ~shadow_host_writable_mask; + new_spte &= ~shadow_mmu_writable_mask; new_spte = mark_spte_for_access_track(new_spte); From patchwork Thu Jan 13 23:30:19 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Matlack X-Patchwork-Id: 12713167 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 3FB48C433FE for ; Thu, 13 Jan 2022 23:30:31 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S238228AbiAMXaa (ORCPT ); Thu, 13 Jan 2022 18:30:30 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:45844 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S238190AbiAMXa3 (ORCPT ); Thu, 13 Jan 2022 18:30:29 -0500 Received: from mail-pj1-x1049.google.com (mail-pj1-x1049.google.com [IPv6:2607:f8b0:4864:20::1049]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id ADC25C061574 for ; Thu, 13 Jan 2022 15:30:29 -0800 (PST) Received: by mail-pj1-x1049.google.com with SMTP id y15-20020a17090a600f00b001b3501d9e7eso11424629pji.8 for ; Thu, 13 Jan 2022 15:30:29 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20210112; h=date:in-reply-to:message-id:mime-version:references:subject:from:to :cc; bh=56rgy4oAyIu0qQhJLuWwTDz6sSfSd/R7QcLnRVQwjhM=; b=lkJLPJWKDbLx4RVpqictGfi0pZJRCPovTyBfIanW9ysxbQhfi7QtWhzoDFyIYhpp3n mgdpzUCqAb+1wSZlOK3JeWmyD9qpV9DK8iInCdNX4I4CLOPMTjqtXGlmo3g4uw/uTsod Yjc+RYEel6XGpfv2WGhiUXhtgeauFD67whbofrD2fWB2m6yx4h9J0EFP8LPCSQTLefbX KwNB2CiXBKYapGSZbKfK2Ly/ocE3rour4E4WMPXzoHqdB1qJTjwpf7UyiYKKQdvdr+12 ec2ht+uVyEqg1NIfTVNw6B5jyoUSISjm8daakGHbttdmsUp/+ToQD40a1gt4WLohocK5 P1VQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=56rgy4oAyIu0qQhJLuWwTDz6sSfSd/R7QcLnRVQwjhM=; b=MGp9MpCAfq/DkTZcULtAngF1BTts/0oxbYDB0tcfbCmCs5lSm/8vIsvs2OI0nJU7wp xKma7hfYWU2bfIa3jhKYGdgL/tJPQHVBq6G0nS1MdD1M25coIjYxpDouWJLt/b7wbcwt Bb6yHJOs6tWiO6B15ipW+Ocf8TIgkPBkr0vySWAU6FzSEtOJbDG5Fo8ryWj9ROZga8Q6 UyXOssRGz3tVACPTrmZqf3N6bdPg5C1vxmM+xhzbMnZ1+K9+Ax6s3cD1q8SlzzG+Ws4w EzhIcNqWgTK6F+0dD3n9dopZ7/prVo8CeSJahSVqrOdBDp33wY7nU8318i+jIXvYMwIY M08Q== X-Gm-Message-State: AOAM5316SVRZGf0fBoPWCFMP/oTOv6evAnpYneHsNt37sUCNiGpzsyqr RL3g4bU3ajg1kz/kcwu1Tx5gM1t1G3+Vyg== X-Google-Smtp-Source: ABdhPJyq9+2wV0pEU7h0g581rjRltoXxzUwvtS/oJRsCX2ioW+Ke5Gc9XzaKibIuRZo0bhovTS3LV7SuPiwLrg== X-Received: from dmatlack-heavy.c.googlers.com ([fda3:e722:ac3:cc00:7f:e700:c0a8:19cd]) (user=dmatlack job=sendgmr) by 2002:a63:694a:: with SMTP id e71mr5901681pgc.37.1642116629233; Thu, 13 Jan 2022 15:30:29 -0800 (PST) Date: Thu, 13 Jan 2022 23:30:19 +0000 In-Reply-To: <20220113233020.3986005-1-dmatlack@google.com> Message-Id: <20220113233020.3986005-4-dmatlack@google.com> Mime-Version: 1.0 References: <20220113233020.3986005-1-dmatlack@google.com> X-Mailer: git-send-email 2.34.1.703.g22d0c6ccf7-goog Subject: [PATCH v2 3/4] KVM: x86/mmu: Document and enforce MMU-writable and Host-writable invariants From: David Matlack To: Paolo Bonzini Cc: Sean Christopherson , Vitaly Kuznetsov , Wanpeng Li , Jim Mattson , Joerg Roedel , Ben Gardon , kvm@vger.kernel.org, David Matlack Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org SPTEs are tagged with software-only bits to indicate if it is "MMU-writable" and "Host-writable". These bits are used to determine why KVM has marked an SPTE as read-only. Document these bits and their invariants, and enforce the invariants with new WARNs in spte_can_locklessly_be_made_writable() to ensure they are not accidentally violated in the future. Opportunistically move DEFAULT_SPTE_{MMU,HOST}_WRITABLE next to EPT_SPTE_{MMU,HOST}_WRITABLE since the new documentation applies to both. No functional change intended. Signed-off-by: David Matlack --- arch/x86/kvm/mmu/spte.h | 42 +++++++++++++++++++++++++++++++++++------ 1 file changed, 36 insertions(+), 6 deletions(-) diff --git a/arch/x86/kvm/mmu/spte.h b/arch/x86/kvm/mmu/spte.h index a4af2a42695c..be6a007a4af3 100644 --- a/arch/x86/kvm/mmu/spte.h +++ b/arch/x86/kvm/mmu/spte.h @@ -60,10 +60,6 @@ static_assert(SPTE_TDP_AD_ENABLED_MASK == 0); (((address) >> PT64_LEVEL_SHIFT(level)) & ((1 << PT64_LEVEL_BITS) - 1)) #define SHADOW_PT_INDEX(addr, level) PT64_INDEX(addr, level) -/* Bits 9 and 10 are ignored by all non-EPT PTEs. */ -#define DEFAULT_SPTE_HOST_WRITEABLE BIT_ULL(9) -#define DEFAULT_SPTE_MMU_WRITEABLE BIT_ULL(10) - /* * The mask/shift to use for saving the original R/X bits when marking the PTE * as not-present for access tracking purposes. We do not save the W bit as the @@ -78,6 +74,35 @@ static_assert(SPTE_TDP_AD_ENABLED_MASK == 0); SHADOW_ACC_TRACK_SAVED_BITS_SHIFT) static_assert(!(SPTE_TDP_AD_MASK & SHADOW_ACC_TRACK_SAVED_MASK)); +/* + * *_SPTE_HOST_WRITEABLE (aka Host-writable) indicates whether the host permits + * writes to the guest page mapped by the SPTE. This bit is cleared on SPTEs + * that map guest pages in read-only memslots and read-only VMAs. + * + * Invariants: + * - If Host-writable is clear, PT_WRITABLE_MASK must be clear. + * + * + * *_SPTE_MMU_WRITEABLE (aka MMU-writable) indicates whether the shadow MMU + * allows writes to the guest page mapped by the SPTE. This bit is cleared when + * the guest page mapped by the SPTE contains a page table that is being + * monitored for shadow paging. In this case the SPTE can only be made writable + * by unsyncing the shadow page under the mmu_lock. + * + * Invariants: + * - If MMU-writable is clear, PT_WRITABLE_MASK must be clear. + * - If MMU-writable is set, Host-writable must be set. + * + * If MMU-writable is set, PT_WRITABLE_MASK is normally set but can be cleared + * to track writes for dirty logging. For such SPTEs, KVM will locklessly set + * PT_WRITABLE_MASK upon the next write from the guest and record the write in + * the dirty log (see fast_page_fault()). + */ + +/* Bits 9 and 10 are ignored by all non-EPT PTEs. */ +#define DEFAULT_SPTE_HOST_WRITEABLE BIT_ULL(9) +#define DEFAULT_SPTE_MMU_WRITEABLE BIT_ULL(10) + /* * Low ignored bits are at a premium for EPT, use high ignored bits, taking care * to not overlap the A/D type mask or the saved access bits of access-tracked @@ -316,8 +341,13 @@ static __always_inline bool is_rsvd_spte(struct rsvd_bits_validate *rsvd_check, static inline bool spte_can_locklessly_be_made_writable(u64 spte) { - return (spte & shadow_host_writable_mask) && - (spte & shadow_mmu_writable_mask); + if (spte & shadow_mmu_writable_mask) { + WARN_ON_ONCE(!(spte & shadow_host_writable_mask)); + return true; + } + + WARN_ON_ONCE(spte & PT_WRITABLE_MASK); + return false; } static inline u64 get_mmio_spte_generation(u64 spte) From patchwork Thu Jan 13 23:30:20 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Matlack X-Patchwork-Id: 12713168 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 302F9C433F5 for ; Thu, 13 Jan 2022 23:30:33 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S238234AbiAMXac (ORCPT ); Thu, 13 Jan 2022 18:30:32 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:45852 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S238218AbiAMXab (ORCPT ); Thu, 13 Jan 2022 18:30:31 -0500 Received: from mail-pf1-x44a.google.com (mail-pf1-x44a.google.com [IPv6:2607:f8b0:4864:20::44a]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 30057C061574 for ; Thu, 13 Jan 2022 15:30:31 -0800 (PST) Received: by mail-pf1-x44a.google.com with SMTP id i16-20020aa78d90000000b004be3e88d746so672384pfr.13 for ; Thu, 13 Jan 2022 15:30:31 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20210112; h=date:in-reply-to:message-id:mime-version:references:subject:from:to :cc; bh=KW0v15GQjuiGlKaCwQeCIorGmGTs86w3rhJM6nHYyNQ=; b=YfcwdgE8KS0HpGOyrh5RYFusKqiUJyUgWtBSWv0XYYHiVclzz4wFfPdzPgi26Jzoxm ItxtbB1Xz7kkQpgw+tBq8PMClIZXqcqWytRYmWn6wXoVW2jbBUw864Y8VXy1lgcR7GpW 7nYBI4TJLrGkdpQY4adAmh+xcTVtHfyK/L2WFb8r1xzdDgih+lBzpGL11sAQS31jFQg/ zEQ4QOhBeajgTT3rzoiwXOBAmsERLphX2M61Vsa1doTJ50DhYyUhNU/BQuy6CujSHHXs Mp9/p0MojHAE/YqWcHB6TbnpIq3y0pn++qtfz/Qzkasjr/0VIJPNW1Xd5//4dAM8oMj6 pbXw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=KW0v15GQjuiGlKaCwQeCIorGmGTs86w3rhJM6nHYyNQ=; b=NvhjFLBfuaYR3JBM8J/jE7S6bHVgHtcRwt0bYsG0E99lhHvQjhFVeKOkJp2E2BvZfH XO1sPWKX7gRehUln5cdelI9dRmBeAGHv+lgf0ogGwZ+Q55KHIC/cPHn6p1/rPbvGbogt 1w3f/My1mrhkgasi3rDOO8u3TgxPdSJQgqXqEOnqzlDmr0xoJA//0E+GHqCbA3prh9XL LbhUEosGoKmMjR+LSAgiTKK37g87jjxgasUJuSopZbC2MFDNs2is7aoI5LPuMHYC4gfP TvdnPUJCFrycvXzspyhpEYvhl/LP6unklrMcBqqPL0f0M3rnxImGJAPbJEGlPiHqCjr9 7j6A== X-Gm-Message-State: AOAM531HRy+3SdzMNsvR0Ugmh2gUkOnThlf/fi9MSGY/6iWtt+Oso2Gr nnCug2GeHNuhH92I654KjmhEiCeaAAP9cw== X-Google-Smtp-Source: ABdhPJzDDEgArwcky65T79OA0kRIOnZD+q2jTxoEmCKozM9xku6MR7JE+xsFMZVQZqBlWT9ywM/VLkL5zT74cw== X-Received: from dmatlack-heavy.c.googlers.com ([fda3:e722:ac3:cc00:7f:e700:c0a8:19cd]) (user=dmatlack job=sendgmr) by 2002:a17:90b:3a83:: with SMTP id om3mr7585109pjb.186.1642116630704; Thu, 13 Jan 2022 15:30:30 -0800 (PST) Date: Thu, 13 Jan 2022 23:30:20 +0000 In-Reply-To: <20220113233020.3986005-1-dmatlack@google.com> Message-Id: <20220113233020.3986005-5-dmatlack@google.com> Mime-Version: 1.0 References: <20220113233020.3986005-1-dmatlack@google.com> X-Mailer: git-send-email 2.34.1.703.g22d0c6ccf7-goog Subject: [PATCH v2 4/4] KVM: x86/mmu: Improve TLB flush comment in kvm_mmu_slot_remove_write_access() From: David Matlack To: Paolo Bonzini Cc: Sean Christopherson , Vitaly Kuznetsov , Wanpeng Li , Jim Mattson , Joerg Roedel , Ben Gardon , kvm@vger.kernel.org, David Matlack Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org Rewrite the comment in kvm_mmu_slot_remove_write_access() that explains why it is safe to flush TLBs outside of the MMU lock after write-protecting SPTEs for dirty logging. The current comment is a long run-on sentence that was difficult to understand. In addition it was specific to the shadow MMU (mentioning mmu_spte_update()) when the TDP MMU has to handle this as well. The new comment explains: - Why the TLB flush is necessary at all. - Why it is desirable to do the TLB flush outside of the MMU lock. - Why it is safe to do the TLB flush outside of the MMU lock. No functional change intended. Signed-off-by: David Matlack Reviewed-by: Sean Christopherson --- arch/x86/kvm/mmu/mmu.c | 31 ++++++++++++++++++++++--------- 1 file changed, 22 insertions(+), 9 deletions(-) diff --git a/arch/x86/kvm/mmu/mmu.c b/arch/x86/kvm/mmu/mmu.c index 1d275e9d76b5..8ed2b42a7aa3 100644 --- a/arch/x86/kvm/mmu/mmu.c +++ b/arch/x86/kvm/mmu/mmu.c @@ -5756,6 +5756,7 @@ static bool __kvm_zap_rmaps(struct kvm *kvm, gfn_t gfn_start, gfn_t gfn_end) continue; flush = slot_handle_level_range(kvm, memslot, kvm_zap_rmapp, + PG_LEVEL_4K, KVM_MAX_HUGEPAGE_LEVEL, start, end - 1, true, flush); } @@ -5825,15 +5826,27 @@ void kvm_mmu_slot_remove_write_access(struct kvm *kvm, } /* - * We can flush all the TLBs out of the mmu lock without TLB - * corruption since we just change the spte from writable to - * readonly so that we only need to care the case of changing - * spte from present to present (changing the spte from present - * to nonpresent will flush all the TLBs immediately), in other - * words, the only case we care is mmu_spte_update() where we - * have checked Host-writable | MMU-writable instead of - * PT_WRITABLE_MASK, that means it does not depend on PT_WRITABLE_MASK - * anymore. + * Flush TLBs if any SPTEs had to be write-protected to ensure that + * guest writes are reflected in the dirty bitmap before the memslot + * update completes, i.e. before enabling dirty logging is visible to + * userspace. + * + * Perform the TLB flush outside the mmu_lock to reduce the amount of + * time the lock is held. However, this does mean that another CPU can + * now grab the mmu_lock and encounter an SPTE that is write-protected + * while CPUs still have writable versions of that SPTE in their TLB. + * + * This is safe but requires KVM to be careful when making decisions + * based on the write-protection status of an SPTE. Specifically, KVM + * also write-protects SPTEs to monitor changes to guest page tables + * during shadow paging, and must guarantee no CPUs can write to those + * page before the lock is dropped. As mentioned in the previous + * paragraph, a write-protected SPTE is no guarantee that CPU cannot + * perform writes. So to determine if a TLB flush is truly required, KVM + * will clear a separate software-only bit (MMU-writable) and skip the + * flush if-and-only-if this bit was already clear. + * + * See DEFAULT_SPTE_MMU_WRITEABLE for more details. */ if (flush) kvm_arch_flush_remote_tlbs_memslot(kvm, memslot);