From patchwork Wed Nov 8 11:17:34 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Nicolas Saenz Julienne X-Patchwork-Id: 13449814 Received: from lindbergh.monkeyblade.net (lindbergh.monkeyblade.net [23.128.96.19]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id B4CC7156FD; Wed, 8 Nov 2023 11:18:32 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=amazon.com header.i=@amazon.com header.b="cY0+dRyP" Received: from smtp-fw-2101.amazon.com (smtp-fw-2101.amazon.com [72.21.196.25]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id B7871186; Wed, 8 Nov 2023 03:18:31 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amazon.com; i=@amazon.com; q=dns/txt; s=amazon201209; t=1699442313; x=1730978313; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=Uq8x7FLfoIR318BRKD0i4hgYzW68ONMdLEubh/+ZbLg=; b=cY0+dRyPVwx/+v+HXR0Jhe77ryGubFD0miWlX3/F6w2LU8JJY03beq5H UlHqWKLtYvOXKD9YLBb5CoWwh+l9/3uWt2XKQauGXUdroYEFFOz7AAtFO G8lWw2z/bZWgPl7FrUHwDU7yi7pL+HkfHPkrhLjv/g2592ptsfSupSXi/ o=; X-IronPort-AV: E=Sophos;i="6.03,286,1694736000"; d="scan'208";a="361602176" Received: from iad12-co-svc-p1-lb1-vlan3.amazon.com (HELO email-inbound-relay-pdx-2a-m6i4x-44b6fc51.us-west-2.amazon.com) ([10.43.8.6]) by smtp-border-fw-2101.iad2.amazon.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 08 Nov 2023 11:18:29 +0000 Received: from smtpout.prod.us-west-2.prod.farcaster.email.amazon.dev (pdx2-ws-svc-p26-lb5-vlan2.pdx.amazon.com [10.39.38.66]) by email-inbound-relay-pdx-2a-m6i4x-44b6fc51.us-west-2.amazon.com (Postfix) with ESMTPS id 8BB26A07F7; Wed, 8 Nov 2023 11:18:26 +0000 (UTC) Received: from EX19MTAEUC001.ant.amazon.com [10.0.10.100:45371] by smtpin.naws.eu-west-1.prod.farcaster.email.amazon.dev [10.0.26.101:2525] with esmtp (Farcaster) id 58210a6b-8421-43e3-b3a6-2ece76fce745; Wed, 8 Nov 2023 11:18:25 +0000 (UTC) X-Farcaster-Flow-ID: 58210a6b-8421-43e3-b3a6-2ece76fce745 Received: from EX19D004EUC001.ant.amazon.com (10.252.51.190) by EX19MTAEUC001.ant.amazon.com (10.252.51.155) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.1118.39; Wed, 8 Nov 2023 11:18:25 +0000 Received: from dev-dsk-nsaenz-1b-189b39ae.eu-west-1.amazon.com (10.13.235.138) by EX19D004EUC001.ant.amazon.com (10.252.51.190) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.1118.39; Wed, 8 Nov 2023 11:18:20 +0000 From: Nicolas Saenz Julienne To: CC: , , , , , , , , , , , , , , , Nicolas Saenz Julienne Subject: [RFC 01/33] KVM: x86: Decouple lapic.h from hyperv.h Date: Wed, 8 Nov 2023 11:17:34 +0000 Message-ID: <20231108111806.92604-2-nsaenz@amazon.com> X-Mailer: git-send-email 2.40.1 In-Reply-To: <20231108111806.92604-1-nsaenz@amazon.com> References: <20231108111806.92604-1-nsaenz@amazon.com> Precedence: bulk X-Mailing-List: kvm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Originating-IP: [10.13.235.138] X-ClientProxiedBy: EX19D037UWC004.ant.amazon.com (10.13.139.254) To EX19D004EUC001.ant.amazon.com (10.252.51.190) lapic.h has no dependencies with hyperv.h, so don't include it there. Additionally, cpuid.c implicitly relied on hyperv.h's inclusion through lapic.h, so include it explicitly there. Signed-off-by: Nicolas Saenz Julienne --- arch/x86/kvm/cpuid.c | 1 + arch/x86/kvm/lapic.h | 1 - 2 files changed, 1 insertion(+), 1 deletion(-) diff --git a/arch/x86/kvm/cpuid.c b/arch/x86/kvm/cpuid.c index 773132c3bf5a..eabd5e9dc003 100644 --- a/arch/x86/kvm/cpuid.c +++ b/arch/x86/kvm/cpuid.c @@ -28,6 +28,7 @@ #include "trace.h" #include "pmu.h" #include "xen.h" +#include "hyperv.h" /* * Unlike "struct cpuinfo_x86.x86_capability", kvm_cpu_caps doesn't need to be diff --git a/arch/x86/kvm/lapic.h b/arch/x86/kvm/lapic.h index 0a0ea4b5dd8c..e1021517cf04 100644 --- a/arch/x86/kvm/lapic.h +++ b/arch/x86/kvm/lapic.h @@ -6,7 +6,6 @@ #include -#include "hyperv.h" #include "smm.h" #define KVM_APIC_INIT 0 From patchwork Wed Nov 8 11:17:35 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Nicolas Saenz Julienne X-Patchwork-Id: 13449815 Received: from lindbergh.monkeyblade.net (lindbergh.monkeyblade.net [23.128.96.19]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 7B75D16429; Wed, 8 Nov 2023 11:18:38 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=amazon.com header.i=@amazon.com header.b="UD8ea73X" Received: from smtp-fw-6002.amazon.com (smtp-fw-6002.amazon.com [52.95.49.90]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 883921BD8; Wed, 8 Nov 2023 03:18:37 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amazon.com; i=@amazon.com; q=dns/txt; s=amazon201209; t=1699442318; x=1730978318; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=S0vg453WVyLXU8F3J/4ltfll9IYzPE7UP39ZvRkTsE0=; b=UD8ea73Xzy/ZISUJsDmyzeYbm6jPHWFJiSf6gYE6ahEt2jUNt3AM5UPd Sn74+de783X/vxdRS3qR2RLRrnLMeWxsgXoqlriXMCvvttJFBS27wzX/a ZBzAUGDqeLYVigpXSl+Ko5djfyY6rT8956P1rIvnyeNwE0b6UsVpUuvE0 4=; X-IronPort-AV: E=Sophos;i="6.03,286,1694736000"; d="scan'208";a="366811567" Received: from iad12-co-svc-p1-lb1-vlan3.amazon.com (HELO email-inbound-relay-pdx-2a-m6i4x-1197e3af.us-west-2.amazon.com) ([10.43.8.6]) by smtp-border-fw-6002.iad6.amazon.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 08 Nov 2023 11:18:33 +0000 Received: from smtpout.prod.us-west-2.prod.farcaster.email.amazon.dev (pdx2-ws-svc-p26-lb5-vlan2.pdx.amazon.com [10.39.38.66]) by email-inbound-relay-pdx-2a-m6i4x-1197e3af.us-west-2.amazon.com (Postfix) with ESMTPS id DD42C100386; Wed, 8 Nov 2023 11:18:31 +0000 (UTC) Received: from EX19MTAEUA001.ant.amazon.com [10.0.10.100:6644] by smtpin.naws.eu-west-1.prod.farcaster.email.amazon.dev [10.0.22.222:2525] with esmtp (Farcaster) id 87b998a2-b394-4602-b733-977feede779e; Wed, 8 Nov 2023 11:18:30 +0000 (UTC) X-Farcaster-Flow-ID: 87b998a2-b394-4602-b733-977feede779e Received: from EX19D004EUC001.ant.amazon.com (10.252.51.190) by EX19MTAEUA001.ant.amazon.com (10.252.50.192) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.1118.39; Wed, 8 Nov 2023 11:18:30 +0000 Received: from dev-dsk-nsaenz-1b-189b39ae.eu-west-1.amazon.com (10.13.235.138) by EX19D004EUC001.ant.amazon.com (10.252.51.190) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.1118.39; Wed, 8 Nov 2023 11:18:25 +0000 From: Nicolas Saenz Julienne To: CC: , , , , , , , , , , , , , , , Anel Orazgaliyeva , "Nicolas Saenz Julienne" Subject: [RFC 02/33] KVM: x86: Introduce KVM_CAP_APIC_ID_GROUPS Date: Wed, 8 Nov 2023 11:17:35 +0000 Message-ID: <20231108111806.92604-3-nsaenz@amazon.com> X-Mailer: git-send-email 2.40.1 In-Reply-To: <20231108111806.92604-1-nsaenz@amazon.com> References: <20231108111806.92604-1-nsaenz@amazon.com> Precedence: bulk X-Mailing-List: kvm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Originating-IP: [10.13.235.138] X-ClientProxiedBy: EX19D037UWC004.ant.amazon.com (10.13.139.254) To EX19D004EUC001.ant.amazon.com (10.252.51.190) From: Anel Orazgaliyeva Introduce KVM_CAP_APIC_ID_GROUPS, this capability segments the VM's APIC ids into two. The lower bits, the physical APIC id, represent the part that's exposed to the guest. The higher bits, which are private to KVM, groups APICs together. APICs in different groups are isolated from each other, and IPIs can only be directed at APICs that share the same group as its source. Furthermore, groups are only relevant to IPIs, anything incoming from outside the local APIC complex: from the IOAPIC, MSIs, or PV-IPIs is targeted at the default APIC group, group 0. When routing IPIs with physical destinations, KVM will OR the source's vCPU APIC group with the ICR's destination ID and use that to resolve the target lAPIC. The APIC physical map is also made group aware in order to speed up this process. For the sake of simplicity, the logical map is not built while KVM_CAP_APIC_ID_GROUPS is in use and we defer IPI routing to the slower per-vCPU scan method. This capability serves as a building block to implement virtualisation based security features like Hyper-V's Virtual Secure Mode (VSM). VSM introduces a para-virtualised switch that allows for guest CPUs to jump into a different execution context, this switches into a different CPU state, lAPIC state, and memory protections. We model this in KVM by using distinct kvm_vcpus for each context. Moreover, execution contexts are hierarchical and its APICs are meant to remain functional even when the context isn't 'scheduled in'. For example, we have to keep track of timers' expirations, and interrupt execution of lesser priority contexts when relevant. Hence the need to alias physical APIC ids, while keeping the ability to target specific execution contexts. Signed-off-by: Anel Orazgaliyeva Co-developed-by: Nicolas Saenz Julienne Signed-off-by: Nicolas Saenz Julienne --- arch/x86/include/asm/kvm_host.h | 3 ++ arch/x86/include/uapi/asm/kvm.h | 5 +++ arch/x86/kvm/lapic.c | 59 ++++++++++++++++++++++++++++----- arch/x86/kvm/lapic.h | 33 ++++++++++++++++++ arch/x86/kvm/x86.c | 15 +++++++++ include/uapi/linux/kvm.h | 2 ++ 6 files changed, 108 insertions(+), 9 deletions(-) diff --git a/arch/x86/include/asm/kvm_host.h b/arch/x86/include/asm/kvm_host.h index dff10051e9b6..a2f224f95404 100644 --- a/arch/x86/include/asm/kvm_host.h +++ b/arch/x86/include/asm/kvm_host.h @@ -1298,6 +1298,9 @@ struct kvm_arch { struct rw_semaphore apicv_update_lock; unsigned long apicv_inhibit_reasons; + u32 apic_id_group_mask; + u8 apic_id_group_shift; + gpa_t wall_clock; bool mwait_in_guest; diff --git a/arch/x86/include/uapi/asm/kvm.h b/arch/x86/include/uapi/asm/kvm.h index a448d0964fc0..f73d137784d7 100644 --- a/arch/x86/include/uapi/asm/kvm.h +++ b/arch/x86/include/uapi/asm/kvm.h @@ -565,4 +565,9 @@ struct kvm_pmu_event_filter { #define KVM_X86_DEFAULT_VM 0 #define KVM_X86_SW_PROTECTED_VM 1 +/* for KVM_SET_APIC_ID_GROUPS */ +struct kvm_apic_id_groups { + __u8 n_bits; /* nr of bits used to represent group in the APIC ID */ +}; + #endif /* _ASM_X86_KVM_H */ diff --git a/arch/x86/kvm/lapic.c b/arch/x86/kvm/lapic.c index 3e977dbbf993..f55d216cb2a0 100644 --- a/arch/x86/kvm/lapic.c +++ b/arch/x86/kvm/lapic.c @@ -141,7 +141,7 @@ static inline int apic_enabled(struct kvm_lapic *apic) static inline u32 kvm_x2apic_id(struct kvm_lapic *apic) { - return apic->vcpu->vcpu_id; + return kvm_apic_id(apic->vcpu); } static bool kvm_can_post_timer_interrupt(struct kvm_vcpu *vcpu) @@ -219,8 +219,8 @@ static int kvm_recalculate_phys_map(struct kvm_apic_map *new, bool *xapic_id_mismatch) { struct kvm_lapic *apic = vcpu->arch.apic; - u32 x2apic_id = kvm_x2apic_id(apic); - u32 xapic_id = kvm_xapic_id(apic); + u32 x2apic_id = kvm_apic_id_and_group(vcpu); + u32 xapic_id = kvm_apic_id_and_group(vcpu); u32 physical_id; /* @@ -299,6 +299,13 @@ static void kvm_recalculate_logical_map(struct kvm_apic_map *new, u16 mask; u32 ldr; + /* + * Using maps for logical destinations when KVM_CAP_APIC_ID_GRUPS is in + * use isn't supported. + */ + if (kvm_apic_group(vcpu)) + new->logical_mode = KVM_APIC_MODE_MAP_DISABLED; + if (new->logical_mode == KVM_APIC_MODE_MAP_DISABLED) return; @@ -370,6 +377,25 @@ enum { DIRTY }; +int kvm_vm_ioctl_set_apic_id_groups(struct kvm *kvm, + struct kvm_apic_id_groups *groups) +{ + u8 n_bits = groups->n_bits; + + if (n_bits > 32) + return -EINVAL; + + kvm->arch.apic_id_group_mask = n_bits ? GENMASK(31, 32 - n_bits): 0; + /* + * Bitshifts >= than the width of the type are UD, so set the + * apic group shift to 0 when n_bits == 0. The group mask above will + * clear the APIC ID, so group querying functions will return the + * correct value. + */ + kvm->arch.apic_id_group_shift = n_bits ? 32 - n_bits : 0; + return 0; +} + void kvm_recalculate_apic_map(struct kvm *kvm) { struct kvm_apic_map *new, *old = NULL; @@ -414,7 +440,7 @@ void kvm_recalculate_apic_map(struct kvm *kvm) kvm_for_each_vcpu(i, vcpu, kvm) if (kvm_apic_present(vcpu)) - max_id = max(max_id, kvm_x2apic_id(vcpu->arch.apic)); + max_id = max(max_id, kvm_apic_id_and_group(vcpu)); new = kvzalloc(sizeof(struct kvm_apic_map) + sizeof(struct kvm_lapic *) * ((u64)max_id + 1), @@ -525,7 +551,7 @@ static inline void kvm_apic_set_x2apic_id(struct kvm_lapic *apic, u32 id) { u32 ldr = kvm_apic_calc_x2apic_ldr(id); - WARN_ON_ONCE(id != apic->vcpu->vcpu_id); + WARN_ON_ONCE(id != kvm_apic_id(apic->vcpu)); kvm_lapic_set_reg(apic, APIC_ID, id); kvm_lapic_set_reg(apic, APIC_LDR, ldr); @@ -1067,6 +1093,17 @@ bool kvm_apic_match_dest(struct kvm_vcpu *vcpu, struct kvm_lapic *source, struct kvm_lapic *target = vcpu->arch.apic; u32 mda = kvm_apic_mda(vcpu, dest, source, target); + /* + * Make sure vCPUs belong to the same APIC group, it's not possible + * to send interrupts across groups. + * + * Non-IPIs and PV-IPIs can only be injected into the default APIC + * group (group 0). + */ + if ((source && !kvm_match_apic_group(source->vcpu, vcpu)) || + kvm_apic_group(vcpu)) + return false; + ASSERT(target); switch (shorthand) { case APIC_DEST_NOSHORT: @@ -1518,6 +1555,10 @@ void kvm_apic_send_ipi(struct kvm_lapic *apic, u32 icr_low, u32 icr_high) else irq.dest_id = GET_XAPIC_DEST_FIELD(icr_high); + if (irq.dest_mode == APIC_DEST_PHYSICAL) + kvm_apic_id_set_group(apic->vcpu->kvm, + kvm_apic_group(apic->vcpu), &irq.dest_id); + trace_kvm_apic_ipi(icr_low, irq.dest_id); kvm_irq_delivery_to_apic(apic->vcpu->kvm, apic, &irq, NULL); @@ -2541,7 +2582,7 @@ void kvm_lapic_set_base(struct kvm_vcpu *vcpu, u64 value) /* update jump label if enable bit changes */ if ((old_value ^ value) & MSR_IA32_APICBASE_ENABLE) { if (value & MSR_IA32_APICBASE_ENABLE) { - kvm_apic_set_xapic_id(apic, vcpu->vcpu_id); + kvm_apic_set_xapic_id(apic, kvm_apic_id(vcpu)); static_branch_slow_dec_deferred(&apic_hw_disabled); /* Check if there are APF page ready requests pending */ kvm_make_request(KVM_REQ_APF_READY, vcpu); @@ -2553,9 +2594,9 @@ void kvm_lapic_set_base(struct kvm_vcpu *vcpu, u64 value) if ((old_value ^ value) & X2APIC_ENABLE) { if (value & X2APIC_ENABLE) - kvm_apic_set_x2apic_id(apic, vcpu->vcpu_id); + kvm_apic_set_x2apic_id(apic, kvm_apic_id(vcpu)); else if (value & MSR_IA32_APICBASE_ENABLE) - kvm_apic_set_xapic_id(apic, vcpu->vcpu_id); + kvm_apic_set_xapic_id(apic, kvm_apic_id(vcpu)); } if ((old_value ^ value) & (MSR_IA32_APICBASE_ENABLE | X2APIC_ENABLE)) { @@ -2685,7 +2726,7 @@ void kvm_lapic_reset(struct kvm_vcpu *vcpu, bool init_event) /* The xAPIC ID is set at RESET even if the APIC was already enabled. */ if (!init_event) - kvm_apic_set_xapic_id(apic, vcpu->vcpu_id); + kvm_apic_set_xapic_id(apic, kvm_apic_id(vcpu)); kvm_apic_set_version(apic->vcpu); for (i = 0; i < apic->nr_lvt_entries; i++) diff --git a/arch/x86/kvm/lapic.h b/arch/x86/kvm/lapic.h index e1021517cf04..542bd208e52b 100644 --- a/arch/x86/kvm/lapic.h +++ b/arch/x86/kvm/lapic.h @@ -97,6 +97,8 @@ void kvm_lapic_set_tpr(struct kvm_vcpu *vcpu, unsigned long cr8); void kvm_lapic_set_eoi(struct kvm_vcpu *vcpu); void kvm_lapic_set_base(struct kvm_vcpu *vcpu, u64 value); u64 kvm_lapic_get_base(struct kvm_vcpu *vcpu); +int kvm_vm_ioctl_set_apic_id_groups(struct kvm *kvm, + struct kvm_apic_id_groups *groups); void kvm_recalculate_apic_map(struct kvm *kvm); void kvm_apic_set_version(struct kvm_vcpu *vcpu); void kvm_apic_after_set_mcg_cap(struct kvm_vcpu *vcpu); @@ -277,4 +279,35 @@ static inline u8 kvm_xapic_id(struct kvm_lapic *apic) return kvm_lapic_get_reg(apic, APIC_ID) >> 24; } +static inline u32 kvm_apic_id(struct kvm_vcpu *vcpu) +{ + return vcpu->vcpu_id & ~vcpu->kvm->arch.apic_id_group_mask; +} + +static inline u32 kvm_apic_id_and_group(struct kvm_vcpu *vcpu) +{ + return vcpu->vcpu_id; +} + +static inline u32 kvm_apic_group(struct kvm_vcpu *vcpu) +{ + struct kvm *kvm = vcpu->kvm; + + return (vcpu->vcpu_id & kvm->arch.apic_id_group_mask) >> + kvm->arch.apic_id_group_shift; +} + +static inline void kvm_apic_id_set_group(struct kvm *kvm, u32 group, + u32 *apic_id) +{ + *apic_id |= ((group << kvm->arch.apic_id_group_shift) & + kvm->arch.apic_id_group_mask); +} + +static inline bool kvm_match_apic_group(struct kvm_vcpu *src, + struct kvm_vcpu *dst) +{ + return kvm_apic_group(src) == kvm_apic_group(dst); +} + #endif diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c index e3eb608b6692..4cd3f00475c1 100644 --- a/arch/x86/kvm/x86.c +++ b/arch/x86/kvm/x86.c @@ -4526,6 +4526,7 @@ int kvm_vm_ioctl_check_extension(struct kvm *kvm, long ext) case KVM_CAP_VM_DISABLE_NX_HUGE_PAGES: case KVM_CAP_IRQFD_RESAMPLE: case KVM_CAP_MEMORY_FAULT_INFO: + case KVM_CAP_APIC_ID_GROUPS: r = 1; break; case KVM_CAP_EXIT_HYPERCALL: @@ -7112,6 +7113,20 @@ int kvm_arch_vm_ioctl(struct file *filp, unsigned int ioctl, unsigned long arg) r = kvm_vm_ioctl_set_msr_filter(kvm, &filter); break; } + case KVM_SET_APIC_ID_GROUPS: { + struct kvm_apic_id_groups groups; + + r = -EINVAL; + if (kvm->created_vcpus) + goto out; + + r = -EFAULT; + if (copy_from_user(&groups, argp, sizeof(groups))) + goto out; + + r = kvm_vm_ioctl_set_apic_id_groups(kvm, &groups); + break; + } default: r = -ENOTTY; } diff --git a/include/uapi/linux/kvm.h b/include/uapi/linux/kvm.h index 5b5820d19e71..d7a01766bf21 100644 --- a/include/uapi/linux/kvm.h +++ b/include/uapi/linux/kvm.h @@ -1219,6 +1219,7 @@ struct kvm_ppc_resize_hpt { #define KVM_CAP_MEMORY_ATTRIBUTES 232 #define KVM_CAP_GUEST_MEMFD 233 #define KVM_CAP_VM_TYPES 234 +#define KVM_CAP_APIC_ID_GROUPS 235 #ifdef KVM_CAP_IRQ_ROUTING @@ -2307,4 +2308,5 @@ struct kvm_create_guest_memfd { #define KVM_GUEST_MEMFD_ALLOW_HUGEPAGE (1ULL << 0) +#define KVM_SET_APIC_ID_GROUPS _IOW(KVMIO, 0xd7, struct kvm_apic_id_groups) #endif /* __LINUX_KVM_H */ From patchwork Wed Nov 8 11:17:36 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Nicolas Saenz Julienne X-Patchwork-Id: 13449816 Received: from lindbergh.monkeyblade.net (lindbergh.monkeyblade.net [23.128.96.19]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 299A1171A0; Wed, 8 Nov 2023 11:19:09 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=amazon.com header.i=@amazon.com header.b="H13rTqvg" Received: from smtp-fw-52004.amazon.com (smtp-fw-52004.amazon.com [52.119.213.154]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id AFAB41729; Wed, 8 Nov 2023 03:19:07 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amazon.com; i=@amazon.com; q=dns/txt; s=amazon201209; t=1699442349; x=1730978349; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=yuGMTR3q9ZG4fjpyArVTAZq+f+G14drPZZ43vxh56XA=; b=H13rTqvgCM+7YyMNXgDOxbs3kVLjOhs1p0gn6psav2wPbvMurt4ONC7a 9JBZiA6VsFjY47es/uCk7SuO/5d+3CQCPgGlFszUvbcsbj6VkgFMEBUxB J7GhMMBld7elvG/ql3A5P47afHnD/sqb0oaWnnVmRPyfZvpahETo7rJri 8=; X-IronPort-AV: E=Sophos;i="6.03,286,1694736000"; d="scan'208";a="164958711" Received: from iad12-co-svc-p1-lb1-vlan2.amazon.com (HELO email-inbound-relay-pdx-2a-m6i4x-1cca8d67.us-west-2.amazon.com) ([10.43.8.2]) by smtp-border-fw-52004.iad7.amazon.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 08 Nov 2023 11:19:02 +0000 Received: from smtpout.prod.us-east-1.prod.farcaster.email.amazon.dev (pdx2-ws-svc-p26-lb5-vlan3.pdx.amazon.com [10.39.38.70]) by email-inbound-relay-pdx-2a-m6i4x-1cca8d67.us-west-2.amazon.com (Postfix) with ESMTPS id 6B97681C1A; Wed, 8 Nov 2023 11:19:00 +0000 (UTC) Received: from EX19MTAEUA002.ant.amazon.com [10.0.10.100:26732] by smtpin.naws.eu-west-1.prod.farcaster.email.amazon.dev [10.0.10.247:2525] with esmtp (Farcaster) id 051368ee-dbbf-415b-be33-7d8aaa805a15; Wed, 8 Nov 2023 11:18:59 +0000 (UTC) X-Farcaster-Flow-ID: 051368ee-dbbf-415b-be33-7d8aaa805a15 Received: from EX19D004EUC001.ant.amazon.com (10.252.51.190) by EX19MTAEUA002.ant.amazon.com (10.252.50.124) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.1118.39; Wed, 8 Nov 2023 11:18:59 +0000 Received: from dev-dsk-nsaenz-1b-189b39ae.eu-west-1.amazon.com (10.13.235.138) by EX19D004EUC001.ant.amazon.com (10.252.51.190) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.1118.39; Wed, 8 Nov 2023 11:18:54 +0000 From: Nicolas Saenz Julienne To: CC: , , , , , , , , , , , , , , , Nicolas Saenz Julienne Subject: [RFC 03/33] KVM: x86: hyper-v: Introduce XMM output support Date: Wed, 8 Nov 2023 11:17:36 +0000 Message-ID: <20231108111806.92604-4-nsaenz@amazon.com> X-Mailer: git-send-email 2.40.1 In-Reply-To: <20231108111806.92604-1-nsaenz@amazon.com> References: <20231108111806.92604-1-nsaenz@amazon.com> Precedence: bulk X-Mailing-List: kvm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Originating-IP: [10.13.235.138] X-ClientProxiedBy: EX19D036UWB003.ant.amazon.com (10.13.139.172) To EX19D004EUC001.ant.amazon.com (10.252.51.190) Prepare infrastructure to be able to return data through the XMM registers when Hyper-V hypercalls are issues in fast mode. The XMM registers are exposed to user-space through KVM_EXIT_HYPERV_HCALL and restored on successful hypercall completion. Signed-off-by: Nicolas Saenz Julienne --- arch/x86/include/asm/hyperv-tlfs.h | 2 +- arch/x86/kvm/hyperv.c | 33 +++++++++++++++++++++++++++++- include/uapi/linux/kvm.h | 6 ++++++ 3 files changed, 39 insertions(+), 2 deletions(-) diff --git a/arch/x86/include/asm/hyperv-tlfs.h b/arch/x86/include/asm/hyperv-tlfs.h index 2ff26f53cd62..af594aa65307 100644 --- a/arch/x86/include/asm/hyperv-tlfs.h +++ b/arch/x86/include/asm/hyperv-tlfs.h @@ -49,7 +49,7 @@ /* Support for physical CPU dynamic partitioning events is available*/ #define HV_X64_CPU_DYNAMIC_PARTITIONING_AVAILABLE BIT(3) /* - * Support for passing hypercall input parameter block via XMM + * Support for passing hypercall input and output parameter block via XMM * registers is available */ #define HV_X64_HYPERCALL_XMM_INPUT_AVAILABLE BIT(4) diff --git a/arch/x86/kvm/hyperv.c b/arch/x86/kvm/hyperv.c index 238afd7335e4..e1bc861ab3b0 100644 --- a/arch/x86/kvm/hyperv.c +++ b/arch/x86/kvm/hyperv.c @@ -1815,6 +1815,7 @@ struct kvm_hv_hcall { u16 rep_idx; bool fast; bool rep; + bool xmm_dirty; sse128_t xmm[HV_HYPERCALL_MAX_XMM_REGISTERS]; /* @@ -2346,9 +2347,33 @@ static int kvm_hv_hypercall_complete(struct kvm_vcpu *vcpu, u64 result) return ret; } +static void kvm_hv_write_xmm(struct kvm_hyperv_xmm_reg *xmm) +{ + int reg; + + kvm_fpu_get(); + for (reg = 0; reg < HV_HYPERCALL_MAX_XMM_REGISTERS; reg++) { + const sse128_t data = sse128(xmm[reg].low, xmm[reg].high); + _kvm_write_sse_reg(reg, &data); + } + kvm_fpu_put(); +} + +static bool kvm_hv_is_xmm_output_hcall(u16 code) +{ + return false; +} + static int kvm_hv_hypercall_complete_userspace(struct kvm_vcpu *vcpu) { - return kvm_hv_hypercall_complete(vcpu, vcpu->run->hyperv.u.hcall.result); + bool fast = !!(vcpu->run->hyperv.u.hcall.input & HV_HYPERCALL_FAST_BIT); + u16 code = vcpu->run->hyperv.u.hcall.input & 0xffff; + u64 result = vcpu->run->hyperv.u.hcall.result; + + if (kvm_hv_is_xmm_output_hcall(code) && hv_result_success(result) && fast) + kvm_hv_write_xmm(vcpu->run->hyperv.u.hcall.xmm); + + return kvm_hv_hypercall_complete(vcpu, result); } static u16 kvm_hvcall_signal_event(struct kvm_vcpu *vcpu, struct kvm_hv_hcall *hc) @@ -2623,6 +2648,9 @@ int kvm_hv_hypercall(struct kvm_vcpu *vcpu) break; } + if ((ret & HV_HYPERCALL_RESULT_MASK) == HV_STATUS_SUCCESS && hc.xmm_dirty) + kvm_hv_write_xmm((struct kvm_hyperv_xmm_reg*)hc.xmm); + hypercall_complete: return kvm_hv_hypercall_complete(vcpu, ret); @@ -2632,6 +2660,8 @@ int kvm_hv_hypercall(struct kvm_vcpu *vcpu) vcpu->run->hyperv.u.hcall.input = hc.param; vcpu->run->hyperv.u.hcall.params[0] = hc.ingpa; vcpu->run->hyperv.u.hcall.params[1] = hc.outgpa; + if (hc.fast) + memcpy(vcpu->run->hyperv.u.hcall.xmm, hc.xmm, sizeof(hc.xmm)); vcpu->arch.complete_userspace_io = kvm_hv_hypercall_complete_userspace; return 0; } @@ -2780,6 +2810,7 @@ int kvm_get_hv_cpuid(struct kvm_vcpu *vcpu, struct kvm_cpuid2 *cpuid, ent->ebx |= HV_ENABLE_EXTENDED_HYPERCALLS; ent->edx |= HV_X64_HYPERCALL_XMM_INPUT_AVAILABLE; + ent->edx |= HV_X64_HYPERCALL_XMM_OUTPUT_AVAILABLE; ent->edx |= HV_FEATURE_FREQUENCY_MSRS_AVAILABLE; ent->edx |= HV_FEATURE_GUEST_CRASH_MSR_AVAILABLE; diff --git a/include/uapi/linux/kvm.h b/include/uapi/linux/kvm.h index d7a01766bf21..5ce06a1eee2b 100644 --- a/include/uapi/linux/kvm.h +++ b/include/uapi/linux/kvm.h @@ -192,6 +192,11 @@ struct kvm_s390_cmma_log { __u64 values; }; +struct kvm_hyperv_xmm_reg { + __u64 low; + __u64 high; +}; + struct kvm_hyperv_exit { #define KVM_EXIT_HYPERV_SYNIC 1 #define KVM_EXIT_HYPERV_HCALL 2 @@ -210,6 +215,7 @@ struct kvm_hyperv_exit { __u64 input; __u64 result; __u64 params[2]; + struct kvm_hyperv_xmm_reg xmm[6]; } hcall; struct { __u32 msr; From patchwork Wed Nov 8 11:17:37 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Nicolas Saenz Julienne X-Patchwork-Id: 13449817 Received: from lindbergh.monkeyblade.net (lindbergh.monkeyblade.net [23.128.96.19]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id D17B9171A5; Wed, 8 Nov 2023 11:19:15 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=amazon.com header.i=@amazon.com header.b="TNe4BukL" Received: from smtp-fw-80007.amazon.com (smtp-fw-80007.amazon.com [99.78.197.218]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 45B491729; Wed, 8 Nov 2023 03:19:15 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amazon.com; i=@amazon.com; q=dns/txt; s=amazon201209; t=1699442355; x=1730978355; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=G8tv4EqIfPFJRUiW9cZCsErULQwL6MkqMdyO6PRnfzY=; b=TNe4BukLRNg0YJfWA90/7JIByXU+dM2VDsbXlcy/qOnFybyCFrYRcHD8 ZQ2wY9eXVgY73blHeJws0Szlh9DQPrDdIa28YsluQgaSq9LIeTWGrNCzO gSWADZe4RXHIN6DqUxD2pE+OXoWeGGMtdU66EPFsK+1fAs3dn0w0wwAlm U=; X-IronPort-AV: E=Sophos;i="6.03,286,1694736000"; d="scan'208";a="251427581" Received: from pdx4-co-svc-p1-lb2-vlan2.amazon.com (HELO email-inbound-relay-iad-1a-m6i4x-edda28d4.us-east-1.amazon.com) ([10.25.36.210]) by smtp-border-fw-80007.pdx80.corp.amazon.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 08 Nov 2023 11:19:09 +0000 Received: from smtpout.prod.us-east-1.prod.farcaster.email.amazon.dev (iad7-ws-svc-p70-lb3-vlan3.iad.amazon.com [10.32.235.38]) by email-inbound-relay-iad-1a-m6i4x-edda28d4.us-east-1.amazon.com (Postfix) with ESMTPS id 8CC3F8057A; Wed, 8 Nov 2023 11:19:05 +0000 (UTC) Received: from EX19MTAEUB001.ant.amazon.com [10.0.43.254:28968] by smtpin.naws.eu-west-1.prod.farcaster.email.amazon.dev [10.0.17.103:2525] with esmtp (Farcaster) id a4bb631b-a22d-4d34-9086-5dcfe9a4daf9; Wed, 8 Nov 2023 11:19:04 +0000 (UTC) X-Farcaster-Flow-ID: a4bb631b-a22d-4d34-9086-5dcfe9a4daf9 Received: from EX19D004EUC001.ant.amazon.com (10.252.51.190) by EX19MTAEUB001.ant.amazon.com (10.252.51.28) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.1118.39; Wed, 8 Nov 2023 11:19:04 +0000 Received: from dev-dsk-nsaenz-1b-189b39ae.eu-west-1.amazon.com (10.13.235.138) by EX19D004EUC001.ant.amazon.com (10.252.51.190) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.1118.39; Wed, 8 Nov 2023 11:18:59 +0000 From: Nicolas Saenz Julienne To: CC: , , , , , , , , , , , , , , , Nicolas Saenz Julienne Subject: [RFC 04/33] KVM: x86: hyper-v: Move hypercall page handling into separate function Date: Wed, 8 Nov 2023 11:17:37 +0000 Message-ID: <20231108111806.92604-5-nsaenz@amazon.com> X-Mailer: git-send-email 2.40.1 In-Reply-To: <20231108111806.92604-1-nsaenz@amazon.com> References: <20231108111806.92604-1-nsaenz@amazon.com> Precedence: bulk X-Mailing-List: kvm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Originating-IP: [10.13.235.138] X-ClientProxiedBy: EX19D036UWB003.ant.amazon.com (10.13.139.172) To EX19D004EUC001.ant.amazon.com (10.252.51.190) The hypercall page patching is about to grow considerably, move it into its own function. No functional change intended. Signed-off-by: Nicolas Saenz Julienne Reviewed-by: Maxim Levitsky --- arch/x86/kvm/hyperv.c | 69 ++++++++++++++++++++++++------------------- 1 file changed, 39 insertions(+), 30 deletions(-) diff --git a/arch/x86/kvm/hyperv.c b/arch/x86/kvm/hyperv.c index e1bc861ab3b0..78d053042667 100644 --- a/arch/x86/kvm/hyperv.c +++ b/arch/x86/kvm/hyperv.c @@ -256,6 +256,42 @@ static void synic_exit(struct kvm_vcpu_hv_synic *synic, u32 msr) kvm_make_request(KVM_REQ_HV_EXIT, vcpu); } +static int patch_hypercall_page(struct kvm_vcpu *vcpu, u64 data) +{ + struct kvm *kvm = vcpu->kvm; + u8 instructions[9]; + int i = 0; + u64 addr; + + /* + * If Xen and Hyper-V hypercalls are both enabled, disambiguate + * the same way Xen itself does, by setting the bit 31 of EAX + * which is RsvdZ in the 32-bit Hyper-V hypercall ABI and just + * going to be clobbered on 64-bit. + */ + if (kvm_xen_hypercall_enabled(kvm)) { + /* orl $0x80000000, %eax */ + instructions[i++] = 0x0d; + instructions[i++] = 0x00; + instructions[i++] = 0x00; + instructions[i++] = 0x00; + instructions[i++] = 0x80; + } + + /* vmcall/vmmcall */ + static_call(kvm_x86_patch_hypercall)(vcpu, instructions + i); + i += 3; + + /* ret */ + ((unsigned char *)instructions)[i++] = 0xc3; + + addr = data & HV_X64_MSR_HYPERCALL_PAGE_ADDRESS_MASK; + if (kvm_vcpu_write_guest(vcpu, addr, instructions, i)) + return 1; + + return 0; +} + static int synic_set_msr(struct kvm_vcpu_hv_synic *synic, u32 msr, u64 data, bool host) { @@ -1338,11 +1374,7 @@ static int kvm_hv_set_msr_pw(struct kvm_vcpu *vcpu, u32 msr, u64 data, if (!hv->hv_guest_os_id) hv->hv_hypercall &= ~HV_X64_MSR_HYPERCALL_ENABLE; break; - case HV_X64_MSR_HYPERCALL: { - u8 instructions[9]; - int i = 0; - u64 addr; - + case HV_X64_MSR_HYPERCALL: /* if guest os id is not set hypercall should remain disabled */ if (!hv->hv_guest_os_id) break; @@ -1351,34 +1383,11 @@ static int kvm_hv_set_msr_pw(struct kvm_vcpu *vcpu, u32 msr, u64 data, break; } - /* - * If Xen and Hyper-V hypercalls are both enabled, disambiguate - * the same way Xen itself does, by setting the bit 31 of EAX - * which is RsvdZ in the 32-bit Hyper-V hypercall ABI and just - * going to be clobbered on 64-bit. - */ - if (kvm_xen_hypercall_enabled(kvm)) { - /* orl $0x80000000, %eax */ - instructions[i++] = 0x0d; - instructions[i++] = 0x00; - instructions[i++] = 0x00; - instructions[i++] = 0x00; - instructions[i++] = 0x80; - } - - /* vmcall/vmmcall */ - static_call(kvm_x86_patch_hypercall)(vcpu, instructions + i); - i += 3; - - /* ret */ - ((unsigned char *)instructions)[i++] = 0xc3; - - addr = data & HV_X64_MSR_HYPERCALL_PAGE_ADDRESS_MASK; - if (kvm_vcpu_write_guest(vcpu, addr, instructions, i)) + if (patch_hypercall_page(vcpu, data)) return 1; + hv->hv_hypercall = data; break; - } case HV_X64_MSR_REFERENCE_TSC: hv->hv_tsc_page = data; if (hv->hv_tsc_page & HV_X64_MSR_TSC_REFERENCE_ENABLE) { From patchwork Wed Nov 8 11:17:38 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Nicolas Saenz Julienne X-Patchwork-Id: 13449818 Received: from lindbergh.monkeyblade.net (lindbergh.monkeyblade.net [23.128.96.19]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 8E609171A7; Wed, 8 Nov 2023 11:19:16 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=amazon.com header.i=@amazon.com header.b="YPK5Up4/" Received: from smtp-fw-52005.amazon.com (smtp-fw-52005.amazon.com [52.119.213.156]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 6D64519BD; Wed, 8 Nov 2023 03:19:15 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amazon.com; i=@amazon.com; q=dns/txt; s=amazon201209; t=1699442356; x=1730978356; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=Xwm80GfbjkdS94nlx2StfaEHABGwZZtCfK9AfoMHOMQ=; b=YPK5Up4/JXj73WVhsq8zCU9Ya+YBhJDPIMsH7BJvTzQnBqvsL1mXOfrH +dztxjHsnisUYBzKgrPs9+ni7x+c1/Gs/lt/hhwumU30MMqti7MYK26fm NYp62hyOQUAfkIBX4sRNlukC0abaP3+4Za61ENg97K5f3xei7i9TFeQZu A=; X-IronPort-AV: E=Sophos;i="6.03,286,1694736000"; d="scan'208";a="614865362" Received: from iad12-co-svc-p1-lb1-vlan3.amazon.com (HELO email-inbound-relay-pdx-2a-m6i4x-1cca8d67.us-west-2.amazon.com) ([10.43.8.6]) by smtp-border-fw-52005.iad7.amazon.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 08 Nov 2023 11:19:12 +0000 Received: from smtpout.prod.us-east-1.prod.farcaster.email.amazon.dev (pdx2-ws-svc-p26-lb5-vlan3.pdx.amazon.com [10.39.38.70]) by email-inbound-relay-pdx-2a-m6i4x-1cca8d67.us-west-2.amazon.com (Postfix) with ESMTPS id 9D678804AE; Wed, 8 Nov 2023 11:19:10 +0000 (UTC) Received: from EX19MTAEUB002.ant.amazon.com [10.0.17.79:22087] by smtpin.naws.eu-west-1.prod.farcaster.email.amazon.dev [10.0.32.187:2525] with esmtp (Farcaster) id d33121c7-11f2-4715-adf8-59384f014902; Wed, 8 Nov 2023 11:19:09 +0000 (UTC) X-Farcaster-Flow-ID: d33121c7-11f2-4715-adf8-59384f014902 Received: from EX19D004EUC001.ant.amazon.com (10.252.51.190) by EX19MTAEUB002.ant.amazon.com (10.252.51.59) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.1118.39; Wed, 8 Nov 2023 11:19:09 +0000 Received: from dev-dsk-nsaenz-1b-189b39ae.eu-west-1.amazon.com (10.13.235.138) by EX19D004EUC001.ant.amazon.com (10.252.51.190) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.1118.39; Wed, 8 Nov 2023 11:19:04 +0000 From: Nicolas Saenz Julienne To: CC: , , , , , , , , , , , , , , , Nicolas Saenz Julienne Subject: [RFC 05/33] KVM: x86: hyper-v: Introduce VTL call/return prologues in hypercall page Date: Wed, 8 Nov 2023 11:17:38 +0000 Message-ID: <20231108111806.92604-6-nsaenz@amazon.com> X-Mailer: git-send-email 2.40.1 In-Reply-To: <20231108111806.92604-1-nsaenz@amazon.com> References: <20231108111806.92604-1-nsaenz@amazon.com> Precedence: bulk X-Mailing-List: kvm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Originating-IP: [10.13.235.138] X-ClientProxiedBy: EX19D036UWB003.ant.amazon.com (10.13.139.172) To EX19D004EUC001.ant.amazon.com (10.252.51.190) VTL call/return hypercalls have their own entry points in the hypercall page because they don't follow normal hyper-v hypercall conventions. Move the VTL call/return control input into ECX/RAX and set the hypercall code into EAX/RCX before calling the hypercall instruction in order to be able to use the Hyper-V hypercall entry function. Guests can read an emulated code page offsets register to know the offsets into the hypercall page for the VTL call/return entries. Signed-off-by: Nicolas Saenz Julienne --- My tree has the additional patch, we're still trying to understand under what conditions Windows expects the offset to be fixed. diff --git a/arch/x86/kvm/hyperv.c b/arch/x86/kvm/hyperv.c index 54f7f36a89bf..9f2ea8c34447 100644 --- a/arch/x86/kvm/hyperv.c +++ b/arch/x86/kvm/hyperv.c @@ -294,6 +294,7 @@ static int patch_hypercall_page(struct kvm_vcpu *vcpu, u64 data) /* VTL call/return entries */ if (!kvm_xen_hypercall_enabled(kvm) && kvm_hv_vsm_enabled(kvm)) { + i = 22; #ifdef CONFIG_X86_64 if (is_64_bit_mode(vcpu)) { /* --- arch/x86/include/asm/kvm_host.h | 2 + arch/x86/kvm/hyperv.c | 78 ++++++++++++++++++++++++++++++- include/asm-generic/hyperv-tlfs.h | 11 +++++ 3 files changed, 90 insertions(+), 1 deletion(-) diff --git a/arch/x86/include/asm/kvm_host.h b/arch/x86/include/asm/kvm_host.h index a2f224f95404..00cd21b09f8c 100644 --- a/arch/x86/include/asm/kvm_host.h +++ b/arch/x86/include/asm/kvm_host.h @@ -1105,6 +1105,8 @@ struct kvm_hv { u64 hv_tsc_emulation_status; u64 hv_invtsc_control; + union hv_register_vsm_code_page_offsets vsm_code_page_offsets; + /* How many vCPUs have VP index != vCPU index */ atomic_t num_mismatched_vp_indexes; diff --git a/arch/x86/kvm/hyperv.c b/arch/x86/kvm/hyperv.c index 78d053042667..d4b1b53ea63d 100644 --- a/arch/x86/kvm/hyperv.c +++ b/arch/x86/kvm/hyperv.c @@ -259,7 +259,8 @@ static void synic_exit(struct kvm_vcpu_hv_synic *synic, u32 msr) static int patch_hypercall_page(struct kvm_vcpu *vcpu, u64 data) { struct kvm *kvm = vcpu->kvm; - u8 instructions[9]; + struct kvm_hv *hv = to_kvm_hv(kvm); + u8 instructions[0x30]; int i = 0; u64 addr; @@ -285,6 +286,81 @@ static int patch_hypercall_page(struct kvm_vcpu *vcpu, u64 data) /* ret */ ((unsigned char *)instructions)[i++] = 0xc3; + /* VTL call/return entries */ + if (!kvm_xen_hypercall_enabled(kvm) && kvm_hv_vsm_enabled(kvm)) { +#ifdef CONFIG_X86_64 + if (is_64_bit_mode(vcpu)) { + /* + * VTL call 64-bit entry prologue: + * mov %rcx, %rax + * mov $0x11, %ecx + * jmp 0: + */ + hv->vsm_code_page_offsets.vtl_call_offset = i; + instructions[i++] = 0x48; + instructions[i++] = 0x89; + instructions[i++] = 0xc8; + instructions[i++] = 0xb9; + instructions[i++] = 0x11; + instructions[i++] = 0x00; + instructions[i++] = 0x00; + instructions[i++] = 0x00; + instructions[i++] = 0xeb; + instructions[i++] = 0xe0; + /* + * VTL return 64-bit entry prologue: + * mov %rcx, %rax + * mov $0x12, %ecx + * jmp 0: + */ + hv->vsm_code_page_offsets.vtl_return_offset = i; + instructions[i++] = 0x48; + instructions[i++] = 0x89; + instructions[i++] = 0xc8; + instructions[i++] = 0xb9; + instructions[i++] = 0x12; + instructions[i++] = 0x00; + instructions[i++] = 0x00; + instructions[i++] = 0x00; + instructions[i++] = 0xeb; + instructions[i++] = 0xd6; + } else +#endif + { + /* + * VTL call 32-bit entry prologue: + * mov %eax, %ecx + * mov $0x11, %eax + * jmp 0: + */ + hv->vsm_code_page_offsets.vtl_call_offset = i; + instructions[i++] = 0x89; + instructions[i++] = 0xc1; + instructions[i++] = 0xb8; + instructions[i++] = 0x11; + instructions[i++] = 0x00; + instructions[i++] = 0x00; + instructions[i++] = 0x00; + instructions[i++] = 0xeb; + instructions[i++] = 0xf3; + /* + * VTL return 32-bit entry prologue: + * mov %eax, %ecx + * mov $0x12, %eax + * jmp 0: + */ + hv->vsm_code_page_offsets.vtl_return_offset = i; + instructions[i++] = 0x89; + instructions[i++] = 0xc1; + instructions[i++] = 0xb8; + instructions[i++] = 0x12; + instructions[i++] = 0x00; + instructions[i++] = 0x00; + instructions[i++] = 0x00; + instructions[i++] = 0xeb; + instructions[i++] = 0xea; + } + } addr = data & HV_X64_MSR_HYPERCALL_PAGE_ADDRESS_MASK; if (kvm_vcpu_write_guest(vcpu, addr, instructions, i)) return 1; diff --git a/include/asm-generic/hyperv-tlfs.h b/include/asm-generic/hyperv-tlfs.h index fdac4a1714ec..0e7643c1ef01 100644 --- a/include/asm-generic/hyperv-tlfs.h +++ b/include/asm-generic/hyperv-tlfs.h @@ -823,4 +823,15 @@ struct hv_mmio_write_input { u8 data[HV_HYPERCALL_MMIO_MAX_DATA_LENGTH]; } __packed; +/* + * VTL call/return hypercall page offsets register + */ +union hv_register_vsm_code_page_offsets { + u64 as_u64; + struct { + u64 vtl_call_offset:12; + u64 vtl_return_offset:12; + u64 reserved:40; + } __packed; +}; #endif From patchwork Wed Nov 8 11:17:39 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Nicolas Saenz Julienne X-Patchwork-Id: 13449819 Received: from lindbergh.monkeyblade.net (lindbergh.monkeyblade.net [23.128.96.19]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 07FFE15E89; Wed, 8 Nov 2023 11:19:46 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=amazon.com header.i=@amazon.com header.b="M4wgJ4HB" Received: from smtp-fw-6002.amazon.com (smtp-fw-6002.amazon.com [52.95.49.90]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id E8CD11FCC; Wed, 8 Nov 2023 03:19:44 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amazon.com; i=@amazon.com; q=dns/txt; s=amazon201209; t=1699442385; x=1730978385; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=wig6s7B83KYNLTquljMX0q4Ptac31Xr4L9axIrcRcNM=; b=M4wgJ4HB1LfCwKdtqmsclWizgthienECE/g9Em3VyZhPBSLDHlnUTXEH hk31EPxUkZA9JSu4lpki0afzSqaXpiVA03u0zmsxVj4TG2XTcKoky/wP9 FFJ2SW5xAi6oB9Lhwu2MDYkkJWye8QUQHOyU0bP7vbKNipDI8Q0sV2TBs U=; X-IronPort-AV: E=Sophos;i="6.03,286,1694736000"; d="scan'208";a="366812182" Received: from iad12-co-svc-p1-lb1-vlan3.amazon.com (HELO email-inbound-relay-pdx-2c-m6i4x-d2040ec1.us-west-2.amazon.com) ([10.43.8.6]) by smtp-border-fw-6002.iad6.amazon.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 08 Nov 2023 11:19:43 +0000 Received: from smtpout.prod.us-east-1.prod.farcaster.email.amazon.dev (pdx2-ws-svc-p26-lb5-vlan2.pdx.amazon.com [10.39.38.66]) by email-inbound-relay-pdx-2c-m6i4x-d2040ec1.us-west-2.amazon.com (Postfix) with ESMTPS id CBF6E40DAE; Wed, 8 Nov 2023 11:19:41 +0000 (UTC) Received: from EX19MTAEUC002.ant.amazon.com [10.0.43.254:55579] by smtpin.naws.eu-west-1.prod.farcaster.email.amazon.dev [10.0.32.187:2525] with esmtp (Farcaster) id a8c00b42-a893-40dc-869b-f1f46fa9119f; Wed, 8 Nov 2023 11:19:40 +0000 (UTC) X-Farcaster-Flow-ID: a8c00b42-a893-40dc-869b-f1f46fa9119f Received: from EX19D004EUC001.ant.amazon.com (10.252.51.190) by EX19MTAEUC002.ant.amazon.com (10.252.51.181) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.1118.39; Wed, 8 Nov 2023 11:19:37 +0000 Received: from dev-dsk-nsaenz-1b-189b39ae.eu-west-1.amazon.com (10.13.235.138) by EX19D004EUC001.ant.amazon.com (10.252.51.190) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.1118.39; Wed, 8 Nov 2023 11:19:33 +0000 From: Nicolas Saenz Julienne To: CC: , , , , , , , , , , , , , , , Nicolas Saenz Julienne Subject: [RFC 06/33] KVM: x86: hyper-v: Introduce VTL awareness to Hyper-V's PV-IPIs Date: Wed, 8 Nov 2023 11:17:39 +0000 Message-ID: <20231108111806.92604-7-nsaenz@amazon.com> X-Mailer: git-send-email 2.40.1 In-Reply-To: <20231108111806.92604-1-nsaenz@amazon.com> References: <20231108111806.92604-1-nsaenz@amazon.com> Precedence: bulk X-Mailing-List: kvm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Originating-IP: [10.13.235.138] X-ClientProxiedBy: EX19D033UWC002.ant.amazon.com (10.13.139.196) To EX19D004EUC001.ant.amazon.com (10.252.51.190) HVCALL_SEND_IPI and HVCALL_SEND_IPI_EX allow targeting specific a specific VTL. Honour the requests. Signed-off-by: Nicolas Saenz Julienne --- arch/x86/kvm/hyperv.c | 24 +++++++++++++++++------- arch/x86/kvm/trace.h | 20 ++++++++++++-------- include/asm-generic/hyperv-tlfs.h | 6 ++++-- 3 files changed, 33 insertions(+), 17 deletions(-) diff --git a/arch/x86/kvm/hyperv.c b/arch/x86/kvm/hyperv.c index d4b1b53ea63d..2cf430f6ddd8 100644 --- a/arch/x86/kvm/hyperv.c +++ b/arch/x86/kvm/hyperv.c @@ -2230,7 +2230,7 @@ static u64 kvm_hv_flush_tlb(struct kvm_vcpu *vcpu, struct kvm_hv_hcall *hc) } static void kvm_hv_send_ipi_to_many(struct kvm *kvm, u32 vector, - u64 *sparse_banks, u64 valid_bank_mask) + u64 *sparse_banks, u64 valid_bank_mask, int vtl) { struct kvm_lapic_irq irq = { .delivery_mode = APIC_DM_FIXED, @@ -2245,6 +2245,9 @@ static void kvm_hv_send_ipi_to_many(struct kvm *kvm, u32 vector, valid_bank_mask, sparse_banks)) continue; + if (kvm_hv_get_active_vtl(vcpu) != vtl) + continue; + /* We fail only when APIC is disabled */ kvm_apic_set_irq(vcpu, &irq, NULL); } @@ -2257,13 +2260,19 @@ static u64 kvm_hv_send_ipi(struct kvm_vcpu *vcpu, struct kvm_hv_hcall *hc) struct kvm *kvm = vcpu->kvm; struct hv_send_ipi_ex send_ipi_ex; struct hv_send_ipi send_ipi; + union hv_input_vtl *in_vtl; u64 valid_bank_mask; u32 vector; bool all_cpus; + u8 vtl; + + /* VTL is at the same offset on both IPI types */ + in_vtl = &send_ipi.in_vtl; + vtl = in_vtl->use_target_vtl ? in_vtl->target_vtl : kvm_hv_get_active_vtl(vcpu); if (hc->code == HVCALL_SEND_IPI) { if (!hc->fast) { - if (unlikely(kvm_read_guest(kvm, hc->ingpa, &send_ipi, + if (unlikely(kvm_vcpu_read_guest(vcpu, hc->ingpa, &send_ipi, sizeof(send_ipi)))) return HV_STATUS_INVALID_HYPERCALL_INPUT; sparse_banks[0] = send_ipi.cpu_mask; @@ -2278,10 +2287,10 @@ static u64 kvm_hv_send_ipi(struct kvm_vcpu *vcpu, struct kvm_hv_hcall *hc) all_cpus = false; valid_bank_mask = BIT_ULL(0); - trace_kvm_hv_send_ipi(vector, sparse_banks[0]); + trace_kvm_hv_send_ipi(vector, sparse_banks[0], vtl); } else { if (!hc->fast) { - if (unlikely(kvm_read_guest(kvm, hc->ingpa, &send_ipi_ex, + if (unlikely(kvm_vcpu_read_guest(vcpu, hc->ingpa, &send_ipi_ex, sizeof(send_ipi_ex)))) return HV_STATUS_INVALID_HYPERCALL_INPUT; } else { @@ -2292,7 +2301,8 @@ static u64 kvm_hv_send_ipi(struct kvm_vcpu *vcpu, struct kvm_hv_hcall *hc) trace_kvm_hv_send_ipi_ex(send_ipi_ex.vector, send_ipi_ex.vp_set.format, - send_ipi_ex.vp_set.valid_bank_mask); + send_ipi_ex.vp_set.valid_bank_mask, + vtl); vector = send_ipi_ex.vector; valid_bank_mask = send_ipi_ex.vp_set.valid_bank_mask; @@ -2322,9 +2332,9 @@ static u64 kvm_hv_send_ipi(struct kvm_vcpu *vcpu, struct kvm_hv_hcall *hc) return HV_STATUS_INVALID_HYPERCALL_INPUT; if (all_cpus) - kvm_hv_send_ipi_to_many(kvm, vector, NULL, 0); + kvm_hv_send_ipi_to_many(kvm, vector, NULL, 0, vtl); else - kvm_hv_send_ipi_to_many(kvm, vector, sparse_banks, valid_bank_mask); + kvm_hv_send_ipi_to_many(kvm, vector, sparse_banks, valid_bank_mask, vtl); ret_success: return HV_STATUS_SUCCESS; diff --git a/arch/x86/kvm/trace.h b/arch/x86/kvm/trace.h index 83843379813e..ab8839c47bc7 100644 --- a/arch/x86/kvm/trace.h +++ b/arch/x86/kvm/trace.h @@ -1606,42 +1606,46 @@ TRACE_EVENT(kvm_hv_flush_tlb_ex, * Tracepoints for kvm_hv_send_ipi. */ TRACE_EVENT(kvm_hv_send_ipi, - TP_PROTO(u32 vector, u64 processor_mask), - TP_ARGS(vector, processor_mask), + TP_PROTO(u32 vector, u64 processor_mask, u8 vtl), + TP_ARGS(vector, processor_mask, vtl), TP_STRUCT__entry( __field(u32, vector) __field(u64, processor_mask) + __field(u8, vtl) ), TP_fast_assign( __entry->vector = vector; __entry->processor_mask = processor_mask; + __entry->vtl = vtl; ), - TP_printk("vector %x processor_mask 0x%llx", - __entry->vector, __entry->processor_mask) + TP_printk("vector %x processor_mask 0x%llx vtl %d", + __entry->vector, __entry->processor_mask, __entry->vtl) ); TRACE_EVENT(kvm_hv_send_ipi_ex, - TP_PROTO(u32 vector, u64 format, u64 valid_bank_mask), - TP_ARGS(vector, format, valid_bank_mask), + TP_PROTO(u32 vector, u64 format, u64 valid_bank_mask, u8 vtl), + TP_ARGS(vector, format, valid_bank_mask, vtl), TP_STRUCT__entry( __field(u32, vector) __field(u64, format) __field(u64, valid_bank_mask) + __field(u8, vtl) ), TP_fast_assign( __entry->vector = vector; __entry->format = format; __entry->valid_bank_mask = valid_bank_mask; + __entry->vtl = vtl; ), - TP_printk("vector %x format %llx valid_bank_mask 0x%llx", + TP_printk("vector %x format %llx valid_bank_mask 0x%llx vtl %d", __entry->vector, __entry->format, - __entry->valid_bank_mask) + __entry->valid_bank_mask, __entry->vtl) ); TRACE_EVENT(kvm_pv_tlb_flush, diff --git a/include/asm-generic/hyperv-tlfs.h b/include/asm-generic/hyperv-tlfs.h index 0e7643c1ef01..40d7dc793c03 100644 --- a/include/asm-generic/hyperv-tlfs.h +++ b/include/asm-generic/hyperv-tlfs.h @@ -424,14 +424,16 @@ struct hv_vpset { /* HvCallSendSyntheticClusterIpi hypercall */ struct hv_send_ipi { u32 vector; - u32 reserved; + union hv_input_vtl in_vtl; + u8 reserved[3]; u64 cpu_mask; } __packed; /* HvCallSendSyntheticClusterIpiEx hypercall */ struct hv_send_ipi_ex { u32 vector; - u32 reserved; + union hv_input_vtl in_vtl; + u8 reserved[3]; struct hv_vpset vp_set; } __packed; From patchwork Wed Nov 8 11:17:40 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Nicolas Saenz Julienne X-Patchwork-Id: 13449821 Received: from lindbergh.monkeyblade.net (lindbergh.monkeyblade.net [23.128.96.19]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 32B8615E8F; Wed, 8 Nov 2023 11:19:55 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=amazon.com header.i=@amazon.com header.b="sE+JHi5R" Received: from smtp-fw-52004.amazon.com (smtp-fw-52004.amazon.com [52.119.213.154]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id D1FAE1BD4; Wed, 8 Nov 2023 03:19:53 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amazon.com; i=@amazon.com; q=dns/txt; s=amazon201209; t=1699442395; x=1730978395; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=LwvxkM973QXPotqf1BmHlmaFhGY0KPkEa30ixGz574A=; b=sE+JHi5RRfWD6k0hVhdwIFulDOEIzbEvBk1b20Dtzd+LgoeZDDXT9ZP1 rK/bYKS8hJlfuxIMXlpuQXE+F0CufSmv3NYAAFrFYY4cesWml5X+UXPTa 83aWhUFHSCCL1xxv2imR9frXrfMK5SvUJ0bjsPDxo6zGpfALx9bFAhamv 0=; X-IronPort-AV: E=Sophos;i="6.03,286,1694736000"; d="scan'208";a="164958920" Received: from iad12-co-svc-p1-lb1-vlan2.amazon.com (HELO email-inbound-relay-iad-1a-m6i4x-b5bd57cf.us-east-1.amazon.com) ([10.43.8.2]) by smtp-border-fw-52004.iad7.amazon.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 08 Nov 2023 11:19:53 +0000 Received: from smtpout.prod.us-east-1.prod.farcaster.email.amazon.dev (iad7-ws-svc-p70-lb3-vlan3.iad.amazon.com [10.32.235.38]) by email-inbound-relay-iad-1a-m6i4x-b5bd57cf.us-east-1.amazon.com (Postfix) with ESMTPS id DDFEC48DA2; Wed, 8 Nov 2023 11:19:49 +0000 (UTC) Received: from EX19MTAEUC001.ant.amazon.com [10.0.10.100:37568] by smtpin.naws.eu-west-1.prod.farcaster.email.amazon.dev [10.0.45.210:2525] with esmtp (Farcaster) id b100eba5-8ee9-421b-9077-66ac2c5bb9c6; Wed, 8 Nov 2023 11:19:48 +0000 (UTC) X-Farcaster-Flow-ID: b100eba5-8ee9-421b-9077-66ac2c5bb9c6 Received: from EX19D004EUC001.ant.amazon.com (10.252.51.190) by EX19MTAEUC001.ant.amazon.com (10.252.51.155) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.1118.39; Wed, 8 Nov 2023 11:19:42 +0000 Received: from dev-dsk-nsaenz-1b-189b39ae.eu-west-1.amazon.com (10.13.235.138) by EX19D004EUC001.ant.amazon.com (10.252.51.190) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.1118.39; Wed, 8 Nov 2023 11:19:38 +0000 From: Nicolas Saenz Julienne To: CC: , , , , , , , , , , , , , , , Nicolas Saenz Julienne Subject: [RFC 07/33] KVM: x86: hyper-v: Introduce KVM_CAP_HYPERV_VSM Date: Wed, 8 Nov 2023 11:17:40 +0000 Message-ID: <20231108111806.92604-8-nsaenz@amazon.com> X-Mailer: git-send-email 2.40.1 In-Reply-To: <20231108111806.92604-1-nsaenz@amazon.com> References: <20231108111806.92604-1-nsaenz@amazon.com> Precedence: bulk X-Mailing-List: kvm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Originating-IP: [10.13.235.138] X-ClientProxiedBy: EX19D033UWC002.ant.amazon.com (10.13.139.196) To EX19D004EUC001.ant.amazon.com (10.252.51.190) Introduce a new capability to enable Hyper-V Virtual Secure Mode (VSM) emulation support. Signed-off-by: Nicolas Saenz Julienne --- arch/x86/include/asm/kvm_host.h | 2 ++ arch/x86/kvm/hyperv.h | 5 +++++ arch/x86/kvm/x86.c | 5 +++++ include/uapi/linux/kvm.h | 1 + 4 files changed, 13 insertions(+) diff --git a/arch/x86/include/asm/kvm_host.h b/arch/x86/include/asm/kvm_host.h index 00cd21b09f8c..7712e31b7537 100644 --- a/arch/x86/include/asm/kvm_host.h +++ b/arch/x86/include/asm/kvm_host.h @@ -1118,6 +1118,8 @@ struct kvm_hv { struct hv_partition_assist_pg *hv_pa_pg; struct kvm_hv_syndbg hv_syndbg; + + bool hv_enable_vsm; }; struct msr_bitmap_range { diff --git a/arch/x86/kvm/hyperv.h b/arch/x86/kvm/hyperv.h index f83b8db72b11..2bfed69ba0db 100644 --- a/arch/x86/kvm/hyperv.h +++ b/arch/x86/kvm/hyperv.h @@ -238,4 +238,9 @@ static inline int kvm_hv_verify_vp_assist(struct kvm_vcpu *vcpu) int kvm_hv_vcpu_flush_tlb(struct kvm_vcpu *vcpu); +static inline bool kvm_hv_vsm_enabled(struct kvm *kvm) +{ + return kvm->arch.hyperv.hv_enable_vsm; +} + #endif diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c index 4cd3f00475c1..b0512e433032 100644 --- a/arch/x86/kvm/x86.c +++ b/arch/x86/kvm/x86.c @@ -4485,6 +4485,7 @@ int kvm_vm_ioctl_check_extension(struct kvm *kvm, long ext) case KVM_CAP_HYPERV_CPUID: case KVM_CAP_HYPERV_ENFORCE_CPUID: case KVM_CAP_SYS_HYPERV_CPUID: + case KVM_CAP_HYPERV_VSM: case KVM_CAP_PCI_SEGMENT: case KVM_CAP_DEBUGREGS: case KVM_CAP_X86_ROBUST_SINGLESTEP: @@ -6519,6 +6520,10 @@ int kvm_vm_ioctl_enable_cap(struct kvm *kvm, } mutex_unlock(&kvm->lock); break; + case KVM_CAP_HYPERV_VSM: + kvm->arch.hyperv.hv_enable_vsm = true; + r = 0; + break; default: r = -EINVAL; break; diff --git a/include/uapi/linux/kvm.h b/include/uapi/linux/kvm.h index 5ce06a1eee2b..168b6ac6ebe5 100644 --- a/include/uapi/linux/kvm.h +++ b/include/uapi/linux/kvm.h @@ -1226,6 +1226,7 @@ struct kvm_ppc_resize_hpt { #define KVM_CAP_GUEST_MEMFD 233 #define KVM_CAP_VM_TYPES 234 #define KVM_CAP_APIC_ID_GROUPS 235 +#define KVM_CAP_HYPERV_VSM 237 #ifdef KVM_CAP_IRQ_ROUTING From patchwork Wed Nov 8 11:17:41 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Nicolas Saenz Julienne X-Patchwork-Id: 13449820 Received: from lindbergh.monkeyblade.net (lindbergh.monkeyblade.net [23.128.96.19]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id CD7D415E89; Wed, 8 Nov 2023 11:19:53 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=amazon.com header.i=@amazon.com header.b="uoXdxXqx" Received: from smtp-fw-6002.amazon.com (smtp-fw-6002.amazon.com [52.95.49.90]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id E0FA91BEB; Wed, 8 Nov 2023 03:19:52 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amazon.com; i=@amazon.com; q=dns/txt; s=amazon201209; t=1699442393; x=1730978393; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=cyTCynOmMb0cQQa0wX5d7cLulbacT3tvj9NSCTRwhHI=; b=uoXdxXqxTJ+aXNNZAiSwqC9CpZyqE0rCCsnQYVD3oz+KZfPUCxjMgi4v DkfMnOq7N8xKvScGfVrJBv3yt4HvH5QadPHsir17js6YhAczT4/E+p9rz oaY9CCXZCcpgJ/iaZplwBa90b6O131+qYzCpi7o2Pu+rtUv12nNOFpB2p Q=; X-IronPort-AV: E=Sophos;i="6.03,286,1694736000"; d="scan'208";a="366812308" Received: from iad12-co-svc-p1-lb1-vlan3.amazon.com (HELO email-inbound-relay-iad-1d-m6i4x-f05d30a1.us-east-1.amazon.com) ([10.43.8.6]) by smtp-border-fw-6002.iad6.amazon.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 08 Nov 2023 11:19:52 +0000 Received: from smtpout.prod.us-east-1.prod.farcaster.email.amazon.dev (iad7-ws-svc-p70-lb3-vlan3.iad.amazon.com [10.32.235.38]) by email-inbound-relay-iad-1d-m6i4x-f05d30a1.us-east-1.amazon.com (Postfix) with ESMTPS id 23B2F80D5F; Wed, 8 Nov 2023 11:19:48 +0000 (UTC) Received: from EX19MTAEUA002.ant.amazon.com [10.0.10.100:19113] by smtpin.naws.eu-west-1.prod.farcaster.email.amazon.dev [10.0.43.105:2525] with esmtp (Farcaster) id fbe08b9c-13d3-4e3e-a014-4f60b4421048; Wed, 8 Nov 2023 11:19:48 +0000 (UTC) X-Farcaster-Flow-ID: fbe08b9c-13d3-4e3e-a014-4f60b4421048 Received: from EX19D004EUC001.ant.amazon.com (10.252.51.190) by EX19MTAEUA002.ant.amazon.com (10.252.50.124) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.1118.39; Wed, 8 Nov 2023 11:19:47 +0000 Received: from dev-dsk-nsaenz-1b-189b39ae.eu-west-1.amazon.com (10.13.235.138) by EX19D004EUC001.ant.amazon.com (10.252.51.190) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.1118.39; Wed, 8 Nov 2023 11:19:43 +0000 From: Nicolas Saenz Julienne To: CC: , , , , , , , , , , , , , , , Nicolas Saenz Julienne Subject: [RFC 08/33] KVM: x86: Don't use hv_timer if CAP_HYPERV_VSM enabled Date: Wed, 8 Nov 2023 11:17:41 +0000 Message-ID: <20231108111806.92604-9-nsaenz@amazon.com> X-Mailer: git-send-email 2.40.1 In-Reply-To: <20231108111806.92604-1-nsaenz@amazon.com> References: <20231108111806.92604-1-nsaenz@amazon.com> Precedence: bulk X-Mailing-List: kvm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Originating-IP: [10.13.235.138] X-ClientProxiedBy: EX19D033UWC002.ant.amazon.com (10.13.139.196) To EX19D004EUC001.ant.amazon.com (10.252.51.190) VSM's VTLs are modeled by using a distinct vCPU per VTL. While one VTL is running the rest of vCPUs are left idle. This doesn't play well with the approach of tracking emulated timer expiration by using the VMX preemption timer. Inactive VTL's timers are still meant to run and inject interrupts regardless of their runstate. Signed-off-by: Nicolas Saenz Julienne --- arch/x86/kvm/lapic.c | 7 ++++--- 1 file changed, 4 insertions(+), 3 deletions(-) diff --git a/arch/x86/kvm/lapic.c b/arch/x86/kvm/lapic.c index f55d216cb2a0..8cc75b24381b 100644 --- a/arch/x86/kvm/lapic.c +++ b/arch/x86/kvm/lapic.c @@ -152,9 +152,10 @@ static bool kvm_can_post_timer_interrupt(struct kvm_vcpu *vcpu) bool kvm_can_use_hv_timer(struct kvm_vcpu *vcpu) { - return kvm_x86_ops.set_hv_timer - && !(kvm_mwait_in_guest(vcpu->kvm) || - kvm_can_post_timer_interrupt(vcpu)); + return kvm_x86_ops.set_hv_timer && + !(kvm_mwait_in_guest(vcpu->kvm) || + kvm_can_post_timer_interrupt(vcpu)) && + !(kvm_hv_vsm_enabled(vcpu->kvm)); } static bool kvm_use_posted_timer_interrupt(struct kvm_vcpu *vcpu) From patchwork Wed Nov 8 11:17:42 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Nicolas Saenz Julienne X-Patchwork-Id: 13449822 Received: from lindbergh.monkeyblade.net (lindbergh.monkeyblade.net [23.128.96.19]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 134EC15E89; Wed, 8 Nov 2023 11:20:20 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=amazon.com header.i=@amazon.com header.b="C0dt2eL7" Received: from smtp-fw-80007.amazon.com (smtp-fw-80007.amazon.com [99.78.197.218]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 8552D1FC8; Wed, 8 Nov 2023 03:20:19 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amazon.com; i=@amazon.com; q=dns/txt; s=amazon201209; t=1699442419; x=1730978419; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=+q6HaRK4F5IqO9uASaPkSnlSL0f+bqvxqAxqsapmXI8=; b=C0dt2eL7HaTLnf+XladYEXWisf8ff8t8FXQGWEY1ORRsP8yKVNUG6dcQ Eq4tEIis1CZvWqwIrhNeHYugTgs6YAFQIKVddISCpF40nigHDaRTCl9L+ gdfO3zAwk0J+wPdQ3mhMrK2UqnhfGb1l5hlOsO7sSxpUU3o7i534VyMoH 8=; X-IronPort-AV: E=Sophos;i="6.03,286,1694736000"; d="scan'208";a="251427918" Received: from pdx4-co-svc-p1-lb2-vlan2.amazon.com (HELO email-inbound-relay-pdx-2b-m6i4x-a893d89c.us-west-2.amazon.com) ([10.25.36.210]) by smtp-border-fw-80007.pdx80.corp.amazon.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 08 Nov 2023 11:20:18 +0000 Received: from smtpout.prod.us-west-2.prod.farcaster.email.amazon.dev (pdx2-ws-svc-p26-lb5-vlan2.pdx.amazon.com [10.39.38.66]) by email-inbound-relay-pdx-2b-m6i4x-a893d89c.us-west-2.amazon.com (Postfix) with ESMTPS id 05BA740D73; Wed, 8 Nov 2023 11:20:17 +0000 (UTC) Received: from EX19MTAEUB001.ant.amazon.com [10.0.17.79:7728] by smtpin.naws.eu-west-1.prod.farcaster.email.amazon.dev [10.0.33.209:2525] with esmtp (Farcaster) id 3cca8fad-d704-4370-b632-03b67dd48c0d; Wed, 8 Nov 2023 11:20:16 +0000 (UTC) X-Farcaster-Flow-ID: 3cca8fad-d704-4370-b632-03b67dd48c0d Received: from EX19D004EUC001.ant.amazon.com (10.252.51.190) by EX19MTAEUB001.ant.amazon.com (10.252.51.28) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.1118.39; Wed, 8 Nov 2023 11:20:16 +0000 Received: from dev-dsk-nsaenz-1b-189b39ae.eu-west-1.amazon.com (10.13.235.138) by EX19D004EUC001.ant.amazon.com (10.252.51.190) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.1118.39; Wed, 8 Nov 2023 11:20:11 +0000 From: Nicolas Saenz Julienne To: CC: , , , , , , , , , , , , , , , Nicolas Saenz Julienne Subject: [RFC 09/33] KVM: x86: hyper-v: Introduce per-VTL vcpu helpers Date: Wed, 8 Nov 2023 11:17:42 +0000 Message-ID: <20231108111806.92604-10-nsaenz@amazon.com> X-Mailer: git-send-email 2.40.1 In-Reply-To: <20231108111806.92604-1-nsaenz@amazon.com> References: <20231108111806.92604-1-nsaenz@amazon.com> Precedence: bulk X-Mailing-List: kvm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Originating-IP: [10.13.235.138] X-ClientProxiedBy: EX19D044UWB002.ant.amazon.com (10.13.139.188) To EX19D004EUC001.ant.amazon.com (10.252.51.190) Introduce two helper functions. The first one queries a vCPU's VTL level, the second one, given a struct kvm_vcpu and VTL pair, returns the corresponding 'sibling' struct kvm_vcpu at the right VTL. We keep track of each VTL's state by having a distinct struct kvm_vpcu for each level. VTL-vCPUs that belong to the same guest CPU share the same physical APIC id, but belong to different APIC groups where the apic group represents the vCPU's VTL. Signed-off-by: Nicolas Saenz Julienne --- arch/x86/kvm/hyperv.h | 18 ++++++++++++++++++ 1 file changed, 18 insertions(+) diff --git a/arch/x86/kvm/hyperv.h b/arch/x86/kvm/hyperv.h index 2bfed69ba0db..5433107e7cc8 100644 --- a/arch/x86/kvm/hyperv.h +++ b/arch/x86/kvm/hyperv.h @@ -23,6 +23,7 @@ #include #include "x86.h" +#include "lapic.h" /* "Hv#1" signature */ #define HYPERV_CPUID_SIGNATURE_EAX 0x31237648 @@ -83,6 +84,23 @@ static inline struct kvm_hv_syndbg *to_hv_syndbg(struct kvm_vcpu *vcpu) return &vcpu->kvm->arch.hyperv.hv_syndbg; } +static inline struct kvm_vcpu *kvm_hv_get_vtl_vcpu(struct kvm_vcpu *vcpu, int vtl) +{ + struct kvm *kvm = vcpu->kvm; + u32 target_id = kvm_apic_id(vcpu); + + kvm_apic_id_set_group(kvm, vtl, &target_id); + if (vcpu->vcpu_id == target_id) + return vcpu; + + return kvm_get_vcpu_by_id(kvm, target_id); +} + +static inline u8 kvm_hv_get_active_vtl(struct kvm_vcpu *vcpu) +{ + return kvm_apic_group(vcpu); +} + static inline u32 kvm_hv_get_vpindex(struct kvm_vcpu *vcpu) { struct kvm_vcpu_hv *hv_vcpu = to_hv_vcpu(vcpu); From patchwork Wed Nov 8 11:17:43 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Nicolas Saenz Julienne X-Patchwork-Id: 13449823 Received: from lindbergh.monkeyblade.net (lindbergh.monkeyblade.net [23.128.96.19]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 725C415E89; Wed, 8 Nov 2023 11:20:29 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=amazon.com header.i=@amazon.com header.b="aykVI1Qf" Received: from smtp-fw-52002.amazon.com (smtp-fw-52002.amazon.com [52.119.213.150]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 4CC5F101; Wed, 8 Nov 2023 03:20:28 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amazon.com; i=@amazon.com; q=dns/txt; s=amazon201209; t=1699442429; x=1730978429; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=O25EX7uu/BZNg+CriP/M93KalXQ52Vh7t09zkAv2Ya4=; b=aykVI1QfFrWWdgvl5K28nJc2jLfTCZz/bsoiUlASsTpFbkDS41nZfZr3 D6azOjVhkVl7UL9tqeYijznMZNP5k8HOiCYJHwm4bfyvTjk6jYm4M++87 qhmwsdlgVGNs+hOQearaS91Oy2ZVQ5ATF/zBhTpyNuvjCuCQE0ToZBBkX I=; X-IronPort-AV: E=Sophos;i="6.03,286,1694736000"; d="scan'208";a="593807472" Received: from iad12-co-svc-p1-lb1-vlan3.amazon.com (HELO email-inbound-relay-iad-1e-m6i4x-529f0975.us-east-1.amazon.com) ([10.43.8.6]) by smtp-border-fw-52002.iad7.amazon.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 08 Nov 2023 11:20:27 +0000 Received: from smtpout.prod.us-west-2.prod.farcaster.email.amazon.dev (iad7-ws-svc-p70-lb3-vlan3.iad.amazon.com [10.32.235.38]) by email-inbound-relay-iad-1e-m6i4x-529f0975.us-east-1.amazon.com (Postfix) with ESMTPS id 527E148E12; Wed, 8 Nov 2023 11:20:23 +0000 (UTC) Received: from EX19MTAEUB002.ant.amazon.com [10.0.17.79:53919] by smtpin.naws.eu-west-1.prod.farcaster.email.amazon.dev [10.0.33.209:2525] with esmtp (Farcaster) id 70e20ac2-f2b6-471e-8f0e-3ab2809c6acc; Wed, 8 Nov 2023 11:20:21 +0000 (UTC) X-Farcaster-Flow-ID: 70e20ac2-f2b6-471e-8f0e-3ab2809c6acc Received: from EX19D004EUC001.ant.amazon.com (10.252.51.190) by EX19MTAEUB002.ant.amazon.com (10.252.51.59) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.1118.39; Wed, 8 Nov 2023 11:20:21 +0000 Received: from dev-dsk-nsaenz-1b-189b39ae.eu-west-1.amazon.com (10.13.235.138) by EX19D004EUC001.ant.amazon.com (10.252.51.190) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.1118.39; Wed, 8 Nov 2023 11:20:16 +0000 From: Nicolas Saenz Julienne To: CC: , , , , , , , , , , , , , , , Nicolas Saenz Julienne Subject: [RFC 10/33] KVM: x86: hyper-v: Introduce KVM_HV_GET_VSM_STATE Date: Wed, 8 Nov 2023 11:17:43 +0000 Message-ID: <20231108111806.92604-11-nsaenz@amazon.com> X-Mailer: git-send-email 2.40.1 In-Reply-To: <20231108111806.92604-1-nsaenz@amazon.com> References: <20231108111806.92604-1-nsaenz@amazon.com> Precedence: bulk X-Mailing-List: kvm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Originating-IP: [10.13.235.138] X-ClientProxiedBy: EX19D044UWB002.ant.amazon.com (10.13.139.188) To EX19D004EUC001.ant.amazon.com (10.252.51.190) HVCALL_GET_VP_REGISTERS exposes the VTL call hypercall page entry offsets to the guest. This hypercall is implemented in user-space while the hypercall page patching happens in-kernel. So expose it as part of the partition wide VSM state. NOTE: Alternatively there is the option of sharing this information through a VTL KVM device attribute (the device is introduced in subsequent patches). Signed-off-by: Nicolas Saenz Julienne --- arch/x86/include/uapi/asm/kvm.h | 5 +++++ arch/x86/kvm/hyperv.c | 8 ++++++++ arch/x86/kvm/hyperv.h | 2 ++ arch/x86/kvm/x86.c | 18 ++++++++++++++++++ include/uapi/linux/kvm.h | 4 ++++ 5 files changed, 37 insertions(+) diff --git a/arch/x86/include/uapi/asm/kvm.h b/arch/x86/include/uapi/asm/kvm.h index f73d137784d7..370483d5d5fd 100644 --- a/arch/x86/include/uapi/asm/kvm.h +++ b/arch/x86/include/uapi/asm/kvm.h @@ -570,4 +570,9 @@ struct kvm_apic_id_groups { __u8 n_bits; /* nr of bits used to represent group in the APIC ID */ }; +/* for KVM_HV_GET_VSM_STATE */ +struct kvm_hv_vsm_state { + __u64 vsm_code_page_offsets; +}; + #endif /* _ASM_X86_KVM_H */ diff --git a/arch/x86/kvm/hyperv.c b/arch/x86/kvm/hyperv.c index 2cf430f6ddd8..caaa859932c5 100644 --- a/arch/x86/kvm/hyperv.c +++ b/arch/x86/kvm/hyperv.c @@ -2990,3 +2990,11 @@ int kvm_get_hv_cpuid(struct kvm_vcpu *vcpu, struct kvm_cpuid2 *cpuid, return 0; } + +int kvm_vm_ioctl_get_hv_vsm_state(struct kvm *kvm, struct kvm_hv_vsm_state *state) +{ + struct kvm_hv* hv = &kvm->arch.hyperv; + + state->vsm_code_page_offsets = hv->vsm_code_page_offsets.as_u64; + return 0; +} diff --git a/arch/x86/kvm/hyperv.h b/arch/x86/kvm/hyperv.h index 5433107e7cc8..b3d1113efe82 100644 --- a/arch/x86/kvm/hyperv.h +++ b/arch/x86/kvm/hyperv.h @@ -261,4 +261,6 @@ static inline bool kvm_hv_vsm_enabled(struct kvm *kvm) return kvm->arch.hyperv.hv_enable_vsm; } +int kvm_vm_ioctl_get_hv_vsm_state(struct kvm *kvm, struct kvm_hv_vsm_state *state); + #endif diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c index b0512e433032..57f9c58e1e32 100644 --- a/arch/x86/kvm/x86.c +++ b/arch/x86/kvm/x86.c @@ -7132,6 +7132,24 @@ int kvm_arch_vm_ioctl(struct file *filp, unsigned int ioctl, unsigned long arg) r = kvm_vm_ioctl_set_apic_id_groups(kvm, &groups); break; } + case KVM_HV_GET_VSM_STATE: { + struct kvm_hv_vsm_state vsm_state; + + r = -EINVAL; + if (!kvm_hv_vsm_enabled(kvm)) + goto out; + + r = kvm_vm_ioctl_get_hv_vsm_state(kvm, &vsm_state); + if (r) + goto out; + + r = -EFAULT; + if (copy_to_user(argp, &vsm_state, sizeof(vsm_state))) + goto out; + + r = 0; + break; + } default: r = -ENOTTY; } diff --git a/include/uapi/linux/kvm.h b/include/uapi/linux/kvm.h index 168b6ac6ebe5..03f5c08fd7aa 100644 --- a/include/uapi/linux/kvm.h +++ b/include/uapi/linux/kvm.h @@ -2316,4 +2316,8 @@ struct kvm_create_guest_memfd { #define KVM_GUEST_MEMFD_ALLOW_HUGEPAGE (1ULL << 0) #define KVM_SET_APIC_ID_GROUPS _IOW(KVMIO, 0xd7, struct kvm_apic_id_groups) + +/* Get/Set Hyper-V VSM state. Available with KVM_CAP_HYPERV_VSM */ +#define KVM_HV_GET_VSM_STATE _IOR(KVMIO, 0xd5, struct kvm_hv_vsm_state) + #endif /* __LINUX_KVM_H */ From patchwork Wed Nov 8 11:17:44 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Nicolas Saenz Julienne X-Patchwork-Id: 13449824 Received: from lindbergh.monkeyblade.net (lindbergh.monkeyblade.net [23.128.96.19]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id BB4F215E96; Wed, 8 Nov 2023 11:20:35 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=amazon.com header.i=@amazon.com header.b="B2oubJH9" Received: from smtp-fw-52003.amazon.com (smtp-fw-52003.amazon.com [52.119.213.152]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 08B351BEC; Wed, 8 Nov 2023 03:20:34 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amazon.com; i=@amazon.com; q=dns/txt; s=amazon201209; t=1699442436; x=1730978436; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=QBVqEnmtZNOjPktuGk9xsojMAUYhuWQTJQf/5jldNU4=; b=B2oubJH9/QQ5t5/Q0OeG0guV4eO6kmOmkBgbZF7D1k6BpayFGxHSYPhJ F8aitM+eksavt7Zw8G7NNTM3kwknGEnRgkGHE9KAL0EiWwkevxBpVLCEV etr9kWKoYI8QxvJhcB84JKM5YembBF0auUEjAUSHxKASziVievrPVAfPY 8=; X-IronPort-AV: E=Sophos;i="6.03,286,1694736000"; d="scan'208";a="618315974" Received: from iad12-co-svc-p1-lb1-vlan3.amazon.com (HELO email-inbound-relay-pdx-2c-m6i4x-f7c754c9.us-west-2.amazon.com) ([10.43.8.6]) by smtp-border-fw-52003.iad7.amazon.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 08 Nov 2023 11:20:32 +0000 Received: from smtpout.prod.us-west-2.prod.farcaster.email.amazon.dev (pdx2-ws-svc-p26-lb5-vlan3.pdx.amazon.com [10.39.38.70]) by email-inbound-relay-pdx-2c-m6i4x-f7c754c9.us-west-2.amazon.com (Postfix) with ESMTPS id B84FB40D92; Wed, 8 Nov 2023 11:20:29 +0000 (UTC) Received: from EX19MTAEUC002.ant.amazon.com [10.0.43.254:62810] by smtpin.naws.eu-west-1.prod.farcaster.email.amazon.dev [10.0.4.34:2525] with esmtp (Farcaster) id 3ff2c4a7-a3e7-46e0-936d-e9e005f722e6; Wed, 8 Nov 2023 11:20:28 +0000 (UTC) X-Farcaster-Flow-ID: 3ff2c4a7-a3e7-46e0-936d-e9e005f722e6 Received: from EX19D004EUC001.ant.amazon.com (10.252.51.190) by EX19MTAEUC002.ant.amazon.com (10.252.51.181) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.1118.39; Wed, 8 Nov 2023 11:20:26 +0000 Received: from dev-dsk-nsaenz-1b-189b39ae.eu-west-1.amazon.com (10.13.235.138) by EX19D004EUC001.ant.amazon.com (10.252.51.190) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.1118.39; Wed, 8 Nov 2023 11:20:21 +0000 From: Nicolas Saenz Julienne To: CC: , , , , , , , , , , , , , , , Nicolas Saenz Julienne Subject: [RFC 11/33] KVM: x86: hyper-v: Handle GET/SET_VP_REGISTER hcall in user-space Date: Wed, 8 Nov 2023 11:17:44 +0000 Message-ID: <20231108111806.92604-12-nsaenz@amazon.com> X-Mailer: git-send-email 2.40.1 In-Reply-To: <20231108111806.92604-1-nsaenz@amazon.com> References: <20231108111806.92604-1-nsaenz@amazon.com> Precedence: bulk X-Mailing-List: kvm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Originating-IP: [10.13.235.138] X-ClientProxiedBy: EX19D044UWB002.ant.amazon.com (10.13.139.188) To EX19D004EUC001.ant.amazon.com (10.252.51.190) Let user-space handle HVCALL_GET_VP_REGISTERS and HVCALL_SET_VP_REGISTERS through the KVM_EXIT_HYPERV_HVCALL exit reason. Additionally, expose the cpuid bit. Signed-off-by: Nicolas Saenz Julienne --- arch/x86/kvm/hyperv.c | 9 +++++++++ include/asm-generic/hyperv-tlfs.h | 1 + 2 files changed, 10 insertions(+) diff --git a/arch/x86/kvm/hyperv.c b/arch/x86/kvm/hyperv.c index caaa859932c5..a3970d52eef1 100644 --- a/arch/x86/kvm/hyperv.c +++ b/arch/x86/kvm/hyperv.c @@ -2456,6 +2456,9 @@ static void kvm_hv_write_xmm(struct kvm_hyperv_xmm_reg *xmm) static bool kvm_hv_is_xmm_output_hcall(u16 code) { + if (code == HVCALL_GET_VP_REGISTERS) + return true; + return false; } @@ -2520,6 +2523,8 @@ static bool is_xmm_fast_hypercall(struct kvm_hv_hcall *hc) case HVCALL_FLUSH_VIRTUAL_ADDRESS_LIST_EX: case HVCALL_FLUSH_VIRTUAL_ADDRESS_SPACE_EX: case HVCALL_SEND_IPI_EX: + case HVCALL_GET_VP_REGISTERS: + case HVCALL_SET_VP_REGISTERS: return true; } @@ -2738,6 +2743,9 @@ int kvm_hv_hypercall(struct kvm_vcpu *vcpu) break; } goto hypercall_userspace_exit; + case HVCALL_GET_VP_REGISTERS: + case HVCALL_SET_VP_REGISTERS: + goto hypercall_userspace_exit; default: ret = HV_STATUS_INVALID_HYPERCALL_CODE; break; @@ -2903,6 +2911,7 @@ int kvm_get_hv_cpuid(struct kvm_vcpu *vcpu, struct kvm_cpuid2 *cpuid, ent->ebx |= HV_POST_MESSAGES; ent->ebx |= HV_SIGNAL_EVENTS; ent->ebx |= HV_ENABLE_EXTENDED_HYPERCALLS; + ent->ebx |= HV_ACCESS_VP_REGISTERS; ent->edx |= HV_X64_HYPERCALL_XMM_INPUT_AVAILABLE; ent->edx |= HV_X64_HYPERCALL_XMM_OUTPUT_AVAILABLE; diff --git a/include/asm-generic/hyperv-tlfs.h b/include/asm-generic/hyperv-tlfs.h index 40d7dc793c03..24ea699a3d8e 100644 --- a/include/asm-generic/hyperv-tlfs.h +++ b/include/asm-generic/hyperv-tlfs.h @@ -89,6 +89,7 @@ #define HV_ACCESS_STATS BIT(8) #define HV_DEBUGGING BIT(11) #define HV_CPU_MANAGEMENT BIT(12) +#define HV_ACCESS_VP_REGISTERS BIT(17) #define HV_ENABLE_EXTENDED_HYPERCALLS BIT(20) #define HV_ISOLATION BIT(22) From patchwork Wed Nov 8 11:17:45 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Nicolas Saenz Julienne X-Patchwork-Id: 13449825 Received: from lindbergh.monkeyblade.net (lindbergh.monkeyblade.net [23.128.96.19]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 762C315E89; Wed, 8 Nov 2023 11:21:04 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=amazon.com header.i=@amazon.com header.b="PBBdMmlK" Received: from smtp-fw-80006.amazon.com (smtp-fw-80006.amazon.com [99.78.197.217]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id DDC0B1BF5; Wed, 8 Nov 2023 03:21:03 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amazon.com; i=@amazon.com; q=dns/txt; s=amazon201209; t=1699442464; x=1730978464; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=7C45WW2plKc9LGS1hg04kUsdvaSyLe7Yr2kIsIf6Lys=; b=PBBdMmlKTVNOph/CauxitBiIZC2KKZDRZoDPYrP9M4s46/tyZCXrGesw EopIKJhR5etsB50qYBfKZhCn3BEkZk7/oahfr0RJhI3xKx8AZ9wy5DxYs QJ1k7B0GZJsSKlIGCX3MXcBrNLAyqtfsi4dGzUoeQB+V5Wi9flFULTzz6 Y=; X-IronPort-AV: E=Sophos;i="6.03,286,1694736000"; d="scan'208";a="250876207" Received: from pdx4-co-svc-p1-lb2-vlan3.amazon.com (HELO email-inbound-relay-iad-1a-m6i4x-edda28d4.us-east-1.amazon.com) ([10.25.36.214]) by smtp-border-fw-80006.pdx80.corp.amazon.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 08 Nov 2023 11:21:01 +0000 Received: from smtpout.prod.us-west-2.prod.farcaster.email.amazon.dev (iad7-ws-svc-p70-lb3-vlan3.iad.amazon.com [10.32.235.38]) by email-inbound-relay-iad-1a-m6i4x-edda28d4.us-east-1.amazon.com (Postfix) with ESMTPS id 11B3F806CD; Wed, 8 Nov 2023 11:20:56 +0000 (UTC) Received: from EX19MTAEUA001.ant.amazon.com [10.0.17.79:3212] by smtpin.naws.eu-west-1.prod.farcaster.email.amazon.dev [10.0.32.187:2525] with esmtp (Farcaster) id 71b58972-fe97-42fd-b068-56f96eb17f93; Wed, 8 Nov 2023 11:20:55 +0000 (UTC) X-Farcaster-Flow-ID: 71b58972-fe97-42fd-b068-56f96eb17f93 Received: from EX19D004EUC001.ant.amazon.com (10.252.51.190) by EX19MTAEUA001.ant.amazon.com (10.252.50.192) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.1118.39; Wed, 8 Nov 2023 11:20:55 +0000 Received: from dev-dsk-nsaenz-1b-189b39ae.eu-west-1.amazon.com (10.13.235.138) by EX19D004EUC001.ant.amazon.com (10.252.51.190) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.1118.39; Wed, 8 Nov 2023 11:20:50 +0000 From: Nicolas Saenz Julienne To: CC: , , , , , , , , , , , , , , , Nicolas Saenz Julienne Subject: [RFC 12/33] KVM: x86: hyper-v: Handle VSM hcalls in user-space Date: Wed, 8 Nov 2023 11:17:45 +0000 Message-ID: <20231108111806.92604-13-nsaenz@amazon.com> X-Mailer: git-send-email 2.40.1 In-Reply-To: <20231108111806.92604-1-nsaenz@amazon.com> References: <20231108111806.92604-1-nsaenz@amazon.com> Precedence: bulk X-Mailing-List: kvm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Originating-IP: [10.13.235.138] X-ClientProxiedBy: EX19D040UWA001.ant.amazon.com (10.13.139.22) To EX19D004EUC001.ant.amazon.com (10.252.51.190) Let user-space handle all hypercalls that fall under the AccessVsm partition privilege flag. That is: - HVCALL_MODIFY_VTL_PROTECTION_MASK: - HVCALL_ENABLE_PARTITION_VTL: - HVCALL_ENABLE_VP_VTL: - HVCALL_VTL_CALL: - HVCALL_VTL_RETURN: The hypercalls are processed through the KVM_EXIT_HYPERV_HVCALL exit. Additionally, expose the cpuid bit. Signed-off-by: Nicolas Saenz Julienne --- arch/x86/kvm/hyperv.c | 15 +++++++++++++++ include/asm-generic/hyperv-tlfs.h | 7 ++++++- 2 files changed, 21 insertions(+), 1 deletion(-) diff --git a/arch/x86/kvm/hyperv.c b/arch/x86/kvm/hyperv.c index a3970d52eef1..a266c5d393f5 100644 --- a/arch/x86/kvm/hyperv.c +++ b/arch/x86/kvm/hyperv.c @@ -2462,6 +2462,11 @@ static bool kvm_hv_is_xmm_output_hcall(u16 code) return false; } +static inline bool kvm_hv_is_vtl_call_return(u16 code) +{ + return code == HVCALL_VTL_CALL || code == HVCALL_VTL_RETURN; +} + static int kvm_hv_hypercall_complete_userspace(struct kvm_vcpu *vcpu) { bool fast = !!(vcpu->run->hyperv.u.hcall.input & HV_HYPERCALL_FAST_BIT); @@ -2471,6 +2476,9 @@ static int kvm_hv_hypercall_complete_userspace(struct kvm_vcpu *vcpu) if (kvm_hv_is_xmm_output_hcall(code) && hv_result_success(result) && fast) kvm_hv_write_xmm(vcpu->run->hyperv.u.hcall.xmm); + if (kvm_hv_is_vtl_call_return(code)) + return kvm_skip_emulated_instruction(vcpu); + return kvm_hv_hypercall_complete(vcpu, result); } @@ -2525,6 +2533,7 @@ static bool is_xmm_fast_hypercall(struct kvm_hv_hcall *hc) case HVCALL_SEND_IPI_EX: case HVCALL_GET_VP_REGISTERS: case HVCALL_SET_VP_REGISTERS: + case HVCALL_MODIFY_VTL_PROTECTION_MASK: return true; } @@ -2745,6 +2754,11 @@ int kvm_hv_hypercall(struct kvm_vcpu *vcpu) goto hypercall_userspace_exit; case HVCALL_GET_VP_REGISTERS: case HVCALL_SET_VP_REGISTERS: + case HVCALL_MODIFY_VTL_PROTECTION_MASK: + case HVCALL_ENABLE_PARTITION_VTL: + case HVCALL_ENABLE_VP_VTL: + case HVCALL_VTL_CALL: + case HVCALL_VTL_RETURN: goto hypercall_userspace_exit; default: ret = HV_STATUS_INVALID_HYPERCALL_CODE; @@ -2912,6 +2926,7 @@ int kvm_get_hv_cpuid(struct kvm_vcpu *vcpu, struct kvm_cpuid2 *cpuid, ent->ebx |= HV_SIGNAL_EVENTS; ent->ebx |= HV_ENABLE_EXTENDED_HYPERCALLS; ent->ebx |= HV_ACCESS_VP_REGISTERS; + ent->ebx |= HV_ACCESS_VSM; ent->edx |= HV_X64_HYPERCALL_XMM_INPUT_AVAILABLE; ent->edx |= HV_X64_HYPERCALL_XMM_OUTPUT_AVAILABLE; diff --git a/include/asm-generic/hyperv-tlfs.h b/include/asm-generic/hyperv-tlfs.h index 24ea699a3d8e..a8b5c8a84bbc 100644 --- a/include/asm-generic/hyperv-tlfs.h +++ b/include/asm-generic/hyperv-tlfs.h @@ -89,6 +89,7 @@ #define HV_ACCESS_STATS BIT(8) #define HV_DEBUGGING BIT(11) #define HV_CPU_MANAGEMENT BIT(12) +#define HV_ACCESS_VSM BIT(16) #define HV_ACCESS_VP_REGISTERS BIT(17) #define HV_ENABLE_EXTENDED_HYPERCALLS BIT(20) #define HV_ISOLATION BIT(22) @@ -147,9 +148,13 @@ union hv_reference_tsc_msr { /* Declare the various hypercall operations. */ #define HVCALL_FLUSH_VIRTUAL_ADDRESS_SPACE 0x0002 #define HVCALL_FLUSH_VIRTUAL_ADDRESS_LIST 0x0003 -#define HVCALL_ENABLE_VP_VTL 0x000f #define HVCALL_NOTIFY_LONG_SPIN_WAIT 0x0008 #define HVCALL_SEND_IPI 0x000b +#define HVCALL_MODIFY_VTL_PROTECTION_MASK 0x000c +#define HVCALL_ENABLE_PARTITION_VTL 0x000d +#define HVCALL_ENABLE_VP_VTL 0x000f +#define HVCALL_VTL_CALL 0x0011 +#define HVCALL_VTL_RETURN 0x0012 #define HVCALL_FLUSH_VIRTUAL_ADDRESS_SPACE_EX 0x0013 #define HVCALL_FLUSH_VIRTUAL_ADDRESS_LIST_EX 0x0014 #define HVCALL_SEND_IPI_EX 0x0015 From patchwork Wed Nov 8 11:17:46 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Nicolas Saenz Julienne X-Patchwork-Id: 13449827 Received: from lindbergh.monkeyblade.net (lindbergh.monkeyblade.net [23.128.96.19]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 54ACF15E96; Wed, 8 Nov 2023 11:21:16 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=amazon.com header.i=@amazon.com header.b="oopPMrgK" Received: from smtp-fw-52003.amazon.com (smtp-fw-52003.amazon.com [52.119.213.152]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 4FBE01BE7; Wed, 8 Nov 2023 03:21:15 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amazon.com; i=@amazon.com; q=dns/txt; s=amazon201209; t=1699442476; x=1730978476; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=WNkxJnNlKH45HLgy7MuxT0PXy6Pk8Rwqfkx/NFZb78M=; b=oopPMrgK3huNH+S58PID/PJGmaZuTodV0aPjByLPAEF84pnf67QPYUyS e2U3hibcw/NpOZdGuV6JX4r0FkStxP/wdkDa+cAJsbnsk71MQuiA5Iqhx r8+EiaDXiDXcjOWtGkKmnbmHTK7aoGUXnZFEehEMICMccuKFru1ijXDmK o=; X-IronPort-AV: E=Sophos;i="6.03,286,1694736000"; d="scan'208";a="618316123" Received: from iad12-co-svc-p1-lb1-vlan3.amazon.com (HELO email-inbound-relay-iad-1d-m6i4x-b404fda3.us-east-1.amazon.com) ([10.43.8.6]) by smtp-border-fw-52003.iad7.amazon.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 08 Nov 2023 11:21:15 +0000 Received: from smtpout.prod.us-west-2.prod.farcaster.email.amazon.dev (iad7-ws-svc-p70-lb3-vlan3.iad.amazon.com [10.32.235.38]) by email-inbound-relay-iad-1d-m6i4x-b404fda3.us-east-1.amazon.com (Postfix) with ESMTPS id CC42F80587; Wed, 8 Nov 2023 11:21:02 +0000 (UTC) Received: from EX19MTAEUA002.ant.amazon.com [10.0.17.79:44839] by smtpin.naws.eu-west-1.prod.farcaster.email.amazon.dev [10.0.22.222:2525] with esmtp (Farcaster) id 18ea7cc4-781f-406b-ba74-c06699172525; Wed, 8 Nov 2023 11:21:00 +0000 (UTC) X-Farcaster-Flow-ID: 18ea7cc4-781f-406b-ba74-c06699172525 Received: from EX19D004EUC001.ant.amazon.com (10.252.51.190) by EX19MTAEUA002.ant.amazon.com (10.252.50.124) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.1118.39; Wed, 8 Nov 2023 11:21:00 +0000 Received: from dev-dsk-nsaenz-1b-189b39ae.eu-west-1.amazon.com (10.13.235.138) by EX19D004EUC001.ant.amazon.com (10.252.51.190) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.1118.39; Wed, 8 Nov 2023 11:20:55 +0000 From: Nicolas Saenz Julienne To: CC: , , , , , , , , , , , , , , , Nicolas Saenz Julienne Subject: [RFC 13/33] KVM: Allow polling vCPUs for events Date: Wed, 8 Nov 2023 11:17:46 +0000 Message-ID: <20231108111806.92604-14-nsaenz@amazon.com> X-Mailer: git-send-email 2.40.1 In-Reply-To: <20231108111806.92604-1-nsaenz@amazon.com> References: <20231108111806.92604-1-nsaenz@amazon.com> Precedence: bulk X-Mailing-List: kvm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Originating-IP: [10.13.235.138] X-ClientProxiedBy: EX19D040UWA001.ant.amazon.com (10.13.139.22) To EX19D004EUC001.ant.amazon.com (10.252.51.190) A number of use cases have surfaced where it'd be beneficial to have a vCPU stop its execution in user-space, as opposed to having it sleep in-kernel. Be it in order to make better use of the pCPU's time while the vCPU is halted, or to implement security features like Hyper-V's VSM. A problem with this approach is that user-space has no way of knowing whether the vCPU has pending events (interrupts, timers, etc...), so we need a new interface to query if they are. poll() turned out to be a very good fit. So enable polling vCPUs. The poll() interface considers a vCPU has a pending event if it didn't enter the guest since being kicked by an event source (being kicked forces a guest exit). Kicking a vCPU that has pollers wakes up the polling threads. NOTES: - There is a race between the 'vcpu->kicked' check in the polling thread and the vCPU thread re-entering the guest. This hardly affects the use-cases stated above, but needs to be fixed. - This was tested alongside a WIP Hyper-V Virtual Trust Level implementation which makes ample use of the poll() interface. Signed-off-by: Nicolas Saenz Julienne --- arch/x86/kvm/x86.c | 2 ++ include/linux/kvm_host.h | 2 ++ virt/kvm/kvm_main.c | 30 ++++++++++++++++++++++++++++++ 3 files changed, 34 insertions(+) diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c index 57f9c58e1e32..bf4891bc044e 100644 --- a/arch/x86/kvm/x86.c +++ b/arch/x86/kvm/x86.c @@ -10788,6 +10788,8 @@ static int vcpu_enter_guest(struct kvm_vcpu *vcpu) goto cancel_injection; } + WRITE_ONCE(vcpu->kicked, false); + if (req_immediate_exit) { kvm_make_request(KVM_REQ_EVENT, vcpu); static_call(kvm_x86_request_immediate_exit)(vcpu); diff --git a/include/linux/kvm_host.h b/include/linux/kvm_host.h index 687589ce9f63..71e1e8cf8936 100644 --- a/include/linux/kvm_host.h +++ b/include/linux/kvm_host.h @@ -336,6 +336,7 @@ struct kvm_vcpu { #endif int mode; u64 requests; + bool kicked; unsigned long guest_debug; struct mutex mutex; @@ -395,6 +396,7 @@ struct kvm_vcpu { */ struct kvm_memory_slot *last_used_slot; u64 last_used_slot_gen; + wait_queue_head_t wqh; }; /* diff --git a/virt/kvm/kvm_main.c b/virt/kvm/kvm_main.c index ad9aab898a0c..fde004a0ac46 100644 --- a/virt/kvm/kvm_main.c +++ b/virt/kvm/kvm_main.c @@ -497,12 +497,14 @@ static void kvm_vcpu_init(struct kvm_vcpu *vcpu, struct kvm *kvm, unsigned id) kvm_vcpu_set_dy_eligible(vcpu, false); vcpu->preempted = false; vcpu->ready = false; + vcpu->kicked = false; preempt_notifier_init(&vcpu->preempt_notifier, &kvm_preempt_ops); vcpu->last_used_slot = NULL; /* Fill the stats id string for the vcpu */ snprintf(vcpu->stats_id, sizeof(vcpu->stats_id), "kvm-%d/vcpu-%d", task_pid_nr(current), id); + init_waitqueue_head(&vcpu->wqh); } static void kvm_vcpu_destroy(struct kvm_vcpu *vcpu) @@ -3970,6 +3972,10 @@ void kvm_vcpu_kick(struct kvm_vcpu *vcpu) if (cpu != me && (unsigned)cpu < nr_cpu_ids && cpu_online(cpu)) smp_send_reschedule(cpu); } + + if (!cmpxchg(&vcpu->kicked, false, true)) + wake_up_interruptible(&vcpu->wqh); + out: put_cpu(); } @@ -4174,6 +4180,29 @@ static int kvm_vcpu_mmap(struct file *file, struct vm_area_struct *vma) return 0; } +static __poll_t kvm_vcpu_poll(struct file *file, poll_table *wait) +{ + struct kvm_vcpu *vcpu = file->private_data; + + poll_wait(file, &vcpu->wqh, wait); + + /* + * Make sure we read vcpu->kicked after adding the vcpu into + * the waitqueue list. Otherwise we might have the following race: + * + * READ_ONCE(vcpu->kicked) + * cmpxchg(&vcpu->kicked, false, true)) + * wake_up_interruptible(&vcpu->wqh) + * list_add_tail(wait, &vcpu->wqh) + */ + smp_mb(); + if (READ_ONCE(vcpu->kicked)) { + return EPOLLIN; + } + + return 0; +} + static int kvm_vcpu_release(struct inode *inode, struct file *filp) { struct kvm_vcpu *vcpu = filp->private_data; @@ -4186,6 +4215,7 @@ static const struct file_operations kvm_vcpu_fops = { .release = kvm_vcpu_release, .unlocked_ioctl = kvm_vcpu_ioctl, .mmap = kvm_vcpu_mmap, + .poll = kvm_vcpu_poll, .llseek = noop_llseek, KVM_COMPAT(kvm_vcpu_compat_ioctl), }; From patchwork Wed Nov 8 11:17:47 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Nicolas Saenz Julienne X-Patchwork-Id: 13449826 Received: from lindbergh.monkeyblade.net (lindbergh.monkeyblade.net [23.128.96.19]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id E818A15E89; Wed, 8 Nov 2023 11:21:11 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=amazon.com header.i=@amazon.com header.b="tTG2Z5me" Received: from smtp-fw-6002.amazon.com (smtp-fw-6002.amazon.com [52.95.49.90]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id E31B3181; Wed, 8 Nov 2023 03:21:10 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amazon.com; i=@amazon.com; q=dns/txt; s=amazon201209; t=1699442471; x=1730978471; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=hsnpzZHwTij1DO+oBadMSuiUb8OXWa0py8QnRL9ObtU=; b=tTG2Z5meFI6gLZozCnUrMYh6fXI1igfyppPiCfK8rMdXuUfJ+cr6P9du p2bla8f9VUANUldCb68SC8CYgEXpujR8nDAALwrnErClO3UmfojOKzt7q EDO9LYKnMuJXcdZGtzE6Mzh57LsCgt3SbP+dtj7GOaSJvkbzWPyC2WCv4 M=; X-IronPort-AV: E=Sophos;i="6.03,286,1694736000"; d="scan'208";a="366812557" Received: from iad12-co-svc-p1-lb1-vlan3.amazon.com (HELO email-inbound-relay-iad-1a-m6i4x-54a853e6.us-east-1.amazon.com) ([10.43.8.6]) by smtp-border-fw-6002.iad6.amazon.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 08 Nov 2023 11:21:10 +0000 Received: from smtpout.prod.us-east-1.prod.farcaster.email.amazon.dev (iad7-ws-svc-p70-lb3-vlan3.iad.amazon.com [10.32.235.38]) by email-inbound-relay-iad-1a-m6i4x-54a853e6.us-east-1.amazon.com (Postfix) with ESMTPS id 3191748ED2; Wed, 8 Nov 2023 11:21:06 +0000 (UTC) Received: from EX19MTAEUA002.ant.amazon.com [10.0.17.79:8159] by smtpin.naws.eu-west-1.prod.farcaster.email.amazon.dev [10.0.26.101:2525] with esmtp (Farcaster) id b036e4e3-6edd-4dbe-817d-58a60e75dd60; Wed, 8 Nov 2023 11:21:06 +0000 (UTC) X-Farcaster-Flow-ID: b036e4e3-6edd-4dbe-817d-58a60e75dd60 Received: from EX19D004EUC001.ant.amazon.com (10.252.51.190) by EX19MTAEUA002.ant.amazon.com (10.252.50.124) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.1118.39; Wed, 8 Nov 2023 11:21:05 +0000 Received: from dev-dsk-nsaenz-1b-189b39ae.eu-west-1.amazon.com (10.13.235.138) by EX19D004EUC001.ant.amazon.com (10.252.51.190) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.1118.39; Wed, 8 Nov 2023 11:21:00 +0000 From: Nicolas Saenz Julienne To: CC: , , , , , , , , , , , , , , , Nicolas Saenz Julienne Subject: [RFC 14/33] KVM: x86: Add VTL to the MMU role Date: Wed, 8 Nov 2023 11:17:47 +0000 Message-ID: <20231108111806.92604-15-nsaenz@amazon.com> X-Mailer: git-send-email 2.40.1 In-Reply-To: <20231108111806.92604-1-nsaenz@amazon.com> References: <20231108111806.92604-1-nsaenz@amazon.com> Precedence: bulk X-Mailing-List: kvm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Originating-IP: [10.13.235.138] X-ClientProxiedBy: EX19D040UWA001.ant.amazon.com (10.13.139.22) To EX19D004EUC001.ant.amazon.com (10.252.51.190) With the upcoming introduction of per-VTL memory protections, make MMU roles VTL aware. This will avoid sharing PTEs between vCPUs that belong to different VTLs, and that have distinct memory access restrictions. Four bits are allocated to store the VTL number in the MMU role, since the TLFS states there is a maximum of 16 levels. Signed-off-by: Nicolas Saenz Julienne --- arch/x86/include/asm/kvm_host.h | 3 ++- arch/x86/kvm/hyperv.h | 6 ++++++ arch/x86/kvm/mmu.h | 1 + arch/x86/kvm/mmu/mmu.c | 3 +++ 4 files changed, 12 insertions(+), 1 deletion(-) diff --git a/arch/x86/include/asm/kvm_host.h b/arch/x86/include/asm/kvm_host.h index 7712e31b7537..1f5a85d461ce 100644 --- a/arch/x86/include/asm/kvm_host.h +++ b/arch/x86/include/asm/kvm_host.h @@ -338,7 +338,8 @@ union kvm_mmu_page_role { unsigned ad_disabled:1; unsigned guest_mode:1; unsigned passthrough:1; - unsigned :5; + unsigned vtl:4; + unsigned :1; /* * This is left at the top of the word so that diff --git a/arch/x86/kvm/hyperv.h b/arch/x86/kvm/hyperv.h index b3d1113efe82..605e80b9e5eb 100644 --- a/arch/x86/kvm/hyperv.h +++ b/arch/x86/kvm/hyperv.h @@ -263,4 +263,10 @@ static inline bool kvm_hv_vsm_enabled(struct kvm *kvm) int kvm_vm_ioctl_get_hv_vsm_state(struct kvm *kvm, struct kvm_hv_vsm_state *state); +static inline void kvm_mmu_role_set_hv_bits(struct kvm_vcpu *vcpu, + union kvm_mmu_page_role *role) +{ + role->vtl = kvm_hv_get_active_vtl(vcpu); +} + #endif diff --git a/arch/x86/kvm/mmu.h b/arch/x86/kvm/mmu.h index 253fb2093d5d..e170388c6da1 100644 --- a/arch/x86/kvm/mmu.h +++ b/arch/x86/kvm/mmu.h @@ -304,4 +304,5 @@ static inline gpa_t kvm_translate_gpa(struct kvm_vcpu *vcpu, return gpa; return translate_nested_gpa(vcpu, gpa, access, exception); } + #endif diff --git a/arch/x86/kvm/mmu/mmu.c b/arch/x86/kvm/mmu/mmu.c index baeba8fc1c38..2afef86863fb 100644 --- a/arch/x86/kvm/mmu/mmu.c +++ b/arch/x86/kvm/mmu/mmu.c @@ -28,6 +28,7 @@ #include "page_track.h" #include "cpuid.h" #include "spte.h" +#include "hyperv.h" #include #include @@ -5197,6 +5198,7 @@ static union kvm_cpu_role kvm_calc_cpu_role(struct kvm_vcpu *vcpu, role.base.smm = is_smm(vcpu); role.base.guest_mode = is_guest_mode(vcpu); role.ext.valid = 1; + kvm_mmu_role_set_hv_bits(vcpu, &role.base); if (!____is_cr0_pg(regs)) { role.base.direct = 1; @@ -5271,6 +5273,7 @@ kvm_calc_tdp_mmu_root_page_role(struct kvm_vcpu *vcpu, role.level = kvm_mmu_get_tdp_level(vcpu); role.direct = true; role.has_4_byte_gpte = false; + kvm_mmu_role_set_hv_bits(vcpu, &role); return role; } From patchwork Wed Nov 8 11:17:48 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Nicolas Saenz Julienne X-Patchwork-Id: 13449828 Received: from lindbergh.monkeyblade.net (lindbergh.monkeyblade.net [23.128.96.19]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id E0EA315E89; Wed, 8 Nov 2023 11:21:47 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=amazon.com header.i=@amazon.com header.b="sNm8tk4b" Received: from smtp-fw-9102.amazon.com (smtp-fw-9102.amazon.com [207.171.184.29]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 55A721BE1; Wed, 8 Nov 2023 03:21:47 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amazon.com; i=@amazon.com; q=dns/txt; s=amazon201209; t=1699442507; x=1730978507; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=iycFoTQhbiLFSko2vQExEp1NsC8B9ilkhIKK0YicW+o=; b=sNm8tk4bdiSv9RcK/p31fYA5P8oPP9zXu7plXO17n8vzt3AEhJxPRgaG HpxMQtnznovUjHqlU4klh8y3Yh5f0cMHfyTFUPpGHvsH5W41JUc8qY1Qf SsuE9Ar5H845RLHucxk3HM+TLuFPWUIMhBV/09vgIKnWKDEPWauz9kkEb k=; X-IronPort-AV: E=Sophos;i="6.03,286,1694736000"; d="scan'208";a="375131996" Received: from pdx4-co-svc-p1-lb2-vlan3.amazon.com (HELO email-inbound-relay-iad-1e-m6i4x-529f0975.us-east-1.amazon.com) ([10.25.36.214]) by smtp-border-fw-9102.sea19.amazon.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 08 Nov 2023 11:21:40 +0000 Received: from smtpout.prod.us-east-1.prod.farcaster.email.amazon.dev (iad7-ws-svc-p70-lb3-vlan3.iad.amazon.com [10.32.235.38]) by email-inbound-relay-iad-1e-m6i4x-529f0975.us-east-1.amazon.com (Postfix) with ESMTPS id BA00148E12; Wed, 8 Nov 2023 11:21:36 +0000 (UTC) Received: from EX19MTAEUC002.ant.amazon.com [10.0.17.79:28792] by smtpin.naws.eu-west-1.prod.farcaster.email.amazon.dev [10.0.4.34:2525] with esmtp (Farcaster) id cc98cadf-a052-4f23-b472-377eb7ac99f7; Wed, 8 Nov 2023 11:21:35 +0000 (UTC) X-Farcaster-Flow-ID: cc98cadf-a052-4f23-b472-377eb7ac99f7 Received: from EX19D004EUC001.ant.amazon.com (10.252.51.190) by EX19MTAEUC002.ant.amazon.com (10.252.51.181) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.1118.39; Wed, 8 Nov 2023 11:21:34 +0000 Received: from dev-dsk-nsaenz-1b-189b39ae.eu-west-1.amazon.com (10.13.235.138) by EX19D004EUC001.ant.amazon.com (10.252.51.190) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.1118.39; Wed, 8 Nov 2023 11:21:29 +0000 From: Nicolas Saenz Julienne To: CC: , , , , , , , , , , , , , , , Nicolas Saenz Julienne Subject: [RFC 15/33] KVM: x86/mmu: Introduce infrastructure to handle non-executable faults Date: Wed, 8 Nov 2023 11:17:48 +0000 Message-ID: <20231108111806.92604-16-nsaenz@amazon.com> X-Mailer: git-send-email 2.40.1 In-Reply-To: <20231108111806.92604-1-nsaenz@amazon.com> References: <20231108111806.92604-1-nsaenz@amazon.com> Precedence: bulk X-Mailing-List: kvm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Originating-IP: [10.13.235.138] X-ClientProxiedBy: EX19D043UWA003.ant.amazon.com (10.13.139.31) To EX19D004EUC001.ant.amazon.com (10.252.51.190) The upcoming per-VTL memory protections support needs to fault in non-executable memory. Introduce a new attribute in struct kvm_page_fault, map_executable, to control whether the gfn range should be mapped as executable. No functional change intended. Signed-off-by: Nicolas Saenz Julienne --- arch/x86/kvm/mmu/mmu.c | 6 +++++- arch/x86/kvm/mmu/mmu_internal.h | 2 ++ arch/x86/kvm/mmu/tdp_mmu.c | 8 ++++++-- 3 files changed, 13 insertions(+), 3 deletions(-) diff --git a/arch/x86/kvm/mmu/mmu.c b/arch/x86/kvm/mmu/mmu.c index 2afef86863fb..4e02d506cc25 100644 --- a/arch/x86/kvm/mmu/mmu.c +++ b/arch/x86/kvm/mmu/mmu.c @@ -3245,6 +3245,7 @@ static int direct_map(struct kvm_vcpu *vcpu, struct kvm_page_fault *fault) struct kvm_mmu_page *sp; int ret; gfn_t base_gfn = fault->gfn; + unsigned access = ACC_ALL; kvm_mmu_hugepage_adjust(vcpu, fault); @@ -3274,7 +3275,10 @@ static int direct_map(struct kvm_vcpu *vcpu, struct kvm_page_fault *fault) if (WARN_ON_ONCE(it.level != fault->goal_level)) return -EFAULT; - ret = mmu_set_spte(vcpu, fault->slot, it.sptep, ACC_ALL, + if (!fault->map_executable) + access &= ~ACC_EXEC_MASK; + + ret = mmu_set_spte(vcpu, fault->slot, it.sptep, access, base_gfn, fault->pfn, fault); if (ret == RET_PF_SPURIOUS) return ret; diff --git a/arch/x86/kvm/mmu/mmu_internal.h b/arch/x86/kvm/mmu/mmu_internal.h index b66a7d47e0e4..bd62c4d5d5f1 100644 --- a/arch/x86/kvm/mmu/mmu_internal.h +++ b/arch/x86/kvm/mmu/mmu_internal.h @@ -239,6 +239,7 @@ struct kvm_page_fault { kvm_pfn_t pfn; hva_t hva; bool map_writable; + bool map_executable; /* * Indicates the guest is trying to write a gfn that contains one or @@ -298,6 +299,7 @@ static inline int kvm_mmu_do_page_fault(struct kvm_vcpu *vcpu, gpa_t cr2_or_gpa, .req_level = PG_LEVEL_4K, .goal_level = PG_LEVEL_4K, .is_private = kvm_mem_is_private(vcpu->kvm, cr2_or_gpa >> PAGE_SHIFT), + .map_executable = true, }; int r; diff --git a/arch/x86/kvm/mmu/tdp_mmu.c b/arch/x86/kvm/mmu/tdp_mmu.c index 6cd4dd631a2f..46f3e72ab770 100644 --- a/arch/x86/kvm/mmu/tdp_mmu.c +++ b/arch/x86/kvm/mmu/tdp_mmu.c @@ -957,14 +957,18 @@ static int tdp_mmu_map_handle_target_level(struct kvm_vcpu *vcpu, u64 new_spte; int ret = RET_PF_FIXED; bool wrprot = false; + unsigned access = ACC_ALL; if (WARN_ON_ONCE(sp->role.level != fault->goal_level)) return RET_PF_RETRY; + if (!fault->map_executable) + access &= ~ACC_EXEC_MASK; + if (unlikely(!fault->slot)) - new_spte = make_mmio_spte(vcpu, iter->gfn, ACC_ALL); + new_spte = make_mmio_spte(vcpu, iter->gfn, access); else - wrprot = make_spte(vcpu, sp, fault->slot, ACC_ALL, iter->gfn, + wrprot = make_spte(vcpu, sp, fault->slot, access, iter->gfn, fault->pfn, iter->old_spte, fault->prefetch, true, fault->map_writable, &new_spte); From patchwork Wed Nov 8 11:17:49 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Nicolas Saenz Julienne X-Patchwork-Id: 13449829 Received: from lindbergh.monkeyblade.net (lindbergh.monkeyblade.net [23.128.96.19]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 9140016400; Wed, 8 Nov 2023 11:21:52 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=amazon.com header.i=@amazon.com header.b="R0A3p8uD" Received: from smtp-fw-2101.amazon.com (smtp-fw-2101.amazon.com [72.21.196.25]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 79FD91BF3; Wed, 8 Nov 2023 03:21:50 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amazon.com; i=@amazon.com; q=dns/txt; s=amazon201209; t=1699442512; x=1730978512; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=oGmQSWddAE8KqMq8lmVDTRLTnkN/hdaUFRsiDOi+ttg=; b=R0A3p8uD3LsJkr+hknwGV/eFTPcdmhtskkROAvqShaRGrh3/7kuIY6J5 E1ayrjy8yPpJDtg5MnX8hQfU94cr+Tn/JzSv3TLHJKJPtXnt4Vb6yc0uE RlmSPbWY/jS/ePgDiCUEHdcPNwDldtVccxL7H3IWSVm+B6ChCt736aqRr 8=; X-IronPort-AV: E=Sophos;i="6.03,286,1694736000"; d="scan'208";a="361603061" Received: from iad12-co-svc-p1-lb1-vlan3.amazon.com (HELO email-inbound-relay-iad-1a-m6i4x-54a853e6.us-east-1.amazon.com) ([10.43.8.6]) by smtp-border-fw-2101.iad2.amazon.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 08 Nov 2023 11:21:50 +0000 Received: from smtpout.prod.us-east-1.prod.farcaster.email.amazon.dev (iad7-ws-svc-p70-lb3-vlan3.iad.amazon.com [10.32.235.38]) by email-inbound-relay-iad-1a-m6i4x-54a853e6.us-east-1.amazon.com (Postfix) with ESMTPS id 9FEC448ED2; Wed, 8 Nov 2023 11:21:46 +0000 (UTC) Received: from EX19MTAEUC001.ant.amazon.com [10.0.10.100:61983] by smtpin.naws.eu-west-1.prod.farcaster.email.amazon.dev [10.0.4.34:2525] with esmtp (Farcaster) id 93cbfb2a-c5dc-4a75-bf0d-dbebc073ea77; Wed, 8 Nov 2023 11:21:45 +0000 (UTC) X-Farcaster-Flow-ID: 93cbfb2a-c5dc-4a75-bf0d-dbebc073ea77 Received: from EX19D004EUC001.ant.amazon.com (10.252.51.190) by EX19MTAEUC001.ant.amazon.com (10.252.51.155) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.1118.39; Wed, 8 Nov 2023 11:21:39 +0000 Received: from dev-dsk-nsaenz-1b-189b39ae.eu-west-1.amazon.com (10.13.235.138) by EX19D004EUC001.ant.amazon.com (10.252.51.190) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.1118.39; Wed, 8 Nov 2023 11:21:34 +0000 From: Nicolas Saenz Julienne To: CC: , , , , , , , , , , , , , , , Nicolas Saenz Julienne Subject: [RFC 16/33] KVM: x86/mmu: Expose R/W/X flags during memory fault exits Date: Wed, 8 Nov 2023 11:17:49 +0000 Message-ID: <20231108111806.92604-17-nsaenz@amazon.com> X-Mailer: git-send-email 2.40.1 In-Reply-To: <20231108111806.92604-1-nsaenz@amazon.com> References: <20231108111806.92604-1-nsaenz@amazon.com> Precedence: bulk X-Mailing-List: kvm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Originating-IP: [10.13.235.138] X-ClientProxiedBy: EX19D043UWA003.ant.amazon.com (10.13.139.31) To EX19D004EUC001.ant.amazon.com (10.252.51.190) Include the fault's read, write and execute status when exiting to user-space. Signed-off-by: Nicolas Saenz Julienne --- arch/x86/kvm/mmu/mmu.c | 4 ++-- include/linux/kvm_host.h | 9 +++++++-- include/uapi/linux/kvm.h | 6 ++++++ 3 files changed, 15 insertions(+), 4 deletions(-) diff --git a/arch/x86/kvm/mmu/mmu.c b/arch/x86/kvm/mmu/mmu.c index 4e02d506cc25..feca077c0210 100644 --- a/arch/x86/kvm/mmu/mmu.c +++ b/arch/x86/kvm/mmu/mmu.c @@ -4300,8 +4300,8 @@ static inline u8 kvm_max_level_for_order(int order) static void kvm_mmu_prepare_memory_fault_exit(struct kvm_vcpu *vcpu, struct kvm_page_fault *fault) { - kvm_prepare_memory_fault_exit(vcpu, fault->gfn << PAGE_SHIFT, - PAGE_SIZE, fault->write, fault->exec, + kvm_prepare_memory_fault_exit(vcpu, fault->gfn << PAGE_SHIFT, PAGE_SIZE, + fault->write, fault->exec, fault->user, fault->is_private); } diff --git a/include/linux/kvm_host.h b/include/linux/kvm_host.h index 71e1e8cf8936..631fd532c97a 100644 --- a/include/linux/kvm_host.h +++ b/include/linux/kvm_host.h @@ -2367,14 +2367,19 @@ static inline void kvm_account_pgtable_pages(void *virt, int nr) static inline void kvm_prepare_memory_fault_exit(struct kvm_vcpu *vcpu, gpa_t gpa, gpa_t size, bool is_write, bool is_exec, - bool is_private) + bool is_read, bool is_private) { vcpu->run->exit_reason = KVM_EXIT_MEMORY_FAULT; vcpu->run->memory_fault.gpa = gpa; vcpu->run->memory_fault.size = size; - /* RWX flags are not (yet) defined or communicated to userspace. */ vcpu->run->memory_fault.flags = 0; + if (is_read) + vcpu->run->memory_fault.flags |= KVM_MEMORY_EXIT_FLAG_READ; + if (is_write) + vcpu->run->memory_fault.flags |= KVM_MEMORY_EXIT_FLAG_WRITE; + if (is_exec) + vcpu->run->memory_fault.flags |= KVM_MEMORY_EXIT_FLAG_EXECUTE; if (is_private) vcpu->run->memory_fault.flags |= KVM_MEMORY_EXIT_FLAG_PRIVATE; } diff --git a/include/uapi/linux/kvm.h b/include/uapi/linux/kvm.h index 03f5c08fd7aa..0ddffb8b0c99 100644 --- a/include/uapi/linux/kvm.h +++ b/include/uapi/linux/kvm.h @@ -533,7 +533,13 @@ struct kvm_run { } notify; /* KVM_EXIT_MEMORY_FAULT */ struct { +#define KVM_MEMORY_EXIT_FLAG_READ (1ULL << 0) +#define KVM_MEMORY_EXIT_FLAG_WRITE (1ULL << 1) +#define KVM_MEMORY_EXIT_FLAG_EXECUTE (1ULL << 2) #define KVM_MEMORY_EXIT_FLAG_PRIVATE (1ULL << 3) +#define KVM_MEMORY_EXIT_NO_ACCESS \ + (KVM_MEMORY_EXIT_FLAG_NR | KVM_MEMORY_EXIT_FLAG_NW | \ + KVM_MEMORY_EXIT_FLAG_NX) __u64 flags; __u64 gpa; __u64 size; From patchwork Wed Nov 8 11:17:50 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Nicolas Saenz Julienne X-Patchwork-Id: 13449830 Received: from lindbergh.monkeyblade.net (lindbergh.monkeyblade.net [23.128.96.19]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 07A6D171D6; Wed, 8 Nov 2023 11:21:56 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=amazon.com header.i=@amazon.com header.b="kKfzBpCq" Received: from smtp-fw-6002.amazon.com (smtp-fw-6002.amazon.com [52.95.49.90]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 0D02D1BE1; Wed, 8 Nov 2023 03:21:54 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amazon.com; i=@amazon.com; q=dns/txt; s=amazon201209; t=1699442515; x=1730978515; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=v9CrHyw1JpyXXRLGKDb9DzQRe5BqtRC9dyxOB/2B9Mo=; b=kKfzBpCqkT/h6wLsqI+jdt228IuVPdJRChlL4wuFcfzpSUgzKmFwfG5l vIXYKjZFv7WgL5B1VfC4pEy0KboL7P6IEWTpVg2mRJHNfSlxcW+hwWQ1V RbkMtzuUuZdG0SsYrK7Jp9RYQ1Snx+lhZ1DhuvRZ612IyVwygY1gsY09i 8=; X-IronPort-AV: E=Sophos;i="6.03,286,1694736000"; d="scan'208";a="366812660" Received: from iad12-co-svc-p1-lb1-vlan3.amazon.com (HELO email-inbound-relay-pdx-2b-m6i4x-f323d91c.us-west-2.amazon.com) ([10.43.8.6]) by smtp-border-fw-6002.iad6.amazon.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 08 Nov 2023 11:21:53 +0000 Received: from smtpout.prod.us-west-2.prod.farcaster.email.amazon.dev (pdx2-ws-svc-p26-lb5-vlan2.pdx.amazon.com [10.39.38.66]) by email-inbound-relay-pdx-2b-m6i4x-f323d91c.us-west-2.amazon.com (Postfix) with ESMTPS id 3352840D95; Wed, 8 Nov 2023 11:21:52 +0000 (UTC) Received: from EX19MTAEUA001.ant.amazon.com [10.0.17.79:36943] by smtpin.naws.eu-west-1.prod.farcaster.email.amazon.dev [10.0.22.222:2525] with esmtp (Farcaster) id a2d9683d-2fd0-47e3-9e16-e83a0bd5b6ea; Wed, 8 Nov 2023 11:21:51 +0000 (UTC) X-Farcaster-Flow-ID: a2d9683d-2fd0-47e3-9e16-e83a0bd5b6ea Received: from EX19D004EUC001.ant.amazon.com (10.252.51.190) by EX19MTAEUA001.ant.amazon.com (10.252.50.192) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.1118.39; Wed, 8 Nov 2023 11:21:44 +0000 Received: from dev-dsk-nsaenz-1b-189b39ae.eu-west-1.amazon.com (10.13.235.138) by EX19D004EUC001.ant.amazon.com (10.252.51.190) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.1118.39; Wed, 8 Nov 2023 11:21:39 +0000 From: Nicolas Saenz Julienne To: CC: , , , , , , , , , , , , , , , Nicolas Saenz Julienne Subject: [RFC 17/33] KVM: x86/mmu: Allow setting memory attributes if VSM enabled Date: Wed, 8 Nov 2023 11:17:50 +0000 Message-ID: <20231108111806.92604-18-nsaenz@amazon.com> X-Mailer: git-send-email 2.40.1 In-Reply-To: <20231108111806.92604-1-nsaenz@amazon.com> References: <20231108111806.92604-1-nsaenz@amazon.com> Precedence: bulk X-Mailing-List: kvm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Originating-IP: [10.13.235.138] X-ClientProxiedBy: EX19D043UWA003.ant.amazon.com (10.13.139.31) To EX19D004EUC001.ant.amazon.com (10.252.51.190) VSM is also a user of memory attributes, so let it use kvm_set_mem_attributes(). Signed-off-by: Nicolas Saenz Julienne --- arch/x86/kvm/mmu/mmu.c | 6 ++++-- 1 file changed, 4 insertions(+), 2 deletions(-) diff --git a/arch/x86/kvm/mmu/mmu.c b/arch/x86/kvm/mmu/mmu.c index feca077c0210..a1fbb905258b 100644 --- a/arch/x86/kvm/mmu/mmu.c +++ b/arch/x86/kvm/mmu/mmu.c @@ -7265,7 +7265,8 @@ bool kvm_arch_pre_set_memory_attributes(struct kvm *kvm, * Zapping SPTEs in this case ensures KVM will reassess whether or not * a hugepage can be used for affected ranges. */ - if (WARN_ON_ONCE(!kvm_arch_has_private_mem(kvm))) + if (WARN_ON_ONCE(!kvm_arch_has_private_mem(kvm) && + !kvm_hv_vsm_enabled(kvm))) return false; return kvm_unmap_gfn_range(kvm, range); @@ -7322,7 +7323,8 @@ bool kvm_arch_post_set_memory_attributes(struct kvm *kvm, * a range that has PRIVATE GFNs, and conversely converting a range to * SHARED may now allow hugepages. */ - if (WARN_ON_ONCE(!kvm_arch_has_private_mem(kvm))) + if (WARN_ON_ONCE(!kvm_arch_has_private_mem(kvm) && + !kvm_hv_vsm_enabled(kvm))) return false; /* From patchwork Wed Nov 8 11:17:51 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Nicolas Saenz Julienne X-Patchwork-Id: 13449831 Received: from lindbergh.monkeyblade.net (lindbergh.monkeyblade.net [23.128.96.19]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id C5F91D267; Wed, 8 Nov 2023 11:22:15 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=amazon.com header.i=@amazon.com header.b="ScG3mjd4" Received: from smtp-fw-80007.amazon.com (smtp-fw-80007.amazon.com [99.78.197.218]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 3C9532595; Wed, 8 Nov 2023 03:22:15 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amazon.com; i=@amazon.com; q=dns/txt; s=amazon201209; t=1699442535; x=1730978535; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=v0xPIztXiJ7BozvZWMLf19nS4eQCyoOv0IyQ7RPZ9oY=; b=ScG3mjd4qwEUTBFvQ+qL53JHqLOqgY4i/iWkFCjCq/HOq1GCAgsSyW3X 6pMcvDICCD68aTy80gFBxxOnZwtbhliEV0N3yLlHqNPCUKXPefTq+npqV 4Frx7Bx5Aay2dODnOlP+XcPDWNWoyGh0dT9ydSBITImWlLnlqWIJ6BIcW w=; X-IronPort-AV: E=Sophos;i="6.03,286,1694736000"; d="scan'208";a="251428368" Received: from pdx4-co-svc-p1-lb2-vlan2.amazon.com (HELO email-inbound-relay-pdx-2c-m6i4x-fa5fe5fb.us-west-2.amazon.com) ([10.25.36.210]) by smtp-border-fw-80007.pdx80.corp.amazon.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 08 Nov 2023 11:22:14 +0000 Received: from smtpout.prod.us-west-2.prod.farcaster.email.amazon.dev (pdx2-ws-svc-p26-lb5-vlan3.pdx.amazon.com [10.39.38.70]) by email-inbound-relay-pdx-2c-m6i4x-fa5fe5fb.us-west-2.amazon.com (Postfix) with ESMTPS id 372E640DB0; Wed, 8 Nov 2023 11:22:14 +0000 (UTC) Received: from EX19MTAEUA002.ant.amazon.com [10.0.17.79:29860] by smtpin.naws.eu-west-1.prod.farcaster.email.amazon.dev [10.0.43.105:2525] with esmtp (Farcaster) id 76b6a16b-a40f-4cc6-97d8-14f01b8119af; Wed, 8 Nov 2023 11:22:13 +0000 (UTC) X-Farcaster-Flow-ID: 76b6a16b-a40f-4cc6-97d8-14f01b8119af Received: from EX19D004EUC001.ant.amazon.com (10.252.51.190) by EX19MTAEUA002.ant.amazon.com (10.252.50.124) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.1118.39; Wed, 8 Nov 2023 11:22:12 +0000 Received: from dev-dsk-nsaenz-1b-189b39ae.eu-west-1.amazon.com (10.13.235.138) by EX19D004EUC001.ant.amazon.com (10.252.51.190) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.1118.39; Wed, 8 Nov 2023 11:22:08 +0000 From: Nicolas Saenz Julienne To: CC: , , , , , , , , , , , , , , , Nicolas Saenz Julienne Subject: [RFC 18/33] KVM: x86: Decouple kvm_get_memory_attributes() from struct kvm's mem_attr_array Date: Wed, 8 Nov 2023 11:17:51 +0000 Message-ID: <20231108111806.92604-19-nsaenz@amazon.com> X-Mailer: git-send-email 2.40.1 In-Reply-To: <20231108111806.92604-1-nsaenz@amazon.com> References: <20231108111806.92604-1-nsaenz@amazon.com> Precedence: bulk X-Mailing-List: kvm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Originating-IP: [10.13.235.138] X-ClientProxiedBy: EX19D046UWB003.ant.amazon.com (10.13.139.174) To EX19D004EUC001.ant.amazon.com (10.252.51.190) Decouple kvm_get_memory_attributes() from struct kvm's mem_attr_array to allow other memory attribute sources to use the function. Signed-off-by: Nicolas Saenz Julienne --- arch/x86/kvm/mmu/mmu.c | 5 +++-- include/linux/kvm_host.h | 8 +++++--- 2 files changed, 8 insertions(+), 5 deletions(-) diff --git a/arch/x86/kvm/mmu/mmu.c b/arch/x86/kvm/mmu/mmu.c index a1fbb905258b..96421234ca88 100644 --- a/arch/x86/kvm/mmu/mmu.c +++ b/arch/x86/kvm/mmu/mmu.c @@ -7301,7 +7301,7 @@ static bool hugepage_has_attrs(struct kvm *kvm, struct kvm_memory_slot *slot, for (gfn = start; gfn < end; gfn += KVM_PAGES_PER_HPAGE(level - 1)) { if (hugepage_test_mixed(slot, gfn, level - 1) || - attrs != kvm_get_memory_attributes(kvm, gfn)) + attrs != kvm_get_memory_attributes(&kvm->mem_attr_array, gfn)) return false; } return true; @@ -7401,7 +7401,8 @@ void kvm_mmu_init_memslot_memory_attributes(struct kvm *kvm, * be manually checked as the attributes may already be mixed. */ for (gfn = start; gfn < end; gfn += nr_pages) { - unsigned long attrs = kvm_get_memory_attributes(kvm, gfn); + unsigned long attrs = + kvm_get_memory_attributes(&kvm->mem_attr_array, gfn); if (hugepage_has_attrs(kvm, slot, gfn, level, attrs)) hugepage_clear_mixed(slot, gfn, level); diff --git a/include/linux/kvm_host.h b/include/linux/kvm_host.h index 631fd532c97a..4242588e3dfb 100644 --- a/include/linux/kvm_host.h +++ b/include/linux/kvm_host.h @@ -2385,9 +2385,10 @@ static inline void kvm_prepare_memory_fault_exit(struct kvm_vcpu *vcpu, } #ifdef CONFIG_KVM_GENERIC_MEMORY_ATTRIBUTES -static inline unsigned long kvm_get_memory_attributes(struct kvm *kvm, gfn_t gfn) +static inline unsigned long +kvm_get_memory_attributes(struct xarray *mem_attr_array, gfn_t gfn) { - return xa_to_value(xa_load(&kvm->mem_attr_array, gfn)); + return xa_to_value(xa_load(mem_attr_array, gfn)); } bool kvm_range_has_memory_attributes(struct kvm *kvm, gfn_t start, gfn_t end, @@ -2400,7 +2401,8 @@ bool kvm_arch_post_set_memory_attributes(struct kvm *kvm, static inline bool kvm_mem_is_private(struct kvm *kvm, gfn_t gfn) { return IS_ENABLED(CONFIG_KVM_PRIVATE_MEM) && - kvm_get_memory_attributes(kvm, gfn) & KVM_MEMORY_ATTRIBUTE_PRIVATE; + kvm_get_memory_attributes(&kvm->mem_attr_array, gfn) & + KVM_MEMORY_ATTRIBUTE_PRIVATE; } #else static inline bool kvm_mem_is_private(struct kvm *kvm, gfn_t gfn) From patchwork Wed Nov 8 11:17:52 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Nicolas Saenz Julienne X-Patchwork-Id: 13449832 Received: from lindbergh.monkeyblade.net (lindbergh.monkeyblade.net [23.128.96.19]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id C5C1F15E96; Wed, 8 Nov 2023 11:22:23 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=amazon.com header.i=@amazon.com header.b="bMupkCSx" Received: from smtp-fw-52002.amazon.com (smtp-fw-52002.amazon.com [52.119.213.150]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id F01271FE3; Wed, 8 Nov 2023 03:22:22 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amazon.com; i=@amazon.com; q=dns/txt; s=amazon201209; t=1699442544; x=1730978544; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=rFj4X075aGu4UwwOIKw0SMbu/vxZLmqvGhwr4IjXpjU=; b=bMupkCSx8kHw7yPBZciU2HV8F0Xl+xtZ09Ws2eUE53PJGeRZCYfT7Nop cnV59XpC57W4oEMdNZq6haUMzkf08ibV3IA+JhnmfJnYKUcqQ7/SNUWEw 4yCNTeBBg/HWh449gZlqWxpqszG8HZucYoctsn85OQPCaAymBvqneZNa4 0=; X-IronPort-AV: E=Sophos;i="6.03,286,1694736000"; d="scan'208";a="593807859" Received: from iad12-co-svc-p1-lb1-vlan3.amazon.com (HELO email-inbound-relay-pdx-2b-m6i4x-189d700f.us-west-2.amazon.com) ([10.43.8.6]) by smtp-border-fw-52002.iad7.amazon.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 08 Nov 2023 11:22:21 +0000 Received: from smtpout.prod.us-west-2.prod.farcaster.email.amazon.dev (pdx2-ws-svc-p26-lb5-vlan3.pdx.amazon.com [10.39.38.70]) by email-inbound-relay-pdx-2b-m6i4x-189d700f.us-west-2.amazon.com (Postfix) with ESMTPS id 452CD40DAA; Wed, 8 Nov 2023 11:22:19 +0000 (UTC) Received: from EX19MTAEUB001.ant.amazon.com [10.0.17.79:28238] by smtpin.naws.eu-west-1.prod.farcaster.email.amazon.dev [10.0.43.105:2525] with esmtp (Farcaster) id dcf98806-3299-4d1a-b110-916760b68cf4; Wed, 8 Nov 2023 11:22:18 +0000 (UTC) X-Farcaster-Flow-ID: dcf98806-3299-4d1a-b110-916760b68cf4 Received: from EX19D004EUC001.ant.amazon.com (10.252.51.190) by EX19MTAEUB001.ant.amazon.com (10.252.51.28) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.1118.39; Wed, 8 Nov 2023 11:22:17 +0000 Received: from dev-dsk-nsaenz-1b-189b39ae.eu-west-1.amazon.com (10.13.235.138) by EX19D004EUC001.ant.amazon.com (10.252.51.190) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.1118.39; Wed, 8 Nov 2023 11:22:13 +0000 From: Nicolas Saenz Julienne To: CC: , , , , , , , , , , , , , , , Nicolas Saenz Julienne Subject: [RFC 19/33] KVM: x86: Decouple kvm_range_has_memory_attributes() from struct kvm's mem_attr_array Date: Wed, 8 Nov 2023 11:17:52 +0000 Message-ID: <20231108111806.92604-20-nsaenz@amazon.com> X-Mailer: git-send-email 2.40.1 In-Reply-To: <20231108111806.92604-1-nsaenz@amazon.com> References: <20231108111806.92604-1-nsaenz@amazon.com> Precedence: bulk X-Mailing-List: kvm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Originating-IP: [10.13.235.138] X-ClientProxiedBy: EX19D046UWB003.ant.amazon.com (10.13.139.174) To EX19D004EUC001.ant.amazon.com (10.252.51.190) Decouple kvm_range_has_memory_attributes() from struct kvm's mem_attr_array to allow other memory attribute sources to use the function. Signed-off-by: Nicolas Saenz Julienne --- arch/x86/kvm/mmu/mmu.c | 3 ++- include/linux/kvm_host.h | 4 ++-- virt/kvm/kvm_main.c | 9 +++++---- 3 files changed, 9 insertions(+), 7 deletions(-) diff --git a/arch/x86/kvm/mmu/mmu.c b/arch/x86/kvm/mmu/mmu.c index 96421234ca88..4ace2f8660b0 100644 --- a/arch/x86/kvm/mmu/mmu.c +++ b/arch/x86/kvm/mmu/mmu.c @@ -7297,7 +7297,8 @@ static bool hugepage_has_attrs(struct kvm *kvm, struct kvm_memory_slot *slot, const unsigned long end = start + KVM_PAGES_PER_HPAGE(level); if (level == PG_LEVEL_2M) - return kvm_range_has_memory_attributes(kvm, start, end, attrs); + return kvm_range_has_memory_attributes(&kvm->mem_attr_array, + start, end, attrs); for (gfn = start; gfn < end; gfn += KVM_PAGES_PER_HPAGE(level - 1)) { if (hugepage_test_mixed(slot, gfn, level - 1) || diff --git a/include/linux/kvm_host.h b/include/linux/kvm_host.h index 4242588e3dfb..32cf05637647 100644 --- a/include/linux/kvm_host.h +++ b/include/linux/kvm_host.h @@ -2391,8 +2391,8 @@ kvm_get_memory_attributes(struct xarray *mem_attr_array, gfn_t gfn) return xa_to_value(xa_load(mem_attr_array, gfn)); } -bool kvm_range_has_memory_attributes(struct kvm *kvm, gfn_t start, gfn_t end, - unsigned long attrs); +bool kvm_range_has_memory_attributes(struct xarray *mem_attr_array, gfn_t start, + gfn_t end, unsigned long attrs); bool kvm_arch_pre_set_memory_attributes(struct kvm *kvm, struct kvm_gfn_range *range); bool kvm_arch_post_set_memory_attributes(struct kvm *kvm, diff --git a/virt/kvm/kvm_main.c b/virt/kvm/kvm_main.c index fde004a0ac46..6bb23eaf7aa6 100644 --- a/virt/kvm/kvm_main.c +++ b/virt/kvm/kvm_main.c @@ -2440,10 +2440,10 @@ static int kvm_vm_ioctl_clear_dirty_log(struct kvm *kvm, * Returns true if _all_ gfns in the range [@start, @end) have attributes * matching @attrs. */ -bool kvm_range_has_memory_attributes(struct kvm *kvm, gfn_t start, gfn_t end, - unsigned long attrs) +bool kvm_range_has_memory_attributes(struct xarray *mem_attr_array, gfn_t start, + gfn_t end, unsigned long attrs) { - XA_STATE(xas, &kvm->mem_attr_array, start); + XA_STATE(xas, mem_attr_array, start); unsigned long index; bool has_attrs; void *entry; @@ -2582,7 +2582,8 @@ static int kvm_vm_set_mem_attributes(struct kvm *kvm, gfn_t start, gfn_t end, mutex_lock(&kvm->slots_lock); /* Nothing to do if the entire range as the desired attributes. */ - if (kvm_range_has_memory_attributes(kvm, start, end, attributes)) + if (kvm_range_has_memory_attributes(&kvm->mem_attr_array, start, end, + attributes)) goto out_unlock; /* From patchwork Wed Nov 8 11:17:53 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Nicolas Saenz Julienne X-Patchwork-Id: 13449833 Received: from lindbergh.monkeyblade.net (lindbergh.monkeyblade.net [23.128.96.19]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id DC12515E96; Wed, 8 Nov 2023 11:22:30 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=amazon.com header.i=@amazon.com header.b="eKTIdGo5" Received: from smtp-fw-52004.amazon.com (smtp-fw-52004.amazon.com [52.119.213.154]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 9B8CC2101; Wed, 8 Nov 2023 03:22:29 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amazon.com; i=@amazon.com; q=dns/txt; s=amazon201209; t=1699442550; x=1730978550; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=J7UoBAKDyQ8FEQ/hZE/ZqdsRX3l1s1mZYyf6joluUVw=; b=eKTIdGo5kTWW7brRwyBs8HubcaCaa+Bio9gajh4QmEzRn3QF8Z6Qha+V VSbpZ0aHF71h6EKaQ38pEVHzFv4Ad/ADlqtVTftyMLTBIqKTdrR4WoJI4 iD/tbY4wpYih0q0sPabhPnQTe3x0G4mHKe+KWGmstGMvvc7D382BP83vg E=; X-IronPort-AV: E=Sophos;i="6.03,286,1694736000"; d="scan'208";a="164959413" Received: from iad12-co-svc-p1-lb1-vlan2.amazon.com (HELO email-inbound-relay-pdx-2c-m6i4x-b1c0e1d0.us-west-2.amazon.com) ([10.43.8.2]) by smtp-border-fw-52004.iad7.amazon.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 08 Nov 2023 11:22:28 +0000 Received: from smtpout.prod.us-west-2.prod.farcaster.email.amazon.dev (pdx2-ws-svc-p26-lb5-vlan3.pdx.amazon.com [10.39.38.70]) by email-inbound-relay-pdx-2c-m6i4x-b1c0e1d0.us-west-2.amazon.com (Postfix) with ESMTPS id 5F0088A33C; Wed, 8 Nov 2023 11:22:24 +0000 (UTC) Received: from EX19MTAEUB002.ant.amazon.com [10.0.17.79:16387] by smtpin.naws.eu-west-1.prod.farcaster.email.amazon.dev [10.0.43.105:2525] with esmtp (Farcaster) id 52d25461-f54c-47d3-a30b-cd79c101be6b; Wed, 8 Nov 2023 11:22:23 +0000 (UTC) X-Farcaster-Flow-ID: 52d25461-f54c-47d3-a30b-cd79c101be6b Received: from EX19D004EUC001.ant.amazon.com (10.252.51.190) by EX19MTAEUB002.ant.amazon.com (10.252.51.59) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.1118.39; Wed, 8 Nov 2023 11:22:22 +0000 Received: from dev-dsk-nsaenz-1b-189b39ae.eu-west-1.amazon.com (10.13.235.138) by EX19D004EUC001.ant.amazon.com (10.252.51.190) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.1118.39; Wed, 8 Nov 2023 11:22:18 +0000 From: Nicolas Saenz Julienne To: CC: , , , , , , , , , , , , , , , Nicolas Saenz Julienne Subject: [RFC 20/33] KVM: x86/mmu: Decouple hugepage_has_attrs() from struct kvm's mem_attr_array Date: Wed, 8 Nov 2023 11:17:53 +0000 Message-ID: <20231108111806.92604-21-nsaenz@amazon.com> X-Mailer: git-send-email 2.40.1 In-Reply-To: <20231108111806.92604-1-nsaenz@amazon.com> References: <20231108111806.92604-1-nsaenz@amazon.com> Precedence: bulk X-Mailing-List: kvm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Originating-IP: [10.13.235.138] X-ClientProxiedBy: EX19D046UWB003.ant.amazon.com (10.13.139.174) To EX19D004EUC001.ant.amazon.com (10.252.51.190) Decouple hugepage_has_attrs() from struct kvm's mem_attr_array to allow other memory attribute sources to use the function. Signed-off-by: Nicolas Saenz Julienne --- arch/x86/kvm/mmu/mmu.c | 18 ++++++++++-------- 1 file changed, 10 insertions(+), 8 deletions(-) diff --git a/arch/x86/kvm/mmu/mmu.c b/arch/x86/kvm/mmu/mmu.c index 4ace2f8660b0..c0fd3afd6be5 100644 --- a/arch/x86/kvm/mmu/mmu.c +++ b/arch/x86/kvm/mmu/mmu.c @@ -7290,19 +7290,19 @@ static void hugepage_set_mixed(struct kvm_memory_slot *slot, gfn_t gfn, lpage_info_slot(gfn, slot, level)->disallow_lpage |= KVM_LPAGE_MIXED_FLAG; } -static bool hugepage_has_attrs(struct kvm *kvm, struct kvm_memory_slot *slot, - gfn_t gfn, int level, unsigned long attrs) +static bool hugepage_has_attrs(struct xarray *mem_attr_array, + struct kvm_memory_slot *slot, gfn_t gfn, + int level, unsigned long attrs) { const unsigned long start = gfn; const unsigned long end = start + KVM_PAGES_PER_HPAGE(level); if (level == PG_LEVEL_2M) - return kvm_range_has_memory_attributes(&kvm->mem_attr_array, - start, end, attrs); + return kvm_range_has_memory_attributes(mem_attr_array, start, end, attrs); for (gfn = start; gfn < end; gfn += KVM_PAGES_PER_HPAGE(level - 1)) { if (hugepage_test_mixed(slot, gfn, level - 1) || - attrs != kvm_get_memory_attributes(&kvm->mem_attr_array, gfn)) + attrs != kvm_get_memory_attributes(mem_attr_array, gfn)) return false; } return true; @@ -7344,7 +7344,8 @@ bool kvm_arch_post_set_memory_attributes(struct kvm *kvm, * misaligned address regardless of memory attributes. */ if (gfn >= slot->base_gfn) { - if (hugepage_has_attrs(kvm, slot, gfn, level, attrs)) + if (hugepage_has_attrs(&kvm->mem_attr_array, + slot, gfn, level, attrs)) hugepage_clear_mixed(slot, gfn, level); else hugepage_set_mixed(slot, gfn, level); @@ -7366,7 +7367,8 @@ bool kvm_arch_post_set_memory_attributes(struct kvm *kvm, */ if (gfn < range->end && (gfn + nr_pages) <= (slot->base_gfn + slot->npages)) { - if (hugepage_has_attrs(kvm, slot, gfn, level, attrs)) + if (hugepage_has_attrs(&kvm->mem_attr_array, slot, gfn, + level, attrs)) hugepage_clear_mixed(slot, gfn, level); else hugepage_set_mixed(slot, gfn, level); @@ -7405,7 +7407,7 @@ void kvm_mmu_init_memslot_memory_attributes(struct kvm *kvm, unsigned long attrs = kvm_get_memory_attributes(&kvm->mem_attr_array, gfn); - if (hugepage_has_attrs(kvm, slot, gfn, level, attrs)) + if (hugepage_has_attrs(&kvm->mem_attr_array, slot, gfn, level, attrs)) hugepage_clear_mixed(slot, gfn, level); else hugepage_set_mixed(slot, gfn, level); From patchwork Wed Nov 8 11:17:54 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Nicolas Saenz Julienne X-Patchwork-Id: 13449834 Received: from lindbergh.monkeyblade.net (lindbergh.monkeyblade.net [23.128.96.19]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id CCAAB15E8F; Wed, 8 Nov 2023 11:22:57 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=amazon.com header.i=@amazon.com header.b="KZxnAQq5" Received: from smtp-fw-52002.amazon.com (smtp-fw-52002.amazon.com [52.119.213.150]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id D763D1FF6; Wed, 8 Nov 2023 03:22:56 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amazon.com; i=@amazon.com; q=dns/txt; s=amazon201209; t=1699442578; x=1730978578; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=RgZghGDVPKK1y7/jRC8uX7Dc8zxHwsko9uAHRrK/b/E=; b=KZxnAQq5x79fecGfeJTdJwy7Asc3OjoDvMstzCK0CsEroWHde6inNPoH bAU2j0GSTfcIUcHmqvV6Wua1ulzGl2alPEfcFhkNCqK5ql5IvNSb6LsP9 lasAo5p7cJN+MQIsFD4w09XMJeESBGC7AfQ4VSo0pakLEqgS+5nD3v4yh k=; X-IronPort-AV: E=Sophos;i="6.03,286,1694736000"; d="scan'208";a="593808027" Received: from iad12-co-svc-p1-lb1-vlan3.amazon.com (HELO email-inbound-relay-iad-1d-m6i4x-d23e07e8.us-east-1.amazon.com) ([10.43.8.6]) by smtp-border-fw-52002.iad7.amazon.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 08 Nov 2023 11:22:57 +0000 Received: from smtpout.prod.us-east-1.prod.farcaster.email.amazon.dev (iad7-ws-svc-p70-lb3-vlan3.iad.amazon.com [10.32.235.38]) by email-inbound-relay-iad-1d-m6i4x-d23e07e8.us-east-1.amazon.com (Postfix) with ESMTPS id 6925380801; Wed, 8 Nov 2023 11:22:52 +0000 (UTC) Received: from EX19MTAEUC002.ant.amazon.com [10.0.17.79:36451] by smtpin.naws.eu-west-1.prod.farcaster.email.amazon.dev [10.0.43.105:2525] with esmtp (Farcaster) id 6ea11377-1737-4520-be2a-aa11258cc468; Wed, 8 Nov 2023 11:22:51 +0000 (UTC) X-Farcaster-Flow-ID: 6ea11377-1737-4520-be2a-aa11258cc468 Received: from EX19D004EUC001.ant.amazon.com (10.252.51.190) by EX19MTAEUC002.ant.amazon.com (10.252.51.181) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.1118.39; Wed, 8 Nov 2023 11:22:51 +0000 Received: from dev-dsk-nsaenz-1b-189b39ae.eu-west-1.amazon.com (10.13.235.138) by EX19D004EUC001.ant.amazon.com (10.252.51.190) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.1118.39; Wed, 8 Nov 2023 11:22:46 +0000 From: Nicolas Saenz Julienne To: CC: , , , , , , , , , , , , , , , Nicolas Saenz Julienne Subject: [RFC 21/33] KVM: Pass memory attribute array as a MMU notifier argument Date: Wed, 8 Nov 2023 11:17:54 +0000 Message-ID: <20231108111806.92604-22-nsaenz@amazon.com> X-Mailer: git-send-email 2.40.1 In-Reply-To: <20231108111806.92604-1-nsaenz@amazon.com> References: <20231108111806.92604-1-nsaenz@amazon.com> Precedence: bulk X-Mailing-List: kvm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Originating-IP: [10.13.235.138] X-ClientProxiedBy: EX19D033UWC001.ant.amazon.com (10.13.139.218) To EX19D004EUC001.ant.amazon.com (10.252.51.190) Pass the memory attribute array through struct kvm_mmu_notifier_arg and use it in kvm_arch_post_set_memory_attributes() instead of defaulting on kvm->mem_attr_array. Signed-off-by: Nicolas Saenz Julienne --- arch/x86/kvm/mmu/mmu.c | 8 ++++---- include/linux/kvm_host.h | 5 ++++- virt/kvm/kvm_main.c | 1 + 3 files changed, 9 insertions(+), 5 deletions(-) diff --git a/arch/x86/kvm/mmu/mmu.c b/arch/x86/kvm/mmu/mmu.c index c0fd3afd6be5..c2bec2be2ba9 100644 --- a/arch/x86/kvm/mmu/mmu.c +++ b/arch/x86/kvm/mmu/mmu.c @@ -7311,6 +7311,7 @@ static bool hugepage_has_attrs(struct xarray *mem_attr_array, bool kvm_arch_post_set_memory_attributes(struct kvm *kvm, struct kvm_gfn_range *range) { + struct xarray *mem_attr_array = range->arg.mem_attr_array; unsigned long attrs = range->arg.attributes; struct kvm_memory_slot *slot = range->slot; int level; @@ -7344,8 +7345,8 @@ bool kvm_arch_post_set_memory_attributes(struct kvm *kvm, * misaligned address regardless of memory attributes. */ if (gfn >= slot->base_gfn) { - if (hugepage_has_attrs(&kvm->mem_attr_array, - slot, gfn, level, attrs)) + if (hugepage_has_attrs(mem_attr_array, slot, + gfn, level, attrs)) hugepage_clear_mixed(slot, gfn, level); else hugepage_set_mixed(slot, gfn, level); @@ -7367,8 +7368,7 @@ bool kvm_arch_post_set_memory_attributes(struct kvm *kvm, */ if (gfn < range->end && (gfn + nr_pages) <= (slot->base_gfn + slot->npages)) { - if (hugepage_has_attrs(&kvm->mem_attr_array, slot, gfn, - level, attrs)) + if (hugepage_has_attrs(mem_attr_array, slot, gfn, level, attrs)) hugepage_clear_mixed(slot, gfn, level); else hugepage_set_mixed(slot, gfn, level); diff --git a/include/linux/kvm_host.h b/include/linux/kvm_host.h index 32cf05637647..652656444c45 100644 --- a/include/linux/kvm_host.h +++ b/include/linux/kvm_host.h @@ -256,7 +256,10 @@ int kvm_async_pf_wakeup_all(struct kvm_vcpu *vcpu); #ifdef CONFIG_KVM_GENERIC_MMU_NOTIFIER union kvm_mmu_notifier_arg { pte_t pte; - unsigned long attributes; + struct { + unsigned long attributes; + struct xarray *mem_attr_array; + }; }; struct kvm_gfn_range { diff --git a/virt/kvm/kvm_main.c b/virt/kvm/kvm_main.c index 6bb23eaf7aa6..f20dafaedc72 100644 --- a/virt/kvm/kvm_main.c +++ b/virt/kvm/kvm_main.c @@ -2569,6 +2569,7 @@ static int kvm_vm_set_mem_attributes(struct kvm *kvm, gfn_t start, gfn_t end, .start = start, .end = end, .arg.attributes = attributes, + .arg.mem_attr_array = &kvm->mem_attr_array, .handler = kvm_arch_post_set_memory_attributes, .on_lock = kvm_mmu_invalidate_end, .may_block = true, From patchwork Wed Nov 8 11:17:55 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Nicolas Saenz Julienne X-Patchwork-Id: 13449835 Received: from lindbergh.monkeyblade.net (lindbergh.monkeyblade.net [23.128.96.19]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id C15B215EBB; Wed, 8 Nov 2023 11:23:02 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=amazon.com header.i=@amazon.com header.b="J1NsGqHF" Received: from smtp-fw-52005.amazon.com (smtp-fw-52005.amazon.com [52.119.213.156]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 9DE072135; Wed, 8 Nov 2023 03:23:01 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amazon.com; i=@amazon.com; q=dns/txt; s=amazon201209; t=1699442582; x=1730978582; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=2uG1W3dIcChJokJ3iAOnGcK/lDIrdgs97U52b9YNSN4=; b=J1NsGqHFv3Xa6QtMyVm8oGK078p8CRMvRUVIsTe1dRNNgJ8icfFfsxiQ 6ba7qHHPlsg41GHi0sNiUWkrftmRfZFN/RcBaD+ADgBAXKkuvy+cBTkOM 5vOSibyj0wKulG/rAgDEIdp1UZMjVuH+TY5+SR05nsNv+DCIr88MABMMY Y=; X-IronPort-AV: E=Sophos;i="6.03,286,1694736000"; d="scan'208";a="614866194" Received: from iad12-co-svc-p1-lb1-vlan3.amazon.com (HELO email-inbound-relay-iad-1e-m6i4x-6e7a78d7.us-east-1.amazon.com) ([10.43.8.6]) by smtp-border-fw-52005.iad7.amazon.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 08 Nov 2023 11:23:00 +0000 Received: from smtpout.prod.us-east-1.prod.farcaster.email.amazon.dev (iad7-ws-svc-p70-lb3-vlan2.iad.amazon.com [10.32.235.34]) by email-inbound-relay-iad-1e-m6i4x-6e7a78d7.us-east-1.amazon.com (Postfix) with ESMTPS id B23D3804F3; Wed, 8 Nov 2023 11:22:57 +0000 (UTC) Received: from EX19MTAEUC001.ant.amazon.com [10.0.10.100:52842] by smtpin.naws.eu-west-1.prod.farcaster.email.amazon.dev [10.0.22.222:2525] with esmtp (Farcaster) id 7959fa46-ecea-470f-903c-012efedff35f; Wed, 8 Nov 2023 11:22:56 +0000 (UTC) X-Farcaster-Flow-ID: 7959fa46-ecea-470f-903c-012efedff35f Received: from EX19D004EUC001.ant.amazon.com (10.252.51.190) by EX19MTAEUC001.ant.amazon.com (10.252.51.155) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.1118.39; Wed, 8 Nov 2023 11:22:56 +0000 Received: from dev-dsk-nsaenz-1b-189b39ae.eu-west-1.amazon.com (10.13.235.138) by EX19D004EUC001.ant.amazon.com (10.252.51.190) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.1118.39; Wed, 8 Nov 2023 11:22:51 +0000 From: Nicolas Saenz Julienne To: CC: , , , , , , , , , , , , , , , Nicolas Saenz Julienne Subject: [RFC 22/33] KVM: Decouple kvm_ioctl_set_mem_attributes() from kvm's mem_attr_array Date: Wed, 8 Nov 2023 11:17:55 +0000 Message-ID: <20231108111806.92604-23-nsaenz@amazon.com> X-Mailer: git-send-email 2.40.1 In-Reply-To: <20231108111806.92604-1-nsaenz@amazon.com> References: <20231108111806.92604-1-nsaenz@amazon.com> Precedence: bulk X-Mailing-List: kvm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Originating-IP: [10.13.235.138] X-ClientProxiedBy: EX19D033UWC001.ant.amazon.com (10.13.139.218) To EX19D004EUC001.ant.amazon.com (10.252.51.190) VSM will keep track of each VTL's memory protections in a separate mem_attr_array. Access to these arrays will happen by issuing KVM_SET_MEMORY_ATTRIBUTES ioctls to their respective KVM VTL devices (which is also introduced in subsequent patches). Let the VTL devices reuse kvm_ioctl_set_mem_attributes() by decoupling it from struct kvm's mem_attr_array. The xarray is now input as a function argument as well as the list of supported memory attributes. Signed-off-by: Nicolas Saenz Julienne --- include/linux/kvm_host.h | 3 +++ virt/kvm/kvm_main.c | 32 ++++++++++++++++++++++---------- 2 files changed, 25 insertions(+), 10 deletions(-) diff --git a/include/linux/kvm_host.h b/include/linux/kvm_host.h index 652656444c45..ad104794037f 100644 --- a/include/linux/kvm_host.h +++ b/include/linux/kvm_host.h @@ -2394,6 +2394,9 @@ kvm_get_memory_attributes(struct xarray *mem_attr_array, gfn_t gfn) return xa_to_value(xa_load(mem_attr_array, gfn)); } +int kvm_ioctl_set_mem_attributes(struct kvm *kvm, struct xarray *mem_attr_array, + u64 supported_attrs, + struct kvm_memory_attributes *attrs); bool kvm_range_has_memory_attributes(struct xarray *mem_attr_array, gfn_t start, gfn_t end, unsigned long attrs); bool kvm_arch_pre_set_memory_attributes(struct kvm *kvm, diff --git a/virt/kvm/kvm_main.c b/virt/kvm/kvm_main.c index f20dafaedc72..74c4c42b2126 100644 --- a/virt/kvm/kvm_main.c +++ b/virt/kvm/kvm_main.c @@ -2554,8 +2554,9 @@ static bool kvm_pre_set_memory_attributes(struct kvm *kvm, } /* Set @attributes for the gfn range [@start, @end). */ -static int kvm_vm_set_mem_attributes(struct kvm *kvm, gfn_t start, gfn_t end, - unsigned long attributes) +static int kvm_set_mem_attributes(struct kvm *kvm, + struct xarray *mem_attr_array, gfn_t start, + gfn_t end, unsigned long attributes) { struct kvm_mmu_notifier_range pre_set_range = { .start = start, @@ -2569,7 +2570,7 @@ static int kvm_vm_set_mem_attributes(struct kvm *kvm, gfn_t start, gfn_t end, .start = start, .end = end, .arg.attributes = attributes, - .arg.mem_attr_array = &kvm->mem_attr_array, + .arg.mem_attr_array = mem_attr_array, .handler = kvm_arch_post_set_memory_attributes, .on_lock = kvm_mmu_invalidate_end, .may_block = true, @@ -2583,7 +2584,7 @@ static int kvm_vm_set_mem_attributes(struct kvm *kvm, gfn_t start, gfn_t end, mutex_lock(&kvm->slots_lock); /* Nothing to do if the entire range as the desired attributes. */ - if (kvm_range_has_memory_attributes(&kvm->mem_attr_array, start, end, + if (kvm_range_has_memory_attributes(mem_attr_array, start, end, attributes)) goto out_unlock; @@ -2592,7 +2593,7 @@ static int kvm_vm_set_mem_attributes(struct kvm *kvm, gfn_t start, gfn_t end, * partway through setting the new attributes. */ for (i = start; i < end; i++) { - r = xa_reserve(&kvm->mem_attr_array, i, GFP_KERNEL_ACCOUNT); + r = xa_reserve(mem_attr_array, i, GFP_KERNEL_ACCOUNT); if (r) goto out_unlock; } @@ -2600,7 +2601,7 @@ static int kvm_vm_set_mem_attributes(struct kvm *kvm, gfn_t start, gfn_t end, kvm_handle_gfn_range(kvm, &pre_set_range); for (i = start; i < end; i++) { - r = xa_err(xa_store(&kvm->mem_attr_array, i, entry, + r = xa_err(xa_store(mem_attr_array, i, entry, GFP_KERNEL_ACCOUNT)); KVM_BUG_ON(r, kvm); } @@ -2612,15 +2613,17 @@ static int kvm_vm_set_mem_attributes(struct kvm *kvm, gfn_t start, gfn_t end, return r; } -static int kvm_vm_ioctl_set_mem_attributes(struct kvm *kvm, - struct kvm_memory_attributes *attrs) + +int kvm_ioctl_set_mem_attributes(struct kvm *kvm, struct xarray *mem_attr_array, + u64 supported_attrs, + struct kvm_memory_attributes *attrs) { gfn_t start, end; /* flags is currently not used. */ if (attrs->flags) return -EINVAL; - if (attrs->attributes & ~kvm_supported_mem_attributes(kvm)) + if (attrs->attributes & ~supported_attrs) return -EINVAL; if (attrs->size == 0 || attrs->address + attrs->size < attrs->address) return -EINVAL; @@ -2637,7 +2640,16 @@ static int kvm_vm_ioctl_set_mem_attributes(struct kvm *kvm, */ BUILD_BUG_ON(sizeof(attrs->attributes) != sizeof(unsigned long)); - return kvm_vm_set_mem_attributes(kvm, start, end, attrs->attributes); + return kvm_set_mem_attributes(kvm, mem_attr_array, start, end, + attrs->attributes); +} + +static int kvm_vm_ioctl_set_mem_attributes(struct kvm *kvm, + struct kvm_memory_attributes *attrs) +{ + return kvm_ioctl_set_mem_attributes(kvm, &kvm->mem_attr_array, + kvm_supported_mem_attributes(kvm), + attrs); } #endif /* CONFIG_KVM_GENERIC_MEMORY_ATTRIBUTES */ From patchwork Wed Nov 8 11:17:56 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Nicolas Saenz Julienne X-Patchwork-Id: 13449836 Received: from lindbergh.monkeyblade.net (lindbergh.monkeyblade.net [23.128.96.19]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id A972A168CD; Wed, 8 Nov 2023 11:23:09 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=amazon.com header.i=@amazon.com header.b="fx21Mtjt" Received: from smtp-fw-52004.amazon.com (smtp-fw-52004.amazon.com [52.119.213.154]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 5AA8A1FE7; Wed, 8 Nov 2023 03:23:08 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amazon.com; i=@amazon.com; q=dns/txt; s=amazon201209; t=1699442589; x=1730978589; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=c+LzbRrbsSUyvlGgFH27jCwf7iTA/jQxajqqAaH5CaQ=; b=fx21MtjtuEYg4WSnWWtyzoP5MoH0tXFLtnQBIffZJ8BwJx9rNFKxkXqY uCthuyHuyYHVocvqF1+i2tQPqXX346ecbzUbofxrbUEfzzZbprlZ32WPv wFZgRWZ1XrOOheF9kYWirku2+mWbD1Gdht/jFGPjEwKkU+/mL2UUSZ3Mq c=; X-IronPort-AV: E=Sophos;i="6.03,286,1694736000"; d="scan'208";a="164959605" Received: from iad12-co-svc-p1-lb1-vlan2.amazon.com (HELO email-inbound-relay-pdx-2b-m6i4x-cadc3fbd.us-west-2.amazon.com) ([10.43.8.2]) by smtp-border-fw-52004.iad7.amazon.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 08 Nov 2023 11:23:07 +0000 Received: from smtpout.prod.us-west-2.prod.farcaster.email.amazon.dev (pdx2-ws-svc-p26-lb5-vlan2.pdx.amazon.com [10.39.38.66]) by email-inbound-relay-pdx-2b-m6i4x-cadc3fbd.us-west-2.amazon.com (Postfix) with ESMTPS id 1A2F0A12E2; Wed, 8 Nov 2023 11:23:02 +0000 (UTC) Received: from EX19MTAEUA002.ant.amazon.com [10.0.17.79:48574] by smtpin.naws.eu-west-1.prod.farcaster.email.amazon.dev [10.0.32.187:2525] with esmtp (Farcaster) id 78327385-25a6-4125-94a0-25fd575b12d8; Wed, 8 Nov 2023 11:23:01 +0000 (UTC) X-Farcaster-Flow-ID: 78327385-25a6-4125-94a0-25fd575b12d8 Received: from EX19D004EUC001.ant.amazon.com (10.252.51.190) by EX19MTAEUA002.ant.amazon.com (10.252.50.124) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.1118.39; Wed, 8 Nov 2023 11:23:01 +0000 Received: from dev-dsk-nsaenz-1b-189b39ae.eu-west-1.amazon.com (10.13.235.138) by EX19D004EUC001.ant.amazon.com (10.252.51.190) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.1118.39; Wed, 8 Nov 2023 11:22:56 +0000 From: Nicolas Saenz Julienne To: CC: , , , , , , , , , , , , , , , Nicolas Saenz Julienne Subject: [RFC 23/33] KVM: Expose memory attribute helper functions unanimously Date: Wed, 8 Nov 2023 11:17:56 +0000 Message-ID: <20231108111806.92604-24-nsaenz@amazon.com> X-Mailer: git-send-email 2.40.1 In-Reply-To: <20231108111806.92604-1-nsaenz@amazon.com> References: <20231108111806.92604-1-nsaenz@amazon.com> Precedence: bulk X-Mailing-List: kvm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Originating-IP: [10.13.235.138] X-ClientProxiedBy: EX19D033UWC001.ant.amazon.com (10.13.139.218) To EX19D004EUC001.ant.amazon.com (10.252.51.190) Expose memory attribute helper functions even when CONFIG_KVM_GENERIC_MEMORY_ATTRIBUTES is disabled. Other KVM features, like Hyper-V VSM, make use of memory attributes but don't rely on the KVM ioctl. Signed-off-by: Nicolas Saenz Julienne --- arch/x86/kvm/mmu/mmu.c | 2 +- include/linux/kvm_host.h | 2 +- virt/kvm/kvm_main.c | 18 +++++++++--------- 3 files changed, 11 insertions(+), 11 deletions(-) diff --git a/arch/x86/kvm/mmu/mmu.c b/arch/x86/kvm/mmu/mmu.c index c2bec2be2ba9..a76028aa8fb3 100644 --- a/arch/x86/kvm/mmu/mmu.c +++ b/arch/x86/kvm/mmu/mmu.c @@ -7250,7 +7250,6 @@ void kvm_mmu_pre_destroy_vm(struct kvm *kvm) kthread_stop(kvm->arch.nx_huge_page_recovery_thread); } -#ifdef CONFIG_KVM_GENERIC_MEMORY_ATTRIBUTES bool kvm_arch_pre_set_memory_attributes(struct kvm *kvm, struct kvm_gfn_range *range) { @@ -7377,6 +7376,7 @@ bool kvm_arch_post_set_memory_attributes(struct kvm *kvm, return false; } +#ifdef CONFIG_KVM_GENERIC_MEMORY_ATTRIBUTES void kvm_mmu_init_memslot_memory_attributes(struct kvm *kvm, struct kvm_memory_slot *slot) { diff --git a/include/linux/kvm_host.h b/include/linux/kvm_host.h index ad104794037f..45e3e261755d 100644 --- a/include/linux/kvm_host.h +++ b/include/linux/kvm_host.h @@ -2387,7 +2387,6 @@ static inline void kvm_prepare_memory_fault_exit(struct kvm_vcpu *vcpu, vcpu->run->memory_fault.flags |= KVM_MEMORY_EXIT_FLAG_PRIVATE; } -#ifdef CONFIG_KVM_GENERIC_MEMORY_ATTRIBUTES static inline unsigned long kvm_get_memory_attributes(struct xarray *mem_attr_array, gfn_t gfn) { @@ -2404,6 +2403,7 @@ bool kvm_arch_pre_set_memory_attributes(struct kvm *kvm, bool kvm_arch_post_set_memory_attributes(struct kvm *kvm, struct kvm_gfn_range *range); +#ifdef CONFIG_KVM_GENERIC_MEMORY_ATTRIBUTES static inline bool kvm_mem_is_private(struct kvm *kvm, gfn_t gfn) { return IS_ENABLED(CONFIG_KVM_PRIVATE_MEM) && diff --git a/virt/kvm/kvm_main.c b/virt/kvm/kvm_main.c index 74c4c42b2126..b3f4b200f438 100644 --- a/virt/kvm/kvm_main.c +++ b/virt/kvm/kvm_main.c @@ -2435,7 +2435,6 @@ static int kvm_vm_ioctl_clear_dirty_log(struct kvm *kvm, } #endif /* CONFIG_KVM_GENERIC_DIRTYLOG_READ_PROTECT */ -#ifdef CONFIG_KVM_GENERIC_MEMORY_ATTRIBUTES /* * Returns true if _all_ gfns in the range [@start, @end) have attributes * matching @attrs. @@ -2472,14 +2471,6 @@ bool kvm_range_has_memory_attributes(struct xarray *mem_attr_array, gfn_t start, return has_attrs; } -static u64 kvm_supported_mem_attributes(struct kvm *kvm) -{ - if (!kvm || kvm_arch_has_private_mem(kvm)) - return KVM_MEMORY_ATTRIBUTE_PRIVATE; - - return 0; -} - static __always_inline void kvm_handle_gfn_range(struct kvm *kvm, struct kvm_mmu_notifier_range *range) { @@ -2644,6 +2635,15 @@ int kvm_ioctl_set_mem_attributes(struct kvm *kvm, struct xarray *mem_attr_array, attrs->attributes); } +#ifdef CONFIG_KVM_GENERIC_MEMORY_ATTRIBUTES +static u64 kvm_supported_mem_attributes(struct kvm *kvm) +{ + if (!kvm || kvm_arch_has_private_mem(kvm)) + return KVM_MEMORY_ATTRIBUTE_PRIVATE; + + return 0; +} + static int kvm_vm_ioctl_set_mem_attributes(struct kvm *kvm, struct kvm_memory_attributes *attrs) { From patchwork Wed Nov 8 11:17:57 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Nicolas Saenz Julienne X-Patchwork-Id: 13449837 Received: from lindbergh.monkeyblade.net (lindbergh.monkeyblade.net [23.128.96.19]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id D649E171A0; Wed, 8 Nov 2023 11:23:38 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=amazon.com header.i=@amazon.com header.b="dsR9+nlN" Received: from smtp-fw-80008.amazon.com (smtp-fw-80008.amazon.com [99.78.197.219]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 40B892121; Wed, 8 Nov 2023 03:23:38 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amazon.com; i=@amazon.com; q=dns/txt; s=amazon201209; t=1699442618; x=1730978618; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=+clN3KXok8ZiZV9OuN8bZTBQbm7LUM4xz0FNLSqP1yE=; b=dsR9+nlNTvZWeN36ejoNmpatN4/sGj4vb+gYBXXzAUK9DNcGWYxrzcHa 5mFbkC4BobvbVurjLKznBiWxyIDlS1NdtuzwFIjhDafgssbNWdE61ND1T LDMga9lGx/YD2m3KACHUM3y5y8TCyykxkt7CheOxLSVlX2DgSJRQWB7VP U=; X-IronPort-AV: E=Sophos;i="6.03,286,1694736000"; d="scan'208";a="42020483" Received: from pdx4-co-svc-p1-lb2-vlan3.amazon.com (HELO email-inbound-relay-iad-1a-m6i4x-366646a6.us-east-1.amazon.com) ([10.25.36.214]) by smtp-border-fw-80008.pdx80.corp.amazon.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 08 Nov 2023 11:23:34 +0000 Received: from smtpout.prod.us-east-1.prod.farcaster.email.amazon.dev (iad7-ws-svc-p70-lb3-vlan2.iad.amazon.com [10.32.235.34]) by email-inbound-relay-iad-1a-m6i4x-366646a6.us-east-1.amazon.com (Postfix) with ESMTPS id 7D955A685E; Wed, 8 Nov 2023 11:23:31 +0000 (UTC) Received: from EX19MTAEUB001.ant.amazon.com [10.0.17.79:25245] by smtpin.naws.eu-west-1.prod.farcaster.email.amazon.dev [10.0.4.34:2525] with esmtp (Farcaster) id 5d05d07b-01a7-4a6a-845a-8ba989f4a5df; Wed, 8 Nov 2023 11:23:30 +0000 (UTC) X-Farcaster-Flow-ID: 5d05d07b-01a7-4a6a-845a-8ba989f4a5df Received: from EX19D004EUC001.ant.amazon.com (10.252.51.190) by EX19MTAEUB001.ant.amazon.com (10.252.51.28) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.1118.39; Wed, 8 Nov 2023 11:23:30 +0000 Received: from dev-dsk-nsaenz-1b-189b39ae.eu-west-1.amazon.com (10.13.235.138) by EX19D004EUC001.ant.amazon.com (10.252.51.190) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.1118.39; Wed, 8 Nov 2023 11:23:25 +0000 From: Nicolas Saenz Julienne To: CC: , , , , , , , , , , , , , , , Nicolas Saenz Julienne Subject: [RFC 24/33] KVM: x86: hyper-v: Introduce KVM VTL device Date: Wed, 8 Nov 2023 11:17:57 +0000 Message-ID: <20231108111806.92604-25-nsaenz@amazon.com> X-Mailer: git-send-email 2.40.1 In-Reply-To: <20231108111806.92604-1-nsaenz@amazon.com> References: <20231108111806.92604-1-nsaenz@amazon.com> Precedence: bulk X-Mailing-List: kvm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Originating-IP: [10.13.235.138] X-ClientProxiedBy: EX19D035UWB001.ant.amazon.com (10.13.138.33) To EX19D004EUC001.ant.amazon.com (10.252.51.190) Introduce a new KVM device aimed at tracking partition wide VTL state, it'll be the one responsible from keeping track of VTL's memory protections. For now its functionality it's limited, it only exposes its VTL level through a device attribute. Additionally, the device type is only registered if the VSM cap is enabled. Signed-off-by: Nicolas Saenz Julienne --- arch/x86/kvm/hyperv.c | 68 ++++++++++++++++++++++++++++++++++++++++ arch/x86/kvm/hyperv.h | 3 ++ arch/x86/kvm/x86.c | 3 ++ include/uapi/linux/kvm.h | 5 +++ 4 files changed, 79 insertions(+) diff --git a/arch/x86/kvm/hyperv.c b/arch/x86/kvm/hyperv.c index a266c5d393f5..0d8402dba596 100644 --- a/arch/x86/kvm/hyperv.c +++ b/arch/x86/kvm/hyperv.c @@ -3022,3 +3022,71 @@ int kvm_vm_ioctl_get_hv_vsm_state(struct kvm *kvm, struct kvm_hv_vsm_state *stat state->vsm_code_page_offsets = hv->vsm_code_page_offsets.as_u64; return 0; } + +struct kvm_hv_vtl_dev { + int vtl; +}; + +static int kvm_hv_vtl_get_attr(struct kvm_device *dev, + struct kvm_device_attr *attr) +{ + struct kvm_hv_vtl_dev *vtl_dev = dev->private; + + switch (attr->group) { + case KVM_DEV_HV_VTL_GROUP: + switch (attr->attr){ + case KVM_DEV_HV_VTL_GROUP_VTLNUM: + return put_user(vtl_dev->vtl, (u32 __user *)attr->addr); + } + } + + return -EINVAL; +} + +static void kvm_hv_vtl_release(struct kvm_device *dev) +{ + struct kvm_hv_vtl_dev *vtl_dev = dev->private; + + kfree(vtl_dev); + kfree(dev); /* alloc by kvm_ioctl_create_device, free by .release */ +} + +static int kvm_hv_vtl_create(struct kvm_device *dev, u32 type); + +static struct kvm_device_ops kvm_hv_vtl_ops = { + .name = "kvm-hv-vtl", + .create = kvm_hv_vtl_create, + .release = kvm_hv_vtl_release, + .get_attr = kvm_hv_vtl_get_attr, +}; + +static int kvm_hv_vtl_create(struct kvm_device *dev, u32 type) +{ + struct kvm_hv_vtl_dev *vtl_dev; + struct kvm_device *tmp; + int vtl = 0; + + vtl_dev = kzalloc(sizeof(*vtl_dev), GFP_KERNEL_ACCOUNT); + if (!vtl_dev) + return -ENOMEM; + + /* Device creation is protected by kvm->lock */ + list_for_each_entry(tmp, &dev->kvm->devices, vm_node) + if (tmp->ops == &kvm_hv_vtl_ops) + vtl++; + + vtl_dev->vtl = vtl; + dev->private = vtl_dev; + + return 0; +} + +int kvm_hv_vtl_dev_register(void) +{ + return kvm_register_device_ops(&kvm_hv_vtl_ops, KVM_DEV_TYPE_HV_VSM_VTL); +} + +void kvm_hv_vtl_dev_unregister(void) +{ + kvm_unregister_device_ops(KVM_DEV_TYPE_HV_VSM_VTL); +} diff --git a/arch/x86/kvm/hyperv.h b/arch/x86/kvm/hyperv.h index 605e80b9e5eb..3cc664e144d8 100644 --- a/arch/x86/kvm/hyperv.h +++ b/arch/x86/kvm/hyperv.h @@ -269,4 +269,7 @@ static inline void kvm_mmu_role_set_hv_bits(struct kvm_vcpu *vcpu, role->vtl = kvm_hv_get_active_vtl(vcpu); } +int kvm_hv_vtl_dev_register(void); +void kvm_hv_vtl_dev_unregister(void); + #endif diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c index bf4891bc044e..82d3b86d9c93 100644 --- a/arch/x86/kvm/x86.c +++ b/arch/x86/kvm/x86.c @@ -6521,6 +6521,7 @@ int kvm_vm_ioctl_enable_cap(struct kvm *kvm, mutex_unlock(&kvm->lock); break; case KVM_CAP_HYPERV_VSM: + kvm_hv_vtl_dev_register(); kvm->arch.hyperv.hv_enable_vsm = true; r = 0; break; @@ -9675,6 +9676,8 @@ void kvm_x86_vendor_exit(void) mutex_lock(&vendor_module_lock); kvm_x86_ops.hardware_enable = NULL; mutex_unlock(&vendor_module_lock); + + kvm_hv_vtl_dev_unregister(); } EXPORT_SYMBOL_GPL(kvm_x86_vendor_exit); diff --git a/include/uapi/linux/kvm.h b/include/uapi/linux/kvm.h index 0ddffb8b0c99..bd97c9852142 100644 --- a/include/uapi/linux/kvm.h +++ b/include/uapi/linux/kvm.h @@ -1471,6 +1471,9 @@ struct kvm_device_attr { #define KVM_DEV_VFIO_GROUP_DEL KVM_DEV_VFIO_FILE_DEL #define KVM_DEV_VFIO_GROUP_SET_SPAPR_TCE 3 +#define KVM_DEV_HV_VTL_GROUP 1 +#define KVM_DEV_HV_VTL_GROUP_VTLNUM 1 + enum kvm_device_type { KVM_DEV_TYPE_FSL_MPIC_20 = 1, #define KVM_DEV_TYPE_FSL_MPIC_20 KVM_DEV_TYPE_FSL_MPIC_20 @@ -1494,6 +1497,8 @@ enum kvm_device_type { #define KVM_DEV_TYPE_ARM_PV_TIME KVM_DEV_TYPE_ARM_PV_TIME KVM_DEV_TYPE_RISCV_AIA, #define KVM_DEV_TYPE_RISCV_AIA KVM_DEV_TYPE_RISCV_AIA + KVM_DEV_TYPE_HV_VSM_VTL, +#define KVM_DEV_TYPE_HV_VSM_VTL KVM_DEV_TYPE_HV_VSM_VTL KVM_DEV_TYPE_MAX, }; From patchwork Wed Nov 8 11:17:58 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Nicolas Saenz Julienne X-Patchwork-Id: 13449838 Received: from lindbergh.monkeyblade.net (lindbergh.monkeyblade.net [23.128.96.19]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id DBB41171BA; Wed, 8 Nov 2023 11:23:40 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=amazon.com header.i=@amazon.com header.b="i97E7MAB" Received: from smtp-fw-52005.amazon.com (smtp-fw-52005.amazon.com [52.119.213.156]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id BE260212E; Wed, 8 Nov 2023 03:23:39 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amazon.com; i=@amazon.com; q=dns/txt; s=amazon201209; t=1699442620; x=1730978620; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=76x6TGe5VGp4tVKNePmei2SbV9igg1iA9dnaIZiL/jY=; b=i97E7MABqSRQDtyaydCEB1rsc8IE0TjnNqrtGQB1JKMxakIFXmg9HS8g jyFUBdrAZWZXmYZxbYssDPmL4sN6rRYDGpd98gfbqQyLdKnF+9/vPEEoR TBOJVkPyqscHKzBCRwvqr6iUjrR3jar3V9/EfC3amdOV9VW2uN/irtgD3 U=; X-IronPort-AV: E=Sophos;i="6.03,286,1694736000"; d="scan'208";a="614866283" Received: from iad12-co-svc-p1-lb1-vlan3.amazon.com (HELO email-inbound-relay-pdx-2a-m6i4x-83883bdb.us-west-2.amazon.com) ([10.43.8.6]) by smtp-border-fw-52005.iad7.amazon.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 08 Nov 2023 11:23:37 +0000 Received: from smtpout.prod.us-east-1.prod.farcaster.email.amazon.dev (pdx2-ws-svc-p26-lb5-vlan2.pdx.amazon.com [10.39.38.66]) by email-inbound-relay-pdx-2a-m6i4x-83883bdb.us-west-2.amazon.com (Postfix) with ESMTPS id 4CDD76098A; Wed, 8 Nov 2023 11:23:36 +0000 (UTC) Received: from EX19MTAEUB002.ant.amazon.com [10.0.43.254:38586] by smtpin.naws.eu-west-1.prod.farcaster.email.amazon.dev [10.0.17.103:2525] with esmtp (Farcaster) id 9f4672c1-7f67-4ab2-9c06-a6e837041bd2; Wed, 8 Nov 2023 11:23:35 +0000 (UTC) X-Farcaster-Flow-ID: 9f4672c1-7f67-4ab2-9c06-a6e837041bd2 Received: from EX19D004EUC001.ant.amazon.com (10.252.51.190) by EX19MTAEUB002.ant.amazon.com (10.252.51.59) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.1118.39; Wed, 8 Nov 2023 11:23:35 +0000 Received: from dev-dsk-nsaenz-1b-189b39ae.eu-west-1.amazon.com (10.13.235.138) by EX19D004EUC001.ant.amazon.com (10.252.51.190) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.1118.39; Wed, 8 Nov 2023 11:23:30 +0000 From: Nicolas Saenz Julienne To: CC: , , , , , , , , , , , , , , , Nicolas Saenz Julienne Subject: [RFC 25/33] KVM: Introduce a set of new memory attributes Date: Wed, 8 Nov 2023 11:17:58 +0000 Message-ID: <20231108111806.92604-26-nsaenz@amazon.com> X-Mailer: git-send-email 2.40.1 In-Reply-To: <20231108111806.92604-1-nsaenz@amazon.com> References: <20231108111806.92604-1-nsaenz@amazon.com> Precedence: bulk X-Mailing-List: kvm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Originating-IP: [10.13.235.138] X-ClientProxiedBy: EX19D035UWB001.ant.amazon.com (10.13.138.33) To EX19D004EUC001.ant.amazon.com (10.252.51.190) Introduce the following memory attributes: - KVM_MEMORY_ATTRIBUTE_READ - KVM_MEMORY_ATTRIBUTE_WRITE - KVM_MEMORY_ATTRIBUTE_EXECUTE - KVM_MEMORY_ATTRIBUTE_NO_ACCESS Note that NO_ACCESS is necessary in order to make a distinction between the lack of attributes for a gfn, which defaults to the memory protections of the backing memory, versus explicitly prohibiting any access to that gfn. These new memory attributes will, for now, only made be available through the VSM KVM device (which we introduce in subsequent patches). Signed-off-by: Nicolas Saenz Julienne --- include/uapi/linux/kvm.h | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/include/uapi/linux/kvm.h b/include/uapi/linux/kvm.h index bd97c9852142..6b875c1040eb 100644 --- a/include/uapi/linux/kvm.h +++ b/include/uapi/linux/kvm.h @@ -2314,7 +2314,11 @@ struct kvm_memory_attributes { __u64 flags; }; +#define KVM_MEMORY_ATTRIBUTE_READ (1ULL << 0) +#define KVM_MEMORY_ATTRIBUTE_WRITE (1ULL << 1) +#define KVM_MEMORY_ATTRIBUTE_EXECUTE (1ULL << 2) #define KVM_MEMORY_ATTRIBUTE_PRIVATE (1ULL << 3) +#define KVM_MEMORY_ATTRIBUTE_NO_ACCESS (1ULL << 4) #define KVM_CREATE_GUEST_MEMFD _IOWR(KVMIO, 0xd4, struct kvm_create_guest_memfd) From patchwork Wed Nov 8 11:17:59 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Nicolas Saenz Julienne X-Patchwork-Id: 13449839 Received: from lindbergh.monkeyblade.net (lindbergh.monkeyblade.net [23.128.96.19]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 7B1E717990; Wed, 8 Nov 2023 11:23:58 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=amazon.com header.i=@amazon.com header.b="n49ZCsyV" Received: from smtp-fw-52004.amazon.com (smtp-fw-52004.amazon.com [52.119.213.154]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id A11AF1FCE; Wed, 8 Nov 2023 03:23:57 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amazon.com; i=@amazon.com; q=dns/txt; s=amazon201209; t=1699442639; x=1730978639; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=xf82pqS3tbHruxRFe/9qoNTFiZgXupnu5dstVI/jtoA=; b=n49ZCsyVQdBfWID3w3tnCBi3s065Hx05roSFRRi6awKchxMxny7b+b69 HhnWSkpHhcSGMlybeE8G089x52+EEHlLGVd4LeZs4cL/WS55zTRzr6NwB c5+xOnVsQ4SzsfvX1nxSycy2CZGiLr1+fZZMkgEOY4RWbkrUIPtEyEE9E 8=; X-IronPort-AV: E=Sophos;i="6.03,286,1694736000"; d="scan'208";a="164959696" Received: from iad12-co-svc-p1-lb1-vlan2.amazon.com (HELO email-inbound-relay-pdx-2c-m6i4x-94edd59b.us-west-2.amazon.com) ([10.43.8.2]) by smtp-border-fw-52004.iad7.amazon.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 08 Nov 2023 11:23:48 +0000 Received: from smtpout.prod.us-east-1.prod.farcaster.email.amazon.dev (pdx2-ws-svc-p26-lb5-vlan3.pdx.amazon.com [10.39.38.70]) by email-inbound-relay-pdx-2c-m6i4x-94edd59b.us-west-2.amazon.com (Postfix) with ESMTPS id 5BC8140D4F; Wed, 8 Nov 2023 11:23:46 +0000 (UTC) Received: from EX19MTAEUC002.ant.amazon.com [10.0.17.79:21927] by smtpin.naws.eu-west-1.prod.farcaster.email.amazon.dev [10.0.4.34:2525] with esmtp (Farcaster) id c9b4010e-6bfd-4a71-a295-fab821898611; Wed, 8 Nov 2023 11:23:45 +0000 (UTC) X-Farcaster-Flow-ID: c9b4010e-6bfd-4a71-a295-fab821898611 Received: from EX19D004EUC001.ant.amazon.com (10.252.51.190) by EX19MTAEUC002.ant.amazon.com (10.252.51.181) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.1118.39; Wed, 8 Nov 2023 11:23:40 +0000 Received: from dev-dsk-nsaenz-1b-189b39ae.eu-west-1.amazon.com (10.13.235.138) by EX19D004EUC001.ant.amazon.com (10.252.51.190) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.1118.39; Wed, 8 Nov 2023 11:23:35 +0000 From: Nicolas Saenz Julienne To: CC: , , , , , , , , , , , , , , , Nicolas Saenz Julienne Subject: [RFC 26/33] KVM: x86: hyper-vsm: Allow setting per-VTL memory attributes Date: Wed, 8 Nov 2023 11:17:59 +0000 Message-ID: <20231108111806.92604-27-nsaenz@amazon.com> X-Mailer: git-send-email 2.40.1 In-Reply-To: <20231108111806.92604-1-nsaenz@amazon.com> References: <20231108111806.92604-1-nsaenz@amazon.com> Precedence: bulk X-Mailing-List: kvm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Originating-IP: [10.13.235.138] X-ClientProxiedBy: EX19D035UWB001.ant.amazon.com (10.13.138.33) To EX19D004EUC001.ant.amazon.com (10.252.51.190) Introduce KVM_SET_MEMORY_ATTRIBUTES ioctl support for VTL KVM devices. The attributes are stored in an xarray private to the VTL device. The following memory attributes are supported: - KVM_MEMORY_ATTRIBUTE_READ - KVM_MEMORY_ATTRIBUTE_WRITE - KVM_MEMORY_ATTRIBUTE_EXECUTE - KVM_MEMORY_ATTRIBUTE_NO_ACCESS Although only some combinations are valid, see code comment below. Signed-off-by: Nicolas Saenz Julienne --- arch/x86/kvm/hyperv.c | 61 +++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 61 insertions(+) diff --git a/arch/x86/kvm/hyperv.c b/arch/x86/kvm/hyperv.c index 0d8402dba596..bcace0258af1 100644 --- a/arch/x86/kvm/hyperv.c +++ b/arch/x86/kvm/hyperv.c @@ -62,6 +62,10 @@ */ #define HV_EXT_CALL_MAX (HV_EXT_CALL_QUERY_CAPABILITIES + 64) +#define KVM_HV_VTL_ATTRS \ + (KVM_MEMORY_ATTRIBUTE_READ | KVM_MEMORY_ATTRIBUTE_WRITE | \ + KVM_MEMORY_ATTRIBUTE_EXECUTE | KVM_MEMORY_ATTRIBUTE_NO_ACCESS) + static void stimer_mark_pending(struct kvm_vcpu_hv_stimer *stimer, bool vcpu_kick); @@ -3025,6 +3029,7 @@ int kvm_vm_ioctl_get_hv_vsm_state(struct kvm *kvm, struct kvm_hv_vsm_state *stat struct kvm_hv_vtl_dev { int vtl; + struct xarray mem_attrs; }; static int kvm_hv_vtl_get_attr(struct kvm_device *dev, @@ -3047,16 +3052,71 @@ static void kvm_hv_vtl_release(struct kvm_device *dev) { struct kvm_hv_vtl_dev *vtl_dev = dev->private; + xa_destroy(&vtl_dev->mem_attrs); kfree(vtl_dev); kfree(dev); /* alloc by kvm_ioctl_create_device, free by .release */ } +/* + * The TLFS lists the valid memory protection combinations (15.9.3): + * - No access + * - Read-only, no execute + * - Read-only, execute + * - Read/write, no execute + * - Read/write, execute + */ +static bool kvm_hv_validate_vtl_mem_attributes(struct kvm_memory_attributes *attrs) +{ + u64 attr = attrs->attributes; + + if (attr & ~KVM_HV_VTL_ATTRS) + return false; + + if (attr == KVM_MEMORY_ATTRIBUTE_NO_ACCESS) + return true; + + if (!(attr & KVM_MEMORY_ATTRIBUTE_READ)) + return false; + + return true; +} + +static long kvm_hv_vtl_ioctl(struct kvm_device *dev, unsigned int ioctl, + unsigned long arg) +{ + switch (ioctl) { + case KVM_SET_MEMORY_ATTRIBUTES: { + struct kvm_hv_vtl_dev *vtl_dev = dev->private; + struct kvm_memory_attributes attrs; + int r; + + if (copy_from_user(&attrs, (void __user *)arg, sizeof(attrs))) + return -EFAULT; + + r = -EINVAL; + if (!kvm_hv_validate_vtl_mem_attributes(&attrs)) + return r; + + r = kvm_ioctl_set_mem_attributes(dev->kvm, &vtl_dev->mem_attrs, + KVM_HV_VTL_ATTRS, &attrs); + if (r) + return r; + break; + } + default: + return -ENOTTY; + } + + return 0; +} + static int kvm_hv_vtl_create(struct kvm_device *dev, u32 type); static struct kvm_device_ops kvm_hv_vtl_ops = { .name = "kvm-hv-vtl", .create = kvm_hv_vtl_create, .release = kvm_hv_vtl_release, + .ioctl = kvm_hv_vtl_ioctl, .get_attr = kvm_hv_vtl_get_attr, }; @@ -3076,6 +3136,7 @@ static int kvm_hv_vtl_create(struct kvm_device *dev, u32 type) vtl++; vtl_dev->vtl = vtl; + xa_init(&vtl_dev->mem_attrs); dev->private = vtl_dev; return 0; From patchwork Wed Nov 8 11:18:00 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Nicolas Saenz Julienne X-Patchwork-Id: 13449840 Received: from lindbergh.monkeyblade.net (lindbergh.monkeyblade.net [23.128.96.19]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 8046017981; Wed, 8 Nov 2023 11:24:16 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=amazon.com header.i=@amazon.com header.b="C/MdZbLS" Received: from smtp-fw-9105.amazon.com (smtp-fw-9105.amazon.com [207.171.188.204]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id AFF6C212A; Wed, 8 Nov 2023 03:24:15 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amazon.com; i=@amazon.com; q=dns/txt; s=amazon201209; t=1699442656; x=1730978656; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=6ST5pXoQneZXqxqWwZOJM6gjg/ZZDrAnOFjwipV1/Y4=; b=C/MdZbLSlXUQzVOmzjqclHrGAyQn/xkIzbBNFKdvvGB/k4+2AxA7reiU pr4iglofX+uAVbi+x3s2UyCnN8I6aWoJ4I1El7gHWCeljwW1zs5lwWqXj zx3KsYhYmD7in2+9B/ABuEMCHl0/kJKks8kpHZkNMgKSbU5S5JlE91Vv9 A=; X-IronPort-AV: E=Sophos;i="6.03,286,1694736000"; d="scan'208";a="683506250" Received: from pdx4-co-svc-p1-lb2-vlan2.amazon.com (HELO email-inbound-relay-iad-1a-m6i4x-93c3b254.us-east-1.amazon.com) ([10.25.36.210]) by smtp-border-fw-9105.sea19.amazon.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 08 Nov 2023 11:24:15 +0000 Received: from smtpout.prod.us-west-2.prod.farcaster.email.amazon.dev (iad7-ws-svc-p70-lb3-vlan2.iad.amazon.com [10.32.235.34]) by email-inbound-relay-iad-1a-m6i4x-93c3b254.us-east-1.amazon.com (Postfix) with ESMTPS id B2EAAE2099; Wed, 8 Nov 2023 11:24:10 +0000 (UTC) Received: from EX19MTAEUB001.ant.amazon.com [10.0.17.79:64382] by smtpin.naws.eu-west-1.prod.farcaster.email.amazon.dev [10.0.17.103:2525] with esmtp (Farcaster) id fa9eeb09-4794-4c6e-b14d-d42e11167bfe; Wed, 8 Nov 2023 11:24:09 +0000 (UTC) X-Farcaster-Flow-ID: fa9eeb09-4794-4c6e-b14d-d42e11167bfe Received: from EX19D004EUC001.ant.amazon.com (10.252.51.190) by EX19MTAEUB001.ant.amazon.com (10.252.51.28) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.1118.39; Wed, 8 Nov 2023 11:24:09 +0000 Received: from dev-dsk-nsaenz-1b-189b39ae.eu-west-1.amazon.com (10.13.235.138) by EX19D004EUC001.ant.amazon.com (10.252.51.190) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.1118.39; Wed, 8 Nov 2023 11:24:04 +0000 From: Nicolas Saenz Julienne To: CC: , , , , , , , , , , , , , , , Nicolas Saenz Julienne Subject: [RFC 27/33] KVM: x86/mmu/hyper-v: Validate memory faults against per-VTL memprots Date: Wed, 8 Nov 2023 11:18:00 +0000 Message-ID: <20231108111806.92604-28-nsaenz@amazon.com> X-Mailer: git-send-email 2.40.1 In-Reply-To: <20231108111806.92604-1-nsaenz@amazon.com> References: <20231108111806.92604-1-nsaenz@amazon.com> Precedence: bulk X-Mailing-List: kvm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Originating-IP: [10.13.235.138] X-ClientProxiedBy: EX19D031UWC003.ant.amazon.com (10.13.139.252) To EX19D004EUC001.ant.amazon.com (10.252.51.190) Introduce a new step in __kvm_faultin_pfn() that'll validate the fault against the vCPU's VTL protections and generate a user space exit when invalid. Note that kvm_hv_faultin_pfn() has to be run after resolving the fault against the memslots, since that operation steps over 'fault->map_writable'. Non VSM users shouldn't see any behaviour change. Signed-off-by: Nicolas Saenz Julienne --- arch/x86/kvm/hyperv.c | 66 ++++++++++++++++++++++++++++++++++++++++++ arch/x86/kvm/hyperv.h | 1 + arch/x86/kvm/mmu/mmu.c | 9 +++++- 3 files changed, 75 insertions(+), 1 deletion(-) diff --git a/arch/x86/kvm/hyperv.c b/arch/x86/kvm/hyperv.c index bcace0258af1..eb6a4848e306 100644 --- a/arch/x86/kvm/hyperv.c +++ b/arch/x86/kvm/hyperv.c @@ -42,6 +42,8 @@ #include "irq.h" #include "fpu.h" +#include "mmu/mmu_internal.h" + #define KVM_HV_MAX_SPARSE_VCPU_SET_BITS DIV_ROUND_UP(KVM_MAX_VCPUS, HV_VCPUS_PER_SPARSE_BANK) /* @@ -3032,6 +3034,55 @@ struct kvm_hv_vtl_dev { struct xarray mem_attrs; }; +static struct xarray *kvm_hv_vsm_get_memprots(struct kvm_vcpu *vcpu); + +bool kvm_hv_vsm_access_valid(struct kvm_page_fault *fault, unsigned long attrs) +{ + if (attrs == KVM_MEMORY_ATTRIBUTE_NO_ACCESS) + return false; + + /* We should never get here without read permissions, force a fault. */ + if (WARN_ON_ONCE(!(attrs & KVM_MEMORY_ATTRIBUTE_READ))) + return false; + + if (fault->write && !(attrs & KVM_MEMORY_ATTRIBUTE_WRITE)) + return false; + + if (fault->exec && !(attrs & KVM_MEMORY_ATTRIBUTE_EXECUTE)) + return false; + + return true; +} + +static unsigned long kvm_hv_vsm_get_memory_attributes(struct kvm_vcpu *vcpu, + gfn_t gfn) +{ + struct xarray *prots = kvm_hv_vsm_get_memprots(vcpu); + + if (!prots) + return 0; + + return xa_to_value(xa_load(prots, gfn)); +} + +int kvm_hv_faultin_pfn(struct kvm_vcpu *vcpu, struct kvm_page_fault *fault) +{ + unsigned long attrs; + + attrs = kvm_hv_vsm_get_memory_attributes(vcpu, fault->gfn); + if (!attrs) + return RET_PF_CONTINUE; + + if (kvm_hv_vsm_access_valid(fault, attrs)) { + fault->map_executable = + !!(attrs & KVM_MEMORY_ATTRIBUTE_EXECUTE); + fault->map_writable = !!(attrs & KVM_MEMORY_ATTRIBUTE_WRITE); + return RET_PF_CONTINUE; + } + + return -EFAULT; +} + static int kvm_hv_vtl_get_attr(struct kvm_device *dev, struct kvm_device_attr *attr) { @@ -3120,6 +3171,21 @@ static struct kvm_device_ops kvm_hv_vtl_ops = { .get_attr = kvm_hv_vtl_get_attr, }; +static struct xarray *kvm_hv_vsm_get_memprots(struct kvm_vcpu *vcpu) +{ + struct kvm_hv_vtl_dev *vtl_dev; + struct kvm_device *tmp; + + list_for_each_entry(tmp, &vcpu->kvm->devices, vm_node) + if (tmp->ops == &kvm_hv_vtl_ops) { + vtl_dev = tmp->private; + if (vtl_dev->vtl == kvm_hv_get_active_vtl(vcpu)) + return &vtl_dev->mem_attrs; + } + + return NULL; +} + static int kvm_hv_vtl_create(struct kvm_device *dev, u32 type) { struct kvm_hv_vtl_dev *vtl_dev; diff --git a/arch/x86/kvm/hyperv.h b/arch/x86/kvm/hyperv.h index 3cc664e144d8..ae781b4d4669 100644 --- a/arch/x86/kvm/hyperv.h +++ b/arch/x86/kvm/hyperv.h @@ -271,5 +271,6 @@ static inline void kvm_mmu_role_set_hv_bits(struct kvm_vcpu *vcpu, int kvm_hv_vtl_dev_register(void); void kvm_hv_vtl_dev_unregister(void); +int kvm_hv_faultin_pfn(struct kvm_vcpu *vcpu, struct kvm_page_fault *fault); #endif diff --git a/arch/x86/kvm/mmu/mmu.c b/arch/x86/kvm/mmu/mmu.c index a76028aa8fb3..ba454c7277dc 100644 --- a/arch/x86/kvm/mmu/mmu.c +++ b/arch/x86/kvm/mmu/mmu.c @@ -4374,7 +4374,7 @@ static int __kvm_faultin_pfn(struct kvm_vcpu *vcpu, struct kvm_page_fault *fault fault->write, &fault->map_writable, &fault->hva); if (!async) - return RET_PF_CONTINUE; /* *pfn has correct page already */ + goto pf_continue; /* *pfn has correct page already */ if (!fault->prefetch && kvm_can_do_async_pf(vcpu)) { trace_kvm_try_async_get_page(fault->addr, fault->gfn); @@ -4395,6 +4395,13 @@ static int __kvm_faultin_pfn(struct kvm_vcpu *vcpu, struct kvm_page_fault *fault fault->pfn = __gfn_to_pfn_memslot(slot, fault->gfn, false, true, NULL, fault->write, &fault->map_writable, &fault->hva); +pf_continue: + if (kvm_hv_vsm_enabled(vcpu->kvm)) { + if (kvm_hv_faultin_pfn(vcpu, fault)) { + kvm_mmu_prepare_memory_fault_exit(vcpu, fault); + return -EFAULT; + } + } return RET_PF_CONTINUE; } From patchwork Wed Nov 8 11:18:01 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Nicolas Saenz Julienne X-Patchwork-Id: 13449841 Received: from lindbergh.monkeyblade.net (lindbergh.monkeyblade.net [23.128.96.19]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id E0B2417981; Wed, 8 Nov 2023 11:24:23 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=amazon.com header.i=@amazon.com header.b="NjJluvCs" Received: from smtp-fw-80009.amazon.com (smtp-fw-80009.amazon.com [99.78.197.220]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 7AB311BD5; Wed, 8 Nov 2023 03:24:23 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amazon.com; i=@amazon.com; q=dns/txt; s=amazon201209; t=1699442663; x=1730978663; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=wLo3D2RUMSrAbuSqouFk5tJc54p3vvp1g/yHmuVwsBM=; b=NjJluvCsCOrrryqvOqaerCThOkjj/PpkYw8LvGx1ve233rFh0+IfCpxW PoywF4AxUXWtszrYkNj8fgDp11qt5QGVFhaP+rxzf/7mmr29gIJb55apl 9ftHBAnSgUSRs7U+e9gUQRS9VRy/q9gseAHzWCywm08lTRdbl9HSp9/do s=; X-IronPort-AV: E=Sophos;i="6.03,286,1694736000"; d="scan'208";a="41962461" Received: from pdx4-co-svc-p1-lb2-vlan2.amazon.com (HELO email-inbound-relay-iad-1d-m6i4x-00fceed5.us-east-1.amazon.com) ([10.25.36.210]) by smtp-border-fw-80009.pdx80.corp.amazon.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 08 Nov 2023 11:24:19 +0000 Received: from smtpout.prod.us-west-2.prod.farcaster.email.amazon.dev (iad7-ws-svc-p70-lb3-vlan3.iad.amazon.com [10.32.235.38]) by email-inbound-relay-iad-1d-m6i4x-00fceed5.us-east-1.amazon.com (Postfix) with ESMTPS id 8D412A0DAB; Wed, 8 Nov 2023 11:24:15 +0000 (UTC) Received: from EX19MTAEUC002.ant.amazon.com [10.0.17.79:9042] by smtpin.naws.eu-west-1.prod.farcaster.email.amazon.dev [10.0.26.101:2525] with esmtp (Farcaster) id ed9999cc-88dc-4223-ba6e-3219e585d98f; Wed, 8 Nov 2023 11:24:14 +0000 (UTC) X-Farcaster-Flow-ID: ed9999cc-88dc-4223-ba6e-3219e585d98f Received: from EX19D004EUC001.ant.amazon.com (10.252.51.190) by EX19MTAEUC002.ant.amazon.com (10.252.51.181) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.1118.39; Wed, 8 Nov 2023 11:24:14 +0000 Received: from dev-dsk-nsaenz-1b-189b39ae.eu-west-1.amazon.com (10.13.235.138) by EX19D004EUC001.ant.amazon.com (10.252.51.190) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.1118.39; Wed, 8 Nov 2023 11:24:09 +0000 From: Nicolas Saenz Julienne To: CC: , , , , , , , , , , , , , , , Nicolas Saenz Julienne Subject: [RFC 28/33] x86/hyper-v: Introduce memory intercept message structure Date: Wed, 8 Nov 2023 11:18:01 +0000 Message-ID: <20231108111806.92604-29-nsaenz@amazon.com> X-Mailer: git-send-email 2.40.1 In-Reply-To: <20231108111806.92604-1-nsaenz@amazon.com> References: <20231108111806.92604-1-nsaenz@amazon.com> Precedence: bulk X-Mailing-List: kvm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Originating-IP: [10.13.235.138] X-ClientProxiedBy: EX19D031UWC003.ant.amazon.com (10.13.139.252) To EX19D004EUC001.ant.amazon.com (10.252.51.190) Introduce struct hv_memory_intercept_message, which is used when issuing memory intercepts to a Hyper-V VSM guest. Signed-off-by: Nicolas Saenz Julienne --- arch/x86/include/asm/hyperv-tlfs.h | 76 ++++++++++++++++++++++++++++++ 1 file changed, 76 insertions(+) diff --git a/arch/x86/include/asm/hyperv-tlfs.h b/arch/x86/include/asm/hyperv-tlfs.h index af594aa65307..d3d74fde6da1 100644 --- a/arch/x86/include/asm/hyperv-tlfs.h +++ b/arch/x86/include/asm/hyperv-tlfs.h @@ -799,6 +799,82 @@ struct hv_get_vp_from_apic_id_in { u32 apic_ids[]; } __packed; + +/* struct hv_intercept_header::access_type_mask */ +#define HV_INTERCEPT_ACCESS_MASK_NONE 0 +#define HV_INTERCEPT_ACCESS_MASK_READ 1 +#define HV_INTERCEPT_ACCESS_MASK_WRITE 2 +#define HV_INTERCEPT_ACCESS_MASK_EXECUTE 4 + +/* struct hv_intercept_exception::cache_type */ +#define HV_X64_CACHE_TYPE_UNCACHED 0 +#define HV_X64_CACHE_TYPE_WRITECOMBINING 1 +#define HV_X64_CACHE_TYPE_WRITETHROUGH 4 +#define HV_X64_CACHE_TYPE_WRITEPROTECTED 5 +#define HV_X64_CACHE_TYPE_WRITEBACK 6 + +/* Intecept message header */ +struct hv_intercept_header { + __u32 vp_index; + __u8 instruction_length; +#define HV_INTERCEPT_ACCESS_READ 0 +#define HV_INTERCEPT_ACCESS_WRITE 1 +#define HV_INTERCEPT_ACCESS_EXECUTE 2 + __u8 access_type_mask; + union { + __u16 as_u16; + struct { + __u16 cpl:2; + __u16 cr0_pe:1; + __u16 cr0_am:1; + __u16 efer_lma:1; + __u16 debug_active:1; + __u16 interruption_pending:1; + __u16 reserved:9; + }; + } exec_state; + struct hv_x64_segment_register cs; + __u64 rip; + __u64 rflags; +} __packed; + +union hv_x64_memory_access_info { + __u8 as_u8; + struct { + __u8 gva_valid:1; + __u8 _reserved:7; + }; +}; + +struct hv_memory_intercept_message { + struct hv_intercept_header header; + __u32 cache_type; + __u8 instruction_byte_count; + union hv_x64_memory_access_info memory_access_info; + __u16 _reserved; + __u64 gva; + __u64 gpa; + __u8 instruction_bytes[16]; + struct hv_x64_segment_register ds; + struct hv_x64_segment_register ss; + __u64 rax; + __u64 rcx; + __u64 rdx; + __u64 rbx; + __u64 rsp; + __u64 rbp; + __u64 rsi; + __u64 rdi; + __u64 r8; + __u64 r9; + __u64 r10; + __u64 r11; + __u64 r12; + __u64 r13; + __u64 r14; + __u64 r15; +} __packed; + #include #endif From patchwork Wed Nov 8 11:18:02 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Nicolas Saenz Julienne X-Patchwork-Id: 13449842 Received: from lindbergh.monkeyblade.net (lindbergh.monkeyblade.net [23.128.96.19]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 79497179AA; Wed, 8 Nov 2023 11:24:26 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=amazon.com header.i=@amazon.com header.b="lRDOzF+2" Received: from smtp-fw-52003.amazon.com (smtp-fw-52003.amazon.com [52.119.213.152]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id C4A871FC2; Wed, 8 Nov 2023 03:24:25 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amazon.com; i=@amazon.com; q=dns/txt; s=amazon201209; t=1699442667; x=1730978667; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=r3WKjqld4qql+FLcnzKJq/fsWtORZpLYW1PJ8L9F7HI=; b=lRDOzF+2/uSWtu6g6xiZPogGFIFZOEPhkU35BDMtOA7U8brSCvQ0mnZX jD8hivVkW40ZS2DOg3FS40/NOB5hG3A4n2gQbMg/ocMKnrcgInsI44svG MLXztXFUWmKvEOTR2olymytYWaNwqvRsY2Z8rlBbsWNEtaQ+AKlJonGxq w=; X-IronPort-AV: E=Sophos;i="6.03,286,1694736000"; d="scan'208";a="618316712" Received: from iad12-co-svc-p1-lb1-vlan3.amazon.com (HELO email-inbound-relay-iad-1a-m6i4x-54a853e6.us-east-1.amazon.com) ([10.43.8.6]) by smtp-border-fw-52003.iad7.amazon.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 08 Nov 2023 11:24:26 +0000 Received: from smtpout.prod.us-east-1.prod.farcaster.email.amazon.dev (iad7-ws-svc-p70-lb3-vlan3.iad.amazon.com [10.32.235.38]) by email-inbound-relay-iad-1a-m6i4x-54a853e6.us-east-1.amazon.com (Postfix) with ESMTPS id 2BB5348ECD; Wed, 8 Nov 2023 11:24:21 +0000 (UTC) Received: from EX19MTAEUC001.ant.amazon.com [10.0.10.100:34530] by smtpin.naws.eu-west-1.prod.farcaster.email.amazon.dev [10.0.22.222:2525] with esmtp (Farcaster) id ada2ae65-467f-4b1a-8aba-0d7dc7fba03f; Wed, 8 Nov 2023 11:24:21 +0000 (UTC) X-Farcaster-Flow-ID: ada2ae65-467f-4b1a-8aba-0d7dc7fba03f Received: from EX19D004EUC001.ant.amazon.com (10.252.51.190) by EX19MTAEUC001.ant.amazon.com (10.252.51.155) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.1118.39; Wed, 8 Nov 2023 11:24:19 +0000 Received: from dev-dsk-nsaenz-1b-189b39ae.eu-west-1.amazon.com (10.13.235.138) by EX19D004EUC001.ant.amazon.com (10.252.51.190) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.1118.39; Wed, 8 Nov 2023 11:24:14 +0000 From: Nicolas Saenz Julienne To: CC: , , , , , , , , , , , , , , , Nicolas Saenz Julienne Subject: [RFC 29/33] KVM: VMX: Save instruction length on EPT violation Date: Wed, 8 Nov 2023 11:18:02 +0000 Message-ID: <20231108111806.92604-30-nsaenz@amazon.com> X-Mailer: git-send-email 2.40.1 In-Reply-To: <20231108111806.92604-1-nsaenz@amazon.com> References: <20231108111806.92604-1-nsaenz@amazon.com> Precedence: bulk X-Mailing-List: kvm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Originating-IP: [10.13.235.138] X-ClientProxiedBy: EX19D031UWC003.ant.amazon.com (10.13.139.252) To EX19D004EUC001.ant.amazon.com (10.252.51.190) Save the length of the instruction that triggered an EPT violation in struct kvm_vcpu_arch. This will be used to populate Hyper-V VSM memory intercept messages. Signed-off-by: Nicolas Saenz Julienne --- arch/x86/include/asm/kvm_host.h | 2 ++ arch/x86/kvm/vmx/vmx.c | 1 + 2 files changed, 3 insertions(+) diff --git a/arch/x86/include/asm/kvm_host.h b/arch/x86/include/asm/kvm_host.h index 1f5a85d461ce..1a854776d91e 100644 --- a/arch/x86/include/asm/kvm_host.h +++ b/arch/x86/include/asm/kvm_host.h @@ -967,6 +967,8 @@ struct kvm_vcpu_arch { /* set at EPT violation at this point */ unsigned long exit_qualification; + u32 exit_instruction_len; + /* pv related host specific info */ struct { bool pv_unhalted; diff --git a/arch/x86/kvm/vmx/vmx.c b/arch/x86/kvm/vmx/vmx.c index 6e502ba93141..9c83ee3a293d 100644 --- a/arch/x86/kvm/vmx/vmx.c +++ b/arch/x86/kvm/vmx/vmx.c @@ -5773,6 +5773,7 @@ static int handle_ept_violation(struct kvm_vcpu *vcpu) PFERR_GUEST_FINAL_MASK : PFERR_GUEST_PAGE_MASK; vcpu->arch.exit_qualification = exit_qualification; + vcpu->arch.exit_instruction_len = vmcs_read32(VM_EXIT_INSTRUCTION_LEN); /* * Check that the GPA doesn't exceed physical memory limits, as that is From patchwork Wed Nov 8 11:18:03 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Nicolas Saenz Julienne X-Patchwork-Id: 13449843 Received: from lindbergh.monkeyblade.net (lindbergh.monkeyblade.net [23.128.96.19]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id D6A24182A3; Wed, 8 Nov 2023 11:24:52 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=amazon.com header.i=@amazon.com header.b="fxtmLd6L" Received: from smtp-fw-6002.amazon.com (smtp-fw-6002.amazon.com [52.95.49.90]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id E66241FF3; Wed, 8 Nov 2023 03:24:51 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amazon.com; i=@amazon.com; q=dns/txt; s=amazon201209; t=1699442693; x=1730978693; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=5PF4/rNtaEl02NErsVfK5XTs+FbISN0CKbNnl5QOpHE=; b=fxtmLd6LCgEgxifWlx4X+luOWu83oDDvV+HeLZN+iSlz/iVZizDQHiC3 TDLOyCfLK1yyvDhGqktCP8G3daYWMXMlJgLiO2FByVu21L8uLcF+N9qwn 8sS1HpyoArvRIl35XeT5zU79HC0eP1NYcSdv7oJTMOI6k2rEWtc2lW+uZ 4=; X-IronPort-AV: E=Sophos;i="6.03,286,1694736000"; d="scan'208";a="366813194" Received: from iad12-co-svc-p1-lb1-vlan3.amazon.com (HELO email-inbound-relay-pdx-2c-m6i4x-e7094f15.us-west-2.amazon.com) ([10.43.8.6]) by smtp-border-fw-6002.iad6.amazon.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 08 Nov 2023 11:24:50 +0000 Received: from smtpout.prod.us-west-2.prod.farcaster.email.amazon.dev (pdx2-ws-svc-p26-lb5-vlan3.pdx.amazon.com [10.39.38.70]) by email-inbound-relay-pdx-2c-m6i4x-e7094f15.us-west-2.amazon.com (Postfix) with ESMTPS id 111B540E63; Wed, 8 Nov 2023 11:24:49 +0000 (UTC) Received: from EX19MTAEUB001.ant.amazon.com [10.0.17.79:18294] by smtpin.naws.eu-west-1.prod.farcaster.email.amazon.dev [10.0.32.187:2525] with esmtp (Farcaster) id 804134a2-3f2d-4fb3-a228-3f0ad905b402; Wed, 8 Nov 2023 11:24:47 +0000 (UTC) X-Farcaster-Flow-ID: 804134a2-3f2d-4fb3-a228-3f0ad905b402 Received: from EX19D004EUC001.ant.amazon.com (10.252.51.190) by EX19MTAEUB001.ant.amazon.com (10.252.51.28) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.1118.39; Wed, 8 Nov 2023 11:24:47 +0000 Received: from dev-dsk-nsaenz-1b-189b39ae.eu-west-1.amazon.com (10.13.235.138) by EX19D004EUC001.ant.amazon.com (10.252.51.190) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.1118.39; Wed, 8 Nov 2023 11:24:42 +0000 From: Nicolas Saenz Julienne To: CC: , , , , , , , , , , , , , , , Nicolas Saenz Julienne Subject: [RFC 30/33] KVM: x86: hyper-v: Introduce KVM_REQ_HV_INJECT_INTERCEPT request Date: Wed, 8 Nov 2023 11:18:03 +0000 Message-ID: <20231108111806.92604-31-nsaenz@amazon.com> X-Mailer: git-send-email 2.40.1 In-Reply-To: <20231108111806.92604-1-nsaenz@amazon.com> References: <20231108111806.92604-1-nsaenz@amazon.com> Precedence: bulk X-Mailing-List: kvm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Originating-IP: [10.13.235.138] X-ClientProxiedBy: EX19D032UWB003.ant.amazon.com (10.13.139.165) To EX19D004EUC001.ant.amazon.com (10.252.51.190) Introduce a new request type, KVM_REQ_HV_INJECT_INTERCEPT which allows injecting out-of-band Hyper-V secure intercepts. For now only memory access intercepts are supported. These are triggered when access a GPA protected by a higher VTL. The memory intercept metadata is filled based on the GPA provided through struct kvm_vcpu_hv_intercept_info, and injected into the guest through SynIC message. Signed-off-by: Nicolas Saenz Julienne --- arch/x86/include/asm/kvm_host.h | 10 +++ arch/x86/kvm/hyperv.c | 114 ++++++++++++++++++++++++++++++++ arch/x86/kvm/hyperv.h | 2 + arch/x86/kvm/x86.c | 3 + 4 files changed, 129 insertions(+) diff --git a/arch/x86/include/asm/kvm_host.h b/arch/x86/include/asm/kvm_host.h index 1a854776d91e..39671e075555 100644 --- a/arch/x86/include/asm/kvm_host.h +++ b/arch/x86/include/asm/kvm_host.h @@ -113,6 +113,7 @@ KVM_ARCH_REQ_FLAGS(31, KVM_REQUEST_WAIT | KVM_REQUEST_NO_WAKEUP) #define KVM_REQ_HV_TLB_FLUSH \ KVM_ARCH_REQ_FLAGS(32, KVM_REQUEST_WAIT | KVM_REQUEST_NO_WAKEUP) +#define KVM_REQ_HV_INJECT_INTERCEPT KVM_ARCH_REQ(33) #define CR0_RESERVED_BITS \ (~(unsigned long)(X86_CR0_PE | X86_CR0_MP | X86_CR0_EM | X86_CR0_TS \ @@ -639,6 +640,13 @@ struct kvm_vcpu_hv_tlb_flush_fifo { DECLARE_KFIFO(entries, u64, KVM_HV_TLB_FLUSH_FIFO_SIZE); }; +struct kvm_vcpu_hv_intercept_info { + struct kvm_vcpu *vcpu; + int type; + u64 gpa; + u8 access; +}; + /* Hyper-V per vcpu emulation context */ struct kvm_vcpu_hv { struct kvm_vcpu *vcpu; @@ -673,6 +681,8 @@ struct kvm_vcpu_hv { u64 vm_id; u32 vp_id; } nested; + + struct kvm_vcpu_hv_intercept_info intercept_info; }; struct kvm_hypervisor_cpuid { diff --git a/arch/x86/kvm/hyperv.c b/arch/x86/kvm/hyperv.c index eb6a4848e306..38ee3abdef9c 100644 --- a/arch/x86/kvm/hyperv.c +++ b/arch/x86/kvm/hyperv.c @@ -2789,6 +2789,120 @@ int kvm_hv_hypercall(struct kvm_vcpu *vcpu) return 0; } +static void store_kvm_segment(const struct kvm_segment *kvmseg, + struct hv_x64_segment_register *reg) +{ + reg->base = kvmseg->base; + reg->limit = kvmseg->limit; + reg->selector = kvmseg->selector; + reg->segment_type = kvmseg->type; + reg->present = kvmseg->present; + reg->descriptor_privilege_level = kvmseg->dpl; + reg->_default = kvmseg->db; + reg->non_system_segment = kvmseg->s; + reg->_long = kvmseg->l; + reg->granularity = kvmseg->g; + reg->available = kvmseg->avl; +} + +static void deliver_gpa_intercept(struct kvm_vcpu *target_vcpu, + struct kvm_vcpu *intercepted_vcpu, u64 gpa, + u64 gva, u8 access_type_mask) +{ + ulong cr0; + struct hv_message msg = { 0 }; + struct hv_memory_intercept_message *intercept = (struct hv_memory_intercept_message *)msg.u.payload; + struct kvm_vcpu_hv *hv_vcpu = to_hv_vcpu(target_vcpu); + struct x86_exception e; + struct kvm_segment kvmseg; + + msg.header.message_type = HVMSG_GPA_INTERCEPT; + msg.header.payload_size = sizeof(*intercept); + + intercept->header.vp_index = to_hv_vcpu(intercepted_vcpu)->vp_index; + intercept->header.instruction_length = intercepted_vcpu->arch.exit_instruction_len; + intercept->header.access_type_mask = access_type_mask; + kvm_x86_ops.get_segment(intercepted_vcpu, &kvmseg, VCPU_SREG_CS); + store_kvm_segment(&kvmseg, &intercept->header.cs); + + cr0 = kvm_read_cr0(intercepted_vcpu); + intercept->header.exec_state.cr0_pe = (cr0 & X86_CR0_PE); + intercept->header.exec_state.cr0_am = (cr0 & X86_CR0_AM); + intercept->header.exec_state.cpl = kvm_x86_ops.get_cpl(intercepted_vcpu); + intercept->header.exec_state.efer_lma = is_long_mode(intercepted_vcpu); + intercept->header.exec_state.debug_active = 0; + intercept->header.exec_state.interruption_pending = 0; + intercept->header.rip = kvm_rip_read(intercepted_vcpu); + intercept->header.rflags = kvm_get_rflags(intercepted_vcpu); + + /* + * For exec violations we don't have a way to decode an instruction that issued a fetch + * to a non-X page because CPU points RIP and GPA to the fetch destination in the faulted page. + * Instruction length though is the length of the fetch source. + * Seems like Hyper-V is aware of that and is not trying to access those fields. + */ + if (access_type_mask == HV_INTERCEPT_ACCESS_EXECUTE) { + intercept->instruction_byte_count = 0; + } else { + intercept->instruction_byte_count = intercepted_vcpu->arch.exit_instruction_len; + if (intercept->instruction_byte_count > sizeof(intercept->instruction_bytes)) + intercept->instruction_byte_count = sizeof(intercept->instruction_bytes); + if (kvm_read_guest_virt(intercepted_vcpu, + kvm_rip_read(intercepted_vcpu), + intercept->instruction_bytes, + intercept->instruction_byte_count, &e)) + goto inject_ud; + } + + intercept->memory_access_info.gva_valid = (gva != 0); + intercept->gva = gva; + intercept->gpa = gpa; + intercept->cache_type = HV_X64_CACHE_TYPE_WRITEBACK; + kvm_x86_ops.get_segment(intercepted_vcpu, &kvmseg, VCPU_SREG_DS); + store_kvm_segment(&kvmseg, &intercept->ds); + kvm_x86_ops.get_segment(intercepted_vcpu, &kvmseg, VCPU_SREG_SS); + store_kvm_segment(&kvmseg, &intercept->ss); + intercept->rax = kvm_rax_read(intercepted_vcpu); + intercept->rcx = kvm_rcx_read(intercepted_vcpu); + intercept->rdx = kvm_rdx_read(intercepted_vcpu); + intercept->rbx = kvm_rbx_read(intercepted_vcpu); + intercept->rsp = kvm_rsp_read(intercepted_vcpu); + intercept->rbp = kvm_rbp_read(intercepted_vcpu); + intercept->rsi = kvm_rsi_read(intercepted_vcpu); + intercept->rdi = kvm_rdi_read(intercepted_vcpu); + intercept->r8 = kvm_r8_read(intercepted_vcpu); + intercept->r9 = kvm_r9_read(intercepted_vcpu); + intercept->r10 = kvm_r10_read(intercepted_vcpu); + intercept->r11 = kvm_r11_read(intercepted_vcpu); + intercept->r12 = kvm_r12_read(intercepted_vcpu); + intercept->r13 = kvm_r13_read(intercepted_vcpu); + intercept->r14 = kvm_r14_read(intercepted_vcpu); + intercept->r15 = kvm_r15_read(intercepted_vcpu); + + if (synic_deliver_msg(&hv_vcpu->synic, 0, &msg, true)) + goto inject_ud; + + return; + +inject_ud: + kvm_queue_exception(target_vcpu, UD_VECTOR); +} + +void kvm_hv_deliver_intercept(struct kvm_vcpu *vcpu) +{ + struct kvm_vcpu_hv_intercept_info *info = &to_hv_vcpu(vcpu)->intercept_info; + + switch (info->type) { + case HVMSG_GPA_INTERCEPT: + deliver_gpa_intercept(vcpu, info->vcpu, info->gpa, 0, + info->access); + break; + default: + pr_warn("Unknown exception\n"); + } +} +EXPORT_SYMBOL_GPL(kvm_hv_deliver_intercept); + void kvm_hv_init_vm(struct kvm *kvm) { struct kvm_hv *hv = to_kvm_hv(kvm); diff --git a/arch/x86/kvm/hyperv.h b/arch/x86/kvm/hyperv.h index ae781b4d4669..8efc4916e0cb 100644 --- a/arch/x86/kvm/hyperv.h +++ b/arch/x86/kvm/hyperv.h @@ -273,4 +273,6 @@ int kvm_hv_vtl_dev_register(void); void kvm_hv_vtl_dev_unregister(void); int kvm_hv_faultin_pfn(struct kvm_vcpu *vcpu, struct kvm_page_fault *fault); +void kvm_hv_deliver_intercept(struct kvm_vcpu *vcpu); + #endif diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c index 82d3b86d9c93..f2581eec39a9 100644 --- a/arch/x86/kvm/x86.c +++ b/arch/x86/kvm/x86.c @@ -10707,6 +10707,9 @@ static int vcpu_enter_guest(struct kvm_vcpu *vcpu) if (kvm_check_request(KVM_REQ_UPDATE_CPU_DIRTY_LOGGING, vcpu)) static_call(kvm_x86_update_cpu_dirty_logging)(vcpu); + + if (kvm_check_request(KVM_REQ_HV_INJECT_INTERCEPT, vcpu)) + kvm_hv_deliver_intercept(vcpu); } if (kvm_check_request(KVM_REQ_EVENT, vcpu) || req_int_win || From patchwork Wed Nov 8 11:18:04 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Nicolas Saenz Julienne X-Patchwork-Id: 13449844 Received: from lindbergh.monkeyblade.net (lindbergh.monkeyblade.net [23.128.96.19]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id AC14F1865A; Wed, 8 Nov 2023 11:24:57 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=amazon.com header.i=@amazon.com header.b="Z/YNBkoS" Received: from smtp-fw-52005.amazon.com (smtp-fw-52005.amazon.com [52.119.213.156]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id CD7381BF5; Wed, 8 Nov 2023 03:24:56 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amazon.com; i=@amazon.com; q=dns/txt; s=amazon201209; t=1699442697; x=1730978697; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=hHT68F7U7dwhmAs9bVBPKto3xQgEmy+Yqg+nDVOZ3bk=; b=Z/YNBkoSDl3C+MQdSyj0WVOqBtScLnHFTYLoa/mgxst4T3XouSIqXDY7 08g7AF1/ysE6OdUdRt+IidJ/ZqoU6FjJiLUuSTm6iCJrIUEnTY+9lYI31 33iAME9kYq483k9N/5zJOejtipoiwZEIG8ED5QAyU0IJ3ZVm8efotJoo3 c=; X-IronPort-AV: E=Sophos;i="6.03,286,1694736000"; d="scan'208";a="614866460" Received: from iad12-co-svc-p1-lb1-vlan3.amazon.com (HELO email-inbound-relay-pdx-1box-2bm6-32cf6363.us-west-2.amazon.com) ([10.43.8.6]) by smtp-border-fw-52005.iad7.amazon.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 08 Nov 2023 11:24:55 +0000 Received: from smtpout.prod.us-west-2.prod.farcaster.email.amazon.dev (pdx2-ws-svc-p26-lb5-vlan2.pdx.amazon.com [10.39.38.66]) by email-inbound-relay-pdx-1box-2bm6-32cf6363.us-west-2.amazon.com (Postfix) with ESMTPS id 14BD280674; Wed, 8 Nov 2023 11:24:54 +0000 (UTC) Received: from EX19MTAEUB002.ant.amazon.com [10.0.43.254:55642] by smtpin.naws.eu-west-1.prod.farcaster.email.amazon.dev [10.0.26.101:2525] with esmtp (Farcaster) id b825c34a-ac0f-48ab-b5f4-78e61c5f00c9; Wed, 8 Nov 2023 11:24:53 +0000 (UTC) X-Farcaster-Flow-ID: b825c34a-ac0f-48ab-b5f4-78e61c5f00c9 Received: from EX19D004EUC001.ant.amazon.com (10.252.51.190) by EX19MTAEUB002.ant.amazon.com (10.252.51.59) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.1118.39; Wed, 8 Nov 2023 11:24:52 +0000 Received: from dev-dsk-nsaenz-1b-189b39ae.eu-west-1.amazon.com (10.13.235.138) by EX19D004EUC001.ant.amazon.com (10.252.51.190) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.1118.39; Wed, 8 Nov 2023 11:24:47 +0000 From: Nicolas Saenz Julienne To: CC: , , , , , , , , , , , , , , , Nicolas Saenz Julienne Subject: [RFC 31/33] KVM: x86: hyper-v: Inject intercept on VTL memory protection fault Date: Wed, 8 Nov 2023 11:18:04 +0000 Message-ID: <20231108111806.92604-32-nsaenz@amazon.com> X-Mailer: git-send-email 2.40.1 In-Reply-To: <20231108111806.92604-1-nsaenz@amazon.com> References: <20231108111806.92604-1-nsaenz@amazon.com> Precedence: bulk X-Mailing-List: kvm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Originating-IP: [10.13.235.138] X-ClientProxiedBy: EX19D032UWB003.ant.amazon.com (10.13.139.165) To EX19D004EUC001.ant.amazon.com (10.252.51.190) Inject a Hyper-V secure intercept when a VTL tries to access memory that was protected by a more privileged VTL. The intercept is injected into the next enabled privileged VTL (for now, this patch takes a shortcut and assumes it's the one right after). After injecting the request, the KVM vCPU that took the fault will exit to user-space with a memory fault. Signed-off-by: Nicolas Saenz Julienne --- arch/x86/kvm/hyperv.c | 27 +++++++++++++++++++++++++++ 1 file changed, 27 insertions(+) diff --git a/arch/x86/kvm/hyperv.c b/arch/x86/kvm/hyperv.c index 38ee3abdef9c..983bf8af5f64 100644 --- a/arch/x86/kvm/hyperv.c +++ b/arch/x86/kvm/hyperv.c @@ -3150,6 +3150,32 @@ struct kvm_hv_vtl_dev { static struct xarray *kvm_hv_vsm_get_memprots(struct kvm_vcpu *vcpu); +static void kvm_hv_inject_gpa_intercept(struct kvm_vcpu *vcpu, + struct kvm_page_fault *fault) +{ + struct kvm_vcpu *target_vcpu = + kvm_hv_get_vtl_vcpu(vcpu, kvm_hv_get_active_vtl(vcpu) + 1); + struct kvm_vcpu_hv_intercept_info *intercept = + &target_vcpu->arch.hyperv->intercept_info; + + /* + * No target VTL available, log a warning and let user-space deal with + * the fault. + */ + if (WARN_ON_ONCE(!target_vcpu)) + return; + + intercept->type = HVMSG_GPA_INTERCEPT; + intercept->gpa = fault->addr; + intercept->access = (fault->user ? HV_INTERCEPT_ACCESS_READ : 0) | + (fault->write ? HV_INTERCEPT_ACCESS_WRITE : 0) | + (fault->exec ? HV_INTERCEPT_ACCESS_EXECUTE : 0); + intercept->vcpu = vcpu; + + kvm_make_request(KVM_REQ_HV_INJECT_INTERCEPT, target_vcpu); + kvm_vcpu_kick(target_vcpu); +} + bool kvm_hv_vsm_access_valid(struct kvm_page_fault *fault, unsigned long attrs) { if (attrs == KVM_MEMORY_ATTRIBUTE_NO_ACCESS) @@ -3194,6 +3220,7 @@ int kvm_hv_faultin_pfn(struct kvm_vcpu *vcpu, struct kvm_page_fault *fault) return RET_PF_CONTINUE; } + kvm_hv_inject_gpa_intercept(vcpu, fault); return -EFAULT; } From patchwork Wed Nov 8 11:18:05 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Nicolas Saenz Julienne X-Patchwork-Id: 13449845 Received: from lindbergh.monkeyblade.net (lindbergh.monkeyblade.net [23.128.96.19]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id ED74515E9A; Wed, 8 Nov 2023 11:25:04 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=amazon.com header.i=@amazon.com header.b="nA0FG8vJ" Received: from smtp-fw-80008.amazon.com (smtp-fw-80008.amazon.com [99.78.197.219]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 263092129; Wed, 8 Nov 2023 03:25:04 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amazon.com; i=@amazon.com; q=dns/txt; s=amazon201209; t=1699442704; x=1730978704; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=9NaCDvWcyp7wzwlWclmY5VS38Gp2lSjf1eIBJksVZ5k=; b=nA0FG8vJfSi2WACdQiag3d6n2OWgLcDgkjIzcczrIZkcIrfAbaaSbyuJ K9teBRcCsqnDxZjbKve2Z2vQ543MpxE0AYPO32gZ37uVKjYDrsfz1Hm8w XG9BVKeVGNQcr0UKUCKgZgn4UWL2k3RNFLbikgTac6NLhyu4K7SRXfNHc k=; X-IronPort-AV: E=Sophos;i="6.03,286,1694736000"; d="scan'208";a="42020831" Received: from pdx4-co-svc-p1-lb2-vlan3.amazon.com (HELO email-inbound-relay-iad-1e-m6i4x-6e7a78d7.us-east-1.amazon.com) ([10.25.36.214]) by smtp-border-fw-80008.pdx80.corp.amazon.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 08 Nov 2023 11:25:02 +0000 Received: from smtpout.prod.us-west-2.prod.farcaster.email.amazon.dev (iad7-ws-svc-p70-lb3-vlan2.iad.amazon.com [10.32.235.34]) by email-inbound-relay-iad-1e-m6i4x-6e7a78d7.us-east-1.amazon.com (Postfix) with ESMTPS id 566FB80832; Wed, 8 Nov 2023 11:24:59 +0000 (UTC) Received: from EX19MTAEUC002.ant.amazon.com [10.0.17.79:35325] by smtpin.naws.eu-west-1.prod.farcaster.email.amazon.dev [10.0.10.247:2525] with esmtp (Farcaster) id 7ff9fbc7-238e-442c-8b96-7408438601b8; Wed, 8 Nov 2023 11:24:57 +0000 (UTC) X-Farcaster-Flow-ID: 7ff9fbc7-238e-442c-8b96-7408438601b8 Received: from EX19D004EUC001.ant.amazon.com (10.252.51.190) by EX19MTAEUC002.ant.amazon.com (10.252.51.181) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.1118.39; Wed, 8 Nov 2023 11:24:57 +0000 Received: from dev-dsk-nsaenz-1b-189b39ae.eu-west-1.amazon.com (10.13.235.138) by EX19D004EUC001.ant.amazon.com (10.252.51.190) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.1118.39; Wed, 8 Nov 2023 11:24:52 +0000 From: Nicolas Saenz Julienne To: CC: , , , , , , , , , , , , , , , Nicolas Saenz Julienne Subject: [RFC 32/33] KVM: x86: hyper-v: Implement HVCALL_TRANSLATE_VIRTUAL_ADDRESS Date: Wed, 8 Nov 2023 11:18:05 +0000 Message-ID: <20231108111806.92604-33-nsaenz@amazon.com> X-Mailer: git-send-email 2.40.1 In-Reply-To: <20231108111806.92604-1-nsaenz@amazon.com> References: <20231108111806.92604-1-nsaenz@amazon.com> Precedence: bulk X-Mailing-List: kvm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Originating-IP: [10.13.235.138] X-ClientProxiedBy: EX19D032UWB003.ant.amazon.com (10.13.139.165) To EX19D004EUC001.ant.amazon.com (10.252.51.190) Introduce HVCALL_TRANSLATE_VIRTUAL_ADDRESS, the hypercall receives a GVA, generally from a less privileged VTL, and returns the GPA backing it. The GVA -> GPA conversion is done by walking the target VTL's vCPU MMU. NOTE: The hypercall implementation is incomplete and only shared for completion. Additionally we'd like to move the VTL aware parts to user-space. Signed-off-by: Nicolas Saenz Julienne --- arch/x86/kvm/hyperv.c | 98 +++++++++++++++++++++++++++++++ arch/x86/kvm/trace.h | 23 ++++++++ include/asm-generic/hyperv-tlfs.h | 28 +++++++++ 3 files changed, 149 insertions(+) diff --git a/arch/x86/kvm/hyperv.c b/arch/x86/kvm/hyperv.c index 983bf8af5f64..1cb53cd0708f 100644 --- a/arch/x86/kvm/hyperv.c +++ b/arch/x86/kvm/hyperv.c @@ -2540,6 +2540,7 @@ static bool is_xmm_fast_hypercall(struct kvm_hv_hcall *hc) case HVCALL_GET_VP_REGISTERS: case HVCALL_SET_VP_REGISTERS: case HVCALL_MODIFY_VTL_PROTECTION_MASK: + case HVCALL_TRANSLATE_VIRTUAL_ADDRESS: return true; } @@ -2556,6 +2557,96 @@ static void kvm_hv_hypercall_read_xmm(struct kvm_hv_hcall *hc) kvm_fpu_put(); } +static bool kvm_hv_xlate_va_validate_input(struct kvm_vcpu* vcpu, + struct hv_xlate_va_input *in, + u8 *vtl, u8 *flags) +{ + union hv_input_vtl in_vtl; + + if (in->partition_id != HV_PARTITION_ID_SELF) + return false; + + if (in->vp_index != HV_VP_INDEX_SELF && + in->vp_index != kvm_hv_get_vpindex(vcpu)) + return false; + + in_vtl.as_uint8 = in->control_flags >> 56; + *flags = in->control_flags & HV_XLATE_GVA_FLAGS_MASK; + if (*flags > (HV_XLATE_GVA_VAL_READ | + HV_XLATE_GVA_VAL_WRITE | + HV_XLATE_GVA_VAL_EXECUTE)) + pr_info_ratelimited("Translate VA control flags unsupported and will be ignored: 0x%llx\n", + in->control_flags); + + *vtl = in_vtl.use_target_vtl ? in_vtl.target_vtl : + kvm_hv_get_active_vtl(vcpu); + if (*vtl > kvm_hv_get_active_vtl(vcpu)) + return false; + + return true; +} + +static u64 kvm_hv_xlate_va_walk(struct kvm_vcpu* vcpu, u64 gva, u8 flags) +{ + struct kvm_mmu *mmu = vcpu->arch.walk_mmu; + u32 access = 0; + + if (flags & HV_XLATE_GVA_VAL_WRITE) + access |= PFERR_WRITE_MASK; + if (flags & HV_XLATE_GVA_VAL_EXECUTE) + access |= PFERR_FETCH_MASK; + + return vcpu->arch.walk_mmu->gva_to_gpa(vcpu, mmu, gva, access, NULL); +} + +static u64 kvm_hv_translate_virtual_address(struct kvm_vcpu* vcpu, + struct kvm_hv_hcall *hc) +{ + struct hv_xlate_va_output output = {}; + struct hv_xlate_va_input input; + struct kvm_vcpu *target_vcpu; + u8 flags, target_vtl; + + if (hc->fast) { + input.partition_id = hc->ingpa; + input.vp_index = hc->outgpa & 0xFFFFFFFF; + input.control_flags = sse128_lo(hc->xmm[0]); + input.gva = sse128_hi(hc->xmm[0]); + } else { + if (kvm_read_guest(vcpu->kvm, hc->ingpa, &input, sizeof(input))) + return HV_STATUS_INVALID_HYPERCALL_INPUT; + } + + trace_kvm_hv_translate_virtual_address(input.partition_id, + input.vp_index, + input.control_flags, input.gva); + + if (!kvm_hv_xlate_va_validate_input(vcpu, &input, &target_vtl, &flags)) + return HV_STATUS_INVALID_HYPERCALL_INPUT; + + target_vcpu = kvm_hv_get_vtl_vcpu(vcpu, target_vtl); + output.gpa = kvm_hv_xlate_va_walk(target_vcpu, input.gva << PAGE_SHIFT, + flags); + if (output.gpa == INVALID_GPA) { + output.result_code = HV_XLATE_GVA_UNMAPPED; + } else { + output.gpa >>= PAGE_SHIFT; + output.result_code = HV_XLATE_GVA_SUCCESS; + output.cache_type = HV_CACHE_TYPE_X64_WB; + } + + if (hc->fast) { + memcpy(&hc->xmm[1], &output, sizeof(output)); + hc->xmm_dirty = true; + } else { + if (kvm_write_guest(vcpu->kvm, hc->outgpa, &output, + sizeof(output))) + return HV_STATUS_INVALID_HYPERCALL_INPUT; + } + + return HV_STATUS_SUCCESS; +} + static bool hv_check_hypercall_access(struct kvm_vcpu_hv *hv_vcpu, u16 code) { if (!hv_vcpu->enforce_cpuid) @@ -2766,6 +2857,13 @@ int kvm_hv_hypercall(struct kvm_vcpu *vcpu) case HVCALL_VTL_CALL: case HVCALL_VTL_RETURN: goto hypercall_userspace_exit; + case HVCALL_TRANSLATE_VIRTUAL_ADDRESS: + if (unlikely(hc.rep_cnt)) { + ret = HV_STATUS_INVALID_HYPERCALL_INPUT; + break; + } + ret = kvm_hv_translate_virtual_address(vcpu, &hc); + break; default: ret = HV_STATUS_INVALID_HYPERCALL_CODE; break; diff --git a/arch/x86/kvm/trace.h b/arch/x86/kvm/trace.h index ab8839c47bc7..6b908671a0cc 100644 --- a/arch/x86/kvm/trace.h +++ b/arch/x86/kvm/trace.h @@ -1372,6 +1372,29 @@ TRACE_EVENT(kvm_hv_stimer_cleanup, __entry->vcpu_id, __entry->timer_index) ); +TRACE_EVENT(kvm_hv_translate_virtual_address, + TP_PROTO(u64 partition_id, u32 vp_index, u64 control_flags, u64 gva), + TP_ARGS(partition_id, vp_index, control_flags, gva), + + TP_STRUCT__entry( + __field(u64, partition_id) + __field(u32, vp_index) + __field(u64, control_flags) + __field(u64, gva) + ), + + TP_fast_assign( + __entry->partition_id = partition_id; + __entry->vp_index = vp_index; + __entry->control_flags = control_flags; + __entry->gva = gva; + ), + + TP_printk("partition id 0x%llx, vp index 0x%x, control flags 0x%llx, gva 0x%llx", + __entry->partition_id, __entry->vp_index, + __entry->control_flags, __entry->gva) +); + TRACE_EVENT(kvm_apicv_inhibit_changed, TP_PROTO(int reason, bool set, unsigned long inhibits), TP_ARGS(reason, set, inhibits), diff --git a/include/asm-generic/hyperv-tlfs.h b/include/asm-generic/hyperv-tlfs.h index a8b5c8a84bbc..24f983222c96 100644 --- a/include/asm-generic/hyperv-tlfs.h +++ b/include/asm-generic/hyperv-tlfs.h @@ -163,6 +163,7 @@ union hv_reference_tsc_msr { #define HVCALL_CREATE_VP 0x004e #define HVCALL_GET_VP_REGISTERS 0x0050 #define HVCALL_SET_VP_REGISTERS 0x0051 +#define HVCALL_TRANSLATE_VIRTUAL_ADDRESS 0x0052 #define HVCALL_POST_MESSAGE 0x005c #define HVCALL_SIGNAL_EVENT 0x005d #define HVCALL_POST_DEBUG_DATA 0x0069 @@ -842,4 +843,31 @@ union hv_register_vsm_code_page_offsets { u64 reserved:40; } __packed; }; + +#define HV_XLATE_GVA_SUCCESS 0 +#define HV_XLATE_GVA_UNMAPPED 1 +#define HV_XLATE_GPA_UNMAPPED 4 +#define HV_CACHE_TYPE_X64_WB 6 + +#define HV_XLATE_GVA_VAL_READ 1 +#define HV_XLATE_GVA_VAL_WRITE 2 +#define HV_XLATE_GVA_VAL_EXECUTE 4 +#define HV_XLATE_GVA_FLAGS_MASK 0x3F + +struct hv_xlate_va_input { + u64 partition_id; + u32 vp_index; + u32 reserved; + u64 control_flags; + u64 gva; +}; + +struct hv_xlate_va_output { + u32 result_code; + u32 cache_type:8; + u32 overlay_page:1; + u32 reserved:23; + u64 gpa; +}; + #endif From patchwork Wed Nov 8 11:18:06 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Nicolas Saenz Julienne X-Patchwork-Id: 13449846 Received: from lindbergh.monkeyblade.net (lindbergh.monkeyblade.net [23.128.96.19]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 7CF73101D2; Wed, 8 Nov 2023 11:25:29 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=amazon.com header.i=@amazon.com header.b="lpv9ixHt" Received: from smtp-fw-9102.amazon.com (smtp-fw-9102.amazon.com [207.171.184.29]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id CE0251FE6; Wed, 8 Nov 2023 03:25:28 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amazon.com; i=@amazon.com; q=dns/txt; s=amazon201209; t=1699442729; x=1730978729; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=pKFAR2whklJOfx0aE2YbJs1djQGs6K3PfD1UPNSSZz8=; b=lpv9ixHtv3vsCQrayxp3rZEZ1aMCCZtv2Ex0GTnkf7QGEq+jVR0RVBHD Gt5PPp2pBc9vsTotBUFYExN9lLZt7EbH78gjtCU8pSc4kLzPE+s5b7wHC +yAhecJ/HG2TTnxJrYXeP0ba+SYcm9xiM7Fv0/64ut0+/NK36E43X2T4F Q=; X-IronPort-AV: E=Sophos;i="6.03,286,1694736000"; d="scan'208";a="375132593" Received: from pdx4-co-svc-p1-lb2-vlan3.amazon.com (HELO email-inbound-relay-pdx-2b-m6i4x-f253a3a3.us-west-2.amazon.com) ([10.25.36.214]) by smtp-border-fw-9102.sea19.amazon.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 08 Nov 2023 11:25:28 +0000 Received: from smtpout.prod.us-west-2.prod.farcaster.email.amazon.dev (pdx2-ws-svc-p26-lb5-vlan2.pdx.amazon.com [10.39.38.66]) by email-inbound-relay-pdx-2b-m6i4x-f253a3a3.us-west-2.amazon.com (Postfix) with ESMTPS id BAE6980718; Wed, 8 Nov 2023 11:25:27 +0000 (UTC) Received: from EX19MTAEUB002.ant.amazon.com [10.0.43.254:32548] by smtpin.naws.eu-west-1.prod.farcaster.email.amazon.dev [10.0.45.210:2525] with esmtp (Farcaster) id 5bc3b812-def8-49f3-9115-267bad567819; Wed, 8 Nov 2023 11:25:26 +0000 (UTC) X-Farcaster-Flow-ID: 5bc3b812-def8-49f3-9115-267bad567819 Received: from EX19D004EUC001.ant.amazon.com (10.252.51.190) by EX19MTAEUB002.ant.amazon.com (10.252.51.59) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.1118.39; Wed, 8 Nov 2023 11:25:26 +0000 Received: from dev-dsk-nsaenz-1b-189b39ae.eu-west-1.amazon.com (10.13.235.138) by EX19D004EUC001.ant.amazon.com (10.252.51.190) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.1118.39; Wed, 8 Nov 2023 11:25:21 +0000 From: Nicolas Saenz Julienne To: CC: , , , , , , , , , , , , , , , Nicolas Saenz Julienne Subject: [RFC 33/33] Documentation: KVM: Introduce "Emulating Hyper-V VSM with KVM" Date: Wed, 8 Nov 2023 11:18:06 +0000 Message-ID: <20231108111806.92604-34-nsaenz@amazon.com> X-Mailer: git-send-email 2.40.1 In-Reply-To: <20231108111806.92604-1-nsaenz@amazon.com> References: <20231108111806.92604-1-nsaenz@amazon.com> Precedence: bulk X-Mailing-List: kvm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Originating-IP: [10.13.235.138] X-ClientProxiedBy: EX19D036UWC002.ant.amazon.com (10.13.139.242) To EX19D004EUC001.ant.amazon.com (10.252.51.190) Introduce "Emulating Hyper-V VSM with KVM", which describes the KVM APIs made available to a VMM that wants to emulate Hyper-V's VSM. Signed-off-by: Nicolas Saenz Julienne --- .../virt/kvm/x86/emulating-hyperv-vsm.rst | 136 ++++++++++++++++++ 1 file changed, 136 insertions(+) create mode 100644 Documentation/virt/kvm/x86/emulating-hyperv-vsm.rst diff --git a/Documentation/virt/kvm/x86/emulating-hyperv-vsm.rst b/Documentation/virt/kvm/x86/emulating-hyperv-vsm.rst new file mode 100644 index 000000000000..8f76bf09c530 --- /dev/null +++ b/Documentation/virt/kvm/x86/emulating-hyperv-vsm.rst @@ -0,0 +1,136 @@ +.. SPDX-License-Identifier: GPL-2.0 + +============================== +Emulating Hyper-V VSM with KVM +============================== + +Hyper-V's Virtual Secure Mode (VSM) is a virtualisation security feature +that leverages the hypervisor to create secure execution environments +within a guest. VSM is documented as part of Microsoft's Hypervisor Top +Level Functional Specification[1]. + +Emulating Hyper-V's Virtual Secure Mode with KVM requires coordination +between KVM and the VMM. Most of the VSM state and configuration is left +to be handled by user-space, but some has made its way into KVM. This +document describes the mechanisms through which a VMM can implement VSM +support. + +Virtual Trust Levels +-------------------- + +The main concept VSM introduces are Virtual Trust Levels or VTLs. Each +VTL is a CPU mode, with its own private CPU architectural state, +interrupt subsystem (limited to a local APIC), and memory access +permissions. VTLs are hierarchical, where VTL0 corresponds to normal +guest execution and VTL > 0 to privileged execution contexts. In +practice, when virtualising Windows on top of KVM, we only see VTL0 and +VTL1. Although the spec allows going all the way to VTL15. VTLs are +orthogonal to ring levels, so each VTL is capable of runnig its own +operating system and user-space[2]. + + ┌──────────────────────────────┐ ┌──────────────────────────────┐ + │ Normal Mode (VTL0) │ │ Secure Mode (VTL1) │ + │ ┌──────────────────────────┐ │ │ ┌──────────────────────────┐ │ + │ │ User-mode Processes │ │ │ │Secure User-mode Processes│ │ + │ └──────────────────────────┘ │ │ └──────────────────────────┘ │ + │ ┌──────────────────────────┐ │ │ ┌──────────────────────────┐ │ + │ │ Kernel │ │ │ │ Secure Kernel │ │ + │ └──────────────────────────┘ │ │ └──────────────────────────┘ │ + └──────────────────────────────┘ └──────────────────────────────┘ + ┌───────────────────────────────────────────────────────────────┐ + │ Hypervisor/KVM │ + └───────────────────────────────────────────────────────────────┘ + ┌───────────────────────────────────────────────────────────────┐ + │ Hardware │ + └───────────────────────────────────────────────────────────────┘ + +VTLs break the core assumption that a vCPU has a single architectural +state, lAPIC state, SynIC state, etc. As such, each VTL is modeled as a +distinct KVM vCPU, with the restriction that only one is allowed to run +at any moment in time. Having multiple KVM vCPUs tracking a single guest +CPU complicates vCPU numbering. VMs that enable VSM are expected to use +CAP_APIC_ID_GROUPS to segregate vCPUs (and their lAPICs) into different +groups. For example, a 4 CPU VSM VM will setup the APIC ID groups feature +so only the first two bits of the APIC ID are exposed to the guest. The +remaining bits represent the vCPU's VTL. The 'sibling' vCPU to VTL0's +vCPU2 at VTL3 will have an APIC ID of 0xE. Using this approach a VMM and +KVM are capable of querying a vCPU's VTL, or finding the vCPU associated +to a specific VTL. + +KVM's lAPIC implementation is aware of groups, and takes note of the +source vCPU's group when delivering IPIs. As such, it shouldn't be +possible to target a different VTL through the APIC. Interrupts are +delivered to the vCPU's lAPIC subsystem regardless of the VTL's runstate, +this also includes timers. Ultimately, any interrupt incoming from an +outside source (IOAPIC/MSIs) is routed to VTL0. + +Moving Between VTLs +------------------- + +All VSM configuration and VTL handling hypercalls are passed through to +user-space. Notably the two primitives that allow switching between VTLs. +All shared state synchronization and KVM vCPU scheduling is left to the +VMM to manage. For example, upon receiving a VTL call, the VMM stops the +vCPU that issued the hypercall, and schedules the vCPU corresponding to +the next privileged VTL. When that privileged vCPU is done executing, it +issues a VTL return hypercall, so the opposite operation happens. All +this is transparent to KVM, which limits itself to running vCPUs. + +An interrupt directed at a privileged VTL always has precedence over the +execution of lower VTLs. To honor this, the VMM can monitor events +targeted at privileged vCPUs with poll(), and should trigger an +asynchronous VTL switch whenever events become available. Additionally, +the target VTL's vCPU VP assist overlay page is used to notify the target +VTL with the reason for the switch. The VMM can keep track of the VP +assist page by installing an MSR filter for HV_X64_MSR_VP_ASSIST_PAGE. + +Hyper-V VP registers +-------------------- + +VP register hypercalls are passed through to user-space. All requests can +be fulfilled either by using already existing KVM state ioctls, or are +related to VSM's configuration, which is already handled by the VMM. Note +that HV_REGISTER_VSM_CODE_PAGE_OFFSETS is the only VSM specific VP +register the kernel controls, as such it is made available through the +KVM_HV_GET_VSM_STATE ioctl. + +Per-VTL Memory Protections +-------------------------- + +A privileged VTL can change the memory access restrictions of lower VTLs. +It does so to hide secrets from them, or to control what they are allowed +to execute. The list of memory protections allowed is[3]: + - No access + - Read-only, no execute + - Read-only, execute + - Read/write, no execute + - Read/write, execute + +VTL memory protection hypercalls are passed through to user-space, but +KVM provides an interface that allows changing memory protections on a +per-VTL basis. This is made possible by the KVM VTL device. VMMs can +create one per VTL and it exposes a ioctl, KVM_SET_MEMORY_ATTRIBUTES, +that controls the memory protections applied to that VTL. The KVM TDP MMU +is VTL aware and page faults are resolved taking into account the +corresponding VTL device's memory attributes. + +When a memory access violates VTL memory protections, KVM issues a secure +memory intercept, which is passed as a SynIC message into the next +privileged VTL. This happens transparently for the VMM. Additionally, KVM +exits with a user-space memory fault. This allows the VMM to stop the +vCPU while the secure intercept is handled by the privileged VTL. In the +good case, the instruction that triggered the fault is emulated and +control is returned to the lower VTL, in the bad case, Windows crashes +gracefully. + +Hyper-V's TLFS also states that DMA should follow VTL0's memory access +restrictions. This is out of scope for this document, as IOMMU mappings +are not handled by KVM. + +[1] https://raw.githubusercontent.com/Microsoft/Virtualization-Documentation/master/tlfs/Hypervisor%20Top%20Level%20Functional%20Specification%20v6.0b.pdf + +[2] Conceptually this design is similar to arm's TrustZone: The +hypervisor plays the role of EL3. Windows (VTL0) runs in Non-Secure +(EL0/EL1) and the secure kernel (VTL1) in Secure World (EL1s/EL0s). + +[3] TLFS 15.9.3