[v3,24/34] KVM: selftests: Hyper-V PV IPI selftest

Message ID	20220414132013.1588929-25-vkuznets@redhat.com (mailing list archive)
State	New, archived
Headers	show Return-Path: <kvm-owner@kernel.org> From: Vitaly Kuznetsov <vkuznets@redhat.com> To: kvm@vger.kernel.org, Paolo Bonzini <pbonzini@redhat.com> Cc: Sean Christopherson <seanjc@google.com>, Wanpeng Li <wanpengli@tencent.com>, Jim Mattson <jmattson@google.com>, Michael Kelley <mikelley@microsoft.com>, Siddharth Chandrasekaran <sidcha@amazon.de>, linux-hyperv@vger.kernel.org, linux-kernel@vger.kernel.org Subject: [PATCH v3 24/34] KVM: selftests: Hyper-V PV IPI selftest Date: Thu, 14 Apr 2022 15:20:03 +0200 Message-Id: <20220414132013.1588929-25-vkuznets@redhat.com> In-Reply-To: <20220414132013.1588929-1-vkuznets@redhat.com> References: <20220414132013.1588929-1-vkuznets@redhat.com> MIME-Version: 1.0 Content-Type: text/plain Content-Transfer-Encoding: 8bit Precedence: bulk
Series	KVM: x86: hyper-v: Fine-grained TLB flush + L2 TLB flush feature \| expand [v3,00/34] KVM: x86: hyper-v: Fine-grained TLB flush + L2 TLB flush feature [v3,01/34] KVM: x86: hyper-v: Resurrect dedicated KVM_REQ_HV_TLB_FLUSH flag [v3,02/34] KVM: x86: hyper-v: Introduce TLB flush ring [v3,03/34] KVM: x86: hyper-v: Add helper to read hypercall data for array [v3,04/34] KVM: x86: hyper-v: Handle HVCALL_FLUSH_VIRTUAL_ADDRESS_LIST{,EX} calls gently [v3,05/34] KVM: x86: hyper-v: Expose support for extended gva ranges for flush hypercalls [v3,06/34] KVM: x86: Prepare kvm_hv_flush_tlb() to handle L2's GPAs [v3,07/34] x86/hyperv: Introduce HV_MAX_SPARSE_VCPU_BANKS/HV_VCPUS_PER_SPARSE_BANK constants [v3,08/34] KVM: x86: hyper-v: Use HV_MAX_SPARSE_VCPU_BANKS/HV_VCPUS_PER_SPARSE_BANK instead of raw … [v3,09/34] KVM: x86: hyper-v: Don't use sparse_set_to_vcpu_mask() in kvm_hv_send_ipi() [v3,10/34] KVM: x86: hyper-v: Create a separate ring for L2 TLB flush [v3,11/34] KVM: x86: hyper-v: Use preallocated buffer in 'struct kvm_vcpu_hv' instead of on-stack '… [v3,12/34] KVM: nVMX: Keep track of hv_vm_id/hv_vp_id when eVMCS is in use [v3,13/34] KVM: nSVM: Keep track of Hyper-V hv_vm_id/hv_vp_id [v3,14/34] KVM: x86: Introduce .post_hv_l2_tlb_flush() nested hook [v3,15/34] KVM: x86: hyper-v: Introduce kvm_hv_is_tlb_flush_hcall() [v3,16/34] KVM: x86: hyper-v: L2 TLB flush [v3,17/34] KVM: x86: hyper-v: Introduce fast kvm_hv_l2_tlb_flush_exposed() check [v3,18/34] x86/hyperv: Fix 'struct hv_enlightened_vmcs' definition [v3,19/34] KVM: nVMX: hyper-v: Enable L2 TLB flush [v3,20/34] KVM: x86: KVM_REQ_TLB_FLUSH_CURRENT is a superset of KVM_REQ_HV_TLB_FLUSH too [v3,21/34] KVM: nSVM: hyper-v: Enable L2 TLB flush [v3,22/34] KVM: x86: Expose Hyper-V L2 TLB flush feature [v3,23/34] KVM: selftests: Better XMM read/write helpers [v3,24/34] KVM: selftests: Hyper-V PV IPI selftest [v3,25/34] KVM: selftests: Make it possible to replace PTEs with __virt_pg_map() [v3,26/34] KVM: selftests: Hyper-V PV TLB flush selftest [v3,27/34] KVM: selftests: Sync 'struct hv_enlightened_vmcs' definition with hyperv-tlfs.h [v3,28/34] KVM: selftests: nVMX: Allocate Hyper-V partition assist page [v3,29/34] KVM: selftests: nSVM: Allocate Hyper-V partition assist and VP assist pages [v3,30/34] KVM: selftests: Sync 'struct hv_vp_assist_page' definition with hyperv-tlfs.h [v3,31/34] KVM: selftests: evmcs_test: Introduce L2 TLB flush test [v3,32/34] KVM: selftests: Move Hyper-V VP assist page enablement out of evmcs.h [v3,33/34] KVM: selftests: hyperv_svm_test: Introduce L2 TLB flush test [v3,34/34] KVM: x86: Rename 'enable_direct_tlbflush' to 'enable_l2_tlb_flush'

Message ID

20220414132013.1588929-25-vkuznets@redhat.com (mailing list archive)

State

New, archived

Headers

From: Vitaly Kuznetsov <vkuznets@redhat.com>
To: kvm@vger.kernel.org, Paolo Bonzini <pbonzini@redhat.com>
Cc: Sean Christopherson <seanjc@google.com>,
        Wanpeng Li <wanpengli@tencent.com>,
        Jim Mattson <jmattson@google.com>,
        Michael Kelley <mikelley@microsoft.com>,
        Siddharth Chandrasekaran <sidcha@amazon.de>,
        linux-hyperv@vger.kernel.org, linux-kernel@vger.kernel.org
Subject: [PATCH v3 24/34] KVM: selftests: Hyper-V PV IPI selftest
Date: Thu, 14 Apr 2022 15:20:03 +0200
Message-Id: <20220414132013.1588929-25-vkuznets@redhat.com>
In-Reply-To: <20220414132013.1588929-1-vkuznets@redhat.com>
References: <20220414132013.1588929-1-vkuznets@redhat.com>
MIME-Version: 1.0
Content-Type: text/plain
Content-Transfer-Encoding: 8bit
Precedence: bulk

Series

KVM: x86: hyper-v: Fine-grained TLB flush + L2 TLB flush feature | expand

Commit Message

Vitaly Kuznetsov April 14, 2022, 1:20 p.m. UTC

Introduce a selftest for Hyper-V PV IPI hypercalls
(HvCallSendSyntheticClusterIpi, HvCallSendSyntheticClusterIpiEx).

The test creates one 'sender' vCPU and two 'receiver' vCPU and then
issues various combinations of send IPI hypercalls in both 'normal'
and 'fast' (with XMM input where necessary) mode. Later, the test
checks whether IPIs were delivered to the expected destination vCPU[s].

Signed-off-by: Vitaly Kuznetsov <vkuznets@redhat.com>
---
 tools/testing/selftests/kvm/.gitignore        |   1 +
 tools/testing/selftests/kvm/Makefile          |   1 +
 .../selftests/kvm/include/x86_64/hyperv.h     |   3 +
 .../selftests/kvm/x86_64/hyperv_features.c    |   5 +-
 .../testing/selftests/kvm/x86_64/hyperv_ipi.c | 374 ++++++++++++++++++
 5 files changed, 381 insertions(+), 3 deletions(-)
 create mode 100644 tools/testing/selftests/kvm/x86_64/hyperv_ipi.c

Comments

Maxim Levitsky May 11, 2022, 11:35 a.m. UTC | #1

On Thu, 2022-04-14 at 15:20 +0200, Vitaly Kuznetsov wrote:
> Introduce a selftest for Hyper-V PV IPI hypercalls
> (HvCallSendSyntheticClusterIpi, HvCallSendSyntheticClusterIpiEx).
> 
> The test creates one 'sender' vCPU and two 'receiver' vCPU and then
> issues various combinations of send IPI hypercalls in both 'normal'
> and 'fast' (with XMM input where necessary) mode. Later, the test
> checks whether IPIs were delivered to the expected destination vCPU[s].
> 
> Signed-off-by: Vitaly Kuznetsov <vkuznets@redhat.com>
> ---
>  tools/testing/selftests/kvm/.gitignore        |   1 +
>  tools/testing/selftests/kvm/Makefile          |   1 +
>  .../selftests/kvm/include/x86_64/hyperv.h     |   3 +
>  .../selftests/kvm/x86_64/hyperv_features.c    |   5 +-
>  .../testing/selftests/kvm/x86_64/hyperv_ipi.c | 374 ++++++++++++++++++
>  5 files changed, 381 insertions(+), 3 deletions(-)
>  create mode 100644 tools/testing/selftests/kvm/x86_64/hyperv_ipi.c
> 
> diff --git a/tools/testing/selftests/kvm/.gitignore b/tools/testing/selftests/kvm/.gitignore
> index 56140068b763..5d5fbb161d56 100644
> --- a/tools/testing/selftests/kvm/.gitignore
> +++ b/tools/testing/selftests/kvm/.gitignore
> @@ -23,6 +23,7 @@
>  /x86_64/hyperv_clock
>  /x86_64/hyperv_cpuid
>  /x86_64/hyperv_features
> +/x86_64/hyperv_ipi
>  /x86_64/hyperv_svm_test
>  /x86_64/mmio_warning_test
>  /x86_64/mmu_role_test
> diff --git a/tools/testing/selftests/kvm/Makefile b/tools/testing/selftests/kvm/Makefile
> index af582d168621..44889f897fe7 100644
> --- a/tools/testing/selftests/kvm/Makefile
> +++ b/tools/testing/selftests/kvm/Makefile
> @@ -52,6 +52,7 @@ TEST_GEN_PROGS_x86_64 += x86_64/fix_hypercall_test
>  TEST_GEN_PROGS_x86_64 += x86_64/hyperv_clock
>  TEST_GEN_PROGS_x86_64 += x86_64/hyperv_cpuid
>  TEST_GEN_PROGS_x86_64 += x86_64/hyperv_features
> +TEST_GEN_PROGS_x86_64 += x86_64/hyperv_ipi
>  TEST_GEN_PROGS_x86_64 += x86_64/hyperv_svm_test
>  TEST_GEN_PROGS_x86_64 += x86_64/kvm_clock_test
>  TEST_GEN_PROGS_x86_64 += x86_64/kvm_pv_test
> diff --git a/tools/testing/selftests/kvm/include/x86_64/hyperv.h b/tools/testing/selftests/kvm/include/x86_64/hyperv.h
> index b66910702c0a..f51d6fab8e93 100644
> --- a/tools/testing/selftests/kvm/include/x86_64/hyperv.h
> +++ b/tools/testing/selftests/kvm/include/x86_64/hyperv.h
> @@ -184,5 +184,8 @@
>  
>  /* hypercall options */
>  #define HV_HYPERCALL_FAST_BIT		BIT(16)
> +#define HV_HYPERCALL_VARHEAD_OFFSET	17
> +
> +#define HYPERV_LINUX_OS_ID ((u64)0x8100 << 48)
>  
>  #endif /* !SELFTEST_KVM_HYPERV_H */
> diff --git a/tools/testing/selftests/kvm/x86_64/hyperv_features.c b/tools/testing/selftests/kvm/x86_64/hyperv_features.c
> index 672915ce73d8..98c020356925 100644
> --- a/tools/testing/selftests/kvm/x86_64/hyperv_features.c
> +++ b/tools/testing/selftests/kvm/x86_64/hyperv_features.c
> @@ -14,7 +14,6 @@
>  #include "hyperv.h"
>  
>  #define VCPU_ID 0
> -#define LINUX_OS_ID ((u64)0x8100 << 48)
>  
>  extern unsigned char rdmsr_start;
>  extern unsigned char rdmsr_end;
> @@ -127,7 +126,7 @@ static void guest_hcall(vm_vaddr_t pgs_gpa, struct hcall_data *hcall)
>  	int i = 0;
>  	u64 res, input, output;
>  
> -	wrmsr(HV_X64_MSR_GUEST_OS_ID, LINUX_OS_ID);
> +	wrmsr(HV_X64_MSR_GUEST_OS_ID, HYPERV_LINUX_OS_ID);
>  	wrmsr(HV_X64_MSR_HYPERCALL, pgs_gpa);
>  
>  	while (hcall->control) {
> @@ -230,7 +229,7 @@ static void guest_test_msrs_access(void)
>  			 */
>  			msr->idx = HV_X64_MSR_GUEST_OS_ID;
>  			msr->write = 1;
> -			msr->write_val = LINUX_OS_ID;
> +			msr->write_val = HYPERV_LINUX_OS_ID;
>  			msr->available = 1;
>  			break;
>  		case 3:

Nitpick: I think that the HYPERV_LINUX_OS_ID change should be in a separate patch.


> diff --git a/tools/testing/selftests/kvm/x86_64/hyperv_ipi.c b/tools/testing/selftests/kvm/x86_64/hyperv_ipi.c
> new file mode 100644
> index 000000000000..075963c32d45
> --- /dev/null
> +++ b/tools/testing/selftests/kvm/x86_64/hyperv_ipi.c
> @@ -0,0 +1,374 @@
> +// SPDX-License-Identifier: GPL-2.0
> +/*
> + * Hyper-V HvCallSendSyntheticClusterIpi{,Ex} tests
> + *
> + * Copyright (C) 2022, Red Hat, Inc.
> + *
> + */
> +
> +#define _GNU_SOURCE /* for program_invocation_short_name */
> +#include <pthread.h>
> +#include <inttypes.h>
> +
> +#include "kvm_util.h"
> +#include "hyperv.h"
> +#include "processor.h"
> +#include "test_util.h"
> +#include "vmx.h"
> +
> +#define SENDER_VCPU_ID   1
> +#define RECEIVER_VCPU_ID_1 2
> +#define RECEIVER_VCPU_ID_2 65
> +
> +#define IPI_VECTOR	 0xfe
> +
> +static volatile uint64_t ipis_rcvd[RECEIVER_VCPU_ID_2 + 1];
> +
> +struct thread_params {
> +	struct kvm_vm *vm;
> +	uint32_t vcpu_id;
> +};
> +
> +struct hv_vpset {
> +	u64 format;
> +	u64 valid_bank_mask;
> +	u64 bank_contents[2];
> +};
> +
> +enum HV_GENERIC_SET_FORMAT {
> +	HV_GENERIC_SET_SPARSE_4K,
> +	HV_GENERIC_SET_ALL,
> +};
> +
> +/* HvCallSendSyntheticClusterIpi hypercall */
> +struct hv_send_ipi {
> +	u32 vector;
> +	u32 reserved;
> +	u64 cpu_mask;
> +};
> +
> +/* HvCallSendSyntheticClusterIpiEx hypercall */
> +struct hv_send_ipi_ex {
> +	u32 vector;
> +	u32 reserved;
> +	struct hv_vpset vp_set;
> +};
> +
> +static inline void hv_init(vm_vaddr_t pgs_gpa)
> +{
> +	wrmsr(HV_X64_MSR_GUEST_OS_ID, HYPERV_LINUX_OS_ID);
> +	wrmsr(HV_X64_MSR_HYPERCALL, pgs_gpa);
> +}
> +
> +static void receiver_code(void *hcall_page, vm_vaddr_t pgs_gpa)
> +{
> +	u32 vcpu_id;
> +
> +	x2apic_enable();
> +	hv_init(pgs_gpa);
> +
> +	vcpu_id = rdmsr(HV_X64_MSR_VP_INDEX);
> +
> +	/* Signal sender vCPU we're ready */
> +	ipis_rcvd[vcpu_id] = (u64)-1;
> +
> +	for (;;)
> +		asm volatile("sti; hlt; cli");
> +}
> +
> +static void guest_ipi_handler(struct ex_regs *regs)
> +{
> +	u32 vcpu_id = rdmsr(HV_X64_MSR_VP_INDEX);
> +
> +	ipis_rcvd[vcpu_id]++;
> +	wrmsr(HV_X64_MSR_EOI, 1);
> +}
> +
> +static inline u64 hypercall(u64 control, vm_vaddr_t arg1, vm_vaddr_t arg2)
> +{
> +	u64 hv_status;
> +
> +	asm volatile("mov %3, %%r8\n"
> +		     "vmcall"
> +		     : "=a" (hv_status),
> +		       "+c" (control), "+d" (arg1)
> +		     :  "r" (arg2)
> +		     : "cc", "memory", "r8", "r9", "r10", "r11");
> +
> +	return hv_status;
> +}
> +
> +static inline void nop_loop(void)
> +{
> +	int i;
> +
> +	for (i = 0; i < 100000000; i++)
> +		asm volatile("nop");
> +}
> +
> +static inline void sync_to_xmm(void *data)
> +{
> +	int i;
> +
> +	for (i = 0; i < 8; i++)
> +		write_sse_reg(i, (sse128_t *)(data + sizeof(sse128_t) * i));
> +}
> +
> +static void sender_guest_code(void *hcall_page, vm_vaddr_t pgs_gpa)
> +{
> +	struct hv_send_ipi *ipi = (struct hv_send_ipi *)hcall_page;
> +	struct hv_send_ipi_ex *ipi_ex = (struct hv_send_ipi_ex *)hcall_page;
> +	int stage = 1, ipis_expected[2] = {0};
> +	u64 res;
> +
> +	hv_init(pgs_gpa);
> +	GUEST_SYNC(stage++);
> +
> +	/* Wait for receiver vCPUs to come up */
> +	while (!ipis_rcvd[RECEIVER_VCPU_ID_1] || !ipis_rcvd[RECEIVER_VCPU_ID_2])
> +		nop_loop();
> +	ipis_rcvd[RECEIVER_VCPU_ID_1] = ipis_rcvd[RECEIVER_VCPU_ID_2] = 0;
> +
> +	/* 'Slow' HvCallSendSyntheticClusterIpi to RECEIVER_VCPU_ID_1 */
> +	ipi->vector = IPI_VECTOR;
> +	ipi->cpu_mask = 1 << RECEIVER_VCPU_ID_1;
> +	res = hypercall(HVCALL_SEND_IPI, pgs_gpa, pgs_gpa + 4096);
> +	GUEST_ASSERT((res & 0xffff) == 0);
> +	nop_loop();
> +	GUEST_ASSERT(ipis_rcvd[RECEIVER_VCPU_ID_1] == ++ipis_expected[0]);
> +	GUEST_ASSERT(ipis_rcvd[RECEIVER_VCPU_ID_2] == ipis_expected[1]);
> +	GUEST_SYNC(stage++);
> +	/* 'Fast' HvCallSendSyntheticClusterIpi to RECEIVER_VCPU_ID_1 */
> +	res = hypercall(HVCALL_SEND_IPI | HV_HYPERCALL_FAST_BIT,
> +			IPI_VECTOR, 1 << RECEIVER_VCPU_ID_1);
> +	GUEST_ASSERT((res & 0xffff) == 0);
> +	nop_loop();
> +	GUEST_ASSERT(ipis_rcvd[RECEIVER_VCPU_ID_1] == ++ipis_expected[0]);
> +	GUEST_ASSERT(ipis_rcvd[RECEIVER_VCPU_ID_2] == ipis_expected[1]);
> +	GUEST_SYNC(stage++);
> +
> +	/* 'Slow' HvCallSendSyntheticClusterIpiEx to RECEIVER_VCPU_ID_1 */
> +	memset(hcall_page, 0, 4096);
> +	ipi_ex->vector = IPI_VECTOR;
> +	ipi_ex->vp_set.format = HV_GENERIC_SET_SPARSE_4K;
> +	ipi_ex->vp_set.valid_bank_mask = 1 << 0;
> +	ipi_ex->vp_set.bank_contents[0] = BIT(RECEIVER_VCPU_ID_1);
> +	res = hypercall(HVCALL_SEND_IPI_EX | (1 << HV_HYPERCALL_VARHEAD_OFFSET),
> +			pgs_gpa, pgs_gpa + 4096);
> +	GUEST_ASSERT((res & 0xffff) == 0);
> +	nop_loop();
> +	GUEST_ASSERT(ipis_rcvd[RECEIVER_VCPU_ID_1] == ++ipis_expected[0]);
> +	GUEST_ASSERT(ipis_rcvd[RECEIVER_VCPU_ID_2] == ipis_expected[1]);
> +	GUEST_SYNC(stage++);
> +	/* 'XMM Fast' HvCallSendSyntheticClusterIpiEx to RECEIVER_VCPU_ID_1 */
> +	sync_to_xmm(&ipi_ex->vp_set.valid_bank_mask);
> +	res = hypercall(HVCALL_SEND_IPI_EX | HV_HYPERCALL_FAST_BIT |
> +			(1 << HV_HYPERCALL_VARHEAD_OFFSET),
> +			IPI_VECTOR, HV_GENERIC_SET_SPARSE_4K);
> +	GUEST_ASSERT((res & 0xffff) == 0);
> +	nop_loop();
> +	GUEST_ASSERT(ipis_rcvd[RECEIVER_VCPU_ID_1] == ++ipis_expected[0]);
> +	GUEST_ASSERT(ipis_rcvd[RECEIVER_VCPU_ID_2] == ipis_expected[1]);
> +	GUEST_SYNC(stage++);
> +
> +	/* 'Slow' HvCallSendSyntheticClusterIpiEx to RECEIVER_VCPU_ID_2 */
> +	memset(hcall_page, 0, 4096);
> +	ipi_ex->vector = IPI_VECTOR;
> +	ipi_ex->vp_set.format = HV_GENERIC_SET_SPARSE_4K;
> +	ipi_ex->vp_set.valid_bank_mask = 1 << 1;
> +	ipi_ex->vp_set.bank_contents[0] = BIT(RECEIVER_VCPU_ID_2 - 64);
> +	res = hypercall(HVCALL_SEND_IPI_EX | (1 << HV_HYPERCALL_VARHEAD_OFFSET),
> +			pgs_gpa, pgs_gpa + 4096);
> +	GUEST_ASSERT((res & 0xffff) == 0);
> +	nop_loop();
> +	GUEST_ASSERT(ipis_rcvd[RECEIVER_VCPU_ID_1] == ipis_expected[0]);
> +	GUEST_ASSERT(ipis_rcvd[RECEIVER_VCPU_ID_2] == ++ipis_expected[1]);
> +	GUEST_SYNC(stage++);
> +	/* 'XMM Fast' HvCallSendSyntheticClusterIpiEx to RECEIVER_VCPU_ID_2 */
> +	sync_to_xmm(&ipi_ex->vp_set.valid_bank_mask);
> +	res = hypercall(HVCALL_SEND_IPI_EX | HV_HYPERCALL_FAST_BIT |
> +			(1 << HV_HYPERCALL_VARHEAD_OFFSET),
> +			IPI_VECTOR, HV_GENERIC_SET_SPARSE_4K);
> +	GUEST_ASSERT((res & 0xffff) == 0);
> +	nop_loop();
> +	GUEST_ASSERT(ipis_rcvd[RECEIVER_VCPU_ID_1] == ipis_expected[0]);
> +	GUEST_ASSERT(ipis_rcvd[RECEIVER_VCPU_ID_2] == ++ipis_expected[1]);
> +	GUEST_SYNC(stage++);
> +
> +	/* 'Slow' HvCallSendSyntheticClusterIpiEx to both RECEIVER_VCPU_ID_{1,2} */
> +	memset(hcall_page, 0, 4096);
> +	ipi_ex->vector = IPI_VECTOR;
> +	ipi_ex->vp_set.format = HV_GENERIC_SET_SPARSE_4K;
> +	ipi_ex->vp_set.valid_bank_mask = 1 << 1 | 1;
> +	ipi_ex->vp_set.bank_contents[0] = BIT(RECEIVER_VCPU_ID_1);
> +	ipi_ex->vp_set.bank_contents[1] = BIT(RECEIVER_VCPU_ID_2 - 64);
> +	res = hypercall(HVCALL_SEND_IPI_EX | (2 << HV_HYPERCALL_VARHEAD_OFFSET),
> +			pgs_gpa, pgs_gpa + 4096);
> +	GUEST_ASSERT((res & 0xffff) == 0);
> +	nop_loop();
> +	GUEST_ASSERT(ipis_rcvd[RECEIVER_VCPU_ID_1] == ++ipis_expected[0]);
> +	GUEST_ASSERT(ipis_rcvd[RECEIVER_VCPU_ID_2] == ++ipis_expected[1]);
> +	GUEST_SYNC(stage++);
> +	/* 'XMM Fast' HvCallSendSyntheticClusterIpiEx to both RECEIVER_VCPU_ID_{1, 2} */
> +	sync_to_xmm(&ipi_ex->vp_set.valid_bank_mask);
> +	res = hypercall(HVCALL_SEND_IPI_EX | HV_HYPERCALL_FAST_BIT |
> +			(2 << HV_HYPERCALL_VARHEAD_OFFSET),
> +			IPI_VECTOR, HV_GENERIC_SET_SPARSE_4K);
> +	GUEST_ASSERT((res & 0xffff) == 0);
> +	nop_loop();
> +	GUEST_ASSERT(ipis_rcvd[RECEIVER_VCPU_ID_1] == ++ipis_expected[0]);
> +	GUEST_ASSERT(ipis_rcvd[RECEIVER_VCPU_ID_2] == ++ipis_expected[1]);
> +	GUEST_SYNC(stage++);
> +
> +	/* 'Slow' HvCallSendSyntheticClusterIpiEx to HV_GENERIC_SET_ALL */
> +	memset(hcall_page, 0, 4096);
> +	ipi_ex->vector = IPI_VECTOR;
> +	ipi_ex->vp_set.format = HV_GENERIC_SET_ALL;
> +	res = hypercall(HVCALL_SEND_IPI_EX,
> +			pgs_gpa, pgs_gpa + 4096);
> +	GUEST_ASSERT((res & 0xffff) == 0);
> +	nop_loop();
> +	GUEST_ASSERT(ipis_rcvd[RECEIVER_VCPU_ID_1] == ++ipis_expected[0]);
> +	GUEST_ASSERT(ipis_rcvd[RECEIVER_VCPU_ID_2] == ++ipis_expected[1]);
> +	GUEST_SYNC(stage++);
> +	/* 'XMM Fast' HvCallSendSyntheticClusterIpiEx to HV_GENERIC_SET_ALL */
> +	sync_to_xmm(&ipi_ex->vp_set.valid_bank_mask);
> +	res = hypercall(HVCALL_SEND_IPI_EX | HV_HYPERCALL_FAST_BIT,
> +			IPI_VECTOR, HV_GENERIC_SET_ALL);
> +	GUEST_ASSERT((res & 0xffff) == 0);
> +	nop_loop();
> +	GUEST_ASSERT(ipis_rcvd[RECEIVER_VCPU_ID_1] == ++ipis_expected[0]);
> +	GUEST_ASSERT(ipis_rcvd[RECEIVER_VCPU_ID_2] == ++ipis_expected[1]);
> +	GUEST_SYNC(stage++);
> +
> +	GUEST_DONE();
> +}
> +
> +static void *vcpu_thread(void *arg)
> +{
> +	struct thread_params *params = (struct thread_params *)arg;
> +	struct ucall uc;
> +	int old;
> +	int r;
> +	unsigned int exit_reason;
> +
> +	r = pthread_setcanceltype(PTHREAD_CANCEL_ASYNCHRONOUS, &old);
> +	TEST_ASSERT(r == 0,
> +		    "pthread_setcanceltype failed on vcpu_id=%u with errno=%d",
> +		    params->vcpu_id, r);
> +
> +	vcpu_run(params->vm, params->vcpu_id);
> +	exit_reason = vcpu_state(params->vm, params->vcpu_id)->exit_reason;
> +
> +	TEST_ASSERT(exit_reason == KVM_EXIT_IO,
> +		    "vCPU %u exited with unexpected exit reason %u-%s, expected KVM_EXIT_IO",
> +		    params->vcpu_id, exit_reason, exit_reason_str(exit_reason));
> +
> +	if (get_ucall(params->vm, params->vcpu_id, &uc) == UCALL_ABORT) {
> +		TEST_ASSERT(false,
> +			    "vCPU %u exited with error: %s.\n",
> +			    params->vcpu_id, (const char *)uc.args[0]);
> +	}
> +
> +	return NULL;
> +}
> +
> +static void cancel_join_vcpu_thread(pthread_t thread, uint32_t vcpu_id)
> +{
> +	void *retval;
> +	int r;
> +
> +	r = pthread_cancel(thread);
> +	TEST_ASSERT(r == 0,
> +		    "pthread_cancel on vcpu_id=%d failed with errno=%d",
> +		    vcpu_id, r);
> +
> +	r = pthread_join(thread, &retval);
> +	TEST_ASSERT(r == 0,
> +		    "pthread_join on vcpu_id=%d failed with errno=%d",
> +		    vcpu_id, r);
> +	TEST_ASSERT(retval == PTHREAD_CANCELED,
> +		    "expected retval=%p, got %p", PTHREAD_CANCELED,
> +		    retval);
> +}
> +
> +int main(int argc, char *argv[])
> +{
> +	int r;
> +	pthread_t threads[2];
> +	struct thread_params params[2];
> +	struct kvm_vm *vm;
> +	struct kvm_run *run;
> +	vm_vaddr_t hcall_page;
> +	struct ucall uc;
> +	int stage = 1;
> +
> +	vm = vm_create_default(SENDER_VCPU_ID, 0, sender_guest_code);
> +	params[0].vm = vm;
> +	params[1].vm = vm;
> +
> +	/* Hypercall input/output */
> +	hcall_page = vm_vaddr_alloc_pages(vm, 2);
> +	memset(addr_gva2hva(vm, hcall_page), 0x0, 2 * getpagesize());
> +
> +	vm_init_descriptor_tables(vm);
> +
> +	vm_vcpu_add_default(vm, RECEIVER_VCPU_ID_1, receiver_code);
> +	vcpu_init_descriptor_tables(vm, RECEIVER_VCPU_ID_1);
> +	vcpu_args_set(vm, RECEIVER_VCPU_ID_1, 2, hcall_page, addr_gva2gpa(vm, hcall_page));
> +	vcpu_set_msr(vm, RECEIVER_VCPU_ID_1, HV_X64_MSR_VP_INDEX, RECEIVER_VCPU_ID_1);
> +	vcpu_set_hv_cpuid(vm, RECEIVER_VCPU_ID_1);
> +
> +	vm_vcpu_add_default(vm, RECEIVER_VCPU_ID_2, receiver_code);
> +	vcpu_init_descriptor_tables(vm, RECEIVER_VCPU_ID_2);
> +	vcpu_args_set(vm, RECEIVER_VCPU_ID_2, 2, hcall_page, addr_gva2gpa(vm, hcall_page));
> +	vcpu_set_msr(vm, RECEIVER_VCPU_ID_2, HV_X64_MSR_VP_INDEX, RECEIVER_VCPU_ID_2);
> +	vcpu_set_hv_cpuid(vm, RECEIVER_VCPU_ID_2);
> +
> +	vm_install_exception_handler(vm, IPI_VECTOR, guest_ipi_handler);
> +
> +	vcpu_args_set(vm, SENDER_VCPU_ID, 2, hcall_page, addr_gva2gpa(vm, hcall_page));
> +	vcpu_set_hv_cpuid(vm, SENDER_VCPU_ID);
> +
> +	params[0].vcpu_id = RECEIVER_VCPU_ID_1;
> +	r = pthread_create(&threads[0], NULL, vcpu_thread, &params[0]);
> +	TEST_ASSERT(r == 0,
> +		    "pthread_create halter failed errno=%d", errno);
> +
> +	params[1].vcpu_id = RECEIVER_VCPU_ID_2;
> +	r = pthread_create(&threads[1], NULL, vcpu_thread, &params[1]);
> +	TEST_ASSERT(r == 0,
> +		    "pthread_create halter failed errno=%d", errno);
> +
> +	run = vcpu_state(vm, SENDER_VCPU_ID);
> +
> +	while (true) {
> +		r = _vcpu_run(vm, SENDER_VCPU_ID);
> +		TEST_ASSERT(!r, "vcpu_run failed: %d\n", r);
> +		TEST_ASSERT(run->exit_reason == KVM_EXIT_IO,
> +			    "unexpected exit reason: %u (%s)",
> +			    run->exit_reason, exit_reason_str(run->exit_reason));
> +
> +		switch (get_ucall(vm, SENDER_VCPU_ID, &uc)) {
> +		case UCALL_SYNC:
> +			TEST_ASSERT(uc.args[1] == stage,
> +				    "Unexpected stage: %ld (%d expected)\n",
> +				    uc.args[1], stage);
> +			break;
> +		case UCALL_ABORT:
> +			TEST_FAIL("%s at %s:%ld", (const char *)uc.args[0],
> +				  __FILE__, uc.args[1]);
> +			return 1;
> +		case UCALL_DONE:
> +			return 0;
> +		}
> +
> +		stage++;
> +	}
> +
> +	cancel_join_vcpu_thread(threads[0], RECEIVER_VCPU_ID_1);
> +	cancel_join_vcpu_thread(threads[1], RECEIVER_VCPU_ID_2);
> +	kvm_vm_free(vm);
> +
> +	return 0;
> +}


Looks overall good to me, but I might have missed something.


Reviewed-by: Maxim Levitsky <mlevitsk@redhat.com>


Best regards,
	Maxim Levitsky

diff --git a/tools/testing/selftests/kvm/.gitignore b/tools/testing/selftests/kvm/.gitignore
index 56140068b763..5d5fbb161d56 100644
--- a/tools/testing/selftests/kvm/.gitignore
+++ b/tools/testing/selftests/kvm/.gitignore
@@ -23,6 +23,7 @@ 
 /x86_64/hyperv_clock
 /x86_64/hyperv_cpuid
 /x86_64/hyperv_features
+/x86_64/hyperv_ipi
 /x86_64/hyperv_svm_test
 /x86_64/mmio_warning_test
 /x86_64/mmu_role_test
diff --git a/tools/testing/selftests/kvm/Makefile b/tools/testing/selftests/kvm/Makefile
index af582d168621..44889f897fe7 100644
--- a/tools/testing/selftests/kvm/Makefile
+++ b/tools/testing/selftests/kvm/Makefile
@@ -52,6 +52,7 @@  TEST_GEN_PROGS_x86_64 += x86_64/fix_hypercall_test
 TEST_GEN_PROGS_x86_64 += x86_64/hyperv_clock
 TEST_GEN_PROGS_x86_64 += x86_64/hyperv_cpuid
 TEST_GEN_PROGS_x86_64 += x86_64/hyperv_features
+TEST_GEN_PROGS_x86_64 += x86_64/hyperv_ipi
 TEST_GEN_PROGS_x86_64 += x86_64/hyperv_svm_test
 TEST_GEN_PROGS_x86_64 += x86_64/kvm_clock_test
 TEST_GEN_PROGS_x86_64 += x86_64/kvm_pv_test
diff --git a/tools/testing/selftests/kvm/include/x86_64/hyperv.h b/tools/testing/selftests/kvm/include/x86_64/hyperv.h
index b66910702c0a..f51d6fab8e93 100644
--- a/tools/testing/selftests/kvm/include/x86_64/hyperv.h
+++ b/tools/testing/selftests/kvm/include/x86_64/hyperv.h
@@ -184,5 +184,8 @@ 
 
 /* hypercall options */
 #define HV_HYPERCALL_FAST_BIT		BIT(16)
+#define HV_HYPERCALL_VARHEAD_OFFSET	17
+
+#define HYPERV_LINUX_OS_ID ((u64)0x8100 << 48)
 
 #endif /* !SELFTEST_KVM_HYPERV_H */
diff --git a/tools/testing/selftests/kvm/x86_64/hyperv_features.c b/tools/testing/selftests/kvm/x86_64/hyperv_features.c
index 672915ce73d8..98c020356925 100644
--- a/tools/testing/selftests/kvm/x86_64/hyperv_features.c
+++ b/tools/testing/selftests/kvm/x86_64/hyperv_features.c
@@ -14,7 +14,6 @@ 
 #include "hyperv.h"
 
 #define VCPU_ID 0
-#define LINUX_OS_ID ((u64)0x8100 << 48)
 
 extern unsigned char rdmsr_start;
 extern unsigned char rdmsr_end;
@@ -127,7 +126,7 @@  static void guest_hcall(vm_vaddr_t pgs_gpa, struct hcall_data *hcall)
 	int i = 0;
 	u64 res, input, output;
 
-	wrmsr(HV_X64_MSR_GUEST_OS_ID, LINUX_OS_ID);
+	wrmsr(HV_X64_MSR_GUEST_OS_ID, HYPERV_LINUX_OS_ID);
 	wrmsr(HV_X64_MSR_HYPERCALL, pgs_gpa);
 
 	while (hcall->control) {
@@ -230,7 +229,7 @@  static void guest_test_msrs_access(void)
 			 */
 			msr->idx = HV_X64_MSR_GUEST_OS_ID;
 			msr->write = 1;
-			msr->write_val = LINUX_OS_ID;
+			msr->write_val = HYPERV_LINUX_OS_ID;
 			msr->available = 1;
 			break;
 		case 3:
diff --git a/tools/testing/selftests/kvm/x86_64/hyperv_ipi.c b/tools/testing/selftests/kvm/x86_64/hyperv_ipi.c
new file mode 100644
index 000000000000..075963c32d45
--- /dev/null
+++ b/tools/testing/selftests/kvm/x86_64/hyperv_ipi.c
@@ -0,0 +1,374 @@ 
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * Hyper-V HvCallSendSyntheticClusterIpi{,Ex} tests
+ *
+ * Copyright (C) 2022, Red Hat, Inc.
+ *
+ */
+
+#define _GNU_SOURCE /* for program_invocation_short_name */
+#include <pthread.h>
+#include <inttypes.h>
+
+#include "kvm_util.h"
+#include "hyperv.h"
+#include "processor.h"
+#include "test_util.h"
+#include "vmx.h"
+
+#define SENDER_VCPU_ID   1
+#define RECEIVER_VCPU_ID_1 2
+#define RECEIVER_VCPU_ID_2 65
+
+#define IPI_VECTOR	 0xfe
+
+static volatile uint64_t ipis_rcvd[RECEIVER_VCPU_ID_2 + 1];
+
+struct thread_params {
+	struct kvm_vm *vm;
+	uint32_t vcpu_id;
+};
+
+struct hv_vpset {
+	u64 format;
+	u64 valid_bank_mask;
+	u64 bank_contents[2];
+};
+
+enum HV_GENERIC_SET_FORMAT {
+	HV_GENERIC_SET_SPARSE_4K,
+	HV_GENERIC_SET_ALL,
+};
+
+/* HvCallSendSyntheticClusterIpi hypercall */
+struct hv_send_ipi {
+	u32 vector;
+	u32 reserved;
+	u64 cpu_mask;
+};
+
+/* HvCallSendSyntheticClusterIpiEx hypercall */
+struct hv_send_ipi_ex {
+	u32 vector;
+	u32 reserved;
+	struct hv_vpset vp_set;
+};
+
+static inline void hv_init(vm_vaddr_t pgs_gpa)
+{
+	wrmsr(HV_X64_MSR_GUEST_OS_ID, HYPERV_LINUX_OS_ID);
+	wrmsr(HV_X64_MSR_HYPERCALL, pgs_gpa);
+}
+
+static void receiver_code(void *hcall_page, vm_vaddr_t pgs_gpa)
+{
+	u32 vcpu_id;
+
+	x2apic_enable();
+	hv_init(pgs_gpa);
+
+	vcpu_id = rdmsr(HV_X64_MSR_VP_INDEX);
+
+	/* Signal sender vCPU we're ready */
+	ipis_rcvd[vcpu_id] = (u64)-1;
+
+	for (;;)
+		asm volatile("sti; hlt; cli");
+}
+
+static void guest_ipi_handler(struct ex_regs *regs)
+{
+	u32 vcpu_id = rdmsr(HV_X64_MSR_VP_INDEX);
+
+	ipis_rcvd[vcpu_id]++;
+	wrmsr(HV_X64_MSR_EOI, 1);
+}
+
+static inline u64 hypercall(u64 control, vm_vaddr_t arg1, vm_vaddr_t arg2)
+{
+	u64 hv_status;
+
+	asm volatile("mov %3, %%r8\n"
+		     "vmcall"
+		     : "=a" (hv_status),
+		       "+c" (control), "+d" (arg1)
+		     :  "r" (arg2)
+		     : "cc", "memory", "r8", "r9", "r10", "r11");
+
+	return hv_status;
+}
+
+static inline void nop_loop(void)
+{
+	int i;
+
+	for (i = 0; i < 100000000; i++)
+		asm volatile("nop");
+}
+
+static inline void sync_to_xmm(void *data)
+{
+	int i;
+
+	for (i = 0; i < 8; i++)
+		write_sse_reg(i, (sse128_t *)(data + sizeof(sse128_t) * i));
+}
+
+static void sender_guest_code(void *hcall_page, vm_vaddr_t pgs_gpa)
+{
+	struct hv_send_ipi *ipi = (struct hv_send_ipi *)hcall_page;
+	struct hv_send_ipi_ex *ipi_ex = (struct hv_send_ipi_ex *)hcall_page;
+	int stage = 1, ipis_expected[2] = {0};
+	u64 res;
+
+	hv_init(pgs_gpa);
+	GUEST_SYNC(stage++);
+
+	/* Wait for receiver vCPUs to come up */
+	while (!ipis_rcvd[RECEIVER_VCPU_ID_1] || !ipis_rcvd[RECEIVER_VCPU_ID_2])
+		nop_loop();
+	ipis_rcvd[RECEIVER_VCPU_ID_1] = ipis_rcvd[RECEIVER_VCPU_ID_2] = 0;
+
+	/* 'Slow' HvCallSendSyntheticClusterIpi to RECEIVER_VCPU_ID_1 */
+	ipi->vector = IPI_VECTOR;
+	ipi->cpu_mask = 1 << RECEIVER_VCPU_ID_1;
+	res = hypercall(HVCALL_SEND_IPI, pgs_gpa, pgs_gpa + 4096);
+	GUEST_ASSERT((res & 0xffff) == 0);
+	nop_loop();
+	GUEST_ASSERT(ipis_rcvd[RECEIVER_VCPU_ID_1] == ++ipis_expected[0]);
+	GUEST_ASSERT(ipis_rcvd[RECEIVER_VCPU_ID_2] == ipis_expected[1]);
+	GUEST_SYNC(stage++);
+	/* 'Fast' HvCallSendSyntheticClusterIpi to RECEIVER_VCPU_ID_1 */
+	res = hypercall(HVCALL_SEND_IPI | HV_HYPERCALL_FAST_BIT,
+			IPI_VECTOR, 1 << RECEIVER_VCPU_ID_1);
+	GUEST_ASSERT((res & 0xffff) == 0);
+	nop_loop();
+	GUEST_ASSERT(ipis_rcvd[RECEIVER_VCPU_ID_1] == ++ipis_expected[0]);
+	GUEST_ASSERT(ipis_rcvd[RECEIVER_VCPU_ID_2] == ipis_expected[1]);
+	GUEST_SYNC(stage++);
+
+	/* 'Slow' HvCallSendSyntheticClusterIpiEx to RECEIVER_VCPU_ID_1 */
+	memset(hcall_page, 0, 4096);
+	ipi_ex->vector = IPI_VECTOR;
+	ipi_ex->vp_set.format = HV_GENERIC_SET_SPARSE_4K;
+	ipi_ex->vp_set.valid_bank_mask = 1 << 0;
+	ipi_ex->vp_set.bank_contents[0] = BIT(RECEIVER_VCPU_ID_1);
+	res = hypercall(HVCALL_SEND_IPI_EX | (1 << HV_HYPERCALL_VARHEAD_OFFSET),
+			pgs_gpa, pgs_gpa + 4096);
+	GUEST_ASSERT((res & 0xffff) == 0);
+	nop_loop();
+	GUEST_ASSERT(ipis_rcvd[RECEIVER_VCPU_ID_1] == ++ipis_expected[0]);
+	GUEST_ASSERT(ipis_rcvd[RECEIVER_VCPU_ID_2] == ipis_expected[1]);
+	GUEST_SYNC(stage++);
+	/* 'XMM Fast' HvCallSendSyntheticClusterIpiEx to RECEIVER_VCPU_ID_1 */
+	sync_to_xmm(&ipi_ex->vp_set.valid_bank_mask);
+	res = hypercall(HVCALL_SEND_IPI_EX | HV_HYPERCALL_FAST_BIT |
+			(1 << HV_HYPERCALL_VARHEAD_OFFSET),
+			IPI_VECTOR, HV_GENERIC_SET_SPARSE_4K);
+	GUEST_ASSERT((res & 0xffff) == 0);
+	nop_loop();
+	GUEST_ASSERT(ipis_rcvd[RECEIVER_VCPU_ID_1] == ++ipis_expected[0]);
+	GUEST_ASSERT(ipis_rcvd[RECEIVER_VCPU_ID_2] == ipis_expected[1]);
+	GUEST_SYNC(stage++);
+
+	/* 'Slow' HvCallSendSyntheticClusterIpiEx to RECEIVER_VCPU_ID_2 */
+	memset(hcall_page, 0, 4096);
+	ipi_ex->vector = IPI_VECTOR;
+	ipi_ex->vp_set.format = HV_GENERIC_SET_SPARSE_4K;
+	ipi_ex->vp_set.valid_bank_mask = 1 << 1;
+	ipi_ex->vp_set.bank_contents[0] = BIT(RECEIVER_VCPU_ID_2 - 64);
+	res = hypercall(HVCALL_SEND_IPI_EX | (1 << HV_HYPERCALL_VARHEAD_OFFSET),
+			pgs_gpa, pgs_gpa + 4096);
+	GUEST_ASSERT((res & 0xffff) == 0);
+	nop_loop();
+	GUEST_ASSERT(ipis_rcvd[RECEIVER_VCPU_ID_1] == ipis_expected[0]);
+	GUEST_ASSERT(ipis_rcvd[RECEIVER_VCPU_ID_2] == ++ipis_expected[1]);
+	GUEST_SYNC(stage++);
+	/* 'XMM Fast' HvCallSendSyntheticClusterIpiEx to RECEIVER_VCPU_ID_2 */
+	sync_to_xmm(&ipi_ex->vp_set.valid_bank_mask);
+	res = hypercall(HVCALL_SEND_IPI_EX | HV_HYPERCALL_FAST_BIT |
+			(1 << HV_HYPERCALL_VARHEAD_OFFSET),
+			IPI_VECTOR, HV_GENERIC_SET_SPARSE_4K);
+	GUEST_ASSERT((res & 0xffff) == 0);
+	nop_loop();
+	GUEST_ASSERT(ipis_rcvd[RECEIVER_VCPU_ID_1] == ipis_expected[0]);
+	GUEST_ASSERT(ipis_rcvd[RECEIVER_VCPU_ID_2] == ++ipis_expected[1]);
+	GUEST_SYNC(stage++);
+
+	/* 'Slow' HvCallSendSyntheticClusterIpiEx to both RECEIVER_VCPU_ID_{1,2} */
+	memset(hcall_page, 0, 4096);
+	ipi_ex->vector = IPI_VECTOR;
+	ipi_ex->vp_set.format = HV_GENERIC_SET_SPARSE_4K;
+	ipi_ex->vp_set.valid_bank_mask = 1 << 1 | 1;
+	ipi_ex->vp_set.bank_contents[0] = BIT(RECEIVER_VCPU_ID_1);
+	ipi_ex->vp_set.bank_contents[1] = BIT(RECEIVER_VCPU_ID_2 - 64);
+	res = hypercall(HVCALL_SEND_IPI_EX | (2 << HV_HYPERCALL_VARHEAD_OFFSET),
+			pgs_gpa, pgs_gpa + 4096);
+	GUEST_ASSERT((res & 0xffff) == 0);
+	nop_loop();
+	GUEST_ASSERT(ipis_rcvd[RECEIVER_VCPU_ID_1] == ++ipis_expected[0]);
+	GUEST_ASSERT(ipis_rcvd[RECEIVER_VCPU_ID_2] == ++ipis_expected[1]);
+	GUEST_SYNC(stage++);
+	/* 'XMM Fast' HvCallSendSyntheticClusterIpiEx to both RECEIVER_VCPU_ID_{1, 2} */
+	sync_to_xmm(&ipi_ex->vp_set.valid_bank_mask);
+	res = hypercall(HVCALL_SEND_IPI_EX | HV_HYPERCALL_FAST_BIT |
+			(2 << HV_HYPERCALL_VARHEAD_OFFSET),
+			IPI_VECTOR, HV_GENERIC_SET_SPARSE_4K);
+	GUEST_ASSERT((res & 0xffff) == 0);
+	nop_loop();
+	GUEST_ASSERT(ipis_rcvd[RECEIVER_VCPU_ID_1] == ++ipis_expected[0]);
+	GUEST_ASSERT(ipis_rcvd[RECEIVER_VCPU_ID_2] == ++ipis_expected[1]);
+	GUEST_SYNC(stage++);
+
+	/* 'Slow' HvCallSendSyntheticClusterIpiEx to HV_GENERIC_SET_ALL */
+	memset(hcall_page, 0, 4096);
+	ipi_ex->vector = IPI_VECTOR;
+	ipi_ex->vp_set.format = HV_GENERIC_SET_ALL;
+	res = hypercall(HVCALL_SEND_IPI_EX,
+			pgs_gpa, pgs_gpa + 4096);
+	GUEST_ASSERT((res & 0xffff) == 0);
+	nop_loop();
+	GUEST_ASSERT(ipis_rcvd[RECEIVER_VCPU_ID_1] == ++ipis_expected[0]);
+	GUEST_ASSERT(ipis_rcvd[RECEIVER_VCPU_ID_2] == ++ipis_expected[1]);
+	GUEST_SYNC(stage++);
+	/* 'XMM Fast' HvCallSendSyntheticClusterIpiEx to HV_GENERIC_SET_ALL */
+	sync_to_xmm(&ipi_ex->vp_set.valid_bank_mask);
+	res = hypercall(HVCALL_SEND_IPI_EX | HV_HYPERCALL_FAST_BIT,
+			IPI_VECTOR, HV_GENERIC_SET_ALL);
+	GUEST_ASSERT((res & 0xffff) == 0);
+	nop_loop();
+	GUEST_ASSERT(ipis_rcvd[RECEIVER_VCPU_ID_1] == ++ipis_expected[0]);
+	GUEST_ASSERT(ipis_rcvd[RECEIVER_VCPU_ID_2] == ++ipis_expected[1]);
+	GUEST_SYNC(stage++);
+
+	GUEST_DONE();
+}
+
+static void *vcpu_thread(void *arg)
+{
+	struct thread_params *params = (struct thread_params *)arg;
+	struct ucall uc;
+	int old;
+	int r;
+	unsigned int exit_reason;
+
+	r = pthread_setcanceltype(PTHREAD_CANCEL_ASYNCHRONOUS, &old);
+	TEST_ASSERT(r == 0,
+		    "pthread_setcanceltype failed on vcpu_id=%u with errno=%d",
+		    params->vcpu_id, r);
+
+	vcpu_run(params->vm, params->vcpu_id);
+	exit_reason = vcpu_state(params->vm, params->vcpu_id)->exit_reason;
+
+	TEST_ASSERT(exit_reason == KVM_EXIT_IO,
+		    "vCPU %u exited with unexpected exit reason %u-%s, expected KVM_EXIT_IO",
+		    params->vcpu_id, exit_reason, exit_reason_str(exit_reason));
+
+	if (get_ucall(params->vm, params->vcpu_id, &uc) == UCALL_ABORT) {
+		TEST_ASSERT(false,
+			    "vCPU %u exited with error: %s.\n",
+			    params->vcpu_id, (const char *)uc.args[0]);
+	}
+
+	return NULL;
+}
+
+static void cancel_join_vcpu_thread(pthread_t thread, uint32_t vcpu_id)
+{
+	void *retval;
+	int r;
+
+	r = pthread_cancel(thread);
+	TEST_ASSERT(r == 0,
+		    "pthread_cancel on vcpu_id=%d failed with errno=%d",
+		    vcpu_id, r);
+
+	r = pthread_join(thread, &retval);
+	TEST_ASSERT(r == 0,
+		    "pthread_join on vcpu_id=%d failed with errno=%d",
+		    vcpu_id, r);
+	TEST_ASSERT(retval == PTHREAD_CANCELED,
+		    "expected retval=%p, got %p", PTHREAD_CANCELED,
+		    retval);
+}
+
+int main(int argc, char *argv[])
+{
+	int r;
+	pthread_t threads[2];
+	struct thread_params params[2];
+	struct kvm_vm *vm;
+	struct kvm_run *run;
+	vm_vaddr_t hcall_page;
+	struct ucall uc;
+	int stage = 1;
+
+	vm = vm_create_default(SENDER_VCPU_ID, 0, sender_guest_code);
+	params[0].vm = vm;
+	params[1].vm = vm;
+
+	/* Hypercall input/output */
+	hcall_page = vm_vaddr_alloc_pages(vm, 2);
+	memset(addr_gva2hva(vm, hcall_page), 0x0, 2 * getpagesize());
+
+	vm_init_descriptor_tables(vm);
+
+	vm_vcpu_add_default(vm, RECEIVER_VCPU_ID_1, receiver_code);
+	vcpu_init_descriptor_tables(vm, RECEIVER_VCPU_ID_1);
+	vcpu_args_set(vm, RECEIVER_VCPU_ID_1, 2, hcall_page, addr_gva2gpa(vm, hcall_page));
+	vcpu_set_msr(vm, RECEIVER_VCPU_ID_1, HV_X64_MSR_VP_INDEX, RECEIVER_VCPU_ID_1);
+	vcpu_set_hv_cpuid(vm, RECEIVER_VCPU_ID_1);
+
+	vm_vcpu_add_default(vm, RECEIVER_VCPU_ID_2, receiver_code);
+	vcpu_init_descriptor_tables(vm, RECEIVER_VCPU_ID_2);
+	vcpu_args_set(vm, RECEIVER_VCPU_ID_2, 2, hcall_page, addr_gva2gpa(vm, hcall_page));
+	vcpu_set_msr(vm, RECEIVER_VCPU_ID_2, HV_X64_MSR_VP_INDEX, RECEIVER_VCPU_ID_2);
+	vcpu_set_hv_cpuid(vm, RECEIVER_VCPU_ID_2);
+
+	vm_install_exception_handler(vm, IPI_VECTOR, guest_ipi_handler);
+
+	vcpu_args_set(vm, SENDER_VCPU_ID, 2, hcall_page, addr_gva2gpa(vm, hcall_page));
+	vcpu_set_hv_cpuid(vm, SENDER_VCPU_ID);
+
+	params[0].vcpu_id = RECEIVER_VCPU_ID_1;
+	r = pthread_create(&threads[0], NULL, vcpu_thread, &params[0]);
+	TEST_ASSERT(r == 0,
+		    "pthread_create halter failed errno=%d", errno);
+
+	params[1].vcpu_id = RECEIVER_VCPU_ID_2;
+	r = pthread_create(&threads[1], NULL, vcpu_thread, &params[1]);
+	TEST_ASSERT(r == 0,
+		    "pthread_create halter failed errno=%d", errno);
+
+	run = vcpu_state(vm, SENDER_VCPU_ID);
+
+	while (true) {
+		r = _vcpu_run(vm, SENDER_VCPU_ID);
+		TEST_ASSERT(!r, "vcpu_run failed: %d\n", r);
+		TEST_ASSERT(run->exit_reason == KVM_EXIT_IO,
+			    "unexpected exit reason: %u (%s)",
+			    run->exit_reason, exit_reason_str(run->exit_reason));
+
+		switch (get_ucall(vm, SENDER_VCPU_ID, &uc)) {
+		case UCALL_SYNC:
+			TEST_ASSERT(uc.args[1] == stage,
+				    "Unexpected stage: %ld (%d expected)\n",
+				    uc.args[1], stage);
+			break;
+		case UCALL_ABORT:
+			TEST_FAIL("%s at %s:%ld", (const char *)uc.args[0],
+				  __FILE__, uc.args[1]);
+			return 1;
+		case UCALL_DONE:
+			return 0;
+		}
+
+		stage++;
+	}
+
+	cancel_join_vcpu_thread(threads[0], RECEIVER_VCPU_ID_1);
+	cancel_join_vcpu_thread(threads[1], RECEIVER_VCPU_ID_2);
+	kvm_vm_free(vm);
+
+	return 0;
+}

[v3,24/34] KVM: selftests: Hyper-V PV IPI selftest

Commit Message

Comments

Patch