From patchwork Thu Nov 28 00:43:39 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sean Christopherson X-Patchwork-Id: 13887462 Received: from mail-pj1-f73.google.com (mail-pj1-f73.google.com [209.85.216.73]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 8BA0CE571 for ; Thu, 28 Nov 2024 00:43:49 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.216.73 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1732754631; cv=none; b=nN3h5ZXS4++9Mqr8jEUlvxKz8DgaaeShAljuugNSJ5f0ORJcahuBSB3VXw/Hobh5Uj/dhtstHBwW5AyF4TZAXHA+JTmzhrbcPksJfaZMjHJiUvu+Cs+p4voIlo5qIsAw6S6zhcHHLhJKRpbJrkSWEo2IrLRhN3Rkq+ZyUa6WGoU= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1732754631; c=relaxed/simple; bh=EACB2yPyU5Yw8+WwKYitNwB9tkKiB/UcMdmwylhn5ac=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=glqqbo59FRrZe0K7RPIjPwKPMZr33NEsKgsdxBB5dMwnHEhTSlKwbHRdZ1mtvB9r/4NVxxC/rOFARyW92jOImNrYkdtUE05cPUVCJoYB6CSunVm4hhtqtYXMC/BCeCp/OM5R3t+SVw7NaC4MuSKX5jBZjWfEM345n/0BvDh3s68= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=v7XxW7Eo; arc=none smtp.client-ip=209.85.216.73 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="v7XxW7Eo" Received: by mail-pj1-f73.google.com with SMTP id 98e67ed59e1d1-2ea65508e51so412717a91.0 for ; Wed, 27 Nov 2024 16:43:49 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1732754629; x=1733359429; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:from:to:cc:subject:date:message-id:reply-to; bh=H6v00VWPrFlV9G8PXjNGHz1lcbArd14LtnA+J13xoHI=; b=v7XxW7EoO+ncriNDu2gIHvbXy3S5xYONjO+Tf0yNsXsxEn8y6N3rh5wETBW6Tkebqa /RSErLXc8BX0QLx/r/i3LGRs4MJwvLuPUkvCxE8EQGrWfd2IM1FVHou0vWBKhB4qHfnR DC626wTtM/z4pDK4dzZeBj3uzSy3Mv7fOWtdC/65d9bGU9oseiO0LmGVrUQQwnSExSCZ SBpi/PSVK5jclZsl47hQHhddfM3q+PapNBtwbBWpJgqV4X+J7d+5jAYGYDr3OYfCJ+SV Hhe1gqn/npwqYlC37ShZibIpIqaM6lA8kjTUVNHXtr0M0gSeI98t6iGOy+81tlouIU9S Y9Dw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1732754629; x=1733359429; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=H6v00VWPrFlV9G8PXjNGHz1lcbArd14LtnA+J13xoHI=; b=MWJXM+Trqu7KhSTVArfiUAkuyVLxTmXHkWAwTk6P+qfUCS4ehWN6VleM8uMsCuTCbw DgH46SoiMXMp1tPHbheE07tsq8LkKyj45txJjUDrauifdji7J9z3yek9qAfzmOpoUO0R QPJXkXG9UeYWMTqhPV78vCbYYiQWmyFCbdxzSTPU5N2msshRc5VilDzFdINXbB+ftP7S Ux/O6fOfMBAvDI+m6Ut7tvMv4/qs4vgL/2Ocg/Ctb9+Z0VDeREu93PAy4LgxDY5NzOLT dSlVSIE26estY/pwVCURFkvq18c357DiINcG453rUlKrPil12pTJtGflw+vYIpDUN9KX Lqfw== X-Gm-Message-State: AOJu0Yygzxq2937JZtqbcxA3O6qwE8KTqSw5Unp0tkjElDktKs0VqyA8 oIrhaMR4wgv+Uqcn8NqhdmUPByPLwn9s04eLXPHGUVSjZ/+s+nDvZMDGtJ1XXEklK8n4rWX+pn6 Ylg== X-Google-Smtp-Source: AGHT+IEK9pbCeaFZWXX0PLadQxkBbAx1o35IfYFdtf0yqxaDLee26TJikEv6kZT/3lUfdcYbGJKEkLcA4Vc= X-Received: from pjbpd1.prod.google.com ([2002:a17:90b:1dc1:b0:2ea:448a:8cd1]) (user=seanjc job=prod-delivery.src-stubby-dispatcher) by 2002:a17:90b:4d0d:b0:2ea:bf1c:1e3a with SMTP id 98e67ed59e1d1-2ee08eb2ae0mr7730683a91.12.1732754629027; Wed, 27 Nov 2024 16:43:49 -0800 (PST) Reply-To: Sean Christopherson Date: Wed, 27 Nov 2024 16:43:39 -0800 In-Reply-To: <20241128004344.4072099-1-seanjc@google.com> Precedence: bulk X-Mailing-List: kvm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20241128004344.4072099-1-seanjc@google.com> X-Mailer: git-send-email 2.47.0.338.g60cca15819-goog Message-ID: <20241128004344.4072099-2-seanjc@google.com> Subject: [PATCH v4 1/6] KVM: x86: Play nice with protected guests in complete_hypercall_exit() From: Sean Christopherson To: Sean Christopherson , Paolo Bonzini Cc: kvm@vger.kernel.org, linux-kernel@vger.kernel.org, Tom Lendacky , Binbin Wu , Isaku Yamahata , Kai Huang , Xiaoyao Li Use is_64_bit_hypercall() instead of is_64_bit_mode() to detect a 64-bit hypercall when completing said hypercall. For guests with protected state, e.g. SEV-ES and SEV-SNP, KVM must assume the hypercall was made in 64-bit mode as the vCPU state needed to detect 64-bit mode is unavailable. Hacking the sev_smoke_test selftest to generate a KVM_HC_MAP_GPA_RANGE hypercall via VMGEXIT trips the WARN: ------------[ cut here ]------------ WARNING: CPU: 273 PID: 326626 at arch/x86/kvm/x86.h:180 complete_hypercall_exit+0x44/0xe0 [kvm] Modules linked in: kvm_amd kvm ... [last unloaded: kvm] CPU: 273 UID: 0 PID: 326626 Comm: sev_smoke_test Not tainted 6.12.0-smp--392e932fa0f3-feat #470 Hardware name: Google Astoria/astoria, BIOS 0.20240617.0-0 06/17/2024 RIP: 0010:complete_hypercall_exit+0x44/0xe0 [kvm] Call Trace: kvm_arch_vcpu_ioctl_run+0x2400/0x2720 [kvm] kvm_vcpu_ioctl+0x54f/0x630 [kvm] __se_sys_ioctl+0x6b/0xc0 do_syscall_64+0x83/0x160 entry_SYSCALL_64_after_hwframe+0x76/0x7e ---[ end trace 0000000000000000 ]--- Fixes: b5aead0064f3 ("KVM: x86: Assume a 64-bit hypercall for guests with protected state") Cc: stable@vger.kernel.org Cc: Tom Lendacky Signed-off-by: Sean Christopherson Reviewed-by: Xiaoyao Li --- arch/x86/kvm/x86.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c index 2e713480933a..0b2fe4aa04a2 100644 --- a/arch/x86/kvm/x86.c +++ b/arch/x86/kvm/x86.c @@ -9976,7 +9976,7 @@ static int complete_hypercall_exit(struct kvm_vcpu *vcpu) { u64 ret = vcpu->run->hypercall.ret; - if (!is_64_bit_mode(vcpu)) + if (!is_64_bit_hypercall(vcpu)) ret = (u32)ret; kvm_rax_write(vcpu, ret); ++vcpu->stat.hypercalls; From patchwork Thu Nov 28 00:43:40 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sean Christopherson X-Patchwork-Id: 13887463 Received: from mail-pj1-f74.google.com (mail-pj1-f74.google.com [209.85.216.74]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 794E61AAC4 for ; Thu, 28 Nov 2024 00:43:51 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.216.74 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1732754632; cv=none; b=DvMMvVLNitoDhGHwfj3t3QZKxnTjCj8eih+dFbDr1K5/eSk9BNqpOeA9n18E+hFMc149OTLQgVHjeouqSl0TNSXG3CJXIZkOSCupuXgoDoDItbls8yvZ4R4cos4LDOgyEcopB3p59+GY6NWE3/bW0mnXIVJf8VzGLi6tArBoTuE= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1732754632; c=relaxed/simple; bh=FRVS2g+fG0f7ZYJIgFJENJzs0OeU6x/7e3vS+WJBqvk=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=Zfp+MQD8seJtckNP8T8pCOP8tEcURspqeGvcOxke7rAgcWql+miVqQqySX4pomlHt/yiLqPaR8v/T4MUmj6c2CItr4dGjtTQXFTw0FDwlSPiOfPo2+DYttSG5LwBnVPzUi3AhDqenzpGhge5d21pKzu1OgzQsMzJM+bpVAIQ8Xs= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=vVv115pt; arc=none smtp.client-ip=209.85.216.74 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="vVv115pt" Received: by mail-pj1-f74.google.com with SMTP id 98e67ed59e1d1-2e9ff7aa7eeso1145252a91.1 for ; Wed, 27 Nov 2024 16:43:51 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1732754631; x=1733359431; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:from:to:cc:subject:date:message-id:reply-to; bh=wXSz7L/2gLB1nXVbSFWAt9DGwp25yrxYRtHtDBXMeOs=; b=vVv115ptRwEuNlic1gAJJK4/+kcJkoiMNDcKeiFEAhXkDWXrppTg3cdfvlZ3ZWb57z FXp/BWwPcJckC/DVW/m7sQkZU9oFLUA4zyonj2q+6IbknIDQJ3WBWjEDKnDXWP7+NNcJ sleNVJbXc/mqQCDP8TmNhOKSc4RlvfDZLq9GuAu44sHuvSYWHygpT06NVMbeaZ1Qw8db /SA5KGsFa3eQ68PDbGN5Kb2XPI2lI/Jw44wrhJ3XUhIN6Cv+SsI5QykUqp/vMdtE57Fk 6mXa3sGUIXTydl4aXVTxyP6/93gpwzNefzBN+ns7JqN8ynbaKNoNKvJ0N90Toaz8qJaw 7yHg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1732754631; x=1733359431; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=wXSz7L/2gLB1nXVbSFWAt9DGwp25yrxYRtHtDBXMeOs=; b=efLRyW0kgRRef+TP8EBZyDvqGpldu5xkHYIZ3tIIdwpiC+OW3dPmpOo3cHuGeeiE+s 9DyE+M/SKHOdqvCU3QI+kaC25bIUs0ABl0a9tuPugEX7J00SwHRW6NO9Gn6Ayy2tTOrO eS6nzxX/lXb5jeouJa1rknAZXfFlfi54aBklf/JDVUjvh0fhKICXuyRoHF/dAhDICdfS 7fP9OUgGMkjaB0/ztTP4aLuOqlyy7K3xRbWc/fEQVZ+OHYwzP+eSjaLCbJ83oceZqWFl xvWvGxzFrd2qqPsbcDA7PLpMPvh+q99I4gBPvKERnAOgS/jpVQQ7vyNh4c4LJqorYUjc rj0Q== X-Gm-Message-State: AOJu0Yz3LDg8/9MKuleBwe2nSi1oTS+IKcYalJYPitLeFwiX02tDNHZ8 6Bin/zSNIOznNrkdqT7szspvTKFD7w2kYqKs4UUbpoAXKFJFrlP9GO/yOuN4iaKScDcVitH7jF4 bFA== X-Google-Smtp-Source: AGHT+IHtc6j3/IEkFk2MarwmYZptNoBmam/jKP/ZpVGuOWQAsWGuBBWLJG0GeFfwVxzsBB4VbyBz5vgvYEs= X-Received: from pjbhl3.prod.google.com ([2002:a17:90b:1343:b0:2e5:ef8a:48c4]) (user=seanjc job=prod-delivery.src-stubby-dispatcher) by 2002:a17:90a:ad8d:b0:2d8:85fc:464c with SMTP id 98e67ed59e1d1-2ee25b3a158mr2128961a91.11.1732754630678; Wed, 27 Nov 2024 16:43:50 -0800 (PST) Reply-To: Sean Christopherson Date: Wed, 27 Nov 2024 16:43:40 -0800 In-Reply-To: <20241128004344.4072099-1-seanjc@google.com> Precedence: bulk X-Mailing-List: kvm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20241128004344.4072099-1-seanjc@google.com> X-Mailer: git-send-email 2.47.0.338.g60cca15819-goog Message-ID: <20241128004344.4072099-3-seanjc@google.com> Subject: [PATCH v4 2/6] KVM: x86: Add a helper to check for user interception of KVM hypercalls From: Sean Christopherson To: Sean Christopherson , Paolo Bonzini Cc: kvm@vger.kernel.org, linux-kernel@vger.kernel.org, Tom Lendacky , Binbin Wu , Isaku Yamahata , Kai Huang , Xiaoyao Li From: Binbin Wu Add and use user_exit_on_hypercall() to check if userspace wants to handle a KVM hypercall instead of open-coding the logic everywhere. No functional change intended. Signed-off-by: Binbin Wu Reviewed-by: Isaku Yamahata Reviewed-by: Kai Huang Reviewed-by: Xiaoyao Li [sean: squash into one patch, keep explicit KVM_HC_MAP_GPA_RANGE check] Signed-off-by: Sean Christopherson --- arch/x86/kvm/svm/sev.c | 4 ++-- arch/x86/kvm/x86.c | 2 +- arch/x86/kvm/x86.h | 5 +++++ 3 files changed, 8 insertions(+), 3 deletions(-) diff --git a/arch/x86/kvm/svm/sev.c b/arch/x86/kvm/svm/sev.c index 72674b8825c4..6ac6312c4d57 100644 --- a/arch/x86/kvm/svm/sev.c +++ b/arch/x86/kvm/svm/sev.c @@ -3640,7 +3640,7 @@ static int snp_begin_psc_msr(struct vcpu_svm *svm, u64 ghcb_msr) return 1; /* resume guest */ } - if (!(vcpu->kvm->arch.hypercall_exit_enabled & (1 << KVM_HC_MAP_GPA_RANGE))) { + if (!user_exit_on_hypercall(vcpu->kvm, KVM_HC_MAP_GPA_RANGE)) { set_ghcb_msr(svm, GHCB_MSR_PSC_RESP_ERROR); return 1; /* resume guest */ } @@ -3723,7 +3723,7 @@ static int snp_begin_psc(struct vcpu_svm *svm, struct psc_buffer *psc) bool huge; u64 gfn; - if (!(vcpu->kvm->arch.hypercall_exit_enabled & (1 << KVM_HC_MAP_GPA_RANGE))) { + if (!user_exit_on_hypercall(vcpu->kvm, KVM_HC_MAP_GPA_RANGE)) { snp_complete_psc(svm, VMGEXIT_PSC_ERROR_GENERIC); return 1; } diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c index 0b2fe4aa04a2..13fe5d6eb8f3 100644 --- a/arch/x86/kvm/x86.c +++ b/arch/x86/kvm/x86.c @@ -10041,7 +10041,7 @@ unsigned long __kvm_emulate_hypercall(struct kvm_vcpu *vcpu, unsigned long nr, u64 gpa = a0, npages = a1, attrs = a2; ret = -KVM_ENOSYS; - if (!(vcpu->kvm->arch.hypercall_exit_enabled & (1 << KVM_HC_MAP_GPA_RANGE))) + if (!user_exit_on_hypercall(vcpu->kvm, KVM_HC_MAP_GPA_RANGE)) break; if (!PAGE_ALIGNED(gpa) || !npages || diff --git a/arch/x86/kvm/x86.h b/arch/x86/kvm/x86.h index ec623d23d13d..45dd53284dbd 100644 --- a/arch/x86/kvm/x86.h +++ b/arch/x86/kvm/x86.h @@ -612,4 +612,9 @@ int kvm_sev_es_string_io(struct kvm_vcpu *vcpu, unsigned int size, unsigned int port, void *data, unsigned int count, int in); +static inline bool user_exit_on_hypercall(struct kvm *kvm, unsigned long hc_nr) +{ + return kvm->arch.hypercall_exit_enabled & BIT(hc_nr); +} + #endif From patchwork Thu Nov 28 00:43:41 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sean Christopherson X-Patchwork-Id: 13887464 Received: from mail-pj1-f74.google.com (mail-pj1-f74.google.com [209.85.216.74]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 09171219FC for ; Thu, 28 Nov 2024 00:43:52 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.216.74 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1732754634; cv=none; b=WtUbLzSZS5bZx5TCqYZaQUlbfNBncQaQJaO+UGCqAl//3Uy9v8GOg+gQUNr99i2fAQlmCgluow39sznFvegmzaAqEw1gBNxZdvAxQZhfYZS6LeLjD5Yf/O4XeXkJEvSPlxxqe37oZoFG0ucxMa9PvHtaBeHEHQQThlgx8UK38FQ= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1732754634; c=relaxed/simple; bh=jyPX3MdAUnX1bDy5TiI1V6ZoW1meXejrHFasqL8eDK4=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=RG6doMLeEt/WeYYip9mzQznH6vhkm04n3Pz6PO22zQN51yXq8O2/aTOTnI8ZdzllKTislwPg4Zsw1XdvKlVM+4+sc+aoHjpB7U23klhzqDPoDIuyaNIRE/bVwOMPWPhHs6u4uqCo5qmuOs6/8BaFVdRQiIHAtApyZBtl+DU8wcY= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=LRbohuJA; arc=none smtp.client-ip=209.85.216.74 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="LRbohuJA" Received: by mail-pj1-f74.google.com with SMTP id 98e67ed59e1d1-2ee0c9962daso311237a91.2 for ; Wed, 27 Nov 2024 16:43:52 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1732754632; x=1733359432; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:from:to:cc:subject:date:message-id:reply-to; bh=1w6JVsg4OUo1AnaxThquyWJswnSGtvGXHMhG9ntSUjE=; b=LRbohuJAneS6Fzwyo0Nu1qq3tkK9wU4TjGm/exnwHhou4HP0qz9ZFla55/i07tyYm0 TlueK+Uk8TDzc9+KfSvPEs3vi+tnaa1jSWLw33uhrBHngTEnQutuPbYrnV50SlOBNfcp aH05IDaep7q9HEE84f0Z2fPgtvY0BzBo1XLza0Rpu7yNgQn3lYFmUiBAyluZWXdgI6M3 uQwDmlvxVqRniicympzZmF0roTylHiFgvy90VFHSUzq6/pM2vbLlEXXyhUOj4+hpHMfV nHsCgtgz7St2OdhdD3x+Hd3PaBmZ2OXQqoRZ3lQEcRgkibYVavk8ZOsx+VI/b6vM0Ow1 55hg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1732754632; x=1733359432; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=1w6JVsg4OUo1AnaxThquyWJswnSGtvGXHMhG9ntSUjE=; b=c3NjGNi23CjVYGfERMFxbq3X0nGMigGD1AKz0DT+Io5b91s2cRIa8QiqJPY5K/oOY7 NUXkmFo4ZIRDHwR1oZQL8dm6nHpKKNBOesfF0rdUnzDIEBfIegRVMsKKlBsF6+tW8hHj 4rFSCUJZFJtIhKn3rCUUDgyPiCo65s5eDBXYCZ5kF0mNFDtQgOQ6tf6lU2rwsw7xpyeP qttCV1CtcplCum4OhIftkLXBl86FAx5UbH8W6JQFEIrShNipAwN6IVlb8gNQBhEFYs3K YPLYQ71uutvLVue7APkmZBbWjG+9fVkP9kQuoQO9ARkkRxDrdB8peeiNOhX8Z8A7dNHh J5AQ== X-Gm-Message-State: AOJu0YwjAUJTAEVgWHimg7bmUKW98AX243etnkJZtSdSH2m/GJBq1dvY 6xVMJIyhhNSH8ByBXhN2H0prGVi/tluqSUPohjPQVbyUmmtsgxm4+U3JS+ItWd7GgdxcxmtEnVd LNQ== X-Google-Smtp-Source: AGHT+IEQTw6NycY68SHBk7WI4L2KbulF4hVElsyaIaLj1wJbiWRwmM/PjWD0Btsx9ro7hcsaxrPuaWvf6Js= X-Received: from pjbli14.prod.google.com ([2002:a17:90b:48ce:b0:2e2:9f67:1ca3]) (user=seanjc job=prod-delivery.src-stubby-dispatcher) by 2002:a17:90b:3d12:b0:2ea:4c5f:923e with SMTP id 98e67ed59e1d1-2ee08e9a0dbmr6652768a91.5.1732754632474; Wed, 27 Nov 2024 16:43:52 -0800 (PST) Reply-To: Sean Christopherson Date: Wed, 27 Nov 2024 16:43:41 -0800 In-Reply-To: <20241128004344.4072099-1-seanjc@google.com> Precedence: bulk X-Mailing-List: kvm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20241128004344.4072099-1-seanjc@google.com> X-Mailer: git-send-email 2.47.0.338.g60cca15819-goog Message-ID: <20241128004344.4072099-4-seanjc@google.com> Subject: [PATCH v4 3/6] KVM: x86: Move "emulate hypercall" function declarations to x86.h From: Sean Christopherson To: Sean Christopherson , Paolo Bonzini Cc: kvm@vger.kernel.org, linux-kernel@vger.kernel.org, Tom Lendacky , Binbin Wu , Isaku Yamahata , Kai Huang , Xiaoyao Li Move the declarations for the hypercall emulation APIs to x86.h. While the helpers are exported, they are intended to be consumed only KVM vendor modules, i.e. don't need to exposed to the kernel at-large. No functional change intended. Signed-off-by: Sean Christopherson Reviewed-by: Xiaoyao Li --- arch/x86/include/asm/kvm_host.h | 6 ------ arch/x86/kvm/x86.h | 6 ++++++ 2 files changed, 6 insertions(+), 6 deletions(-) diff --git a/arch/x86/include/asm/kvm_host.h b/arch/x86/include/asm/kvm_host.h index e159e44a6a1b..c1251b371421 100644 --- a/arch/x86/include/asm/kvm_host.h +++ b/arch/x86/include/asm/kvm_host.h @@ -2181,12 +2181,6 @@ static inline void kvm_clear_apicv_inhibit(struct kvm *kvm, kvm_set_or_clear_apicv_inhibit(kvm, reason, false); } -unsigned long __kvm_emulate_hypercall(struct kvm_vcpu *vcpu, unsigned long nr, - unsigned long a0, unsigned long a1, - unsigned long a2, unsigned long a3, - int op_64_bit, int cpl); -int kvm_emulate_hypercall(struct kvm_vcpu *vcpu); - int kvm_mmu_page_fault(struct kvm_vcpu *vcpu, gpa_t cr2_or_gpa, u64 error_code, void *insn, int insn_len); void kvm_mmu_print_sptes(struct kvm_vcpu *vcpu, gpa_t gpa, const char *msg); diff --git a/arch/x86/kvm/x86.h b/arch/x86/kvm/x86.h index 45dd53284dbd..6db13b696468 100644 --- a/arch/x86/kvm/x86.h +++ b/arch/x86/kvm/x86.h @@ -617,4 +617,10 @@ static inline bool user_exit_on_hypercall(struct kvm *kvm, unsigned long hc_nr) return kvm->arch.hypercall_exit_enabled & BIT(hc_nr); } +unsigned long __kvm_emulate_hypercall(struct kvm_vcpu *vcpu, unsigned long nr, + unsigned long a0, unsigned long a1, + unsigned long a2, unsigned long a3, + int op_64_bit, int cpl); +int kvm_emulate_hypercall(struct kvm_vcpu *vcpu); + #endif From patchwork Thu Nov 28 00:43:42 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sean Christopherson X-Patchwork-Id: 13887465 Received: from mail-pj1-f74.google.com (mail-pj1-f74.google.com [209.85.216.74]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 8E6A93C463 for ; Thu, 28 Nov 2024 00:43:54 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.216.74 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1732754636; cv=none; b=TJFkaVQBKYAqRq/GG+zA5DFFiJ+pduL7RNr2kHk6cyElR7xnmKNF/SwQyUWuszpZk7aX8vjjPL7u+KjM7n87X3vraS4ppZg0ugPqeQq++HIcGDZqUrXiA8e48aSEanar8XZHmlcFQKQrOgrPrb4to7Lhhmv0lyFS9AYRyUwtLXE= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1732754636; c=relaxed/simple; bh=AgF1nLFRJT2jGgGNEvzsKs6m0B0/bKeLHYnQ3NQUIic=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=ua263tfaBbwWUIqCSJhXjvioRNr4Ufxa6DGn7a/9O+QMjvKHr0UwlZhl+O9OnleWb2R4gSdM9/KizxIWgHfenUdUfi9qcdap9GvDVDQE9obNR7tyTCjW47oWtYM26qr/F3EF+hmwo7m0kC9CMGE6r/5ztoCuQq3tvyCaxzHRzvw= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=iMcPu1mW; arc=none smtp.client-ip=209.85.216.74 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="iMcPu1mW" Received: by mail-pj1-f74.google.com with SMTP id 98e67ed59e1d1-2ea9d209e75so292516a91.3 for ; Wed, 27 Nov 2024 16:43:54 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1732754634; x=1733359434; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:from:to:cc:subject:date:message-id:reply-to; bh=dDZT1bmtmeMkj/4PbsRTAxMKbyk+c3Rb1LRLdcLSYzc=; b=iMcPu1mW5FyKzgEe/m80oOhGh0/ZDI2Ang0Nn0OEB2CphL89imrYEyCXvd+/PP/mxT 0zstjftCgJVBzenM9yIhb71rtHgX5cHWNIJ+OJrf1vD1KyXBEOEpKprsPFQe73bfkEW1 FiIliBS7fGZVwTCObDuwgxAOD8kslb8+Ks6nrJ8W80tQhI8qKzE+e4DAJ8uOzgn86eBs dNgwI3BWSD6/fb3cOHl54oXzMo0iz0yW87Vhife8aEtQZ4jVsVepYt83AqveVaYNs+ef SBmxzFLpyD4jnJWcz9d2SrusZNrY22scmALgyqzK8P9K4WMt4nsfxOT79fnUZDoxbIhp BebQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1732754634; x=1733359434; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=dDZT1bmtmeMkj/4PbsRTAxMKbyk+c3Rb1LRLdcLSYzc=; b=R0iOkuG90MybQkS5jOxy5IvglttstPnan8PZviLGmZI51tpglAN4sZIK9BczE+RQvY w6t1lwIcUc9ChI4UAuojGkMf47cWXUMrnxu+4wjXKZEeqF4/s6HkjIRoOL7CyBG9oxkC HQdJeGsvHwmIrorSV87T87Yl8qhaxlldxv8VC6mSEQYWA1OSJDjHu25Ubo8l3qZTZbIl HAjwrZPNEPtDMs31t+/QQUHYyNOEuoT1fYrrnaitMukjulPKiynXC04Dm6yvwnBoBL7I REJRfzjZOx9kzNVcJeg1rFg/zYmZ0GsUw03kH7ZzuHAPqJ200VuT2oogrRqlbBmwba9m GTHQ== X-Gm-Message-State: AOJu0YzyMNDShV3lNoMrbhmok+1ubB0ayWxiyM6zV3nzhm9L05QT6Hwm U1uyebgGwljs/YLSq4Rx7AtiMwQ48BXhJZhIYO0zky1gVozQHv0uO06Chkvr+u5DIJpCkiLXYlY YJA== X-Google-Smtp-Source: AGHT+IFvrKrb++Xh6P0AIe+4NdzweBJ/xiuvfkWJ8bzV4vHyK2OmIv7X9EgrfXBEvIn57suFkD1uefwv9ZA= X-Received: from pjbsq16.prod.google.com ([2002:a17:90b:5310:b0:2ea:22cd:7ff1]) (user=seanjc job=prod-delivery.src-stubby-dispatcher) by 2002:a17:90b:4c85:b0:2ea:aa69:106a with SMTP id 98e67ed59e1d1-2ee08e9bd37mr6163133a91.2.1732754634166; Wed, 27 Nov 2024 16:43:54 -0800 (PST) Reply-To: Sean Christopherson Date: Wed, 27 Nov 2024 16:43:42 -0800 In-Reply-To: <20241128004344.4072099-1-seanjc@google.com> Precedence: bulk X-Mailing-List: kvm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20241128004344.4072099-1-seanjc@google.com> X-Mailer: git-send-email 2.47.0.338.g60cca15819-goog Message-ID: <20241128004344.4072099-5-seanjc@google.com> Subject: [PATCH v4 4/6] KVM: x86: Bump hypercall stat prior to fully completing hypercall From: Sean Christopherson To: Sean Christopherson , Paolo Bonzini Cc: kvm@vger.kernel.org, linux-kernel@vger.kernel.org, Tom Lendacky , Binbin Wu , Isaku Yamahata , Kai Huang , Xiaoyao Li Increment the "hypercalls" stat for KVM hypercalls as soon as KVM knows it will skip the guest instruction, i.e. once KVM is committed to emulating the hypercall. Waiting until completion adds no known value, and creates a discrepancy where the stat will be bumped if KVM exits to userspace as a result of trying to skip the instruction, but not if the hypercall itself exits. Handling the stat in common code will also avoid the need for another helper to dedup code when TDX comes along (TDX needs a separate completion path due to GPR usage differences). Signed-off-by: Sean Christopherson Reviewed-by: Xiaoyao Li --- arch/x86/kvm/x86.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c index 13fe5d6eb8f3..11434752b467 100644 --- a/arch/x86/kvm/x86.c +++ b/arch/x86/kvm/x86.c @@ -9979,7 +9979,6 @@ static int complete_hypercall_exit(struct kvm_vcpu *vcpu) if (!is_64_bit_hypercall(vcpu)) ret = (u32)ret; kvm_rax_write(vcpu, ret); - ++vcpu->stat.hypercalls; return kvm_skip_emulated_instruction(vcpu); } @@ -9990,6 +9989,8 @@ unsigned long __kvm_emulate_hypercall(struct kvm_vcpu *vcpu, unsigned long nr, { unsigned long ret; + ++vcpu->stat.hypercalls; + trace_kvm_hypercall(nr, a0, a1, a2, a3); if (!op_64_bit) { @@ -10070,7 +10071,6 @@ unsigned long __kvm_emulate_hypercall(struct kvm_vcpu *vcpu, unsigned long nr, } out: - ++vcpu->stat.hypercalls; return ret; } EXPORT_SYMBOL_GPL(__kvm_emulate_hypercall); From patchwork Thu Nov 28 00:43:43 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sean Christopherson X-Patchwork-Id: 13887466 Received: from mail-pj1-f74.google.com (mail-pj1-f74.google.com [209.85.216.74]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 7C4831A270 for ; Thu, 28 Nov 2024 00:43:56 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.216.74 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1732754638; cv=none; b=VbFjjtc04we+lQleJlr7FJ54GJ3aRQx6XDSScB7Shjyt74gl+yDzS6RMUsQD9ildziL/FkqP+fp3ccy9q1rTxCOszo2R9MyoSJCZHl+YhMKcL2lr+FHNxSqcFVZhImU2XRAbCw5Gvo7BFm5Cg1bpjGAPTg7Bsaj8wijG9XEbGH4= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1732754638; c=relaxed/simple; bh=AEDUeCL/zaARx8onXji3t/YtfDw26liZdxyZL4gPBeI=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=VOLsNLLggXXPMAQR+cTB22xa1VKnqPdrxICBxJOQ3oyAERdy4WvXsO8+6mivB0k2I41BA603L2BvBxMm+favDvWhEHRoMZ0okYz2NjPMLqmoiXP0chK0vSqOS/yoSl8yX+hhgnuJm8DvhQHcW3NZ+Q+rP5wz1MqB4V/6Oz6odAA= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=MZtoZRpu; arc=none smtp.client-ip=209.85.216.74 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="MZtoZRpu" Received: by mail-pj1-f74.google.com with SMTP id 98e67ed59e1d1-2edeef8a994so339944a91.2 for ; Wed, 27 Nov 2024 16:43:56 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1732754636; x=1733359436; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:from:to:cc:subject:date:message-id:reply-to; bh=b+5GMpw/w1WHRIobpEWm5+/qy2gHIU4YsE7Y113p5dg=; b=MZtoZRpu338MJJmHtfYKDG3uwOHPGZhCnZm8GIf4tqm7RH1/zF1uxd19ucKx+2kR7I Z2TZ0yj/J4FGEjMswLH5BOp1Jo9br5CPk3AfnAEK4fBNt0KLywDDACvH8iGZ2r+GHO1s AX8dNAqsPiFUnYjKSG2fObPhN+6HwSzLo3V9YGRk7t3hpxTbcRCzGI10T8V0moGMxqUA MQjVNzYVELrzaoWSGPH7KIH9i/BttK1uuMvWSwbQsA81SMvFPfm7g9njcYMj8Nrqur9C qV1tjaKctHyQecApKVZIsMMpIVF64gXGEHRp+GVZYA0PY0jLacygQGGqxyczhEAON5M2 afVA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1732754636; x=1733359436; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=b+5GMpw/w1WHRIobpEWm5+/qy2gHIU4YsE7Y113p5dg=; b=bKzzlL6xvpXb8hU14RTn6+EKCnxYv0RWcY++tz9A3YRujXxjxvz30G7W3Ad9T3O4Kd xTiX2iBPEfZXoHjXihQyrisIQ1jgkXtb6Sp3IZ74xNkBZOFZwXmWyu/iIbzyymmCeY3I JPbAqDYYczmEvyrcxWc8ev+5LthxGuavN8sJwGh85cLusLsTGyfSkZqblmdDT6rg2C4O 1jrJObBD2AQEejRvUKKhtOU3kQmXZw0jKwMEySZXqwuNjQyVO3sexjgmhnY4cNHVFbH9 nxqBYvCqv4NF/fxd5r0MIQcP8LEqfIY53Vvo9CHgd16/a/vStXL88xGHDUfj0FnTjDdd KRTQ== X-Gm-Message-State: AOJu0YwcQ+8OsRQ1xj/Q6bCyajrhJjmL3NyIOKhWWZY5D1v14jXHmSW5 /JyjENYoddaXm7AZv5SrqycJBjKB6Tj6RsRgqEQzcaq+iQ5izkHyLbBAtxhz0Ud+MxiuqaqN5Ek d9A== X-Google-Smtp-Source: AGHT+IGosbHKAV+rkitlr6k2kWiYww6vT/dEz80J0WwFs8XqNGHk+Uosu6KGRZs/IjB6xkH6D7120BnwA0M= X-Received: from pjbli14.prod.google.com ([2002:a17:90b:48ce:b0:2e2:9f67:1ca3]) (user=seanjc job=prod-delivery.src-stubby-dispatcher) by 2002:a17:90b:48c1:b0:2ea:9f38:993c with SMTP id 98e67ed59e1d1-2ee08ecc4d9mr6651891a91.18.1732754635965; Wed, 27 Nov 2024 16:43:55 -0800 (PST) Reply-To: Sean Christopherson Date: Wed, 27 Nov 2024 16:43:43 -0800 In-Reply-To: <20241128004344.4072099-1-seanjc@google.com> Precedence: bulk X-Mailing-List: kvm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20241128004344.4072099-1-seanjc@google.com> X-Mailer: git-send-email 2.47.0.338.g60cca15819-goog Message-ID: <20241128004344.4072099-6-seanjc@google.com> Subject: [PATCH v4 5/6] KVM: x86: Always complete hypercall via function callback From: Sean Christopherson To: Sean Christopherson , Paolo Bonzini Cc: kvm@vger.kernel.org, linux-kernel@vger.kernel.org, Tom Lendacky , Binbin Wu , Isaku Yamahata , Kai Huang , Xiaoyao Li Finish "emulation" of KVM hypercalls by function callback, even when the hypercall is handled entirely within KVM, i.e. doesn't require an exit to userspace, and refactor __kvm_emulate_hypercall()'s return value to *only* communicate whether or not KVM should exit to userspace or resume the guest. (Ab)Use vcpu->run->hypercall.ret to propagate the return value to the callback, purely to avoid having to add a trampoline for every completion callback. Using the function return value for KVM's control flow eliminates the multiplexed return value, where '0' for KVM_HC_MAP_GPA_RANGE (and only that hypercall) means "exit to userspace". Note, the unnecessary extra indirect call and thus potential retpoline will be eliminated in the near future by converting the intermediate layer to a macro. Suggested-by: Binbin Wu Suggested-by: Kai Huang Signed-off-by: Sean Christopherson --- arch/x86/kvm/x86.c | 29 ++++++++++++----------------- arch/x86/kvm/x86.h | 10 ++++++---- 2 files changed, 18 insertions(+), 21 deletions(-) diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c index 11434752b467..39be2a891ab4 100644 --- a/arch/x86/kvm/x86.c +++ b/arch/x86/kvm/x86.c @@ -9982,10 +9982,11 @@ static int complete_hypercall_exit(struct kvm_vcpu *vcpu) return kvm_skip_emulated_instruction(vcpu); } -unsigned long __kvm_emulate_hypercall(struct kvm_vcpu *vcpu, unsigned long nr, - unsigned long a0, unsigned long a1, - unsigned long a2, unsigned long a3, - int op_64_bit, int cpl) +int __kvm_emulate_hypercall(struct kvm_vcpu *vcpu, unsigned long nr, + unsigned long a0, unsigned long a1, + unsigned long a2, unsigned long a3, + int op_64_bit, int cpl, + int (*complete_hypercall)(struct kvm_vcpu *)) { unsigned long ret; @@ -10061,7 +10062,7 @@ unsigned long __kvm_emulate_hypercall(struct kvm_vcpu *vcpu, unsigned long nr, vcpu->run->hypercall.flags |= KVM_EXIT_HYPERCALL_LONG_MODE; WARN_ON_ONCE(vcpu->run->hypercall.flags & KVM_EXIT_HYPERCALL_MBZ); - vcpu->arch.complete_userspace_io = complete_hypercall_exit; + vcpu->arch.complete_userspace_io = complete_hypercall; /* stat is incremented on completion. */ return 0; } @@ -10071,13 +10072,15 @@ unsigned long __kvm_emulate_hypercall(struct kvm_vcpu *vcpu, unsigned long nr, } out: - return ret; + vcpu->run->hypercall.ret = ret; + complete_hypercall(vcpu); + return 1; } EXPORT_SYMBOL_GPL(__kvm_emulate_hypercall); int kvm_emulate_hypercall(struct kvm_vcpu *vcpu) { - unsigned long nr, a0, a1, a2, a3, ret; + unsigned long nr, a0, a1, a2, a3; int op_64_bit; int cpl; @@ -10095,16 +10098,8 @@ int kvm_emulate_hypercall(struct kvm_vcpu *vcpu) op_64_bit = is_64_bit_hypercall(vcpu); cpl = kvm_x86_call(get_cpl)(vcpu); - ret = __kvm_emulate_hypercall(vcpu, nr, a0, a1, a2, a3, op_64_bit, cpl); - if (nr == KVM_HC_MAP_GPA_RANGE && !ret) - /* MAP_GPA tosses the request to the user space. */ - return 0; - - if (!op_64_bit) - ret = (u32)ret; - kvm_rax_write(vcpu, ret); - - return kvm_skip_emulated_instruction(vcpu); + return __kvm_emulate_hypercall(vcpu, nr, a0, a1, a2, a3, op_64_bit, cpl, + complete_hypercall_exit); } EXPORT_SYMBOL_GPL(kvm_emulate_hypercall); diff --git a/arch/x86/kvm/x86.h b/arch/x86/kvm/x86.h index 6db13b696468..28adc8ea04bf 100644 --- a/arch/x86/kvm/x86.h +++ b/arch/x86/kvm/x86.h @@ -617,10 +617,12 @@ static inline bool user_exit_on_hypercall(struct kvm *kvm, unsigned long hc_nr) return kvm->arch.hypercall_exit_enabled & BIT(hc_nr); } -unsigned long __kvm_emulate_hypercall(struct kvm_vcpu *vcpu, unsigned long nr, - unsigned long a0, unsigned long a1, - unsigned long a2, unsigned long a3, - int op_64_bit, int cpl); +int __kvm_emulate_hypercall(struct kvm_vcpu *vcpu, unsigned long nr, + unsigned long a0, unsigned long a1, + unsigned long a2, unsigned long a3, + int op_64_bit, int cpl, + int (*complete_hypercall)(struct kvm_vcpu *)); + int kvm_emulate_hypercall(struct kvm_vcpu *vcpu); #endif From patchwork Thu Nov 28 00:43:44 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sean Christopherson X-Patchwork-Id: 13887467 Received: from mail-pg1-f201.google.com (mail-pg1-f201.google.com [209.85.215.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 3105882D91 for ; Thu, 28 Nov 2024 00:43:58 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.215.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1732754639; cv=none; b=PY/7f4qI6fF2yzWwWutqFwra19u4g6pGsVHVnmPzbIMYi4tA55/V5Li3LsYSbKs/kTZsX4OVSR2ush9nnopdHAoaa+pujG8vjNrGMwpkxbRcOSy9XdfxTcRyJp7lBq/CXoqwdXT9YPuedp3G4W4PnX21j43RCmQUxEyI75N4oco= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1732754639; c=relaxed/simple; bh=iMEWMeSgtpX2vDiZAZHNPxwDcscsHshHwaR+WGYfMz8=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=ALFV+umhKMhrwwQjJDSovaNH16x3t568O/91h2Fib3zohnJl8TAj/q1SEV0AtjjRb7jRowHgbIbV01RICuDsAG37QlOj5FevjdnSq7Sk/SwsmGqUaXpnLk1aEaZohcsMGuRT4cb1oTriaQ6Dqb4DWkVg8UTH6xoibfsoRzpkI2o= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=Dm4qnYCo; arc=none smtp.client-ip=209.85.215.201 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="Dm4qnYCo" Received: by mail-pg1-f201.google.com with SMTP id 41be03b00d2f7-7ea8baba60dso309223a12.3 for ; Wed, 27 Nov 2024 16:43:58 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1732754637; x=1733359437; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:from:to:cc:subject:date:message-id:reply-to; bh=Kb2dr0T5bp6rNnjcREbp+SqCdYs3HsUs4UOFpqH2Uio=; b=Dm4qnYCoQ441Mvab8cdSfmnRZWVfohDjPcUmSdbu3CGH5T+rzr0H/TNYK8pfLsIyPn 9qyKQgNv1A28bdHGrAFS5t2DXlW6fkEKWu4fHfjf2Zr7rKQwFidhdpic+AB1cXVRiyS9 P+8G754SxNVhIkU0YICN9NuNpmTecZIaoNEzbQj1WoZ5tnJRzd74K7xu5wySExxuxDH3 kKXVYWRotgqYaO4rGcU7/wN328Reo7sMuIqT99csvMEH2+Ir7GmVaD1cj4M/Ws5ShoV3 XLJQ4WFY2Ua+9oB1ksyEJ8tUyQ6vBPzkL9hXcX3UkNIKuHep3D3CjxkmVz7IhMCEQnK2 6/5w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1732754637; x=1733359437; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=Kb2dr0T5bp6rNnjcREbp+SqCdYs3HsUs4UOFpqH2Uio=; b=uYC0Qb6SQlm5fQe4FkSSL/cNPP/sLpR9rfqIrHK2Wb9E08e5O31btHWdqLU60JvLtF yQVq0yW1fHvaj9bPCmceLsDloQhkHmrGJw9gM7wmihXx1QiY4GYKzgPaih0NRQqEgd1Y TNlsiDzLrnvz345ysTVqskDvedrlU0Wd2EcaEwJNkFlelhhKfyHHN268VfuWldo9dnjN Mt3Kv4KjUKU0Vyp2w12KHpRlAuXZabotc4GxbYCLgyuRBzCeZxTHXXBJrVsG7rp0uFQq jk2j6heDwXYpgSntzTQWrs2wIiHlb0wzhMRwywUitYY20RuOa1K+PIw4fxuKq4vYFK3a gFJQ== X-Gm-Message-State: AOJu0Yy9+kxW5B8MGg0/DmU6XU8JAcuiT8W0VzUw2Jd0sVqBD6sdPH4O S2HAJveb8hxRg1zVlgpCyOeD3RpGhh8M5Kh2GiatOUWPVtbVp8KKAD/KQ0X6qOUFBFER2q/iZ8F tMw== X-Google-Smtp-Source: AGHT+IFAR5YUseLMEmNyaxfAMVHlZLF3OQCHcVX3UaR+ESdLafctVjBieHniv07m1mzMJyd37ZzdGkN19OE= X-Received: from pjd6.prod.google.com ([2002:a17:90b:54c6:b0:2ea:7174:2101]) (user=seanjc job=prod-delivery.src-stubby-dispatcher) by 2002:a05:6a20:734a:b0:1d9:4837:ada2 with SMTP id adf61e73a8af0-1e0e0b7e3a4mr7225792637.35.1732754637627; Wed, 27 Nov 2024 16:43:57 -0800 (PST) Reply-To: Sean Christopherson Date: Wed, 27 Nov 2024 16:43:44 -0800 In-Reply-To: <20241128004344.4072099-1-seanjc@google.com> Precedence: bulk X-Mailing-List: kvm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20241128004344.4072099-1-seanjc@google.com> X-Mailer: git-send-email 2.47.0.338.g60cca15819-goog Message-ID: <20241128004344.4072099-7-seanjc@google.com> Subject: [PATCH v4 6/6] KVM: x86: Refactor __kvm_emulate_hypercall() into a macro From: Sean Christopherson To: Sean Christopherson , Paolo Bonzini Cc: kvm@vger.kernel.org, linux-kernel@vger.kernel.org, Tom Lendacky , Binbin Wu , Isaku Yamahata , Kai Huang , Xiaoyao Li Rework __kvm_emulate_hypercall() into a macro so that completion of hypercalls that don't exit to userspace use direct function calls to the completion helper, i.e. don't trigger a retpoline when RETPOLINE=y. Opportunistically take the names of the input registers, as opposed to taking the input values, to preemptively dedup more of the calling code (TDX needs to use different registers). Use the direct GPR accessors to read values to avoid the pointless marking of the registers as available (KVM requires GPRs to always be available). Signed-off-by: Sean Christopherson --- arch/x86/kvm/x86.c | 29 +++++++++-------------------- arch/x86/kvm/x86.h | 25 ++++++++++++++++++++----- 2 files changed, 29 insertions(+), 25 deletions(-) diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c index 39be2a891ab4..fef8b4e63d25 100644 --- a/arch/x86/kvm/x86.c +++ b/arch/x86/kvm/x86.c @@ -9982,11 +9982,11 @@ static int complete_hypercall_exit(struct kvm_vcpu *vcpu) return kvm_skip_emulated_instruction(vcpu); } -int __kvm_emulate_hypercall(struct kvm_vcpu *vcpu, unsigned long nr, - unsigned long a0, unsigned long a1, - unsigned long a2, unsigned long a3, - int op_64_bit, int cpl, - int (*complete_hypercall)(struct kvm_vcpu *)) +int ____kvm_emulate_hypercall(struct kvm_vcpu *vcpu, unsigned long nr, + unsigned long a0, unsigned long a1, + unsigned long a2, unsigned long a3, + int op_64_bit, int cpl, + int (*complete_hypercall)(struct kvm_vcpu *)) { unsigned long ret; @@ -10073,32 +10073,21 @@ int __kvm_emulate_hypercall(struct kvm_vcpu *vcpu, unsigned long nr, out: vcpu->run->hypercall.ret = ret; - complete_hypercall(vcpu); return 1; } -EXPORT_SYMBOL_GPL(__kvm_emulate_hypercall); +EXPORT_SYMBOL_GPL(____kvm_emulate_hypercall); int kvm_emulate_hypercall(struct kvm_vcpu *vcpu) { - unsigned long nr, a0, a1, a2, a3; - int op_64_bit; - int cpl; - if (kvm_xen_hypercall_enabled(vcpu->kvm)) return kvm_xen_hypercall(vcpu); if (kvm_hv_hypercall_enabled(vcpu)) return kvm_hv_hypercall(vcpu); - nr = kvm_rax_read(vcpu); - a0 = kvm_rbx_read(vcpu); - a1 = kvm_rcx_read(vcpu); - a2 = kvm_rdx_read(vcpu); - a3 = kvm_rsi_read(vcpu); - op_64_bit = is_64_bit_hypercall(vcpu); - cpl = kvm_x86_call(get_cpl)(vcpu); - - return __kvm_emulate_hypercall(vcpu, nr, a0, a1, a2, a3, op_64_bit, cpl, + return __kvm_emulate_hypercall(vcpu, rax, rbx, rcx, rdx, rsi, + is_64_bit_hypercall(vcpu), + kvm_x86_call(get_cpl)(vcpu), complete_hypercall_exit); } EXPORT_SYMBOL_GPL(kvm_emulate_hypercall); diff --git a/arch/x86/kvm/x86.h b/arch/x86/kvm/x86.h index 28adc8ea04bf..ad6fe6159dea 100644 --- a/arch/x86/kvm/x86.h +++ b/arch/x86/kvm/x86.h @@ -617,11 +617,26 @@ static inline bool user_exit_on_hypercall(struct kvm *kvm, unsigned long hc_nr) return kvm->arch.hypercall_exit_enabled & BIT(hc_nr); } -int __kvm_emulate_hypercall(struct kvm_vcpu *vcpu, unsigned long nr, - unsigned long a0, unsigned long a1, - unsigned long a2, unsigned long a3, - int op_64_bit, int cpl, - int (*complete_hypercall)(struct kvm_vcpu *)); +int ____kvm_emulate_hypercall(struct kvm_vcpu *vcpu, unsigned long nr, + unsigned long a0, unsigned long a1, + unsigned long a2, unsigned long a3, + int op_64_bit, int cpl, + int (*complete_hypercall)(struct kvm_vcpu *)); + +#define __kvm_emulate_hypercall(_vcpu, nr, a0, a1, a2, a3, op_64_bit, cpl, complete_hypercall) \ +({ \ + int __ret; \ + \ + __ret = ____kvm_emulate_hypercall(_vcpu, \ + kvm_##nr##_read(_vcpu), kvm_##a0##_read(_vcpu), \ + kvm_##a1##_read(_vcpu), kvm_##a2##_read(_vcpu), \ + kvm_##a3##_read(_vcpu), op_64_bit, cpl, \ + complete_hypercall); \ + \ + if (__ret > 0) \ + complete_hypercall(_vcpu); \ + __ret; \ +}) int kvm_emulate_hypercall(struct kvm_vcpu *vcpu);