From patchwork Sat Jan 11 01:24:46 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sean Christopherson X-Patchwork-Id: 13935752 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id B479AE7719A for ; Sat, 11 Jan 2025 03:37:53 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender:Reply-To:List-Subscribe: List-Help:List-Post:List-Archive:List-Unsubscribe:List-Id:Content-Type:Cc:To: From:Subject:Message-ID:References:Mime-Version:In-Reply-To:Date: Content-Transfer-Encoding:Content-ID:Content-Description:Resent-Date: Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=L3TyIoxqzKbU5TOqXbLBWWXwSPvFkN5kGJBIZi+nPuA=; b=Ex1Et5ghDQNXBUuiXO0RWd6oUm M2Ial2q0ZqjiotsoY1E+H+nZqM0Yu+0wbF/Q1Za2FGU7nT2V6tK1nY9aOxjKWwtsEs1Vx3UWKZ2v0 I4yuhahfa4ytJa3Q2mBnHbX0VrL5h2uGZqRzDH8hGthbKnbtX4W5h8WPBQbAQiua9Z2ilfXVKmnjy OeuHJ5kK9qq24ySOPbea4xE74wE62CUmCGL07kM24zkeuLu+BekmQ/vKOF8iuRlGvQ6vBg3DsvJ71 wipUjz+04VgdJb0/1tTqrL8LGJZ9ryf630EGZ87SnMPGIHNey60U1KsnwRwTrTxVchyf8z73VZO+u 6OeXNbqw==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.98 #2 (Red Hat Linux)) id 1tWSJj-000000004Tq-38NL; Sat, 11 Jan 2025 03:37:43 +0000 Received: from mail-pj1-x104a.google.com ([2607:f8b0:4864:20::104a]) by bombadil.infradead.org with esmtps (Exim 4.98 #2 (Red Hat Linux)) id 1tWQFD-0000000HUQz-36un for linux-arm-kernel@lists.infradead.org; Sat, 11 Jan 2025 01:24:56 +0000 Received: by mail-pj1-x104a.google.com with SMTP id 98e67ed59e1d1-2ef6ef9ba3fso4790115a91.2 for ; Fri, 10 Jan 2025 17:24:54 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1736558694; x=1737163494; darn=lists.infradead.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:from:to:cc:subject:date:message-id:reply-to; bh=L3TyIoxqzKbU5TOqXbLBWWXwSPvFkN5kGJBIZi+nPuA=; b=MZama5LY0TAW+qKnvCzVZfDiLty1i3X1R2et7hn9QtmfESyqNSh8j52ECO84N+vWLK WOQx1uwgQxcLYjSXBz+OyFDIQBOTeuhoCVrYolvHhdB0Vg1sNz415OJ78hXbErIhjZLe rRR//ZJBb2tvq2mEeVEcaW63bFgbBadf6lsRE2lUwbu6BJXZPGbEj157OGlS0afWTDp5 ShYTAbPm9xlIZ8K/ZsNo44eNarkfh9nmw+e0n8GB/kg7HcO1mtfczSYS6C6IjPdzsBLm mZrHuWxVUKi6eJvzdUWj+xGu1cuUoBu/fIcsZS7PhS4ZC3+H7tjJpZeVIiKYeJc5PLrp 2ygQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1736558694; x=1737163494; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=L3TyIoxqzKbU5TOqXbLBWWXwSPvFkN5kGJBIZi+nPuA=; b=If7Pe2zbNoEPe709HZiFFyfMYths3zQB9SntxbErrzuaNaxtlEY7kvYw4BFm8xIdoz mYooHRjgRo1qgLVenIqTl/nP6pN2JRWcUYhYo2uTWs3K+GbIMyO7lQzgi7r7pwbq7LTT VyJBnblWgl58/uSu8xU1GDl3J4xqoUpclJNuJSlhIFUE6UjDfWnb4HlYIFO0uu/0hhm0 ydsRZI9v6PB38xNpTulSty7OV+YtB/yoOzrXKbhsTAsps3gk7SLa1Uvf9FBeoEimmtmK hpvZDydKmn9KjJxV+DnTV47OoV0VpwvyLJAV6M3f0yCaUXNgcRXXsnyyduLeO32TBahx Is9Q== X-Forwarded-Encrypted: i=1; AJvYcCW6TtI1BkC13fbM/WA0BPt9hOvcd2p14Bqxj4oQSbD1zmUa/TypT60O2MgIUR9a48ASW3ZWGoleExZCa5SqA5Iw@lists.infradead.org X-Gm-Message-State: AOJu0YzuuAilBkeq84JBuCtrJ4PIQE5oXpyh2yfy+5iL4IqAVSJsp3hG sHEeHhxEy/5CgCwF+L/Pqk2N14ARxeJjLmHvr/yxqpOQOq345Z7q8lUYuCgOPhRCJkaSGYv0gMM 8sw== X-Google-Smtp-Source: AGHT+IHQFtnrz9Qz4G4DVM+w+HCWHLq9SWFcOY5MsmF5EmSUkf18xpdPguK9lOowTA48c954eXDA8ByTfJE= X-Received: from pfld20.prod.google.com ([2002:a05:6a00:1994:b0:728:b3dd:ba8c]) (user=seanjc job=prod-delivery.src-stubby-dispatcher) by 2002:a05:6a00:114e:b0:728:f337:a721 with SMTP id d2e1a72fcca58-72d21f29214mr17702842b3a.7.1736558694264; Fri, 10 Jan 2025 17:24:54 -0800 (PST) Date: Fri, 10 Jan 2025 17:24:46 -0800 In-Reply-To: <20250111012450.1262638-1-seanjc@google.com> Mime-Version: 1.0 References: <20250111012450.1262638-1-seanjc@google.com> X-Mailer: git-send-email 2.47.1.613.gc27f4b7a9f-goog Message-ID: <20250111012450.1262638-2-seanjc@google.com> Subject: [PATCH 1/5] KVM: x86: Document that KVM_EXIT_HYPERCALL requires completion From: Sean Christopherson To: Paolo Bonzini , Marc Zyngier , Oliver Upton , Michael Ellerman , Sean Christopherson Cc: kvm@vger.kernel.org, linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev, linuxppc-dev@lists.ozlabs.org, linux-kernel@vger.kernel.org X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20250110_172455_782593_400F46CC X-CRM114-Status: GOOD ( 13.31 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: Sean Christopherson Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Update KVM's documentation to call out that KVM_EXIT_HYPERCALL requires userspace to do KVM_RUN before state save/restore, so that KVM can skip the hypercall instruction, otherwise resuming the vCPU after restore will restart the instruction and potentially lead to a spurious MAP_GPA_RANGE. Fixes: 0dbb11230437 ("KVM: X86: Introduce KVM_HC_MAP_GPA_RANGE hypercall") Cc: stable@vger.kernel.org Signed-off-by: Sean Christopherson --- Documentation/virt/kvm/api.rst | 39 +++++++++++++++++----------------- 1 file changed, 20 insertions(+), 19 deletions(-) diff --git a/Documentation/virt/kvm/api.rst b/Documentation/virt/kvm/api.rst index 454c2aaa155e..c92c8d4e8779 100644 --- a/Documentation/virt/kvm/api.rst +++ b/Documentation/virt/kvm/api.rst @@ -6615,13 +6615,29 @@ The 'data' member contains, in its first 'len' bytes, the value as it would appear if the VCPU performed a load or store of the appropriate width directly to the byte array. +It is strongly recommended that userspace use ``KVM_EXIT_IO`` (x86) or +``KVM_EXIT_MMIO`` (all except s390) to implement functionality that +requires a guest to interact with host userspace. + +.. note:: KVM_EXIT_IO is significantly faster than KVM_EXIT_MMIO. + + /* KVM_EXIT_HYPERCALL */ + struct { + __u64 nr; + __u64 args[6]; + __u64 ret; + __u64 flags; + } hypercall; + + .. note:: For KVM_EXIT_IO, KVM_EXIT_MMIO, KVM_EXIT_OSI, KVM_EXIT_PAPR, KVM_EXIT_XEN, - KVM_EXIT_EPR, KVM_EXIT_X86_RDMSR and KVM_EXIT_X86_WRMSR the corresponding - operations are complete (and guest state is consistent) only after userspace - has re-entered the kernel with KVM_RUN. The kernel side will first finish - incomplete operations and then check for pending signals. + KVM_EXIT_EPR, KVM_EXIT_X86_RDMSR, KVM_EXIT_X86_WRMSR, and KVM_EXIT_HYPERCALL + the corresponding operations are complete (and guest state is consistent) + only after userspace has re-entered the kernel with KVM_RUN. The kernel + side will first finish incomplete operations and then check for pending + signals. The pending state of the operation is not preserved in state which is visible to userspace, thus userspace should ensure that the operation is @@ -6632,21 +6648,6 @@ to the byte array. :: - /* KVM_EXIT_HYPERCALL */ - struct { - __u64 nr; - __u64 args[6]; - __u64 ret; - __u64 flags; - } hypercall; - - -It is strongly recommended that userspace use ``KVM_EXIT_IO`` (x86) or -``KVM_EXIT_MMIO`` (all except s390) to implement functionality that -requires a guest to interact with host userspace. - -.. note:: KVM_EXIT_IO is significantly faster than KVM_EXIT_MMIO. - For arm64: ---------- From patchwork Sat Jan 11 01:24:47 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sean Christopherson X-Patchwork-Id: 13935716 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 6B8B0E77188 for ; Sat, 11 Jan 2025 02:37:55 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender:Reply-To:List-Subscribe: List-Help:List-Post:List-Archive:List-Unsubscribe:List-Id:Content-Type:Cc:To: From:Subject:Message-ID:References:Mime-Version:In-Reply-To:Date: Content-Transfer-Encoding:Content-ID:Content-Description:Resent-Date: Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=yPMtRfEi+Y38oJbaqxUUwsgL1BiiO9TX2S4BZFXkjhs=; b=meBBFzk2mOZ1IP34NGXxqDNAfH bkZYfnIaRqimmc1Ra5a2gbBzD7cFENkL4g5r5/xlFUGRBtqFzFN04WtCBvtvR9WINc21R2J573H9x CUQAdto4rsGK8AEgxTvyiqX3wEOFjJQyuI+LFkPcao5Z1GMv+zhZZkZ3TXgWqqfofSYcGLtJK+ZEN RIrOo49c4xBjCCKYmzDjkk5B2JBRtpg6AC6oXeoFYCVMeYqRWOjM8R0EmBeNSCJqgi2Mb3SDK4s2/ eUhnj1dh5jrg4pY9wkIubFYd7WpTIZrslSuHyS5dJFxayhaPJBIJ4YeTVm9yJBZabqjtgd+8fj/tC UvwbIgtQ==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.98 #2 (Red Hat Linux)) id 1tWRNe-0000000Hadt-2juB; Sat, 11 Jan 2025 02:37:42 +0000 Received: from desiato.infradead.org ([2001:8b0:10b:1:d65d:64ff:fe57:4e05]) by bombadil.infradead.org with esmtps (Exim 4.98 #2 (Red Hat Linux)) id 1tWQFL-0000000HUSg-1BoN for linux-arm-kernel@bombadil.infradead.org; Sat, 11 Jan 2025 01:25:03 +0000 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=desiato.20200630; h=Content-Type:Cc:To:From:Subject: Message-ID:References:Mime-Version:In-Reply-To:Date:Reply-To:Sender: Content-Transfer-Encoding:Content-ID:Content-Description; bh=yPMtRfEi+Y38oJbaqxUUwsgL1BiiO9TX2S4BZFXkjhs=; b=RvRDNRsCxSKs1ZrmYmnyETAMmZ cYhkm+T+vAUbrjQ0us/T3Gs89AZ1iZFPVa9nhD5LRUzp3YqFIsgk8GQlMKm3L20S2rKasv65e5uLa FaVekl/vq0GrbBKQrAegJeV7+npUvoZUn+9eCPFgKpxrfHo01oHT4qIT0iWAFACW0yvix4cm4K1yy vGOcSrHlW4k73cLbokYA53/nJ4u0SvazeZuY++sSi2mB5f0zFwiSFg7EYrLGCs7xvObF49THIwxCj 1lHUTP4JyhYIOeOhxMhXFVvb+zQ1+nhJEmZ55m0Leo4DawGsdELb0e90W7coEy4HFC6LakY58WJQd SEbJCvqw==; Received: from mail-pl1-x649.google.com ([2607:f8b0:4864:20::649]) by desiato.infradead.org with esmtps (Exim 4.98 #2 (Red Hat Linux)) id 1tWQFH-00000009zI7-2HhN for linux-arm-kernel@lists.infradead.org; Sat, 11 Jan 2025 01:25:01 +0000 Received: by mail-pl1-x649.google.com with SMTP id d9443c01a7336-2164fad3792so44515695ad.0 for ; Fri, 10 Jan 2025 17:24:57 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1736558696; x=1737163496; darn=lists.infradead.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:from:to:cc:subject:date:message-id:reply-to; bh=yPMtRfEi+Y38oJbaqxUUwsgL1BiiO9TX2S4BZFXkjhs=; b=Gu6hSbv7R/6iidsVDHhM26j0r1U5RTGam98h9GxA+mskGNOUF96nOrykSWBavGOZYQ D0+wUS5/ZIKO1hgOSHVqGdGVDLIMawp3RihysIzx2Ofu+T3xdjdR9C+ffo8vYrA7IF59 GPh792lQiBYlFsD//sCYxVn/U7FF7wbO43IWOIbWKbTklmk42iRw7EAricUr+r9g0Bdn CjFcXIj+xoXmEzG2C9Aks8+NNp+ggakedpuQ5mP9SnaNR+Om7gsiBYaMGsIKMOEuOvn6 unE4r1Dm3OMeOl1FMA2s/vOBnQ1Lnl7S0+CYNBgPWllJX59iqcEdGp9vbOQcSck2J0bX 67Hw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1736558696; x=1737163496; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=yPMtRfEi+Y38oJbaqxUUwsgL1BiiO9TX2S4BZFXkjhs=; b=OkUCOdhNgWC9KGVDbbip8xVfF3xDRbtBzTQe/sijxaBx/gWMmXPxqGinwyI1hyhpN1 tknVXJM9RyO4STgg6c7twO7+vsAlJ5cTsDIM+xgMng+L8OkK6cSK6UcEUfxrvwr3VM1/ 3tHRufUJAr/nA3+WGIHcAUNYxX/bBOh4AR+6VHQbKL5XtcwwTthlolk9BSZsuxNSBlNN yHOVSEZn/gGmGNcB4vKdYdoOwEywbg+fhY9TgzTu0w9KK0YuwAdfIGmV9g4Wcqtqy5hC OBMhGVeiIGgHovQJGOkEhluOYqZkd/IXFvH0SuTuWhrgc709UXRDDcWJizdnQGtKpGQS T4rg== X-Forwarded-Encrypted: i=1; AJvYcCUHI+npLRsykoBD5pAVcKHAuzW2uMnAcTH3257LmQohEzF2dUKw+s0PQs/9Qdjb/hEeFyJV2wCbWAhvxgOSdVoc@lists.infradead.org X-Gm-Message-State: AOJu0YxD/f0eV48VBIV/fI/bDN0J5EhI9gvpjiMcVShFzGjfkIvhhjvY THJxMvMwqhaI3mXQSqPVNrVoqIsSZbDZxTjKr2nPi/ddB6gZnEJgqzeJVuwKOrEtNQGsqXy7b9k Iug== X-Google-Smtp-Source: AGHT+IFD3f0YjwZz/pTHPUxUy4yfVqFt92VRSC3L2LvrexSYRapcV9Xo2L4A2CKFVBcFdjD8/mn4jkTqU/U= X-Received: from pgkp11.prod.google.com ([2002:a63:f44b:0:b0:7fc:fac3:7df6]) (user=seanjc job=prod-delivery.src-stubby-dispatcher) by 2002:a05:6a20:4325:b0:1e1:b062:f3fa with SMTP id adf61e73a8af0-1e88d1dba97mr25121737637.34.1736558695870; Fri, 10 Jan 2025 17:24:55 -0800 (PST) Date: Fri, 10 Jan 2025 17:24:47 -0800 In-Reply-To: <20250111012450.1262638-1-seanjc@google.com> Mime-Version: 1.0 References: <20250111012450.1262638-1-seanjc@google.com> X-Mailer: git-send-email 2.47.1.613.gc27f4b7a9f-goog Message-ID: <20250111012450.1262638-3-seanjc@google.com> Subject: [PATCH 2/5] KVM: Clear vcpu->run->flags at start of KVM_RUN for all architectures From: Sean Christopherson To: Paolo Bonzini , Marc Zyngier , Oliver Upton , Michael Ellerman , Sean Christopherson Cc: kvm@vger.kernel.org, linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev, linuxppc-dev@lists.ozlabs.org, linux-kernel@vger.kernel.org X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20250111_012500_305857_EAA0DCFC X-CRM114-Status: GOOD ( 14.08 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: Sean Christopherson Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Clear kvm_run.flags at the start of KVM_RUN for all architectures to minimize the probability of leaving a stale flag set. Signed-off-by: Sean Christopherson --- arch/arm64/kvm/arm.c | 1 - arch/arm64/kvm/handle_exit.c | 2 +- arch/powerpc/kvm/book3s_hv.c | 4 +--- arch/x86/kvm/x86.c | 1 - virt/kvm/kvm_main.c | 3 +++ 5 files changed, 5 insertions(+), 6 deletions(-) diff --git a/arch/arm64/kvm/arm.c b/arch/arm64/kvm/arm.c index a102c3aebdbc..925fa010bb7b 100644 --- a/arch/arm64/kvm/arm.c +++ b/arch/arm64/kvm/arm.c @@ -1128,7 +1128,6 @@ int kvm_arch_vcpu_ioctl_run(struct kvm_vcpu *vcpu) ret = 1; run->exit_reason = KVM_EXIT_UNKNOWN; - run->flags = 0; while (ret > 0) { /* * Check conditions before entering the guest diff --git a/arch/arm64/kvm/handle_exit.c b/arch/arm64/kvm/handle_exit.c index d7c2990e7c9e..63692c254a07 100644 --- a/arch/arm64/kvm/handle_exit.c +++ b/arch/arm64/kvm/handle_exit.c @@ -186,7 +186,7 @@ static int kvm_handle_guest_debug(struct kvm_vcpu *vcpu) run->exit_reason = KVM_EXIT_DEBUG; run->debug.arch.hsr = lower_32_bits(esr); run->debug.arch.hsr_high = upper_32_bits(esr); - run->flags = KVM_DEBUG_ARCH_HSR_HIGH_VALID; + run->flags |= KVM_DEBUG_ARCH_HSR_HIGH_VALID; switch (ESR_ELx_EC(esr)) { case ESR_ELx_EC_WATCHPT_LOW: diff --git a/arch/powerpc/kvm/book3s_hv.c b/arch/powerpc/kvm/book3s_hv.c index 25429905ae90..b253f7372774 100644 --- a/arch/powerpc/kvm/book3s_hv.c +++ b/arch/powerpc/kvm/book3s_hv.c @@ -1704,9 +1704,7 @@ static int kvmppc_handle_exit_hv(struct kvm_vcpu *vcpu, /* Exit to guest with KVM_EXIT_NMI as exit reason */ run->exit_reason = KVM_EXIT_NMI; run->hw.hardware_exit_reason = vcpu->arch.trap; - /* Clear out the old NMI status from run->flags */ - run->flags &= ~KVM_RUN_PPC_NMI_DISP_MASK; - /* Now set the NMI status */ + /* Note, run->flags is cleared at the start of KVM_RUN. */ if (vcpu->arch.mce_evt.disposition == MCE_DISPOSITION_RECOVERED) run->flags |= KVM_RUN_PPC_NMI_DISP_FULLY_RECOV; else diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c index 1b04092ec76a..a8aa12e0911d 100644 --- a/arch/x86/kvm/x86.c +++ b/arch/x86/kvm/x86.c @@ -11465,7 +11465,6 @@ int kvm_arch_vcpu_ioctl_run(struct kvm_vcpu *vcpu) vcpu_load(vcpu); kvm_sigset_activate(vcpu); - kvm_run->flags = 0; kvm_load_guest_fpu(vcpu); kvm_vcpu_srcu_read_lock(vcpu); diff --git a/virt/kvm/kvm_main.c b/virt/kvm/kvm_main.c index de2c11dae231..7d2076439081 100644 --- a/virt/kvm/kvm_main.c +++ b/virt/kvm/kvm_main.c @@ -4336,6 +4336,9 @@ static long kvm_vcpu_ioctl(struct file *filp, put_pid(oldpid); } + + vcpu->run->flags = 0; + vcpu->wants_to_run = !READ_ONCE(vcpu->run->immediate_exit__unsafe); r = kvm_arch_vcpu_ioctl_run(vcpu); vcpu->wants_to_run = false; From patchwork Sat Jan 11 01:24:48 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sean Christopherson X-Patchwork-Id: 13935697 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 19753E77188 for ; Sat, 11 Jan 2025 01:28:52 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender:Reply-To:List-Subscribe: List-Help:List-Post:List-Archive:List-Unsubscribe:List-Id:Content-Type:Cc:To: From:Subject:Message-ID:References:Mime-Version:In-Reply-To:Date: Content-Transfer-Encoding:Content-ID:Content-Description:Resent-Date: Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=eb/3+mKu6t8Zhrqra/tibkxSDkT7W4OQb1wzc5Zn/9I=; b=4yrp7c0d50wgq2lpTHelcCNtg1 AWr9pB932xDJHbi8jgGhwbyHH+iQ0rHtW16lhOySMoqr1FHVUvziBP+FXhvqIJSoqzo8ir5wQUVcd Bu+eIfIdsfEyc7MFlTotmwNs/ceuflYSB49+wIhKEjp2Q8TyJamcewUBZpb20saVh9GLVRbyTlFPq iH4JUhS2DDYKvaVR20NqLt3LsXn7uwV5W5GvgpBDSlhMvkBevTgQ3ZTQ1Gn/knY1pd2Ghf6ChdJ7m EVhsWKF+D2rUUkJbFAbbR2+CnYjEtmf+uVdj3mCrewhiNY5takUYDQrixMOHnlJ5klv1UifVUNWpQ cQVc7HyA==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.98 #2 (Red Hat Linux)) id 1tWQIo-0000000HV9o-41hb; Sat, 11 Jan 2025 01:28:38 +0000 Received: from mail-pj1-x104a.google.com ([2607:f8b0:4864:20::104a]) by bombadil.infradead.org with esmtps (Exim 4.98 #2 (Red Hat Linux)) id 1tWQFG-0000000HURb-2p89 for linux-arm-kernel@lists.infradead.org; Sat, 11 Jan 2025 01:24:59 +0000 Received: by mail-pj1-x104a.google.com with SMTP id 98e67ed59e1d1-2efa74481fdso4801323a91.1 for ; Fri, 10 Jan 2025 17:24:57 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1736558697; x=1737163497; darn=lists.infradead.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:from:to:cc:subject:date:message-id:reply-to; bh=eb/3+mKu6t8Zhrqra/tibkxSDkT7W4OQb1wzc5Zn/9I=; b=DnVmTwiTzvSkWFQaS24Gjq0zqMKUffaueJVs8LDgWzScDrcFXb1rHWypcWwjHqEGL2 z0jzC/xbeRUNvp/8Klhdoh2GOcQyQNXmYpXPrnTY51/Il979t2M5KNq3JL66LHj2o8W1 x6832L/WL843O6a0PpHydBxVwxj3/UB+ax7to4WNVRJfdG7tpuY67l4+Rwbzm4nz7GsC gwqg644vj4d1+su9XPYL/WwctsmIeyTYsUVJc3yqhz8OBy+YFa7dZ/5PpZGRtRoTmhkw xpdP1g2fwV4ptgwCoi822uQ1wP7OxpPDslhljuO95App+2Ihk4hQkHeaIL7V8IE3MYXZ 03jw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1736558697; x=1737163497; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=eb/3+mKu6t8Zhrqra/tibkxSDkT7W4OQb1wzc5Zn/9I=; b=wyafgBjDWmRTK84vyrhExOnHN8gG0IqZHucqBYBH/MhNgESqmzxpLZGpa4aTb78CmY PNDUAVFGQyYnVp3puHmvhXAP6V0nMiGSEiAAdKagjt1a4n0qGBZKtIn36p3m+t4v1cyt 2pCfrRk6Ii28TBLZ01JR7EA6NIIGJyp1jkV7qPWY5NHdopQcHFTZF/EqQ7A4TGNdmAgX wtVgL/MYjFEvm+ehiFN5a45RtcLDuymBV8Ma/3mtP37NgadD7Op+pXLHT7CmPuvsHCAO 7wTqsZKTmQSxgTBSwnmHFfpPP4G7EJzMWFnChlmvmQlZf0e/8n8szFdMIE6c85Txqg6v jwSw== X-Forwarded-Encrypted: i=1; AJvYcCXU55Opai8DGEjBI1fV5U03bqv4qdfjx/Qt3qCJmz3cOlN30H2VFt1XwHdDH8Uvq99V8qI23kCxH/IwMNIcuIqQ@lists.infradead.org X-Gm-Message-State: AOJu0YwYT4/TGqfIgpiUWVmJEutVhVMk+pYGdTN47rZ+36fo8dxBQjnI jF+6s7CO2HngtkPQ5onzCapJv9Z8htGpwOb80tIN3GXPltF+ymKtUSRYrH7iWAm4PaldWMbvdHK aMQ== X-Google-Smtp-Source: AGHT+IEahfph4iX899pUZU2sWlx08q+szoXMpcKFVnSL/6Z/oUV0Zh5RCrkeMGXrUzIUgcOK6vulcmkt/Qg= X-Received: from pjvb13.prod.google.com ([2002:a17:90a:d88d:b0:2ee:4679:4a6b]) (user=seanjc job=prod-delivery.src-stubby-dispatcher) by 2002:a17:90b:2f03:b0:2ee:ed1c:e451 with SMTP id 98e67ed59e1d1-2f548eb32ffmr20060830a91.15.1736558697483; Fri, 10 Jan 2025 17:24:57 -0800 (PST) Date: Fri, 10 Jan 2025 17:24:48 -0800 In-Reply-To: <20250111012450.1262638-1-seanjc@google.com> Mime-Version: 1.0 References: <20250111012450.1262638-1-seanjc@google.com> X-Mailer: git-send-email 2.47.1.613.gc27f4b7a9f-goog Message-ID: <20250111012450.1262638-4-seanjc@google.com> Subject: [PATCH 3/5] KVM: Add a common kvm_run flag to communicate an exit needs completion From: Sean Christopherson To: Paolo Bonzini , Marc Zyngier , Oliver Upton , Michael Ellerman , Sean Christopherson Cc: kvm@vger.kernel.org, linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev, linuxppc-dev@lists.ozlabs.org, linux-kernel@vger.kernel.org X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20250110_172458_717279_9BA8CBEF X-CRM114-Status: GOOD ( 23.92 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: Sean Christopherson Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Add a kvm_run flag, KVM_RUN_NEEDS_COMPLETION, to communicate to userspace that KVM_RUN needs to be re-executed prior to save/restore in order to complete the instruction/operation that triggered the userspace exit. KVM's current approach of adding notes in the Documentation is beyond brittle, e.g. there is at least one known case where a KVM developer added a new userspace exit type, and then that same developer forgot to handle completion when adding userspace support. On x86, there are multiple exits that need completion, but they are all conveniently funneled through a single callback, i.e. in theory, this is a one-time thing for KVM x86 (and other architectures could follow suit with additional refactoring). Signed-off-by: Sean Christopherson --- Documentation/virt/kvm/api.rst | 48 ++++++++++++++++++++++--------- arch/powerpc/kvm/book3s_emulate.c | 1 + arch/powerpc/kvm/book3s_hv.c | 1 + arch/powerpc/kvm/book3s_pr.c | 2 ++ arch/powerpc/kvm/booke.c | 1 + arch/x86/include/uapi/asm/kvm.h | 7 +++-- arch/x86/kvm/x86.c | 2 ++ include/uapi/linux/kvm.h | 3 ++ virt/kvm/kvm_main.c | 1 + 9 files changed, 49 insertions(+), 17 deletions(-) diff --git a/Documentation/virt/kvm/api.rst b/Documentation/virt/kvm/api.rst index c92c8d4e8779..8e172675d8d6 100644 --- a/Documentation/virt/kvm/api.rst +++ b/Documentation/virt/kvm/api.rst @@ -6505,7 +6505,7 @@ local APIC is not used. __u16 flags; -More architecture-specific flags detailing state of the VCPU that may +Common and architecture-specific flags detailing state of the VCPU that may affect the device's behavior. Current defined flags:: /* x86, set if the VCPU is in system management mode */ @@ -6518,6 +6518,8 @@ affect the device's behavior. Current defined flags:: /* arm64, set for KVM_EXIT_DEBUG */ #define KVM_DEBUG_ARCH_HSR_HIGH_VALID (1 << 0) + /* all architectures, set when the exit needs completion (via KVM_RUN) */ + #define KVM_RUN_NEEDS_COMPLETION (1 << 15) :: /* in (pre_kvm_run), out (post_kvm_run) */ @@ -6632,19 +6634,10 @@ requires a guest to interact with host userspace. .. note:: - For KVM_EXIT_IO, KVM_EXIT_MMIO, KVM_EXIT_OSI, KVM_EXIT_PAPR, KVM_EXIT_XEN, - KVM_EXIT_EPR, KVM_EXIT_X86_RDMSR, KVM_EXIT_X86_WRMSR, and KVM_EXIT_HYPERCALL - the corresponding operations are complete (and guest state is consistent) - only after userspace has re-entered the kernel with KVM_RUN. The kernel - side will first finish incomplete operations and then check for pending - signals. + For some exits, userspace must re-enter the kernel with KVM_RUN to + complete the exit and ensure guest state is consistent. - The pending state of the operation is not preserved in state which is - visible to userspace, thus userspace should ensure that the operation is - completed before performing a live migration. Userspace can re-enter the - guest with an unmasked signal pending or with the immediate_exit field set - to complete pending operations without allowing any further instructions - to be executed. + See KVM_CAP_NEEDS_COMPLETION for details. :: @@ -8239,7 +8232,7 @@ Note: Userspace is responsible for correctly configuring CPUID 0x15, a.k.a. the core crystal clock frequency, if a non-zero CPUID 0x15 is exposed to the guest. 7.36 KVM_CAP_X86_GUEST_MODE ------------------------------- +--------------------------- :Architectures: x86 :Returns: Informational only, -EINVAL on direct KVM_ENABLE_CAP. @@ -8252,6 +8245,33 @@ KVM exits with the register state of either the L1 or L2 guest depending on which executed at the time of an exit. Userspace must take care to differentiate between these cases. +7.37 KVM_CAP_NEEDS_COMPLETION +----------------------------- + +:Architectures: all +:Returns: Informational only, -EINVAL on direct KVM_ENABLE_CAP. + +The presence of this capability indicates that KVM_RUN will set +KVM_RUN_NEEDS_COMPLETION in kvm_run.flags if KVM requires userspace to re-enter +the kernel KVM_RUN to complete the exit. + +For select exits, userspace must re-enter the kernel with KVM_RUN to complete +the corresponding operation, only after which is guest state guaranteed to be +consistent. On such a KVM_RUN, the kernel side will first finish incomplete +operations and then check for pending signals. + +The pending state of the operation for such exits is not preserved in state +which is visible to userspace, thus userspace should ensure that the operation +is completed before performing state save/restore, e.g. for live migration. +Userspace can re-enter the guest with an unmasked signal pending or with the +immediate_exit field set to complete pending operations without allowing any +further instructions to be executed. + +Without KVM_CAP_NEEDS_COMPLETION, KVM_RUN_NEEDS_COMPLETION will never be set +and userspace must assume that exits of type KVM_EXIT_IO, KVM_EXIT_MMIO, +KVM_EXIT_OSI, KVM_EXIT_PAPR, KVM_EXIT_XEN, KVM_EXIT_EPR, KVM_EXIT_X86_RDMSR, +KVM_EXIT_X86_WRMSR, and KVM_EXIT_HYPERCALL require completion. + 8. Other capabilities. ====================== diff --git a/arch/powerpc/kvm/book3s_emulate.c b/arch/powerpc/kvm/book3s_emulate.c index de126d153328..15014a66c167 100644 --- a/arch/powerpc/kvm/book3s_emulate.c +++ b/arch/powerpc/kvm/book3s_emulate.c @@ -374,6 +374,7 @@ int kvmppc_core_emulate_op_pr(struct kvm_vcpu *vcpu, } vcpu->run->exit_reason = KVM_EXIT_PAPR_HCALL; + vcpu->run->flags |= KVM_RUN_NEEDS_COMPLETION; vcpu->arch.hcall_needed = 1; emulated = EMULATE_EXIT_USER; break; diff --git a/arch/powerpc/kvm/book3s_hv.c b/arch/powerpc/kvm/book3s_hv.c index b253f7372774..05ad0c3494f1 100644 --- a/arch/powerpc/kvm/book3s_hv.c +++ b/arch/powerpc/kvm/book3s_hv.c @@ -1767,6 +1767,7 @@ static int kvmppc_handle_exit_hv(struct kvm_vcpu *vcpu, for (i = 0; i < 9; ++i) run->papr_hcall.args[i] = kvmppc_get_gpr(vcpu, 4 + i); run->exit_reason = KVM_EXIT_PAPR_HCALL; + run->flags |= KVM_RUN_NEEDS_COMPLETION; vcpu->arch.hcall_needed = 1; r = RESUME_HOST; break; diff --git a/arch/powerpc/kvm/book3s_pr.c b/arch/powerpc/kvm/book3s_pr.c index 83bcdc80ce51..c45beb64905a 100644 --- a/arch/powerpc/kvm/book3s_pr.c +++ b/arch/powerpc/kvm/book3s_pr.c @@ -1310,6 +1310,7 @@ int kvmppc_handle_exit_pr(struct kvm_vcpu *vcpu, unsigned int exit_nr) run->papr_hcall.args[i] = gpr; } run->exit_reason = KVM_EXIT_PAPR_HCALL; + run->flags |= KVM_RUN_NEEDS_COMPLETION; vcpu->arch.hcall_needed = 1; r = RESUME_HOST; } else if (vcpu->arch.osi_enabled && @@ -1320,6 +1321,7 @@ int kvmppc_handle_exit_pr(struct kvm_vcpu *vcpu, unsigned int exit_nr) int i; run->exit_reason = KVM_EXIT_OSI; + run->flags |= KVM_RUN_NEEDS_COMPLETION; for (i = 0; i < 32; i++) gprs[i] = kvmppc_get_gpr(vcpu, i); vcpu->arch.osi_needed = 1; diff --git a/arch/powerpc/kvm/booke.c b/arch/powerpc/kvm/booke.c index 6a5be025a8af..3a0e00178fa5 100644 --- a/arch/powerpc/kvm/booke.c +++ b/arch/powerpc/kvm/booke.c @@ -751,6 +751,7 @@ int kvmppc_core_check_requests(struct kvm_vcpu *vcpu) vcpu->run->epr.epr = 0; vcpu->arch.epr_needed = true; vcpu->run->exit_reason = KVM_EXIT_EPR; + vcpu->run->flags |= KVM_RUN_NEEDS_COMPLETION; r = 0; } diff --git a/arch/x86/include/uapi/asm/kvm.h b/arch/x86/include/uapi/asm/kvm.h index 88585c1de416..e2ec32a8970c 100644 --- a/arch/x86/include/uapi/asm/kvm.h +++ b/arch/x86/include/uapi/asm/kvm.h @@ -104,9 +104,10 @@ struct kvm_ioapic_state { #define KVM_IRQCHIP_IOAPIC 2 #define KVM_NR_IRQCHIPS 3 -#define KVM_RUN_X86_SMM (1 << 0) -#define KVM_RUN_X86_BUS_LOCK (1 << 1) -#define KVM_RUN_X86_GUEST_MODE (1 << 2) +#define KVM_RUN_X86_SMM (1 << 0) +#define KVM_RUN_X86_BUS_LOCK (1 << 1) +#define KVM_RUN_X86_GUEST_MODE (1 << 2) +#define KVM_RUN_X86_NEEDS_COMPLETION (1 << 2) /* for KVM_GET_REGS and KVM_SET_REGS */ struct kvm_regs { diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c index a8aa12e0911d..67034b089371 100644 --- a/arch/x86/kvm/x86.c +++ b/arch/x86/kvm/x86.c @@ -10154,6 +10154,8 @@ static void post_kvm_run_save(struct kvm_vcpu *vcpu) kvm_run->flags |= KVM_RUN_X86_SMM; if (is_guest_mode(vcpu)) kvm_run->flags |= KVM_RUN_X86_GUEST_MODE; + if (vcpu->arch.complete_userspace_io) + kvm_run->flags |= KVM_RUN_NEEDS_COMPLETION; } static void update_cr8_intercept(struct kvm_vcpu *vcpu) diff --git a/include/uapi/linux/kvm.h b/include/uapi/linux/kvm.h index 343de0a51797..91dbee3ae0bc 100644 --- a/include/uapi/linux/kvm.h +++ b/include/uapi/linux/kvm.h @@ -192,6 +192,8 @@ struct kvm_xen_exit { /* Flags that describe what fields in emulation_failure hold valid data. */ #define KVM_INTERNAL_ERROR_EMULATION_FLAG_INSTRUCTION_BYTES (1ULL << 0) +#define KVM_RUN_NEEDS_COMPLETION (1 << 15) + /* * struct kvm_run can be modified by userspace at any time, so KVM must be * careful to avoid TOCTOU bugs. In order to protect KVM, HINT_UNSAFE_IN_KVM() @@ -933,6 +935,7 @@ struct kvm_enable_cap { #define KVM_CAP_PRE_FAULT_MEMORY 236 #define KVM_CAP_X86_APIC_BUS_CYCLES_NS 237 #define KVM_CAP_X86_GUEST_MODE 238 +#define KVM_CAP_NEEDS_COMPLETION 239 struct kvm_irq_routing_irqchip { __u32 irqchip; diff --git a/virt/kvm/kvm_main.c b/virt/kvm/kvm_main.c index 7d2076439081..28aa89e5ba85 100644 --- a/virt/kvm/kvm_main.c +++ b/virt/kvm/kvm_main.c @@ -4746,6 +4746,7 @@ static int kvm_vm_ioctl_check_extension_generic(struct kvm *kvm, long arg) case KVM_CAP_CHECK_EXTENSION_VM: case KVM_CAP_ENABLE_CAP_VM: case KVM_CAP_HALT_POLL: + case KVM_CAP_NEEDS_COMPLETION: return 1; #ifdef CONFIG_KVM_MMIO case KVM_CAP_COALESCED_MMIO: From patchwork Sat Jan 11 01:24:49 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sean Christopherson X-Patchwork-Id: 13935704 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 0F40BE77188 for ; Sat, 11 Jan 2025 01:31:18 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender:Reply-To:List-Subscribe: List-Help:List-Post:List-Archive:List-Unsubscribe:List-Id:Content-Type:Cc:To: From:Subject:Message-ID:References:Mime-Version:In-Reply-To:Date: Content-Transfer-Encoding:Content-ID:Content-Description:Resent-Date: Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=3UIVRx5jS5uHOad/ZRxuC+ZyfJfWx7WVNf7AHGunc9Q=; b=M3h2JMqIQC3c+pxdJRKm6jRWsz g86W3DpFWS7mdx6nEo81ujE39ydJwSDaK99ZebGQF5zzPNdKH4mMqLoiM2FTSfKg6mOKEMifBVWXR GrUyGwoDaEvFdTuQ9jTEvQS6+tvo7MRZtsxbi1Xwk3q2/FB/iAEOag/jKQ3TEFCMra5new8mi557F TwsEdinXUA4QEvqDOUZlGXWcq6U6cgbz6J/MIgaCLrCk7Yk0X3X4Kh+5e+QPR82z6vzfCGeIby5lp swrOquHoTJI3bc8Qyqk8Q61el40izsahhUSKVNB1JOdQCwJ95a6CTlXIlvMCO7dY3IQwkGXZ35Sz+ sbJRbzfw==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.98 #2 (Red Hat Linux)) id 1tWQLB-0000000HVV4-3856; Sat, 11 Jan 2025 01:31:05 +0000 Received: from desiato.infradead.org ([2001:8b0:10b:1:d65d:64ff:fe57:4e05]) by bombadil.infradead.org with esmtps (Exim 4.98 #2 (Red Hat Linux)) id 1tWQFM-0000000HUT6-1Lmp for linux-arm-kernel@bombadil.infradead.org; Sat, 11 Jan 2025 01:25:04 +0000 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=desiato.20200630; h=Content-Type:Cc:To:From:Subject: Message-ID:References:Mime-Version:In-Reply-To:Date:Reply-To:Sender: Content-Transfer-Encoding:Content-ID:Content-Description; bh=3UIVRx5jS5uHOad/ZRxuC+ZyfJfWx7WVNf7AHGunc9Q=; b=qUKEFdA5fIgGjPuYf/0ytLSvH0 NiUMy2SzLc3q6adzjbyB8wywAkmCycw+8VTD7Ek7MGMdmW8LpUYRi0TrDpBkWGg+8LCS7K5zVdGrE RLUS4Ycu0oaO/k0h4wypX5V3CvKJyW6BnPJsdQLKGvSG8EbGKhXXT9c27gEjTHfptTmqfmkWYmwYr S7oR8lNZmytKs5FHvyFYhRUJgmsRhaT+T5HT9M/sn0lUNsj3mq6tpeBo8kJqdQhDim5al1k1utxwo porqrp04RXAnY4dQqmAl+LE/DR4+OMHLrXjAolmnZUzMr85cRUrA+gArLQa7uftf+mLBd7KrLDIKZ WP0NpqTQ==; Received: from mail-pj1-x104a.google.com ([2607:f8b0:4864:20::104a]) by desiato.infradead.org with esmtps (Exim 4.98 #2 (Red Hat Linux)) id 1tWQFJ-00000009zIB-0NiF for linux-arm-kernel@lists.infradead.org; Sat, 11 Jan 2025 01:25:02 +0000 Received: by mail-pj1-x104a.google.com with SMTP id 98e67ed59e1d1-2eeeb5b7022so4818735a91.0 for ; Fri, 10 Jan 2025 17:25:00 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1736558699; x=1737163499; darn=lists.infradead.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:from:to:cc:subject:date:message-id:reply-to; bh=3UIVRx5jS5uHOad/ZRxuC+ZyfJfWx7WVNf7AHGunc9Q=; b=23dgqUCpv9Xgjh09BpxpCdW8sVlLnE2vDx9YEsBy/Y9F0AYQWuGAMeWuHPmgMAFKuj XfMDuUItO4tG5TF3ODqCS6cNUh2iV5BP2vJCK2aJO0vbVtqMiZS6sW21EPNzOkPS/pj1 Mo/1LPqdI5vCNhDlfKMfYaYoRWv1oryFix3fEpljefVniC9pG3jqN9UP5929F4sdSMt3 GWFc9Ovh4rpmvmjxKrwrlc+UiffhiPWViZ3uFEnJbbRiHkwjnBrQDfG5dDtoHv/mb9UW 3UW7crh6MydeSDg4gYWG6LZDLoHZCbpojkF88t6wv4RbzzmVaEV61HmLxbdMDC1VgF94 9IjQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1736558699; x=1737163499; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=3UIVRx5jS5uHOad/ZRxuC+ZyfJfWx7WVNf7AHGunc9Q=; b=LC+7Hsy5RsKPwKTZyHwcdUxNwVvesYmuEXQahTQkqcniyT0qqZsdxrKUpNvBfbmjVe 8d8KPECsY1PsU5KoVHPEOElGyPks/SyqYlDoL+7ryCipMnE+aKaZ7HNjASa4MtXzkHwV VlD6mihBgnYnwsoqs+BL/hrTTvDbZuExNJalYZDczVmRtE0Fo37GrTm5Zj1mwnfdDU99 x05QPYz4DH+IwMoEZmPqWI10r7Uipeaaqv1cRB3Iuwyn/k0XgzqARhncSBmQZtvKCMK9 3iY2x0ZYzfg5DbSZKepCzPtfFRreZ9p9Q5GqqmJQCLo2byhUc2DPVZObPdC7C1GmSpQg gu9A== X-Forwarded-Encrypted: i=1; AJvYcCW7745YboIl38iPwqdTzsfjmlnsqYyIPIrLet75ly6w60kipPGaNWtqLGl/E4wScYV/xUq6NT0ASCZD0ayNXEku@lists.infradead.org X-Gm-Message-State: AOJu0YxrW3YHFKmAvbbetL0IqIUTQzuG/TkJqe/uwkyXdfkafCn4MPk/ 7LwkVbcH0ph2zYxn2j5hYNrhPzJPpT6GctSzrX3Lvf2vNNWTP9PTquuGNNwcVRXKt6JBZ9cI6Rs lAw== X-Google-Smtp-Source: AGHT+IHf025t1EVvlV+j91gbU3uZgm+9mZiPk2JrEEttgvDlmrVpnhWnRwbzCB96nhWXuJS4v4uP1KYHMJk= X-Received: from pjbnw13.prod.google.com ([2002:a17:90b:254d:b0:2ee:4b69:50e1]) (user=seanjc job=prod-delivery.src-stubby-dispatcher) by 2002:a17:90b:5188:b0:2ee:df57:b194 with SMTP id 98e67ed59e1d1-2f548eceb00mr17022072a91.21.1736558698928; Fri, 10 Jan 2025 17:24:58 -0800 (PST) Date: Fri, 10 Jan 2025 17:24:49 -0800 In-Reply-To: <20250111012450.1262638-1-seanjc@google.com> Mime-Version: 1.0 References: <20250111012450.1262638-1-seanjc@google.com> X-Mailer: git-send-email 2.47.1.613.gc27f4b7a9f-goog Message-ID: <20250111012450.1262638-5-seanjc@google.com> Subject: [PATCH 4/5] KVM: selftests: Provide separate helper for KVM_RUN with immediate_exit From: Sean Christopherson To: Paolo Bonzini , Marc Zyngier , Oliver Upton , Michael Ellerman , Sean Christopherson Cc: kvm@vger.kernel.org, linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev, linuxppc-dev@lists.ozlabs.org, linux-kernel@vger.kernel.org X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20250111_012501_322612_ECDC3AE7 X-CRM114-Status: GOOD ( 11.19 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: Sean Christopherson Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Turn vcpu_run_complete_io() into a wrapper for a dedicated helper for doing KVM_RUN with immediate_exit = true, so that a future patch can do userspace exit completion if and only if it's actually necessary, whereas x86's nested exceptions test wants to unconditionally do KVM_RUN with an immediate exit. No functional change intended. Signed-off-by: Sean Christopherson --- tools/testing/selftests/kvm/include/kvm_util.h | 9 ++++++++- tools/testing/selftests/kvm/lib/kvm_util.c | 2 +- tools/testing/selftests/kvm/x86/nested_exceptions_test.c | 3 +-- 3 files changed, 10 insertions(+), 4 deletions(-) diff --git a/tools/testing/selftests/kvm/include/kvm_util.h b/tools/testing/selftests/kvm/include/kvm_util.h index 4c4e5a847f67..78fd597c1b60 100644 --- a/tools/testing/selftests/kvm/include/kvm_util.h +++ b/tools/testing/selftests/kvm/include/kvm_util.h @@ -631,7 +631,14 @@ static inline int __vcpu_run(struct kvm_vcpu *vcpu) return __vcpu_ioctl(vcpu, KVM_RUN, NULL); } -void vcpu_run_complete_io(struct kvm_vcpu *vcpu); + +void vcpu_run_immediate_exit(struct kvm_vcpu *vcpu); + +static inline void vcpu_run_complete_io(struct kvm_vcpu *vcpu) +{ + vcpu_run_immediate_exit(vcpu); +} + struct kvm_reg_list *vcpu_get_reg_list(struct kvm_vcpu *vcpu); static inline void vcpu_enable_cap(struct kvm_vcpu *vcpu, uint32_t cap, diff --git a/tools/testing/selftests/kvm/lib/kvm_util.c b/tools/testing/selftests/kvm/lib/kvm_util.c index 33fefeb3ca44..c9a33766f673 100644 --- a/tools/testing/selftests/kvm/lib/kvm_util.c +++ b/tools/testing/selftests/kvm/lib/kvm_util.c @@ -1665,7 +1665,7 @@ void vcpu_run(struct kvm_vcpu *vcpu) TEST_ASSERT(!ret, KVM_IOCTL_ERROR(KVM_RUN, ret)); } -void vcpu_run_complete_io(struct kvm_vcpu *vcpu) +void vcpu_run_immediate_exit(struct kvm_vcpu *vcpu) { int ret; diff --git a/tools/testing/selftests/kvm/x86/nested_exceptions_test.c b/tools/testing/selftests/kvm/x86/nested_exceptions_test.c index 3eb0313ffa39..4f144576a6da 100644 --- a/tools/testing/selftests/kvm/x86/nested_exceptions_test.c +++ b/tools/testing/selftests/kvm/x86/nested_exceptions_test.c @@ -238,8 +238,7 @@ int main(int argc, char *argv[]) /* Pend #SS and request immediate exit. #SS should still be pending. */ queue_ss_exception(vcpu, false); - vcpu->run->immediate_exit = true; - vcpu_run_complete_io(vcpu); + vcpu_run_immediate_exit(vcpu); /* Verify the pending events comes back out the same as it went in. */ vcpu_events_get(vcpu, &events); From patchwork Sat Jan 11 01:24:50 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sean Christopherson X-Patchwork-Id: 13935717 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 44CC6E77188 for ; Sat, 11 Jan 2025 02:38:03 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender:Reply-To:List-Subscribe: List-Help:List-Post:List-Archive:List-Unsubscribe:List-Id:Content-Type:Cc:To: From:Subject:Message-ID:References:Mime-Version:In-Reply-To:Date: Content-Transfer-Encoding:Content-ID:Content-Description:Resent-Date: Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=bnP9l4TBgZavFgl9ingoWbV7IQlMfBLutKO12YuuQSs=; b=pjn6RZYFuaEvPscH6q8hj5ck+m 0yNgt2HlGR+kg3z1M8/CcjuEt0kGoPNZPlD1Arlhlf86UBgA0MrSJWv31JF7GzLCvpZ7dyD+IcUaT +aFzaopTizQhM5KCtHQ1gjokdWhLvKWcfOkV7wAatYDSf2qO+GBeL7NgiZ7dPQeSvtjPh2ilnELrA GNhEh3crIUBbHJI+9fVfzgpxOQt5Oz06C4BD3FGrapKsi0LRJ6XJoHMreV0UpzBXXS7h3GuuSwGBV 3hqf8SbYFaC9x2r9NP08SYslDir9tuSIi2Jnw5FIOm5TyR1aq3i2HvX0rM8vQ2kN53yKymRH6B1Jm jreQsblw==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.98 #2 (Red Hat Linux)) id 1tWRNf-0000000Hadz-0kp3; Sat, 11 Jan 2025 02:37:43 +0000 Received: from desiato.infradead.org ([2001:8b0:10b:1:d65d:64ff:fe57:4e05]) by bombadil.infradead.org with esmtps (Exim 4.98 #2 (Red Hat Linux)) id 1tWQFN-0000000HUTD-0k2p for linux-arm-kernel@bombadil.infradead.org; Sat, 11 Jan 2025 01:25:05 +0000 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=desiato.20200630; h=Content-Type:Cc:To:From:Subject: Message-ID:References:Mime-Version:In-Reply-To:Date:Reply-To:Sender: Content-Transfer-Encoding:Content-ID:Content-Description; bh=bnP9l4TBgZavFgl9ingoWbV7IQlMfBLutKO12YuuQSs=; b=NqSg7lIOafpZ784gGjs1ksBIkC +WHQ9ekUmaDjt/v3BsMOd8l1wUDKcPVjAOdqUE4ryd0Ed3me9wKJ55Ujfz8y1zJak+/d8OM8FMurK ly7FcMlUq2V/xqK/hg5KNQawJT9tv8zhWDlN4Vuy2e7/fdnOYf8SFcqnzXHRTPUsIv+P8/BfhBqqp 5QF0g0ZPXyMbMKnysChWyysR6yzFpKXdHTpy+HHuQs7GuiKwvP0wRwvjFX6LJ4EZGGiDBZUxIzBPC 6nJ4KvhgXgmy5B3q6AZ6Lpjfp19wiGw7kPk+GLFt1EOJ+4eWHWZSlty3HyYe0na5Kr44HvWliXZa/ FWi4M33A==; Received: from mail-pl1-x64a.google.com ([2607:f8b0:4864:20::64a]) by desiato.infradead.org with esmtps (Exim 4.98 #2 (Red Hat Linux)) id 1tWQFK-00000009zIj-1Xgz for linux-arm-kernel@lists.infradead.org; Sat, 11 Jan 2025 01:25:04 +0000 Received: by mail-pl1-x64a.google.com with SMTP id d9443c01a7336-2161d5b3eb5so46318605ad.3 for ; Fri, 10 Jan 2025 17:25:01 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1736558700; x=1737163500; darn=lists.infradead.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:from:to:cc:subject:date:message-id:reply-to; bh=bnP9l4TBgZavFgl9ingoWbV7IQlMfBLutKO12YuuQSs=; b=XNiT0lGQkId/9VTF7JBWYRtXguwgRxp1EiZlXoFtgSProOZEDo7w2XV31muuC90N1P 2h9NvxCFJ83PhwlmS6mIPtqVVwFGkrwC278U9E3Vj+rh5NGUgUU02+PUaNg15BjFwmHC bJ9jOfA+xkodPvR9GwkSAKTxqwbtjhdkgruCA01oDYXD6QoMk1VP6WYveMdApDO2nw0o 4Pc1IqJhoBKaZ6pRV/sRZFlXFfa2kOm8cmMeDk7EadPg2zKhygCLm9xhUT2QTewLSznP GDCExrvSCGq1T9jFV7EQ+uC8JJZKERwID/vfQCrHnlNN7oWdosCzY0MXHBfOQPj+oLdq bTDw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1736558700; x=1737163500; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=bnP9l4TBgZavFgl9ingoWbV7IQlMfBLutKO12YuuQSs=; b=Lf+LlAsjkO1OjK45IIyEbq2k+fhpv0lsNFX9R6w+myuf1jd6b7ohlw8hVkgxzPo6YW BguMJ7fIgMZb5gCELYhwxthImAcOgeDQD5N4gPgfejDKH1DNmrPyKz5VT4AIr8SBN/kJ PYWwDmDrRirkYpu9nIGJ6A1mLf6J7o4Cz6auxekuk41e/OcOvdWLDb7BP9U3khbeKijf Q27NOYTgQL51TK1X4b/XE9/g5I+3ZMSg8YknWXLCw8qEHqsL1QBGRPgRzTuYzQne4i0I sLZKkjwlXSUklbkZTTgre9yeYR5vxekGAzgnDHacnsvKsymJ7iB5DrbHfOdUgeZTcVtH L/tQ== X-Forwarded-Encrypted: i=1; AJvYcCVTBYBkwBQC1unNEwyF0UVqBXxnwc6YkZYHPRQZjD46bYBpDXc6GpeUsRg76r0mUMksOeDduTp2sXkhoUaBNpkl@lists.infradead.org X-Gm-Message-State: AOJu0Yzaanehq6owrJ4j5JLxVNXopizABv7PH29tygJJppXo76SzNJE0 8vfTlcmelBFGxZ7YH/Qy4H5dlmh+fRjtLqAY05desm7GnhD9MGLDkSlZ4dQaeAKkU7tnBZ+N/sn jbg== X-Google-Smtp-Source: AGHT+IH1NdERQE8OQLjVGWXsUxxFzmL931Ova+zIR4kc3ay4XMIaQ3TzbpO5jT/NYBJvXHUAPCJZD+fSckk= X-Received: from pfbfh41.prod.google.com ([2002:a05:6a00:3929:b0:72a:bc54:8507]) (user=seanjc job=prod-delivery.src-stubby-dispatcher) by 2002:a05:6a20:4393:b0:1e1:ca91:b0e3 with SMTP id adf61e73a8af0-1e88d09e17fmr21120449637.36.1736558700525; Fri, 10 Jan 2025 17:25:00 -0800 (PST) Date: Fri, 10 Jan 2025 17:24:50 -0800 In-Reply-To: <20250111012450.1262638-1-seanjc@google.com> Mime-Version: 1.0 References: <20250111012450.1262638-1-seanjc@google.com> X-Mailer: git-send-email 2.47.1.613.gc27f4b7a9f-goog Message-ID: <20250111012450.1262638-6-seanjc@google.com> Subject: [PATCH 5/5] KVM: selftests: Rely on KVM_RUN_NEEDS_COMPLETION to complete userspace exits From: Sean Christopherson To: Paolo Bonzini , Marc Zyngier , Oliver Upton , Michael Ellerman , Sean Christopherson Cc: kvm@vger.kernel.org, linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev, linuxppc-dev@lists.ozlabs.org, linux-kernel@vger.kernel.org X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20250111_012502_527671_6A335354 X-CRM114-Status: GOOD ( 14.49 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: Sean Christopherson Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Add selftests coverage for KVM_RUN_NEEDS_COMPLETION by redoing KVM_RUN if and only if KVM states that completion is required. Opportunistically rename the helper to replace "io" with "exit", as exits that require completion are no longer limited to I/O. Signed-off-by: Sean Christopherson --- tools/testing/selftests/kvm/include/kvm_util.h | 8 ++++++-- tools/testing/selftests/kvm/lib/kvm_util.c | 4 ++++ tools/testing/selftests/kvm/lib/ucall_common.c | 2 +- tools/testing/selftests/kvm/lib/x86/processor.c | 8 +------- tools/testing/selftests/kvm/x86/triple_fault_event_test.c | 3 +-- 5 files changed, 13 insertions(+), 12 deletions(-) diff --git a/tools/testing/selftests/kvm/include/kvm_util.h b/tools/testing/selftests/kvm/include/kvm_util.h index 78fd597c1b60..86e1850e4e49 100644 --- a/tools/testing/selftests/kvm/include/kvm_util.h +++ b/tools/testing/selftests/kvm/include/kvm_util.h @@ -29,6 +29,8 @@ #define NSEC_PER_SEC 1000000000L +extern bool kvm_has_needs_completion; + struct userspace_mem_region { struct kvm_userspace_memory_region2 region; struct sparsebit *unused_phy_pages; @@ -634,9 +636,11 @@ static inline int __vcpu_run(struct kvm_vcpu *vcpu) void vcpu_run_immediate_exit(struct kvm_vcpu *vcpu); -static inline void vcpu_run_complete_io(struct kvm_vcpu *vcpu) +static inline void vcpu_run_complete_exit(struct kvm_vcpu *vcpu) { - vcpu_run_immediate_exit(vcpu); + if (!kvm_has_needs_completion || + (vcpu->run->flags & KVM_RUN_NEEDS_COMPLETION)) + vcpu_run_immediate_exit(vcpu); } struct kvm_reg_list *vcpu_get_reg_list(struct kvm_vcpu *vcpu); diff --git a/tools/testing/selftests/kvm/lib/kvm_util.c b/tools/testing/selftests/kvm/lib/kvm_util.c index c9a33766f673..95ac9b981049 100644 --- a/tools/testing/selftests/kvm/lib/kvm_util.c +++ b/tools/testing/selftests/kvm/lib/kvm_util.c @@ -19,6 +19,8 @@ #define KVM_UTIL_MIN_PFN 2 +bool kvm_has_needs_completion; + uint32_t guest_random_seed; struct guest_random_state guest_rng; static uint32_t last_guest_seed; @@ -2253,6 +2255,8 @@ void __attribute((constructor)) kvm_selftest_init(void) /* Tell stdout not to buffer its content. */ setbuf(stdout, NULL); + kvm_has_needs_completion = kvm_check_cap(KVM_CAP_NEEDS_COMPLETION); + guest_random_seed = last_guest_seed = random(); pr_info("Random seed: 0x%x\n", guest_random_seed); diff --git a/tools/testing/selftests/kvm/lib/ucall_common.c b/tools/testing/selftests/kvm/lib/ucall_common.c index 42151e571953..125584a94ba8 100644 --- a/tools/testing/selftests/kvm/lib/ucall_common.c +++ b/tools/testing/selftests/kvm/lib/ucall_common.c @@ -154,7 +154,7 @@ uint64_t get_ucall(struct kvm_vcpu *vcpu, struct ucall *uc) "Guest failed to allocate ucall struct"); memcpy(uc, addr, sizeof(*uc)); - vcpu_run_complete_io(vcpu); + vcpu_run_complete_exit(vcpu); } else { memset(uc, 0, sizeof(*uc)); } diff --git a/tools/testing/selftests/kvm/lib/x86/processor.c b/tools/testing/selftests/kvm/lib/x86/processor.c index bd5a802fa7a5..1db4764e413b 100644 --- a/tools/testing/selftests/kvm/lib/x86/processor.c +++ b/tools/testing/selftests/kvm/lib/x86/processor.c @@ -1077,13 +1077,7 @@ struct kvm_x86_state *vcpu_save_state(struct kvm_vcpu *vcpu) nested_size, sizeof(state->nested_)); } - /* - * When KVM exits to userspace with KVM_EXIT_IO, KVM guarantees - * guest state is consistent only after userspace re-enters the - * kernel with KVM_RUN. Complete IO prior to migrating state - * to a new VM. - */ - vcpu_run_complete_io(vcpu); + vcpu_run_complete_exit(vcpu); state = malloc(sizeof(*state) + msr_list->nmsrs * sizeof(state->msrs.entries[0])); TEST_ASSERT(state, "-ENOMEM when allocating kvm state"); diff --git a/tools/testing/selftests/kvm/x86/triple_fault_event_test.c b/tools/testing/selftests/kvm/x86/triple_fault_event_test.c index 56306a19144a..82d732788bc1 100644 --- a/tools/testing/selftests/kvm/x86/triple_fault_event_test.c +++ b/tools/testing/selftests/kvm/x86/triple_fault_event_test.c @@ -97,8 +97,7 @@ int main(void) events.flags |= KVM_VCPUEVENT_VALID_TRIPLE_FAULT; events.triple_fault.pending = true; vcpu_events_set(vcpu, &events); - run->immediate_exit = true; - vcpu_run_complete_io(vcpu); + vcpu_run_complete_exit(vcpu); vcpu_events_get(vcpu, &events); TEST_ASSERT(events.flags & KVM_VCPUEVENT_VALID_TRIPLE_FAULT,