From patchwork Tue Jun 8 15:48:02 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jean-Philippe Brucker X-Patchwork-Id: 12307283 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-18.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED, USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 4D79EC47082 for ; Tue, 8 Jun 2021 15:54:07 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 3183261363 for ; Tue, 8 Jun 2021 15:54:07 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232031AbhFHPz7 (ORCPT ); Tue, 8 Jun 2021 11:55:59 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:50080 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231175AbhFHPz4 (ORCPT ); Tue, 8 Jun 2021 11:55:56 -0400 Received: from mail-wr1-x432.google.com (mail-wr1-x432.google.com [IPv6:2a00:1450:4864:20::432]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 34630C061787 for ; Tue, 8 Jun 2021 08:54:03 -0700 (PDT) Received: by mail-wr1-x432.google.com with SMTP id c9so13442102wrt.5 for ; Tue, 08 Jun 2021 08:54:03 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=BRXQuaY4B7tLzh+9+26u4SBQDqXVmvyu3lGkiJd9OsE=; b=gr/gglQLIVxWHQCeK/ivVghBcHcoS3AZLiI+39Ob4NoNZW41wlCK9c24hTBSR5ke43 fHeHoYJh3ii5RmoLBw/CW8i9xLQHTkDwnUeagqqOvprPrSo33yOPMjD00O9qQx4variP wAOxxaDwYJuP0L6+4hIE5QrRBHiMb4m7YKyyPoINsi+QANcEjGqYtUh+ociH3y4IBZ8V jAZXSvRXgREbES9TgpI+AdGsYZyWh8glxmBUdFcBPeP71icony0OCulH2ORerU3vtXV8 0rLFCmm2YegiS3GyBPg6CHBjsxJyL2HITbLT27oJfFVg8ODdXSfIPyG0kVPmEW+C96zg /8/g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=BRXQuaY4B7tLzh+9+26u4SBQDqXVmvyu3lGkiJd9OsE=; b=pQFE18McJNSUJb0+BJL5Z2QW2L0O7MNI+PRlwDarI+t1chlzEgKkV7hUmrnwLlmmA2 XIfR625bj00ZWe8Ky/oJHY4lL9OevoC+P2CWn0/DRYsYC+cVkWgQQW15+fz3CVTAg0nm R2AKfEKv5EFHNUw2FXUKTWLEge1cv2YVHPfcDq5pTIdZt5YZEqNmdC6BgQB0JMfCTaVr dIorDYmj6JGHVUqCuRtzYr+ko8MpSDRuXV4qpUwCmw5BItmHxaf4PtoA8fz8D2FgPEkl rkAC98LzhBeVAR8PaOVMAPLkVdZhRqrrpm9bUfwmF+yEZ2/iuIOduSpcX5WpvbPsbGk/ beig== X-Gm-Message-State: AOAM5325W1ZIBqyU0hKSgLGVB+Hgt02FjfnMED7jC8O3baZAVR0BSvQD jDAYLkGUhzCbfIQflhe+mhByIw== X-Google-Smtp-Source: ABdhPJykr6vORYCQFK0zdalZ1dmo1FvGrQrMtsX1IEM0HdNX3NO+P/oNrCPC+8IGF+lGOb5mFL1IQA== X-Received: by 2002:adf:eac3:: with SMTP id o3mr23276114wrn.157.1623167641696; Tue, 08 Jun 2021 08:54:01 -0700 (PDT) Received: from localhost.localdomain (adsl-84-226-111-173.adslplus.ch. [84.226.111.173]) by smtp.gmail.com with ESMTPSA id l31sm3314180wms.16.2021.06.08.08.54.00 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 08 Jun 2021 08:54:01 -0700 (PDT) From: Jean-Philippe Brucker To: maz@kernel.org Cc: kvm@vger.kernel.org, kvmarm@lists.cs.columbia.edu, linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, pbonzini@redhat.com, corbet@lwn.net, james.morse@arm.com, alexandru.elisei@arm.com, suzuki.poulose@arm.com, catalin.marinas@arm.com, will@kernel.org, lorenzo.pieralisi@arm.com, salil.mehta@huawei.com, shameerali.kolothum.thodi@huawei.com, jonathan.cameron@huawei.com, Jean-Philippe Brucker Subject: [RFC PATCH 1/5] KVM: arm64: Replace power_off with mp_state in struct kvm_vcpu_arch Date: Tue, 8 Jun 2021 17:48:02 +0200 Message-Id: <20210608154805.216869-2-jean-philippe@linaro.org> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20210608154805.216869-1-jean-philippe@linaro.org> References: <20210608154805.216869-1-jean-philippe@linaro.org> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org In order to add a new "suspend" power state, replace power_off with mp_state in struct kvm_vcpu_arch. Factor the vcpu_off() function while we're here. No functional change intended. Signed-off-by: Jean-Philippe Brucker --- arch/arm64/include/asm/kvm_host.h | 6 ++++-- arch/arm64/kvm/arm.c | 29 +++++++++++++++-------------- arch/arm64/kvm/psci.c | 19 ++++++------------- 3 files changed, 25 insertions(+), 29 deletions(-) diff --git a/arch/arm64/include/asm/kvm_host.h b/arch/arm64/include/asm/kvm_host.h index 7cd7d5c8c4bc..55a04f4d5919 100644 --- a/arch/arm64/include/asm/kvm_host.h +++ b/arch/arm64/include/asm/kvm_host.h @@ -340,8 +340,8 @@ struct kvm_vcpu_arch { u32 mdscr_el1; } guest_debug_preserved; - /* vcpu power-off state */ - bool power_off; + /* vcpu power state (runnable, stopped, halted) */ + u32 mp_state; /* Don't run the guest (internal implementation need) */ bool pause; @@ -720,6 +720,8 @@ int kvm_arm_vcpu_arch_get_attr(struct kvm_vcpu *vcpu, struct kvm_device_attr *attr); int kvm_arm_vcpu_arch_has_attr(struct kvm_vcpu *vcpu, struct kvm_device_attr *attr); +void kvm_arm_vcpu_power_off(struct kvm_vcpu *vcpu); +bool kvm_arm_vcpu_is_off(struct kvm_vcpu *vcpu); /* Guest/host FPSIMD coordination helpers */ int kvm_arch_vcpu_run_map_fp(struct kvm_vcpu *vcpu); diff --git a/arch/arm64/kvm/arm.c b/arch/arm64/kvm/arm.c index e720148232a0..bcc24adb9c0a 100644 --- a/arch/arm64/kvm/arm.c +++ b/arch/arm64/kvm/arm.c @@ -435,21 +435,22 @@ void kvm_arch_vcpu_put(struct kvm_vcpu *vcpu) vcpu->cpu = -1; } -static void vcpu_power_off(struct kvm_vcpu *vcpu) +void kvm_arm_vcpu_power_off(struct kvm_vcpu *vcpu) { - vcpu->arch.power_off = true; + vcpu->arch.mp_state = KVM_MP_STATE_STOPPED; kvm_make_request(KVM_REQ_SLEEP, vcpu); kvm_vcpu_kick(vcpu); } +bool kvm_arm_vcpu_is_off(struct kvm_vcpu *vcpu) +{ + return vcpu->arch.mp_state == KVM_MP_STATE_STOPPED; +} + int kvm_arch_vcpu_ioctl_get_mpstate(struct kvm_vcpu *vcpu, struct kvm_mp_state *mp_state) { - if (vcpu->arch.power_off) - mp_state->mp_state = KVM_MP_STATE_STOPPED; - else - mp_state->mp_state = KVM_MP_STATE_RUNNABLE; - + mp_state->mp_state = vcpu->arch.mp_state; return 0; } @@ -460,10 +461,10 @@ int kvm_arch_vcpu_ioctl_set_mpstate(struct kvm_vcpu *vcpu, switch (mp_state->mp_state) { case KVM_MP_STATE_RUNNABLE: - vcpu->arch.power_off = false; + vcpu->arch.mp_state = KVM_MP_STATE_RUNNABLE; break; case KVM_MP_STATE_STOPPED: - vcpu_power_off(vcpu); + kvm_arm_vcpu_power_off(vcpu); break; default: ret = -EINVAL; @@ -483,7 +484,7 @@ int kvm_arch_vcpu_runnable(struct kvm_vcpu *v) { bool irq_lines = *vcpu_hcr(v) & (HCR_VI | HCR_VF); return ((irq_lines || kvm_vgic_vcpu_pending_irq(v)) - && !v->arch.power_off && !v->arch.pause); + && !kvm_arm_vcpu_is_off(v) && !v->arch.pause); } bool kvm_arch_vcpu_in_kernel(struct kvm_vcpu *vcpu) @@ -643,10 +644,10 @@ static void vcpu_req_sleep(struct kvm_vcpu *vcpu) struct rcuwait *wait = kvm_arch_vcpu_get_wait(vcpu); rcuwait_wait_event(wait, - (!vcpu->arch.power_off) &&(!vcpu->arch.pause), + !kvm_arm_vcpu_is_off(vcpu) && !vcpu->arch.pause, TASK_INTERRUPTIBLE); - if (vcpu->arch.power_off || vcpu->arch.pause) { + if (kvm_arm_vcpu_is_off(vcpu) || vcpu->arch.pause) { /* Awaken to handle a signal, request we sleep again later. */ kvm_make_request(KVM_REQ_SLEEP, vcpu); } @@ -1087,9 +1088,9 @@ static int kvm_arch_vcpu_ioctl_vcpu_init(struct kvm_vcpu *vcpu, * Handle the "start in power-off" case. */ if (test_bit(KVM_ARM_VCPU_POWER_OFF, vcpu->arch.features)) - vcpu_power_off(vcpu); + kvm_arm_vcpu_power_off(vcpu); else - vcpu->arch.power_off = false; + vcpu->arch.mp_state = KVM_MP_STATE_RUNNABLE; return 0; } diff --git a/arch/arm64/kvm/psci.c b/arch/arm64/kvm/psci.c index db4056ecccfd..24b4a2265dbd 100644 --- a/arch/arm64/kvm/psci.c +++ b/arch/arm64/kvm/psci.c @@ -52,13 +52,6 @@ static unsigned long kvm_psci_vcpu_suspend(struct kvm_vcpu *vcpu) return PSCI_RET_SUCCESS; } -static void kvm_psci_vcpu_off(struct kvm_vcpu *vcpu) -{ - vcpu->arch.power_off = true; - kvm_make_request(KVM_REQ_SLEEP, vcpu); - kvm_vcpu_kick(vcpu); -} - static unsigned long kvm_psci_vcpu_on(struct kvm_vcpu *source_vcpu) { struct vcpu_reset_state *reset_state; @@ -78,7 +71,7 @@ static unsigned long kvm_psci_vcpu_on(struct kvm_vcpu *source_vcpu) */ if (!vcpu) return PSCI_RET_INVALID_PARAMS; - if (!vcpu->arch.power_off) { + if (!kvm_arm_vcpu_is_off(vcpu)) { if (kvm_psci_version(source_vcpu, kvm) != KVM_ARM_PSCI_0_1) return PSCI_RET_ALREADY_ON; else @@ -107,7 +100,7 @@ static unsigned long kvm_psci_vcpu_on(struct kvm_vcpu *source_vcpu) */ smp_wmb(); - vcpu->arch.power_off = false; + vcpu->arch.mp_state = KVM_MP_STATE_RUNNABLE; kvm_vcpu_wake_up(vcpu); return PSCI_RET_SUCCESS; @@ -142,7 +135,7 @@ static unsigned long kvm_psci_vcpu_affinity_info(struct kvm_vcpu *vcpu) mpidr = kvm_vcpu_get_mpidr_aff(tmp); if ((mpidr & target_affinity_mask) == target_affinity) { matching_cpus++; - if (!tmp->arch.power_off) + if (!kvm_arm_vcpu_is_off(tmp)) return PSCI_0_2_AFFINITY_LEVEL_ON; } } @@ -168,7 +161,7 @@ static void kvm_prepare_system_event(struct kvm_vcpu *vcpu, u32 type) * re-initialized. */ kvm_for_each_vcpu(i, tmp, vcpu->kvm) - tmp->arch.power_off = true; + tmp->arch.mp_state = KVM_MP_STATE_STOPPED; kvm_make_all_cpus_request(vcpu->kvm, KVM_REQ_SLEEP); memset(&vcpu->run->system_event, 0, sizeof(vcpu->run->system_event)); @@ -237,7 +230,7 @@ static int kvm_psci_0_2_call(struct kvm_vcpu *vcpu) val = kvm_psci_vcpu_suspend(vcpu); break; case PSCI_0_2_FN_CPU_OFF: - kvm_psci_vcpu_off(vcpu); + kvm_arm_vcpu_power_off(vcpu); val = PSCI_RET_SUCCESS; break; case PSCI_0_2_FN_CPU_ON: @@ -350,7 +343,7 @@ static int kvm_psci_0_1_call(struct kvm_vcpu *vcpu) switch (psci_fn) { case KVM_PSCI_FN_CPU_OFF: - kvm_psci_vcpu_off(vcpu); + kvm_arm_vcpu_power_off(vcpu); val = PSCI_RET_SUCCESS; break; case KVM_PSCI_FN_CPU_ON: From patchwork Tue Jun 8 15:48:03 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jean-Philippe Brucker X-Patchwork-Id: 12307289 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-18.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED, USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 513F4C47082 for ; Tue, 8 Jun 2021 15:55:09 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 3797860FEA for ; Tue, 8 Jun 2021 15:55:09 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233055AbhFHP47 (ORCPT ); Tue, 8 Jun 2021 11:56:59 -0400 Received: from mail-wr1-f54.google.com ([209.85.221.54]:45661 "EHLO mail-wr1-f54.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S233009AbhFHP44 (ORCPT ); Tue, 8 Jun 2021 11:56:56 -0400 Received: by mail-wr1-f54.google.com with SMTP id z8so22095929wrp.12 for ; Tue, 08 Jun 2021 08:55:03 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=luZ2XBy4kIjKsUNN5YLr1hr2hYOSweOt5shsxweZY5c=; b=sbCxjbeI/oicVowr1ScpDXLBsVThc8WALV+maymt0+dWYJi+SdK1EjS3CBamvobctn oVj9jPoiFl+XFyqVbz7oy3/wgxvfgv4TUhXNsZtIJYnlHpvoajG5yUBb4YDZeadnQyZ6 FVMM9S8Byjv7mzoU+5SISyAl31bOpdFIzZ//vvWfgeVaD+uvShhJnQfPo+J+Wj1q9caq U3Hk1jw15VgUv/U1tleMfSt+xrJk0XKUTYZF7Qb+jytlId0YYKfFcJnS///J517/MU38 hcdfZin3+XvAGgVk0iRCkX8FCL1xBqvocAV1qiayCXWYGUiFRbLXitgw6SGtGQZndgP9 UYFg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=luZ2XBy4kIjKsUNN5YLr1hr2hYOSweOt5shsxweZY5c=; b=OdLiQim5whhbq0WbUX1yP7fJtBnOyq9TJKQL0Ao234HGvdBpVz5PHdaVLBrjzGV1q8 28IGhNWWKnyEFHbrKHbwO5h3wn86eYwkTd5VlJXK7CHI642sryaedc+tAVO+ymFPxaU+ TBImStUacBUmLjia3Qd8Cksluw+BYrMwhe+mkf7V99kWKFlMWenP9mDpcEENcwmRG88x sEnD8+2J1TkHcmfDcL+HAqDcrLQ1YhW/mGKj2ysqyimOKtjPzb5skF3DqtUamDfJL1pY C3ypwB0TDhejgXvlsYqV+IWp6JE7glDStGJS6DovwgcJUsrObp32uK2BzkVKylQseKob 6WCA== X-Gm-Message-State: AOAM532+AQqvgRPEU6y4sXhplXakwGpa/q4IgGkuFniIETaOduKWbDvv FClaRwJI2YSqRfjfP5pRSQUpNg== X-Google-Smtp-Source: ABdhPJzkPP5/N5I0QEPYl+or03a+RJK+q3fbfrxy8aomfkBbcwKtQQDp8ZjdsnAAKwnqh5+OZC4YyA== X-Received: by 2002:a5d:4a4b:: with SMTP id v11mr23583559wrs.246.1623167642648; Tue, 08 Jun 2021 08:54:02 -0700 (PDT) Received: from localhost.localdomain (adsl-84-226-111-173.adslplus.ch. [84.226.111.173]) by smtp.gmail.com with ESMTPSA id l31sm3314180wms.16.2021.06.08.08.54.01 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 08 Jun 2021 08:54:02 -0700 (PDT) From: Jean-Philippe Brucker To: maz@kernel.org Cc: kvm@vger.kernel.org, kvmarm@lists.cs.columbia.edu, linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, pbonzini@redhat.com, corbet@lwn.net, james.morse@arm.com, alexandru.elisei@arm.com, suzuki.poulose@arm.com, catalin.marinas@arm.com, will@kernel.org, lorenzo.pieralisi@arm.com, salil.mehta@huawei.com, shameerali.kolothum.thodi@huawei.com, jonathan.cameron@huawei.com, Jean-Philippe Brucker Subject: [RFC PATCH 2/5] KVM: arm64: Move WFI execution to check_vcpu_requests() Date: Tue, 8 Jun 2021 17:48:03 +0200 Message-Id: <20210608154805.216869-3-jean-philippe@linaro.org> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20210608154805.216869-1-jean-philippe@linaro.org> References: <20210608154805.216869-1-jean-philippe@linaro.org> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org Prepare for WFI requests from userspace, by adding a suspend request and moving the WFI execution into check_vcpu_requests(), next to the power-off logic. vcpu->arch.mp_state, previously only RUNNABLE or STOPPED, supports an additional state HALTED and two new state transitions: RUNNABLE -> HALTED from WFI or PSCI CPU_SUSPEND (same vCPU) HALTED -> RUNNABLE vGIC IRQ, pending timer, signal There shouldn't be any functional change with this patch, even though the KVM_GET_MP_STATE ioctl could now in theory return KVM_MP_STATE_HALTED, which would break some users' mp_state support. In practice it should not happen because we do not return to userspace with HALTED state. Both WFI and PSCI CPU_SUSPEND stay in the vCPU run loop until the suspend request is consumed. It does feel fragile though, maybe we should explicitly return RUNNABLE in KVM_GET_MP_STATE in place of HALTED, to prevent future breakage. Signed-off-by: Jean-Philippe Brucker --- arch/arm64/include/asm/kvm_host.h | 2 ++ arch/arm64/kvm/arm.c | 18 ++++++++++++++- arch/arm64/kvm/handle_exit.c | 3 +-- arch/arm64/kvm/psci.c | 37 +++++++++++++------------------ 4 files changed, 35 insertions(+), 25 deletions(-) diff --git a/arch/arm64/include/asm/kvm_host.h b/arch/arm64/include/asm/kvm_host.h index 55a04f4d5919..3ca732feb9a5 100644 --- a/arch/arm64/include/asm/kvm_host.h +++ b/arch/arm64/include/asm/kvm_host.h @@ -46,6 +46,7 @@ #define KVM_REQ_VCPU_RESET KVM_ARCH_REQ(2) #define KVM_REQ_RECORD_STEAL KVM_ARCH_REQ(3) #define KVM_REQ_RELOAD_GICv4 KVM_ARCH_REQ(4) +#define KVM_REQ_SUSPEND KVM_ARCH_REQ(5) #define KVM_DIRTY_LOG_MANUAL_CAPS (KVM_DIRTY_LOG_MANUAL_PROTECT_ENABLE | \ KVM_DIRTY_LOG_INITIALLY_SET) @@ -722,6 +723,7 @@ int kvm_arm_vcpu_arch_has_attr(struct kvm_vcpu *vcpu, struct kvm_device_attr *attr); void kvm_arm_vcpu_power_off(struct kvm_vcpu *vcpu); bool kvm_arm_vcpu_is_off(struct kvm_vcpu *vcpu); +void kvm_arm_vcpu_suspend(struct kvm_vcpu *vcpu); /* Guest/host FPSIMD coordination helpers */ int kvm_arch_vcpu_run_map_fp(struct kvm_vcpu *vcpu); diff --git a/arch/arm64/kvm/arm.c b/arch/arm64/kvm/arm.c index bcc24adb9c0a..d8cbaa0373c7 100644 --- a/arch/arm64/kvm/arm.c +++ b/arch/arm64/kvm/arm.c @@ -447,6 +447,12 @@ bool kvm_arm_vcpu_is_off(struct kvm_vcpu *vcpu) return vcpu->arch.mp_state == KVM_MP_STATE_STOPPED; } +void kvm_arm_vcpu_suspend(struct kvm_vcpu *vcpu) +{ + vcpu->arch.mp_state = KVM_MP_STATE_HALTED; + kvm_make_request(KVM_REQ_SUSPEND, vcpu); +} + int kvm_arch_vcpu_ioctl_get_mpstate(struct kvm_vcpu *vcpu, struct kvm_mp_state *mp_state) { @@ -667,6 +673,8 @@ static int kvm_vcpu_initialized(struct kvm_vcpu *vcpu) static void check_vcpu_requests(struct kvm_vcpu *vcpu) { + bool irq_pending; + if (kvm_request_pending(vcpu)) { if (kvm_check_request(KVM_REQ_SLEEP, vcpu)) vcpu_req_sleep(vcpu); @@ -678,7 +686,7 @@ static void check_vcpu_requests(struct kvm_vcpu *vcpu) * Clear IRQ_PENDING requests that were made to guarantee * that a VCPU sees new virtual interrupts. */ - kvm_check_request(KVM_REQ_IRQ_PENDING, vcpu); + irq_pending = kvm_check_request(KVM_REQ_IRQ_PENDING, vcpu); if (kvm_check_request(KVM_REQ_RECORD_STEAL, vcpu)) kvm_update_stolen_time(vcpu); @@ -690,6 +698,14 @@ static void check_vcpu_requests(struct kvm_vcpu *vcpu) vgic_v4_load(vcpu); preempt_enable(); } + + if (kvm_check_request(KVM_REQ_SUSPEND, vcpu)) { + if (!irq_pending) { + kvm_vcpu_block(vcpu); + kvm_clear_request(KVM_REQ_UNHALT, vcpu); + } + vcpu->arch.mp_state = KVM_MP_STATE_RUNNABLE; + } } } diff --git a/arch/arm64/kvm/handle_exit.c b/arch/arm64/kvm/handle_exit.c index 6f48336b1d86..9717df3104cf 100644 --- a/arch/arm64/kvm/handle_exit.c +++ b/arch/arm64/kvm/handle_exit.c @@ -95,8 +95,7 @@ static int kvm_handle_wfx(struct kvm_vcpu *vcpu) } else { trace_kvm_wfx_arm64(*vcpu_pc(vcpu), false); vcpu->stat.wfi_exit_stat++; - kvm_vcpu_block(vcpu); - kvm_clear_request(KVM_REQ_UNHALT, vcpu); + kvm_arm_vcpu_suspend(vcpu); } kvm_incr_pc(vcpu); diff --git a/arch/arm64/kvm/psci.c b/arch/arm64/kvm/psci.c index 24b4a2265dbd..42a307ceb95f 100644 --- a/arch/arm64/kvm/psci.c +++ b/arch/arm64/kvm/psci.c @@ -31,27 +31,6 @@ static unsigned long psci_affinity_mask(unsigned long affinity_level) return 0; } -static unsigned long kvm_psci_vcpu_suspend(struct kvm_vcpu *vcpu) -{ - /* - * NOTE: For simplicity, we make VCPU suspend emulation to be - * same-as WFI (Wait-for-interrupt) emulation. - * - * This means for KVM the wakeup events are interrupts and - * this is consistent with intended use of StateID as described - * in section 5.4.1 of PSCI v0.2 specification (ARM DEN 0022A). - * - * Further, we also treat power-down request to be same as - * stand-by request as-per section 5.4.2 clause 3 of PSCI v0.2 - * specification (ARM DEN 0022A). This means all suspend states - * for KVM will preserve the register state. - */ - kvm_vcpu_block(vcpu); - kvm_clear_request(KVM_REQ_UNHALT, vcpu); - - return PSCI_RET_SUCCESS; -} - static unsigned long kvm_psci_vcpu_on(struct kvm_vcpu *source_vcpu) { struct vcpu_reset_state *reset_state; @@ -227,7 +206,21 @@ static int kvm_psci_0_2_call(struct kvm_vcpu *vcpu) break; case PSCI_0_2_FN_CPU_SUSPEND: case PSCI_0_2_FN64_CPU_SUSPEND: - val = kvm_psci_vcpu_suspend(vcpu); + /* + * NOTE: For simplicity, we make VCPU suspend emulation to be + * same-as WFI (Wait-for-interrupt) emulation. + * + * This means for KVM the wakeup events are interrupts and this + * is consistent with intended use of StateID as described in + * section 5.4.1 of PSCI v0.2 specification (ARM DEN 0022A). + * + * Further, we also treat power-down request to be same as + * stand-by request as-per section 5.4.2 clause 3 of PSCI v0.2 + * specification (ARM DEN 0022A). This means all suspend states + * for KVM will preserve the register state. + */ + kvm_arm_vcpu_suspend(vcpu); + val = PSCI_RET_SUCCESS; break; case PSCI_0_2_FN_CPU_OFF: kvm_arm_vcpu_power_off(vcpu); From patchwork Tue Jun 8 15:48:04 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jean-Philippe Brucker X-Patchwork-Id: 12307285 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-18.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED, USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id E79EFC4743D for ; Tue, 8 Jun 2021 15:54:07 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id C9C5B6134F for ; Tue, 8 Jun 2021 15:54:07 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231844AbhFHP4A (ORCPT ); Tue, 8 Jun 2021 11:56:00 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:50088 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231865AbhFHPz6 (ORCPT ); Tue, 8 Jun 2021 11:55:58 -0400 Received: from mail-wr1-x431.google.com (mail-wr1-x431.google.com [IPv6:2a00:1450:4864:20::431]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 020ECC061787 for ; Tue, 8 Jun 2021 08:54:05 -0700 (PDT) Received: by mail-wr1-x431.google.com with SMTP id c5so22091480wrq.9 for ; Tue, 08 Jun 2021 08:54:04 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=6VUTi6r0zGjvD+ox5LbEVdcu7SOJ3CxYqosX/6quRvc=; b=iZxQMnxOlNisd6FMWcutFJg9ihWBRrP18+4KoCR7isLbBgJRjV2h9bm7ro5tM0SuA5 MKA963p2Fjmaerhs0VLNc3mXjmVJbt4LXKOds3/hhLBfPZTNhM4ALrICmh9wRHLm3mDj 9X0bwod0yko2OzqVbHqT6E91S1tWi3hL1+YyfpA/AefIXhN0izWgnv2m4AsrQM4gTRFb XPGRIVmgyNb6TPkqf5zH5+OpFuMLdvJItw67EM2UPkwxqiE40pkNyBxiSgrBs7QNMC1Q UqgLxRznUGNuKj9K/rYCy6GTXUiiCElOBKA3oqupM1W8sLyaqix9NLGflqG3lQaM8Z05 rHZg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=6VUTi6r0zGjvD+ox5LbEVdcu7SOJ3CxYqosX/6quRvc=; b=SC+WTTWtaZ4p6Rwqh5hd3Wfa6PVKU0VwCTWoYeMaaiRor8dxipDzv/w9b887ATuzuY QnPQg8Sd8lFaU21Sc36uDD+UDfEnObnxZU1JSqhPW6YGMAM1272oPqaqfj5OxdnraICr gGUGilNrLOtM0ePC1y37flMWaWJAyaFfTfE470p8495l3hNoe+fmDhwZMF03kY6kPyxc E6XOHmuUf5rU5PeSP9CSChfvM5oMZlPDutpfMgLjPv3Oh4qFTCHXTWYUkeKfqxjGbiVz jDUanNfLGAyJRFbs2cx0F4u5+Li2ukeQvma3R/VSrjgBQpABEaaYkbNb3Q3Yxm6Zg9A5 /v6w== X-Gm-Message-State: AOAM533YGuyAg0K4iexDt0GPc+2vZg2nkw7l11EpV0jLJHlwZ2famxkd YyJ8wzFZLPW04iAR877i20aDdw== X-Google-Smtp-Source: ABdhPJy61JSG9YYU4eU/zql91JN1YArLPTW12IIWCRmtkz+g9RAUJUbtnqHF/W7eyUKs8LNMBHfbCw== X-Received: by 2002:adf:e50b:: with SMTP id j11mr22681729wrm.377.1623167643627; Tue, 08 Jun 2021 08:54:03 -0700 (PDT) Received: from localhost.localdomain (adsl-84-226-111-173.adslplus.ch. [84.226.111.173]) by smtp.gmail.com with ESMTPSA id l31sm3314180wms.16.2021.06.08.08.54.02 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 08 Jun 2021 08:54:03 -0700 (PDT) From: Jean-Philippe Brucker To: maz@kernel.org Cc: kvm@vger.kernel.org, kvmarm@lists.cs.columbia.edu, linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, pbonzini@redhat.com, corbet@lwn.net, james.morse@arm.com, alexandru.elisei@arm.com, suzuki.poulose@arm.com, catalin.marinas@arm.com, will@kernel.org, lorenzo.pieralisi@arm.com, salil.mehta@huawei.com, shameerali.kolothum.thodi@huawei.com, jonathan.cameron@huawei.com, Jean-Philippe Brucker Subject: [RFC PATCH 3/5] KVM: arm64: Allow userspace to request WFI Date: Tue, 8 Jun 2021 17:48:04 +0200 Message-Id: <20210608154805.216869-4-jean-philippe@linaro.org> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20210608154805.216869-1-jean-philippe@linaro.org> References: <20210608154805.216869-1-jean-philippe@linaro.org> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org To help userspace implement PSCI CPU_SUSPEND, allow setting the "HALTED" MP state to request a WFI before returning to the guest. Userspace won't obtain a HALTED mp_state from a KVM_GET_MP_STATE call unless they set it themselves. When set by KVM, to handle wfi or CPU_SUSPEND, it is consumed before returning to userspace. Signed-off-by: Jean-Philippe Brucker --- Documentation/virt/kvm/api.rst | 15 +++++++++------ include/uapi/linux/kvm.h | 1 + arch/arm64/kvm/arm.c | 11 ++++++++++- 3 files changed, 20 insertions(+), 7 deletions(-) diff --git a/Documentation/virt/kvm/api.rst b/Documentation/virt/kvm/api.rst index 7fcb2fd38f42..e4fe7fb60d5d 100644 --- a/Documentation/virt/kvm/api.rst +++ b/Documentation/virt/kvm/api.rst @@ -1416,8 +1416,8 @@ Possible values are: which has not yet received an INIT signal [x86] KVM_MP_STATE_INIT_RECEIVED the vcpu has received an INIT signal, and is now ready for a SIPI [x86] - KVM_MP_STATE_HALTED the vcpu has executed a HLT instruction and - is waiting for an interrupt [x86] + KVM_MP_STATE_HALTED the vcpu has executed a HLT/WFI instruction + and is waiting for an interrupt [x86,arm64] KVM_MP_STATE_SIPI_RECEIVED the vcpu has just received a SIPI (vector accessible via KVM_GET_VCPU_EVENTS) [x86] KVM_MP_STATE_STOPPED the vcpu is stopped [s390,arm/arm64] @@ -1435,8 +1435,9 @@ these architectures. For arm/arm64: ^^^^^^^^^^^^^^ -The only states that are valid are KVM_MP_STATE_STOPPED and -KVM_MP_STATE_RUNNABLE which reflect if the vcpu is paused or not. +Valid states are KVM_MP_STATE_STOPPED and KVM_MP_STATE_RUNNABLE which reflect +if the vcpu is paused or not. If KVM_CAP_ARM_MP_HALTED is present, state +KVM_MP_STATE_HALTED is also valid. 4.39 KVM_SET_MP_STATE --------------------- @@ -1457,8 +1458,10 @@ these architectures. For arm/arm64: ^^^^^^^^^^^^^^ -The only states that are valid are KVM_MP_STATE_STOPPED and -KVM_MP_STATE_RUNNABLE which reflect if the vcpu should be paused or not. +Valid states are KVM_MP_STATE_STOPPED and KVM_MP_STATE_RUNNABLE which reflect +if the vcpu should be paused or not. If KVM_CAP_ARM_MP_HALTED is present, +KVM_MP_STATE_HALTED can be set, to wait for interrupts targeted at the vcpu +before running it. 4.40 KVM_SET_IDENTITY_MAP_ADDR ------------------------------ diff --git a/include/uapi/linux/kvm.h b/include/uapi/linux/kvm.h index 79d9c44d1ad7..06ba64c49737 100644 --- a/include/uapi/linux/kvm.h +++ b/include/uapi/linux/kvm.h @@ -1083,6 +1083,7 @@ struct kvm_ppc_resize_hpt { #define KVM_CAP_SGX_ATTRIBUTE 196 #define KVM_CAP_VM_COPY_ENC_CONTEXT_FROM 197 #define KVM_CAP_PTP_KVM 198 +#define KVM_CAP_ARM_MP_HALTED 199 #ifdef KVM_CAP_IRQ_ROUTING diff --git a/arch/arm64/kvm/arm.c b/arch/arm64/kvm/arm.c index d8cbaa0373c7..d6ad977fea5f 100644 --- a/arch/arm64/kvm/arm.c +++ b/arch/arm64/kvm/arm.c @@ -207,6 +207,7 @@ int kvm_vm_ioctl_check_extension(struct kvm *kvm, long ext) case KVM_CAP_SET_GUEST_DEBUG: case KVM_CAP_VCPU_ATTRIBUTES: case KVM_CAP_PTP_KVM: + case KVM_CAP_ARM_MP_HALTED: r = 1; break; case KVM_CAP_SET_GUEST_DEBUG2: @@ -469,6 +470,9 @@ int kvm_arch_vcpu_ioctl_set_mpstate(struct kvm_vcpu *vcpu, case KVM_MP_STATE_RUNNABLE: vcpu->arch.mp_state = KVM_MP_STATE_RUNNABLE; break; + case KVM_MP_STATE_HALTED: + kvm_arm_vcpu_suspend(vcpu); + break; case KVM_MP_STATE_STOPPED: kvm_arm_vcpu_power_off(vcpu); break; @@ -699,7 +703,12 @@ static void check_vcpu_requests(struct kvm_vcpu *vcpu) preempt_enable(); } - if (kvm_check_request(KVM_REQ_SUSPEND, vcpu)) { + /* + * Check mp_state again in case userspace changed their mind + * after requesting suspend. + */ + if (kvm_check_request(KVM_REQ_SUSPEND, vcpu) && + vcpu->arch.mp_state == KVM_MP_STATE_HALTED) { if (!irq_pending) { kvm_vcpu_block(vcpu); kvm_clear_request(KVM_REQ_UNHALT, vcpu); From patchwork Tue Jun 8 15:48:05 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jean-Philippe Brucker X-Patchwork-Id: 12307287 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-18.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED, USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 98EC5C4743E for ; Tue, 8 Jun 2021 15:54:08 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 7217B6136E for ; Tue, 8 Jun 2021 15:54:08 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232487AbhFHP4A (ORCPT ); Tue, 8 Jun 2021 11:56:00 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:50092 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232269AbhFHPz7 (ORCPT ); Tue, 8 Jun 2021 11:55:59 -0400 Received: from mail-wr1-x435.google.com (mail-wr1-x435.google.com [IPv6:2a00:1450:4864:20::435]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 1F40BC06178B for ; Tue, 8 Jun 2021 08:54:06 -0700 (PDT) Received: by mail-wr1-x435.google.com with SMTP id c5so22091542wrq.9 for ; Tue, 08 Jun 2021 08:54:06 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=vY60FWlLV1xpe2gtDWbEapubpchtwsNSBFFHT7yOxe0=; b=qfSlHca66UxSFszZIYf1Ho34VW7NdqKkuU+Pc8aTRoFu2WAU8C7R5vlR511TO3aKYr kroybsX+yhcmbjt2IlLgNnQyKw0ZOnCws+R1zr3ywgOUz849ROejIKxhybmKnv35DELZ RfP8PTeca+tIUXQI0EWgp8V2dYYsKqx3T9LOmH/cg0FmJ+Hu53AIaZ19HmoIEvkat938 DQiEIaOKoBAAGW3iJRpIigw7h6Y6v0/rxv/yzhJE1hUuY96aGWcRtfcaZfTX6QbW/wuT vDqpUUDN/7RRZTZxr/NgRPEWBhBw+zaNf4KTdybDBajcKU32apwOlG3cP00Yvu3r9GpF 09Lw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=vY60FWlLV1xpe2gtDWbEapubpchtwsNSBFFHT7yOxe0=; b=QMPBnq4gmwjr0qwQx2YeJ8HMpYc1FJm/uPUquzJQM3CVx04ULqfX0diUhtqtifY4Q6 2tLl6RC2UBm75JwZsKM8T+d6e5FOoiIjqL9Or+yen1Dl0pd+4wPqfBTH9fvgm3zTPJ2Y Zl1LBqGwYIxLFsKvPyot8aXTBASpyEmUbvw9iM39r/dQjKucwbYH2yyy/IO/RAoasIPr qQih6nfCsHhXPgwT6/gY8PT2e9G56Bqhrm0/NPH7ghj6Kb0vlOCOVfchPBPTdfBCxhub yHo0LEUHVHHAKqTSRcobjdULuNlIeh/M0RTqyGgLoVQlquF7Xuz5kiNtGhMy0yx2h3tA AL4w== X-Gm-Message-State: AOAM531sI9geohwYJlLcP0zTmGtRtzIBUbAIURBYvjw6mp6o6Lp4xzIU tAUv4aU5cat6Z+k9jNMHF7MkaA== X-Google-Smtp-Source: ABdhPJyKyuA8FB6TyDgi8X2Ffbrwu7e7pRxSFAZugb7LawFRVBOmbO7jA2vBPjlJibJuploId8Mjkw== X-Received: by 2002:a5d:6b81:: with SMTP id n1mr23036101wrx.144.1623167644621; Tue, 08 Jun 2021 08:54:04 -0700 (PDT) Received: from localhost.localdomain (adsl-84-226-111-173.adslplus.ch. [84.226.111.173]) by smtp.gmail.com with ESMTPSA id l31sm3314180wms.16.2021.06.08.08.54.03 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 08 Jun 2021 08:54:04 -0700 (PDT) From: Jean-Philippe Brucker To: maz@kernel.org Cc: kvm@vger.kernel.org, kvmarm@lists.cs.columbia.edu, linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, pbonzini@redhat.com, corbet@lwn.net, james.morse@arm.com, alexandru.elisei@arm.com, suzuki.poulose@arm.com, catalin.marinas@arm.com, will@kernel.org, lorenzo.pieralisi@arm.com, salil.mehta@huawei.com, shameerali.kolothum.thodi@huawei.com, jonathan.cameron@huawei.com, Jean-Philippe Brucker Subject: [RFC PATCH 4/5] KVM: arm64: Pass hypercalls to userspace Date: Tue, 8 Jun 2021 17:48:05 +0200 Message-Id: <20210608154805.216869-5-jean-philippe@linaro.org> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20210608154805.216869-1-jean-philippe@linaro.org> References: <20210608154805.216869-1-jean-philippe@linaro.org> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org Let userspace request to handle all hypercalls that aren't handled by KVM, by setting the KVM_CAP_ARM_HVC_TO_USER capability. With the help of another capability, this will allow userspace to handle PSCI calls. Suggested-by: James Morse Signed-off-by: Jean-Philippe Brucker --- Notes on this implementation: * A similar mechanism was proposed for SDEI some time ago [1]. This RFC generalizes the idea to all hypercalls, since that was suggested on the list [2, 3]. * We're reusing kvm_run.hypercall. I copied x0-x5 into kvm_run.hypercall.args[] to help userspace but I'm tempted to remove this, because: - Most user handlers will need to write results back into the registers (x0-x3 for SMCCC), so if we keep this shortcut we should go all the way and synchronize them on return to kernel. - QEMU doesn't care about this shortcut, it pulls all vcpu regs before handling the call. - SMCCC uses x0-x16 for parameters. x0 does contain the SMCCC function ID and may be useful for fast dispatch, we could keep that plus the immediate number. * Should we add a flag in the kvm_run.hypercall telling whether this is HVC or SMC? Can be added later in those bottom longmode and pad fields. * On top of this we could share with userspace which HVC ranges are available and which ones are handled by KVM. That can actually be added independently, through a vCPU/VM device attribute (which doesn't consume a new ioctl): - userspace issues HAS_ATTR ioctl on the VM fd to query whether this feature is available. - userspace queries the number N of HVC ranges using one GET_ATTR. - userspace passes an array of N ranges using another GET_ATTR. The array is filled and returned by KVM. * Untested for AArch32 guests. [1] https://lore.kernel.org/linux-arm-kernel/20170808164616.25949-12-james.morse@arm.com/ [2] https://lore.kernel.org/linux-arm-kernel/bf7e83f1-c58e-8d65-edd0-d08f27b8b766@arm.com/ [3] https://lore.kernel.org/linux-arm-kernel/f56cf420-affc-35f0-2355-801a924b8a35@arm.com/ --- Documentation/virt/kvm/api.rst | 17 +++++++++++++++-- arch/arm64/include/asm/kvm_host.h | 1 + include/kvm/arm_psci.h | 4 ++++ include/uapi/linux/kvm.h | 1 + arch/arm64/kvm/arm.c | 5 +++++ arch/arm64/kvm/hypercalls.c | 28 +++++++++++++++++++++++++++- 6 files changed, 53 insertions(+), 3 deletions(-) diff --git a/Documentation/virt/kvm/api.rst b/Documentation/virt/kvm/api.rst index e4fe7fb60d5d..3d8c1661e7b2 100644 --- a/Documentation/virt/kvm/api.rst +++ b/Documentation/virt/kvm/api.rst @@ -5228,8 +5228,12 @@ to the byte array. __u32 pad; } hypercall; -Unused. This was once used for 'hypercall to userspace'. To implement -such functionality, use KVM_EXIT_IO (x86) or KVM_EXIT_MMIO (all except s390). +On x86 this was once used for 'hypercall to userspace'. To implement such +functionality, use KVM_EXIT_IO (x86) or KVM_EXIT_MMIO (all except s390). + +On arm64 it is used for hypercalls, when the KVM_CAP_ARM_HVC_TO_USER capability +is enabled. 'nr' contains the HVC or SMC immediate. 'args' contains registers +x0 - x5. The other parameters are unused. .. note:: KVM_EXIT_IO is significantly faster than KVM_EXIT_MMIO. @@ -6894,3 +6898,12 @@ This capability is always enabled. This capability indicates that the KVM virtual PTP service is supported in the host. A VMM can check whether the service is available to the guest on migration. + +8.33 KVM_CAP_ARM_HVC_TO_USER +---------------------------- + +:Architecture: arm64 + +This capability indicates that KVM can pass unhandled hypercalls to userspace, +if the VMM enables it. Hypercalls are passed with KVM_EXIT_HYPERCALL in +kvm_run::hypercall. diff --git a/arch/arm64/include/asm/kvm_host.h b/arch/arm64/include/asm/kvm_host.h index 3ca732feb9a5..25554ce97045 100644 --- a/arch/arm64/include/asm/kvm_host.h +++ b/arch/arm64/include/asm/kvm_host.h @@ -123,6 +123,7 @@ struct kvm_arch { * supported. */ bool return_nisv_io_abort_to_user; + bool hvc_to_user; /* * VM-wide PMU filter, implemented as a bitmap and big enough for diff --git a/include/kvm/arm_psci.h b/include/kvm/arm_psci.h index 5b58bd2fe088..d6b71a48fbb1 100644 --- a/include/kvm/arm_psci.h +++ b/include/kvm/arm_psci.h @@ -16,6 +16,10 @@ #define KVM_ARM_PSCI_LATEST KVM_ARM_PSCI_1_0 +#define KVM_PSCI_FN_LAST KVM_PSCI_FN(3) +#define PSCI_0_2_FN_LAST PSCI_0_2_FN(0x3f) +#define PSCI_0_2_FN64_LAST PSCI_0_2_FN64(0x3f) + /* * We need the KVM pointer independently from the vcpu as we can call * this from HYP, and need to apply kern_hyp_va on it... diff --git a/include/uapi/linux/kvm.h b/include/uapi/linux/kvm.h index 06ba64c49737..aa831986a399 100644 --- a/include/uapi/linux/kvm.h +++ b/include/uapi/linux/kvm.h @@ -1084,6 +1084,7 @@ struct kvm_ppc_resize_hpt { #define KVM_CAP_VM_COPY_ENC_CONTEXT_FROM 197 #define KVM_CAP_PTP_KVM 198 #define KVM_CAP_ARM_MP_HALTED 199 +#define KVM_CAP_ARM_HVC_TO_USER 200 #ifdef KVM_CAP_IRQ_ROUTING diff --git a/arch/arm64/kvm/arm.c b/arch/arm64/kvm/arm.c index d6ad977fea5f..074197721e97 100644 --- a/arch/arm64/kvm/arm.c +++ b/arch/arm64/kvm/arm.c @@ -93,6 +93,10 @@ int kvm_vm_ioctl_enable_cap(struct kvm *kvm, r = 0; kvm->arch.return_nisv_io_abort_to_user = true; break; + case KVM_CAP_ARM_HVC_TO_USER: + r = 0; + kvm->arch.hvc_to_user = true; + break; default: r = -EINVAL; break; @@ -208,6 +212,7 @@ int kvm_vm_ioctl_check_extension(struct kvm *kvm, long ext) case KVM_CAP_VCPU_ATTRIBUTES: case KVM_CAP_PTP_KVM: case KVM_CAP_ARM_MP_HALTED: + case KVM_CAP_ARM_HVC_TO_USER: r = 1; break; case KVM_CAP_SET_GUEST_DEBUG2: diff --git a/arch/arm64/kvm/hypercalls.c b/arch/arm64/kvm/hypercalls.c index 30da78f72b3b..ccc2015eddf9 100644 --- a/arch/arm64/kvm/hypercalls.c +++ b/arch/arm64/kvm/hypercalls.c @@ -58,6 +58,28 @@ static void kvm_ptp_get_time(struct kvm_vcpu *vcpu, u64 *val) val[3] = lower_32_bits(cycles); } +static int kvm_hvc_user(struct kvm_vcpu *vcpu) +{ + int i; + struct kvm_run *run = vcpu->run; + + if (!vcpu->kvm->arch.hvc_to_user) { + smccc_set_retval(vcpu, SMCCC_RET_NOT_SUPPORTED, 0, 0, 0); + return 1; + } + + run->exit_reason = KVM_EXIT_HYPERCALL; + run->hypercall.nr = kvm_vcpu_hvc_get_imm(vcpu); + /* Copy the first parameters for fast access */ + for (i = 0; i < 6; i++) + run->hypercall.args[i] = vcpu_get_reg(vcpu, i); + run->hypercall.ret = 0; + run->hypercall.longmode = 0; + run->hypercall.pad = 0; + + return 0; +} + int kvm_hvc_call_handler(struct kvm_vcpu *vcpu) { u32 func_id = smccc_get_function(vcpu); @@ -139,8 +161,12 @@ int kvm_hvc_call_handler(struct kvm_vcpu *vcpu) case ARM_SMCCC_TRNG_RND32: case ARM_SMCCC_TRNG_RND64: return kvm_trng_call(vcpu); - default: + case KVM_PSCI_FN_BASE...KVM_PSCI_FN_LAST: + case PSCI_0_2_FN_BASE...PSCI_0_2_FN_LAST: + case PSCI_0_2_FN64_BASE...PSCI_0_2_FN64_LAST: return kvm_psci_call(vcpu); + default: + return kvm_hvc_user(vcpu); } smccc_set_retval(vcpu, val[0], val[1], val[2], val[3]); From patchwork Tue Jun 8 15:48:06 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jean-Philippe Brucker X-Patchwork-Id: 12307291 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-18.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED, USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 5AEE3C48BCD for ; Tue, 8 Jun 2021 15:55:22 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 3CC1F6135D for ; Tue, 8 Jun 2021 15:55:22 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233142AbhFHP5O (ORCPT ); Tue, 8 Jun 2021 11:57:14 -0400 Received: from mail-wr1-f43.google.com ([209.85.221.43]:39799 "EHLO mail-wr1-f43.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231548AbhFHP5N (ORCPT ); Tue, 8 Jun 2021 11:57:13 -0400 Received: by mail-wr1-f43.google.com with SMTP id l2so22166002wrw.6 for ; Tue, 08 Jun 2021 08:55:07 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=KHEWXTvaTgGt1BVLxIfWQF63jGSa3hhv5Aeen36ISYE=; b=JFN/xiy0BsolYRMuvUu/vBBp8n6h7jDdIzTqFsPh8onZqCMTUkEyfq6+dShb/4jooa 4BfzwPmDLIIGk+APV232oFKjNH8TwCxOn945UJ6HhpeB4bqxxI4sANfxPmyky5JN68ci /AeQ6D6u41JuYGXQIHDrsSjX9YypR07kCATFqSrHzVd6Hna45k6evaAFB/Ta/+knrIbg E9OfDY92p3KvHMlQALtdInffDDrQ4cNn9DMFrpOICKsXVojgNB0T664irg2FH/xP1tkt nFMHjilT2SK06OxmVp0ISWLldj3mSlNHtriZZyZOaHWdkzaZHNlefhrr6r7IKTaPn8iE 1tsQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=KHEWXTvaTgGt1BVLxIfWQF63jGSa3hhv5Aeen36ISYE=; b=lzCiDSCtQFk73t75jpjLQjSErokc/6U1+mnb1Mr/5GnTaoLW1c21aUM9EfEJ7jzCvz ID7XDzjkvWn0DEjLxd8sUE6qTGJrkCNEf1PNLeR+3NC2NfPiHunwHCTRudEJ9WpuwMZp nh2wIomVWQPxoxOYEAOsWif4QhAdirVDGBC7TT41lK8mdbWI5RI3XqmNYkNdAAdLyOrq vQasaTXUoSUttkNhuC4f3oipHz+v5rviRZ2mIGHJRR8sAxZsnJxKeHe+ex1V76onwP3N aIwV4Gs4DLgaXSIdeq7I3WlO1cavqG1n3Hh1Gjs4L/+KqdoJ6+lrRA8mo2fm6f0Jlq04 1YyA== X-Gm-Message-State: AOAM533ZUfpslWLR3e29v+y/B77UaZ+UXt+0VySOhqsoKj6omI6XO7br 6RJ232HkYB7GXosg1cT+izJiRQ== X-Google-Smtp-Source: ABdhPJxIqvImjc8EYF1I8fYhJ8WzhBv5pnbK6ynBNA3UWFEPYAyfdGSMfbYuZieceMItwHj8Wy0EPA== X-Received: by 2002:adf:f94c:: with SMTP id q12mr6581214wrr.417.1623167646586; Tue, 08 Jun 2021 08:54:06 -0700 (PDT) Received: from localhost.localdomain (adsl-84-226-111-173.adslplus.ch. [84.226.111.173]) by smtp.gmail.com with ESMTPSA id l31sm3314180wms.16.2021.06.08.08.54.04 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 08 Jun 2021 08:54:05 -0700 (PDT) From: Jean-Philippe Brucker To: maz@kernel.org Cc: kvm@vger.kernel.org, kvmarm@lists.cs.columbia.edu, linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, pbonzini@redhat.com, corbet@lwn.net, james.morse@arm.com, alexandru.elisei@arm.com, suzuki.poulose@arm.com, catalin.marinas@arm.com, will@kernel.org, lorenzo.pieralisi@arm.com, salil.mehta@huawei.com, shameerali.kolothum.thodi@huawei.com, jonathan.cameron@huawei.com, Jean-Philippe Brucker Subject: [RFC PATCH 5/5] KVM: arm64: Pass PSCI calls to userspace Date: Tue, 8 Jun 2021 17:48:06 +0200 Message-Id: <20210608154805.216869-6-jean-philippe@linaro.org> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20210608154805.216869-1-jean-philippe@linaro.org> References: <20210608154805.216869-1-jean-philippe@linaro.org> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org Let userspace request to handle PSCI calls, by setting the new KVM_CAP_ARM_PSCI_TO_USER capability. SMCCC probe requires PSCI v1.x. If userspace only implements PSCI v0.2, the guest won't query SMCCC support through PSCI and won't use the spectre workarounds. We could hijack PSCI_VERSION and pretend to support v1.0 if userspace does not, then handle all v1.0 calls ourselves (including guessing the PSCI feature set implemented by the guest), but that seems unnecessary. After all the API already allows userspace to force a version lower than v1.0 using the firmware pseudo-registers. The KVM_REG_ARM_PSCI_VERSION pseudo-register currently resets to either v0.1 if userspace doesn't set KVM_ARM_VCPU_PSCI_0_2, or KVM_ARM_PSCI_LATEST (1.0). Suggested-by: James Morse Signed-off-by: Jean-Philippe Brucker --- Documentation/virt/kvm/api.rst | 14 ++++++++++++++ Documentation/virt/kvm/arm/psci.rst | 1 + arch/arm64/include/asm/kvm_host.h | 1 + include/kvm/arm_hypercalls.h | 1 + include/uapi/linux/kvm.h | 1 + arch/arm64/kvm/arm.c | 10 +++++++--- arch/arm64/kvm/hypercalls.c | 2 +- arch/arm64/kvm/psci.c | 13 +++++++++++++ 8 files changed, 39 insertions(+), 4 deletions(-) diff --git a/Documentation/virt/kvm/api.rst b/Documentation/virt/kvm/api.rst index 3d8c1661e7b2..f24eb70e575d 100644 --- a/Documentation/virt/kvm/api.rst +++ b/Documentation/virt/kvm/api.rst @@ -6907,3 +6907,17 @@ available to the guest on migration. This capability indicates that KVM can pass unhandled hypercalls to userspace, if the VMM enables it. Hypercalls are passed with KVM_EXIT_HYPERCALL in kvm_run::hypercall. + +8.34 KVM_CAP_ARM_PSCI_TO_USER +----------------------------- + +:Architectures: arm64 + +When the VMM enables this capability, all PSCI calls are passed to userspace +instead of being handled by KVM. Capability KVM_CAP_ARM_HVC_TO_USER must be +enabled first. + +Userspace should support at least PSCI v1.0. Otherwise SMCCC features won't be +available to the guest. Userspace does not need to handle the SMCCC_VERSION +parameter for the PSCI_FEATURES function. The KVM_ARM_VCPU_PSCI_0_2 vCPU +feature should be set even if this capability is enabled. diff --git a/Documentation/virt/kvm/arm/psci.rst b/Documentation/virt/kvm/arm/psci.rst index d52c2e83b5b8..110011d1fa3f 100644 --- a/Documentation/virt/kvm/arm/psci.rst +++ b/Documentation/virt/kvm/arm/psci.rst @@ -34,6 +34,7 @@ The following register is defined: - Allows any PSCI version implemented by KVM and compatible with v0.2 to be set with SET_ONE_REG - Affects the whole VM (even if the register view is per-vcpu) + - Defaults to PSCI 1.0 if userspace enables KVM_CAP_ARM_PSCI_TO_USER. * KVM_REG_ARM_SMCCC_ARCH_WORKAROUND_1: Holds the state of the firmware support to mitigate CVE-2017-5715, as diff --git a/arch/arm64/include/asm/kvm_host.h b/arch/arm64/include/asm/kvm_host.h index 25554ce97045..5d74b769c16d 100644 --- a/arch/arm64/include/asm/kvm_host.h +++ b/arch/arm64/include/asm/kvm_host.h @@ -124,6 +124,7 @@ struct kvm_arch { */ bool return_nisv_io_abort_to_user; bool hvc_to_user; + bool psci_to_user; /* * VM-wide PMU filter, implemented as a bitmap and big enough for diff --git a/include/kvm/arm_hypercalls.h b/include/kvm/arm_hypercalls.h index 0e2509d27910..b66c6a000ef3 100644 --- a/include/kvm/arm_hypercalls.h +++ b/include/kvm/arm_hypercalls.h @@ -6,6 +6,7 @@ #include +int kvm_hvc_user(struct kvm_vcpu *vcpu); int kvm_hvc_call_handler(struct kvm_vcpu *vcpu); static inline u32 smccc_get_function(struct kvm_vcpu *vcpu) diff --git a/include/uapi/linux/kvm.h b/include/uapi/linux/kvm.h index aa831986a399..2b8e55aa7e1e 100644 --- a/include/uapi/linux/kvm.h +++ b/include/uapi/linux/kvm.h @@ -1085,6 +1085,7 @@ struct kvm_ppc_resize_hpt { #define KVM_CAP_PTP_KVM 198 #define KVM_CAP_ARM_MP_HALTED 199 #define KVM_CAP_ARM_HVC_TO_USER 200 +#define KVM_CAP_ARM_PSCI_TO_USER 201 #ifdef KVM_CAP_IRQ_ROUTING diff --git a/arch/arm64/kvm/arm.c b/arch/arm64/kvm/arm.c index 074197721e97..bc3e63b0b3ad 100644 --- a/arch/arm64/kvm/arm.c +++ b/arch/arm64/kvm/arm.c @@ -83,7 +83,7 @@ int kvm_arch_check_processor_compat(void *opaque) int kvm_vm_ioctl_enable_cap(struct kvm *kvm, struct kvm_enable_cap *cap) { - int r; + int r = -EINVAL; if (cap->flags) return -EINVAL; @@ -97,8 +97,11 @@ int kvm_vm_ioctl_enable_cap(struct kvm *kvm, r = 0; kvm->arch.hvc_to_user = true; break; - default: - r = -EINVAL; + case KVM_CAP_ARM_PSCI_TO_USER: + if (kvm->arch.hvc_to_user) { + r = 0; + kvm->arch.psci_to_user = true; + } break; } @@ -213,6 +216,7 @@ int kvm_vm_ioctl_check_extension(struct kvm *kvm, long ext) case KVM_CAP_PTP_KVM: case KVM_CAP_ARM_MP_HALTED: case KVM_CAP_ARM_HVC_TO_USER: + case KVM_CAP_ARM_PSCI_TO_USER: r = 1; break; case KVM_CAP_SET_GUEST_DEBUG2: diff --git a/arch/arm64/kvm/hypercalls.c b/arch/arm64/kvm/hypercalls.c index ccc2015eddf9..621d5a5b7e48 100644 --- a/arch/arm64/kvm/hypercalls.c +++ b/arch/arm64/kvm/hypercalls.c @@ -58,7 +58,7 @@ static void kvm_ptp_get_time(struct kvm_vcpu *vcpu, u64 *val) val[3] = lower_32_bits(cycles); } -static int kvm_hvc_user(struct kvm_vcpu *vcpu) +int kvm_hvc_user(struct kvm_vcpu *vcpu) { int i; struct kvm_run *run = vcpu->run; diff --git a/arch/arm64/kvm/psci.c b/arch/arm64/kvm/psci.c index 42a307ceb95f..7f44ee527966 100644 --- a/arch/arm64/kvm/psci.c +++ b/arch/arm64/kvm/psci.c @@ -353,6 +353,16 @@ static int kvm_psci_0_1_call(struct kvm_vcpu *vcpu) return 1; } +static bool kvm_psci_call_is_user(struct kvm_vcpu *vcpu) +{ + /* Handle the special case of SMCCC probe through PSCI */ + if (smccc_get_function(vcpu) == PSCI_1_0_FN_PSCI_FEATURES && + smccc_get_arg1(vcpu) == ARM_SMCCC_VERSION_FUNC_ID) + return false; + + return vcpu->kvm->arch.psci_to_user; +} + /** * kvm_psci_call - handle PSCI call if r0 value is in range * @vcpu: Pointer to the VCPU struct @@ -369,6 +379,9 @@ static int kvm_psci_0_1_call(struct kvm_vcpu *vcpu) */ int kvm_psci_call(struct kvm_vcpu *vcpu) { + if (kvm_psci_call_is_user(vcpu)) + return kvm_hvc_user(vcpu); + switch (kvm_psci_version(vcpu, vcpu->kvm)) { case KVM_ARM_PSCI_1_0: return kvm_psci_1_0_call(vcpu);