From patchwork Tue Jun 8 15:48:02 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jean-Philippe Brucker X-Patchwork-Id: 12307293 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.9 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED, USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 3406EC4743E for ; Tue, 8 Jun 2021 15:57:27 +0000 (UTC) Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 05E85610A2 for ; Tue, 8 Jun 2021 15:57:27 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 05E85610A2 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=linaro.org Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Message-Id:Date:Subject:Cc:To:From:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=977rtwF8KBmDIk0Wo90yB4T5gsqI45f+9iCwKF+/aRs=; b=ClFOStc00WKowO nG/zWRSjmc6mW4tlH5V/gwlteduDKzj8K+UpaYxkl2Cnztyn0CisxVxO7zj4Ms6x4DUt70+FF5UEF 2AdvecgxEBr7fKPtkRgBQTLGkLg4vsfTEmY5uBgN5vSU6bgyJwdB32fl9RFKTK5jtEoVRIp/bYTVA JyyAR/U+4uo/ahuNkcYbP6RGLRqCeYnd+XNJV6cwPWtR7oA7ZXVVZm6f/ym9AYAhtHvCZYS6kcLi0 R2ddbJwbWb2lT6p3j8F1kn7hZEWhcosJA/NjXK7vFt8AC3MvAGLkBUiOqChkXjM7EEdONj6O6JdZZ CpnTHvx7DiD/Le4Z8X5g==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1lqe4f-009OWJ-GY; Tue, 08 Jun 2021 15:55:29 +0000 Received: from mail-wr1-f41.google.com ([209.85.221.41]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1lqe4F-009OMv-Gi for linux-arm-kernel@lists.infradead.org; Tue, 08 Jun 2021 15:55:05 +0000 Received: by mail-wr1-f41.google.com with SMTP id c5so22091375wrq.9 for ; Tue, 08 Jun 2021 08:55:02 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=BRXQuaY4B7tLzh+9+26u4SBQDqXVmvyu3lGkiJd9OsE=; b=gr/gglQLIVxWHQCeK/ivVghBcHcoS3AZLiI+39Ob4NoNZW41wlCK9c24hTBSR5ke43 fHeHoYJh3ii5RmoLBw/CW8i9xLQHTkDwnUeagqqOvprPrSo33yOPMjD00O9qQx4variP wAOxxaDwYJuP0L6+4hIE5QrRBHiMb4m7YKyyPoINsi+QANcEjGqYtUh+ociH3y4IBZ8V jAZXSvRXgREbES9TgpI+AdGsYZyWh8glxmBUdFcBPeP71icony0OCulH2ORerU3vtXV8 0rLFCmm2YegiS3GyBPg6CHBjsxJyL2HITbLT27oJfFVg8ODdXSfIPyG0kVPmEW+C96zg /8/g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=BRXQuaY4B7tLzh+9+26u4SBQDqXVmvyu3lGkiJd9OsE=; b=QLIpPGDlclFlvCItOg0d6KYkErj0+9t/CdGK+iZimTO5Zro4MmqAiFPKmCbH/cQM+Q YVmqALB2ZjiE5ZuAQIZCGiyG9WebAq7e6NPdyfTR3kLw9cFsO0ASCJpCHnUL7iK745HY oMmvxLx21/wB2L0vXwKWkzaw8BrA9CnJ3dcS8LRhwH7qrQ6HiRkpzgUS9ynK+LvMT74b fS2nUFc1Wy6BolvkwXFY0cjQwA1z0UcUUlu/vTfvTKOsFanpzDhzn1o4exlfIa1ajrU2 9zB3EavczorJIwFKi1HTt4XMnl5gGp51pgc2TZhYjBicAIPQo1gqNKTI0iReQWuX7ucW QgUA== X-Gm-Message-State: AOAM531BUOA6JuR4bpoueFkFyn8aHPPT1LrXYtje70tS/s7mw4bLmx4T x9NjFJuI2U49eAf2j/fzK2Ob0g== X-Google-Smtp-Source: ABdhPJykr6vORYCQFK0zdalZ1dmo1FvGrQrMtsX1IEM0HdNX3NO+P/oNrCPC+8IGF+lGOb5mFL1IQA== X-Received: by 2002:adf:eac3:: with SMTP id o3mr23276114wrn.157.1623167641696; Tue, 08 Jun 2021 08:54:01 -0700 (PDT) Received: from localhost.localdomain (adsl-84-226-111-173.adslplus.ch. [84.226.111.173]) by smtp.gmail.com with ESMTPSA id l31sm3314180wms.16.2021.06.08.08.54.00 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 08 Jun 2021 08:54:01 -0700 (PDT) From: Jean-Philippe Brucker To: maz@kernel.org Cc: kvm@vger.kernel.org, kvmarm@lists.cs.columbia.edu, linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, pbonzini@redhat.com, corbet@lwn.net, james.morse@arm.com, alexandru.elisei@arm.com, suzuki.poulose@arm.com, catalin.marinas@arm.com, will@kernel.org, lorenzo.pieralisi@arm.com, salil.mehta@huawei.com, shameerali.kolothum.thodi@huawei.com, jonathan.cameron@huawei.com, Jean-Philippe Brucker Subject: [RFC PATCH 1/5] KVM: arm64: Replace power_off with mp_state in struct kvm_vcpu_arch Date: Tue, 8 Jun 2021 17:48:02 +0200 Message-Id: <20210608154805.216869-2-jean-philippe@linaro.org> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20210608154805.216869-1-jean-philippe@linaro.org> References: <20210608154805.216869-1-jean-philippe@linaro.org> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20210608_085503_600441_B2134860 X-CRM114-Status: GOOD ( 17.44 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org In order to add a new "suspend" power state, replace power_off with mp_state in struct kvm_vcpu_arch. Factor the vcpu_off() function while we're here. No functional change intended. Signed-off-by: Jean-Philippe Brucker --- arch/arm64/include/asm/kvm_host.h | 6 ++++-- arch/arm64/kvm/arm.c | 29 +++++++++++++++-------------- arch/arm64/kvm/psci.c | 19 ++++++------------- 3 files changed, 25 insertions(+), 29 deletions(-) diff --git a/arch/arm64/include/asm/kvm_host.h b/arch/arm64/include/asm/kvm_host.h index 7cd7d5c8c4bc..55a04f4d5919 100644 --- a/arch/arm64/include/asm/kvm_host.h +++ b/arch/arm64/include/asm/kvm_host.h @@ -340,8 +340,8 @@ struct kvm_vcpu_arch { u32 mdscr_el1; } guest_debug_preserved; - /* vcpu power-off state */ - bool power_off; + /* vcpu power state (runnable, stopped, halted) */ + u32 mp_state; /* Don't run the guest (internal implementation need) */ bool pause; @@ -720,6 +720,8 @@ int kvm_arm_vcpu_arch_get_attr(struct kvm_vcpu *vcpu, struct kvm_device_attr *attr); int kvm_arm_vcpu_arch_has_attr(struct kvm_vcpu *vcpu, struct kvm_device_attr *attr); +void kvm_arm_vcpu_power_off(struct kvm_vcpu *vcpu); +bool kvm_arm_vcpu_is_off(struct kvm_vcpu *vcpu); /* Guest/host FPSIMD coordination helpers */ int kvm_arch_vcpu_run_map_fp(struct kvm_vcpu *vcpu); diff --git a/arch/arm64/kvm/arm.c b/arch/arm64/kvm/arm.c index e720148232a0..bcc24adb9c0a 100644 --- a/arch/arm64/kvm/arm.c +++ b/arch/arm64/kvm/arm.c @@ -435,21 +435,22 @@ void kvm_arch_vcpu_put(struct kvm_vcpu *vcpu) vcpu->cpu = -1; } -static void vcpu_power_off(struct kvm_vcpu *vcpu) +void kvm_arm_vcpu_power_off(struct kvm_vcpu *vcpu) { - vcpu->arch.power_off = true; + vcpu->arch.mp_state = KVM_MP_STATE_STOPPED; kvm_make_request(KVM_REQ_SLEEP, vcpu); kvm_vcpu_kick(vcpu); } +bool kvm_arm_vcpu_is_off(struct kvm_vcpu *vcpu) +{ + return vcpu->arch.mp_state == KVM_MP_STATE_STOPPED; +} + int kvm_arch_vcpu_ioctl_get_mpstate(struct kvm_vcpu *vcpu, struct kvm_mp_state *mp_state) { - if (vcpu->arch.power_off) - mp_state->mp_state = KVM_MP_STATE_STOPPED; - else - mp_state->mp_state = KVM_MP_STATE_RUNNABLE; - + mp_state->mp_state = vcpu->arch.mp_state; return 0; } @@ -460,10 +461,10 @@ int kvm_arch_vcpu_ioctl_set_mpstate(struct kvm_vcpu *vcpu, switch (mp_state->mp_state) { case KVM_MP_STATE_RUNNABLE: - vcpu->arch.power_off = false; + vcpu->arch.mp_state = KVM_MP_STATE_RUNNABLE; break; case KVM_MP_STATE_STOPPED: - vcpu_power_off(vcpu); + kvm_arm_vcpu_power_off(vcpu); break; default: ret = -EINVAL; @@ -483,7 +484,7 @@ int kvm_arch_vcpu_runnable(struct kvm_vcpu *v) { bool irq_lines = *vcpu_hcr(v) & (HCR_VI | HCR_VF); return ((irq_lines || kvm_vgic_vcpu_pending_irq(v)) - && !v->arch.power_off && !v->arch.pause); + && !kvm_arm_vcpu_is_off(v) && !v->arch.pause); } bool kvm_arch_vcpu_in_kernel(struct kvm_vcpu *vcpu) @@ -643,10 +644,10 @@ static void vcpu_req_sleep(struct kvm_vcpu *vcpu) struct rcuwait *wait = kvm_arch_vcpu_get_wait(vcpu); rcuwait_wait_event(wait, - (!vcpu->arch.power_off) &&(!vcpu->arch.pause), + !kvm_arm_vcpu_is_off(vcpu) && !vcpu->arch.pause, TASK_INTERRUPTIBLE); - if (vcpu->arch.power_off || vcpu->arch.pause) { + if (kvm_arm_vcpu_is_off(vcpu) || vcpu->arch.pause) { /* Awaken to handle a signal, request we sleep again later. */ kvm_make_request(KVM_REQ_SLEEP, vcpu); } @@ -1087,9 +1088,9 @@ static int kvm_arch_vcpu_ioctl_vcpu_init(struct kvm_vcpu *vcpu, * Handle the "start in power-off" case. */ if (test_bit(KVM_ARM_VCPU_POWER_OFF, vcpu->arch.features)) - vcpu_power_off(vcpu); + kvm_arm_vcpu_power_off(vcpu); else - vcpu->arch.power_off = false; + vcpu->arch.mp_state = KVM_MP_STATE_RUNNABLE; return 0; } diff --git a/arch/arm64/kvm/psci.c b/arch/arm64/kvm/psci.c index db4056ecccfd..24b4a2265dbd 100644 --- a/arch/arm64/kvm/psci.c +++ b/arch/arm64/kvm/psci.c @@ -52,13 +52,6 @@ static unsigned long kvm_psci_vcpu_suspend(struct kvm_vcpu *vcpu) return PSCI_RET_SUCCESS; } -static void kvm_psci_vcpu_off(struct kvm_vcpu *vcpu) -{ - vcpu->arch.power_off = true; - kvm_make_request(KVM_REQ_SLEEP, vcpu); - kvm_vcpu_kick(vcpu); -} - static unsigned long kvm_psci_vcpu_on(struct kvm_vcpu *source_vcpu) { struct vcpu_reset_state *reset_state; @@ -78,7 +71,7 @@ static unsigned long kvm_psci_vcpu_on(struct kvm_vcpu *source_vcpu) */ if (!vcpu) return PSCI_RET_INVALID_PARAMS; - if (!vcpu->arch.power_off) { + if (!kvm_arm_vcpu_is_off(vcpu)) { if (kvm_psci_version(source_vcpu, kvm) != KVM_ARM_PSCI_0_1) return PSCI_RET_ALREADY_ON; else @@ -107,7 +100,7 @@ static unsigned long kvm_psci_vcpu_on(struct kvm_vcpu *source_vcpu) */ smp_wmb(); - vcpu->arch.power_off = false; + vcpu->arch.mp_state = KVM_MP_STATE_RUNNABLE; kvm_vcpu_wake_up(vcpu); return PSCI_RET_SUCCESS; @@ -142,7 +135,7 @@ static unsigned long kvm_psci_vcpu_affinity_info(struct kvm_vcpu *vcpu) mpidr = kvm_vcpu_get_mpidr_aff(tmp); if ((mpidr & target_affinity_mask) == target_affinity) { matching_cpus++; - if (!tmp->arch.power_off) + if (!kvm_arm_vcpu_is_off(tmp)) return PSCI_0_2_AFFINITY_LEVEL_ON; } } @@ -168,7 +161,7 @@ static void kvm_prepare_system_event(struct kvm_vcpu *vcpu, u32 type) * re-initialized. */ kvm_for_each_vcpu(i, tmp, vcpu->kvm) - tmp->arch.power_off = true; + tmp->arch.mp_state = KVM_MP_STATE_STOPPED; kvm_make_all_cpus_request(vcpu->kvm, KVM_REQ_SLEEP); memset(&vcpu->run->system_event, 0, sizeof(vcpu->run->system_event)); @@ -237,7 +230,7 @@ static int kvm_psci_0_2_call(struct kvm_vcpu *vcpu) val = kvm_psci_vcpu_suspend(vcpu); break; case PSCI_0_2_FN_CPU_OFF: - kvm_psci_vcpu_off(vcpu); + kvm_arm_vcpu_power_off(vcpu); val = PSCI_RET_SUCCESS; break; case PSCI_0_2_FN_CPU_ON: @@ -350,7 +343,7 @@ static int kvm_psci_0_1_call(struct kvm_vcpu *vcpu) switch (psci_fn) { case KVM_PSCI_FN_CPU_OFF: - kvm_psci_vcpu_off(vcpu); + kvm_arm_vcpu_power_off(vcpu); val = PSCI_RET_SUCCESS; break; case KVM_PSCI_FN_CPU_ON: From patchwork Tue Jun 8 15:48:03 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jean-Philippe Brucker X-Patchwork-Id: 12307277 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.9 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED, USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 934E0C4743F for ; Tue, 8 Jun 2021 15:55:50 +0000 (UTC) Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 659CC6134F for ; Tue, 8 Jun 2021 15:55:50 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 659CC6134F Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=linaro.org Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Message-Id:Date:Subject:Cc:To:From:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=YrrMgfX17lyE7nGAZKmo0Lg331oBTxQLqr/fFEsZx0E=; b=gBtvNpHf01xlK1 R+9nBsha5s9obRXPI5SnSyzib8WcYksuwZ0ReifWPIwoRuFAAHhgD91z8bOvjBDqeQG0KZZEhswa/ aZYGaIn1BWX/vrIGkM6kOY3U6pB98cREAEi4P4+yVpv5C6b/w9AvpM/cw/oV7PIF5rpg7ciF+WuOJ 82l5bz5OfiY0tW43shgVeZmXxxFsstEVIo2bHKGKicvSCXi9uFM1CTYag3U0Qzok52Dbg8543zuEh Q3YvYq3tjoAWug8jEFakncNfewCZHRkiWwFTDR+Zf8OzFa8DzR4Kvi7ZkuOnpZ/8kZemulxgwlM9x rkAaEnuV0/laUjIgDGdA==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1lqe3P-009O9E-AG; Tue, 08 Jun 2021 15:54:11 +0000 Received: from mail-wr1-x433.google.com ([2a00:1450:4864:20::433]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1lqe3K-009O7F-23 for linux-arm-kernel@lists.infradead.org; Tue, 08 Jun 2021 15:54:07 +0000 Received: by mail-wr1-x433.google.com with SMTP id r9so5434938wrz.10 for ; Tue, 08 Jun 2021 08:54:03 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=luZ2XBy4kIjKsUNN5YLr1hr2hYOSweOt5shsxweZY5c=; b=sbCxjbeI/oicVowr1ScpDXLBsVThc8WALV+maymt0+dWYJi+SdK1EjS3CBamvobctn oVj9jPoiFl+XFyqVbz7oy3/wgxvfgv4TUhXNsZtIJYnlHpvoajG5yUBb4YDZeadnQyZ6 FVMM9S8Byjv7mzoU+5SISyAl31bOpdFIzZ//vvWfgeVaD+uvShhJnQfPo+J+Wj1q9caq U3Hk1jw15VgUv/U1tleMfSt+xrJk0XKUTYZF7Qb+jytlId0YYKfFcJnS///J517/MU38 hcdfZin3+XvAGgVk0iRCkX8FCL1xBqvocAV1qiayCXWYGUiFRbLXitgw6SGtGQZndgP9 UYFg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=luZ2XBy4kIjKsUNN5YLr1hr2hYOSweOt5shsxweZY5c=; b=Kufe9bDia1DvGkMb6F9+Sce4+iSCYHTcbkPMRyOpBgUaXHzeWKd9g73oPWxHBC8L7K hZEXy4p3o2v8JR0t5pZy4M0GsqFoearkx2VFpM8jyYvdwKyqiP7iDo8m/yExulVXpsqU xuWicRrFiYXTZeNVQl60aAXVcHgWajkJ5gVi+LqsXyW3tI4wK7VbmAcHYvsXU1q0QIq2 29FwRchrSQlKDRd9d730cP8gDk1TTKeFJ4KbQLOtRtEASZER7hIrsjWq8DhCdKlyfpR2 ywbseSba2abb27s2wA6ZcmYiCI+OhMmTP8tMQNmkVydnnGdGnL+scCmZzhzxo5NfH7kQ iMdg== X-Gm-Message-State: AOAM530HcWwgM/u9Ml3gxH3xYu00Mdvwvph2lZA6GZ6Prl6rLjGiL97T hIfyWzx82Tp9uMWrk1ZUkQJTmQ== X-Google-Smtp-Source: ABdhPJzkPP5/N5I0QEPYl+or03a+RJK+q3fbfrxy8aomfkBbcwKtQQDp8ZjdsnAAKwnqh5+OZC4YyA== X-Received: by 2002:a5d:4a4b:: with SMTP id v11mr23583559wrs.246.1623167642648; Tue, 08 Jun 2021 08:54:02 -0700 (PDT) Received: from localhost.localdomain (adsl-84-226-111-173.adslplus.ch. [84.226.111.173]) by smtp.gmail.com with ESMTPSA id l31sm3314180wms.16.2021.06.08.08.54.01 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 08 Jun 2021 08:54:02 -0700 (PDT) From: Jean-Philippe Brucker To: maz@kernel.org Cc: kvm@vger.kernel.org, kvmarm@lists.cs.columbia.edu, linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, pbonzini@redhat.com, corbet@lwn.net, james.morse@arm.com, alexandru.elisei@arm.com, suzuki.poulose@arm.com, catalin.marinas@arm.com, will@kernel.org, lorenzo.pieralisi@arm.com, salil.mehta@huawei.com, shameerali.kolothum.thodi@huawei.com, jonathan.cameron@huawei.com, Jean-Philippe Brucker Subject: [RFC PATCH 2/5] KVM: arm64: Move WFI execution to check_vcpu_requests() Date: Tue, 8 Jun 2021 17:48:03 +0200 Message-Id: <20210608154805.216869-3-jean-philippe@linaro.org> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20210608154805.216869-1-jean-philippe@linaro.org> References: <20210608154805.216869-1-jean-philippe@linaro.org> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20210608_085406_125936_264F661B X-CRM114-Status: GOOD ( 21.98 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Prepare for WFI requests from userspace, by adding a suspend request and moving the WFI execution into check_vcpu_requests(), next to the power-off logic. vcpu->arch.mp_state, previously only RUNNABLE or STOPPED, supports an additional state HALTED and two new state transitions: RUNNABLE -> HALTED from WFI or PSCI CPU_SUSPEND (same vCPU) HALTED -> RUNNABLE vGIC IRQ, pending timer, signal There shouldn't be any functional change with this patch, even though the KVM_GET_MP_STATE ioctl could now in theory return KVM_MP_STATE_HALTED, which would break some users' mp_state support. In practice it should not happen because we do not return to userspace with HALTED state. Both WFI and PSCI CPU_SUSPEND stay in the vCPU run loop until the suspend request is consumed. It does feel fragile though, maybe we should explicitly return RUNNABLE in KVM_GET_MP_STATE in place of HALTED, to prevent future breakage. Signed-off-by: Jean-Philippe Brucker --- arch/arm64/include/asm/kvm_host.h | 2 ++ arch/arm64/kvm/arm.c | 18 ++++++++++++++- arch/arm64/kvm/handle_exit.c | 3 +-- arch/arm64/kvm/psci.c | 37 +++++++++++++------------------ 4 files changed, 35 insertions(+), 25 deletions(-) diff --git a/arch/arm64/include/asm/kvm_host.h b/arch/arm64/include/asm/kvm_host.h index 55a04f4d5919..3ca732feb9a5 100644 --- a/arch/arm64/include/asm/kvm_host.h +++ b/arch/arm64/include/asm/kvm_host.h @@ -46,6 +46,7 @@ #define KVM_REQ_VCPU_RESET KVM_ARCH_REQ(2) #define KVM_REQ_RECORD_STEAL KVM_ARCH_REQ(3) #define KVM_REQ_RELOAD_GICv4 KVM_ARCH_REQ(4) +#define KVM_REQ_SUSPEND KVM_ARCH_REQ(5) #define KVM_DIRTY_LOG_MANUAL_CAPS (KVM_DIRTY_LOG_MANUAL_PROTECT_ENABLE | \ KVM_DIRTY_LOG_INITIALLY_SET) @@ -722,6 +723,7 @@ int kvm_arm_vcpu_arch_has_attr(struct kvm_vcpu *vcpu, struct kvm_device_attr *attr); void kvm_arm_vcpu_power_off(struct kvm_vcpu *vcpu); bool kvm_arm_vcpu_is_off(struct kvm_vcpu *vcpu); +void kvm_arm_vcpu_suspend(struct kvm_vcpu *vcpu); /* Guest/host FPSIMD coordination helpers */ int kvm_arch_vcpu_run_map_fp(struct kvm_vcpu *vcpu); diff --git a/arch/arm64/kvm/arm.c b/arch/arm64/kvm/arm.c index bcc24adb9c0a..d8cbaa0373c7 100644 --- a/arch/arm64/kvm/arm.c +++ b/arch/arm64/kvm/arm.c @@ -447,6 +447,12 @@ bool kvm_arm_vcpu_is_off(struct kvm_vcpu *vcpu) return vcpu->arch.mp_state == KVM_MP_STATE_STOPPED; } +void kvm_arm_vcpu_suspend(struct kvm_vcpu *vcpu) +{ + vcpu->arch.mp_state = KVM_MP_STATE_HALTED; + kvm_make_request(KVM_REQ_SUSPEND, vcpu); +} + int kvm_arch_vcpu_ioctl_get_mpstate(struct kvm_vcpu *vcpu, struct kvm_mp_state *mp_state) { @@ -667,6 +673,8 @@ static int kvm_vcpu_initialized(struct kvm_vcpu *vcpu) static void check_vcpu_requests(struct kvm_vcpu *vcpu) { + bool irq_pending; + if (kvm_request_pending(vcpu)) { if (kvm_check_request(KVM_REQ_SLEEP, vcpu)) vcpu_req_sleep(vcpu); @@ -678,7 +686,7 @@ static void check_vcpu_requests(struct kvm_vcpu *vcpu) * Clear IRQ_PENDING requests that were made to guarantee * that a VCPU sees new virtual interrupts. */ - kvm_check_request(KVM_REQ_IRQ_PENDING, vcpu); + irq_pending = kvm_check_request(KVM_REQ_IRQ_PENDING, vcpu); if (kvm_check_request(KVM_REQ_RECORD_STEAL, vcpu)) kvm_update_stolen_time(vcpu); @@ -690,6 +698,14 @@ static void check_vcpu_requests(struct kvm_vcpu *vcpu) vgic_v4_load(vcpu); preempt_enable(); } + + if (kvm_check_request(KVM_REQ_SUSPEND, vcpu)) { + if (!irq_pending) { + kvm_vcpu_block(vcpu); + kvm_clear_request(KVM_REQ_UNHALT, vcpu); + } + vcpu->arch.mp_state = KVM_MP_STATE_RUNNABLE; + } } } diff --git a/arch/arm64/kvm/handle_exit.c b/arch/arm64/kvm/handle_exit.c index 6f48336b1d86..9717df3104cf 100644 --- a/arch/arm64/kvm/handle_exit.c +++ b/arch/arm64/kvm/handle_exit.c @@ -95,8 +95,7 @@ static int kvm_handle_wfx(struct kvm_vcpu *vcpu) } else { trace_kvm_wfx_arm64(*vcpu_pc(vcpu), false); vcpu->stat.wfi_exit_stat++; - kvm_vcpu_block(vcpu); - kvm_clear_request(KVM_REQ_UNHALT, vcpu); + kvm_arm_vcpu_suspend(vcpu); } kvm_incr_pc(vcpu); diff --git a/arch/arm64/kvm/psci.c b/arch/arm64/kvm/psci.c index 24b4a2265dbd..42a307ceb95f 100644 --- a/arch/arm64/kvm/psci.c +++ b/arch/arm64/kvm/psci.c @@ -31,27 +31,6 @@ static unsigned long psci_affinity_mask(unsigned long affinity_level) return 0; } -static unsigned long kvm_psci_vcpu_suspend(struct kvm_vcpu *vcpu) -{ - /* - * NOTE: For simplicity, we make VCPU suspend emulation to be - * same-as WFI (Wait-for-interrupt) emulation. - * - * This means for KVM the wakeup events are interrupts and - * this is consistent with intended use of StateID as described - * in section 5.4.1 of PSCI v0.2 specification (ARM DEN 0022A). - * - * Further, we also treat power-down request to be same as - * stand-by request as-per section 5.4.2 clause 3 of PSCI v0.2 - * specification (ARM DEN 0022A). This means all suspend states - * for KVM will preserve the register state. - */ - kvm_vcpu_block(vcpu); - kvm_clear_request(KVM_REQ_UNHALT, vcpu); - - return PSCI_RET_SUCCESS; -} - static unsigned long kvm_psci_vcpu_on(struct kvm_vcpu *source_vcpu) { struct vcpu_reset_state *reset_state; @@ -227,7 +206,21 @@ static int kvm_psci_0_2_call(struct kvm_vcpu *vcpu) break; case PSCI_0_2_FN_CPU_SUSPEND: case PSCI_0_2_FN64_CPU_SUSPEND: - val = kvm_psci_vcpu_suspend(vcpu); + /* + * NOTE: For simplicity, we make VCPU suspend emulation to be + * same-as WFI (Wait-for-interrupt) emulation. + * + * This means for KVM the wakeup events are interrupts and this + * is consistent with intended use of StateID as described in + * section 5.4.1 of PSCI v0.2 specification (ARM DEN 0022A). + * + * Further, we also treat power-down request to be same as + * stand-by request as-per section 5.4.2 clause 3 of PSCI v0.2 + * specification (ARM DEN 0022A). This means all suspend states + * for KVM will preserve the register state. + */ + kvm_arm_vcpu_suspend(vcpu); + val = PSCI_RET_SUCCESS; break; case PSCI_0_2_FN_CPU_OFF: kvm_arm_vcpu_power_off(vcpu); From patchwork Tue Jun 8 15:48:04 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jean-Philippe Brucker X-Patchwork-Id: 12307295 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.9 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED, USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 52ABFC47082 for ; Tue, 8 Jun 2021 15:57:53 +0000 (UTC) Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 20669610A2 for ; Tue, 8 Jun 2021 15:57:53 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 20669610A2 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=linaro.org Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Message-Id:Date:Subject:Cc:To:From:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=K0ZvRjp9flhLLSGXiBcVKdJ7v8p3W5vVS59fOAQB0oA=; b=c40W9fmoyX8ARY xY+ActvyDPZH9HZV1Z6yTV6kNcupoPMsTDJIi+rAnkz0I6rRuQw0jlWUuVx6PCGUUpRMeYAeGNDf8 OkuAlrOftSADxNl4C/T0FNP0Al0OPj2ydslKKOA2f5pFwvZTHimYIgPHwLutchg1X1MdWu5oq1C2j EPwn5AMlXCSSpyl8SU2NiQWepwCNUPgluFLBHxvfWd8KmWk90YC8HtMWGMULKiBtuQ0R0mrsG3YCc EEkdLtddsKp/YSOmM3x5QPQ1K0j8sNYGncy8yJ4l8HZZT/ZSTTuI+59JEuVw3OiXenOsytWPqMs85 fI1YGNaWPDnhcpkz2yOw==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1lqe4u-009OZW-F7; Tue, 08 Jun 2021 15:55:45 +0000 Received: from mail-wr1-f46.google.com ([209.85.221.46]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1lqe4H-009ONl-DL for linux-arm-kernel@lists.infradead.org; Tue, 08 Jun 2021 15:55:07 +0000 Received: by mail-wr1-f46.google.com with SMTP id e11so11936680wrg.3 for ; Tue, 08 Jun 2021 08:55:04 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=6VUTi6r0zGjvD+ox5LbEVdcu7SOJ3CxYqosX/6quRvc=; b=iZxQMnxOlNisd6FMWcutFJg9ihWBRrP18+4KoCR7isLbBgJRjV2h9bm7ro5tM0SuA5 MKA963p2Fjmaerhs0VLNc3mXjmVJbt4LXKOds3/hhLBfPZTNhM4ALrICmh9wRHLm3mDj 9X0bwod0yko2OzqVbHqT6E91S1tWi3hL1+YyfpA/AefIXhN0izWgnv2m4AsrQM4gTRFb XPGRIVmgyNb6TPkqf5zH5+OpFuMLdvJItw67EM2UPkwxqiE40pkNyBxiSgrBs7QNMC1Q UqgLxRznUGNuKj9K/rYCy6GTXUiiCElOBKA3oqupM1W8sLyaqix9NLGflqG3lQaM8Z05 rHZg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=6VUTi6r0zGjvD+ox5LbEVdcu7SOJ3CxYqosX/6quRvc=; b=AZUJKC2S9vpLkmRGBRE0O4+S+zbQHDncJtkhi7wNVoWaRwxr5ePDinqCSt945uzgLM qMOyhtZ7v4SQ1GpA1Z4bmoPoNsBM8331xvXaF/yaIQpJXkbY0i3MmJ5DpVYZ9ftM2ZOT K84eRlOVkyCZ3gOtWvs+3iKXPdacowfSstFHSGDRcS20xxymkMru5ijB1yXvdwebsevv oXezcPdjSOxxy8rNNCwQdkcxmHdt85rVwdY4JfY2gSrvhXbZAl93ZtDmCAdD3V88HIOs uDNino3qrK03NBwDamrujam25WYfNbHGg8jmv4Npi3jl3fmI5NPDgFi7WV5ugbD8DzaG Rmzg== X-Gm-Message-State: AOAM533tQlaXqH8IJwzx5xHy1C/p2MiMJ0cRQMJ0N33ijMxWOsJN8cAI ozZTkGF9QNtiMqVCxqaEb24Ynw== X-Google-Smtp-Source: ABdhPJy61JSG9YYU4eU/zql91JN1YArLPTW12IIWCRmtkz+g9RAUJUbtnqHF/W7eyUKs8LNMBHfbCw== X-Received: by 2002:adf:e50b:: with SMTP id j11mr22681729wrm.377.1623167643627; Tue, 08 Jun 2021 08:54:03 -0700 (PDT) Received: from localhost.localdomain (adsl-84-226-111-173.adslplus.ch. [84.226.111.173]) by smtp.gmail.com with ESMTPSA id l31sm3314180wms.16.2021.06.08.08.54.02 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 08 Jun 2021 08:54:03 -0700 (PDT) From: Jean-Philippe Brucker To: maz@kernel.org Cc: kvm@vger.kernel.org, kvmarm@lists.cs.columbia.edu, linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, pbonzini@redhat.com, corbet@lwn.net, james.morse@arm.com, alexandru.elisei@arm.com, suzuki.poulose@arm.com, catalin.marinas@arm.com, will@kernel.org, lorenzo.pieralisi@arm.com, salil.mehta@huawei.com, shameerali.kolothum.thodi@huawei.com, jonathan.cameron@huawei.com, Jean-Philippe Brucker Subject: [RFC PATCH 3/5] KVM: arm64: Allow userspace to request WFI Date: Tue, 8 Jun 2021 17:48:04 +0200 Message-Id: <20210608154805.216869-4-jean-philippe@linaro.org> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20210608154805.216869-1-jean-philippe@linaro.org> References: <20210608154805.216869-1-jean-philippe@linaro.org> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20210608_085505_480849_63376F3C X-CRM114-Status: GOOD ( 16.49 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org To help userspace implement PSCI CPU_SUSPEND, allow setting the "HALTED" MP state to request a WFI before returning to the guest. Userspace won't obtain a HALTED mp_state from a KVM_GET_MP_STATE call unless they set it themselves. When set by KVM, to handle wfi or CPU_SUSPEND, it is consumed before returning to userspace. Signed-off-by: Jean-Philippe Brucker --- Documentation/virt/kvm/api.rst | 15 +++++++++------ include/uapi/linux/kvm.h | 1 + arch/arm64/kvm/arm.c | 11 ++++++++++- 3 files changed, 20 insertions(+), 7 deletions(-) diff --git a/Documentation/virt/kvm/api.rst b/Documentation/virt/kvm/api.rst index 7fcb2fd38f42..e4fe7fb60d5d 100644 --- a/Documentation/virt/kvm/api.rst +++ b/Documentation/virt/kvm/api.rst @@ -1416,8 +1416,8 @@ Possible values are: which has not yet received an INIT signal [x86] KVM_MP_STATE_INIT_RECEIVED the vcpu has received an INIT signal, and is now ready for a SIPI [x86] - KVM_MP_STATE_HALTED the vcpu has executed a HLT instruction and - is waiting for an interrupt [x86] + KVM_MP_STATE_HALTED the vcpu has executed a HLT/WFI instruction + and is waiting for an interrupt [x86,arm64] KVM_MP_STATE_SIPI_RECEIVED the vcpu has just received a SIPI (vector accessible via KVM_GET_VCPU_EVENTS) [x86] KVM_MP_STATE_STOPPED the vcpu is stopped [s390,arm/arm64] @@ -1435,8 +1435,9 @@ these architectures. For arm/arm64: ^^^^^^^^^^^^^^ -The only states that are valid are KVM_MP_STATE_STOPPED and -KVM_MP_STATE_RUNNABLE which reflect if the vcpu is paused or not. +Valid states are KVM_MP_STATE_STOPPED and KVM_MP_STATE_RUNNABLE which reflect +if the vcpu is paused or not. If KVM_CAP_ARM_MP_HALTED is present, state +KVM_MP_STATE_HALTED is also valid. 4.39 KVM_SET_MP_STATE --------------------- @@ -1457,8 +1458,10 @@ these architectures. For arm/arm64: ^^^^^^^^^^^^^^ -The only states that are valid are KVM_MP_STATE_STOPPED and -KVM_MP_STATE_RUNNABLE which reflect if the vcpu should be paused or not. +Valid states are KVM_MP_STATE_STOPPED and KVM_MP_STATE_RUNNABLE which reflect +if the vcpu should be paused or not. If KVM_CAP_ARM_MP_HALTED is present, +KVM_MP_STATE_HALTED can be set, to wait for interrupts targeted at the vcpu +before running it. 4.40 KVM_SET_IDENTITY_MAP_ADDR ------------------------------ diff --git a/include/uapi/linux/kvm.h b/include/uapi/linux/kvm.h index 79d9c44d1ad7..06ba64c49737 100644 --- a/include/uapi/linux/kvm.h +++ b/include/uapi/linux/kvm.h @@ -1083,6 +1083,7 @@ struct kvm_ppc_resize_hpt { #define KVM_CAP_SGX_ATTRIBUTE 196 #define KVM_CAP_VM_COPY_ENC_CONTEXT_FROM 197 #define KVM_CAP_PTP_KVM 198 +#define KVM_CAP_ARM_MP_HALTED 199 #ifdef KVM_CAP_IRQ_ROUTING diff --git a/arch/arm64/kvm/arm.c b/arch/arm64/kvm/arm.c index d8cbaa0373c7..d6ad977fea5f 100644 --- a/arch/arm64/kvm/arm.c +++ b/arch/arm64/kvm/arm.c @@ -207,6 +207,7 @@ int kvm_vm_ioctl_check_extension(struct kvm *kvm, long ext) case KVM_CAP_SET_GUEST_DEBUG: case KVM_CAP_VCPU_ATTRIBUTES: case KVM_CAP_PTP_KVM: + case KVM_CAP_ARM_MP_HALTED: r = 1; break; case KVM_CAP_SET_GUEST_DEBUG2: @@ -469,6 +470,9 @@ int kvm_arch_vcpu_ioctl_set_mpstate(struct kvm_vcpu *vcpu, case KVM_MP_STATE_RUNNABLE: vcpu->arch.mp_state = KVM_MP_STATE_RUNNABLE; break; + case KVM_MP_STATE_HALTED: + kvm_arm_vcpu_suspend(vcpu); + break; case KVM_MP_STATE_STOPPED: kvm_arm_vcpu_power_off(vcpu); break; @@ -699,7 +703,12 @@ static void check_vcpu_requests(struct kvm_vcpu *vcpu) preempt_enable(); } - if (kvm_check_request(KVM_REQ_SUSPEND, vcpu)) { + /* + * Check mp_state again in case userspace changed their mind + * after requesting suspend. + */ + if (kvm_check_request(KVM_REQ_SUSPEND, vcpu) && + vcpu->arch.mp_state == KVM_MP_STATE_HALTED) { if (!irq_pending) { kvm_vcpu_block(vcpu); kvm_clear_request(KVM_REQ_UNHALT, vcpu); From patchwork Tue Jun 8 15:48:05 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jean-Philippe Brucker X-Patchwork-Id: 12307297 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.9 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED, USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 6E8ECC47082 for ; Tue, 8 Jun 2021 15:58:20 +0000 (UTC) Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 371E360FEA for ; Tue, 8 Jun 2021 15:58:20 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 371E360FEA Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=linaro.org Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Message-Id:Date:Subject:Cc:To:From:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=w15M/oJGVC9nuK0yUVB3T6UQ1gJZwhq2bt7o6Cw9dQ0=; b=H/9PtQzcUCYO8J Lj0LOidOwnG3j38AwB8mHpHfJ1ne7awKa2SLgQVPkIU4W+OaAPaJV9+tTUr8EDsTDY1ZgN2b0/ZoT e8F92e77fOYxti/LsLWfWF0YThzl/QUsquiNfSM0M8Xlvk+XorlwK31jq6MnW2IW01etf7OfxVe+O k/rRJbHc1/WLUDhYcahsJsOE6FIc17jIV6UjjB7ykO0K2979scaBM1JuKcFAdV59CUzXPJfe6cZRC wjHUUmYYD2Q2uWKH2CMcprIo1CBjTfPIH4/3rncC54+2ARVzPK5GqzQ1m1B6MoG2p2p/ttO/7HRk9 5fTDdqdOiZK5jkyZnung==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1lqe5S-009Oj4-4E; Tue, 08 Jun 2021 15:56:19 +0000 Received: from mail-wr1-f42.google.com ([209.85.221.42]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1lqe4I-009OOF-H9 for linux-arm-kernel@lists.infradead.org; Tue, 08 Jun 2021 15:55:09 +0000 Received: by mail-wr1-f42.google.com with SMTP id y7so17499083wrh.7 for ; Tue, 08 Jun 2021 08:55:05 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=vY60FWlLV1xpe2gtDWbEapubpchtwsNSBFFHT7yOxe0=; b=qfSlHca66UxSFszZIYf1Ho34VW7NdqKkuU+Pc8aTRoFu2WAU8C7R5vlR511TO3aKYr kroybsX+yhcmbjt2IlLgNnQyKw0ZOnCws+R1zr3ywgOUz849ROejIKxhybmKnv35DELZ RfP8PTeca+tIUXQI0EWgp8V2dYYsKqx3T9LOmH/cg0FmJ+Hu53AIaZ19HmoIEvkat938 DQiEIaOKoBAAGW3iJRpIigw7h6Y6v0/rxv/yzhJE1hUuY96aGWcRtfcaZfTX6QbW/wuT vDqpUUDN/7RRZTZxr/NgRPEWBhBw+zaNf4KTdybDBajcKU32apwOlG3cP00Yvu3r9GpF 09Lw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=vY60FWlLV1xpe2gtDWbEapubpchtwsNSBFFHT7yOxe0=; b=pepp3oihfiMVKoGWuHzUzSmgPaufRFotXGNBqChLyhN96n2SISIedV1AJ4SdbYZwD6 chpVeAhw+DB9CmN3TBJ+0njodXeyNBvOUd7HclJeazacrRkEhe8O1VM2DdJgWhrTVss5 WDCnt86pfm+h8uPo4G7ZHAnneMT0DzVPXQ4UdztpUoHxdGbtT6gKiy+BENgdLeSlN3Mh Aovb/KPAeRHAwMxnBfX781ykLTeXdLd05jzInJaW4xagCZ0VHHjyeuSGxh0y8GeH8YOw Ooo6kNxsp0UetUy3rOqPWOX0uw8+PWFQ2FR/uD46AwgaRjkEEe1d33pfzEG3V3gtv7L0 nwUA== X-Gm-Message-State: AOAM532NoiXXNigCjks6cWfulXyVYbUpqbeXa0Hg/rCs2QS6DBdilAGP kDhTBkXgkVscO5c8GR6BWjINrg== X-Google-Smtp-Source: ABdhPJyKyuA8FB6TyDgi8X2Ffbrwu7e7pRxSFAZugb7LawFRVBOmbO7jA2vBPjlJibJuploId8Mjkw== X-Received: by 2002:a5d:6b81:: with SMTP id n1mr23036101wrx.144.1623167644621; Tue, 08 Jun 2021 08:54:04 -0700 (PDT) Received: from localhost.localdomain (adsl-84-226-111-173.adslplus.ch. [84.226.111.173]) by smtp.gmail.com with ESMTPSA id l31sm3314180wms.16.2021.06.08.08.54.03 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 08 Jun 2021 08:54:04 -0700 (PDT) From: Jean-Philippe Brucker To: maz@kernel.org Cc: kvm@vger.kernel.org, kvmarm@lists.cs.columbia.edu, linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, pbonzini@redhat.com, corbet@lwn.net, james.morse@arm.com, alexandru.elisei@arm.com, suzuki.poulose@arm.com, catalin.marinas@arm.com, will@kernel.org, lorenzo.pieralisi@arm.com, salil.mehta@huawei.com, shameerali.kolothum.thodi@huawei.com, jonathan.cameron@huawei.com, Jean-Philippe Brucker Subject: [RFC PATCH 4/5] KVM: arm64: Pass hypercalls to userspace Date: Tue, 8 Jun 2021 17:48:05 +0200 Message-Id: <20210608154805.216869-5-jean-philippe@linaro.org> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20210608154805.216869-1-jean-philippe@linaro.org> References: <20210608154805.216869-1-jean-philippe@linaro.org> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20210608_085506_624245_0A827CEB X-CRM114-Status: GOOD ( 25.38 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Let userspace request to handle all hypercalls that aren't handled by KVM, by setting the KVM_CAP_ARM_HVC_TO_USER capability. With the help of another capability, this will allow userspace to handle PSCI calls. Suggested-by: James Morse Signed-off-by: Jean-Philippe Brucker --- Notes on this implementation: * A similar mechanism was proposed for SDEI some time ago [1]. This RFC generalizes the idea to all hypercalls, since that was suggested on the list [2, 3]. * We're reusing kvm_run.hypercall. I copied x0-x5 into kvm_run.hypercall.args[] to help userspace but I'm tempted to remove this, because: - Most user handlers will need to write results back into the registers (x0-x3 for SMCCC), so if we keep this shortcut we should go all the way and synchronize them on return to kernel. - QEMU doesn't care about this shortcut, it pulls all vcpu regs before handling the call. - SMCCC uses x0-x16 for parameters. x0 does contain the SMCCC function ID and may be useful for fast dispatch, we could keep that plus the immediate number. * Should we add a flag in the kvm_run.hypercall telling whether this is HVC or SMC? Can be added later in those bottom longmode and pad fields. * On top of this we could share with userspace which HVC ranges are available and which ones are handled by KVM. That can actually be added independently, through a vCPU/VM device attribute (which doesn't consume a new ioctl): - userspace issues HAS_ATTR ioctl on the VM fd to query whether this feature is available. - userspace queries the number N of HVC ranges using one GET_ATTR. - userspace passes an array of N ranges using another GET_ATTR. The array is filled and returned by KVM. * Untested for AArch32 guests. [1] https://lore.kernel.org/linux-arm-kernel/20170808164616.25949-12-james.morse@arm.com/ [2] https://lore.kernel.org/linux-arm-kernel/bf7e83f1-c58e-8d65-edd0-d08f27b8b766@arm.com/ [3] https://lore.kernel.org/linux-arm-kernel/f56cf420-affc-35f0-2355-801a924b8a35@arm.com/ --- Documentation/virt/kvm/api.rst | 17 +++++++++++++++-- arch/arm64/include/asm/kvm_host.h | 1 + include/kvm/arm_psci.h | 4 ++++ include/uapi/linux/kvm.h | 1 + arch/arm64/kvm/arm.c | 5 +++++ arch/arm64/kvm/hypercalls.c | 28 +++++++++++++++++++++++++++- 6 files changed, 53 insertions(+), 3 deletions(-) diff --git a/Documentation/virt/kvm/api.rst b/Documentation/virt/kvm/api.rst index e4fe7fb60d5d..3d8c1661e7b2 100644 --- a/Documentation/virt/kvm/api.rst +++ b/Documentation/virt/kvm/api.rst @@ -5228,8 +5228,12 @@ to the byte array. __u32 pad; } hypercall; -Unused. This was once used for 'hypercall to userspace'. To implement -such functionality, use KVM_EXIT_IO (x86) or KVM_EXIT_MMIO (all except s390). +On x86 this was once used for 'hypercall to userspace'. To implement such +functionality, use KVM_EXIT_IO (x86) or KVM_EXIT_MMIO (all except s390). + +On arm64 it is used for hypercalls, when the KVM_CAP_ARM_HVC_TO_USER capability +is enabled. 'nr' contains the HVC or SMC immediate. 'args' contains registers +x0 - x5. The other parameters are unused. .. note:: KVM_EXIT_IO is significantly faster than KVM_EXIT_MMIO. @@ -6894,3 +6898,12 @@ This capability is always enabled. This capability indicates that the KVM virtual PTP service is supported in the host. A VMM can check whether the service is available to the guest on migration. + +8.33 KVM_CAP_ARM_HVC_TO_USER +---------------------------- + +:Architecture: arm64 + +This capability indicates that KVM can pass unhandled hypercalls to userspace, +if the VMM enables it. Hypercalls are passed with KVM_EXIT_HYPERCALL in +kvm_run::hypercall. diff --git a/arch/arm64/include/asm/kvm_host.h b/arch/arm64/include/asm/kvm_host.h index 3ca732feb9a5..25554ce97045 100644 --- a/arch/arm64/include/asm/kvm_host.h +++ b/arch/arm64/include/asm/kvm_host.h @@ -123,6 +123,7 @@ struct kvm_arch { * supported. */ bool return_nisv_io_abort_to_user; + bool hvc_to_user; /* * VM-wide PMU filter, implemented as a bitmap and big enough for diff --git a/include/kvm/arm_psci.h b/include/kvm/arm_psci.h index 5b58bd2fe088..d6b71a48fbb1 100644 --- a/include/kvm/arm_psci.h +++ b/include/kvm/arm_psci.h @@ -16,6 +16,10 @@ #define KVM_ARM_PSCI_LATEST KVM_ARM_PSCI_1_0 +#define KVM_PSCI_FN_LAST KVM_PSCI_FN(3) +#define PSCI_0_2_FN_LAST PSCI_0_2_FN(0x3f) +#define PSCI_0_2_FN64_LAST PSCI_0_2_FN64(0x3f) + /* * We need the KVM pointer independently from the vcpu as we can call * this from HYP, and need to apply kern_hyp_va on it... diff --git a/include/uapi/linux/kvm.h b/include/uapi/linux/kvm.h index 06ba64c49737..aa831986a399 100644 --- a/include/uapi/linux/kvm.h +++ b/include/uapi/linux/kvm.h @@ -1084,6 +1084,7 @@ struct kvm_ppc_resize_hpt { #define KVM_CAP_VM_COPY_ENC_CONTEXT_FROM 197 #define KVM_CAP_PTP_KVM 198 #define KVM_CAP_ARM_MP_HALTED 199 +#define KVM_CAP_ARM_HVC_TO_USER 200 #ifdef KVM_CAP_IRQ_ROUTING diff --git a/arch/arm64/kvm/arm.c b/arch/arm64/kvm/arm.c index d6ad977fea5f..074197721e97 100644 --- a/arch/arm64/kvm/arm.c +++ b/arch/arm64/kvm/arm.c @@ -93,6 +93,10 @@ int kvm_vm_ioctl_enable_cap(struct kvm *kvm, r = 0; kvm->arch.return_nisv_io_abort_to_user = true; break; + case KVM_CAP_ARM_HVC_TO_USER: + r = 0; + kvm->arch.hvc_to_user = true; + break; default: r = -EINVAL; break; @@ -208,6 +212,7 @@ int kvm_vm_ioctl_check_extension(struct kvm *kvm, long ext) case KVM_CAP_VCPU_ATTRIBUTES: case KVM_CAP_PTP_KVM: case KVM_CAP_ARM_MP_HALTED: + case KVM_CAP_ARM_HVC_TO_USER: r = 1; break; case KVM_CAP_SET_GUEST_DEBUG2: diff --git a/arch/arm64/kvm/hypercalls.c b/arch/arm64/kvm/hypercalls.c index 30da78f72b3b..ccc2015eddf9 100644 --- a/arch/arm64/kvm/hypercalls.c +++ b/arch/arm64/kvm/hypercalls.c @@ -58,6 +58,28 @@ static void kvm_ptp_get_time(struct kvm_vcpu *vcpu, u64 *val) val[3] = lower_32_bits(cycles); } +static int kvm_hvc_user(struct kvm_vcpu *vcpu) +{ + int i; + struct kvm_run *run = vcpu->run; + + if (!vcpu->kvm->arch.hvc_to_user) { + smccc_set_retval(vcpu, SMCCC_RET_NOT_SUPPORTED, 0, 0, 0); + return 1; + } + + run->exit_reason = KVM_EXIT_HYPERCALL; + run->hypercall.nr = kvm_vcpu_hvc_get_imm(vcpu); + /* Copy the first parameters for fast access */ + for (i = 0; i < 6; i++) + run->hypercall.args[i] = vcpu_get_reg(vcpu, i); + run->hypercall.ret = 0; + run->hypercall.longmode = 0; + run->hypercall.pad = 0; + + return 0; +} + int kvm_hvc_call_handler(struct kvm_vcpu *vcpu) { u32 func_id = smccc_get_function(vcpu); @@ -139,8 +161,12 @@ int kvm_hvc_call_handler(struct kvm_vcpu *vcpu) case ARM_SMCCC_TRNG_RND32: case ARM_SMCCC_TRNG_RND64: return kvm_trng_call(vcpu); - default: + case KVM_PSCI_FN_BASE...KVM_PSCI_FN_LAST: + case PSCI_0_2_FN_BASE...PSCI_0_2_FN_LAST: + case PSCI_0_2_FN64_BASE...PSCI_0_2_FN64_LAST: return kvm_psci_call(vcpu); + default: + return kvm_hvc_user(vcpu); } smccc_set_retval(vcpu, val[0], val[1], val[2], val[3]); From patchwork Tue Jun 8 15:48:06 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jean-Philippe Brucker X-Patchwork-Id: 12307299 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.9 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,URIBL_BLOCKED,USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id C742CC47082 for ; Tue, 8 Jun 2021 15:59:02 +0000 (UTC) Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 9A44A60FEA for ; Tue, 8 Jun 2021 15:59:02 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 9A44A60FEA Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=linaro.org Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Message-Id:Date:Subject:Cc:To:From:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=21BejRbHYUiNRqUUC6//QOXHxFfh/VsA59NCGxzPc2s=; b=SbGdfSoWDDqn5Z 1FTH2T8aZyftKKZKKhhwksfeFeJbsahGqqS+ILxO4juDrDjvNIk291kTyNtp/QCuXTSO8gTw540B0 0ENVROoA0/wKVui69BZD2qA6tRgE6ecpx1guRx4AmGKNQ6sMfPHdSgkRJuurI1jLRPvqGapXIpQf1 dWkvnIt5goTq2pNlRROIzc3RrFtqUb7vZ4IUwqvW5yOac76C2mwajnnOe6Ai6QKl7kCC2hfKwf6ao gGOUNDFbtgveUbEsENeUr/VLoAtmtyfy3qsuvwGh9+V9ZT7re2ruNqFHAG2dETra3KpHCjnlHOohF Bfg737Pb0yn+0cQlviIQ==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1lqe62-009Oxf-RY; Tue, 08 Jun 2021 15:56:55 +0000 Received: from mail-wr1-f47.google.com ([209.85.221.47]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1lqe4K-009OPI-Kw for linux-arm-kernel@lists.infradead.org; Tue, 08 Jun 2021 15:55:11 +0000 Received: by mail-wr1-f47.google.com with SMTP id a11so20264960wrt.13 for ; Tue, 08 Jun 2021 08:55:07 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=KHEWXTvaTgGt1BVLxIfWQF63jGSa3hhv5Aeen36ISYE=; b=JFN/xiy0BsolYRMuvUu/vBBp8n6h7jDdIzTqFsPh8onZqCMTUkEyfq6+dShb/4jooa 4BfzwPmDLIIGk+APV232oFKjNH8TwCxOn945UJ6HhpeB4bqxxI4sANfxPmyky5JN68ci /AeQ6D6u41JuYGXQIHDrsSjX9YypR07kCATFqSrHzVd6Hna45k6evaAFB/Ta/+knrIbg E9OfDY92p3KvHMlQALtdInffDDrQ4cNn9DMFrpOICKsXVojgNB0T664irg2FH/xP1tkt nFMHjilT2SK06OxmVp0ISWLldj3mSlNHtriZZyZOaHWdkzaZHNlefhrr6r7IKTaPn8iE 1tsQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=KHEWXTvaTgGt1BVLxIfWQF63jGSa3hhv5Aeen36ISYE=; b=dQGnVmXC1iYGp48BsuYb6e4McQD77J5NmD9ZoSaiM+1LvXmIZPIH9TnPqNoOYJ8piQ I5R75pbAiHkGhYRSkWxVqKqj1uio4VNQbU8dqqIlMHetKOKaf2v+DM0nEuOzEykqCKLv yiIVjzY6JVZyI549mYSnuAFjWDQY9Ibb4DlBRLHXCxIOe/SF94SPUMk3r2Fw4szJMXRA f7ept+CV5COuA1Fmyb1RdJa8CiPg0JOVQzsjrErYSpy1CMa5UY3CiQ7lyEUprTxieNAZ QGcAJOMnX1DtcsdFrtDdDZ3VAW1EjPxN/Gi7Ad26TQtTPbVZ+q9XelLxtunfL+3lbfoI 1K0w== X-Gm-Message-State: AOAM533t6NFikFOvn849XkZHJmvdKpuD+AEhyo4o21JDiQUzJ2MGnIH5 Lu3lcb74EpzpkHu9H+gP9RDTzQ== X-Google-Smtp-Source: ABdhPJxIqvImjc8EYF1I8fYhJ8WzhBv5pnbK6ynBNA3UWFEPYAyfdGSMfbYuZieceMItwHj8Wy0EPA== X-Received: by 2002:adf:f94c:: with SMTP id q12mr6581214wrr.417.1623167646586; Tue, 08 Jun 2021 08:54:06 -0700 (PDT) Received: from localhost.localdomain (adsl-84-226-111-173.adslplus.ch. [84.226.111.173]) by smtp.gmail.com with ESMTPSA id l31sm3314180wms.16.2021.06.08.08.54.04 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 08 Jun 2021 08:54:05 -0700 (PDT) From: Jean-Philippe Brucker To: maz@kernel.org Cc: kvm@vger.kernel.org, kvmarm@lists.cs.columbia.edu, linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, pbonzini@redhat.com, corbet@lwn.net, james.morse@arm.com, alexandru.elisei@arm.com, suzuki.poulose@arm.com, catalin.marinas@arm.com, will@kernel.org, lorenzo.pieralisi@arm.com, salil.mehta@huawei.com, shameerali.kolothum.thodi@huawei.com, jonathan.cameron@huawei.com, Jean-Philippe Brucker Subject: [RFC PATCH 5/5] KVM: arm64: Pass PSCI calls to userspace Date: Tue, 8 Jun 2021 17:48:06 +0200 Message-Id: <20210608154805.216869-6-jean-philippe@linaro.org> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20210608154805.216869-1-jean-philippe@linaro.org> References: <20210608154805.216869-1-jean-philippe@linaro.org> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20210608_085508_723890_EB3E75C9 X-CRM114-Status: GOOD ( 25.54 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Let userspace request to handle PSCI calls, by setting the new KVM_CAP_ARM_PSCI_TO_USER capability. SMCCC probe requires PSCI v1.x. If userspace only implements PSCI v0.2, the guest won't query SMCCC support through PSCI and won't use the spectre workarounds. We could hijack PSCI_VERSION and pretend to support v1.0 if userspace does not, then handle all v1.0 calls ourselves (including guessing the PSCI feature set implemented by the guest), but that seems unnecessary. After all the API already allows userspace to force a version lower than v1.0 using the firmware pseudo-registers. The KVM_REG_ARM_PSCI_VERSION pseudo-register currently resets to either v0.1 if userspace doesn't set KVM_ARM_VCPU_PSCI_0_2, or KVM_ARM_PSCI_LATEST (1.0). Suggested-by: James Morse Signed-off-by: Jean-Philippe Brucker --- Documentation/virt/kvm/api.rst | 14 ++++++++++++++ Documentation/virt/kvm/arm/psci.rst | 1 + arch/arm64/include/asm/kvm_host.h | 1 + include/kvm/arm_hypercalls.h | 1 + include/uapi/linux/kvm.h | 1 + arch/arm64/kvm/arm.c | 10 +++++++--- arch/arm64/kvm/hypercalls.c | 2 +- arch/arm64/kvm/psci.c | 13 +++++++++++++ 8 files changed, 39 insertions(+), 4 deletions(-) diff --git a/Documentation/virt/kvm/api.rst b/Documentation/virt/kvm/api.rst index 3d8c1661e7b2..f24eb70e575d 100644 --- a/Documentation/virt/kvm/api.rst +++ b/Documentation/virt/kvm/api.rst @@ -6907,3 +6907,17 @@ available to the guest on migration. This capability indicates that KVM can pass unhandled hypercalls to userspace, if the VMM enables it. Hypercalls are passed with KVM_EXIT_HYPERCALL in kvm_run::hypercall. + +8.34 KVM_CAP_ARM_PSCI_TO_USER +----------------------------- + +:Architectures: arm64 + +When the VMM enables this capability, all PSCI calls are passed to userspace +instead of being handled by KVM. Capability KVM_CAP_ARM_HVC_TO_USER must be +enabled first. + +Userspace should support at least PSCI v1.0. Otherwise SMCCC features won't be +available to the guest. Userspace does not need to handle the SMCCC_VERSION +parameter for the PSCI_FEATURES function. The KVM_ARM_VCPU_PSCI_0_2 vCPU +feature should be set even if this capability is enabled. diff --git a/Documentation/virt/kvm/arm/psci.rst b/Documentation/virt/kvm/arm/psci.rst index d52c2e83b5b8..110011d1fa3f 100644 --- a/Documentation/virt/kvm/arm/psci.rst +++ b/Documentation/virt/kvm/arm/psci.rst @@ -34,6 +34,7 @@ The following register is defined: - Allows any PSCI version implemented by KVM and compatible with v0.2 to be set with SET_ONE_REG - Affects the whole VM (even if the register view is per-vcpu) + - Defaults to PSCI 1.0 if userspace enables KVM_CAP_ARM_PSCI_TO_USER. * KVM_REG_ARM_SMCCC_ARCH_WORKAROUND_1: Holds the state of the firmware support to mitigate CVE-2017-5715, as diff --git a/arch/arm64/include/asm/kvm_host.h b/arch/arm64/include/asm/kvm_host.h index 25554ce97045..5d74b769c16d 100644 --- a/arch/arm64/include/asm/kvm_host.h +++ b/arch/arm64/include/asm/kvm_host.h @@ -124,6 +124,7 @@ struct kvm_arch { */ bool return_nisv_io_abort_to_user; bool hvc_to_user; + bool psci_to_user; /* * VM-wide PMU filter, implemented as a bitmap and big enough for diff --git a/include/kvm/arm_hypercalls.h b/include/kvm/arm_hypercalls.h index 0e2509d27910..b66c6a000ef3 100644 --- a/include/kvm/arm_hypercalls.h +++ b/include/kvm/arm_hypercalls.h @@ -6,6 +6,7 @@ #include +int kvm_hvc_user(struct kvm_vcpu *vcpu); int kvm_hvc_call_handler(struct kvm_vcpu *vcpu); static inline u32 smccc_get_function(struct kvm_vcpu *vcpu) diff --git a/include/uapi/linux/kvm.h b/include/uapi/linux/kvm.h index aa831986a399..2b8e55aa7e1e 100644 --- a/include/uapi/linux/kvm.h +++ b/include/uapi/linux/kvm.h @@ -1085,6 +1085,7 @@ struct kvm_ppc_resize_hpt { #define KVM_CAP_PTP_KVM 198 #define KVM_CAP_ARM_MP_HALTED 199 #define KVM_CAP_ARM_HVC_TO_USER 200 +#define KVM_CAP_ARM_PSCI_TO_USER 201 #ifdef KVM_CAP_IRQ_ROUTING diff --git a/arch/arm64/kvm/arm.c b/arch/arm64/kvm/arm.c index 074197721e97..bc3e63b0b3ad 100644 --- a/arch/arm64/kvm/arm.c +++ b/arch/arm64/kvm/arm.c @@ -83,7 +83,7 @@ int kvm_arch_check_processor_compat(void *opaque) int kvm_vm_ioctl_enable_cap(struct kvm *kvm, struct kvm_enable_cap *cap) { - int r; + int r = -EINVAL; if (cap->flags) return -EINVAL; @@ -97,8 +97,11 @@ int kvm_vm_ioctl_enable_cap(struct kvm *kvm, r = 0; kvm->arch.hvc_to_user = true; break; - default: - r = -EINVAL; + case KVM_CAP_ARM_PSCI_TO_USER: + if (kvm->arch.hvc_to_user) { + r = 0; + kvm->arch.psci_to_user = true; + } break; } @@ -213,6 +216,7 @@ int kvm_vm_ioctl_check_extension(struct kvm *kvm, long ext) case KVM_CAP_PTP_KVM: case KVM_CAP_ARM_MP_HALTED: case KVM_CAP_ARM_HVC_TO_USER: + case KVM_CAP_ARM_PSCI_TO_USER: r = 1; break; case KVM_CAP_SET_GUEST_DEBUG2: diff --git a/arch/arm64/kvm/hypercalls.c b/arch/arm64/kvm/hypercalls.c index ccc2015eddf9..621d5a5b7e48 100644 --- a/arch/arm64/kvm/hypercalls.c +++ b/arch/arm64/kvm/hypercalls.c @@ -58,7 +58,7 @@ static void kvm_ptp_get_time(struct kvm_vcpu *vcpu, u64 *val) val[3] = lower_32_bits(cycles); } -static int kvm_hvc_user(struct kvm_vcpu *vcpu) +int kvm_hvc_user(struct kvm_vcpu *vcpu) { int i; struct kvm_run *run = vcpu->run; diff --git a/arch/arm64/kvm/psci.c b/arch/arm64/kvm/psci.c index 42a307ceb95f..7f44ee527966 100644 --- a/arch/arm64/kvm/psci.c +++ b/arch/arm64/kvm/psci.c @@ -353,6 +353,16 @@ static int kvm_psci_0_1_call(struct kvm_vcpu *vcpu) return 1; } +static bool kvm_psci_call_is_user(struct kvm_vcpu *vcpu) +{ + /* Handle the special case of SMCCC probe through PSCI */ + if (smccc_get_function(vcpu) == PSCI_1_0_FN_PSCI_FEATURES && + smccc_get_arg1(vcpu) == ARM_SMCCC_VERSION_FUNC_ID) + return false; + + return vcpu->kvm->arch.psci_to_user; +} + /** * kvm_psci_call - handle PSCI call if r0 value is in range * @vcpu: Pointer to the VCPU struct @@ -369,6 +379,9 @@ static int kvm_psci_0_1_call(struct kvm_vcpu *vcpu) */ int kvm_psci_call(struct kvm_vcpu *vcpu) { + if (kvm_psci_call_is_user(vcpu)) + return kvm_hvc_user(vcpu); + switch (kvm_psci_version(vcpu, vcpu->kvm)) { case KVM_ARM_PSCI_1_0: return kvm_psci_1_0_call(vcpu);