[87/89] KVM: arm64: Expose memory sharing hypercalls to protected guests

Message ID	20220519134204.5379-88-will@kernel.org (mailing list archive)
State	New, archived
Headers	show Return-Path: <kvm-owner@kernel.org> From: Will Deacon <will@kernel.org> To: kvmarm@lists.cs.columbia.edu Cc: Will Deacon <will@kernel.org>, Ard Biesheuvel <ardb@kernel.org>, Sean Christopherson <seanjc@google.com>, Alexandru Elisei <alexandru.elisei@arm.com>, Andy Lutomirski <luto@amacapital.net>, Catalin Marinas <catalin.marinas@arm.com>, James Morse <james.morse@arm.com>, Chao Peng <chao.p.peng@linux.intel.com>, Quentin Perret <qperret@google.com>, Suzuki K Poulose <suzuki.poulose@arm.com>, Michael Roth <michael.roth@amd.com>, Mark Rutland <mark.rutland@arm.com>, Fuad Tabba <tabba@google.com>, Oliver Upton <oupton@google.com>, Marc Zyngier <maz@kernel.org>, kernel-team@android.com, kvm@vger.kernel.org, linux-arm-kernel@lists.infradead.org Subject: [PATCH 87/89] KVM: arm64: Expose memory sharing hypercalls to protected guests Date: Thu, 19 May 2022 14:42:02 +0100 Message-Id: <20220519134204.5379-88-will@kernel.org> In-Reply-To: <20220519134204.5379-1-will@kernel.org> References: <20220519134204.5379-1-will@kernel.org> MIME-Version: 1.0 Content-Transfer-Encoding: 8bit Precedence: bulk
Series	KVM: arm64: Base support for the pKVM hypervisor at EL2 \| expand [00/89] KVM: arm64: Base support for the pKVM hypervisor at EL2 [01/89] KVM: arm64: Handle all ID registers trapped for a protected VM [02/89] KVM: arm64: Remove redundant hyp_assert_lock_held() assertions [03/89] KVM: arm64: Return error from kvm_arch_init_vm() on allocation failure [04/89] KVM: arm64: Ignore 'kvm-arm.mode=protected' when using VHE [05/89] KVM: arm64: Extend comment in has_vhe() [06/89] KVM: arm64: Drop stale comment [07/89] KVM: arm64: Move hyp refcount manipulation helpers [08/89] KVM: arm64: Back hyp_vmemmap for all of memory [09/89] KVM: arm64: Unify identifiers used to distinguish host and hypervisor [10/89] KVM: arm64: Implement do_donate() helper for donating memory [11/89] KVM: arm64: Prevent the donation of no-map pages [12/89] KVM: arm64: Add helpers to pin memory shared with hyp [13/89] KVM: arm64: Include asm/kvm_mmu.h in nvhe/mem_protect.h [14/89] KVM: arm64: Add hyp_spinlock_t static initializer [15/89] KVM: arm64: Introduce shadow VM state at EL2 [16/89] KVM: arm64: Instantiate VM shadow data from EL1 [17/89] KVM: arm64: Make hyp stage-1 refcnt correct on the whole range [18/89] KVM: arm64: Factor out private range VA allocation [19/89] KVM: arm64: Add pcpu fixmap infrastructure at EL2 [20/89] KVM: arm64: Provide I-cache invalidation by VA at EL2 [21/89] KVM: arm64: Allow non-coallescable pages in a hyp_pool [22/89] KVM: arm64: Add generic hyp_memcache helpers [23/89] KVM: arm64: Instantiate guest stage-2 page-tables at EL2 [24/89] KVM: arm64: Return guest memory from EL2 via dedicated teardown memcache [25/89] KVM: arm64: Add flags to struct hyp_page [26/89] KVM: arm64: Provide a hypercall for the host to reclaim guest memory [27/89] KVM: arm64: Extend memory sharing to allow host-to-guest transitions [28/89] KVM: arm64: Consolidate stage-2 init in one function [29/89] KVM: arm64: Check for PTE validity when checking for executable/cacheable [30/89] KVM: arm64: Do not allow memslot changes after first VM run under pKVM [31/89] KVM: arm64: Disallow dirty logging and RO memslots with pKVM [32/89] KVM: arm64: Use the shadow vCPU structure in handle___kvm_vcpu_run() [33/89] KVM: arm64: Handle guest stage-2 page-tables entirely at EL2 [34/89] KVM: arm64: Don't access kvm_arm_hyp_percpu_base at EL1 [35/89] KVM: arm64: Unmap kvm_arm_hyp_percpu_base from the host [36/89] KVM: arm64: Maintain a copy of 'kvm_arm_vmid_bits' at EL2 [37/89] KVM: arm64: Explicitly map kvm_vgic_global_state at EL2 [38/89] KVM: arm64: Don't map host sections in pkvm [39/89] KVM: arm64: Extend memory donation to allow host-to-guest transitions [40/89] KVM: arm64: Split up nvhe/fixed_config.h [41/89] KVM: arm64: Make vcpu_{read,write}_sys_reg available to HYP code [42/89] KVM: arm64: Simplify vgic-v3 hypercalls [43/89] KVM: arm64: Add the {flush,sync}_vgic_state() primitives [44/89] KVM: arm64: Introduce predicates to check for protected state [45/89] KVM: arm64: Add the {flush,sync}_timer_state() primitives [46/89] KVM: arm64: Introduce the pkvm_vcpu_{load,put} hypercalls [47/89] KVM: arm64: Add current vcpu and shadow_state lookup primitive [48/89] KVM: arm64: Skip __kvm_adjust_pc() for protected vcpus [49/89] KVM: arm64: Add hyp per_cpu variable to track current physical cpu number [50/89] KVM: arm64: Ensure that TLBs and I-cache are private to each vcpu [51/89] KVM: arm64: Introduce per-EC entry/exit handlers [52/89] KVM: arm64: Introduce lazy-ish state sync for non-protected VMs [53/89] KVM: arm64: Lazy host FP save/restore [54/89] KVM: arm64: Reduce host/shadow vcpu state copying [55/89] KVM: arm64: Do not pass the vcpu to __pkvm_host_map_guest() [56/89] KVM: arm64: Check directly whether the vcpu is protected [57/89] KVM: arm64: Trap debug break and watch from guest [58/89] KVM: arm64: Restrict protected VM capabilities [59/89] KVM: arm64: Do not support MTE for protected VMs [60/89] KVM: arm64: Refactor reset_mpidr to extract its computation [61/89] KVM: arm64: Reset sysregs for protected VMs [62/89] KVM: arm64: Move pkvm_vcpu_init_traps to shadow vcpu init [63/89] KVM: arm64: Fix initializing traps in protected mode [64/89] KVM: arm64: Advertise GICv3 sysreg interface to protected guests [65/89] KVM: arm64: Force injection of a data abort on NISV MMIO exit [66/89] KVM: arm64: Donate memory to protected guests [67/89] KVM: arm64: Add EL2 entry/exit handlers for pKVM guests [68/89] KVM: arm64: Move vgic state between host and shadow vcpu structures [69/89] KVM: arm64: Do not update virtual timer state for protected VMs [70/89] KVM: arm64: Refactor kvm_vcpu_enable_ptrauth() for hyp use [71/89] KVM: arm64: Initialize shadow vm state at hyp [72/89] KVM: arm64: Track the SVE state in the shadow vcpu [73/89] KVM: arm64: Add HVC handling for protected guests at EL2 [74/89] KVM: arm64: Move pstate reset values to kvm_arm.h [75/89] KVM: arm64: Move some kvm_psci functions to a shared header [76/89] KVM: arm64: Factor out vcpu_reset code for core registers and PSCI [77/89] KVM: arm64: Handle PSCI for protected VMs in EL2 [78/89] KVM: arm64: Don't expose TLBI hypercalls after de-privilege [79/89] KVM: arm64: Add is_pkvm_initialized() helper [80/89] KVM: arm64: Refactor enter_exception64() [81/89] KVM: arm64: Inject SIGSEGV on illegal accesses [82/89] KVM: arm64: Support TLB invalidation in guest context [83/89] KVM: arm64: Avoid BBM when changing only s/w bits in Stage-2 PTE [84/89] KVM: arm64: Extend memory sharing to allow guest-to-host transitions [85/89] KVM: arm64: Document the KVM/arm64-specific calls in hypercalls.rst [86/89] KVM: arm64: Reformat/beautify PTP hypercall documentation [87/89] KVM: arm64: Expose memory sharing hypercalls to protected guests [88/89] KVM: arm64: Introduce KVM_VM_TYPE_ARM_PROTECTED machine type for PVMs [89/89] Documentation: KVM: Add some documentation for Protected KVM on arm64

diff --git a/Documentation/virt/kvm/arm/hypercalls.rst b/Documentation/virt/kvm/arm/hypercalls.rst index 17be111f493f..d96c9fd7d8c5 100644 --- a/Documentation/virt/kvm/arm/hypercalls.rst +++ b/Documentation/virt/kvm/arm/hypercalls.rst @@ -44,3 +44,75 @@ Provides a discovery mechanism for other KVM/arm64 hypercalls. ---------------------------------------- See ptp_kvm.rst + +``ARM_SMCCC_KVM_FUNC_HYP_MEMINFO`` +---------------------------------- + +Query the memory protection parameters for a protected virtual machine. + ++---------------------+-------------------------------------------------------------+ +| Presence: | Optional; protected guests only. | ++---------------------+-------------------------------------------------------------+ +| Calling convention: | HVC64 | ++---------------------+----------+--------------------------------------------------+ +| Function ID: | (uint32) | 0xC6000002 | ++---------------------+----------+----+---------------------------------------------+ +| Arguments: | (uint64) | R1 | Reserved / Must be zero | +| +----------+----+---------------------------------------------+ +| | (uint64) | R2 | Reserved / Must be zero | +| +----------+----+---------------------------------------------+ +| | (uint64) | R3 | Reserved / Must be zero | ++---------------------+----------+----+---------------------------------------------+ +| Return Values: | (int64) | R0 | ``INVALID_PARAMETER (-3)`` on error, else | +| | | | memory protection granule in bytes | ++---------------------+----------+----+---------------------------------------------+ + +``ARM_SMCCC_KVM_FUNC_MEM_SHARE`` +-------------------------------- + +Share a region of memory with the KVM host, granting it read, write and execute +permissions. The size of the region is equal to the memory protection granule +advertised by ``ARM_SMCCC_KVM_FUNC_HYP_MEMINFO``. + ++---------------------+-------------------------------------------------------------+ +| Presence: | Optional; protected guests only. | ++---------------------+-------------------------------------------------------------+ +| Calling convention: | HVC64 | ++---------------------+----------+--------------------------------------------------+ +| Function ID: | (uint32) | 0xC6000003 | ++---------------------+----------+----+---------------------------------------------+ +| Arguments: | (uint64) | R1 | Base IPA of memory region to share | +| +----------+----+---------------------------------------------+ +| | (uint64) | R2 | Reserved / Must be zero | +| +----------+----+---------------------------------------------+ +| | (uint64) | R3 | Reserved / Must be zero | ++---------------------+----------+----+---------------------------------------------+ +| Return Values: | (int64) | R0 | ``SUCCESS (0)`` | +| | | +---------------------------------------------+ +| | | | ``INVALID_PARAMETER (-3)`` | ++---------------------+----------+----+---------------------------------------------+ + +``ARM_SMCCC_KVM_FUNC_MEM_UNSHARE`` +---------------------------------- + +Revoke access permission from the KVM host to a memory region previously shared +with ``ARM_SMCCC_KVM_FUNC_MEM_SHARE``. The size of the region is equal to the +memory protection granule advertised by ``ARM_SMCCC_KVM_FUNC_HYP_MEMINFO``. + ++---------------------+-------------------------------------------------------------+ +| Presence: | Optional; protected guests only. | ++---------------------+-------------------------------------------------------------+ +| Calling convention: | HVC64 | ++---------------------+----------+--------------------------------------------------+ +| Function ID: | (uint32) | 0xC6000004 | ++---------------------+----------+----+---------------------------------------------+ +| Arguments: | (uint64) | R1 | Base IPA of memory region to unshare | +| +----------+----+---------------------------------------------+ +| | (uint64) | R2 | Reserved / Must be zero | +| +----------+----+---------------------------------------------+ +| | (uint64) | R3 | Reserved / Must be zero | ++---------------------+----------+----+---------------------------------------------+ +| Return Values: | (int64) | R0 | ``SUCCESS (0)`` | +| | | +---------------------------------------------+ +| | | | ``INVALID_PARAMETER (-3)`` | ++---------------------+----------+----+---------------------------------------------+ diff --git a/arch/arm64/kvm/hyp/nvhe/hyp-main.c b/arch/arm64/kvm/hyp/nvhe/hyp-main.c index 694e0071b13e..be000d457e44 100644 --- a/arch/arm64/kvm/hyp/nvhe/hyp-main.c +++ b/arch/arm64/kvm/hyp/nvhe/hyp-main.c @@ -45,7 +45,7 @@ static void handle_pvm_entry_wfx(struct kvm_vcpu *host_vcpu, struct kvm_vcpu *sh KVM_ARM64_INCREMENT_PC; } -static void handle_pvm_entry_hvc64(struct kvm_vcpu *host_vcpu, struct kvm_vcpu *shadow_vcpu) +static void handle_pvm_entry_psci(struct kvm_vcpu *host_vcpu, struct kvm_vcpu *shadow_vcpu) { u32 psci_fn = smccc_get_function(shadow_vcpu); u64 ret = READ_ONCE(host_vcpu->arch.ctxt.regs.regs[0]); @@ -81,6 +81,22 @@ static void handle_pvm_entry_hvc64(struct kvm_vcpu *host_vcpu, struct kvm_vcpu * vcpu_set_reg(shadow_vcpu, 0, ret); } +static void handle_pvm_entry_hvc64(struct kvm_vcpu *host_vcpu, struct kvm_vcpu *shadow_vcpu) +{ + u32 fn = smccc_get_function(shadow_vcpu); + + switch (fn) { + case ARM_SMCCC_VENDOR_HYP_KVM_MEM_SHARE_FUNC_ID: + fallthrough; + case ARM_SMCCC_VENDOR_HYP_KVM_MEM_UNSHARE_FUNC_ID: + vcpu_set_reg(shadow_vcpu, 0, SMCCC_RET_SUCCESS); + break; + default: + handle_pvm_entry_psci(host_vcpu, shadow_vcpu); + break; + } +} + static void handle_pvm_entry_sys64(struct kvm_vcpu *host_vcpu, struct kvm_vcpu *shadow_vcpu) { unsigned long host_flags; @@ -257,6 +273,12 @@ static void handle_pvm_exit_hvc64(struct kvm_vcpu *host_vcpu, struct kvm_vcpu *s n = 1; break; + case ARM_SMCCC_VENDOR_HYP_KVM_MEM_SHARE_FUNC_ID: + fallthrough; + case ARM_SMCCC_VENDOR_HYP_KVM_MEM_UNSHARE_FUNC_ID: + n = 4; + break; + case PSCI_1_1_FN_SYSTEM_RESET2: case PSCI_1_1_FN64_SYSTEM_RESET2: n = 3; diff --git a/arch/arm64/kvm/hyp/nvhe/pkvm.c b/arch/arm64/kvm/hyp/nvhe/pkvm.c index d44f524c936b..e33ba9067d7b 100644 --- a/arch/arm64/kvm/hyp/nvhe/pkvm.c +++ b/arch/arm64/kvm/hyp/nvhe/pkvm.c @@ -1118,6 +1118,82 @@ static bool pkvm_handle_psci(struct kvm_vcpu *vcpu) return pvm_psci_not_supported(vcpu); } +static u64 __pkvm_memshare_page_req(struct kvm_vcpu *vcpu, u64 ipa) +{ + u64 elr; + + /* Fake up a data abort (Level 3 translation fault on write) */ + vcpu->arch.fault.esr_el2 = (u32)ESR_ELx_EC_DABT_LOW << ESR_ELx_EC_SHIFT | + ESR_ELx_WNR | ESR_ELx_FSC_FAULT | + FIELD_PREP(ESR_ELx_FSC_LEVEL, 3); + + /* Shuffle the IPA around into the HPFAR */ + vcpu->arch.fault.hpfar_el2 = (ipa >> 8) & HPFAR_MASK; + + /* This is a virtual address. 0's good. Let's go with 0. */ + vcpu->arch.fault.far_el2 = 0; + + /* Rewind the ELR so we return to the HVC once the IPA is mapped */ + elr = read_sysreg(elr_el2); + elr -= 4; + write_sysreg(elr, elr_el2); + + return ARM_EXCEPTION_TRAP; +} + +static bool pkvm_memshare_call(struct kvm_vcpu *vcpu, u64 *exit_code) +{ + u64 ipa = smccc_get_arg1(vcpu); + u64 arg2 = smccc_get_arg2(vcpu); + u64 arg3 = smccc_get_arg3(vcpu); + int err; + + if (arg2 || arg3) + goto out_guest_err; + + err = __pkvm_guest_share_host(vcpu, ipa); + switch (err) { + case 0: + /* Success! Now tell the host. */ + goto out_host; + case -EFAULT: + /* + * Convert the exception into a data abort so that the page + * being shared is mapped into the guest next time. + */ + *exit_code = __pkvm_memshare_page_req(vcpu, ipa); + goto out_host; + } + +out_guest_err: + smccc_set_retval(vcpu, SMCCC_RET_INVALID_PARAMETER, 0, 0, 0); + return true; + +out_host: + return false; +} + +static bool pkvm_memunshare_call(struct kvm_vcpu *vcpu) +{ + u64 ipa = smccc_get_arg1(vcpu); + u64 arg2 = smccc_get_arg2(vcpu); + u64 arg3 = smccc_get_arg3(vcpu); + int err; + + if (arg2 || arg3) + goto out_guest_err; + + err = __pkvm_guest_unshare_host(vcpu, ipa); + if (err) + goto out_guest_err; + + return false; + +out_guest_err: + smccc_set_retval(vcpu, SMCCC_RET_INVALID_PARAMETER, 0, 0, 0); + return true; +} + /* * Handler for protected VM HVC calls. * @@ -1127,13 +1203,42 @@ static bool pkvm_handle_psci(struct kvm_vcpu *vcpu) bool kvm_handle_pvm_hvc64(struct kvm_vcpu *vcpu, u64 *exit_code) { u32 fn = smccc_get_function(vcpu); + u64 val[4] = { SMCCC_RET_NOT_SUPPORTED }; switch (fn) { case ARM_SMCCC_VERSION_FUNC_ID: /* Nothing to be handled by the host. Go back to the guest. */ - smccc_set_retval(vcpu, ARM_SMCCC_VERSION_1_1, 0, 0, 0); - return true; + val[0] = ARM_SMCCC_VERSION_1_1; + break; + case ARM_SMCCC_VENDOR_HYP_CALL_UID_FUNC_ID: + val[0] = ARM_SMCCC_VENDOR_HYP_UID_KVM_REG_0; + val[1] = ARM_SMCCC_VENDOR_HYP_UID_KVM_REG_1; + val[2] = ARM_SMCCC_VENDOR_HYP_UID_KVM_REG_2; + val[3] = ARM_SMCCC_VENDOR_HYP_UID_KVM_REG_3; + break; + case ARM_SMCCC_VENDOR_HYP_KVM_FEATURES_FUNC_ID: + val[0] = BIT(ARM_SMCCC_KVM_FUNC_FEATURES); + val[0] |= BIT(ARM_SMCCC_KVM_FUNC_HYP_MEMINFO); + val[0] |= BIT(ARM_SMCCC_KVM_FUNC_MEM_SHARE); + val[0] |= BIT(ARM_SMCCC_KVM_FUNC_MEM_UNSHARE); + break; + case ARM_SMCCC_VENDOR_HYP_KVM_HYP_MEMINFO_FUNC_ID: + if (smccc_get_arg1(vcpu) || + smccc_get_arg2(vcpu) || + smccc_get_arg3(vcpu)) { + val[0] = SMCCC_RET_INVALID_PARAMETER; + } else { + val[0] = PAGE_SIZE; + } + break; + case ARM_SMCCC_VENDOR_HYP_KVM_MEM_SHARE_FUNC_ID: + return pkvm_memshare_call(vcpu, exit_code); + case ARM_SMCCC_VENDOR_HYP_KVM_MEM_UNSHARE_FUNC_ID: + return pkvm_memunshare_call(vcpu); default: return pkvm_handle_psci(vcpu); } + + smccc_set_retval(vcpu, val[0], val[1], val[2], val[3]); + return true; } diff --git a/include/linux/arm-smccc.h b/include/linux/arm-smccc.h index 220c8c60e021..25c576a910df 100644 --- a/include/linux/arm-smccc.h +++ b/include/linux/arm-smccc.h @@ -112,6 +112,9 @@ /* KVM "vendor specific" services */ #define ARM_SMCCC_KVM_FUNC_FEATURES 0 #define ARM_SMCCC_KVM_FUNC_PTP 1 +#define ARM_SMCCC_KVM_FUNC_HYP_MEMINFO 2 +#define ARM_SMCCC_KVM_FUNC_MEM_SHARE 3 +#define ARM_SMCCC_KVM_FUNC_MEM_UNSHARE 4 #define ARM_SMCCC_KVM_FUNC_FEATURES_2 127 #define ARM_SMCCC_KVM_NUM_FUNCS 128 @@ -134,6 +137,24 @@ ARM_SMCCC_OWNER_VENDOR_HYP, \ ARM_SMCCC_KVM_FUNC_PTP) +#define ARM_SMCCC_VENDOR_HYP_KVM_HYP_MEMINFO_FUNC_ID \ + ARM_SMCCC_CALL_VAL(ARM_SMCCC_FAST_CALL, \ + ARM_SMCCC_SMC_64, \ + ARM_SMCCC_OWNER_VENDOR_HYP, \ + ARM_SMCCC_KVM_FUNC_HYP_MEMINFO) + +#define ARM_SMCCC_VENDOR_HYP_KVM_MEM_SHARE_FUNC_ID \ + ARM_SMCCC_CALL_VAL(ARM_SMCCC_FAST_CALL, \ + ARM_SMCCC_SMC_64, \ + ARM_SMCCC_OWNER_VENDOR_HYP, \ + ARM_SMCCC_KVM_FUNC_MEM_SHARE) + +#define ARM_SMCCC_VENDOR_HYP_KVM_MEM_UNSHARE_FUNC_ID \ + ARM_SMCCC_CALL_VAL(ARM_SMCCC_FAST_CALL, \ + ARM_SMCCC_SMC_64, \ + ARM_SMCCC_OWNER_VENDOR_HYP, \ + ARM_SMCCC_KVM_FUNC_MEM_UNSHARE) + /* ptp_kvm counter type ID */ #define KVM_PTP_VIRT_COUNTER 0 #define KVM_PTP_PHYS_COUNTER 1

[87/89] KVM: arm64: Expose memory sharing hypercalls to protected guests

Commit Message

Patch