From patchwork Wed Jun 14 12:25:59 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Mostafa Saleh X-Patchwork-Id: 13279995 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 96D60EB64DA for ; Wed, 14 Jun 2023 12:26:37 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:Cc:To:From:Subject:Message-ID: Mime-Version:Date:Reply-To:Content-ID:Content-Description:Resent-Date: Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:In-Reply-To: References:List-Owner; bh=1NVepAlBos84CVHxUSZOuLE9gfk+NxURIBZAwZN6x9s=; b=bAK +km5nav+173+X3Wx0Lh0Lzsp9Yv6L7uj79DAxIoEt/+DVtX9/F2zbc5LJj19UmI1/t1EVsxr6YGl1 5x++b/v/JE8CZXXjxWQhIjFEZVuUJkRuMdNiVFBkOe8dUTmHs8jhYYpE0ogq4/pzNrHFQE5yTVM3t F1F6dbRVo+8gnY0E8rj1Ls1PGqiK5BzIsniETATG8XKOqhC+ZOwnQJ7XCaPSknlzYbCQp1NeYg5DO l+m5+ZYyY1EltCFvXXkzZWOk0f5eHaQ7ISIC65W4/0aRWVFY2Jp2mIZ4A+CtwJR0RCd548Dqsgz0J ubIoXoJXAzAK8biQ/lIcfvSju/g6X2A==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.96 #2 (Red Hat Linux)) id 1q9PZn-00BcWJ-0H; Wed, 14 Jun 2023 12:26:15 +0000 Received: from mail-lj1-x249.google.com ([2a00:1450:4864:20::249]) by bombadil.infradead.org with esmtps (Exim 4.96 #2 (Red Hat Linux)) id 1q9PZk-00BcVk-0x for linux-arm-kernel@lists.infradead.org; Wed, 14 Jun 2023 12:26:13 +0000 Received: by mail-lj1-x249.google.com with SMTP id 38308e7fff4ca-2b1c60977e3so5562521fa.0 for ; Wed, 14 Jun 2023 05:26:10 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20221208; t=1686745569; x=1689337569; h=cc:to:from:subject:message-id:mime-version:date:from:to:cc:subject :date:message-id:reply-to; bh=yCzCiWoRELswxyKR3MRcDLOgrFXaFbCkfTHkOK7uwD0=; b=gllsBxNq6gokE1zbrq87NmmsaIIb3mTAtMg7dO3GDuDj14v7zFtVy4Lcq4BZLStnZh hoP5wHluCVhHdkY/oo+tU6vjDyKxLcLLjj9jNXCDdXeq9/ISRg5MBtFyVdjjnNkdUPzL oCCHGVo1oanf2kaO2CroU/lWg9/MIshWsJTbsrw82EgF12g+6vvkCqfMAl5CqkIVlUIK VjL4P+LgD8QRi5l3dqlQFdTq0IK+pCCvORdRHncrJ4ssTn+VqLNrmRC5Cy0q/iZ3cgTv rrkAOGHzScUdeeZxDhGcxERAPcY4ZVU7KOQKXCUOo3roAVnOPGDn1Sjf7Z9mAM7FkOu4 qgeA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1686745569; x=1689337569; h=cc:to:from:subject:message-id:mime-version:date:x-gm-message-state :from:to:cc:subject:date:message-id:reply-to; bh=yCzCiWoRELswxyKR3MRcDLOgrFXaFbCkfTHkOK7uwD0=; b=C+SEJhdjDgKHfn2s1nKgmjGdgnVgQCHFAwxlXVurcGAbnMCRugGHmccPixqMP0u45K zUOMOy6gfYk/iZuGZTqiEeKEub5RyNfaFwM7CHcce5bEEgfpgYJ0p9CgQoiUa7PTH05A TW9Z36ACy3Sv7nm5j4CmBnK7Y93LL8oxSx1M2LRmWb5/r6h9MkqH9n+l6bMQPDFua4At ehgMgDeNBY/88rPQe9vJbEVkXQlj10sa9r41cmgi5tFD31tNa22yBuT2FFYThwXl2UYr O5M4f5q8ywhIcranl1vHNiqBhqZXazSYPLENRXAKnix8FWhcAcf8d6hGvI3QOR+ff2bH +HOQ== X-Gm-Message-State: AC+VfDze3GP6mSuP4rrgmhRfcurwlCgClrTywmPiwcGJiI6E6tcBYiOP Bc2LigkWRpAl/+eP4qfa2YXUOK4r35RE2w== X-Google-Smtp-Source: ACHHUZ4uuMjHNfCmftPMbMtzKVaT5lwd5w6tsM1f8YtmMu9PT5/eEj8tScBG8lmCSia5m2Ir13MR585drZdnKw== X-Received: from mostafa.c.googlers.com ([fda3:e722:ac3:cc00:28:9cb1:c0a8:333c]) (user=smostafa job=sendgmr) by 2002:a2e:7310:0:b0:2ad:7f86:44e2 with SMTP id o16-20020a2e7310000000b002ad7f8644e2mr3059289ljc.9.1686745568914; Wed, 14 Jun 2023 05:26:08 -0700 (PDT) Date: Wed, 14 Jun 2023 12:25:59 +0000 Mime-Version: 1.0 X-Mailer: git-send-email 2.41.0.162.gfafddb0af9-goog Message-ID: <20230614122600.2098901-1-smostafa@google.com> Subject: [PATCH v2] KVM: arm64: Use different pointer authentication keys for pKVM From: Mostafa Saleh To: maz@kernel.org, oliver.upton@linux.dev, linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev, linux-kernel@vger.kernel.org Cc: tabba@google.com, kaleshsingh@google.com, will@kernel.org, catalin.marinas@arm.com, yuzenghui@huawei.com, suzuki.poulose@arm.com, james.morse@arm.com, Mostafa Saleh X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20230614_052612_361271_566C32D8 X-CRM114-Status: GOOD ( 19.07 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org When the use of pointer authentication is enabled in the kernel it applies to both the kernel itself as well as KVM's nVHE hypervisor. The same keys are used for both the kernel and the nVHE hypervisor, which is less than desirable for pKVM as the host is not trusted at runtime. Naturally, the fix is to use a different set of keys for the hypervisor when running in protected mode. Have the host generate a new set of keys for the hypervisor before deprivileging the kernel. While there might be other sources of random directly available at EL2, this keeps the implementation simple, and the host is trusted anyways until it is deprivileged. Since the host and hypervisor no longer share a set of pointer authentication keys, start context switching them on the host entry/exit path exactly as we do for guest entry/exit. There is no need to handle CPU migration as the nVHE code is not migratable in the first place. Signed-off-by: Mostafa Saleh --- v2: - Add missing isb after updating hyp keys. - Reword commit message from Oliver. --- arch/arm64/kvm/arm.c | 26 ++++++++++++++++++++++++ arch/arm64/kvm/hyp/nvhe/host.S | 36 +++++++++++++++++++++++++++++++++- 2 files changed, 61 insertions(+), 1 deletion(-) diff --git a/arch/arm64/kvm/arm.c b/arch/arm64/kvm/arm.c index 14391826241c..dd03b52f035d 100644 --- a/arch/arm64/kvm/arm.c +++ b/arch/arm64/kvm/arm.c @@ -51,6 +51,8 @@ DECLARE_KVM_HYP_PER_CPU(unsigned long, kvm_hyp_vector); DEFINE_PER_CPU(unsigned long, kvm_arm_hyp_stack_page); DECLARE_KVM_NVHE_PER_CPU(struct kvm_nvhe_init_params, kvm_init_params); +DECLARE_KVM_NVHE_PER_CPU(struct kvm_cpu_context, kvm_hyp_ctxt); + static bool vgic_present; static DEFINE_PER_CPU(unsigned char, kvm_arm_hardware_enabled); @@ -2067,6 +2069,26 @@ static int __init kvm_hyp_init_protection(u32 hyp_va_bits) return 0; } +static void pkvm_hyp_init_ptrauth(void) +{ + struct kvm_cpu_context *hyp_ctxt; + int cpu; + + for_each_possible_cpu(cpu) { + hyp_ctxt = per_cpu_ptr_nvhe_sym(kvm_hyp_ctxt, cpu); + hyp_ctxt->sys_regs[APIAKEYLO_EL1] = get_random_long(); + hyp_ctxt->sys_regs[APIAKEYHI_EL1] = get_random_long(); + hyp_ctxt->sys_regs[APIBKEYLO_EL1] = get_random_long(); + hyp_ctxt->sys_regs[APIBKEYHI_EL1] = get_random_long(); + hyp_ctxt->sys_regs[APDAKEYLO_EL1] = get_random_long(); + hyp_ctxt->sys_regs[APDAKEYHI_EL1] = get_random_long(); + hyp_ctxt->sys_regs[APDBKEYLO_EL1] = get_random_long(); + hyp_ctxt->sys_regs[APDBKEYHI_EL1] = get_random_long(); + hyp_ctxt->sys_regs[APGAKEYLO_EL1] = get_random_long(); + hyp_ctxt->sys_regs[APGAKEYHI_EL1] = get_random_long(); + } +} + /* Inits Hyp-mode on all online CPUs */ static int __init init_hyp_mode(void) { @@ -2228,6 +2250,10 @@ static int __init init_hyp_mode(void) kvm_hyp_init_symbols(); if (is_protected_kvm_enabled()) { + if (IS_ENABLED(CONFIG_ARM64_PTR_AUTH_KERNEL) && + cpus_have_const_cap(ARM64_HAS_ADDRESS_AUTH)) + pkvm_hyp_init_ptrauth(); + init_cpu_logical_map(); if (!init_psci_relay()) { diff --git a/arch/arm64/kvm/hyp/nvhe/host.S b/arch/arm64/kvm/hyp/nvhe/host.S index b6c0188c4b35..c87c63133e10 100644 --- a/arch/arm64/kvm/hyp/nvhe/host.S +++ b/arch/arm64/kvm/hyp/nvhe/host.S @@ -10,6 +10,7 @@ #include #include #include +#include .text @@ -37,10 +38,43 @@ SYM_FUNC_START(__host_exit) /* Save the host context pointer in x29 across the function call */ mov x29, x0 + +#ifdef CONFIG_ARM64_PTR_AUTH_KERNEL +alternative_if_not ARM64_HAS_ADDRESS_AUTH +b __skip_pauth_save +alternative_else_nop_endif + +alternative_if ARM64_KVM_PROTECTED_MODE + /* Save kernel ptrauth keys. */ + add x18, x29, #CPU_APIAKEYLO_EL1 + ptrauth_save_state x18, x19, x20 + + /* Use hyp keys. */ + adr_this_cpu x18, kvm_hyp_ctxt, x19 + add x18, x18, #CPU_APIAKEYLO_EL1 + ptrauth_restore_state x18, x19, x20 + isb +alternative_else_nop_endif +__skip_pauth_save: +#endif /* CONFIG_ARM64_PTR_AUTH_KERNEL */ + bl handle_trap - /* Restore host regs x0-x17 */ __host_enter_restore_full: + /* Restore kernel keys. */ +#ifdef CONFIG_ARM64_PTR_AUTH_KERNEL +alternative_if_not ARM64_HAS_ADDRESS_AUTH +b __skip_pauth_restore +alternative_else_nop_endif + +alternative_if ARM64_KVM_PROTECTED_MODE + add x18, x29, #CPU_APIAKEYLO_EL1 + ptrauth_restore_state x18, x19, x20 +alternative_else_nop_endif +__skip_pauth_restore: +#endif /* CONFIG_ARM64_PTR_AUTH_KERNEL */ + + /* Restore host regs x0-x17 */ ldp x0, x1, [x29, #CPU_XREG_OFFSET(0)] ldp x2, x3, [x29, #CPU_XREG_OFFSET(2)] ldp x4, x5, [x29, #CPU_XREG_OFFSET(4)]