From patchwork Mon Aug 9 15:24:28 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Quentin Perret X-Patchwork-Id: 12426801 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-17.5 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_ADSP_CUSTOM_MED,DKIM_SIGNED,DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, URIBL_BLOCKED,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 6857EC4338F for ; Mon, 9 Aug 2021 15:28:14 +0000 (UTC) Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 334FC60EDF for ; Mon, 9 Aug 2021 15:28:14 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.4.1 mail.kernel.org 334FC60EDF Authentication-Results: mail.kernel.org; dmarc=fail (p=reject dis=none) header.from=google.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:Cc:To:From:Subject:References: Mime-Version:Message-Id:In-Reply-To:Date:Reply-To:Content-ID: Content-Description:Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc :Resent-Message-ID:List-Owner; bh=IHpuEiENXAfu3cds7xynejLC0V1+0XHZiSXMjLZvWQw=; b=F0jSgkO58PbMwKxOBE02Cur1ZJ b3E9v0xPnZyKydYAipKX3+oNgcyrbcphcDkRdina8N0iQDBJAi9HXK+Lzu+M8rVKKY+Xqyq6Bcboo 8Ft4/DOgHse6TMeRRB6+gQSWrLqO4UUM5cfdUEVOO96YrlB3ey++WDcK6ZgPemPyCVwZUBPSYG3OZ pCRY/rkb5YeOsycip3ZNJgoL5DsYGWFnwmZniGQy3Opy8ZXONGtV3VtikJEg8X6pD7Vx2O7vnqa3x Gs/Zo0j2DPNYaw80e/hFyOhF67uqW2YPxTjbjTOApeSF8OpHq+nlq28fyF2IexpRQqnjPmBx92Tm7 2sKwgCpw==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1mD79I-0017ta-My; Mon, 09 Aug 2021 15:25:08 +0000 Received: from mail-qk1-x749.google.com ([2607:f8b0:4864:20::749]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1mD795-0017pn-13 for linux-arm-kernel@lists.infradead.org; Mon, 09 Aug 2021 15:24:56 +0000 Received: by mail-qk1-x749.google.com with SMTP id h186-20020a37b7c30000b02903b914d9e335so12719706qkf.17 for ; Mon, 09 Aug 2021 08:24:53 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=date:in-reply-to:message-id:mime-version:references:subject:from:to :cc; bh=9BduUGnjsWum8FcOsM11gO3G+u8+79slhTeMYYKry4w=; b=g983I9YSVoWH/a7/cl0ZycwVjzqZeyo8x+fLCew59C7a0BnEu2wL6cysC50kAFs8Vi ehEdfCFNUnOWvVuDeMXGfwSfjkPO6irIfVDI1BOyVgOb2mistjn/3qc9aekGj3nyWfEP mv2IxEl7l/db3vL40dr7vcCKECuWFlGEQTsXsxS+OPHN8h4PW005Da05hkZ4QrGVi6/c iy5E1D2xv8HABlnaCg5SV0NZvQskMmKX3vG7+v7NRPhL/iuVTDX753+y/AxDWOc3l6hM igtE3JtznbdG+M1JR7b47U5TTSHB4/v8dAR/v0CBrRKmc9w4iH1rKAt2xMXY4Rmb58AP IwiA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=9BduUGnjsWum8FcOsM11gO3G+u8+79slhTeMYYKry4w=; b=m585Y+tqtByawvBwNqLbCW7YjQgK1xphnWCOPXxIPof0WKUlPQGC99R3CwJaYndn6v c1af6x3v/yrx7TrMXsYXRYlqyt5VZJobNOpzS4ExFuhMAvQNKb+EJicxxuUQ/95enj5J pvGRDC97Rts8UExEvVJKsva9JuKfC6rFcH2/oeLUsjqXJwdX3wxMtrZfpykainckra/N eVtuWFMD9SfcPmGliiVhPGpUCW+hLINx6rxGc9g7WfdYeKAMOPsrM2oeWwKJ4ArAvvih PYwuQ6FWcXjR1aJQi3lbP93dE+hVxiE53TQhai+/gPry+vvj7xIp8eiVK5YjRcWrEx67 zYeA== X-Gm-Message-State: AOAM530L6jYkwFGwBXYW6l+Wo2VPhPzQ/Q0O4FhpS8F4siQclJhn7ZnS yTdBCB1tOBOnlAZNx5KI02+1Q4ly+mIi X-Google-Smtp-Source: ABdhPJxcZ9biebm1fDKv+UMiqbVL5l0nCshM5eLfjTeBhNH2YYSYI90HGbAFPRYEaW9dglJFBj8tlcbiknXL X-Received: from luke.lon.corp.google.com ([2a00:79e0:d:210:b0e8:d460:758b:a0ae]) (user=qperret job=sendgmr) by 2002:a0c:e609:: with SMTP id z9mr9405599qvm.37.1628522692904; Mon, 09 Aug 2021 08:24:52 -0700 (PDT) Date: Mon, 9 Aug 2021 16:24:28 +0100 In-Reply-To: <20210809152448.1810400-1-qperret@google.com> Message-Id: <20210809152448.1810400-2-qperret@google.com> Mime-Version: 1.0 References: <20210809152448.1810400-1-qperret@google.com> X-Mailer: git-send-email 2.32.0.605.g8dce9f2422-goog Subject: [PATCH v4 01/21] KVM: arm64: Add hyp_spin_is_locked() for basic locking assertions at EL2 From: Quentin Perret To: maz@kernel.org, james.morse@arm.com, alexandru.elisei@arm.com, suzuki.poulose@arm.com, catalin.marinas@arm.com, will@kernel.org Cc: linux-arm-kernel@lists.infradead.org, kvmarm@lists.cs.columbia.edu, linux-kernel@vger.kernel.org, ardb@kernel.org, qwandor@google.com, tabba@google.com, dbrazdil@google.com, kernel-team@android.com, qperret@google.com X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20210809_082455_115915_582EC312 X-CRM114-Status: GOOD ( 11.70 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org From: Will Deacon Introduce hyp_spin_is_locked() so that functions can easily assert that a given lock is held (albeit possibly by another CPU!) without having to drag full lockdep support up to EL2. Signed-off-by: Will Deacon Signed-off-by: Quentin Perret --- arch/arm64/kvm/hyp/include/nvhe/spinlock.h | 8 ++++++++ 1 file changed, 8 insertions(+) diff --git a/arch/arm64/kvm/hyp/include/nvhe/spinlock.h b/arch/arm64/kvm/hyp/include/nvhe/spinlock.h index 76b537f8d1c6..04f65b655fcf 100644 --- a/arch/arm64/kvm/hyp/include/nvhe/spinlock.h +++ b/arch/arm64/kvm/hyp/include/nvhe/spinlock.h @@ -15,6 +15,7 @@ #include #include +#include typedef union hyp_spinlock { u32 __val; @@ -89,4 +90,11 @@ static inline void hyp_spin_unlock(hyp_spinlock_t *lock) : "memory"); } +static inline bool hyp_spin_is_locked(hyp_spinlock_t *lock) +{ + hyp_spinlock_t lockval = READ_ONCE(*lock); + + return lockval.owner != lockval.next; +} + #endif /* __ARM64_KVM_NVHE_SPINLOCK_H__ */ From patchwork Mon Aug 9 15:24:29 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Quentin Perret X-Patchwork-Id: 12426803 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-17.5 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_ADSP_CUSTOM_MED,DKIM_SIGNED,DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, URIBL_BLOCKED,USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 73127C4338F for ; Mon, 9 Aug 2021 15:28:17 +0000 (UTC) Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 3439B60EDF for ; Mon, 9 Aug 2021 15:28:17 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.4.1 mail.kernel.org 3439B60EDF Authentication-Results: mail.kernel.org; dmarc=fail (p=reject dis=none) header.from=google.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:Cc:To:From:Subject:References: Mime-Version:Message-Id:In-Reply-To:Date:Reply-To:Content-ID: Content-Description:Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc :Resent-Message-ID:List-Owner; bh=F/lS3ON9nGUp7CS+8zWR2P6T7nAA534ScvqzB/sSuBI=; b=KqFakroiEQJ+ElAsGCoKsUAhzw rksJUpNSvDz62M5ODjt1/f9bbqe4kDrp0LJ/Qx5CAN5HMX5MPg5Xf9GTml6rm45bfQ6d+glgB9dfd FSpJDD+3fHXn0U92sxomX9xO1of9URNgnchr8k/7AH7XQGYYmBp1i4k0lbW34PUVp2vkFuE4lzeuX e1gdoPlZ/T5eVNPX/KU9XHjYBhRwdLOQIS6k+xbF1k4bBRdEDbTrlB3H11J4/ib5VSCvCRqnyrIvh zJH+8B0PK74vwJMqoK5grrOZGRHEHe1agVUTk+/zCNL7mz71JY1TGuf3IQTLtdFXzQvZQ8C+XKitW aXCUzf5g==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1mD79W-0017yB-RY; Mon, 09 Aug 2021 15:25:23 +0000 Received: from mail-qv1-xf49.google.com ([2607:f8b0:4864:20::f49]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1mD797-0017qQ-Pf for linux-arm-kernel@lists.infradead.org; Mon, 09 Aug 2021 15:24:59 +0000 Received: by mail-qv1-xf49.google.com with SMTP id z25-20020a0ca9590000b029033ba243ffa1so12619350qva.0 for ; Mon, 09 Aug 2021 08:24:56 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=date:in-reply-to:message-id:mime-version:references:subject:from:to :cc; bh=BeIaolisaUZtGeC9k2EMLoK7VySoIN8YqNGAmtfkvk0=; b=iotHDKdGhx5vD9VI+nB2hXHLhXQLu/dD/nLQap0ZXipIAdjHfTrBtWxzjBLF27fJvX V8y2aGlVI3WevnGv3a/oygvB8DWA83QPRUnoFEaP2mA/jZ5OzBE3dKjBkI9lUapxC4nI kvU0G+t3hmhMJm5nfdoXxXB7+P0mv3iMd/wNs0oYDeM9lA/FwvqtmAfoJDexoKDFfwCM /KfNeDHdg0hyO5xlCEKkrzl90VhfWtHnfpPf+mjOAQJeoMnDgQiNXR4arITrxhV1tsQO oADMUZZt36O4VZv5wDHUL3HnQX+JwOi7hViCI1DDa2kxcUEA214zGyr/oVnSEsHC2Sef V3cw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=BeIaolisaUZtGeC9k2EMLoK7VySoIN8YqNGAmtfkvk0=; b=f60pW23M0rysuGOtXZOUlh+tYmc/R+lMjOrY6fAfrhIL2lst7oc7Sy/jOHy4xjqgGO G7U3ftphLnYtNZO060RGWrwps0gKYUrMcWHNcYcMyJ8eZDOgxfmUBeH9sFmukhBHr9cx fmSGVqdtTlxVHycy6ew9EVC33SIE6RD4mgqPRg9cFliFmDJnW/rODLwJ7JQXH9nfY/JC WsOapy4dsgATdczQgi/3U5Zx+CaPEb83NUFfplLodWaybvcLeOSllF8pZaImWrI8VD59 X5JU6xFCgFn54sqsD3c36JjhEk46xbmXox+LczlhYgIYmQWdy1KWMB7wENqX+GeEqS8r Tr5w== X-Gm-Message-State: AOAM531/jIjwTkc5QyMWGs4I1TVKxnGZsy2vhN1YPVlDKwLHBYP/XabO J8nZZFQP+RQp420QXjWFZFc0Tl16iSmB X-Google-Smtp-Source: ABdhPJwwkJx0vgQujLnN6O80r1QMwphuw2OhWcj7Ct18NZVCBAh2Ykl3Y1mymMGph4BEdeJQyMlOnYmLV0m0 X-Received: from luke.lon.corp.google.com ([2a00:79e0:d:210:b0e8:d460:758b:a0ae]) (user=qperret job=sendgmr) by 2002:ad4:59c5:: with SMTP id el5mr10407143qvb.43.1628522695300; Mon, 09 Aug 2021 08:24:55 -0700 (PDT) Date: Mon, 9 Aug 2021 16:24:29 +0100 In-Reply-To: <20210809152448.1810400-1-qperret@google.com> Message-Id: <20210809152448.1810400-3-qperret@google.com> Mime-Version: 1.0 References: <20210809152448.1810400-1-qperret@google.com> X-Mailer: git-send-email 2.32.0.605.g8dce9f2422-goog Subject: [PATCH v4 02/21] KVM: arm64: Introduce hyp_assert_lock_held() From: Quentin Perret To: maz@kernel.org, james.morse@arm.com, alexandru.elisei@arm.com, suzuki.poulose@arm.com, catalin.marinas@arm.com, will@kernel.org Cc: linux-arm-kernel@lists.infradead.org, kvmarm@lists.cs.columbia.edu, linux-kernel@vger.kernel.org, ardb@kernel.org, qwandor@google.com, tabba@google.com, dbrazdil@google.com, kernel-team@android.com, qperret@google.com X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20210809_082457_887909_80FC3D50 X-CRM114-Status: GOOD ( 13.86 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Introduce a poor man's lockdep implementation at EL2 which allows to BUG() whenever a hyp spinlock is not held when it should. Hide this feature behind a new Kconfig option that targets the EL2 object specifically, instead of piggy backing on the existing CONFIG_LOCKDEP. EL2 cannot WARN() cleanly to report locking issues, hence BUG() is the only option and it is not clear whether we want this widely enabled. This is most likely going to be useful for local testing until the EL2 WARN() situation has improved. Signed-off-by: Quentin Perret --- arch/arm64/kvm/Kconfig | 9 +++++++++ arch/arm64/kvm/hyp/include/nvhe/spinlock.h | 17 +++++++++++++++++ 2 files changed, 26 insertions(+) diff --git a/arch/arm64/kvm/Kconfig b/arch/arm64/kvm/Kconfig index a4eba0908bfa..9b9721895e5c 100644 --- a/arch/arm64/kvm/Kconfig +++ b/arch/arm64/kvm/Kconfig @@ -46,6 +46,15 @@ if KVM source "virt/kvm/Kconfig" +config NVHE_EL2_DEBUG + bool "Debug mode for non-VHE EL2 object" + help + Say Y here to enable the debug mode for the non-VHE KVM EL2 object. + Failure reports will BUG() in the hypervisor. This is intended for + local EL2 hypervisor development. + + If unsure, say N. + endif # KVM endif # VIRTUALIZATION diff --git a/arch/arm64/kvm/hyp/include/nvhe/spinlock.h b/arch/arm64/kvm/hyp/include/nvhe/spinlock.h index 04f65b655fcf..4652fd04bdbe 100644 --- a/arch/arm64/kvm/hyp/include/nvhe/spinlock.h +++ b/arch/arm64/kvm/hyp/include/nvhe/spinlock.h @@ -97,4 +97,21 @@ static inline bool hyp_spin_is_locked(hyp_spinlock_t *lock) return lockval.owner != lockval.next; } +#ifdef CONFIG_NVHE_EL2_DEBUG +static inline void hyp_assert_lock_held(hyp_spinlock_t *lock) +{ + /* + * The __pkvm_init() path accesses protected data-structures without + * holding locks as the other CPUs are guaranteed to not enter EL2 + * concurrently at this point in time. The point by which EL2 is + * initialized on all CPUs is reflected in the pkvm static key, so + * wait until it is set before checking the lock state. + */ + if (static_branch_likely(&kvm_protected_mode_initialized)) + BUG_ON(!hyp_spin_is_locked(lock)); +} +#else +static inline void hyp_assert_lock_held(hyp_spinlock_t *lock) { } +#endif + #endif /* __ARM64_KVM_NVHE_SPINLOCK_H__ */ From patchwork Mon Aug 9 15:24:30 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Quentin Perret X-Patchwork-Id: 12426793 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-17.5 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_ADSP_CUSTOM_MED,DKIM_SIGNED,DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, URIBL_BLOCKED,USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id C8101C432BE for ; Mon, 9 Aug 2021 15:27:35 +0000 (UTC) Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 7BECB60EBB for ; Mon, 9 Aug 2021 15:27:35 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.4.1 mail.kernel.org 7BECB60EBB Authentication-Results: mail.kernel.org; dmarc=fail (p=reject dis=none) header.from=google.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:Cc:To:From:Subject:References: Mime-Version:Message-Id:In-Reply-To:Date:Reply-To:Content-ID: Content-Description:Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc :Resent-Message-ID:List-Owner; bh=FDDCHIiifPt42yA8+y6fai5bRUuFkMUfkHEDwICX4C4=; b=TAiNJdjQEndItvfaYlJkGYPz4+ gSn36GmhbEl5Jk5HB1xAPgEpzjS1pWgzL1/ystknSskmqIrpEuzIXSlxjC+ZDC3R3jAN49LRcTkNX Hrp7UpZMmyIyX46glNa1mPcilq0pIfQBvzJWuKXN+rGvaKIJ3veM0+qSn1txviY8+oWCmkkdgvcsd 0JkGoV/afuTelpXe8+F63vZLPQrPqumx8/GBV7I+VTSLA9DyG4hyCfNJQoinbDJxzHvVUK+W5OLVo pK+DouHcJVWpqZ4oZ2a13EDA1XNLjrMv/uy+c7EFUKt5Wt80TtB+8Igpq6wBbgvj527d54DGxshoO J8xMeShw==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1mD79m-00186T-Un; Mon, 09 Aug 2021 15:25:39 +0000 Received: from mail-wr1-x449.google.com ([2a00:1450:4864:20::449]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1mD79A-0017r0-5R for linux-arm-kernel@lists.infradead.org; Mon, 09 Aug 2021 15:25:01 +0000 Received: by mail-wr1-x449.google.com with SMTP id v18-20020adfe2920000b029013bbfb19640so5452087wri.17 for ; Mon, 09 Aug 2021 08:24:58 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=date:in-reply-to:message-id:mime-version:references:subject:from:to :cc; bh=g0Tec8zCIIV6B7UFt/na0xdI02dI+qYcG36GmUIxQNs=; b=u/8T1pZALwA4/sFdX62scy5dCtljsLkrBi8I1Wo5YlC53+ou/YjrvDL8JNsEIB71bE LRivkMFVgCH0BtjYzhigCs/bBsZnzm1aZdrrbJrAhXN2dDwjESl94VxP5qszcWj4CCEd kb6ZNqQAPWs3owfE5gBV3uIxdOads4TKkeSBC+CEa86jU1FkSsXByuLM+TGmtl+JwI1s V5+s+6x3trNBHDLQF9LXi2sMfrQr2qHgz74SlBLo1h0tqRcsn8LkQJbmZFl6TVJkfNSa l+1c8KgXcM3JN+xcOybD93Qo/Y8JcdC1KdNJjrp98+bqF+lnmdb8LFXpm1F/xsbnlAZG wATw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=g0Tec8zCIIV6B7UFt/na0xdI02dI+qYcG36GmUIxQNs=; b=aGPaA9NSafWeALoDCG84iL8goP1uHlHrHIf4UOP7J32mOR4JaFb8iqCxE3dG2N5Hb0 KyyeS4/69Lr1yK82Ort7P26XNzVgXjiQNX9hJOhxTw7IugkufCwri0/2aDVVFJZD1Z7U c8MbWYrTDSDbeCji8w+HaqMhJ2Id1qwifPDw7ybIYVIrzNdBNFi/LqaeuqBQ/CurAIde 4VXoLSXfGy800iI9WLbJpfn5gaE1u3Rk/ArPnDeKNlPGpojdabe4TfCVnqrIVdVtgSvT 4Y0RKfYXxP95nzpq30OVwDcc2HuqSna350SORn9cG1As5zrmMt+ml7qYR3wUQrpdxIJe EQKQ== X-Gm-Message-State: AOAM532wfE49H6pZcGxiRND618ux6CixXUadhF5gjIqpVf3qBrde2Th3 wjnf3Lx2vBJZ5IxUWtqyPc6gyXUcwcUC X-Google-Smtp-Source: ABdhPJwdhbneOWgiW5/bK5oKtxthf7W2lS34xOPxnKsvszEVciXZLPa8/F4hD9ZjXEHPO3uTXXdiY7U47vX5 X-Received: from luke.lon.corp.google.com ([2a00:79e0:d:210:b0e8:d460:758b:a0ae]) (user=qperret job=sendgmr) by 2002:a05:6000:232:: with SMTP id l18mr25664566wrz.24.1628522697474; Mon, 09 Aug 2021 08:24:57 -0700 (PDT) Date: Mon, 9 Aug 2021 16:24:30 +0100 In-Reply-To: <20210809152448.1810400-1-qperret@google.com> Message-Id: <20210809152448.1810400-4-qperret@google.com> Mime-Version: 1.0 References: <20210809152448.1810400-1-qperret@google.com> X-Mailer: git-send-email 2.32.0.605.g8dce9f2422-goog Subject: [PATCH v4 03/21] KVM: arm64: Provide the host_stage2_try() helper macro From: Quentin Perret To: maz@kernel.org, james.morse@arm.com, alexandru.elisei@arm.com, suzuki.poulose@arm.com, catalin.marinas@arm.com, will@kernel.org Cc: linux-arm-kernel@lists.infradead.org, kvmarm@lists.cs.columbia.edu, linux-kernel@vger.kernel.org, ardb@kernel.org, qwandor@google.com, tabba@google.com, dbrazdil@google.com, kernel-team@android.com, qperret@google.com X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20210809_082500_283922_80663062 X-CRM114-Status: GOOD ( 16.73 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org We currently unmap all MMIO mappings from the host stage-2 to recycle the pages whenever we run out. In order to make this pattern easy to re-use from other places, factor the logic out into a dedicated macro. While at it, apply the macro for the kvm_pgtable_stage2_set_owner() calls. They're currently only called early on and are guaranteed to succeed, but making them robust to the -ENOMEM case doesn't hurt and will avoid painful debugging sessions later on. Reviewed-by: Fuad Tabba Signed-off-by: Quentin Perret --- arch/arm64/kvm/hyp/nvhe/mem_protect.c | 40 +++++++++++++++------------ 1 file changed, 22 insertions(+), 18 deletions(-) diff --git a/arch/arm64/kvm/hyp/nvhe/mem_protect.c b/arch/arm64/kvm/hyp/nvhe/mem_protect.c index d938ce95d3bd..74280a753efb 100644 --- a/arch/arm64/kvm/hyp/nvhe/mem_protect.c +++ b/arch/arm64/kvm/hyp/nvhe/mem_protect.c @@ -208,6 +208,25 @@ static inline int __host_stage2_idmap(u64 start, u64 end, prot, &host_s2_pool); } +/* + * The pool has been provided with enough pages to cover all of memory with + * page granularity, but it is difficult to know how much of the MMIO range + * we will need to cover upfront, so we may need to 'recycle' the pages if we + * run out. + */ +#define host_stage2_try(fn, ...) \ + ({ \ + int __ret; \ + hyp_assert_lock_held(&host_kvm.lock); \ + __ret = fn(__VA_ARGS__); \ + if (__ret == -ENOMEM) { \ + __ret = host_stage2_unmap_dev_all(); \ + if (!__ret) \ + __ret = fn(__VA_ARGS__); \ + } \ + __ret; \ + }) + static int host_stage2_idmap(u64 addr) { enum kvm_pgtable_prot prot = KVM_PGTABLE_PROT_R | KVM_PGTABLE_PROT_W; @@ -223,22 +242,7 @@ static int host_stage2_idmap(u64 addr) if (ret) goto unlock; - ret = __host_stage2_idmap(range.start, range.end, prot); - if (ret != -ENOMEM) - goto unlock; - - /* - * The pool has been provided with enough pages to cover all of memory - * with page granularity, but it is difficult to know how much of the - * MMIO range we will need to cover upfront, so we may need to 'recycle' - * the pages if we run out. - */ - ret = host_stage2_unmap_dev_all(); - if (ret) - goto unlock; - - ret = __host_stage2_idmap(range.start, range.end, prot); - + ret = host_stage2_try(__host_stage2_idmap, range.start, range.end, prot); unlock: hyp_spin_unlock(&host_kvm.lock); @@ -257,8 +261,8 @@ int __pkvm_mark_hyp(phys_addr_t start, phys_addr_t end) return -EINVAL; hyp_spin_lock(&host_kvm.lock); - ret = kvm_pgtable_stage2_set_owner(&host_kvm.pgt, start, end - start, - &host_s2_pool, pkvm_hyp_id); + ret = host_stage2_try(kvm_pgtable_stage2_set_owner, &host_kvm.pgt, + start, end - start, &host_s2_pool, pkvm_hyp_id); hyp_spin_unlock(&host_kvm.lock); return ret != -EAGAIN ? ret : 0; From patchwork Mon Aug 9 15:24:31 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Quentin Perret X-Patchwork-Id: 12426795 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-17.5 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_ADSP_CUSTOM_MED,DKIM_SIGNED,DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, URIBL_BLOCKED,USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 4C350C4338F for ; Mon, 9 Aug 2021 15:27:41 +0000 (UTC) Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 16D6B6101D for ; Mon, 9 Aug 2021 15:27:41 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.4.1 mail.kernel.org 16D6B6101D Authentication-Results: mail.kernel.org; dmarc=fail (p=reject dis=none) header.from=google.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:Cc:To:From:Subject:References: Mime-Version:Message-Id:In-Reply-To:Date:Reply-To:Content-ID: Content-Description:Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc :Resent-Message-ID:List-Owner; bh=+M5HZBCN3/6JDBu71w4tktAsrGkfS2/6fC0TPjLPD/8=; b=ObOExywhjpoeg+nQ/alpzHhX8A tLN5fwog5xXVm4w+avHC3K0rs4Wu9ImuZV7173J5QXJhJxQ18Dpv0OFFHnwDP2iOAVpltFW0ka4YV EG7gYsf3cdfbewQSpt2lBLLLIYbA6EcF/Y1ZAeA4k+/Wqv++pZG8ZTUQJyPubb4wrSukny+jKtuX6 i0QaDjyw75MJ16kK1WMhH4gcZ3y7Rxmk+epFOMOPOPtYlBN4B2Tyt0OFsIfm4Ahvc9rW9RFJ6/1Ry b/PSppiRHpasPiaQe9PxiYlddaTYHGCrzDvjz/tCEpER65v4oNL+9+Sy5Op3sZLf3TQnlKlEN4tQo sI+BDEkw==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1mD7A2-0018CA-I8; Mon, 09 Aug 2021 15:25:54 +0000 Received: from mail-ed1-x549.google.com ([2a00:1450:4864:20::549]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1mD79C-0017ro-Er for linux-arm-kernel@lists.infradead.org; Mon, 09 Aug 2021 15:25:03 +0000 Received: by mail-ed1-x549.google.com with SMTP id j15-20020aa7c40f0000b02903be5fbe68a9so2435269edq.2 for ; Mon, 09 Aug 2021 08:25:01 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=date:in-reply-to:message-id:mime-version:references:subject:from:to :cc; bh=GMZNfzuPpqQplgJREqrbVT8SGjVRH/ICHDZ3APociKw=; b=UHGjFJ88t++uDbi+ho+q/XdFtIa+W/7vj7rOtKa5ztsftv96V3hJF8oNqfWIZeOQaL aUP+VWMsxv/Ww6yCE/11VKrtsrFRo4r1jE9zZCcHCxeW4QTXs3Hf0eg59+DEpi0AsG9+ pQpwB2Zfd62iUZFWqRr7WkBLq6/W/YQO57q79BUO2PTtMUj5uiQIu4UTW0GFsdzup9UE 9v9sMuWxhogJDOCknqN7ch+GClEeK3D78t6XMjW2bjoowOIRZuE8PbSwxbSu3F3AkfXj J5a/V4E7m6hSw9dq9XYFJeT/AzeYnLZo6tEmPEs60tS9JdK639KAKwpr/ZkQ/mj98px6 3hRw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=GMZNfzuPpqQplgJREqrbVT8SGjVRH/ICHDZ3APociKw=; b=oe2F1p4zzLulDHQdrjDOW/62jKnR6dVLDUuq2M8/yF4w0Ui0t6tpJpHa95KI1+sQPE Utr4xlPNQ3ygKgerZctojt5kQt3lfOpEe0ZTdDZ/SpNfO1LtiFfasBNv3cl4ZpJrERNc ERz0W2fJ8ouQpN/jye/psFtWKCNaUBkhMbTtngcgW+RACWjb/4ruX5uVLUotzvWnoujl dpUBxcpo93afHnGe7xyD0+Dbq/KFnW3VlbTISH3JLSW2Ycn5YN+dd7fmQeyT9scmQ7xI x5qNTW3IfTCCf76/zhzK7PQ9VIIv7bMNUHehFkCvDjMpt9SnUA9Jep36VoMK7zqI6zMR Dx4g== X-Gm-Message-State: AOAM530vzfqXG7lbP93LAf6GT16QRRGSnBz43HUTHTiW70CYWwyjmxcW eHsmqm/xmtX8UFacI1ofkir2V7Vo91wJ X-Google-Smtp-Source: ABdhPJyqzY0L1dCaL3AwOH4Fm7LMkYjK+OM+7knyOMrkv2ibGaXiqJGzChD13pqf4rXPj/gIvGULi/3nex3K X-Received: from luke.lon.corp.google.com ([2a00:79e0:d:210:b0e8:d460:758b:a0ae]) (user=qperret job=sendgmr) by 2002:aa7:d899:: with SMTP id u25mr30548023edq.151.1628522699812; Mon, 09 Aug 2021 08:24:59 -0700 (PDT) Date: Mon, 9 Aug 2021 16:24:31 +0100 In-Reply-To: <20210809152448.1810400-1-qperret@google.com> Message-Id: <20210809152448.1810400-5-qperret@google.com> Mime-Version: 1.0 References: <20210809152448.1810400-1-qperret@google.com> X-Mailer: git-send-email 2.32.0.605.g8dce9f2422-goog Subject: [PATCH v4 04/21] KVM: arm64: Introduce helper to retrieve a PTE and its level From: Quentin Perret To: maz@kernel.org, james.morse@arm.com, alexandru.elisei@arm.com, suzuki.poulose@arm.com, catalin.marinas@arm.com, will@kernel.org Cc: linux-arm-kernel@lists.infradead.org, kvmarm@lists.cs.columbia.edu, linux-kernel@vger.kernel.org, ardb@kernel.org, qwandor@google.com, tabba@google.com, dbrazdil@google.com, kernel-team@android.com, qperret@google.com X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20210809_082502_560116_50F9312F X-CRM114-Status: GOOD ( 13.85 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org From: Marc Zyngier It is becoming a common need to fetch the PTE for a given address together with its level. Add such a helper. Signed-off-by: Marc Zyngier Signed-off-by: Quentin Perret Reviewed-by: Andrew Walbran --- arch/arm64/include/asm/kvm_pgtable.h | 19 ++++++++++++++ arch/arm64/kvm/hyp/pgtable.c | 39 ++++++++++++++++++++++++++++ 2 files changed, 58 insertions(+) diff --git a/arch/arm64/include/asm/kvm_pgtable.h b/arch/arm64/include/asm/kvm_pgtable.h index f004c0115d89..082b9d65f40b 100644 --- a/arch/arm64/include/asm/kvm_pgtable.h +++ b/arch/arm64/include/asm/kvm_pgtable.h @@ -432,6 +432,25 @@ int kvm_pgtable_stage2_flush(struct kvm_pgtable *pgt, u64 addr, u64 size); int kvm_pgtable_walk(struct kvm_pgtable *pgt, u64 addr, u64 size, struct kvm_pgtable_walker *walker); +/** + * kvm_pgtable_get_leaf() - Walk a page-table and retrieve the leaf entry + * with its level. + * @pgt: Page-table structure initialised by kvm_pgtable_*_init(). + * @addr: Input address for the start of the walk. + * @ptep: Pointer to storage for the retrieved PTE. + * @level: Pointer to storage for the level of the retrieved PTE. + * + * The offset of @addr within a page is ignored. + * + * The walker will walk the page-table entries corresponding to the input + * address specified, retrieving the leaf corresponding to this address. + * Invalid entries are treated as leaf entries. + * + * Return: 0 on success, negative error code on failure. + */ +int kvm_pgtable_get_leaf(struct kvm_pgtable *pgt, u64 addr, + kvm_pte_t *ptep, u32 *level); + /** * kvm_pgtable_stage2_find_range() - Find a range of Intermediate Physical * Addresses with compatible permission diff --git a/arch/arm64/kvm/hyp/pgtable.c b/arch/arm64/kvm/hyp/pgtable.c index 05321f4165e3..78f36bd5df6c 100644 --- a/arch/arm64/kvm/hyp/pgtable.c +++ b/arch/arm64/kvm/hyp/pgtable.c @@ -326,6 +326,45 @@ int kvm_pgtable_walk(struct kvm_pgtable *pgt, u64 addr, u64 size, return _kvm_pgtable_walk(&walk_data); } +struct leaf_walk_data { + kvm_pte_t pte; + u32 level; +}; + +static int leaf_walker(u64 addr, u64 end, u32 level, kvm_pte_t *ptep, + enum kvm_pgtable_walk_flags flag, void * const arg) +{ + struct leaf_walk_data *data = arg; + + data->pte = *ptep; + data->level = level; + + return 0; +} + +int kvm_pgtable_get_leaf(struct kvm_pgtable *pgt, u64 addr, + kvm_pte_t *ptep, u32 *level) +{ + struct leaf_walk_data data; + struct kvm_pgtable_walker walker = { + .cb = leaf_walker, + .flags = KVM_PGTABLE_WALK_LEAF, + .arg = &data, + }; + int ret; + + ret = kvm_pgtable_walk(pgt, ALIGN_DOWN(addr, PAGE_SIZE), + PAGE_SIZE, &walker); + if (!ret) { + if (ptep) + *ptep = data.pte; + if (level) + *level = data.level; + } + + return ret; +} + struct hyp_map_data { u64 phys; kvm_pte_t attr; From patchwork Mon Aug 9 15:24:32 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Quentin Perret X-Patchwork-Id: 12426797 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-17.5 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_ADSP_CUSTOM_MED,DKIM_SIGNED,DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, URIBL_BLOCKED,USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id BF88BC4320A for ; Mon, 9 Aug 2021 15:27:55 +0000 (UTC) Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 891A06101D for ; Mon, 9 Aug 2021 15:27:55 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.4.1 mail.kernel.org 891A06101D Authentication-Results: mail.kernel.org; dmarc=fail (p=reject dis=none) header.from=google.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:Cc:To:From:Subject:References: Mime-Version:Message-Id:In-Reply-To:Date:Reply-To:Content-ID: Content-Description:Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc :Resent-Message-ID:List-Owner; bh=RxXayfi2QtWY1DpVRoJkKKZYfOeRtqe08i9ufn2TA8Y=; b=tvmTeqpXiAFeUtPKlZ84dSzdn8 qhm++hKEEkd5DjRdcLH30Xg8dZvMOTZFmFODVsvj5R41dc/2NZoPnSoWoY4cEAkU26DF1X7Exp1ZL nJqxcynVj/02GkLySTrpMdHdpQs5WDWovpDgz/7yuVIuJcezPC9zVJAT1RoY4SJ3WhOeL6ryAcSBF Z1tdmF1UI17fGqPWGBrHBrc/okebYkml3RLMqiWy5pGmsmSw6XCshmEYvgkIiwLDZ74k7DSy+t5Y1 /8KtHuQPmz8lF91dHrJz9PnELgHFdvD0aGO8S74HrwBZUi/KlOLDMlLoZiLU+Nj/zQRze2SPODRXk eHpxLmtg==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1mD7AI-0018Ij-0G; Mon, 09 Aug 2021 15:26:10 +0000 Received: from mail-qk1-x74a.google.com ([2607:f8b0:4864:20::74a]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1mD79D-0017sJ-JD for linux-arm-kernel@lists.infradead.org; Mon, 09 Aug 2021 15:25:05 +0000 Received: by mail-qk1-x74a.google.com with SMTP id p16-20020a05620a22b0b02903ca40d6cc81so7465426qkh.13 for ; Mon, 09 Aug 2021 08:25:02 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=date:in-reply-to:message-id:mime-version:references:subject:from:to :cc; bh=nXWh1aBsmvvArSdynKqs/F6MGOyla+y37XGDQDAw0Zw=; b=cYfnUsIb5gGe+lWEdMn95RmDSxRUQoPfJdok4pDTRDFpRWfOU0ggbk9mfKP2zsxOUh GscOCxW3U2TBjao5XjoBagZImLZH1sNAULkmo2jw8VpjVzlpCivcuhOuCZwZPuvY+KCl hAD2kr6od+cAD1+GqOldj1CvFuTxUb7B8NZ51j4ufwBiHfiW6Tm4JyHxRwYCziDocagZ uoFADrALl5V/33jTieDKrHukSUoZfweS5IakvG1sJ5RrSD7eHFpVjbt0/Fh1sSvEF3c2 KqkKTxt+xL7wbhcTpQUvE+6MzgrWIlXHS4E06USqLnvqPIi0HL3+gAKM1SKKoqlrQGdF 8UsQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=nXWh1aBsmvvArSdynKqs/F6MGOyla+y37XGDQDAw0Zw=; b=tb37Su6QQUbzHXapK1rvCaFickko5E+7/h4kXPKlCJv6SK69tcSmI5EXkhjJJG9CUB Hdp1A8899o/DqZLJlcBKFYAM6KeSRvIktbHXL+nA2tkAD5LOj5jxdGiprHiUcWMkP2K/ o25gzELgcz6e1KpthYNBQ3unrAYdSfcHXiGlS13j4A2tc72EdREt7oJQywBJKXq5+Qmk 2JM8+sy+xYQxitswC4sb4r2+SHjvZPw67tniMaaN6W0/wpVCHQ1JcNixctQxcb4HD8IF f+WbfjXV9MisCjtLuvsKpuRXzUkA8jWLfhb7H/TjERYEUcGi0pgtlBUCZ70Gsz8IVYLs v10Q== X-Gm-Message-State: AOAM532lR3Ps6NCYjfEr32ztOluNjeGygJEo6lpFPYEffoeRATHfUFgc qL8eqMvEpel6UDCYyHl4nCxRCwxkb+02 X-Google-Smtp-Source: ABdhPJzInJfMmMfgDqEILlYqnmWDME8lUUPA1VgIQarM5NixXRbdPCHLe8fmcxdPmKC8j+G1kAF2nZVLpZyc X-Received: from luke.lon.corp.google.com ([2a00:79e0:d:210:b0e8:d460:758b:a0ae]) (user=qperret job=sendgmr) by 2002:a0c:fe6a:: with SMTP id b10mr24245399qvv.6.1628522702203; Mon, 09 Aug 2021 08:25:02 -0700 (PDT) Date: Mon, 9 Aug 2021 16:24:32 +0100 In-Reply-To: <20210809152448.1810400-1-qperret@google.com> Message-Id: <20210809152448.1810400-6-qperret@google.com> Mime-Version: 1.0 References: <20210809152448.1810400-1-qperret@google.com> X-Mailer: git-send-email 2.32.0.605.g8dce9f2422-goog Subject: [PATCH v4 05/21] KVM: arm64: Expose page-table helpers From: Quentin Perret To: maz@kernel.org, james.morse@arm.com, alexandru.elisei@arm.com, suzuki.poulose@arm.com, catalin.marinas@arm.com, will@kernel.org Cc: linux-arm-kernel@lists.infradead.org, kvmarm@lists.cs.columbia.edu, linux-kernel@vger.kernel.org, ardb@kernel.org, qwandor@google.com, tabba@google.com, dbrazdil@google.com, kernel-team@android.com, qperret@google.com X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20210809_082503_685622_E420BA46 X-CRM114-Status: GOOD ( 15.35 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org The KVM pgtable API exposes the kvm_pgtable_walk() function to allow the definition of walkers outside of pgtable.c. However, it is not easy to implement any of those walkers without some of the low-level helpers. Move some of them to the header file to allow re-use from other places. Signed-off-by: Quentin Perret Reviewed-by: Fuad Tabba --- arch/arm64/include/asm/kvm_pgtable.h | 40 ++++++++++++++++++++++++++++ arch/arm64/kvm/hyp/pgtable.c | 39 --------------------------- 2 files changed, 40 insertions(+), 39 deletions(-) diff --git a/arch/arm64/include/asm/kvm_pgtable.h b/arch/arm64/include/asm/kvm_pgtable.h index 082b9d65f40b..6938eac72c1f 100644 --- a/arch/arm64/include/asm/kvm_pgtable.h +++ b/arch/arm64/include/asm/kvm_pgtable.h @@ -25,6 +25,46 @@ static inline u64 kvm_get_parange(u64 mmfr0) typedef u64 kvm_pte_t; +#define KVM_PTE_VALID BIT(0) + +#define KVM_PTE_ADDR_MASK GENMASK(47, PAGE_SHIFT) +#define KVM_PTE_ADDR_51_48 GENMASK(15, 12) + +static inline bool kvm_pte_valid(kvm_pte_t pte) +{ + return pte & KVM_PTE_VALID; +} + +static inline u64 kvm_pte_to_phys(kvm_pte_t pte) +{ + u64 pa = pte & KVM_PTE_ADDR_MASK; + + if (PAGE_SHIFT == 16) + pa |= FIELD_GET(KVM_PTE_ADDR_51_48, pte) << 48; + + return pa; +} + +static inline u64 kvm_granule_shift(u32 level) +{ + /* Assumes KVM_PGTABLE_MAX_LEVELS is 4 */ + return ARM64_HW_PGTABLE_LEVEL_SHIFT(level); +} + +static inline u64 kvm_granule_size(u32 level) +{ + return BIT(kvm_granule_shift(level)); +} + +static inline bool kvm_level_supports_block_mapping(u32 level) +{ + /* + * Reject invalid block mappings and don't bother with 4TB mappings for + * 52-bit PAs. + */ + return !(level == 0 || (PAGE_SIZE != SZ_4K && level == 1)); +} + /** * struct kvm_pgtable_mm_ops - Memory management callbacks. * @zalloc_page: Allocate a single zeroed memory page. diff --git a/arch/arm64/kvm/hyp/pgtable.c b/arch/arm64/kvm/hyp/pgtable.c index 78f36bd5df6c..49d768b92997 100644 --- a/arch/arm64/kvm/hyp/pgtable.c +++ b/arch/arm64/kvm/hyp/pgtable.c @@ -11,16 +11,12 @@ #include #include -#define KVM_PTE_VALID BIT(0) #define KVM_PTE_TYPE BIT(1) #define KVM_PTE_TYPE_BLOCK 0 #define KVM_PTE_TYPE_PAGE 1 #define KVM_PTE_TYPE_TABLE 1 -#define KVM_PTE_ADDR_MASK GENMASK(47, PAGE_SHIFT) -#define KVM_PTE_ADDR_51_48 GENMASK(15, 12) - #define KVM_PTE_LEAF_ATTR_LO GENMASK(11, 2) #define KVM_PTE_LEAF_ATTR_LO_S1_ATTRIDX GENMASK(4, 2) @@ -61,17 +57,6 @@ struct kvm_pgtable_walk_data { u64 end; }; -static u64 kvm_granule_shift(u32 level) -{ - /* Assumes KVM_PGTABLE_MAX_LEVELS is 4 */ - return ARM64_HW_PGTABLE_LEVEL_SHIFT(level); -} - -static u64 kvm_granule_size(u32 level) -{ - return BIT(kvm_granule_shift(level)); -} - #define KVM_PHYS_INVALID (-1ULL) static bool kvm_phys_is_valid(u64 phys) @@ -79,15 +64,6 @@ static bool kvm_phys_is_valid(u64 phys) return phys < BIT(id_aa64mmfr0_parange_to_phys_shift(ID_AA64MMFR0_PARANGE_MAX)); } -static bool kvm_level_supports_block_mapping(u32 level) -{ - /* - * Reject invalid block mappings and don't bother with 4TB mappings for - * 52-bit PAs. - */ - return !(level == 0 || (PAGE_SIZE != SZ_4K && level == 1)); -} - static bool kvm_block_mapping_supported(u64 addr, u64 end, u64 phys, u32 level) { u64 granule = kvm_granule_size(level); @@ -135,11 +111,6 @@ static u32 kvm_pgd_pages(u32 ia_bits, u32 start_level) return __kvm_pgd_page_idx(&pgt, -1ULL) + 1; } -static bool kvm_pte_valid(kvm_pte_t pte) -{ - return pte & KVM_PTE_VALID; -} - static bool kvm_pte_table(kvm_pte_t pte, u32 level) { if (level == KVM_PGTABLE_MAX_LEVELS - 1) @@ -151,16 +122,6 @@ static bool kvm_pte_table(kvm_pte_t pte, u32 level) return FIELD_GET(KVM_PTE_TYPE, pte) == KVM_PTE_TYPE_TABLE; } -static u64 kvm_pte_to_phys(kvm_pte_t pte) -{ - u64 pa = pte & KVM_PTE_ADDR_MASK; - - if (PAGE_SHIFT == 16) - pa |= FIELD_GET(KVM_PTE_ADDR_51_48, pte) << 48; - - return pa; -} - static kvm_pte_t kvm_phys_to_pte(u64 pa) { kvm_pte_t pte = pa & KVM_PTE_ADDR_MASK; From patchwork Mon Aug 9 15:24:33 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Quentin Perret X-Patchwork-Id: 12426805 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-17.5 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_ADSP_CUSTOM_MED,DKIM_SIGNED,DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, URIBL_BLOCKED,USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id ECD39C4338F for ; Mon, 9 Aug 2021 15:28:30 +0000 (UTC) Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id ADB8C60F8F for ; Mon, 9 Aug 2021 15:28:30 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.4.1 mail.kernel.org ADB8C60F8F Authentication-Results: mail.kernel.org; dmarc=fail (p=reject dis=none) header.from=google.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:Cc:To:From:Subject:References: Mime-Version:Message-Id:In-Reply-To:Date:Reply-To:Content-ID: Content-Description:Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc :Resent-Message-ID:List-Owner; bh=o29Dr9BqLqtpUk+J/I/03l4B0AVWgRSKe0wI5ed/E5A=; b=dsHS6Xgwd8ELWTBa/ILhbDQFCB Zik+DN0TIowrUJGDRHtRY3RLQVMC0yC4KYL+myW2hp1GK7cL41t7FU14Yo+pJAAhelzV2RuTryS2G UVQtQf8Xk5+b/PVz6+0jX4q5z5wOHTaufV2Bh5CeO1WM8YH1eM/Cs0E7G9ZXGdhNDlWJCBuaEdY99 A/fmzYbnBF5BKii2EvjvPHkplQ1ZRETVfJU1R5kzdcbrC4lqvjFgdZrg0bwDFb7uJTDVGPJ9RgJSm lHs8G3FUo6/8xZPzsPYu+CXxIAvMD2z0wTheKTyg4YGLELgjXN7IHJe8e8e/MfcoRv8AcLxOrM1hz hbcq22Ew==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1mD7Ak-0018VR-OE; Mon, 09 Aug 2021 15:26:38 +0000 Received: from mail-ed1-x54a.google.com ([2a00:1450:4864:20::54a]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1mD79H-0017t7-3v for linux-arm-kernel@lists.infradead.org; Mon, 09 Aug 2021 15:25:09 +0000 Received: by mail-ed1-x54a.google.com with SMTP id u4-20020a50eac40000b02903bddc52675eso8885399edp.4 for ; Mon, 09 Aug 2021 08:25:05 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=date:in-reply-to:message-id:mime-version:references:subject:from:to :cc; bh=V6FXsDOTW6vekbs/rXCZ1qg9Zi2BQ8Ystt2E8nbUo50=; b=S39TUDBkvVCpPK7CjHvagYIL9jJP+3RrAyJHX7sacKU7lwHNVmQYS3eMcRisxm6eR7 9fRa6hm639YDKJFTrY8Qeo3xl2hld6cFffTwoFR3r28lRSSnydTdpEEU9BVg09olwL0f MnUZ4qiCBGGtB6kKqc+SMQ/03BzhaoG0UONxUot4IL/qGlAWr84cQVOKoxRK4iHBeevS jgXEcdJCeOyoEyb1anUFDCyGqsJjnLqyLNpMrNa0Lty6cbXVmnROrMNQyoQjAqX2FkA1 H5wa3f58y/1R21B2SEFLZGQHDp6MLiOuVEqXQZ8/QTOiKpOu7OlLS9T3OXW7hNtKarqh 4cdg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=V6FXsDOTW6vekbs/rXCZ1qg9Zi2BQ8Ystt2E8nbUo50=; b=c/Hm7GMC49G2fFMB0+YK5pg6k5NI1+h4nKtxXFTyG1MxrVJMP+s7hx5CrLYphF848V st+/jHc0k7RvsXNAeofVDjMpWO1piUoE9mE5zA68+c6739e3tsjcZHpApKjDKEZvlY+r +NFXCkxVv+UUdsGTCZtDkpUcCryi+Vz8XgcvXthUSG5paon4Rj1KVg7ONDV0VDCS43J7 s5QHkcwQPWHkbsrMSeK2Ts3/x3i17YsM6JZbQEYh1WSBgfjuUNlv3qI3KML24ABeAFsi LsxxywHzAcC4Y1Rn5nWIbwUd2HMSfxC33R2+C/u1DV7GEQz0fHwaCd3VCe5lkoOJajga JqTA== X-Gm-Message-State: AOAM532IO/pSBHf4OoGZW0sGeOpA57VqzhWWvTopgBtMZg1b+YhX603p cd1Ox1q3L21Q/FaONJj5j5Lo87+oEWAO X-Google-Smtp-Source: ABdhPJw9Ff26FcaF7JXa4LyfU3GCWK7Zy3K86lTJTMJcHdOfcHGrzRq10T4KxoJzQD+Vj7eG+XC/UGFBJRhv X-Received: from luke.lon.corp.google.com ([2a00:79e0:d:210:b0e8:d460:758b:a0ae]) (user=qperret job=sendgmr) by 2002:a05:6402:160c:: with SMTP id f12mr30717665edv.122.1628522704406; Mon, 09 Aug 2021 08:25:04 -0700 (PDT) Date: Mon, 9 Aug 2021 16:24:33 +0100 In-Reply-To: <20210809152448.1810400-1-qperret@google.com> Message-Id: <20210809152448.1810400-7-qperret@google.com> Mime-Version: 1.0 References: <20210809152448.1810400-1-qperret@google.com> X-Mailer: git-send-email 2.32.0.605.g8dce9f2422-goog Subject: [PATCH v4 06/21] KVM: arm64: Optimize host memory aborts From: Quentin Perret To: maz@kernel.org, james.morse@arm.com, alexandru.elisei@arm.com, suzuki.poulose@arm.com, catalin.marinas@arm.com, will@kernel.org Cc: linux-arm-kernel@lists.infradead.org, kvmarm@lists.cs.columbia.edu, linux-kernel@vger.kernel.org, ardb@kernel.org, qwandor@google.com, tabba@google.com, dbrazdil@google.com, kernel-team@android.com, qperret@google.com X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20210809_082507_200675_AE17465F X-CRM114-Status: GOOD ( 26.01 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org The kvm_pgtable_stage2_find_range() function is used in the host memory abort path to try and look for the largest block mapping that can be used to map the faulting address. In order to do so, the function currently walks the stage-2 page-table and looks for existing incompatible mappings within the range of the largest possible block. If incompatible mappings are found, it tries the same procedure again, but using a smaller block range, and repeats until a matching range is found (potentially up to page granularity). While this approach has benefits (mostly in the fact that it proactively coalesces host stage-2 mappings), it can be slow if the ranges are fragmented, and it isn't optimized to deal with CPUs faulting on the same IPA as all of them will do all the work every time. To avoid these issues, remove kvm_pgtable_stage2_find_range(), and walk the page-table only once in the host_mem_abort() path to find the closest leaf to the input address. With this, use the corresponding range if it is invalid and not owned by another entity. If a valid leaf is found, return -EAGAIN similar to what is done in the kvm_pgtable_stage2_map() path to optimize concurrent faults. Reviewed-by: Fuad Tabba Signed-off-by: Quentin Perret --- arch/arm64/include/asm/kvm_pgtable.h | 30 ----------- arch/arm64/kvm/hyp/nvhe/mem_protect.c | 45 +++++++++++++++- arch/arm64/kvm/hyp/pgtable.c | 74 --------------------------- 3 files changed, 44 insertions(+), 105 deletions(-) diff --git a/arch/arm64/include/asm/kvm_pgtable.h b/arch/arm64/include/asm/kvm_pgtable.h index 6938eac72c1f..83c5c97d9eac 100644 --- a/arch/arm64/include/asm/kvm_pgtable.h +++ b/arch/arm64/include/asm/kvm_pgtable.h @@ -154,16 +154,6 @@ enum kvm_pgtable_prot { #define PAGE_HYP_RO (KVM_PGTABLE_PROT_R) #define PAGE_HYP_DEVICE (PAGE_HYP | KVM_PGTABLE_PROT_DEVICE) -/** - * struct kvm_mem_range - Range of Intermediate Physical Addresses - * @start: Start of the range. - * @end: End of the range. - */ -struct kvm_mem_range { - u64 start; - u64 end; -}; - /** * enum kvm_pgtable_walk_flags - Flags to control a depth-first page-table walk. * @KVM_PGTABLE_WALK_LEAF: Visit leaf entries, including invalid @@ -490,24 +480,4 @@ int kvm_pgtable_walk(struct kvm_pgtable *pgt, u64 addr, u64 size, */ int kvm_pgtable_get_leaf(struct kvm_pgtable *pgt, u64 addr, kvm_pte_t *ptep, u32 *level); - -/** - * kvm_pgtable_stage2_find_range() - Find a range of Intermediate Physical - * Addresses with compatible permission - * attributes. - * @pgt: Page-table structure initialised by kvm_pgtable_stage2_init*(). - * @addr: Address that must be covered by the range. - * @prot: Protection attributes that the range must be compatible with. - * @range: Range structure used to limit the search space at call time and - * that will hold the result. - * - * The offset of @addr within a page is ignored. An IPA is compatible with @prot - * iff its corresponding stage-2 page-table entry has default ownership and, if - * valid, is mapped with protection attributes identical to @prot. - * - * Return: 0 on success, negative error code on failure. - */ -int kvm_pgtable_stage2_find_range(struct kvm_pgtable *pgt, u64 addr, - enum kvm_pgtable_prot prot, - struct kvm_mem_range *range); #endif /* __ARM64_KVM_PGTABLE_H__ */ diff --git a/arch/arm64/kvm/hyp/nvhe/mem_protect.c b/arch/arm64/kvm/hyp/nvhe/mem_protect.c index 74280a753efb..2148d3968aa5 100644 --- a/arch/arm64/kvm/hyp/nvhe/mem_protect.c +++ b/arch/arm64/kvm/hyp/nvhe/mem_protect.c @@ -159,6 +159,11 @@ static int host_stage2_unmap_dev_all(void) return kvm_pgtable_stage2_unmap(pgt, addr, BIT(pgt->ia_bits) - addr); } +struct kvm_mem_range { + u64 start; + u64 end; +}; + static bool find_mem_range(phys_addr_t addr, struct kvm_mem_range *range) { int cur, left = 0, right = hyp_memblock_nr; @@ -227,6 +232,44 @@ static inline int __host_stage2_idmap(u64 start, u64 end, __ret; \ }) +static inline bool range_included(struct kvm_mem_range *child, + struct kvm_mem_range *parent) +{ + return parent->start <= child->start && child->end <= parent->end; +} + +static int host_stage2_adjust_range(u64 addr, struct kvm_mem_range *range) +{ + struct kvm_mem_range cur; + kvm_pte_t pte; + u32 level; + int ret; + + hyp_assert_lock_held(&host_kvm.lock); + ret = kvm_pgtable_get_leaf(&host_kvm.pgt, addr, &pte, &level); + if (ret) + return ret; + + if (kvm_pte_valid(pte)) + return -EAGAIN; + + if (pte) + return -EPERM; + + do { + u64 granule = kvm_granule_size(level); + cur.start = ALIGN_DOWN(addr, granule); + cur.end = cur.start + granule; + level++; + } while ((level < KVM_PGTABLE_MAX_LEVELS) && + !(kvm_level_supports_block_mapping(level) && + range_included(&cur, range))); + + *range = cur; + + return 0; +} + static int host_stage2_idmap(u64 addr) { enum kvm_pgtable_prot prot = KVM_PGTABLE_PROT_R | KVM_PGTABLE_PROT_W; @@ -238,7 +281,7 @@ static int host_stage2_idmap(u64 addr) prot |= KVM_PGTABLE_PROT_X; hyp_spin_lock(&host_kvm.lock); - ret = kvm_pgtable_stage2_find_range(&host_kvm.pgt, addr, prot, &range); + ret = host_stage2_adjust_range(addr, &range); if (ret) goto unlock; diff --git a/arch/arm64/kvm/hyp/pgtable.c b/arch/arm64/kvm/hyp/pgtable.c index 49d768b92997..4dff2ad39ee4 100644 --- a/arch/arm64/kvm/hyp/pgtable.c +++ b/arch/arm64/kvm/hyp/pgtable.c @@ -1102,77 +1102,3 @@ void kvm_pgtable_stage2_destroy(struct kvm_pgtable *pgt) pgt->mm_ops->free_pages_exact(pgt->pgd, pgd_sz); pgt->pgd = NULL; } - -#define KVM_PTE_LEAF_S2_COMPAT_MASK (KVM_PTE_LEAF_ATTR_S2_PERMS | \ - KVM_PTE_LEAF_ATTR_LO_S2_MEMATTR | \ - KVM_PTE_LEAF_ATTR_S2_IGNORED) - -static int stage2_check_permission_walker(u64 addr, u64 end, u32 level, - kvm_pte_t *ptep, - enum kvm_pgtable_walk_flags flag, - void * const arg) -{ - kvm_pte_t old_attr, pte = *ptep, *new_attr = arg; - - /* - * Compatible mappings are either invalid and owned by the page-table - * owner (whose id is 0), or valid with matching permission attributes. - */ - if (kvm_pte_valid(pte)) { - old_attr = pte & KVM_PTE_LEAF_S2_COMPAT_MASK; - if (old_attr != *new_attr) - return -EEXIST; - } else if (pte) { - return -EEXIST; - } - - return 0; -} - -int kvm_pgtable_stage2_find_range(struct kvm_pgtable *pgt, u64 addr, - enum kvm_pgtable_prot prot, - struct kvm_mem_range *range) -{ - kvm_pte_t attr; - struct kvm_pgtable_walker check_perm_walker = { - .cb = stage2_check_permission_walker, - .flags = KVM_PGTABLE_WALK_LEAF, - .arg = &attr, - }; - u64 granule, start, end; - u32 level; - int ret; - - ret = stage2_set_prot_attr(pgt, prot, &attr); - if (ret) - return ret; - attr &= KVM_PTE_LEAF_S2_COMPAT_MASK; - - for (level = pgt->start_level; level < KVM_PGTABLE_MAX_LEVELS; level++) { - granule = kvm_granule_size(level); - start = ALIGN_DOWN(addr, granule); - end = start + granule; - - if (!kvm_level_supports_block_mapping(level)) - continue; - - if (start < range->start || range->end < end) - continue; - - /* - * Check the presence of existing mappings with incompatible - * permissions within the current block range, and try one level - * deeper if one is found. - */ - ret = kvm_pgtable_walk(pgt, start, granule, &check_perm_walker); - if (ret != -EEXIST) - break; - } - - if (!ret) { - range->start = start; - range->end = end; - } - - return ret; -} From patchwork Mon Aug 9 15:24:34 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Quentin Perret X-Patchwork-Id: 12426807 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-17.5 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_ADSP_CUSTOM_MED,DKIM_SIGNED,DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, URIBL_BLOCKED,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 949F7C4338F for ; Mon, 9 Aug 2021 15:29:08 +0000 (UTC) Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 54B5460F93 for ; Mon, 9 Aug 2021 15:29:08 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.4.1 mail.kernel.org 54B5460F93 Authentication-Results: mail.kernel.org; dmarc=fail (p=reject dis=none) header.from=google.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:Cc:To:From:Subject:References: Mime-Version:Message-Id:In-Reply-To:Date:Reply-To:Content-ID: Content-Description:Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc :Resent-Message-ID:List-Owner; bh=yJNCtFyJ4jMcEkQKTXEHSiiy3NpYV/XOzE1cuIwTXjQ=; b=J1oiu1R9D+5KUi/9LY0aV30MWw 01VDn7Kq+EYEZVx3AxkwPOwcR/3ObFOV+SjwoWQ+bWik1kiL1a/JUeVoYxvHaFNOSpi4VBgcuT+qn t7+xhMI0UskSQ0GJPX7i+GL065Hmb+HME85W19FfTH9uxeQ/bN7SAjM6fZi42eICAc9hlzBF6Hea1 fJNtXt4lAVLuH5SLfeTHAtTuk589BhCcwqdYTM5MIum9uSPl7eHXyrzhMmCsplfBO+CM8/M48mMVi ikxCqzvPU18+xdqYLn+ksE/vXlWL2eOla/f+YEb7flvFbrJwdqAwC8qoEHuWxEjfEbiblIgvxDVzD gCtwdYLQ==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1mD7BT-0018tq-Ak; Mon, 09 Aug 2021 15:27:26 +0000 Received: from mail-qv1-xf4a.google.com ([2607:f8b0:4864:20::f4a]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1mD79J-0017tY-2Q for linux-arm-kernel@lists.infradead.org; Mon, 09 Aug 2021 15:25:10 +0000 Received: by mail-qv1-xf4a.google.com with SMTP id u6-20020ad448660000b02903500bf28866so4934048qvy.23 for ; Mon, 09 Aug 2021 08:25:07 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=date:in-reply-to:message-id:mime-version:references:subject:from:to :cc; bh=0U6bEbFLcZw+EVlWCQFxbjscDvzC1GFCLaFxX8mML7w=; b=kOLCXZaTuLa+M/eAksDlmTXxarvjJi/4V0U691zcl9If3pch1JmCrYhjGOY8XVKtJE 6NuTYclkw0GPt0FK2mSGr3hTr4kwtJkTW+ezZALU33xRPk28YsNg05w+3w7O4PtRSL8i wrjld/G5kuClFhvVfczeYTwUpwU7xzbTze9JQU4DBSUbnKUAk+5TmjEX1GPeAbpvo7EB 8VtQ67ClSRfBWW+SLvG80L6+JEtKS6B+eU5WjxhSeBvp0pLR+HNQV+YuuMmfIvZp+APm fiAK3U3VTWGVuBaz5uZbFLU0vd0fRk8vYM5Ajv1XJbbeZUdR+aL0PvPbhBv8YESCUjpZ TKFA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=0U6bEbFLcZw+EVlWCQFxbjscDvzC1GFCLaFxX8mML7w=; b=Lee3QwHJfiahpWsWwk4sm7YXTnHugPKhhzbyl7ZVrmmw1Er8hs6Z0H0w0LbgHmcOXt xSEtma8eiAFrOcSBtpZNCjg3ZDVl5T6kn/fdsew6sGw5maSsHfy9dq07VBroY4HMHBlt tpGPSsffW42AFOhaA2+RJsLLN5LkxceHcp4GOWJrDA4leSpmGTcgVh+UdXunsvNWPa2N 6grMrriXi3lvb6XECgAYFddkOXYySEhDL0sCRoSGM3m3bWgOa1v/1WnzbF09dPaoZd+W vya2bFOeiyTGvDH053sLPMK1cC6ls4krO3WFlltrJyXnjfewkurxwOilnoAMmdBwf/yt NZig== X-Gm-Message-State: AOAM530xQbZHZniUrGGyZMLj5K2ZOlIzgoqvL7wniPv3yW8n0IM77Oc+ XXa28y6Gv4abji0W3EFdf5S+XuzCgMrg X-Google-Smtp-Source: ABdhPJyLymu2D60zHUrmLqd+Yu9r4hpSVIXHgBvGpH3s/oeTvKgBK0EWZGKeXuUJmU+jyvIS3H1jipiit27T X-Received: from luke.lon.corp.google.com ([2a00:79e0:d:210:b0e8:d460:758b:a0ae]) (user=qperret job=sendgmr) by 2002:a05:6214:62c:: with SMTP id a12mr14308495qvx.49.1628522706714; Mon, 09 Aug 2021 08:25:06 -0700 (PDT) Date: Mon, 9 Aug 2021 16:24:34 +0100 In-Reply-To: <20210809152448.1810400-1-qperret@google.com> Message-Id: <20210809152448.1810400-8-qperret@google.com> Mime-Version: 1.0 References: <20210809152448.1810400-1-qperret@google.com> X-Mailer: git-send-email 2.32.0.605.g8dce9f2422-goog Subject: [PATCH v4 07/21] KVM: arm64: Rename KVM_PTE_LEAF_ATTR_S2_IGNORED From: Quentin Perret To: maz@kernel.org, james.morse@arm.com, alexandru.elisei@arm.com, suzuki.poulose@arm.com, catalin.marinas@arm.com, will@kernel.org Cc: linux-arm-kernel@lists.infradead.org, kvmarm@lists.cs.columbia.edu, linux-kernel@vger.kernel.org, ardb@kernel.org, qwandor@google.com, tabba@google.com, dbrazdil@google.com, kernel-team@android.com, qperret@google.com X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20210809_082509_166167_E71A9654 X-CRM114-Status: GOOD ( 11.66 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org The ignored bits for both stage-1 and stage-2 page and block descriptors are in [55:58], so rename KVM_PTE_LEAF_ATTR_S2_IGNORED to make it applicable to both. And while at it, since these bits are more commonly known as 'software' bits, rename accordingly. Reviewed-by: Fuad Tabba Signed-off-by: Quentin Perret --- arch/arm64/kvm/hyp/pgtable.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/arch/arm64/kvm/hyp/pgtable.c b/arch/arm64/kvm/hyp/pgtable.c index 4dff2ad39ee4..59a394d82de3 100644 --- a/arch/arm64/kvm/hyp/pgtable.c +++ b/arch/arm64/kvm/hyp/pgtable.c @@ -36,6 +36,8 @@ #define KVM_PTE_LEAF_ATTR_HI GENMASK(63, 51) +#define KVM_PTE_LEAF_ATTR_HI_SW GENMASK(58, 55) + #define KVM_PTE_LEAF_ATTR_HI_S1_XN BIT(54) #define KVM_PTE_LEAF_ATTR_HI_S2_XN BIT(54) @@ -44,8 +46,6 @@ KVM_PTE_LEAF_ATTR_LO_S2_S2AP_W | \ KVM_PTE_LEAF_ATTR_HI_S2_XN) -#define KVM_PTE_LEAF_ATTR_S2_IGNORED GENMASK(58, 55) - #define KVM_INVALID_PTE_OWNER_MASK GENMASK(63, 56) #define KVM_MAX_OWNER_ID 1 From patchwork Mon Aug 9 15:24:35 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Quentin Perret X-Patchwork-Id: 12426811 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-17.5 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_ADSP_CUSTOM_MED,DKIM_SIGNED,DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, URIBL_BLOCKED,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id E25CFC4338F for ; Mon, 9 Aug 2021 15:30:37 +0000 (UTC) Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id A76446101D for ; Mon, 9 Aug 2021 15:30:37 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.4.1 mail.kernel.org A76446101D Authentication-Results: mail.kernel.org; dmarc=fail (p=reject dis=none) header.from=google.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:Cc:To:From:Subject:References: Mime-Version:Message-Id:In-Reply-To:Date:Reply-To:Content-ID: Content-Description:Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc :Resent-Message-ID:List-Owner; bh=IJDUrtuUs5yUS0UYiOGM5qcSIWiMZyL23HWmSZDiHJc=; b=zl0qZ458McmnuVPW2Se5zehsfA nBqOAUukmrlEqHSpwI2U2B9VX395wUWUYlBiAw9nPXJhhfeAdk10pqYZ77TZ41oWL0N2Wh+t/Ezoe 88wwNUPz3UWQ7wxkEIvLBSGF1MLUcsYZaSQvBofjy/BAOAVrdUJzou8nK9R/oZOZi2GGLlJNO0rqg dxPbNWXGLSDmovDocVBQ4OuL3TNSUGl5ZsF8QilROIrkuyHASq1Uppt/03lK//kczUbVC2vUrkSXq rUSGSfeBA90h2CEq53+T95vnw3MDlNV/7coOocgPEWKYQpBXRyAu8rgB5OGsFW907F4vR3ZJs1MRO Mv4RGRfA==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1mD7CH-0019NO-QG; Mon, 09 Aug 2021 15:28:14 +0000 Received: from mail-wm1-x34a.google.com ([2a00:1450:4864:20::34a]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1mD79L-0017ub-Uj for linux-arm-kernel@lists.infradead.org; Mon, 09 Aug 2021 15:25:13 +0000 Received: by mail-wm1-x34a.google.com with SMTP id k5-20020a7bc3050000b02901e081f69d80so4136580wmj.8 for ; Mon, 09 Aug 2021 08:25:10 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=date:in-reply-to:message-id:mime-version:references:subject:from:to :cc; bh=Io0EDWR3PIis5nkH0gOz1mjrCvg673G9g4IXwa5GIus=; b=jfaqF6Wq0Q9e4kkR+jmQAYTa2BnTkBr9q1lmRTdsQdJw/Lh0h/noCoRqRFHW979Vpn FDKKbqzoWj8dqTthh4xSOmxkqcsPWcuNm+/ndBqE4q2EeOVzZY5mqOV525AiuZvLD4vO sS+vwhpt6tu/ukL509vBuZlbxwKpNeB2AAN77VyY2yLrcUYdbxSOO5/Yxf4wwTvU8W1N AmTljU+kIMZA+bYoLJRjetYMG/tcu4In0+GCs4r0zidT6ZPDR4xX/vsgr9gTLMWH1g8h xAk0qlE1XeZybchFEHU0YlCCJpuW++Vrxa1Y0KQ43VfgFVirxcu6/bwjge3Al4LJOlkE Jf3Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=Io0EDWR3PIis5nkH0gOz1mjrCvg673G9g4IXwa5GIus=; b=K1bLntCE8QPsVW+2zNQD/A/PpHwB+ptrm5cwr5BRIOgTDXvMXHn1nBX/ykXHe5iv1E s+MAXTkkweJHjbfkLH/hfxz6XsyWV87WpjOwUiuRfYITDtZZW6rPreZRVDSnmIXAjqMb IANVv41CJ9rFdl57U2lbAbuC94vAY4xuG9jfwSYMl7iBrM9dnIcVxPb/VQL2KgkBKkES 0S7AeU10FqC00um1zLwYQ5yhdk7XyG4OKNQ5oIH0T3azYN3DG8xuwVQPW4RL2HUJqmGW K1a7pmt9xmHmE1gG6Gg7UnwUuONhF0Clxsx5YlUjh3x1IysxyOx9ybyEDUFdF9WIp9WU CfVQ== X-Gm-Message-State: AOAM53383dULgcEmWJuk9CzxNqK8WXoeI7EII7L5DJ5KHOKXXq0k6MON QqPpfvVMZm38T21nbgPBJ/7Y1Mgchc7Y X-Google-Smtp-Source: ABdhPJz9dh/8xSQuBObFyZiVhTM2lFEjdAUdZfHfPEw63Mx6TEFaOYVnO3fFNdHkbUVeYnSFomkPJvPUytlF X-Received: from luke.lon.corp.google.com ([2a00:79e0:d:210:b0e8:d460:758b:a0ae]) (user=qperret job=sendgmr) by 2002:a05:600c:410b:: with SMTP id j11mr34327897wmi.27.1628522709110; Mon, 09 Aug 2021 08:25:09 -0700 (PDT) Date: Mon, 9 Aug 2021 16:24:35 +0100 In-Reply-To: <20210809152448.1810400-1-qperret@google.com> Message-Id: <20210809152448.1810400-9-qperret@google.com> Mime-Version: 1.0 References: <20210809152448.1810400-1-qperret@google.com> X-Mailer: git-send-email 2.32.0.605.g8dce9f2422-goog Subject: [PATCH v4 08/21] KVM: arm64: Don't overwrite software bits with owner id From: Quentin Perret To: maz@kernel.org, james.morse@arm.com, alexandru.elisei@arm.com, suzuki.poulose@arm.com, catalin.marinas@arm.com, will@kernel.org Cc: linux-arm-kernel@lists.infradead.org, kvmarm@lists.cs.columbia.edu, linux-kernel@vger.kernel.org, ardb@kernel.org, qwandor@google.com, tabba@google.com, dbrazdil@google.com, kernel-team@android.com, qperret@google.com X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20210809_082512_025767_8D021026 X-CRM114-Status: GOOD ( 14.00 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org We will soon start annotating page-tables with new flags to track shared pages and such, and we will do so in valid mappings using software bits in the PTEs, as provided by the architecture. However, it is possible that we will need to use those flags to annotate invalid mappings as well in the future, similar to what we do to track page ownership in the host stage-2. In order to facilitate the annotation of invalid mappings with such flags, it would be preferable to re-use the same bits as for valid mappings (bits [58-55]), but these are currently used for ownership encoding. Since we have plenty of bits left to use in invalid mappings, move the ownership bits further down the PTE to avoid the conflict. Reviewed-by: Fuad Tabba Signed-off-by: Quentin Perret --- arch/arm64/kvm/hyp/pgtable.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/arch/arm64/kvm/hyp/pgtable.c b/arch/arm64/kvm/hyp/pgtable.c index 59a394d82de3..1ee1168ac32d 100644 --- a/arch/arm64/kvm/hyp/pgtable.c +++ b/arch/arm64/kvm/hyp/pgtable.c @@ -46,7 +46,7 @@ KVM_PTE_LEAF_ATTR_LO_S2_S2AP_W | \ KVM_PTE_LEAF_ATTR_HI_S2_XN) -#define KVM_INVALID_PTE_OWNER_MASK GENMASK(63, 56) +#define KVM_INVALID_PTE_OWNER_MASK GENMASK(9, 2) #define KVM_MAX_OWNER_ID 1 struct kvm_pgtable_walk_data { From patchwork Mon Aug 9 15:24:36 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Quentin Perret X-Patchwork-Id: 12426813 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-17.5 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_ADSP_CUSTOM_MED,DKIM_SIGNED,DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, URIBL_BLOCKED,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 7D036C4338F for ; Mon, 9 Aug 2021 15:30:49 +0000 (UTC) Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 3970660F93 for ; Mon, 9 Aug 2021 15:30:49 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.4.1 mail.kernel.org 3970660F93 Authentication-Results: mail.kernel.org; dmarc=fail (p=reject dis=none) header.from=google.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:Cc:To:From:Subject:References: Mime-Version:Message-Id:In-Reply-To:Date:Reply-To:Content-ID: Content-Description:Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc :Resent-Message-ID:List-Owner; bh=y7mTmRiK+rZ6tgD08SRNynvl94DVnDi32KqjMeaukyM=; b=2GE+yxnil3uWeNcbqqTqi9cQqy 1BTkvZA431E12wkokI0A2Jj5ZzD+FWagFzNhCu92KcegUF1jdEgnCA9aejIomWJJZ3t7v+AqpEfvX aKmzIgtH1VCdRVxFufh0eFENKH4Mss0votbYGJ8CQ3wl7I8DGIyEyQrD1YwRSTVGcLDDalbojLb9B dYHc5GrFdRueonhFAphjfCmrV3ZsRsL54vQNKigQ4uaWE5ThGFUkMUGxmT0u0oY38kSRCsDKfOgzp ljAaARI+RqD7+fLuW845MsaHA7IEsPjQ6shTKO2PmxeUwlm3cQCGylaLOX+1eShui7W44+W2ZjU+7 0eHgtKOA==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1mD7Cq-0019h9-HN; Mon, 09 Aug 2021 15:28:49 +0000 Received: from mail-qv1-xf49.google.com ([2607:f8b0:4864:20::f49]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1mD79M-0017v4-IO for linux-arm-kernel@lists.infradead.org; Mon, 09 Aug 2021 15:25:13 +0000 Received: by mail-qv1-xf49.google.com with SMTP id z25-20020a0ca9590000b029033ba243ffa1so12621397qva.0 for ; Mon, 09 Aug 2021 08:25:11 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=date:in-reply-to:message-id:mime-version:references:subject:from:to :cc; bh=WndYnCDMOQzlPd71DewOIEPIaBhgx4ZmPbtWCPoDRhI=; b=XGuSt0YhxsImBr6msmOACUEnWDe5RjoMRsWfC6joinmJxH/ccrEqwcaqr0K9ea4kDM PLHmWbi+Lhh8yLHGdXomBavPSRAVZMAL8cyJlzE7Ayf/3/45f4IoNaWo2Hb4qacFglSJ +BJ8TUJOkcwkRMUt0arMJGnO1cL6VjSD/1EiVvMtauBxIazQsTDmLji32PyBKfkGscqY O+JEUiGOAV/z1LTCyODiwmScRaa//NHIuIiu2LpdEoVb3iH/ax0Akw30emloxQ5Y3LWh H7zcq+JTl/1swFcTt7/katDOOq5zfZrHT4eTqWSmVRKgGfAImEknabYM4aLPjNb8p8J4 d1wA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=WndYnCDMOQzlPd71DewOIEPIaBhgx4ZmPbtWCPoDRhI=; b=H2VrTQz+NOFZxyKoN/EEbNc9836dQ2UQddthEyJIuhSuzjulNiWCQHa4+41hLt6cjG WVhGH91ur3gJhkU/W2161DN9mGHfjnTqp9h1K9Li+b9HK0KA3Lpx3pIRuLSen3T/H2CH 5QDyejOJg24pP5JhjO6+to6U1sOXtxcBJqvEwWLkqXaKYiQ1kFmjtIiwaBtsOGAZfnDl gPzjPpSeTjA84Cy7shJ6N1yH2S9ajl23ZilwzYccLF7+4VAnEA4esc2UPCcy0SAAMwOY T3zLg+tB4HUys/7woIEivasj8GeK+muwZOj4dAhJfedKzOr2TVvDmZ/96moi0wWzNYv6 VIfw== X-Gm-Message-State: AOAM531CDzQX1yQDigc2zbSR3ubCzi08Gj5yCH077FuhP7RYeJT2CLj9 JIsZXwnm81mFdl066hcaMRB65bp54SV2 X-Google-Smtp-Source: ABdhPJxKbzxXq3vACntXFb3WeX5U2azY/J4P0m64KOuxC+yq1GUbq1EaY8YQtx4SiagyVZzS09eoTaGzaXW+ X-Received: from luke.lon.corp.google.com ([2a00:79e0:d:210:b0e8:d460:758b:a0ae]) (user=qperret job=sendgmr) by 2002:a05:6214:10c4:: with SMTP id r4mr24444934qvs.58.1628522711297; Mon, 09 Aug 2021 08:25:11 -0700 (PDT) Date: Mon, 9 Aug 2021 16:24:36 +0100 In-Reply-To: <20210809152448.1810400-1-qperret@google.com> Message-Id: <20210809152448.1810400-10-qperret@google.com> Mime-Version: 1.0 References: <20210809152448.1810400-1-qperret@google.com> X-Mailer: git-send-email 2.32.0.605.g8dce9f2422-goog Subject: [PATCH v4 09/21] KVM: arm64: Tolerate re-creating hyp mappings to set software bits From: Quentin Perret To: maz@kernel.org, james.morse@arm.com, alexandru.elisei@arm.com, suzuki.poulose@arm.com, catalin.marinas@arm.com, will@kernel.org Cc: linux-arm-kernel@lists.infradead.org, kvmarm@lists.cs.columbia.edu, linux-kernel@vger.kernel.org, ardb@kernel.org, qwandor@google.com, tabba@google.com, dbrazdil@google.com, kernel-team@android.com, qperret@google.com X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20210809_082512_654941_F5AA0E41 X-CRM114-Status: GOOD ( 12.87 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org The current hypervisor stage-1 mapping code doesn't allow changing an existing valid mapping. Relax this condition by allowing changes that only target software bits, as that will soon be needed to annotate shared pages. Reviewed-by: Fuad Tabba Signed-off-by: Quentin Perret --- arch/arm64/kvm/hyp/pgtable.c | 18 ++++++++++++++++-- 1 file changed, 16 insertions(+), 2 deletions(-) diff --git a/arch/arm64/kvm/hyp/pgtable.c b/arch/arm64/kvm/hyp/pgtable.c index 1ee1168ac32d..2689fcb7901d 100644 --- a/arch/arm64/kvm/hyp/pgtable.c +++ b/arch/arm64/kvm/hyp/pgtable.c @@ -362,6 +362,21 @@ static int hyp_set_prot_attr(enum kvm_pgtable_prot prot, kvm_pte_t *ptep) return 0; } +static bool hyp_pte_needs_update(kvm_pte_t old, kvm_pte_t new) +{ + /* + * Tolerate KVM recreating the exact same mapping, or changing software + * bits if the existing mapping was valid. + */ + if (old == new) + return false; + + if (!kvm_pte_valid(old)) + return true; + + return !WARN_ON((old ^ new) & ~KVM_PTE_LEAF_ATTR_HI_SW); +} + static bool hyp_map_walker_try_leaf(u64 addr, u64 end, u32 level, kvm_pte_t *ptep, struct hyp_map_data *data) { @@ -371,9 +386,8 @@ static bool hyp_map_walker_try_leaf(u64 addr, u64 end, u32 level, if (!kvm_block_mapping_supported(addr, end, phys, level)) return false; - /* Tolerate KVM recreating the exact same mapping */ new = kvm_init_valid_leaf_pte(phys, data->attr, level); - if (old != new && !WARN_ON(kvm_pte_valid(old))) + if (hyp_pte_needs_update(old, new)) smp_store_release(ptep, new); data->phys += granule; From patchwork Mon Aug 9 15:24:37 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Quentin Perret X-Patchwork-Id: 12426815 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-17.5 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_ADSP_CUSTOM_MED,DKIM_SIGNED,DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, URIBL_BLOCKED,USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 2766AC4338F for ; Mon, 9 Aug 2021 15:31:34 +0000 (UTC) Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id DD48560EFF for ; Mon, 9 Aug 2021 15:31:33 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.4.1 mail.kernel.org DD48560EFF Authentication-Results: mail.kernel.org; dmarc=fail (p=reject dis=none) header.from=google.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:Cc:To:From:Subject:References: Mime-Version:Message-Id:In-Reply-To:Date:Reply-To:Content-ID: Content-Description:Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc :Resent-Message-ID:List-Owner; bh=KEZclsizEBwlib3n7NieRUnrtxNcSfYWqQZcbJ7bVjI=; b=p1iWfv5wtkdlGzJ4RLHLopeh/9 BHFEb9OcPzFnNCba5YrgGqSixlYoDuLiF2S5BdRJU0sqdu5OMUwt2rU/2Xi9CvGj4dffR9HBgaUEV CKdh+ZODcQtDMI3yvB3mA98LWYX0QKjlIo6Q55yUgCN7HWrp8ZzxafqZqaMtByZqBBCHxClPSJgqH U2HsPjCXDQZfIafF4yvJE27yMyTu9W4S1XfswsBDjxEYZ2Vt98vST1cpoAJ1pxCI9NB0DfwidJiRX FoG73Dze2cl16U3zUWcETgmOYfqHzC0Sq4IqewctfiqCO/gO/mhn6QRSPVY4NJE5lymMK8XNGJj8+ B5127qGw==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1mD7DO-0019vQ-Pr; Mon, 09 Aug 2021 15:29:22 +0000 Received: from mail-qv1-xf49.google.com ([2607:f8b0:4864:20::f49]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1mD79O-0017vs-Ty for linux-arm-kernel@lists.infradead.org; Mon, 09 Aug 2021 15:25:16 +0000 Received: by mail-qv1-xf49.google.com with SMTP id u6-20020ad448660000b02903500bf28866so4935218qvy.23 for ; Mon, 09 Aug 2021 08:25:14 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=date:in-reply-to:message-id:mime-version:references:subject:from:to :cc; bh=70o1p9T2yrXnR7kwCBzcsvYiWZbSw7GxMLQVcwsnfAk=; b=XdV5jZzO88qZVD7CT6bTH1V+RO7/ta+lS3AGiKGYxx2q/zbEX/XeIP043P2PvpaI42 LhLQzSWzuaKN0AB2Hu4Dn/5gmOKR44PrV/e99ZLMhwr0nUXykbyi8jo5rqxSLxTZ7tu8 M7WVnYNO7XOR8Io/u0e8grpvD/d4VVb/+pr0nA7UUlHRT3uvGQiREG8Zgx5YVCqHzUAx yILJCdHUv99cDWlcDuYnGPeg9+GO4ZGYtbegHSFmfIVnlDpJ9HJqInCbIBMA0DUz49Y3 VRFPuu7rus9aySOAREFUtnE5uW07LJ0wOtyQnzitAYH7DWVtBCh6ZQAotzkjEkt4TQft Z6uw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=70o1p9T2yrXnR7kwCBzcsvYiWZbSw7GxMLQVcwsnfAk=; b=s2CTBvYajyEk69nBuSxwezwYBvR9sfvFoNmPq8AYr8n82QMsGjqDTQkliuIPNaLfyu reHiqq2a0CswgT0j3jVBU2v+IWC+Xi6WGX3ScvQCdPFhuEKRzEoAU2d8FgoymSOG6isx 8KdHkaOtuOassoQtcV856T1Le1ZFsIJcmt4HNeL9Z0B7SlMpBn7BmBvS5oLLC8BiSuue vJUJYqMjSR6J3crHZ4YYiYlfSKTx3q0/5Ga0FfAPsoR335MU95n8E7bNs4SzK5gftYxd 791iKo470o0/8rYZZdwYdvbYQ+0XMdYe9zFUYcq/ruBGuwBoGptpxiluWpfVN+PvDreF 12BA== X-Gm-Message-State: AOAM530QjrE3Etqx0SD5idzyIYJOkfYyd/NWVcLW4zVDaC17dN8Ppu7Y vL94P2X3RY8c3LK1CCcZYQC2gbrxNwtb X-Google-Smtp-Source: ABdhPJycvuemF1/YjunyxQTI+Hc1BBpUtCt6LUgKSmJpnGd04AlbbCnK/c5/bJkNsshxXK1v5uhMC1oLfHJF X-Received: from luke.lon.corp.google.com ([2a00:79e0:d:210:b0e8:d460:758b:a0ae]) (user=qperret job=sendgmr) by 2002:a0c:f28f:: with SMTP id k15mr3144025qvl.43.1628522713431; Mon, 09 Aug 2021 08:25:13 -0700 (PDT) Date: Mon, 9 Aug 2021 16:24:37 +0100 In-Reply-To: <20210809152448.1810400-1-qperret@google.com> Message-Id: <20210809152448.1810400-11-qperret@google.com> Mime-Version: 1.0 References: <20210809152448.1810400-1-qperret@google.com> X-Mailer: git-send-email 2.32.0.605.g8dce9f2422-goog Subject: [PATCH v4 10/21] KVM: arm64: Enable forcing page-level stage-2 mappings From: Quentin Perret To: maz@kernel.org, james.morse@arm.com, alexandru.elisei@arm.com, suzuki.poulose@arm.com, catalin.marinas@arm.com, will@kernel.org Cc: linux-arm-kernel@lists.infradead.org, kvmarm@lists.cs.columbia.edu, linux-kernel@vger.kernel.org, ardb@kernel.org, qwandor@google.com, tabba@google.com, dbrazdil@google.com, kernel-team@android.com, qperret@google.com X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20210809_082515_027825_75BF9819 X-CRM114-Status: GOOD ( 25.95 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Much of the stage-2 manipulation logic relies on being able to destroy block mappings if e.g. installing a smaller mapping in the range. The rationale for this behaviour is that stage-2 mappings can always be re-created lazily. However, this gets more complicated when the stage-2 page-table is used to store metadata about the underlying pages. In such cases, destroying a block mapping may lead to losing part of the state, and confuse the user of those metadata (such as the hypervisor in nVHE protected mode). To avoid this, introduce a callback function in the pgtable struct which is called during all map operations to determine whether the mappings can use blocks, or should be forced to page granularity. This is used by the hypervisor when creating the host stage-2 to force page-level mappings when using non-default protection attributes. Signed-off-by: Quentin Perret Reviewed-by: Fuad Tabba --- arch/arm64/include/asm/kvm_pgtable.h | 66 +++++++++++++++++---------- arch/arm64/kvm/hyp/nvhe/mem_protect.c | 34 ++++++++++++-- arch/arm64/kvm/hyp/pgtable.c | 29 ++++++++++-- 3 files changed, 94 insertions(+), 35 deletions(-) diff --git a/arch/arm64/include/asm/kvm_pgtable.h b/arch/arm64/include/asm/kvm_pgtable.h index 83c5c97d9eac..2c090b0eee77 100644 --- a/arch/arm64/include/asm/kvm_pgtable.h +++ b/arch/arm64/include/asm/kvm_pgtable.h @@ -115,25 +115,6 @@ enum kvm_pgtable_stage2_flags { KVM_PGTABLE_S2_IDMAP = BIT(1), }; -/** - * struct kvm_pgtable - KVM page-table. - * @ia_bits: Maximum input address size, in bits. - * @start_level: Level at which the page-table walk starts. - * @pgd: Pointer to the first top-level entry of the page-table. - * @mm_ops: Memory management callbacks. - * @mmu: Stage-2 KVM MMU struct. Unused for stage-1 page-tables. - */ -struct kvm_pgtable { - u32 ia_bits; - u32 start_level; - kvm_pte_t *pgd; - struct kvm_pgtable_mm_ops *mm_ops; - - /* Stage-2 only */ - struct kvm_s2_mmu *mmu; - enum kvm_pgtable_stage2_flags flags; -}; - /** * enum kvm_pgtable_prot - Page-table permissions and attributes. * @KVM_PGTABLE_PROT_X: Execute permission. @@ -149,11 +130,43 @@ enum kvm_pgtable_prot { KVM_PGTABLE_PROT_DEVICE = BIT(3), }; -#define PAGE_HYP (KVM_PGTABLE_PROT_R | KVM_PGTABLE_PROT_W) +#define KVM_PGTABLE_PROT_RW (KVM_PGTABLE_PROT_R | KVM_PGTABLE_PROT_W) +#define KVM_PGTABLE_PROT_RWX (KVM_PGTABLE_PROT_RW | KVM_PGTABLE_PROT_X) + +#define PKVM_HOST_MEM_PROT KVM_PGTABLE_PROT_RWX +#define PKVM_HOST_MMIO_PROT KVM_PGTABLE_PROT_RW + +#define PAGE_HYP KVM_PGTABLE_PROT_RW #define PAGE_HYP_EXEC (KVM_PGTABLE_PROT_R | KVM_PGTABLE_PROT_X) #define PAGE_HYP_RO (KVM_PGTABLE_PROT_R) #define PAGE_HYP_DEVICE (PAGE_HYP | KVM_PGTABLE_PROT_DEVICE) +typedef bool (*kvm_pgtable_force_pte_cb_t)(u64 addr, u64 end, + enum kvm_pgtable_prot prot); + +/** + * struct kvm_pgtable - KVM page-table. + * @ia_bits: Maximum input address size, in bits. + * @start_level: Level at which the page-table walk starts. + * @pgd: Pointer to the first top-level entry of the page-table. + * @mm_ops: Memory management callbacks. + * @mmu: Stage-2 KVM MMU struct. Unused for stage-1 page-tables. + * @flags: Stage-2 page-table flags. + * @force_pte_cb: Function that returns true if page level mappings must + * be used instead of block mappings. + */ +struct kvm_pgtable { + u32 ia_bits; + u32 start_level; + kvm_pte_t *pgd; + struct kvm_pgtable_mm_ops *mm_ops; + + /* Stage-2 only */ + struct kvm_s2_mmu *mmu; + enum kvm_pgtable_stage2_flags flags; + kvm_pgtable_force_pte_cb_t force_pte_cb; +}; + /** * enum kvm_pgtable_walk_flags - Flags to control a depth-first page-table walk. * @KVM_PGTABLE_WALK_LEAF: Visit leaf entries, including invalid @@ -246,21 +259,24 @@ int kvm_pgtable_hyp_map(struct kvm_pgtable *pgt, u64 addr, u64 size, u64 phys, u64 kvm_get_vtcr(u64 mmfr0, u64 mmfr1, u32 phys_shift); /** - * kvm_pgtable_stage2_init_flags() - Initialise a guest stage-2 page-table. + * __kvm_pgtable_stage2_init() - Initialise a guest stage-2 page-table. * @pgt: Uninitialised page-table structure to initialise. * @arch: Arch-specific KVM structure representing the guest virtual * machine. * @mm_ops: Memory management callbacks. * @flags: Stage-2 configuration flags. + * @force_pte_cb: Function that returns true if page level mappings must + * be used instead of block mappings. * * Return: 0 on success, negative error code on failure. */ -int kvm_pgtable_stage2_init_flags(struct kvm_pgtable *pgt, struct kvm_arch *arch, - struct kvm_pgtable_mm_ops *mm_ops, - enum kvm_pgtable_stage2_flags flags); +int __kvm_pgtable_stage2_init(struct kvm_pgtable *pgt, struct kvm_arch *arch, + struct kvm_pgtable_mm_ops *mm_ops, + enum kvm_pgtable_stage2_flags flags, + kvm_pgtable_force_pte_cb_t force_pte_cb); #define kvm_pgtable_stage2_init(pgt, arch, mm_ops) \ - kvm_pgtable_stage2_init_flags(pgt, arch, mm_ops, 0) + __kvm_pgtable_stage2_init(pgt, arch, mm_ops, 0, NULL) /** * kvm_pgtable_stage2_destroy() - Destroy an unused guest stage-2 page-table. diff --git a/arch/arm64/kvm/hyp/nvhe/mem_protect.c b/arch/arm64/kvm/hyp/nvhe/mem_protect.c index 2148d3968aa5..6fed6772c673 100644 --- a/arch/arm64/kvm/hyp/nvhe/mem_protect.c +++ b/arch/arm64/kvm/hyp/nvhe/mem_protect.c @@ -89,6 +89,8 @@ static void prepare_host_vtcr(void) id_aa64mmfr1_el1_sys_val, phys_shift); } +static bool host_stage2_force_pte_cb(u64 addr, u64 end, enum kvm_pgtable_prot prot); + int kvm_host_prepare_stage2(void *pgt_pool_base) { struct kvm_s2_mmu *mmu = &host_kvm.arch.mmu; @@ -101,8 +103,9 @@ int kvm_host_prepare_stage2(void *pgt_pool_base) if (ret) return ret; - ret = kvm_pgtable_stage2_init_flags(&host_kvm.pgt, &host_kvm.arch, - &host_kvm.mm_ops, KVM_HOST_S2_FLAGS); + ret = __kvm_pgtable_stage2_init(&host_kvm.pgt, &host_kvm.arch, + &host_kvm.mm_ops, KVM_HOST_S2_FLAGS, + host_stage2_force_pte_cb); if (ret) return ret; @@ -270,15 +273,36 @@ static int host_stage2_adjust_range(u64 addr, struct kvm_mem_range *range) return 0; } +static bool host_stage2_force_pte_cb(u64 addr, u64 end, enum kvm_pgtable_prot prot) +{ + /* + * Block mappings must be used with care in the host stage-2 as a + * kvm_pgtable_stage2_map() operation targeting a page in the range of + * an existing block will delete the block under the assumption that + * mappings in the rest of the block range can always be rebuilt lazily. + * That assumption is correct for the host stage-2 with RWX mappings + * targeting memory or RW mappings targeting MMIO ranges (see + * host_stage2_idmap() below which implements some of the host memory + * abort logic). However, this is not safe for any other mappings where + * the host stage-2 page-table is in fact the only place where this + * state is stored. In all those cases, it is safer to use page-level + * mappings, hence avoiding to lose the state because of side-effects in + * kvm_pgtable_stage2_map(). + */ + if (range_is_memory(addr, end)) + return prot != PKVM_HOST_MEM_PROT; + else + return prot != PKVM_HOST_MMIO_PROT; +} + static int host_stage2_idmap(u64 addr) { - enum kvm_pgtable_prot prot = KVM_PGTABLE_PROT_R | KVM_PGTABLE_PROT_W; struct kvm_mem_range range; bool is_memory = find_mem_range(addr, &range); + enum kvm_pgtable_prot prot; int ret; - if (is_memory) - prot |= KVM_PGTABLE_PROT_X; + prot = is_memory ? PKVM_HOST_MEM_PROT : PKVM_HOST_MMIO_PROT; hyp_spin_lock(&host_kvm.lock); ret = host_stage2_adjust_range(addr, &range); diff --git a/arch/arm64/kvm/hyp/pgtable.c b/arch/arm64/kvm/hyp/pgtable.c index 2689fcb7901d..e25d829587b9 100644 --- a/arch/arm64/kvm/hyp/pgtable.c +++ b/arch/arm64/kvm/hyp/pgtable.c @@ -452,6 +452,8 @@ int kvm_pgtable_hyp_init(struct kvm_pgtable *pgt, u32 va_bits, pgt->start_level = KVM_PGTABLE_MAX_LEVELS - levels; pgt->mm_ops = mm_ops; pgt->mmu = NULL; + pgt->force_pte_cb = NULL; + return 0; } @@ -489,6 +491,9 @@ struct stage2_map_data { void *memcache; struct kvm_pgtable_mm_ops *mm_ops; + + /* Force mappings to page granularity */ + bool force_pte; }; u64 kvm_get_vtcr(u64 mmfr0, u64 mmfr1, u32 phys_shift) @@ -602,6 +607,15 @@ static bool stage2_pte_executable(kvm_pte_t pte) return !(pte & KVM_PTE_LEAF_ATTR_HI_S2_XN); } +static bool stage2_leaf_mapping_allowed(u64 addr, u64 end, u32 level, + struct stage2_map_data *data) +{ + if (data->force_pte && (level < (KVM_PGTABLE_MAX_LEVELS - 1))) + return false; + + return kvm_block_mapping_supported(addr, end, data->phys, level); +} + static int stage2_map_walker_try_leaf(u64 addr, u64 end, u32 level, kvm_pte_t *ptep, struct stage2_map_data *data) @@ -611,7 +625,7 @@ static int stage2_map_walker_try_leaf(u64 addr, u64 end, u32 level, struct kvm_pgtable *pgt = data->mmu->pgt; struct kvm_pgtable_mm_ops *mm_ops = data->mm_ops; - if (!kvm_block_mapping_supported(addr, end, phys, level)) + if (!stage2_leaf_mapping_allowed(addr, end, level, data)) return -E2BIG; if (kvm_phys_is_valid(phys)) @@ -655,7 +669,7 @@ static int stage2_map_walk_table_pre(u64 addr, u64 end, u32 level, if (data->anchor) return 0; - if (!kvm_block_mapping_supported(addr, end, data->phys, level)) + if (!stage2_leaf_mapping_allowed(addr, end, level, data)) return 0; data->childp = kvm_pte_follow(*ptep, data->mm_ops); @@ -785,6 +799,7 @@ int kvm_pgtable_stage2_map(struct kvm_pgtable *pgt, u64 addr, u64 size, .mmu = pgt->mmu, .memcache = mc, .mm_ops = pgt->mm_ops, + .force_pte = pgt->force_pte_cb && pgt->force_pte_cb(addr, addr + size, prot), }; struct kvm_pgtable_walker walker = { .cb = stage2_map_walker, @@ -816,6 +831,7 @@ int kvm_pgtable_stage2_set_owner(struct kvm_pgtable *pgt, u64 addr, u64 size, .memcache = mc, .mm_ops = pgt->mm_ops, .owner_id = owner_id, + .force_pte = true, }; struct kvm_pgtable_walker walker = { .cb = stage2_map_walker, @@ -1057,9 +1073,11 @@ int kvm_pgtable_stage2_flush(struct kvm_pgtable *pgt, u64 addr, u64 size) return kvm_pgtable_walk(pgt, addr, size, &walker); } -int kvm_pgtable_stage2_init_flags(struct kvm_pgtable *pgt, struct kvm_arch *arch, - struct kvm_pgtable_mm_ops *mm_ops, - enum kvm_pgtable_stage2_flags flags) + +int __kvm_pgtable_stage2_init(struct kvm_pgtable *pgt, struct kvm_arch *arch, + struct kvm_pgtable_mm_ops *mm_ops, + enum kvm_pgtable_stage2_flags flags, + kvm_pgtable_force_pte_cb_t force_pte_cb) { size_t pgd_sz; u64 vtcr = arch->vtcr; @@ -1077,6 +1095,7 @@ int kvm_pgtable_stage2_init_flags(struct kvm_pgtable *pgt, struct kvm_arch *arch pgt->mm_ops = mm_ops; pgt->mmu = &arch->mmu; pgt->flags = flags; + pgt->force_pte_cb = force_pte_cb; /* Ensure zeroed PGD pages are visible to the hardware walker */ dsb(ishst); From patchwork Mon Aug 9 15:24:38 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Quentin Perret X-Patchwork-Id: 12426817 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-17.5 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_ADSP_CUSTOM_MED,DKIM_SIGNED,DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, URIBL_BLOCKED,USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id B2F76C4338F for ; Mon, 9 Aug 2021 15:32:13 +0000 (UTC) Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 73F886101D for ; Mon, 9 Aug 2021 15:32:13 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.4.1 mail.kernel.org 73F886101D Authentication-Results: mail.kernel.org; dmarc=fail (p=reject dis=none) header.from=google.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:Cc:To:From:Subject:References: Mime-Version:Message-Id:In-Reply-To:Date:Reply-To:Content-ID: Content-Description:Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc :Resent-Message-ID:List-Owner; bh=7Ql0WwXDfF/vKAdkJCUhqeWqxxkDF0koFWHQVT4cfSg=; b=jdMKrIohSJak4q+VVS9AImhXgl RX/sd9Bj0YTuoqDufoA10/12siALz02tECz39zEKq4SpuoAW7EbEbcpokdc0OhJFtuKD1uXKk04d+ Me+rXjr/p+N03frVdlenfVfhXUYHoSe6uiHUdCN2vArE8p5O5OK7ZFstuwCxtBaTXZjaGfIAIPW7T 2zZrXoPme9WSuHEF9e7DfJzSkzsFtDVHy7KjOukIzzNf+gnybMopKIbZ+FHRKuvGzDAbjWD9z4dD3 CN1K6xkIZ7sCScaNQicfyEFdxOtZWiyb2MSEi2FDcqUbhOiq9tsBiqHooVLx+o4ONsGE0hBaC7KQk K4eNgaXA==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1mD7E2-001ACV-St; Mon, 09 Aug 2021 15:30:03 +0000 Received: from mail-qv1-xf4a.google.com ([2607:f8b0:4864:20::f4a]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1mD79R-0017wT-7V for linux-arm-kernel@lists.infradead.org; Mon, 09 Aug 2021 15:25:18 +0000 Received: by mail-qv1-xf4a.google.com with SMTP id b8-20020a0562141148b02902f1474ce8b7so12548428qvt.20 for ; Mon, 09 Aug 2021 08:25:16 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=date:in-reply-to:message-id:mime-version:references:subject:from:to :cc; bh=4CFmU9uB5cuxc2lPCoWChXDNr9f3K+SqIKdnjLv1P4s=; b=rZmm2VKnjSmuiIf5p19o76FrNYszuOISRqkab3N4YFD95ogVFJvhagSQdpHXMeyhyj c3A82CuZtPTSFGFh2BXtGtUxrxi+jv2tB4LAp9D4hIFry+Hza8zHQht3NLujuVS1Ibus 92eAwBDgemHLSymFMTKMs6lCfK4aAEP3jvkHWkXoLKMUSzIQUgc1+A9Ex2PaRj+rpOb9 rxDag7IKYRu/MIgk/849635PQ18cH9sGG+H1T5zF/KKhXHTs5KpPyb2wXNjdavHHI1KM y/oFj0sH5zDEHl5qt/kGTjIthWR1z2MGqphl5V8yT7U28ZnIoRYxuG0VWw9zZDJSn9wM UdoA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=4CFmU9uB5cuxc2lPCoWChXDNr9f3K+SqIKdnjLv1P4s=; b=ZT9shPgvesQMTQjjmJBeJLEpOiWCNQ1RPsiGRQvkNe7tqRCYT7jzKWzdHERNuvgmwy B/p8X6a1jL3WbkCIu5UbPUQFixQzCbNFKZG00fatEKRXqe0/J/sl6yNYokSEF6zRq4DK 3adiRA8kvb8D+mOoSRlw5zpoo9XP/VDrg45b1ELHMxEyPlYltOnbxdg3tM5pG04jigP9 2rCB9doIytQ4ASSvqqJXoP0YqI0FBF6KlhS0F5nTu7xaqhcTDKYRRdTYqzN9Fn0JPuWk Ar1HjfO86MBAvz6QhKVF7ohGSvocv45PgKg+Xs/xbU42Zvr7xHjAg5/dg2U1eWTg562G kPUQ== X-Gm-Message-State: AOAM531dX6uqf19dJOXn4g/uAW0Yb1ien+tBt4874DFyUcbjk6h543nc kw3gUDV0EsDja24S6G6TQ9DbtkDuMTIX X-Google-Smtp-Source: ABdhPJxlnJqrt/Vc6kgjycoOm+7KHQ8sxsdJFc8/TA+ubWDblTmiAVGwH3I7qjBHoAKuZruQygdEooL2YeNk X-Received: from luke.lon.corp.google.com ([2a00:79e0:d:210:b0e8:d460:758b:a0ae]) (user=qperret job=sendgmr) by 2002:a05:6214:10c8:: with SMTP id r8mr24109684qvs.28.1628522715621; Mon, 09 Aug 2021 08:25:15 -0700 (PDT) Date: Mon, 9 Aug 2021 16:24:38 +0100 In-Reply-To: <20210809152448.1810400-1-qperret@google.com> Message-Id: <20210809152448.1810400-12-qperret@google.com> Mime-Version: 1.0 References: <20210809152448.1810400-1-qperret@google.com> X-Mailer: git-send-email 2.32.0.605.g8dce9f2422-goog Subject: [PATCH v4 11/21] KVM: arm64: Allow populating software bits From: Quentin Perret To: maz@kernel.org, james.morse@arm.com, alexandru.elisei@arm.com, suzuki.poulose@arm.com, catalin.marinas@arm.com, will@kernel.org Cc: linux-arm-kernel@lists.infradead.org, kvmarm@lists.cs.columbia.edu, linux-kernel@vger.kernel.org, ardb@kernel.org, qwandor@google.com, tabba@google.com, dbrazdil@google.com, kernel-team@android.com, qperret@google.com X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20210809_082517_314525_B137A962 X-CRM114-Status: GOOD ( 16.51 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Introduce infrastructure allowing to manipulate software bits in stage-1 and stage-2 page-tables using additional entries in the kvm_pgtable_prot enum. This is heavily inspired by Marc's implementation of a similar feature in the NV patch series, but adapted to allow stage-1 changes as well: https://lore.kernel.org/kvmarm/20210510165920.1913477-56-maz@kernel.org/ Suggested-by: Marc Zyngier Signed-off-by: Quentin Perret Reviewed-by: Fuad Tabba --- arch/arm64/include/asm/kvm_pgtable.h | 12 +++++++++++- arch/arm64/kvm/hyp/pgtable.c | 5 +++++ 2 files changed, 16 insertions(+), 1 deletion(-) diff --git a/arch/arm64/include/asm/kvm_pgtable.h b/arch/arm64/include/asm/kvm_pgtable.h index 2c090b0eee77..ff9d52f8073a 100644 --- a/arch/arm64/include/asm/kvm_pgtable.h +++ b/arch/arm64/include/asm/kvm_pgtable.h @@ -121,6 +121,10 @@ enum kvm_pgtable_stage2_flags { * @KVM_PGTABLE_PROT_W: Write permission. * @KVM_PGTABLE_PROT_R: Read permission. * @KVM_PGTABLE_PROT_DEVICE: Device attributes. + * @KVM_PGTABLE_PROT_SW0: Software bit 0. + * @KVM_PGTABLE_PROT_SW1: Software bit 1. + * @KVM_PGTABLE_PROT_SW2: Software bit 2. + * @KVM_PGTABLE_PROT_SW3: Software bit 3. */ enum kvm_pgtable_prot { KVM_PGTABLE_PROT_X = BIT(0), @@ -128,6 +132,11 @@ enum kvm_pgtable_prot { KVM_PGTABLE_PROT_R = BIT(2), KVM_PGTABLE_PROT_DEVICE = BIT(3), + + KVM_PGTABLE_PROT_SW0 = BIT(55), + KVM_PGTABLE_PROT_SW1 = BIT(56), + KVM_PGTABLE_PROT_SW2 = BIT(57), + KVM_PGTABLE_PROT_SW3 = BIT(58), }; #define KVM_PGTABLE_PROT_RW (KVM_PGTABLE_PROT_R | KVM_PGTABLE_PROT_W) @@ -420,7 +429,8 @@ kvm_pte_t kvm_pgtable_stage2_mkold(struct kvm_pgtable *pgt, u64 addr); * If there is a valid, leaf page-table entry used to translate @addr, then * relax the permissions in that entry according to the read, write and * execute permissions specified by @prot. No permissions are removed, and - * TLB invalidation is performed after updating the entry. + * TLB invalidation is performed after updating the entry. Software bits cannot + * be set or cleared using kvm_pgtable_stage2_relax_perms(). * * Return: 0 on success, negative error code on failure. */ diff --git a/arch/arm64/kvm/hyp/pgtable.c b/arch/arm64/kvm/hyp/pgtable.c index e25d829587b9..cff744136044 100644 --- a/arch/arm64/kvm/hyp/pgtable.c +++ b/arch/arm64/kvm/hyp/pgtable.c @@ -357,6 +357,7 @@ static int hyp_set_prot_attr(enum kvm_pgtable_prot prot, kvm_pte_t *ptep) attr |= FIELD_PREP(KVM_PTE_LEAF_ATTR_LO_S1_AP, ap); attr |= FIELD_PREP(KVM_PTE_LEAF_ATTR_LO_S1_SH, sh); attr |= KVM_PTE_LEAF_ATTR_LO_S1_AF; + attr |= prot & KVM_PTE_LEAF_ATTR_HI_SW; *ptep = attr; return 0; @@ -558,6 +559,7 @@ static int stage2_set_prot_attr(struct kvm_pgtable *pgt, enum kvm_pgtable_prot p attr |= FIELD_PREP(KVM_PTE_LEAF_ATTR_LO_S2_SH, sh); attr |= KVM_PTE_LEAF_ATTR_LO_S2_AF; + attr |= prot & KVM_PTE_LEAF_ATTR_HI_SW; *ptep = attr; return 0; @@ -1025,6 +1027,9 @@ int kvm_pgtable_stage2_relax_perms(struct kvm_pgtable *pgt, u64 addr, u32 level; kvm_pte_t set = 0, clr = 0; + if (prot & KVM_PTE_LEAF_ATTR_HI_SW) + return -EINVAL; + if (prot & KVM_PGTABLE_PROT_R) set |= KVM_PTE_LEAF_ATTR_LO_S2_S2AP_R; From patchwork Mon Aug 9 15:24:39 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Quentin Perret X-Patchwork-Id: 12426819 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-17.5 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_ADSP_CUSTOM_MED,DKIM_SIGNED,DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, URIBL_BLOCKED,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 3E47AC4338F for ; Mon, 9 Aug 2021 15:33:09 +0000 (UTC) Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id E82876101E for ; Mon, 9 Aug 2021 15:33:08 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.4.1 mail.kernel.org E82876101E Authentication-Results: mail.kernel.org; dmarc=fail (p=reject dis=none) header.from=google.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:Cc:To:From:Subject:References: Mime-Version:Message-Id:In-Reply-To:Date:Reply-To:Content-ID: Content-Description:Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc :Resent-Message-ID:List-Owner; bh=iaExK6vqDLKGVwC3u1PAYYbrPYicc+mkAHdX08h709o=; b=N1/1PrjKQLPNOIE1IdaEHlhPSD 4Y8sxBRelF6dVUeXJLx9Jc6vfjFMM/Egd4JkjbN7dSWR+0pKwoJFQ0rAUk9QfoU0TAP904c19L801 zY2nrj4hCoByJ53dQFJzLgvVJ6OO0jrWnGzGIRGyoOdv0CYuBhId6m7B8bSdBmuxyBtBbNeIu5tYI 1vYbVbRCBli7ozf6HtLTw6jCXbXVMnGCWW7+UI0Eteoi+29+543X6QuUiAJ5GbAnJVo3FbDeaWJKl eNxXEoNYil7v8bbIAjijSQGmEYjqfNsCcbls88x3LCU55gRq5I2yx69ENtrO6DWpj53Tg/YQuQgjA E1/FyDgA==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1mD7Ed-001AT1-G7; Mon, 09 Aug 2021 15:30:40 +0000 Received: from mail-qv1-xf4a.google.com ([2607:f8b0:4864:20::f4a]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1mD79T-0017x4-BD for linux-arm-kernel@lists.infradead.org; Mon, 09 Aug 2021 15:25:21 +0000 Received: by mail-qv1-xf4a.google.com with SMTP id k7-20020ad453c70000b02902f36ca6afdcso8293661qvv.7 for ; Mon, 09 Aug 2021 08:25:18 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=date:in-reply-to:message-id:mime-version:references:subject:from:to :cc; bh=8jz2lJsOdyZcDtlCHC3Dvx0yZKTs2N3b55wAk/v95ow=; b=BDFa2yN6cmKJG3x9zJSFyiSyGXrWwbQI3EkRXEyAoiDsNiZxnCuM7yr8Oa4sS4dxId 5eKr+FKgGlQq97sho5QfjNNpY4QVEgOwGe4qnto8+PH1sPYKxvsSFFEWd/WmFfbdUvkD c3sroI1akznW1wvpagXKal01dux4IP23CQRoyQzKlB2Hq7vZwlRmeuzMiiGUULtM74EO vqk/PboQA93VAnMy1vjV9PrMoZNR67MMoGR8eoMC8skkuWKBSfLRX65XrYlx1k9PzQEL B4Vu9eiDVi86Co2QUmAnVC43oq9HYU5EMtVUKZQVLmLRE/lJ3PSGixyaFvXfy8rmzq2H k/VQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=8jz2lJsOdyZcDtlCHC3Dvx0yZKTs2N3b55wAk/v95ow=; b=E0PYh9Qa5eFcqhJrJ/jEI4NCK3wD/Iqgd+eBB4Lit1ufrT+gwSDrxoQIt8gECfJtpz gdh3w5pAuWELXKRJPAz5/CooEdz7GfE3xCcF2DSsqKUxHAaMw+ZGAxDCZsaI187IhSdZ y4UPfwlT9ScQ8swIssPt+OOuYdCsy03A7GLEAhV/CZ1FqMsM6nR30aOVYzIkiNCIz1Pp r2eTHvjsRj8yg2ZxMYbwqUJ1OUNMmHsEqjOdSg0Ettur+d+2OPGT1JyKiL92pQ5fd/UG LjLBCrNEcoBSpAUAVVn/U/N29njlRcoJh/XXlJg4NqFKgSVaHB96/hM9QTcE8d4W7xj0 XINQ== X-Gm-Message-State: AOAM531NQia/NVsezwHjg/+wf1Ro2B4l+Ryj2bos8uaj4kU4y4DpXpMG N3hcp2gXdvNeUY9MOVD1Nohmr7PZfZqs X-Google-Smtp-Source: ABdhPJz2l1NOBwMrE26TsCA7occvb2o9beTqoSppx2h3dHXwe0dIzSOg5MwgreJKvwJvtlmSCeojgXOCjH28 X-Received: from luke.lon.corp.google.com ([2a00:79e0:d:210:b0e8:d460:758b:a0ae]) (user=qperret job=sendgmr) by 2002:a05:6214:10c4:: with SMTP id r4mr24445444qvs.58.1628522717738; Mon, 09 Aug 2021 08:25:17 -0700 (PDT) Date: Mon, 9 Aug 2021 16:24:39 +0100 In-Reply-To: <20210809152448.1810400-1-qperret@google.com> Message-Id: <20210809152448.1810400-13-qperret@google.com> Mime-Version: 1.0 References: <20210809152448.1810400-1-qperret@google.com> X-Mailer: git-send-email 2.32.0.605.g8dce9f2422-goog Subject: [PATCH v4 12/21] KVM: arm64: Add helpers to tag shared pages in SW bits From: Quentin Perret To: maz@kernel.org, james.morse@arm.com, alexandru.elisei@arm.com, suzuki.poulose@arm.com, catalin.marinas@arm.com, will@kernel.org Cc: linux-arm-kernel@lists.infradead.org, kvmarm@lists.cs.columbia.edu, linux-kernel@vger.kernel.org, ardb@kernel.org, qwandor@google.com, tabba@google.com, dbrazdil@google.com, kernel-team@android.com, qperret@google.com X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20210809_082519_444901_CC1AC6F4 X-CRM114-Status: GOOD ( 12.82 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org We will soon start annotating shared pages in page-tables in nVHE protected mode. Define all the states in which a page can be (owned, shared and owned, shared and borrowed), and provide helpers allowing to convert this into SW bits annotations using the matching prot attributes. Reviewed-by: Fuad Tabba Signed-off-by: Quentin Perret --- arch/arm64/kvm/hyp/include/nvhe/mem_protect.h | 26 +++++++++++++++++++ 1 file changed, 26 insertions(+) diff --git a/arch/arm64/kvm/hyp/include/nvhe/mem_protect.h b/arch/arm64/kvm/hyp/include/nvhe/mem_protect.h index 9c227d87c36d..87b1690c439f 100644 --- a/arch/arm64/kvm/hyp/include/nvhe/mem_protect.h +++ b/arch/arm64/kvm/hyp/include/nvhe/mem_protect.h @@ -12,6 +12,32 @@ #include #include +/* + * SW bits 0-1 are reserved to track the memory ownership state of each page: + * 00: The page is owned exclusively by the page-table owner. + * 01: The page is owned by the page-table owner, but is shared + * with another entity. + * 10: The page is shared with, but not owned by the page-table owner. + * 11: Reserved for future use (lending). + */ +enum pkvm_page_state { + PKVM_PAGE_OWNED = 0ULL, + PKVM_PAGE_SHARED_OWNED = KVM_PGTABLE_PROT_SW0, + PKVM_PAGE_SHARED_BORROWED = KVM_PGTABLE_PROT_SW1, +}; + +#define PKVM_PAGE_STATE_PROT_MASK (KVM_PGTABLE_PROT_SW0 | KVM_PGTABLE_PROT_SW1) +static inline enum kvm_pgtable_prot pkvm_mkstate(enum kvm_pgtable_prot prot, + enum pkvm_page_state state) +{ + return (prot & ~PKVM_PAGE_STATE_PROT_MASK) | state; +} + +static inline enum pkvm_page_state pkvm_getstate(enum kvm_pgtable_prot prot) +{ + return prot & PKVM_PAGE_STATE_PROT_MASK; +} + struct host_kvm { struct kvm_arch arch; struct kvm_pgtable pgt; From patchwork Mon Aug 9 15:24:40 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Quentin Perret X-Patchwork-Id: 12426821 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-17.5 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_ADSP_CUSTOM_MED,DKIM_SIGNED,DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, URIBL_BLOCKED,USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id CC624C4338F for ; Mon, 9 Aug 2021 15:33:57 +0000 (UTC) Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 78D8C6101D for ; Mon, 9 Aug 2021 15:33:57 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.4.1 mail.kernel.org 78D8C6101D Authentication-Results: mail.kernel.org; dmarc=fail (p=reject dis=none) header.from=google.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:Cc:To:From:Subject:References: Mime-Version:Message-Id:In-Reply-To:Date:Reply-To:Content-ID: Content-Description:Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc :Resent-Message-ID:List-Owner; bh=zCE9/3a5vHuargD2L4pcNio5BL4cdqED9IOCjNj3DmY=; b=Qp7GWt/ZyP9SdggL5Z2x4pZ09k Q/Kh+HIr/N9qqwwq7V90sY688860UCao+brFkg736Bmzgv0F7YbKMTIUwoA/r7NnZ0QYmlWJDnUsw nPHG9nqhTBANzBooim5HSpV1P7BAMOpmcY4Qczl5bH29/1nU5S8B2gyWFEWeKirpIqotcsw5wOei3 38w7b2/uiT9Juhw214AEV794CKJeEXvHNBydLxD/yHmnvdp5x8zqRP6+3W2nmk30od6snLBB7mHh7 nNLELNaNVi0S8KQsqt75eaB+IxJ4DtyORVm0lWPTY74TTVUpNnEP3z8vGGufQXtfwXQMnTgDcJzV0 EJyHLm0A==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1mD7FN-001AlA-E4; Mon, 09 Aug 2021 15:31:26 +0000 Received: from mail-qt1-x849.google.com ([2607:f8b0:4864:20::849]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1mD79V-0017xs-Uz for linux-arm-kernel@lists.infradead.org; Mon, 09 Aug 2021 15:25:23 +0000 Received: by mail-qt1-x849.google.com with SMTP id x15-20020ac87a8f0000b0290294d1eeaa1dso706770qtr.13 for ; Mon, 09 Aug 2021 08:25:20 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=date:in-reply-to:message-id:mime-version:references:subject:from:to :cc; bh=3dBIi1YzqMEm+UlYWRCSS5kKhcRIzlI2kVO6oR1gAJY=; b=pwqfES+b/309iPvlv8E1ftMhSnhlZy8mcMABvDRuAotG2budtPOdgJb73oPI6/MSGD ILhOvPTq8aIdf06m1VIM0yJOfipj9IDRQygTQK0MXZQ78+f+s6FJNc4eruv9GBpHUATi 3DPVRIxc+WjDdgR385vu2hDXy4mP0G3jEB7NeE0p5xd7Tsw9AwVodm/pABL8JAkqLUJm Dedj5LkITe1ybQH+GMWWBsZx05lk6S7Iqa2v9OVbbFvui/pxn7ZMcFeqJTXh1zGzeV7/ wCMz42JxwEpefj4CSksac16Vm16+9iQ0mxJgbtQYa+MRG8v10SN4Entl6tuQ2OhQ2q3X lIXA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=3dBIi1YzqMEm+UlYWRCSS5kKhcRIzlI2kVO6oR1gAJY=; b=dkzJuPecRjvQGQuT5mM2uCO2TIAwjDEzOqqqVIpla+EuL7xUPrj1gyRmBiLcis71sh n19QzyuaiKGzsO5tHgTtA0OdNfYRgCL7+Wh9mZLTlbxwDk6rejeCBvVZFCPWuUfocN91 4XoE6q1RVaVI9aCwn/vSU06M/p+EvcvbJ4UtqniFcvTEPJ6/tix51BtMrSD4u1m5/IV4 Xb3Y0j7wd8mHL1Q5w347DxBbOWe2W7tK0kaJh0R8Vaxq5wgNBBRerfWNLF+/8MLafjmS wsLaeOi74BLU2Vok/bR9EfdzdNX78YbS4l3W7X8MMNZpy7Qpioffu3FwNbhezU59/5U5 j7sw== X-Gm-Message-State: AOAM5336YdQJNcrMOkoxopQevLt/cbFnpCqSyptyKkW1qERUIJ/9L9d/ fM/coGP8I7cm9tqffTDVe5obBPZnTtbF X-Google-Smtp-Source: ABdhPJwTAF1XYWSKb67BxPuZMFA3SIWFIr0NEvX9FTJFVSSti+daWMWMqdeklw8++yGxusAYBtawMwClTWQ8 X-Received: from luke.lon.corp.google.com ([2a00:79e0:d:210:b0e8:d460:758b:a0ae]) (user=qperret job=sendgmr) by 2002:a05:6214:d4b:: with SMTP id 11mr12803043qvr.44.1628522719790; Mon, 09 Aug 2021 08:25:19 -0700 (PDT) Date: Mon, 9 Aug 2021 16:24:40 +0100 In-Reply-To: <20210809152448.1810400-1-qperret@google.com> Message-Id: <20210809152448.1810400-14-qperret@google.com> Mime-Version: 1.0 References: <20210809152448.1810400-1-qperret@google.com> X-Mailer: git-send-email 2.32.0.605.g8dce9f2422-goog Subject: [PATCH v4 13/21] KVM: arm64: Expose host stage-2 manipulation helpers From: Quentin Perret To: maz@kernel.org, james.morse@arm.com, alexandru.elisei@arm.com, suzuki.poulose@arm.com, catalin.marinas@arm.com, will@kernel.org Cc: linux-arm-kernel@lists.infradead.org, kvmarm@lists.cs.columbia.edu, linux-kernel@vger.kernel.org, ardb@kernel.org, qwandor@google.com, tabba@google.com, dbrazdil@google.com, kernel-team@android.com, qperret@google.com X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20210809_082522_065707_81A2888B X-CRM114-Status: GOOD ( 12.51 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org We will need to manipulate the host stage-2 page-table from outside mem_protect.c soon. Introduce two functions allowing this, and make them usable to users of mem_protect.h. Signed-off-by: Quentin Perret Reviewed-by: Fuad Tabba --- arch/arm64/kvm/hyp/include/nvhe/mem_protect.h | 2 ++ arch/arm64/kvm/hyp/nvhe/mem_protect.c | 18 +++++++++++++++++- 2 files changed, 19 insertions(+), 1 deletion(-) diff --git a/arch/arm64/kvm/hyp/include/nvhe/mem_protect.h b/arch/arm64/kvm/hyp/include/nvhe/mem_protect.h index 87b1690c439f..0849ee8fa260 100644 --- a/arch/arm64/kvm/hyp/include/nvhe/mem_protect.h +++ b/arch/arm64/kvm/hyp/include/nvhe/mem_protect.h @@ -49,6 +49,8 @@ extern struct host_kvm host_kvm; int __pkvm_prot_finalize(void); int __pkvm_mark_hyp(phys_addr_t start, phys_addr_t end); +int host_stage2_idmap_locked(phys_addr_t addr, u64 size, enum kvm_pgtable_prot prot); +int host_stage2_set_owner_locked(phys_addr_t addr, u64 size, u8 owner_id); int kvm_host_prepare_stage2(void *pgt_pool_base); void handle_host_mem_abort(struct kvm_cpu_context *host_ctxt); diff --git a/arch/arm64/kvm/hyp/nvhe/mem_protect.c b/arch/arm64/kvm/hyp/nvhe/mem_protect.c index 6fed6772c673..f95a5a4aa09c 100644 --- a/arch/arm64/kvm/hyp/nvhe/mem_protect.c +++ b/arch/arm64/kvm/hyp/nvhe/mem_protect.c @@ -273,6 +273,22 @@ static int host_stage2_adjust_range(u64 addr, struct kvm_mem_range *range) return 0; } +int host_stage2_idmap_locked(phys_addr_t addr, u64 size, + enum kvm_pgtable_prot prot) +{ + hyp_assert_lock_held(&host_kvm.lock); + + return host_stage2_try(__host_stage2_idmap, addr, addr + size, prot); +} + +int host_stage2_set_owner_locked(phys_addr_t addr, u64 size, u8 owner_id) +{ + hyp_assert_lock_held(&host_kvm.lock); + + return host_stage2_try(kvm_pgtable_stage2_set_owner, &host_kvm.pgt, + addr, size, &host_s2_pool, owner_id); +} + static bool host_stage2_force_pte_cb(u64 addr, u64 end, enum kvm_pgtable_prot prot) { /* @@ -309,7 +325,7 @@ static int host_stage2_idmap(u64 addr) if (ret) goto unlock; - ret = host_stage2_try(__host_stage2_idmap, range.start, range.end, prot); + ret = host_stage2_idmap_locked(range.start, range.end - range.start, prot); unlock: hyp_spin_unlock(&host_kvm.lock); From patchwork Mon Aug 9 15:24:41 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Quentin Perret X-Patchwork-Id: 12426823 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-17.5 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_ADSP_CUSTOM_MED,DKIM_SIGNED,DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, URIBL_BLOCKED,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id D6154C4338F for ; Mon, 9 Aug 2021 15:34:35 +0000 (UTC) Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 92C4B6101E for ; Mon, 9 Aug 2021 15:34:35 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.4.1 mail.kernel.org 92C4B6101E Authentication-Results: mail.kernel.org; dmarc=fail (p=reject dis=none) header.from=google.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:Cc:To:From:Subject:References: Mime-Version:Message-Id:In-Reply-To:Date:Reply-To:Content-ID: Content-Description:Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc :Resent-Message-ID:List-Owner; bh=6bL7c1/Fx6X5vCUPSNaZ1CyBx74IH//3JsLFgkG7MRA=; b=p6kq4smMqKSG9+h4fR+uY6VgnW B38Mtqk4uWUHE8w4/Ah2wTmOns313Q2G4VYKlwuHeO6bcXQe46FR9IELVHxMY4gEJkbtRygB+WgGZ SWSwzwAME+zScet8UdDthRHmQyC2N9Gjom/WbETD4bQ+kuwedWt6/JbmFv0t1HwFHzXtEVbFlmAbP wq8BUt/zp3t63UntzonNnQDkRKQlQ/Cih+ZMP3B9zQJ/JFfEYxAZqNZTnt8TjlDNHf0avmoK87zXU w7bUsnzQ55rnvoBFjMGSMt8QjHlAdZ/kFKBDh9WcMkWnWzbLMCoNA42itMJod14R2PC6gR5hSvB1k GOrVBWmQ==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1mD7G7-001B4G-4x; Mon, 09 Aug 2021 15:32:12 +0000 Received: from mail-qt1-x849.google.com ([2607:f8b0:4864:20::849]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1mD79X-0017ya-AA for linux-arm-kernel@lists.infradead.org; Mon, 09 Aug 2021 15:25:24 +0000 Received: by mail-qt1-x849.google.com with SMTP id c11-20020ac87dcb0000b0290293566e00b1so2063672qte.15 for ; Mon, 09 Aug 2021 08:25:22 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=date:in-reply-to:message-id:mime-version:references:subject:from:to :cc; bh=PF2it3NjGIyKq5ORZJDtBzFFsiYhiJLkV2yaXKhO9X4=; b=ftXs+EPnLTaD5rCUC9K5I63oGlpyo8cK1JWBx8UMjZ9M93skpv50giZG25JXZCnqR3 P4/bzs5IMikwsrEoPn4s5CIbKMUP2g9FrdX9jEn37cKvQer2/nFU7JUQGu84PZkYKdGE gpcZQBLTk7tFHRo7TSjmGRlJsxEFHP4XQsoDp4RyZR5TldotXatjcdMTOVVOTTnD+PyE eJg9fO9N/IZNYFbV/fpOFXkoJkn69ctU91Sa2C5RC/VOqM5oXnCCBiwB7oQviBUSERLr uASVreo2KG9expDo8/UU/xVSjkvRLYkgukVyElK5YqFZ7V2OhNzd0/5x5vxtQw21L+ex 2/zA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=PF2it3NjGIyKq5ORZJDtBzFFsiYhiJLkV2yaXKhO9X4=; b=eSx7tk2x9ssi6Tg8eSOaUjbskiVlh256+mctcbA1niZsTi1acxi1FsIYjHFUkzqaQQ xm7BzGwIkhtSedFejZQgfQoDmhDmWeVxjPhjsBYili5pvYxZBFUMHN3CCRpA5B10537B zJS+O2dIv2TY42zyv++ai+yS+P1wXB0RXY9R85HKhGcUz09TWn52v9qzH/doezpSDhen HL6kw2Dl5rQvV1Q2h9d20rhLrMzYh3cekSCzyMqzRuaXCrcvDHccZsojSmFxPiChAIQd 2RkW6Wl/Lk3Iuvh4twFzImkLxvrM4vM6aAY7Pn62+RCM2NzvbyHeqz+nOIS0n+3afSCQ qEIA== X-Gm-Message-State: AOAM530Yn4VJUFek/uzNNU+5OzmUzwh5JFGR92zWWP8CxWgSBGJpHmiQ bzFRVJIGTX9QwSRU33vKLhpNnZdYnN51 X-Google-Smtp-Source: ABdhPJykT3ZqBNKwN2TK8pulE0GyjvqWRyK+t1M7X0TDYfKeIo+MjBoUBZPBPTR6JMY1GKpSBOFS/zgqRiv2 X-Received: from luke.lon.corp.google.com ([2a00:79e0:d:210:b0e8:d460:758b:a0ae]) (user=qperret job=sendgmr) by 2002:a05:6214:10e6:: with SMTP id q6mr24439882qvt.11.1628522721784; Mon, 09 Aug 2021 08:25:21 -0700 (PDT) Date: Mon, 9 Aug 2021 16:24:41 +0100 In-Reply-To: <20210809152448.1810400-1-qperret@google.com> Message-Id: <20210809152448.1810400-15-qperret@google.com> Mime-Version: 1.0 References: <20210809152448.1810400-1-qperret@google.com> X-Mailer: git-send-email 2.32.0.605.g8dce9f2422-goog Subject: [PATCH v4 14/21] KVM: arm64: Expose pkvm_hyp_id From: Quentin Perret To: maz@kernel.org, james.morse@arm.com, alexandru.elisei@arm.com, suzuki.poulose@arm.com, catalin.marinas@arm.com, will@kernel.org Cc: linux-arm-kernel@lists.infradead.org, kvmarm@lists.cs.columbia.edu, linux-kernel@vger.kernel.org, ardb@kernel.org, qwandor@google.com, tabba@google.com, dbrazdil@google.com, kernel-team@android.com, qperret@google.com X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20210809_082523_474347_C661C398 X-CRM114-Status: GOOD ( 12.26 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Allow references to the hypervisor's owner id from outside mem_protect.c. Signed-off-by: Quentin Perret Reviewed-by: Fuad Tabba --- arch/arm64/kvm/hyp/include/nvhe/mem_protect.h | 2 ++ arch/arm64/kvm/hyp/nvhe/mem_protect.c | 2 +- 2 files changed, 3 insertions(+), 1 deletion(-) diff --git a/arch/arm64/kvm/hyp/include/nvhe/mem_protect.h b/arch/arm64/kvm/hyp/include/nvhe/mem_protect.h index 0849ee8fa260..23316a021880 100644 --- a/arch/arm64/kvm/hyp/include/nvhe/mem_protect.h +++ b/arch/arm64/kvm/hyp/include/nvhe/mem_protect.h @@ -46,6 +46,8 @@ struct host_kvm { }; extern struct host_kvm host_kvm; +extern const u8 pkvm_hyp_id; + int __pkvm_prot_finalize(void); int __pkvm_mark_hyp(phys_addr_t start, phys_addr_t end); diff --git a/arch/arm64/kvm/hyp/nvhe/mem_protect.c b/arch/arm64/kvm/hyp/nvhe/mem_protect.c index f95a5a4aa09c..ee255171945c 100644 --- a/arch/arm64/kvm/hyp/nvhe/mem_protect.c +++ b/arch/arm64/kvm/hyp/nvhe/mem_protect.c @@ -31,7 +31,7 @@ static struct hyp_pool host_s2_pool; u64 id_aa64mmfr0_el1_sys_val; u64 id_aa64mmfr1_el1_sys_val; -static const u8 pkvm_hyp_id = 1; +const u8 pkvm_hyp_id = 1; static void *host_s2_zalloc_pages_exact(size_t size) { From patchwork Mon Aug 9 15:24:42 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Quentin Perret X-Patchwork-Id: 12426825 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-17.5 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_ADSP_CUSTOM_MED,DKIM_SIGNED,DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, URIBL_BLOCKED,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 0FB6EC4338F for ; Mon, 9 Aug 2021 15:35:35 +0000 (UTC) Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id C81A46101D for ; Mon, 9 Aug 2021 15:35:34 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.4.1 mail.kernel.org C81A46101D Authentication-Results: mail.kernel.org; dmarc=fail (p=reject dis=none) header.from=google.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:Cc:To:From:Subject:References: Mime-Version:Message-Id:In-Reply-To:Date:Reply-To:Content-ID: Content-Description:Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc :Resent-Message-ID:List-Owner; bh=OCTuiOlh044Hohu7iTYXdhqNHUmGTTP2lnbJUVuhyh4=; b=L0iCWitI641rbKF0f1xSlfQ/hS a9rqgTajJoAEt1VLd6+V67N7LTOunK/d/8CQNX/51hgjngWKadnKo2ZJcBRTIGPOOGIWb0tfNc0HE 1KSYzYBEal5506kpwu/g6YqNGIM+s6TiNkMtctMie2w+OLgW2h/StuTlU+6zGH8an0cMDfEHaH8i0 h4SWtcMRzySbsGsrs4g0HUo1JdxtsgwsDTWQrtvV2XxI2TYqKVyKgFg8z4O91leyk9hGIt+pc8jCK 6WUuWoDOVUJXNXCY9z/qgALJnPEYkcFkmC/yO7B59ofHCRdDtWVIfomlX2IesXuYYibyZbicirsoj oWx0fZIw==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1mD7H7-001BS7-Ma; Mon, 09 Aug 2021 15:33:14 +0000 Received: from mail-wm1-x349.google.com ([2a00:1450:4864:20::349]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1mD79a-0017zs-Io for linux-arm-kernel@lists.infradead.org; Mon, 09 Aug 2021 15:25:27 +0000 Received: by mail-wm1-x349.google.com with SMTP id w25-20020a1cf6190000b0290252505ddd56so4138060wmc.3 for ; Mon, 09 Aug 2021 08:25:25 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=date:in-reply-to:message-id:mime-version:references:subject:from:to :cc; bh=sKHyMUAJ6pCT8dHKiixvyVxL5Tun5Zpw8bNGyjRM47Q=; b=bWc7R9nxej9wKPmFw7s2yZ0f0yAf+KeBkNmo6SjyRjbG3l1cLWkJ//rfxcBol1QBM7 bPkIAEFKVfFoHoK13PoLaqLoJvb6nZ3IXRULnN60HmZseVOfW7IqrDGLo5xPMyCUrFYa QNp5B+AsF9CfudlTwOj1BQzPPvVR8mxu0Bizz9YrQIoIl8n/VrGwG34KxyaGSU7CCCg1 ws45RKPPDGF2xUU2uTgxTbXSiMCwkNafZGCQQq9m57SYZjOC8Q9Sj6QFY5BE4m4nTDdN hWUNQ6xQnQ4fYbDe7R/P172LCOSA+nh7JJzs1dGHaL8xRuNJIqL3Qo7NbiHOilWilPRF LnzA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=sKHyMUAJ6pCT8dHKiixvyVxL5Tun5Zpw8bNGyjRM47Q=; b=QCdsJuGFISUu+rokzTx4PA5PXi6kjCdFlGLgHt3SGiM7nT/GDPAP/P85cRMPwHzwCx xOesEHmInnT8VJzuAN3PNNTnVjCSDmlHc7enbqumKqDMj77fXOi7P54sn1YajWziZWA/ X6Ny90AZCEDHhzGdGsE9XDgQjGtZuezFL7qZF4BtzbZ+ontQ9Xg9VOdbGrj8WJ7EUEET LrpOmFCfpGRw4leiHo/O6a/gY+G2IChz8+2fRObcjJUkyJLrro4Y+BFL5l6FopmKEV1N 7plhokRHi2QJGnFUlo/eiZ12WsYn3phTTMP9Uxv303sUROa7n6UozLOPIic88RIB+EMT v1Cg== X-Gm-Message-State: AOAM5327W8fSBGNyLFYSLWaY3S0KUjfU74YQMEzV/maJujZNy9OfOdLX oF7xhUyUz24GTQ7YFpQZsjeUBSL4xJvD X-Google-Smtp-Source: ABdhPJwsP/UADjMl4ruQIGRGBcaP9Ad7CMQNtdJiMCBKVwCsIFF2Gv3r4iqTKkuIpNzqlGVjFMa4yXaF8yoo X-Received: from luke.lon.corp.google.com ([2a00:79e0:d:210:b0e8:d460:758b:a0ae]) (user=qperret job=sendgmr) by 2002:a05:600c:4f46:: with SMTP id m6mr17616703wmq.68.1628522724126; Mon, 09 Aug 2021 08:25:24 -0700 (PDT) Date: Mon, 9 Aug 2021 16:24:42 +0100 In-Reply-To: <20210809152448.1810400-1-qperret@google.com> Message-Id: <20210809152448.1810400-16-qperret@google.com> Mime-Version: 1.0 References: <20210809152448.1810400-1-qperret@google.com> X-Mailer: git-send-email 2.32.0.605.g8dce9f2422-goog Subject: [PATCH v4 15/21] KVM: arm64: Introduce addr_is_memory() From: Quentin Perret To: maz@kernel.org, james.morse@arm.com, alexandru.elisei@arm.com, suzuki.poulose@arm.com, catalin.marinas@arm.com, will@kernel.org Cc: linux-arm-kernel@lists.infradead.org, kvmarm@lists.cs.columbia.edu, linux-kernel@vger.kernel.org, ardb@kernel.org, qwandor@google.com, tabba@google.com, dbrazdil@google.com, kernel-team@android.com, qperret@google.com X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20210809_082526_665460_72047D94 X-CRM114-Status: GOOD ( 10.34 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Introduce a helper usable in nVHE protected mode to check whether a physical address is in a RAM region or not. Signed-off-by: Quentin Perret Reviewed-by: Fuad Tabba --- arch/arm64/kvm/hyp/include/nvhe/mem_protect.h | 1 + arch/arm64/kvm/hyp/nvhe/mem_protect.c | 7 +++++++ 2 files changed, 8 insertions(+) diff --git a/arch/arm64/kvm/hyp/include/nvhe/mem_protect.h b/arch/arm64/kvm/hyp/include/nvhe/mem_protect.h index 23316a021880..49db0ec5a606 100644 --- a/arch/arm64/kvm/hyp/include/nvhe/mem_protect.h +++ b/arch/arm64/kvm/hyp/include/nvhe/mem_protect.h @@ -51,6 +51,7 @@ extern const u8 pkvm_hyp_id; int __pkvm_prot_finalize(void); int __pkvm_mark_hyp(phys_addr_t start, phys_addr_t end); +bool addr_is_memory(phys_addr_t phys); int host_stage2_idmap_locked(phys_addr_t addr, u64 size, enum kvm_pgtable_prot prot); int host_stage2_set_owner_locked(phys_addr_t addr, u64 size, u8 owner_id); int kvm_host_prepare_stage2(void *pgt_pool_base); diff --git a/arch/arm64/kvm/hyp/nvhe/mem_protect.c b/arch/arm64/kvm/hyp/nvhe/mem_protect.c index ee255171945c..cb023d31666e 100644 --- a/arch/arm64/kvm/hyp/nvhe/mem_protect.c +++ b/arch/arm64/kvm/hyp/nvhe/mem_protect.c @@ -197,6 +197,13 @@ static bool find_mem_range(phys_addr_t addr, struct kvm_mem_range *range) return false; } +bool addr_is_memory(phys_addr_t phys) +{ + struct kvm_mem_range range; + + return find_mem_range(phys, &range); +} + static bool range_is_memory(u64 start, u64 end) { struct kvm_mem_range r1, r2; From patchwork Mon Aug 9 15:24:43 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Quentin Perret X-Patchwork-Id: 12426827 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-17.5 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_ADSP_CUSTOM_MED,DKIM_SIGNED,DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, URIBL_BLOCKED,USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 8C08DC4338F for ; Mon, 9 Aug 2021 15:38:22 +0000 (UTC) Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 4C3BB60EDF for ; Mon, 9 Aug 2021 15:38:22 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.4.1 mail.kernel.org 4C3BB60EDF Authentication-Results: mail.kernel.org; dmarc=fail (p=reject dis=none) header.from=google.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:Cc:To:From:Subject:References: Mime-Version:Message-Id:In-Reply-To:Date:Reply-To:Content-ID: Content-Description:Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc :Resent-Message-ID:List-Owner; bh=v+w41Y1MrxvaSOhR8SQn9cR1ERHD/4bKnB9N85CIt0w=; b=Y7Qk5G8FBTXmRCjE+XYlJ8J4e3 tYuxLk/uJFiOxARdiXcd1kv05naJJPLZg0BlQ2lEHf2yYbkotnKhrZeO0plxLAhXGVaqcMZfpZNd7 aTRNgAI707vTbYsErW0+ebUOAM0XZMqO7ryo7BmV8sTTs90TFpnxsIArEUnLWo0FOhmHcGOpDkh/6 tfcJYzDeH6izQA76xDU9Rxz0js5zlMLoNB3orvFQeS3VzTvwkpVNLwa8CD2x3mYpMtPwMTaoOXvQx PxgWAZZPtWZ+AjDj9UkBCSB5MNFLY4GdRs8C4AIUCqtFi1wJ4uUO/UCyU0yxe/WiMCiPcdp6iqPSf fi7NYi5A==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1mD7JL-001CIU-Ne; Mon, 09 Aug 2021 15:35:32 +0000 Received: from mail-wr1-x44a.google.com ([2a00:1450:4864:20::44a]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1mD79d-00180x-4m for linux-arm-kernel@lists.infradead.org; Mon, 09 Aug 2021 15:25:31 +0000 Received: by mail-wr1-x44a.google.com with SMTP id z10-20020a5d440a0000b0290154e0f00348so1999700wrq.4 for ; Mon, 09 Aug 2021 08:25:27 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=date:in-reply-to:message-id:mime-version:references:subject:from:to :cc; bh=aj8MBR15jNb/eqSbuMjCVp1+vMaSCorP22m0gXjnWP4=; b=KglZAeR1qSe4Fux9NKVSqLH5ETyNyLwo5BwAI8rIRoar0U1eIzKsZBbQskglFPq/tl 5wqV/Ut+gPY5oqueMzmWHR5oDZDZJeH0fiPF9GJU/HsJOHnSdK+3cGq7qAQUidtxQH/R 7TVpr0n9seO4pWJOJIN2bYZ0Tx4rSBgKFVt4FjlPQyP6I0BU+Lww/FVIozy0ZUYBFzRr 23ElyYL09R13mKCKacrpORmkccN37M4BnHPdt9fwA1QAK9rJOYlIOiqiUwwVUoAnbfkK INPA8y0/M0cYEr4FkzwW5r1AbfmtNKc81EiE4J3BrikgtA/oqRHJPLojwEKVnIbWh5de NQBg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=aj8MBR15jNb/eqSbuMjCVp1+vMaSCorP22m0gXjnWP4=; b=e9Jkjqo3RbVj8P6eZ3bGMzqFzBDxoMUuWaRC6rizteAoKjMQoqBttYe8TCOodPRYbo nKg4s20QHwOt18+oo4i0sUahGQ622tTumCADztnuqUO93LtA+rG7AiavzGeco5vSZlus uMPyc8EUGJmms7txxE3NOzlKFe2KIOggzQuCfLq19iueO3zV1dJb1R/wQlAat6IIfSzZ KJSOfo863uDwcqT0u7Me0HYTl6TSl53RF3c87i18YpkUFSeiLverbiXTFVhAQtbBtnbZ e4aODV/odM/E2PY8rx819a09EeldIcH5ASEhiua3gA5xg8i1+goazxFuEuy4Wr9MgAEM 4D6A== X-Gm-Message-State: AOAM531BLtpuvMYrQEcX60TKClDKXg/WOS4GSucUl+FxG5te83Fvicmw tSsKOjbfd+GwnEiX9KWJ78W1TsPy7uw4 X-Google-Smtp-Source: ABdhPJxO6Ief2rDrUHALAL2m66Q1K9ZTpGeNI4IRrYBo8N3BxkN9OXPutXcNR00+t/uvXxU7PCGCiVC7nza1 X-Received: from luke.lon.corp.google.com ([2a00:79e0:d:210:b0e8:d460:758b:a0ae]) (user=qperret job=sendgmr) by 2002:a7b:cb09:: with SMTP id u9mr34069444wmj.63.1628522726484; Mon, 09 Aug 2021 08:25:26 -0700 (PDT) Date: Mon, 9 Aug 2021 16:24:43 +0100 In-Reply-To: <20210809152448.1810400-1-qperret@google.com> Message-Id: <20210809152448.1810400-17-qperret@google.com> Mime-Version: 1.0 References: <20210809152448.1810400-1-qperret@google.com> X-Mailer: git-send-email 2.32.0.605.g8dce9f2422-goog Subject: [PATCH v4 16/21] KVM: arm64: Enable retrieving protections attributes of PTEs From: Quentin Perret To: maz@kernel.org, james.morse@arm.com, alexandru.elisei@arm.com, suzuki.poulose@arm.com, catalin.marinas@arm.com, will@kernel.org Cc: linux-arm-kernel@lists.infradead.org, kvmarm@lists.cs.columbia.edu, linux-kernel@vger.kernel.org, ardb@kernel.org, qwandor@google.com, tabba@google.com, dbrazdil@google.com, kernel-team@android.com, qperret@google.com X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20210809_082529_273965_7A432EA9 X-CRM114-Status: GOOD ( 13.08 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Introduce helper functions in the KVM stage-2 and stage-1 page-table manipulation library allowing to retrieve the enum kvm_pgtable_prot of a PTE. This will be useful to implement custom walkers outside of pgtable.c. Signed-off-by: Quentin Perret Reviewed-by: Fuad Tabba --- arch/arm64/include/asm/kvm_pgtable.h | 20 +++++++++++++++ arch/arm64/kvm/hyp/pgtable.c | 37 ++++++++++++++++++++++++++++ 2 files changed, 57 insertions(+) diff --git a/arch/arm64/include/asm/kvm_pgtable.h b/arch/arm64/include/asm/kvm_pgtable.h index ff9d52f8073a..f1651e0153ad 100644 --- a/arch/arm64/include/asm/kvm_pgtable.h +++ b/arch/arm64/include/asm/kvm_pgtable.h @@ -506,4 +506,24 @@ int kvm_pgtable_walk(struct kvm_pgtable *pgt, u64 addr, u64 size, */ int kvm_pgtable_get_leaf(struct kvm_pgtable *pgt, u64 addr, kvm_pte_t *ptep, u32 *level); + +/** + * kvm_pgtable_stage2_pte_prot() - Retrieve the protection attributes of a + * stage-2 Page-Table Entry. + * @pte: Page-table entry + * + * Return: protection attributes of the page-table entry in the enum + * kvm_pgtable_prot format. + */ +enum kvm_pgtable_prot kvm_pgtable_stage2_pte_prot(kvm_pte_t pte); + +/** + * kvm_pgtable_hyp_pte_prot() - Retrieve the protection attributes of a stage-1 + * Page-Table Entry. + * @pte: Page-table entry + * + * Return: protection attributes of the page-table entry in the enum + * kvm_pgtable_prot format. + */ +enum kvm_pgtable_prot kvm_pgtable_hyp_pte_prot(kvm_pte_t pte); #endif /* __ARM64_KVM_PGTABLE_H__ */ diff --git a/arch/arm64/kvm/hyp/pgtable.c b/arch/arm64/kvm/hyp/pgtable.c index cff744136044..f8ceebe4982e 100644 --- a/arch/arm64/kvm/hyp/pgtable.c +++ b/arch/arm64/kvm/hyp/pgtable.c @@ -363,6 +363,26 @@ static int hyp_set_prot_attr(enum kvm_pgtable_prot prot, kvm_pte_t *ptep) return 0; } +enum kvm_pgtable_prot kvm_pgtable_hyp_pte_prot(kvm_pte_t pte) +{ + enum kvm_pgtable_prot prot = pte & KVM_PTE_LEAF_ATTR_HI_SW; + u32 ap; + + if (!kvm_pte_valid(pte)) + return prot; + + if (!(pte & KVM_PTE_LEAF_ATTR_HI_S1_XN)) + prot |= KVM_PGTABLE_PROT_X; + + ap = FIELD_GET(KVM_PTE_LEAF_ATTR_LO_S1_AP, pte); + if (ap == KVM_PTE_LEAF_ATTR_LO_S1_AP_RO) + prot |= KVM_PGTABLE_PROT_R; + else if (ap == KVM_PTE_LEAF_ATTR_LO_S1_AP_RW) + prot |= KVM_PGTABLE_PROT_RW; + + return prot; +} + static bool hyp_pte_needs_update(kvm_pte_t old, kvm_pte_t new) { /* @@ -565,6 +585,23 @@ static int stage2_set_prot_attr(struct kvm_pgtable *pgt, enum kvm_pgtable_prot p return 0; } +enum kvm_pgtable_prot kvm_pgtable_stage2_pte_prot(kvm_pte_t pte) +{ + enum kvm_pgtable_prot prot = pte & KVM_PTE_LEAF_ATTR_HI_SW; + + if (!kvm_pte_valid(pte)) + return prot; + + if (pte & KVM_PTE_LEAF_ATTR_LO_S2_S2AP_R) + prot |= KVM_PGTABLE_PROT_R; + if (pte & KVM_PTE_LEAF_ATTR_LO_S2_S2AP_W) + prot |= KVM_PGTABLE_PROT_W; + if (!(pte & KVM_PTE_LEAF_ATTR_HI_S2_XN)) + prot |= KVM_PGTABLE_PROT_X; + + return prot; +} + static bool stage2_pte_needs_update(kvm_pte_t old, kvm_pte_t new) { if (!kvm_pte_valid(old) || !kvm_pte_valid(new)) From patchwork Mon Aug 9 15:24:44 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Quentin Perret X-Patchwork-Id: 12426829 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-17.5 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_ADSP_CUSTOM_MED,DKIM_SIGNED,DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, URIBL_BLOCKED,USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id C540DC4338F for ; Mon, 9 Aug 2021 15:39:22 +0000 (UTC) Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 74FF160F8F for ; Mon, 9 Aug 2021 15:39:22 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.4.1 mail.kernel.org 74FF160F8F Authentication-Results: mail.kernel.org; dmarc=fail (p=reject dis=none) header.from=google.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:Cc:To:From:Subject:References: Mime-Version:Message-Id:In-Reply-To:Date:Reply-To:Content-ID: Content-Description:Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc :Resent-Message-ID:List-Owner; bh=rlCOMJMK0RgnQrIQqaHPMFByqdMJpil3oGy5DER7Lvo=; b=q/aBwW4uBB5vq5AYJ4bok8gZ3t 9ltF1/hbd3FNf7WTJYxoyyZ0pjJPzvk6HtETWAA+DjehgMFVY+LuF1ezh+zPsKj5yc+bMgVJz5E+p DS20n7hqAp209RLJe4N4vW9ATRL/CYiSCYNnLvru8xaMprT1iN6R1+iUqAsgLI04eOD7h9mh2X01z h55msL7NquEmY6bxExnB+tJeYrMB7/rpAY7bgE1OKRknhKuAUn+acsGd+/9+vLOpYZGvofn53AoB6 b218llSF0rzprZ42ANx43S/Q/8+QeMvMybgg7jHOB9S+gXgkgen+IdV4+/31YwKlj3x0dHex/QlOr xO1h2hpQ==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1mD7KY-001Cja-Dw; Mon, 09 Aug 2021 15:36:47 +0000 Received: from mail-ej1-x649.google.com ([2a00:1450:4864:20::649]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1mD79f-00182L-DW for linux-arm-kernel@lists.infradead.org; Mon, 09 Aug 2021 15:25:33 +0000 Received: by mail-ej1-x649.google.com with SMTP id v19-20020a170906b013b02905b2f1bbf8f3so579304ejy.6 for ; Mon, 09 Aug 2021 08:25:30 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=date:in-reply-to:message-id:mime-version:references:subject:from:to :cc; bh=dKAdWq+um7+5WCTr7spdVMqonShc4XNlQTBou41SonA=; b=X5MaIqpMDpsgElSyO+XEpl5K8TmNOlWy9iyRmNdtwE3HyEvHRuU55AsAcW15E0tQE+ jA7MindYid7mLIBWyt24HUfu2l9pN2kywv5tYtKGVJmJSXG3gEPe8HQuJ1uE3t82jb2R R6I208HaQXPyBO8Gljjafvuzdzjrx0AwC2KgLjl1Or4B38Z2ft0Z6BYxVAt6gyfpwqgX w/kusFQ9wey4B0Z5v7QvjKeXWm8jUbg0/0P+Q1bns7OQUUAjlTXjCQ02a2bo96MuFmHS bWb4AiP60ALuRjZ7a9zAqWfFUrcREzLPOMvxQHN87KICnFZF4oBK0KjWIxyjZ+LcJyBH yhMQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=dKAdWq+um7+5WCTr7spdVMqonShc4XNlQTBou41SonA=; b=hPQil4sHFkEfB6PTRsV/QXgBKls1l35AulRWeTGBOAnE4lTRFuTWRP9Yb2p51ug1yd QvLLUXwNHLEe/G+UkD1kV+6bHmdWurDESSAtiNbTx5/X7jnwK0zGrFh+DttlgxERRocQ Rl08pDqKW9S7RQ6iJNG1T2BEzrznP+nOmr2074ONrye/xWtjCEfJI3uVSz82muiQYEPI aMdmQC+JpB28SbxKT+WM6OCyh13pjRO3rJKG5UJiGtAyHUEzoTj6fXKWsYfA7AYYtgY0 uGJJeYwW9WHv/wLJh5Da319ExwWW0/+pS99kw1wCiChLKQcKmGNROlWt3Fz6zZYjLmRh FOpg== X-Gm-Message-State: AOAM532I8m2rIJCAX1MSdiBmYCK2bhwP1UdFEhl+aYDmmN8oc7BwWs55 5ghvnuDwZQ/BM6gyFy6YTBfPt1nuBuC6 X-Google-Smtp-Source: ABdhPJzPp9qFi5jNPXGgwRFOFgI/f0ff/7l9UlxcQ8qnZH4O21lc+UlEQ/M6EMuoSsJAxxc6uAUQfLyq71O/ X-Received: from luke.lon.corp.google.com ([2a00:79e0:d:210:b0e8:d460:758b:a0ae]) (user=qperret job=sendgmr) by 2002:aa7:cd9a:: with SMTP id x26mr30279924edv.26.1628522728841; Mon, 09 Aug 2021 08:25:28 -0700 (PDT) Date: Mon, 9 Aug 2021 16:24:44 +0100 In-Reply-To: <20210809152448.1810400-1-qperret@google.com> Message-Id: <20210809152448.1810400-18-qperret@google.com> Mime-Version: 1.0 References: <20210809152448.1810400-1-qperret@google.com> X-Mailer: git-send-email 2.32.0.605.g8dce9f2422-goog Subject: [PATCH v4 17/21] KVM: arm64: Mark host bss and rodata section as shared From: Quentin Perret To: maz@kernel.org, james.morse@arm.com, alexandru.elisei@arm.com, suzuki.poulose@arm.com, catalin.marinas@arm.com, will@kernel.org Cc: linux-arm-kernel@lists.infradead.org, kvmarm@lists.cs.columbia.edu, linux-kernel@vger.kernel.org, ardb@kernel.org, qwandor@google.com, tabba@google.com, dbrazdil@google.com, kernel-team@android.com, qperret@google.com X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20210809_082531_529391_F88EDEF7 X-CRM114-Status: GOOD ( 20.48 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org As the hypervisor maps the host's .bss and .rodata sections in its stage-1, make sure to tag them as shared in hyp and host page-tables. But since the hypervisor relies on the presence of these mappings, we cannot let the host in complete control of the memory regions -- it must not unshare or donate them to another entity for example. To prevent this, let's transfer the ownership of those ranges to the hypervisor itself, and share the pages back with the host. Signed-off-by: Quentin Perret Reviewed-by: Fuad Tabba --- arch/arm64/kvm/hyp/nvhe/setup.c | 82 +++++++++++++++++++++++++++++---- 1 file changed, 74 insertions(+), 8 deletions(-) diff --git a/arch/arm64/kvm/hyp/nvhe/setup.c b/arch/arm64/kvm/hyp/nvhe/setup.c index 0b574d106519..57c27846320f 100644 --- a/arch/arm64/kvm/hyp/nvhe/setup.c +++ b/arch/arm64/kvm/hyp/nvhe/setup.c @@ -58,6 +58,7 @@ static int recreate_hyp_mappings(phys_addr_t phys, unsigned long size, { void *start, *end, *virt = hyp_phys_to_virt(phys); unsigned long pgt_size = hyp_s1_pgtable_pages() << PAGE_SHIFT; + enum kvm_pgtable_prot prot; int ret, i; /* Recreate the hyp page-table using the early page allocator */ @@ -83,10 +84,6 @@ static int recreate_hyp_mappings(phys_addr_t phys, unsigned long size, if (ret) return ret; - ret = pkvm_create_mappings(__start_rodata, __end_rodata, PAGE_HYP_RO); - if (ret) - return ret; - ret = pkvm_create_mappings(__hyp_rodata_start, __hyp_rodata_end, PAGE_HYP_RO); if (ret) return ret; @@ -95,10 +92,6 @@ static int recreate_hyp_mappings(phys_addr_t phys, unsigned long size, if (ret) return ret; - ret = pkvm_create_mappings(__hyp_bss_end, __bss_stop, PAGE_HYP_RO); - if (ret) - return ret; - ret = pkvm_create_mappings(virt, virt + size, PAGE_HYP); if (ret) return ret; @@ -117,6 +110,24 @@ static int recreate_hyp_mappings(phys_addr_t phys, unsigned long size, return ret; } + /* + * Map the host's .bss and .rodata sections RO in the hypervisor, but + * transfer the ownership from the host to the hypervisor itself to + * make sure it can't be donated or shared with another entity. + * + * The ownership transition requires matching changes in the host + * stage-2. This will be done later (see finalize_host_mappings()) once + * the hyp_vmemmap is addressable. + */ + prot = pkvm_mkstate(PAGE_HYP_RO, PKVM_PAGE_SHARED_OWNED); + ret = pkvm_create_mappings(__start_rodata, __end_rodata, prot); + if (ret) + return ret; + + ret = pkvm_create_mappings(__hyp_bss_end, __bss_stop, prot); + if (ret) + return ret; + return 0; } @@ -148,6 +159,57 @@ static void hpool_put_page(void *addr) hyp_put_page(&hpool, addr); } +static int finalize_host_mappings_walker(u64 addr, u64 end, u32 level, + kvm_pte_t *ptep, + enum kvm_pgtable_walk_flags flag, + void * const arg) +{ + enum kvm_pgtable_prot prot; + enum pkvm_page_state state; + kvm_pte_t pte = *ptep; + phys_addr_t phys; + + if (!kvm_pte_valid(pte)) + return 0; + + if (level != (KVM_PGTABLE_MAX_LEVELS - 1)) + return -EINVAL; + + phys = kvm_pte_to_phys(pte); + if (!addr_is_memory(phys)) + return 0; + + /* + * Adjust the host stage-2 mappings to match the ownership attributes + * configured in the hypervisor stage-1. + */ + state = pkvm_getstate(kvm_pgtable_hyp_pte_prot(pte)); + switch (state) { + case PKVM_PAGE_OWNED: + return host_stage2_set_owner_locked(phys, PAGE_SIZE, pkvm_hyp_id); + case PKVM_PAGE_SHARED_OWNED: + prot = pkvm_mkstate(PKVM_HOST_MEM_PROT, PKVM_PAGE_SHARED_BORROWED); + break; + case PKVM_PAGE_SHARED_BORROWED: + prot = pkvm_mkstate(PKVM_HOST_MEM_PROT, PKVM_PAGE_SHARED_OWNED); + break; + default: + return -EINVAL; + } + + return host_stage2_idmap_locked(phys, PAGE_SIZE, prot); +} + +static int finalize_host_mappings(void) +{ + struct kvm_pgtable_walker walker = { + .cb = finalize_host_mappings_walker, + .flags = KVM_PGTABLE_WALK_LEAF, + }; + + return kvm_pgtable_walk(&pkvm_pgtable, 0, BIT(pkvm_pgtable.ia_bits), &walker); +} + void __noreturn __pkvm_init_finalise(void) { struct kvm_host_data *host_data = this_cpu_ptr(&kvm_host_data); @@ -167,6 +229,10 @@ void __noreturn __pkvm_init_finalise(void) if (ret) goto out; + ret = finalize_host_mappings(); + if (ret) + goto out; + pkvm_pgtable_mm_ops = (struct kvm_pgtable_mm_ops) { .zalloc_page = hyp_zalloc_hyp_page, .phys_to_virt = hyp_phys_to_virt, From patchwork Mon Aug 9 15:24:45 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Quentin Perret X-Patchwork-Id: 12426831 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-17.5 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_ADSP_CUSTOM_MED,DKIM_SIGNED,DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, URIBL_BLOCKED,USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 7B436C4338F for ; Mon, 9 Aug 2021 15:40:53 +0000 (UTC) Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 2126E60FF2 for ; Mon, 9 Aug 2021 15:40:52 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.4.1 mail.kernel.org 2126E60FF2 Authentication-Results: mail.kernel.org; dmarc=fail (p=reject dis=none) header.from=google.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:Cc:To:From:Subject:References: Mime-Version:Message-Id:In-Reply-To:Date:Reply-To:Content-ID: Content-Description:Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc :Resent-Message-ID:List-Owner; bh=nPwjmrUnyDLXN6vFbSzdUlPHsqjW7MoA+H0qZ4+BIVU=; b=2XkAiqCMGgDa8qYmqrVaDvt7aD DPyATqfDPjC3epHCJNykjW8RXoiu7pJLaNE4m+w1ixKq9KSvmw5PHrfMNbQ3jf6LtMAugMgCXpHM9 iS1l8sCqofqSgH46OLTJ+311PPf/kAVXtN5btYXd+L6exlL/iVtEFOeg8bZfp1Avu+LQwUZX3obdE BqvwutkibtN9TlMyq3afmsJeg8a5WURm7VhZWelC/8bdQS4Qtl855Z9EgI53bnYyaVjA7m4zv/tRQ ajehwAQBP+xxKQ5TooXhiTIq3MKeyq98ZIHj9XJJCEW0GZKR78VRVBfsm0c/D3CqlW/kmbYrFMTOs UdXQEogw==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1mD7Lr-001DAW-8C; Mon, 09 Aug 2021 15:38:08 +0000 Received: from mail-qk1-x74a.google.com ([2607:f8b0:4864:20::74a]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1mD79g-001833-MO for linux-arm-kernel@lists.infradead.org; Mon, 09 Aug 2021 15:25:34 +0000 Received: by mail-qk1-x74a.google.com with SMTP id a133-20020a37668b0000b02903cee5e97f26so5794695qkc.11 for ; Mon, 09 Aug 2021 08:25:31 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=date:in-reply-to:message-id:mime-version:references:subject:from:to :cc; bh=MNGuBytX5NXMQPGh03MXz48l5IrVanSNLiyhXmu6HLc=; b=PARTJpvAdy7O5RqWHTAlFxcXu05/69ppZA0ujC1MkD/awV08LT++mW1Ot31gr/o2oE L7mInUCVpVyODycSxKTxKYGD7JUtSxBxwDtsTSLz4lX0jlNM2ZmxK+CywSCOB5nMw02q yJsSlJNdEe8QcdmwQo5IEqgbCrY9NLZRHKscsVzqrj8H6XZ0z2glBxgii77jHP/IrnNf ycRYzjVA6vW1A8kqhst9lDLKuRB3MV7VBWf7GNDsfD6uxVDkw/mErsEmhhp3lPDDd6p1 8bMcTEPHuzFSiF3VHwnYVS/IR5yd2vUrfefNVmIVJfrlBSdnquN33PGrAdR1ZKp4pfDj f1Xg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=MNGuBytX5NXMQPGh03MXz48l5IrVanSNLiyhXmu6HLc=; b=ck/rULvE0ucoOp/nKGA1HgygEEh9vdxpyFr0/FtJKBnadAATQVyuQde/rwvctM1HWQ HN9tj7HYQQDWk3+dg6gG6/ZkH6gVQnsLKnaaKI/atGPcOoarbYmuAojEVjcYtjYmoihr nHxAYkgnQRYUI4IyX9MV9+ujrQrGjRWjBlIUL5+69dxJavxxrJBjPR/KF9Awg0Wnfdz5 GdTAxd3tV26U8wHDppw/qSMAEJD89aWBLAqq76opqgP3LdnZ08lAZN+SnKlHbL5rHfXR WQcfby0lqUGrHOgn6goeCmF6KHSq6NOatUPacmvQj5Dk9DUx1oIbqt4eD9Um7WMjGZ8p 0nOA== X-Gm-Message-State: AOAM531STeBDSfnWusJHU/cbOzXN3vVLlkckDaKGa4SPzK76woTj4lqP szttgqGwFHkFKBbVKrbbYjJHCIYtF4VY X-Google-Smtp-Source: ABdhPJw4mf35HGX4mhejZnF84rlhADJ8aR2DHNMQwNPUuRpUPyARXjrqqyYC3OYWgsFBlllhXreLHcRP/iXd X-Received: from luke.lon.corp.google.com ([2a00:79e0:d:210:b0e8:d460:758b:a0ae]) (user=qperret job=sendgmr) by 2002:a0c:f5c8:: with SMTP id q8mr6468132qvm.5.1628522731047; Mon, 09 Aug 2021 08:25:31 -0700 (PDT) Date: Mon, 9 Aug 2021 16:24:45 +0100 In-Reply-To: <20210809152448.1810400-1-qperret@google.com> Message-Id: <20210809152448.1810400-19-qperret@google.com> Mime-Version: 1.0 References: <20210809152448.1810400-1-qperret@google.com> X-Mailer: git-send-email 2.32.0.605.g8dce9f2422-goog Subject: [PATCH v4 18/21] KVM: arm64: Remove __pkvm_mark_hyp From: Quentin Perret To: maz@kernel.org, james.morse@arm.com, alexandru.elisei@arm.com, suzuki.poulose@arm.com, catalin.marinas@arm.com, will@kernel.org Cc: linux-arm-kernel@lists.infradead.org, kvmarm@lists.cs.columbia.edu, linux-kernel@vger.kernel.org, ardb@kernel.org, qwandor@google.com, tabba@google.com, dbrazdil@google.com, kernel-team@android.com, qperret@google.com X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20210809_082532_797491_321DBEF3 X-CRM114-Status: GOOD ( 16.21 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Now that we mark memory owned by the hypervisor in the host stage-2 during __pkvm_init(), we no longer need to rely on the host to explicitly mark the hyp sections later on. Remove the __pkvm_mark_hyp() hypercall altogether. Signed-off-by: Quentin Perret Reviewed-by: Fuad Tabba --- arch/arm64/include/asm/kvm_asm.h | 3 +- arch/arm64/kvm/arm.c | 46 ------------------- arch/arm64/kvm/hyp/include/nvhe/mem_protect.h | 1 - arch/arm64/kvm/hyp/nvhe/hyp-main.c | 9 ---- arch/arm64/kvm/hyp/nvhe/mem_protect.c | 19 -------- 5 files changed, 1 insertion(+), 77 deletions(-) diff --git a/arch/arm64/include/asm/kvm_asm.h b/arch/arm64/include/asm/kvm_asm.h index 9f0bf2109be7..432a9ea1f02e 100644 --- a/arch/arm64/include/asm/kvm_asm.h +++ b/arch/arm64/include/asm/kvm_asm.h @@ -63,8 +63,7 @@ #define __KVM_HOST_SMCCC_FUNC___pkvm_create_private_mapping 17 #define __KVM_HOST_SMCCC_FUNC___pkvm_cpu_set_vector 18 #define __KVM_HOST_SMCCC_FUNC___pkvm_prot_finalize 19 -#define __KVM_HOST_SMCCC_FUNC___pkvm_mark_hyp 20 -#define __KVM_HOST_SMCCC_FUNC___kvm_adjust_pc 21 +#define __KVM_HOST_SMCCC_FUNC___kvm_adjust_pc 20 #ifndef __ASSEMBLY__ diff --git a/arch/arm64/kvm/arm.c b/arch/arm64/kvm/arm.c index e9a2b8f27792..2f378482471b 100644 --- a/arch/arm64/kvm/arm.c +++ b/arch/arm64/kvm/arm.c @@ -1954,57 +1954,11 @@ static void _kvm_host_prot_finalize(void *discard) WARN_ON(kvm_call_hyp_nvhe(__pkvm_prot_finalize)); } -static inline int pkvm_mark_hyp(phys_addr_t start, phys_addr_t end) -{ - return kvm_call_hyp_nvhe(__pkvm_mark_hyp, start, end); -} - -#define pkvm_mark_hyp_section(__section) \ - pkvm_mark_hyp(__pa_symbol(__section##_start), \ - __pa_symbol(__section##_end)) - static int finalize_hyp_mode(void) { - int cpu, ret; - if (!is_protected_kvm_enabled()) return 0; - ret = pkvm_mark_hyp_section(__hyp_idmap_text); - if (ret) - return ret; - - ret = pkvm_mark_hyp_section(__hyp_text); - if (ret) - return ret; - - ret = pkvm_mark_hyp_section(__hyp_rodata); - if (ret) - return ret; - - ret = pkvm_mark_hyp_section(__hyp_bss); - if (ret) - return ret; - - ret = pkvm_mark_hyp(hyp_mem_base, hyp_mem_base + hyp_mem_size); - if (ret) - return ret; - - for_each_possible_cpu(cpu) { - phys_addr_t start = virt_to_phys((void *)kvm_arm_hyp_percpu_base[cpu]); - phys_addr_t end = start + (PAGE_SIZE << nvhe_percpu_order()); - - ret = pkvm_mark_hyp(start, end); - if (ret) - return ret; - - start = virt_to_phys((void *)per_cpu(kvm_arm_hyp_stack_page, cpu)); - end = start + PAGE_SIZE; - ret = pkvm_mark_hyp(start, end); - if (ret) - return ret; - } - /* * Flip the static key upfront as that may no longer be possible * once the host stage 2 is installed. diff --git a/arch/arm64/kvm/hyp/include/nvhe/mem_protect.h b/arch/arm64/kvm/hyp/include/nvhe/mem_protect.h index 49db0ec5a606..0118527b07b0 100644 --- a/arch/arm64/kvm/hyp/include/nvhe/mem_protect.h +++ b/arch/arm64/kvm/hyp/include/nvhe/mem_protect.h @@ -49,7 +49,6 @@ extern struct host_kvm host_kvm; extern const u8 pkvm_hyp_id; int __pkvm_prot_finalize(void); -int __pkvm_mark_hyp(phys_addr_t start, phys_addr_t end); bool addr_is_memory(phys_addr_t phys); int host_stage2_idmap_locked(phys_addr_t addr, u64 size, enum kvm_pgtable_prot prot); diff --git a/arch/arm64/kvm/hyp/nvhe/hyp-main.c b/arch/arm64/kvm/hyp/nvhe/hyp-main.c index 1632f001f4ed..7900d5b66ba3 100644 --- a/arch/arm64/kvm/hyp/nvhe/hyp-main.c +++ b/arch/arm64/kvm/hyp/nvhe/hyp-main.c @@ -163,14 +163,6 @@ static void handle___pkvm_prot_finalize(struct kvm_cpu_context *host_ctxt) { cpu_reg(host_ctxt, 1) = __pkvm_prot_finalize(); } - -static void handle___pkvm_mark_hyp(struct kvm_cpu_context *host_ctxt) -{ - DECLARE_REG(phys_addr_t, start, host_ctxt, 1); - DECLARE_REG(phys_addr_t, end, host_ctxt, 2); - - cpu_reg(host_ctxt, 1) = __pkvm_mark_hyp(start, end); -} typedef void (*hcall_t)(struct kvm_cpu_context *); #define HANDLE_FUNC(x) [__KVM_HOST_SMCCC_FUNC_##x] = (hcall_t)handle_##x @@ -196,7 +188,6 @@ static const hcall_t host_hcall[] = { HANDLE_FUNC(__pkvm_create_mappings), HANDLE_FUNC(__pkvm_create_private_mapping), HANDLE_FUNC(__pkvm_prot_finalize), - HANDLE_FUNC(__pkvm_mark_hyp), }; static void handle_host_hcall(struct kvm_cpu_context *host_ctxt) diff --git a/arch/arm64/kvm/hyp/nvhe/mem_protect.c b/arch/arm64/kvm/hyp/nvhe/mem_protect.c index cb023d31666e..2991dc6996b9 100644 --- a/arch/arm64/kvm/hyp/nvhe/mem_protect.c +++ b/arch/arm64/kvm/hyp/nvhe/mem_protect.c @@ -339,25 +339,6 @@ static int host_stage2_idmap(u64 addr) return ret; } -int __pkvm_mark_hyp(phys_addr_t start, phys_addr_t end) -{ - int ret; - - /* - * host_stage2_unmap_dev_all() currently relies on MMIO mappings being - * non-persistent, so don't allow changing page ownership in MMIO range. - */ - if (!range_is_memory(start, end)) - return -EINVAL; - - hyp_spin_lock(&host_kvm.lock); - ret = host_stage2_try(kvm_pgtable_stage2_set_owner, &host_kvm.pgt, - start, end - start, &host_s2_pool, pkvm_hyp_id); - hyp_spin_unlock(&host_kvm.lock); - - return ret != -EAGAIN ? ret : 0; -} - void handle_host_mem_abort(struct kvm_cpu_context *host_ctxt) { struct kvm_vcpu_fault_info fault; From patchwork Mon Aug 9 15:24:46 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Quentin Perret X-Patchwork-Id: 12426833 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-17.5 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_ADSP_CUSTOM_MED,DKIM_SIGNED,DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, URIBL_BLOCKED,USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id D99B8C4338F for ; Mon, 9 Aug 2021 15:42:00 +0000 (UTC) Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 7A6656101D for ; Mon, 9 Aug 2021 15:42:00 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.4.1 mail.kernel.org 7A6656101D Authentication-Results: mail.kernel.org; dmarc=fail (p=reject dis=none) header.from=google.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:Cc:To:From:Subject:References: Mime-Version:Message-Id:In-Reply-To:Date:Reply-To:Content-ID: Content-Description:Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc :Resent-Message-ID:List-Owner; bh=VYnKtRsEMVxIFldd6bXc9HyPztC2X39gH4WSFYSOtsM=; b=pcLBBjq7ulAouAsdWfn7/kp9yy flcaGKtFa+i2Xuadd5UzLjn/MT7Y1dOGrxk0OS03dU1ayWYUhytgHACeXhEcnaYFsDnk93vmtVoj0 2c9N37r9BbGv4v9d5lHNZkcW0lIOkifgH8Dy4606XxHNqNGnwUhZAkTa0DkxgobyhPby3d/MZ1Kel EeORFb//v/r170tIES429Vccf1BywOuc96KGNXNUJdI4hHlXEodwHzn1LpaxrAZIbU5uZ2FWwJBUh dTuh0vBmm+YZ4Ce/CbCLC3tDksNzAefI+uibpkyL091Nv2ZE1oTTRmPDOOEuo83PaQZR24YXEBv+o WogrmIaw==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1mD7N3-001Dbi-0H; Mon, 09 Aug 2021 15:39:21 +0000 Received: from mail-wm1-x34a.google.com ([2a00:1450:4864:20::34a]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1mD79j-00184l-8Y for linux-arm-kernel@lists.infradead.org; Mon, 09 Aug 2021 15:25:38 +0000 Received: by mail-wm1-x34a.google.com with SMTP id l38-20020a05600c1d26b0290259bef426efso113681wms.8 for ; Mon, 09 Aug 2021 08:25:34 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=date:in-reply-to:message-id:mime-version:references:subject:from:to :cc; bh=0vnFX0HCkFp8ZLTU0btBJYZAdzWXKCWbI3FuYOfCODA=; b=edLLNeSUJUuSeZh6U5LgHuiFbQdxbqRKi/NQ1RSRnsUEn2S/8ryLev1RWgMSfDbltm Om9IrwxtxrsFpwCskBaMEJ7xBMB6hChrlzhzMzZ2whQvKh0X0/PsQAyhN2UbcMabEKhJ kFITxof5dzgHb27rpT/TUwsw9eujgADfYDSuydpjHXLWRa3f3mUrl1r2tJHF5HcYrnXM ZVct81ad5/jrKV4yq33XFH/l0LAM02qWUJwCBxnaoSuNVHOUDowrr9GS0w/R/KbWvIzV XsiLCk2wB36vg/uxYgid0GyY+bEKFJ/s794tI7nnae4HsUsaHCOpJbS240pLizQYB85F EWrA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=0vnFX0HCkFp8ZLTU0btBJYZAdzWXKCWbI3FuYOfCODA=; b=uDBt5nppB9UfxLWTJDmdvv53BIuxLfK59WK5li3wHJpnWgBnSLFt+kh6C6D3GjfQbS EizTL36Y6Yz3JesKPf5gOgqFhS9CS2R3UyQD7eZYocCB88pZDE2aztnUnM+2MhZQ3YSu LKQ+IYPRC6unKx3dLCR0vNcGDOxnAKLIjKsLvt8h7Hr7utAxQWwWYwenIuuXJNm+bWis /A0idQp/gOoZimqpBQJKJ+97tIK+lHbqNWeCSPNEnLaiTMtvWwgZAIwPzIwyH54KaZKE w46FrNZVxsQW06EejVGOMoFEEOJLSqWYDvunKJmzuk/DkMW1V9tbHzXHr8UL4575vaE1 fLmA== X-Gm-Message-State: AOAM531sE50C6GmSvC6zIOjWuPTEpX0NDvm9kL3cchkVU6ep0VSOqWBE REYMrZndWBxWXOqKfHXMsKIhorEcbfhm X-Google-Smtp-Source: ABdhPJyIOtnzxP4S0R25oU2iubMhTBhg+aNqUgDs+Hxip8y4Nj2Lxx59WuSTFwfisX1gdgavPWTXBcPgInrK X-Received: from luke.lon.corp.google.com ([2a00:79e0:d:210:b0e8:d460:758b:a0ae]) (user=qperret job=sendgmr) by 2002:a7b:cb99:: with SMTP id m25mr33616400wmi.90.1628522733253; Mon, 09 Aug 2021 08:25:33 -0700 (PDT) Date: Mon, 9 Aug 2021 16:24:46 +0100 In-Reply-To: <20210809152448.1810400-1-qperret@google.com> Message-Id: <20210809152448.1810400-20-qperret@google.com> Mime-Version: 1.0 References: <20210809152448.1810400-1-qperret@google.com> X-Mailer: git-send-email 2.32.0.605.g8dce9f2422-goog Subject: [PATCH v4 19/21] KVM: arm64: Refactor protected nVHE stage-1 locking From: Quentin Perret To: maz@kernel.org, james.morse@arm.com, alexandru.elisei@arm.com, suzuki.poulose@arm.com, catalin.marinas@arm.com, will@kernel.org Cc: linux-arm-kernel@lists.infradead.org, kvmarm@lists.cs.columbia.edu, linux-kernel@vger.kernel.org, ardb@kernel.org, qwandor@google.com, tabba@google.com, dbrazdil@google.com, kernel-team@android.com, qperret@google.com X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20210809_082535_402050_C9F199B5 X-CRM114-Status: GOOD ( 13.90 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Refactor the hypervisor stage-1 locking in nVHE protected mode to expose a new pkvm_create_mappings_locked() function. This will be used in later patches to allow walking and changing the hypervisor stage-1 without releasing the lock. Signed-off-by: Quentin Perret Reviewed-by: Fuad Tabba --- arch/arm64/kvm/hyp/include/nvhe/mm.h | 1 + arch/arm64/kvm/hyp/nvhe/mm.c | 18 ++++++++++++++++-- 2 files changed, 17 insertions(+), 2 deletions(-) diff --git a/arch/arm64/kvm/hyp/include/nvhe/mm.h b/arch/arm64/kvm/hyp/include/nvhe/mm.h index 8ec3a5a7744b..c76d7136ed9b 100644 --- a/arch/arm64/kvm/hyp/include/nvhe/mm.h +++ b/arch/arm64/kvm/hyp/include/nvhe/mm.h @@ -23,6 +23,7 @@ int hyp_map_vectors(void); int hyp_back_vmemmap(phys_addr_t phys, unsigned long size, phys_addr_t back); int pkvm_cpu_set_vector(enum arm64_hyp_spectre_vector slot); int pkvm_create_mappings(void *from, void *to, enum kvm_pgtable_prot prot); +int pkvm_create_mappings_locked(void *from, void *to, enum kvm_pgtable_prot prot); int __pkvm_create_mappings(unsigned long start, unsigned long size, unsigned long phys, enum kvm_pgtable_prot prot); unsigned long __pkvm_create_private_mapping(phys_addr_t phys, size_t size, diff --git a/arch/arm64/kvm/hyp/nvhe/mm.c b/arch/arm64/kvm/hyp/nvhe/mm.c index a8efdf0f9003..6fbe8e8030f6 100644 --- a/arch/arm64/kvm/hyp/nvhe/mm.c +++ b/arch/arm64/kvm/hyp/nvhe/mm.c @@ -67,13 +67,15 @@ unsigned long __pkvm_create_private_mapping(phys_addr_t phys, size_t size, return addr; } -int pkvm_create_mappings(void *from, void *to, enum kvm_pgtable_prot prot) +int pkvm_create_mappings_locked(void *from, void *to, enum kvm_pgtable_prot prot) { unsigned long start = (unsigned long)from; unsigned long end = (unsigned long)to; unsigned long virt_addr; phys_addr_t phys; + hyp_assert_lock_held(&pkvm_pgd_lock); + start = start & PAGE_MASK; end = PAGE_ALIGN(end); @@ -81,7 +83,8 @@ int pkvm_create_mappings(void *from, void *to, enum kvm_pgtable_prot prot) int err; phys = hyp_virt_to_phys((void *)virt_addr); - err = __pkvm_create_mappings(virt_addr, PAGE_SIZE, phys, prot); + err = kvm_pgtable_hyp_map(&pkvm_pgtable, virt_addr, PAGE_SIZE, + phys, prot); if (err) return err; } @@ -89,6 +92,17 @@ int pkvm_create_mappings(void *from, void *to, enum kvm_pgtable_prot prot) return 0; } +int pkvm_create_mappings(void *from, void *to, enum kvm_pgtable_prot prot) +{ + int ret; + + hyp_spin_lock(&pkvm_pgd_lock); + ret = pkvm_create_mappings_locked(from, to, prot); + hyp_spin_unlock(&pkvm_pgd_lock); + + return ret; +} + int hyp_back_vmemmap(phys_addr_t phys, unsigned long size, phys_addr_t back) { unsigned long start, end; From patchwork Mon Aug 9 15:24:47 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Quentin Perret X-Patchwork-Id: 12426835 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-17.5 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_ADSP_CUSTOM_MED,DKIM_SIGNED,DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, URIBL_BLOCKED,USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id D7965C4338F for ; Mon, 9 Aug 2021 15:43:13 +0000 (UTC) Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 7546660F93 for ; Mon, 9 Aug 2021 15:43:13 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.4.1 mail.kernel.org 7546660F93 Authentication-Results: mail.kernel.org; dmarc=fail (p=reject dis=none) header.from=google.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:Cc:To:From:Subject:References: Mime-Version:Message-Id:In-Reply-To:Date:Reply-To:Content-ID: Content-Description:Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc :Resent-Message-ID:List-Owner; bh=GYUuoyA5hHJdoV+S81d4j9RUbBuC9ZJiNvM+MyNmAxw=; b=y9avq4brlWWBB967G3ZwlfxtkA A5zjf1OsmWtkTT0K7v6usE6npFzGj8HWj/0w+Yesloga5xVyVJIiDK8qLTH8wZsARdt4LlNHuTMs8 M7AZQkIErkLP/ZeetZiU3dBGj35ow4Nj2RO9sJr/7/ZN0RYgclSy3M5R2A65YE/C9jXdMn4xmdUty EVrO4J/IHDaP/w3zk9DZm0sDVT/CoySLyauWSGWn+ARfBre5Joqqlux0wL1L9gxK/z5T/Y00DEuH8 RpDgbSOrhBq+LwHrYt81QlGFZC6qYAfX5WWj/6shPikk3rBJqm15U6anVbigSf6Kl9alpal22Grop rV5eTctA==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1mD7OE-001E0R-CF; Mon, 09 Aug 2021 15:40:35 +0000 Received: from mail-wm1-x349.google.com ([2a00:1450:4864:20::349]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1mD79l-00186F-SJ for linux-arm-kernel@lists.infradead.org; Mon, 09 Aug 2021 15:25:39 +0000 Received: by mail-wm1-x349.google.com with SMTP id 17-20020a1c04110000b02902e687e6276dso1635084wme.3 for ; Mon, 09 Aug 2021 08:25:36 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=date:in-reply-to:message-id:mime-version:references:subject:from:to :cc; bh=KFZVGZ+Mzblb6mTT1CbKeaimyRVzKWDYIpGPBkKJMH8=; b=pTWmztROLpoZ/YEoV+ztylPoctL8IiBHjZvfPn5BF25G67RHEP1SM5mfeOIA3mYaqD jTjA77K/PVHAviLqbXufrID4P5t/K9hJjg9RIyFKzrsEnXGIHCy0RQlttOVSQrZ96C/8 26dycaUIiMRShVlPsHmbBSkGaAWpT3F5sT5SC1Gq2DSYmzeVqSnHZ2tJBkQyQPnDNrI2 uEBimzTF1mF8hYtql2vkagTqfB3QunZkJ7dmhkO7vmnwI9PhbmY96O9HhCMk6jKvaxYv E8kNrnjWesO1Y1nHStFmPOK75x+zDLL0ERSj/IflF1OU+GZPW93aOnQYF5Hv8ouk+Dx7 Rmhg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=KFZVGZ+Mzblb6mTT1CbKeaimyRVzKWDYIpGPBkKJMH8=; b=PIb5l9sCCq5OwBzZanDUVyE5zUeBT+CMEcevO0jR5bC0A3vd/OLEMemNOP6GV6sefO EnXUiwEGtfiZszQRsr6qgJ159gMq61af6Qe9U/wYdXu6FbgOPqTts8/PgS4wokWre3Vq 5UdqpRUXvAbTZU+7qa3yCS/2jhKITmZMBafIhcNWArpn4ktOrYXp4wjhg3s/3xVxJZVO Cu9l8Q8MS/l1D2ZivyhsGSJU+C5v2iamwEjOenkfLrPy4xlcIKDrUtIa82P+iiTgX4q8 kzPsDBAekQm7s1h4EAxrEByZzbOh0S2FByhUeomTvJmKjPqHTXo1GbEjht1TCCp2naty 3Qhw== X-Gm-Message-State: AOAM532Tda8DcbZ8WSDQ0IDRkDWH4BMDNigkGKNYCB4t9gC1TfjF9lH3 RMlDsU+/PAaqwiYEkK0Vp7V3BJ1QBWlW X-Google-Smtp-Source: ABdhPJyvCDOjAG2Hu7aKsLby4Z+3QpOn0rmCk7GJ0Z+BakNZ9SXQUH+1+hp9v3IxblCUxUDyje28JERIawZx X-Received: from luke.lon.corp.google.com ([2a00:79e0:d:210:b0e8:d460:758b:a0ae]) (user=qperret job=sendgmr) by 2002:a05:600c:5116:: with SMTP id o22mr72964wms.0.1628522735422; Mon, 09 Aug 2021 08:25:35 -0700 (PDT) Date: Mon, 9 Aug 2021 16:24:47 +0100 In-Reply-To: <20210809152448.1810400-1-qperret@google.com> Message-Id: <20210809152448.1810400-21-qperret@google.com> Mime-Version: 1.0 References: <20210809152448.1810400-1-qperret@google.com> X-Mailer: git-send-email 2.32.0.605.g8dce9f2422-goog Subject: [PATCH v4 20/21] KVM: arm64: Restrict EL2 stage-1 changes in protected mode From: Quentin Perret To: maz@kernel.org, james.morse@arm.com, alexandru.elisei@arm.com, suzuki.poulose@arm.com, catalin.marinas@arm.com, will@kernel.org Cc: linux-arm-kernel@lists.infradead.org, kvmarm@lists.cs.columbia.edu, linux-kernel@vger.kernel.org, ardb@kernel.org, qwandor@google.com, tabba@google.com, dbrazdil@google.com, kernel-team@android.com, qperret@google.com X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20210809_082537_986945_A5A7E5EB X-CRM114-Status: GOOD ( 22.75 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org The host kernel is currently able to change EL2 stage-1 mappings without restrictions thanks to the __pkvm_create_mappings() hypercall. But in a world where the host is no longer part of the TCB, this clearly poses a problem. To fix this, introduce a new hypercall to allow the host to share a physical memory page with the hypervisor, and remove the __pkvm_create_mappings() variant. The new hypercall implements ownership and permission checks before allowing the sharing operation, and it annotates the shared page in the hypervisor stage-1 and host stage-2 page-tables. Signed-off-by: Quentin Perret Reviewed-by: Fuad Tabba --- arch/arm64/include/asm/kvm_asm.h | 2 +- arch/arm64/kvm/hyp/include/nvhe/mem_protect.h | 1 + arch/arm64/kvm/hyp/nvhe/hyp-main.c | 11 +-- arch/arm64/kvm/hyp/nvhe/mem_protect.c | 88 +++++++++++++++++++ arch/arm64/kvm/mmu.c | 28 +++++- 5 files changed, 118 insertions(+), 12 deletions(-) diff --git a/arch/arm64/include/asm/kvm_asm.h b/arch/arm64/include/asm/kvm_asm.h index 432a9ea1f02e..aed2aa61766a 100644 --- a/arch/arm64/include/asm/kvm_asm.h +++ b/arch/arm64/include/asm/kvm_asm.h @@ -59,7 +59,7 @@ #define __KVM_HOST_SMCCC_FUNC___vgic_v3_save_aprs 13 #define __KVM_HOST_SMCCC_FUNC___vgic_v3_restore_aprs 14 #define __KVM_HOST_SMCCC_FUNC___pkvm_init 15 -#define __KVM_HOST_SMCCC_FUNC___pkvm_create_mappings 16 +#define __KVM_HOST_SMCCC_FUNC___pkvm_host_share_hyp 16 #define __KVM_HOST_SMCCC_FUNC___pkvm_create_private_mapping 17 #define __KVM_HOST_SMCCC_FUNC___pkvm_cpu_set_vector 18 #define __KVM_HOST_SMCCC_FUNC___pkvm_prot_finalize 19 diff --git a/arch/arm64/kvm/hyp/include/nvhe/mem_protect.h b/arch/arm64/kvm/hyp/include/nvhe/mem_protect.h index 0118527b07b0..03e604f842e2 100644 --- a/arch/arm64/kvm/hyp/include/nvhe/mem_protect.h +++ b/arch/arm64/kvm/hyp/include/nvhe/mem_protect.h @@ -49,6 +49,7 @@ extern struct host_kvm host_kvm; extern const u8 pkvm_hyp_id; int __pkvm_prot_finalize(void); +int __pkvm_host_share_hyp(u64 pfn); bool addr_is_memory(phys_addr_t phys); int host_stage2_idmap_locked(phys_addr_t addr, u64 size, enum kvm_pgtable_prot prot); diff --git a/arch/arm64/kvm/hyp/nvhe/hyp-main.c b/arch/arm64/kvm/hyp/nvhe/hyp-main.c index 7900d5b66ba3..2da6aa8da868 100644 --- a/arch/arm64/kvm/hyp/nvhe/hyp-main.c +++ b/arch/arm64/kvm/hyp/nvhe/hyp-main.c @@ -140,14 +140,11 @@ static void handle___pkvm_cpu_set_vector(struct kvm_cpu_context *host_ctxt) cpu_reg(host_ctxt, 1) = pkvm_cpu_set_vector(slot); } -static void handle___pkvm_create_mappings(struct kvm_cpu_context *host_ctxt) +static void handle___pkvm_host_share_hyp(struct kvm_cpu_context *host_ctxt) { - DECLARE_REG(unsigned long, start, host_ctxt, 1); - DECLARE_REG(unsigned long, size, host_ctxt, 2); - DECLARE_REG(unsigned long, phys, host_ctxt, 3); - DECLARE_REG(enum kvm_pgtable_prot, prot, host_ctxt, 4); + DECLARE_REG(u64, pfn, host_ctxt, 1); - cpu_reg(host_ctxt, 1) = __pkvm_create_mappings(start, size, phys, prot); + cpu_reg(host_ctxt, 1) = __pkvm_host_share_hyp(pfn); } static void handle___pkvm_create_private_mapping(struct kvm_cpu_context *host_ctxt) @@ -185,7 +182,7 @@ static const hcall_t host_hcall[] = { HANDLE_FUNC(__vgic_v3_restore_aprs), HANDLE_FUNC(__pkvm_init), HANDLE_FUNC(__pkvm_cpu_set_vector), - HANDLE_FUNC(__pkvm_create_mappings), + HANDLE_FUNC(__pkvm_host_share_hyp), HANDLE_FUNC(__pkvm_create_private_mapping), HANDLE_FUNC(__pkvm_prot_finalize), }; diff --git a/arch/arm64/kvm/hyp/nvhe/mem_protect.c b/arch/arm64/kvm/hyp/nvhe/mem_protect.c index 2991dc6996b9..8165390d3ec9 100644 --- a/arch/arm64/kvm/hyp/nvhe/mem_protect.c +++ b/arch/arm64/kvm/hyp/nvhe/mem_protect.c @@ -339,6 +339,94 @@ static int host_stage2_idmap(u64 addr) return ret; } +static inline bool check_prot(enum kvm_pgtable_prot prot, + enum kvm_pgtable_prot required, + enum kvm_pgtable_prot denied) +{ + return (prot & (required | denied)) == required; +} + +int __pkvm_host_share_hyp(u64 pfn) +{ + phys_addr_t addr = hyp_pfn_to_phys(pfn); + enum kvm_pgtable_prot prot, cur; + void *virt = __hyp_va(addr); + enum pkvm_page_state state; + kvm_pte_t pte; + int ret; + + if (!addr_is_memory(addr)) + return -EINVAL; + + hyp_spin_lock(&host_kvm.lock); + hyp_spin_lock(&pkvm_pgd_lock); + + ret = kvm_pgtable_get_leaf(&host_kvm.pgt, addr, &pte, NULL); + if (ret) + goto unlock; + if (!pte) + goto map_shared; + + /* + * Check attributes in the host stage-2 PTE. We need the page to be: + * - mapped RWX as we're sharing memory; + * - not borrowed, as that implies absence of ownership. + * Otherwise, we can't let it got through + */ + cur = kvm_pgtable_stage2_pte_prot(pte); + prot = pkvm_mkstate(0, PKVM_PAGE_SHARED_BORROWED); + if (!check_prot(cur, PKVM_HOST_MEM_PROT, prot)) { + ret = -EPERM; + goto unlock; + } + + state = pkvm_getstate(cur); + if (state == PKVM_PAGE_OWNED) + goto map_shared; + + /* + * Tolerate double-sharing the same page, but this requires + * cross-checking the hypervisor stage-1. + */ + if (state != PKVM_PAGE_SHARED_OWNED) { + ret = -EPERM; + goto unlock; + } + + ret = kvm_pgtable_get_leaf(&pkvm_pgtable, (u64)virt, &pte, NULL); + if (ret) + goto unlock; + + /* + * If the page has been shared with the hypervisor, it must be + * already mapped as SHARED_BORROWED in its stage-1. + */ + cur = kvm_pgtable_hyp_pte_prot(pte); + prot = pkvm_mkstate(PAGE_HYP, PKVM_PAGE_SHARED_BORROWED); + if (!check_prot(cur, prot, ~prot)) + ret = EPERM; + goto unlock; + +map_shared: + /* + * If the page is not yet shared, adjust mappings in both page-tables + * while both locks are held. + */ + prot = pkvm_mkstate(PAGE_HYP, PKVM_PAGE_SHARED_BORROWED); + ret = pkvm_create_mappings_locked(virt, virt + PAGE_SIZE, prot); + BUG_ON(ret); + + prot = pkvm_mkstate(PKVM_HOST_MEM_PROT, PKVM_PAGE_SHARED_OWNED); + ret = host_stage2_idmap_locked(addr, PAGE_SIZE, prot); + BUG_ON(ret); + +unlock: + hyp_spin_unlock(&pkvm_pgd_lock); + hyp_spin_unlock(&host_kvm.lock); + + return ret; +} + void handle_host_mem_abort(struct kvm_cpu_context *host_ctxt) { struct kvm_vcpu_fault_info fault; diff --git a/arch/arm64/kvm/mmu.c b/arch/arm64/kvm/mmu.c index 0625bf2353c2..cbab146cda6a 100644 --- a/arch/arm64/kvm/mmu.c +++ b/arch/arm64/kvm/mmu.c @@ -259,10 +259,8 @@ static int __create_hyp_mappings(unsigned long start, unsigned long size, { int err; - if (!kvm_host_owns_hyp_mappings()) { - return kvm_call_hyp_nvhe(__pkvm_create_mappings, - start, size, phys, prot); - } + if (WARN_ON(!kvm_host_owns_hyp_mappings())) + return -EINVAL; mutex_lock(&kvm_hyp_pgd_mutex); err = kvm_pgtable_hyp_map(hyp_pgtable, start, size, phys, prot); @@ -282,6 +280,21 @@ static phys_addr_t kvm_kaddr_to_phys(void *kaddr) } } +static int pkvm_share_hyp(phys_addr_t start, phys_addr_t end) +{ + phys_addr_t addr; + int ret; + + for (addr = ALIGN_DOWN(start, PAGE_SIZE); addr < end; addr += PAGE_SIZE) { + ret = kvm_call_hyp_nvhe(__pkvm_host_share_hyp, + __phys_to_pfn(addr)); + if (ret) + return ret; + } + + return 0; +} + /** * create_hyp_mappings - duplicate a kernel virtual address range in Hyp mode * @from: The virtual kernel start address of the range @@ -302,6 +315,13 @@ int create_hyp_mappings(void *from, void *to, enum kvm_pgtable_prot prot) if (is_kernel_in_hyp_mode()) return 0; + if (!kvm_host_owns_hyp_mappings()) { + if (WARN_ON(prot != PAGE_HYP)) + return -EPERM; + return pkvm_share_hyp(kvm_kaddr_to_phys(from), + kvm_kaddr_to_phys(to)); + } + start = start & PAGE_MASK; end = PAGE_ALIGN(end); From patchwork Mon Aug 9 15:24:48 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Quentin Perret X-Patchwork-Id: 12426837 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-17.5 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_ADSP_CUSTOM_MED,DKIM_SIGNED,DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, URIBL_BLOCKED,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 8E9FFC4338F for ; Mon, 9 Aug 2021 15:43:59 +0000 (UTC) Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 51CAC61019 for ; Mon, 9 Aug 2021 15:43:59 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.4.1 mail.kernel.org 51CAC61019 Authentication-Results: mail.kernel.org; dmarc=fail (p=reject dis=none) header.from=google.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:Cc:To:From:Subject:References: Mime-Version:Message-Id:In-Reply-To:Date:Reply-To:Content-ID: Content-Description:Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc :Resent-Message-ID:List-Owner; bh=VOYfElNmApW1utC0IeJF2nIJWBiuHka501eFRxftu80=; b=YgSMrNbbVwa3HPOi+CA1Q+jDau z9P/zEa9EBkf6Pu8WskZNnQXBvecC3vEgMermkuh1aBqupXY0EIZ0E3g8SB03MR+KnySEGiL2pcHq 1dlAVQgMQEBik736IlzzqKXFGTXtp12tttgljz6qIzmXu/mAGWM8/pzwTOxmVa5j8S6Mhau56oXL+ NbfLrFK64eNsnEHTxntoSv5wNwLBEAQW0Eq8X7qyW4QRW0+0FYMIjOoDDxgmzUe24Ro9jHqMWjvNU pvrqX2zUUDAcxPiUUgWIOhtnofujIRFdfukVsnxQeKNXRNJJVOSadIXF+fDMBYzs3nckUQD9pbeCW bOY1zJtA==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1mD7PI-001ENa-W0; Mon, 09 Aug 2021 15:41:41 +0000 Received: from mail-qv1-xf4a.google.com ([2607:f8b0:4864:20::f4a]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1mD79n-00187J-5l for linux-arm-kernel@lists.infradead.org; Mon, 09 Aug 2021 15:25:40 +0000 Received: by mail-qv1-xf4a.google.com with SMTP id u11-20020ad45aab0000b029033e289c093aso12572060qvg.10 for ; Mon, 09 Aug 2021 08:25:38 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=date:in-reply-to:message-id:mime-version:references:subject:from:to :cc; bh=sdImyBobVK9C7JHfpgjE6AhabFh6YsuoqYzTsW8O26w=; b=kqZ+48HlyWqF47Nk0k2ceqQM9Fre+x1T8hoYnPiuq5PTJ4KtKnXsvPfN8z0efxyo2K 1Cv/Y6nA1ri8uEP9fKKSe8ggvj8zxMwVMtyqCvsN0tbgapBu1uX/FRexEORPLaD7xkmC +iI0zK4K0HntIu3IDkwo8zkeuMEYV0LRMLftETQ0wMYxrrYH6anUuwIVrnBfhc6O89s9 xaIOQKixOTX5PUFhjVwPfO+yshqSpKx+8GMXNTlSVVnsyFkqHsNp1k7CmVAv6G0yrlA6 56I8EZTN3dqzfYafoy6B+aA2TdNoUdpIuu9GGm/oIh8KBQuT/PdupYj05j1672MfhIaQ EJ9g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=sdImyBobVK9C7JHfpgjE6AhabFh6YsuoqYzTsW8O26w=; b=oIuS62bEbOsSz9a6Kpn5VXY2ifmw2MRAQPeTTaKkE7qo25UsbvN2FQpN/nvKOSLKvX cITYc6GERgvsGiQvO/k7/o3B8UN6MzQVahqJIQb0ptmaARGE/uuj7b9odZTSznlvsSoi xY+Dea8QnnK4ht6RL/A1QnhTXWMRiWS/KNTMY0kIUu+1740Y+Q9W750sM5ij5nPjnClP MoBwtgbxixAbLYD0CVqmzjL0NTRulZ1AnGxf4dqcSc4A6qrh7xgR/N+v8ZOS720YESgs 8tl08PKCpTl6An5/14UBJ60j4HGOLpaGpULyWUY9yxIdhK8JZVnnv1cKAVwMJC9eEAB9 KB1g== X-Gm-Message-State: AOAM533KnqNZrXkdg4hMjdLINC1ujJ8FzdZ0Yg72AhSiGvkmMLTkNyvx LKTfeWSDJvYlqtFZpVMF3yxVnkrS2vS3 X-Google-Smtp-Source: ABdhPJzOPEvCyZwawEXBs8XVbjr2ljQD5haVjnGzWMLXjl905o6lFfqoGXZgRYKpOaaVjXYS/gQ/pMM6dBBo X-Received: from luke.lon.corp.google.com ([2a00:79e0:d:210:b0e8:d460:758b:a0ae]) (user=qperret job=sendgmr) by 2002:a05:6214:6af:: with SMTP id s15mr6751903qvz.52.1628522737726; Mon, 09 Aug 2021 08:25:37 -0700 (PDT) Date: Mon, 9 Aug 2021 16:24:48 +0100 In-Reply-To: <20210809152448.1810400-1-qperret@google.com> Message-Id: <20210809152448.1810400-22-qperret@google.com> Mime-Version: 1.0 References: <20210809152448.1810400-1-qperret@google.com> X-Mailer: git-send-email 2.32.0.605.g8dce9f2422-goog Subject: [PATCH v4 21/21] KVM: arm64: Make __pkvm_create_mappings static From: Quentin Perret To: maz@kernel.org, james.morse@arm.com, alexandru.elisei@arm.com, suzuki.poulose@arm.com, catalin.marinas@arm.com, will@kernel.org Cc: linux-arm-kernel@lists.infradead.org, kvmarm@lists.cs.columbia.edu, linux-kernel@vger.kernel.org, ardb@kernel.org, qwandor@google.com, tabba@google.com, dbrazdil@google.com, kernel-team@android.com, qperret@google.com X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20210809_082539_273659_28DABB84 X-CRM114-Status: GOOD ( 11.37 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org The __pkvm_create_mappings() function is no longer used outside of nvhe/mm.c, make it static. Signed-off-by: Quentin Perret Reviewed-by: Fuad Tabba --- arch/arm64/kvm/hyp/include/nvhe/mm.h | 2 -- arch/arm64/kvm/hyp/nvhe/mm.c | 4 ++-- 2 files changed, 2 insertions(+), 4 deletions(-) diff --git a/arch/arm64/kvm/hyp/include/nvhe/mm.h b/arch/arm64/kvm/hyp/include/nvhe/mm.h index c76d7136ed9b..c9a8f535212e 100644 --- a/arch/arm64/kvm/hyp/include/nvhe/mm.h +++ b/arch/arm64/kvm/hyp/include/nvhe/mm.h @@ -24,8 +24,6 @@ int hyp_back_vmemmap(phys_addr_t phys, unsigned long size, phys_addr_t back); int pkvm_cpu_set_vector(enum arm64_hyp_spectre_vector slot); int pkvm_create_mappings(void *from, void *to, enum kvm_pgtable_prot prot); int pkvm_create_mappings_locked(void *from, void *to, enum kvm_pgtable_prot prot); -int __pkvm_create_mappings(unsigned long start, unsigned long size, - unsigned long phys, enum kvm_pgtable_prot prot); unsigned long __pkvm_create_private_mapping(phys_addr_t phys, size_t size, enum kvm_pgtable_prot prot); diff --git a/arch/arm64/kvm/hyp/nvhe/mm.c b/arch/arm64/kvm/hyp/nvhe/mm.c index 6fbe8e8030f6..2fabeceb889a 100644 --- a/arch/arm64/kvm/hyp/nvhe/mm.c +++ b/arch/arm64/kvm/hyp/nvhe/mm.c @@ -23,8 +23,8 @@ u64 __io_map_base; struct memblock_region hyp_memory[HYP_MEMBLOCK_REGIONS]; unsigned int hyp_memblock_nr; -int __pkvm_create_mappings(unsigned long start, unsigned long size, - unsigned long phys, enum kvm_pgtable_prot prot) +static int __pkvm_create_mappings(unsigned long start, unsigned long size, + unsigned long phys, enum kvm_pgtable_prot prot) { int err;