From patchwork Thu Jul 29 13:27:58 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Quentin Perret X-Patchwork-Id: 12408711 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-17.5 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_ADSP_CUSTOM_MED,DKIM_SIGNED,DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, URIBL_BLOCKED,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id CFD5EC4338F for ; Thu, 29 Jul 2021 13:30:10 +0000 (UTC) Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id A288B60EBC for ; Thu, 29 Jul 2021 13:30:10 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.4.1 mail.kernel.org A288B60EBC Authentication-Results: mail.kernel.org; dmarc=fail (p=reject dis=none) header.from=google.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:Cc:To:From:Subject:References: Mime-Version:Message-Id:In-Reply-To:Date:Reply-To:Content-ID: Content-Description:Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc :Resent-Message-ID:List-Owner; bh=MESCDWPD/BcBaUXtohoscSS4U0ubpWQL9dv47TZqdX0=; b=q1azHVr7teQlgPe2DpPtI4G6/O Nxr63Ql/fxvdU6rWEewFzDfI0ei0Li53xwRLgma7OpdiufaMLjqKEt1eh7elUQXhUegWwhz4qud9r mC7Ga4Rtuwkv7HeK1qfXad3QpIwEkLxJeYMRJ+QVIrXtSe4NqcNEn8EEak7CIvJm2TRdZJLek35VV 2fgBtFW5UrmnfkrzUTVsO/GItKykLFg2dLvs7tIuiA2OBJJxvFpGS6EydQTeMQ021tx/DBwSbKQob 0IMo+wkkkIHt75IEIBOHlnGnuPWd0gyJAXOXBa0MtCLahfCIuRzZDGAsBPeqRO/q9CAaYmBywuc29 n5LS+yYQ==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1m965X-004Ecz-PZ; Thu, 29 Jul 2021 13:28:39 +0000 Received: from mail-qk1-x74a.google.com ([2607:f8b0:4864:20::74a]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1m965J-004EZV-V8 for linux-arm-kernel@lists.infradead.org; Thu, 29 Jul 2021 13:28:27 +0000 Received: by mail-qk1-x74a.google.com with SMTP id o2-20020a05620a1102b02903b9ade0af31so3392644qkk.1 for ; Thu, 29 Jul 2021 06:28:25 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=date:in-reply-to:message-id:mime-version:references:subject:from:to :cc; bh=crBqYydgsM0lLmpXbxrQzqnfwJboPEMWJ5uFG+CB388=; b=JOIVCVkTKSE7PwC39f/uN10vnOvi7Vzp2RTafhg3kyIB15DlDdxcNRRjKoM75ogyTS sjmXizIUunrQ4Q2yIkfgmJDrNF8nuB33MW7tB7YPCbcNhgcnStKG61TKLA9OHyXOwzKH i7uDNzA+PPHFpfTHJs2M5QoYOwbb6NUXBh++UvXlYe3cdezMzPl1EZ3sOx0jBVOxIit7 DsJ4Rk2OJv96rif4hnZqeL50UBKsRJGWc8A2EaMc0YZVDJkZTXTjcJfL1x72E+Z4uyBi PGyvbycBPgY92pQ84g5f29LjEY96+E8Pod3jV3Xa3E41vkI8XuIPX9jDrr0GlJ2pzzRS RQTw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=crBqYydgsM0lLmpXbxrQzqnfwJboPEMWJ5uFG+CB388=; b=LkUyVxfPXFopKz2FxJapYfp6ODtXl7J0GDV+LFxESwxhno7LQIdLmZeKSoCY9/4JWG g53vd7Nq9eXRjyGuj/QQ1w27asydXaGfbxhKcDE4GZh4f3GHZn/mJexXJQjYAarJcICI fAdG6P+WbBilSxkS1BRh2mgRmfzgONnlDm+8c+aqKlFRnxUZIWIJ7JA1+2xP18vcePn3 idwXva0xCi/RB9n6lCsm6BZq3fdxBAiTqSKvT/SMrOwnet+xbeutlUvfWQZ2mCdMCSY6 7FsrvtAbocBPgvJ/XNuGDyz7C6GcbZFlEYytS7uTksMPU7zRTBplhLXy945ZBOhLE+Rl +CnQ== X-Gm-Message-State: AOAM530bXiUtGwHoP6R0DmKl4bQPfmojoBCDIIlUYNvh5LgwDumv1WEn 6bfDj+V98ns5yDIiFOZM4txZC6USTeP8 X-Google-Smtp-Source: ABdhPJxHRTBRw3rhOotBjYPWwjVk04U8m2F1PsiYgMcXVSq1/9Rd7rzCjHaUtz4DsAzceZrGz5P9KOU1HQzs X-Received: from luke.lon.corp.google.com ([2a00:79e0:d:210:293a:bc89:7514:5218]) (user=qperret job=sendgmr) by 2002:a05:6214:1021:: with SMTP id k1mr5318862qvr.4.1627565304476; Thu, 29 Jul 2021 06:28:24 -0700 (PDT) Date: Thu, 29 Jul 2021 14:27:58 +0100 In-Reply-To: <20210729132818.4091769-1-qperret@google.com> Message-Id: <20210729132818.4091769-2-qperret@google.com> Mime-Version: 1.0 References: <20210729132818.4091769-1-qperret@google.com> X-Mailer: git-send-email 2.32.0.432.gabb21c7263-goog Subject: [PATCH v3 01/21] KVM: arm64: Add hyp_spin_is_locked() for basic locking assertions at EL2 From: Quentin Perret To: maz@kernel.org, james.morse@arm.com, alexandru.elisei@arm.com, suzuki.poulose@arm.com, catalin.marinas@arm.com, will@kernel.org Cc: linux-arm-kernel@lists.infradead.org, kvmarm@lists.cs.columbia.edu, linux-kernel@vger.kernel.org, ardb@kernel.org, qwandor@google.com, tabba@google.com, dbrazdil@google.com, kernel-team@android.com, Quentin Perret X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20210729_062826_047608_8897BCC1 X-CRM114-Status: GOOD ( 11.06 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org From: Will Deacon Introduce hyp_spin_is_locked() so that functions can easily assert that a given lock is held (albeit possibly by another CPU!) without having to drag full lockdep support up to EL2. Signed-off-by: Will Deacon Signed-off-by: Quentin Perret --- arch/arm64/kvm/hyp/include/nvhe/spinlock.h | 8 ++++++++ 1 file changed, 8 insertions(+) diff --git a/arch/arm64/kvm/hyp/include/nvhe/spinlock.h b/arch/arm64/kvm/hyp/include/nvhe/spinlock.h index 76b537f8d1c6..04f65b655fcf 100644 --- a/arch/arm64/kvm/hyp/include/nvhe/spinlock.h +++ b/arch/arm64/kvm/hyp/include/nvhe/spinlock.h @@ -15,6 +15,7 @@ #include #include +#include typedef union hyp_spinlock { u32 __val; @@ -89,4 +90,11 @@ static inline void hyp_spin_unlock(hyp_spinlock_t *lock) : "memory"); } +static inline bool hyp_spin_is_locked(hyp_spinlock_t *lock) +{ + hyp_spinlock_t lockval = READ_ONCE(*lock); + + return lockval.owner != lockval.next; +} + #endif /* __ARM64_KVM_NVHE_SPINLOCK_H__ */ From patchwork Thu Jul 29 13:27:59 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Quentin Perret X-Patchwork-Id: 12408713 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-17.5 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_ADSP_CUSTOM_MED,DKIM_SIGNED,DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, URIBL_BLOCKED,USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 602B4C4338F for ; Thu, 29 Jul 2021 13:30:26 +0000 (UTC) Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 363E760184 for ; Thu, 29 Jul 2021 13:30:26 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.4.1 mail.kernel.org 363E760184 Authentication-Results: mail.kernel.org; dmarc=fail (p=reject dis=none) header.from=google.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:Cc:To:From:Subject:References: Mime-Version:Message-Id:In-Reply-To:Date:Reply-To:Content-ID: Content-Description:Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc :Resent-Message-ID:List-Owner; bh=u6Sfk/gx2W9ITh6urT1JzmnIUdKBa1v0Ceca+3lQtcg=; b=yY039IOwgEw+zDeYxeqL+XcZCC xkBfdDkxIWJUIVc3DAU8BUs7pG7s25pEHDswml6jFJQGOW3bMp6u0RCwuwIHb+fjcCaKd8Z8ap65O EtKqiWawVJM1THNha9uoEhvXSjjyG9ZpeQNlz2a+XCwNfyCsPmBNFoaPeuRytwK87M/b/tSVU5zgj Ko4stRgeE5mtAzzOvHFd/AAJAiKVNSDKo6nWm0//R1EVXwWSp2Fr2qrC9jMFygjNXgM15/f3mtWRu 087GH7ESjNef4lH9/VW+vX4tkYIAFf6U+JVBoM5S/6mXgI4AKEHAXaAFqC6U4NHbDv1/mfDxYpcpc dWYZsCLw==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1m965k-004Egs-LX; Thu, 29 Jul 2021 13:28:53 +0000 Received: from mail-wm1-x34a.google.com ([2a00:1450:4864:20::34a]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1m965N-004Ea6-JX for linux-arm-kernel@lists.infradead.org; Thu, 29 Jul 2021 13:28:31 +0000 Received: by mail-wm1-x34a.google.com with SMTP id r125-20020a1c2b830000b0290197a4be97b7so1992773wmr.9 for ; Thu, 29 Jul 2021 06:28:28 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=date:in-reply-to:message-id:mime-version:references:subject:from:to :cc; bh=t7+5ofN1mhSPCYy31zt7RU0I2nBRXXWPYqlSGOk7ubE=; b=EOWUMMsZPsB2L4bsN4GxoGi3y0igtMQJVuIWpJTOwkroahO2vrvgGKDlL7RtzJMlsM 0oE50dz19H3398h3CH7lGvES9i7AZ+VyzBn3LLKoJN/6N09gfxN4nZrgog/WIOpdxF48 B/jbMHiG9QnFkf2f639cMBjVse7pVDqHc+1Ylei1mCqOmtyDH269kxdDb+J0Kuh6hwTv 9+n8fZ5n5Q8wA6fg82FoPSbY9mbZRd1dd4E3hdMvxHuv/9ztJK9IqX7gYuN2lWpjx5+s +HFOzx5RoBxyC3x0/qNim7tnNdldKJhuLJ7GxVdP5C7TH7ZDgtXRup/AoaGjp/YnLSie c8VA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=t7+5ofN1mhSPCYy31zt7RU0I2nBRXXWPYqlSGOk7ubE=; b=drNpJ6gFelHxemzzjSJB9qqwLNgFsX6tYT35jiLdyuB0wLvZ7lIatQaLauD0TQQtp/ Q+jpycXr2bsnRrUm9QqdQLyhVj1puKkznoiT9pEzBXjhKWCW2TVsvNT3ppNYGhKX43Y2 RaKbFt0ruI+vXZZHehAi7Qy+uVKvrz8+tjBdDUV/UtXO+imROivxzbSl65QslnBrn0eE Ru0Lxv6JLwL5pytz1gllgXDAp6zT3jaQo9jZuwTPOyRkL2N+B5iayujAaKD2QOmqW1nX 4DtszlafqG9dEiFi+v55FJFOck5b2AA0F14CUF4uuxoDE+vNUqs1h/uB58FMdMFB+rjG PUHg== X-Gm-Message-State: AOAM533TaNnow/Dap2KM0jA7VNIh41qGO7AKXZqX8Cf/+CB/prWXXJ9d t97clVhzjGva7BfwW9qhAB3XdAP9QGrF X-Google-Smtp-Source: ABdhPJyA9LhjEf8A/taAM06dydN+uO7RN2g31fVg3fvbmNEB2A5ceVlvMQQeLCoX2VjPrd82TvrUI4VZN1Ys X-Received: from luke.lon.corp.google.com ([2a00:79e0:d:210:293a:bc89:7514:5218]) (user=qperret job=sendgmr) by 2002:a1c:4b0a:: with SMTP id y10mr4499wma.1.1627565306816; Thu, 29 Jul 2021 06:28:26 -0700 (PDT) Date: Thu, 29 Jul 2021 14:27:59 +0100 In-Reply-To: <20210729132818.4091769-1-qperret@google.com> Message-Id: <20210729132818.4091769-3-qperret@google.com> Mime-Version: 1.0 References: <20210729132818.4091769-1-qperret@google.com> X-Mailer: git-send-email 2.32.0.432.gabb21c7263-goog Subject: [PATCH v3 02/21] KVM: arm64: Introduce hyp_assert_lock_held() From: Quentin Perret To: maz@kernel.org, james.morse@arm.com, alexandru.elisei@arm.com, suzuki.poulose@arm.com, catalin.marinas@arm.com, will@kernel.org Cc: linux-arm-kernel@lists.infradead.org, kvmarm@lists.cs.columbia.edu, linux-kernel@vger.kernel.org, ardb@kernel.org, qwandor@google.com, tabba@google.com, dbrazdil@google.com, kernel-team@android.com, Quentin Perret X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20210729_062829_696636_616C91B2 X-CRM114-Status: GOOD ( 13.21 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Introduce a poor man's lockdep implementation at EL2 which allows to BUG() whenever a hyp spinlock is not held when it should. Hide this feature behind a new Kconfig option that targets the EL2 object specifically, instead of piggy backing on the existing CONFIG_LOCKDEP. EL2 cannot WARN() cleanly to report locking issues, hence BUG() is the only option and it is not clear whether we want this widely enabled. This is most likely going to be useful for local testing until the EL2 WARN() situation has improved. Signed-off-by: Quentin Perret --- arch/arm64/kvm/Kconfig | 9 +++++++++ arch/arm64/kvm/hyp/include/nvhe/spinlock.h | 17 +++++++++++++++++ 2 files changed, 26 insertions(+) diff --git a/arch/arm64/kvm/Kconfig b/arch/arm64/kvm/Kconfig index a4eba0908bfa..9b9721895e5c 100644 --- a/arch/arm64/kvm/Kconfig +++ b/arch/arm64/kvm/Kconfig @@ -46,6 +46,15 @@ if KVM source "virt/kvm/Kconfig" +config NVHE_EL2_DEBUG + bool "Debug mode for non-VHE EL2 object" + help + Say Y here to enable the debug mode for the non-VHE KVM EL2 object. + Failure reports will BUG() in the hypervisor. This is intended for + local EL2 hypervisor development. + + If unsure, say N. + endif # KVM endif # VIRTUALIZATION diff --git a/arch/arm64/kvm/hyp/include/nvhe/spinlock.h b/arch/arm64/kvm/hyp/include/nvhe/spinlock.h index 04f65b655fcf..4652fd04bdbe 100644 --- a/arch/arm64/kvm/hyp/include/nvhe/spinlock.h +++ b/arch/arm64/kvm/hyp/include/nvhe/spinlock.h @@ -97,4 +97,21 @@ static inline bool hyp_spin_is_locked(hyp_spinlock_t *lock) return lockval.owner != lockval.next; } +#ifdef CONFIG_NVHE_EL2_DEBUG +static inline void hyp_assert_lock_held(hyp_spinlock_t *lock) +{ + /* + * The __pkvm_init() path accesses protected data-structures without + * holding locks as the other CPUs are guaranteed to not enter EL2 + * concurrently at this point in time. The point by which EL2 is + * initialized on all CPUs is reflected in the pkvm static key, so + * wait until it is set before checking the lock state. + */ + if (static_branch_likely(&kvm_protected_mode_initialized)) + BUG_ON(!hyp_spin_is_locked(lock)); +} +#else +static inline void hyp_assert_lock_held(hyp_spinlock_t *lock) { } +#endif + #endif /* __ARM64_KVM_NVHE_SPINLOCK_H__ */ From patchwork Thu Jul 29 13:28:00 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Quentin Perret X-Patchwork-Id: 12408715 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-17.5 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_ADSP_CUSTOM_MED,DKIM_SIGNED,DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, URIBL_BLOCKED,USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 8BC6FC4338F for ; Thu, 29 Jul 2021 13:30:49 +0000 (UTC) Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 615B460EBC for ; Thu, 29 Jul 2021 13:30:49 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.4.1 mail.kernel.org 615B460EBC Authentication-Results: mail.kernel.org; dmarc=fail (p=reject dis=none) header.from=google.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:Cc:To:From:Subject:References: Mime-Version:Message-Id:In-Reply-To:Date:Reply-To:Content-ID: Content-Description:Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc :Resent-Message-ID:List-Owner; bh=UuWmqKwukZSOcrSBaSVeXe7029kE9zbILPVPa+tQYsY=; b=zL9bx8G986iRzawfQoYc5w6Hqv pfkLM1hXqcaXf3325HFiyyC2mLj/zkvYzXEZaMxkks37yEahMJqFNTfKR0+KkrsOJ7MmNFU5BAmDs xbU6ZqUqPyaiJUdxeOg7/8zvgnMi3RAsEsJdja+hCxN4zeUDcUH5m56R27JC/nPIE+Vlm4OXuXTpM v6LgZKyBhGP0L8k0NZrRkZnKE7KYm4pkRQ4YfKDdvsZe2YM1sU0X+VoOBrQscJbIwVHA5dLJg7k5c 3kYxs8CqBTHtEtHtuiNlw2WNJhO6sMjje37jA8VujlqrXChB+pCnwGre7YRMvxS0ck3eWuksGWD9/ iGI5yrlA==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1m965y-004Em8-Ct; Thu, 29 Jul 2021 13:29:06 +0000 Received: from mail-qv1-xf4a.google.com ([2607:f8b0:4864:20::f4a]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1m965P-004Eah-Lz for linux-arm-kernel@lists.infradead.org; Thu, 29 Jul 2021 13:28:33 +0000 Received: by mail-qv1-xf4a.google.com with SMTP id r14-20020a0c8d0e0000b02902e82df307f0so3932864qvb.4 for ; Thu, 29 Jul 2021 06:28:30 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=date:in-reply-to:message-id:mime-version:references:subject:from:to :cc; bh=ATiyLC5m+9bmdBYcDBkPgetNyCcHjVe1BNbOPK3Xa6Q=; b=OqRh2DivWRwS1PBtKegpyfDBYkMloX5jagqHe1vyeZJWSLQKdZRgp2d+xwx0GJIkQW QZ2QSanSfxgy9Wce9m9rmjL3UA1sTmhe2tgCos0VPpCxjlLtM0IGduNpJprR59Tu2CT+ cIR1sequ0SyiSX6FhZl8dyIBIA11l5qUpOPwKsxeuIYoU2ZpHynVwaJOe+G3r8ykGWFo gdbjk7UlL1nYQ6wR9CGXdTP8kBswWWKumbD5kUNfvKtWm1So6t8OXteOZkdFy5HrMy0A 3WECcSIO9ofZeH5sX4UUWEpcnFCrYOUdZwNbYRPL2dxux0HVY+y7WPY5DaUP47qFAmP6 yjQg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=ATiyLC5m+9bmdBYcDBkPgetNyCcHjVe1BNbOPK3Xa6Q=; b=ggsjnoVxv7ej1ywV6hx0JHsXHfswSGQc9/QYSCa9N8/9a4iBH7lBGkoykgDDC+B0kW Emm+cMW4hYD10SjRpVqsJsKi2WaGZArRkKFKR1mEBp4LEnlCBd6a4HfHfwR27nLodGnb QusYdjxX9fkFTjatvQyHGtGy4KSpbaNvBX1b4k9Y+ZeJgP1EAQ9K7Y1CQJJdepyMJKbY 5Z/QaqP9tdLPVD2plbA64X4U9URmoXBnjP8hLFbuE2tdwtNb+V8iXl8T3XocmmWK4eYu Zd0P8Ln6ykfZAXpGC0Nbc3PpTK3FZD1ATW0cTgslEh8pxl4hRSY4F76BLIxTye2YqV27 bJDw== X-Gm-Message-State: AOAM533YkxMaf5pHUSGh1PTfHLqNgzXGQzdh+tNbhnAHaAsug0guYxyF 3EqZvyf+z1K1vvg2G1hhpAgTB9WUY9dk X-Google-Smtp-Source: ABdhPJyvj7S5X1fDRFz1PU5wtUnJrRchL+QMKuVeP46VjzReAMKnCpUDVjPlF/qwJy6z74GCRKY+6nq3HeHT X-Received: from luke.lon.corp.google.com ([2a00:79e0:d:210:293a:bc89:7514:5218]) (user=qperret job=sendgmr) by 2002:a05:6214:501d:: with SMTP id jo29mr5191775qvb.43.1627565309560; Thu, 29 Jul 2021 06:28:29 -0700 (PDT) Date: Thu, 29 Jul 2021 14:28:00 +0100 In-Reply-To: <20210729132818.4091769-1-qperret@google.com> Message-Id: <20210729132818.4091769-4-qperret@google.com> Mime-Version: 1.0 References: <20210729132818.4091769-1-qperret@google.com> X-Mailer: git-send-email 2.32.0.432.gabb21c7263-goog Subject: [PATCH v3 03/21] KVM: arm64: Provide the host_stage2_try() helper macro From: Quentin Perret To: maz@kernel.org, james.morse@arm.com, alexandru.elisei@arm.com, suzuki.poulose@arm.com, catalin.marinas@arm.com, will@kernel.org Cc: linux-arm-kernel@lists.infradead.org, kvmarm@lists.cs.columbia.edu, linux-kernel@vger.kernel.org, ardb@kernel.org, qwandor@google.com, tabba@google.com, dbrazdil@google.com, kernel-team@android.com, Quentin Perret X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20210729_062831_765094_D5C865D1 X-CRM114-Status: GOOD ( 16.06 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org We currently unmap all MMIO mappings from the host stage-2 to recycle the pages whenever we run out. In order to make this pattern easy to re-use from other places, factor the logic out into a dedicated macro. While at it, apply the macro for the kvm_pgtable_stage2_set_owner() calls. They're currently only called early on and are guaranteed to succeed, but making them robust to the -ENOMEM case doesn't hurt and will avoid painful debugging sessions later on. Signed-off-by: Quentin Perret Reviewed-by: Fuad Tabba --- arch/arm64/kvm/hyp/nvhe/mem_protect.c | 40 +++++++++++++++------------ 1 file changed, 22 insertions(+), 18 deletions(-) diff --git a/arch/arm64/kvm/hyp/nvhe/mem_protect.c b/arch/arm64/kvm/hyp/nvhe/mem_protect.c index d938ce95d3bd..74280a753efb 100644 --- a/arch/arm64/kvm/hyp/nvhe/mem_protect.c +++ b/arch/arm64/kvm/hyp/nvhe/mem_protect.c @@ -208,6 +208,25 @@ static inline int __host_stage2_idmap(u64 start, u64 end, prot, &host_s2_pool); } +/* + * The pool has been provided with enough pages to cover all of memory with + * page granularity, but it is difficult to know how much of the MMIO range + * we will need to cover upfront, so we may need to 'recycle' the pages if we + * run out. + */ +#define host_stage2_try(fn, ...) \ + ({ \ + int __ret; \ + hyp_assert_lock_held(&host_kvm.lock); \ + __ret = fn(__VA_ARGS__); \ + if (__ret == -ENOMEM) { \ + __ret = host_stage2_unmap_dev_all(); \ + if (!__ret) \ + __ret = fn(__VA_ARGS__); \ + } \ + __ret; \ + }) + static int host_stage2_idmap(u64 addr) { enum kvm_pgtable_prot prot = KVM_PGTABLE_PROT_R | KVM_PGTABLE_PROT_W; @@ -223,22 +242,7 @@ static int host_stage2_idmap(u64 addr) if (ret) goto unlock; - ret = __host_stage2_idmap(range.start, range.end, prot); - if (ret != -ENOMEM) - goto unlock; - - /* - * The pool has been provided with enough pages to cover all of memory - * with page granularity, but it is difficult to know how much of the - * MMIO range we will need to cover upfront, so we may need to 'recycle' - * the pages if we run out. - */ - ret = host_stage2_unmap_dev_all(); - if (ret) - goto unlock; - - ret = __host_stage2_idmap(range.start, range.end, prot); - + ret = host_stage2_try(__host_stage2_idmap, range.start, range.end, prot); unlock: hyp_spin_unlock(&host_kvm.lock); @@ -257,8 +261,8 @@ int __pkvm_mark_hyp(phys_addr_t start, phys_addr_t end) return -EINVAL; hyp_spin_lock(&host_kvm.lock); - ret = kvm_pgtable_stage2_set_owner(&host_kvm.pgt, start, end - start, - &host_s2_pool, pkvm_hyp_id); + ret = host_stage2_try(kvm_pgtable_stage2_set_owner, &host_kvm.pgt, + start, end - start, &host_s2_pool, pkvm_hyp_id); hyp_spin_unlock(&host_kvm.lock); return ret != -EAGAIN ? ret : 0; From patchwork Thu Jul 29 13:28:01 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Quentin Perret X-Patchwork-Id: 12408717 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-17.5 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_ADSP_CUSTOM_MED,DKIM_SIGNED,DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, URIBL_BLOCKED,USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 509D0C4338F for ; Thu, 29 Jul 2021 13:31:07 +0000 (UTC) Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 10729601FC for ; Thu, 29 Jul 2021 13:31:07 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.4.1 mail.kernel.org 10729601FC Authentication-Results: mail.kernel.org; dmarc=fail (p=reject dis=none) header.from=google.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:Cc:To:From:Subject:References: Mime-Version:Message-Id:In-Reply-To:Date:Reply-To:Content-ID: Content-Description:Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc :Resent-Message-ID:List-Owner; bh=J9yLLqnAZ3DXDtXSOobwdtuXGHkQCNNcKeF4soEIxEk=; b=KlsHUByL4CBUCJ6a3rHJaUWrdM +9ZPWLVm2r4cVvz/NQuKBfEqNOJj9KKCj7PGGj5va0YRpHZehe9fXOCpFEiqyIPecOYQq+BVLzG2c uDH3LO1YSiUGW7/q7wLkx++sOMUn2JrsV4JHW19NZNq7EePLVF77uMCG1MrqIlODgFBjPSyD9C2Ld 33Hq3K8hgd2q0F3TNZJFUUCA1k941VWAdkzUTvBX5jh9g0CUR9COyARPTNWwmFF7T2v0H2AeAP6pg g6z89kxzvxMDmWgJ/gNQx8cMnKYSFEaNRSimbOLDBWYjLf5WN7yhwAouZMBsADeZK6jVeJK8QbX+C uTI5obtA==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1m966I-004Et5-4m; Thu, 29 Jul 2021 13:29:26 +0000 Received: from mail-wr1-x44a.google.com ([2a00:1450:4864:20::44a]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1m965S-004Eb9-B3 for linux-arm-kernel@lists.infradead.org; Thu, 29 Jul 2021 13:28:35 +0000 Received: by mail-wr1-x44a.google.com with SMTP id s16-20020adfdb100000b0290140a25efc6dso2251051wri.5 for ; Thu, 29 Jul 2021 06:28:33 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=date:in-reply-to:message-id:mime-version:references:subject:from:to :cc; bh=Gjf1jLwbr60oU/lDUo56+HNoAVcvWGbxZryQe2MxgVI=; b=liAJ/B9k752zaUtEr2dNmyI6PNIq95D2uxyTofIPP2Y9x50NOXw4/JT8oVxsLvIkmc P7ulUWGLvyLx+t+fWD8rdAsH4RvIVZHUhm9AOhec5RvozMBkX+zZj7nLaIAXqPqIyTGm qMSgBtOd1z4GkgAg8Aq2iTh9+h3RRsOREBBjibJ8VnR4P30ZOtFv7y65TudPffgn0bRg RznpmwquKro9JhqASPV8OQxONr9moaG80DsU8VJsjXcHn5Yyqn3qHvtX9DT4MJ6bUy36 iuW7lgbeDizByaPiTm3DcjyGsmrR4gr7L+ezKHNRzsImGGj1FE1SjSaaY2EcjvFz2eYA lr2g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=Gjf1jLwbr60oU/lDUo56+HNoAVcvWGbxZryQe2MxgVI=; b=PapmxnVbAg5RR40l7oTNAu/MxhlaRnAN5yPY1U2nFmIRUi2CR/YOXBn5L7AsFfpkPr wrDW9k+3S1H+4AMp1D/IqELOwmLoIK6Fm0XrYYAYxNz3uEUGu95iYit8+iROJ3qZCsM6 66h1Ijc1cKbtKs4I8gDZC9UlRDe1reGh7n5J0BGLWZ6deP/O9qGK/9dHrxYH83ml2tPo 3auODPLqyTltJRQRMgRumYrnj/t3FBpzCppZMvxRL5Swyk3UA/9NmE3l8Tl8cS+6dg4+ pRCk8dJLEfs1qFdyXYfT0HvicZb4VCgx0NJ7B//zvK8/l+8GGnyKGoUxhPREucA88FVc nMnA== X-Gm-Message-State: AOAM532fjEjE8p2NDunqreQh76rlMs/74p5jPapvRNKhzf3fxca5F5iB ZHSX72Fn1tZzOCa8SsqlMmGOXByUl9Si X-Google-Smtp-Source: ABdhPJyC7oJeo9hm9rZKLPbAlJuErb6lfSMGOtMtJPYFP3+ud7lcAhFVhw7sxxXnSC8c2pWtkaZZ9KzshhQh X-Received: from luke.lon.corp.google.com ([2a00:79e0:d:210:293a:bc89:7514:5218]) (user=qperret job=sendgmr) by 2002:a7b:c147:: with SMTP id z7mr4842725wmi.10.1627565311894; Thu, 29 Jul 2021 06:28:31 -0700 (PDT) Date: Thu, 29 Jul 2021 14:28:01 +0100 In-Reply-To: <20210729132818.4091769-1-qperret@google.com> Message-Id: <20210729132818.4091769-5-qperret@google.com> Mime-Version: 1.0 References: <20210729132818.4091769-1-qperret@google.com> X-Mailer: git-send-email 2.32.0.432.gabb21c7263-goog Subject: [PATCH v3 04/21] KVM: arm64: Introduce helper to retrieve a PTE and its level From: Quentin Perret To: maz@kernel.org, james.morse@arm.com, alexandru.elisei@arm.com, suzuki.poulose@arm.com, catalin.marinas@arm.com, will@kernel.org Cc: linux-arm-kernel@lists.infradead.org, kvmarm@lists.cs.columbia.edu, linux-kernel@vger.kernel.org, ardb@kernel.org, qwandor@google.com, tabba@google.com, dbrazdil@google.com, kernel-team@android.com, Quentin Perret X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20210729_062834_440297_A37A64C8 X-CRM114-Status: GOOD ( 13.20 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org From: Marc Zyngier It is becoming a common need to fetch the PTE for a given address together with its level. Add such a helper. Signed-off-by: Marc Zyngier Signed-off-by: Quentin Perret --- arch/arm64/include/asm/kvm_pgtable.h | 19 ++++++++++++++ arch/arm64/kvm/hyp/pgtable.c | 39 ++++++++++++++++++++++++++++ 2 files changed, 58 insertions(+) diff --git a/arch/arm64/include/asm/kvm_pgtable.h b/arch/arm64/include/asm/kvm_pgtable.h index f004c0115d89..082b9d65f40b 100644 --- a/arch/arm64/include/asm/kvm_pgtable.h +++ b/arch/arm64/include/asm/kvm_pgtable.h @@ -432,6 +432,25 @@ int kvm_pgtable_stage2_flush(struct kvm_pgtable *pgt, u64 addr, u64 size); int kvm_pgtable_walk(struct kvm_pgtable *pgt, u64 addr, u64 size, struct kvm_pgtable_walker *walker); +/** + * kvm_pgtable_get_leaf() - Walk a page-table and retrieve the leaf entry + * with its level. + * @pgt: Page-table structure initialised by kvm_pgtable_*_init(). + * @addr: Input address for the start of the walk. + * @ptep: Pointer to storage for the retrieved PTE. + * @level: Pointer to storage for the level of the retrieved PTE. + * + * The offset of @addr within a page is ignored. + * + * The walker will walk the page-table entries corresponding to the input + * address specified, retrieving the leaf corresponding to this address. + * Invalid entries are treated as leaf entries. + * + * Return: 0 on success, negative error code on failure. + */ +int kvm_pgtable_get_leaf(struct kvm_pgtable *pgt, u64 addr, + kvm_pte_t *ptep, u32 *level); + /** * kvm_pgtable_stage2_find_range() - Find a range of Intermediate Physical * Addresses with compatible permission diff --git a/arch/arm64/kvm/hyp/pgtable.c b/arch/arm64/kvm/hyp/pgtable.c index 05321f4165e3..78f36bd5df6c 100644 --- a/arch/arm64/kvm/hyp/pgtable.c +++ b/arch/arm64/kvm/hyp/pgtable.c @@ -326,6 +326,45 @@ int kvm_pgtable_walk(struct kvm_pgtable *pgt, u64 addr, u64 size, return _kvm_pgtable_walk(&walk_data); } +struct leaf_walk_data { + kvm_pte_t pte; + u32 level; +}; + +static int leaf_walker(u64 addr, u64 end, u32 level, kvm_pte_t *ptep, + enum kvm_pgtable_walk_flags flag, void * const arg) +{ + struct leaf_walk_data *data = arg; + + data->pte = *ptep; + data->level = level; + + return 0; +} + +int kvm_pgtable_get_leaf(struct kvm_pgtable *pgt, u64 addr, + kvm_pte_t *ptep, u32 *level) +{ + struct leaf_walk_data data; + struct kvm_pgtable_walker walker = { + .cb = leaf_walker, + .flags = KVM_PGTABLE_WALK_LEAF, + .arg = &data, + }; + int ret; + + ret = kvm_pgtable_walk(pgt, ALIGN_DOWN(addr, PAGE_SIZE), + PAGE_SIZE, &walker); + if (!ret) { + if (ptep) + *ptep = data.pte; + if (level) + *level = data.level; + } + + return ret; +} + struct hyp_map_data { u64 phys; kvm_pte_t attr; From patchwork Thu Jul 29 13:28:02 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Quentin Perret X-Patchwork-Id: 12408721 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-17.5 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_ADSP_CUSTOM_MED,DKIM_SIGNED,DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, URIBL_BLOCKED,USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 5A877C4338F for ; Thu, 29 Jul 2021 13:31:45 +0000 (UTC) Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 22E6860EBC for ; Thu, 29 Jul 2021 13:31:45 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.4.1 mail.kernel.org 22E6860EBC Authentication-Results: mail.kernel.org; dmarc=fail (p=reject dis=none) header.from=google.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:Cc:To:From:Subject:References: Mime-Version:Message-Id:In-Reply-To:Date:Reply-To:Content-ID: Content-Description:Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc :Resent-Message-ID:List-Owner; bh=aosc2pmI+pKjDY9afeY2J1iwHoHhK2VnPyGWTdgHi54=; b=nrtpwI5zJo8cjktPPmakjaJ6kp QH3Eow7TNan3pHpeccCHSquWUC1PNX2OpgpUojs5yEFv+29pjRMnDMHNvohYEfqerJFiUZWrmmzyk pZjiEhciHJ1zET0qktQ30bM/TVcVkwm+WkNGCc+PK7+w4dwbhF7FqmzCxZpYveZ3wbOvQ6pg7jPDV GQHdZCOXzCh4X8qbX7HaMu4PiU5KxErZWPxGGuM43NtJBxj9O4hHABqsbpokJl6bsR9vOFmwz31jb XdClN05gwNXdY4gop4LNjKwPV0ybKIUhMxRr3cJ/IdjHiExsCeIKaWYWqGpLTC8sAxLatNgZ9vAT/ BOwBH9hg==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1m966k-004F5Q-0Q; Thu, 29 Jul 2021 13:29:54 +0000 Received: from mail-qv1-xf4a.google.com ([2607:f8b0:4864:20::f4a]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1m965T-004Ec5-QF for linux-arm-kernel@lists.infradead.org; Thu, 29 Jul 2021 13:28:37 +0000 Received: by mail-qv1-xf4a.google.com with SMTP id o32-20020a0c85a30000b0290328f91ede2bso3944732qva.1 for ; Thu, 29 Jul 2021 06:28:35 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=date:in-reply-to:message-id:mime-version:references:subject:from:to :cc; bh=Cv7PfXmwJJr5mkYfUmsrSSKe76wdqOHk3bYX3NKK1Ak=; b=AbwsfISePdR+JcDYjzhjFaE1bsJCrdANuvRjTecRQQceHXv3IeTh0J7jJlARwdsQ8G lK2RNR40PdH2ybX+4B8/SG5a625FiCm/DG6T+1x0H/dzjUtP5pUIoKsaQjGt8zEfItZi PA2DJfQTPDIWY7IetC/uKiLRnyWOgkqH2qZXfcaBRUV/7fy/9vJazfZA57wTRE+X99Hs NLie91Ls49QGVPdQSkA8SAEtC+Bglx/wmSdONWh4h0p+VikY71TxLpKomLxhPbNKrqWS xWl4/oTTHN7Ikt7nqOsL9aqOKYUcBZl+aN86Yz90LGWUGeKx1Y9CVB4ZlhG1Ky5d+eEO 92kQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=Cv7PfXmwJJr5mkYfUmsrSSKe76wdqOHk3bYX3NKK1Ak=; b=ph8pMXxg9ywtBgARVbOILdBnBiiRHCm6UTqpHuYn6xSchsddH0lUGiXd+JV/8TCtMv Lkd9itSOZi/d5mJQkL7cZRjC4JtqeMivHU9ms7WbHIht5dH8O5/HozoJkRY3B4rDjiZa crEsiIwqeoAAXIZU/jRThs2Ts1W082g1AbkF67RLbLQf7LTgUErGsVB3Ve/058pGY5NH qgY5QqEwXndhhmVOh6VV5Gk13LW2Ex5WsuynWsbpb3L74dUDw503VPzQWG6ACFyvvEeg CTvfEm28rIp6ljHD1TeQQ6ICdv8xLAjZnxGIHnRv3APPL81+rxed22tolqGFzUrILDS9 vSHg== X-Gm-Message-State: AOAM5313YZSjlFZx9mqZ82k8fzZ/wezC25cJDi06JcJW8jvT/E+erxom V4khWvLKSm3COcgZZ7RzoZLJsRWe25fI X-Google-Smtp-Source: ABdhPJyLqRxp0HVqSAdMS3S3QIEpwdG08MnQZBGS56P78o0eMqoGVMy2zgiz2Vzx15NSrvOb6w+TzZN0qWva X-Received: from luke.lon.corp.google.com ([2a00:79e0:d:210:293a:bc89:7514:5218]) (user=qperret job=sendgmr) by 2002:a05:6214:84:: with SMTP id n4mr5338927qvr.4.1627565314337; Thu, 29 Jul 2021 06:28:34 -0700 (PDT) Date: Thu, 29 Jul 2021 14:28:02 +0100 In-Reply-To: <20210729132818.4091769-1-qperret@google.com> Message-Id: <20210729132818.4091769-6-qperret@google.com> Mime-Version: 1.0 References: <20210729132818.4091769-1-qperret@google.com> X-Mailer: git-send-email 2.32.0.432.gabb21c7263-goog Subject: [PATCH v3 05/21] KVM: arm64: Expose page-table helpers From: Quentin Perret To: maz@kernel.org, james.morse@arm.com, alexandru.elisei@arm.com, suzuki.poulose@arm.com, catalin.marinas@arm.com, will@kernel.org Cc: linux-arm-kernel@lists.infradead.org, kvmarm@lists.cs.columbia.edu, linux-kernel@vger.kernel.org, ardb@kernel.org, qwandor@google.com, tabba@google.com, dbrazdil@google.com, kernel-team@android.com, Quentin Perret X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20210729_062835_986302_D1CFE42D X-CRM114-Status: GOOD ( 14.70 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org The KVM pgtable API exposes the kvm_pgtable_walk() function to allow the definition of walkers outside of pgtable.c. However, it is not easy to implement any of those walkers without some of the low-level helpers. Move some of them to the header file to allow re-use from other places. Signed-off-by: Quentin Perret --- arch/arm64/include/asm/kvm_pgtable.h | 40 ++++++++++++++++++++++++++++ arch/arm64/kvm/hyp/pgtable.c | 39 --------------------------- 2 files changed, 40 insertions(+), 39 deletions(-) diff --git a/arch/arm64/include/asm/kvm_pgtable.h b/arch/arm64/include/asm/kvm_pgtable.h index 082b9d65f40b..6938eac72c1f 100644 --- a/arch/arm64/include/asm/kvm_pgtable.h +++ b/arch/arm64/include/asm/kvm_pgtable.h @@ -25,6 +25,46 @@ static inline u64 kvm_get_parange(u64 mmfr0) typedef u64 kvm_pte_t; +#define KVM_PTE_VALID BIT(0) + +#define KVM_PTE_ADDR_MASK GENMASK(47, PAGE_SHIFT) +#define KVM_PTE_ADDR_51_48 GENMASK(15, 12) + +static inline bool kvm_pte_valid(kvm_pte_t pte) +{ + return pte & KVM_PTE_VALID; +} + +static inline u64 kvm_pte_to_phys(kvm_pte_t pte) +{ + u64 pa = pte & KVM_PTE_ADDR_MASK; + + if (PAGE_SHIFT == 16) + pa |= FIELD_GET(KVM_PTE_ADDR_51_48, pte) << 48; + + return pa; +} + +static inline u64 kvm_granule_shift(u32 level) +{ + /* Assumes KVM_PGTABLE_MAX_LEVELS is 4 */ + return ARM64_HW_PGTABLE_LEVEL_SHIFT(level); +} + +static inline u64 kvm_granule_size(u32 level) +{ + return BIT(kvm_granule_shift(level)); +} + +static inline bool kvm_level_supports_block_mapping(u32 level) +{ + /* + * Reject invalid block mappings and don't bother with 4TB mappings for + * 52-bit PAs. + */ + return !(level == 0 || (PAGE_SIZE != SZ_4K && level == 1)); +} + /** * struct kvm_pgtable_mm_ops - Memory management callbacks. * @zalloc_page: Allocate a single zeroed memory page. diff --git a/arch/arm64/kvm/hyp/pgtable.c b/arch/arm64/kvm/hyp/pgtable.c index 78f36bd5df6c..49d768b92997 100644 --- a/arch/arm64/kvm/hyp/pgtable.c +++ b/arch/arm64/kvm/hyp/pgtable.c @@ -11,16 +11,12 @@ #include #include -#define KVM_PTE_VALID BIT(0) #define KVM_PTE_TYPE BIT(1) #define KVM_PTE_TYPE_BLOCK 0 #define KVM_PTE_TYPE_PAGE 1 #define KVM_PTE_TYPE_TABLE 1 -#define KVM_PTE_ADDR_MASK GENMASK(47, PAGE_SHIFT) -#define KVM_PTE_ADDR_51_48 GENMASK(15, 12) - #define KVM_PTE_LEAF_ATTR_LO GENMASK(11, 2) #define KVM_PTE_LEAF_ATTR_LO_S1_ATTRIDX GENMASK(4, 2) @@ -61,17 +57,6 @@ struct kvm_pgtable_walk_data { u64 end; }; -static u64 kvm_granule_shift(u32 level) -{ - /* Assumes KVM_PGTABLE_MAX_LEVELS is 4 */ - return ARM64_HW_PGTABLE_LEVEL_SHIFT(level); -} - -static u64 kvm_granule_size(u32 level) -{ - return BIT(kvm_granule_shift(level)); -} - #define KVM_PHYS_INVALID (-1ULL) static bool kvm_phys_is_valid(u64 phys) @@ -79,15 +64,6 @@ static bool kvm_phys_is_valid(u64 phys) return phys < BIT(id_aa64mmfr0_parange_to_phys_shift(ID_AA64MMFR0_PARANGE_MAX)); } -static bool kvm_level_supports_block_mapping(u32 level) -{ - /* - * Reject invalid block mappings and don't bother with 4TB mappings for - * 52-bit PAs. - */ - return !(level == 0 || (PAGE_SIZE != SZ_4K && level == 1)); -} - static bool kvm_block_mapping_supported(u64 addr, u64 end, u64 phys, u32 level) { u64 granule = kvm_granule_size(level); @@ -135,11 +111,6 @@ static u32 kvm_pgd_pages(u32 ia_bits, u32 start_level) return __kvm_pgd_page_idx(&pgt, -1ULL) + 1; } -static bool kvm_pte_valid(kvm_pte_t pte) -{ - return pte & KVM_PTE_VALID; -} - static bool kvm_pte_table(kvm_pte_t pte, u32 level) { if (level == KVM_PGTABLE_MAX_LEVELS - 1) @@ -151,16 +122,6 @@ static bool kvm_pte_table(kvm_pte_t pte, u32 level) return FIELD_GET(KVM_PTE_TYPE, pte) == KVM_PTE_TYPE_TABLE; } -static u64 kvm_pte_to_phys(kvm_pte_t pte) -{ - u64 pa = pte & KVM_PTE_ADDR_MASK; - - if (PAGE_SHIFT == 16) - pa |= FIELD_GET(KVM_PTE_ADDR_51_48, pte) << 48; - - return pa; -} - static kvm_pte_t kvm_phys_to_pte(u64 pa) { kvm_pte_t pte = pa & KVM_PTE_ADDR_MASK; From patchwork Thu Jul 29 13:28:03 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Quentin Perret X-Patchwork-Id: 12408723 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-17.5 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_ADSP_CUSTOM_MED,DKIM_SIGNED,DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, URIBL_BLOCKED,USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id C13EAC4338F for ; Thu, 29 Jul 2021 13:32:26 +0000 (UTC) Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 8680760184 for ; Thu, 29 Jul 2021 13:32:26 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.4.1 mail.kernel.org 8680760184 Authentication-Results: mail.kernel.org; dmarc=fail (p=reject dis=none) header.from=google.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:Cc:To:From:Subject:References: Mime-Version:Message-Id:In-Reply-To:Date:Reply-To:Content-ID: Content-Description:Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc :Resent-Message-ID:List-Owner; bh=wG/YAHEZRemHMp9X5du+4iBC3UEU8Xbfs5RmFY7C3po=; b=0TAXt18V/tABrqO3yrEaMs0Ll7 p+3BLTk0weBcFBV2Zb4CVVUm+3MRC2oynK08k3sF0dG1gc2XOssRG53PYf1yMswdep0xsWMZhLm8b fh+L29WpWpOzh8Gt0zhgPFO1TA9nk81LQ8u8e5PP/yjCNb57BWMDIEdmtcLzlP4A1Ht5u943TMCQg ru1+J6YNoCqrPA6AruzRuXMtCZscyGfeyAe4syV2x7KPtatHqJhyvwPM8EEWHSUXTiC+g/hEe3O55 nFjftQIFcgqzgvEErkhsz4qjNbHTfm5Uc9DAHOt0rw9dgWqtIh8NfrjDGbMwDFVTJnJY/HkFF4WQx YedHnicg==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1m967N-004FPP-R1; Thu, 29 Jul 2021 13:30:34 +0000 Received: from mail-qk1-x74a.google.com ([2607:f8b0:4864:20::74a]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1m965W-004Ecd-DA for linux-arm-kernel@lists.infradead.org; Thu, 29 Jul 2021 13:28:40 +0000 Received: by mail-qk1-x74a.google.com with SMTP id c3-20020a05620a0ce3b02903b8eff05707so3225605qkj.5 for ; Thu, 29 Jul 2021 06:28:37 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=date:in-reply-to:message-id:mime-version:references:subject:from:to :cc; bh=Vtw+99RHnSwt8m0h/K1748WonMCx6xnq/TU9kqpCDps=; b=SS0ObHT0a2jGjjWNqwJMq9EFMptgPm8qfoASrI1QO04/EtIPlBgkkAh0xNtRaPevsI LWBXYoGGL88UNvCSa++qVPcQzClCnwobO9slUzG2Qy11yLfFPhSFbAjLAfNhouCFqGTr AebRRvfliZozrMSyBac2X9+c6oFhJ0y9lreCEWx/IPrM9wCyOwPwYITeX0D48RVcDYyL IEm9MsgqQGwKnTzT6IKjX8MrjHbZw5a4+YPRXD8LW7mFNskPBd3TSAleOGe2NWPF9rai cRS2jVrnRWSgThRnTqExzS5rC/yVviKDUlpV20jsd8NRjRg7kQWMpMJ7dZDTumf7WYB0 hHOQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=Vtw+99RHnSwt8m0h/K1748WonMCx6xnq/TU9kqpCDps=; b=Gyyp6ZmSzvqkdyfXIPCf6vR5M9V41MCcQNqMAJ6p0pm0p/mw0/EStok9/zlUHfr4nx eQ6rgoXisRT2x4dI8AoEx1kPlCGHZbeu94XUKEXdj0ZF41oGLnIUe+n+Yy+m2Eke0tVi X9bFU9Utd5sA/vlwzx3T1IoSF61vuv81w6q59hl78DxoFkyPw3iILlv+gjuD5dVhVXk5 4T3B6GcZrhMvjx/j4rvx3tVMtooXr0fS7tXPmRGBkYkNqjR1WIAoHlpc68MT//UP5R1S WuXVGzc9m3uwB6ArAFygU2ekHBeUcdAGy1pqo3P924ci2Z+1c7wJz2FPRZ3cvCZzYukr ehbA== X-Gm-Message-State: AOAM533YopSbJFopjZQRoKtFdvSc4YhdQI0XDVrmbawEKDgMVqsAf/9j YoXU2fbVUCC+BKW9OGpLpZ5L53RIaAVc X-Google-Smtp-Source: ABdhPJxX9lBRmrW4UHAZNvD4OuS7Yq+AjD04kFvGufhGX6CXVguYeexFfDeG9IwwoKxyn8t2nOd7KrL/BUAg X-Received: from luke.lon.corp.google.com ([2a00:79e0:d:210:293a:bc89:7514:5218]) (user=qperret job=sendgmr) by 2002:a05:6214:178d:: with SMTP id ct13mr4937778qvb.22.1627565316579; Thu, 29 Jul 2021 06:28:36 -0700 (PDT) Date: Thu, 29 Jul 2021 14:28:03 +0100 In-Reply-To: <20210729132818.4091769-1-qperret@google.com> Message-Id: <20210729132818.4091769-7-qperret@google.com> Mime-Version: 1.0 References: <20210729132818.4091769-1-qperret@google.com> X-Mailer: git-send-email 2.32.0.432.gabb21c7263-goog Subject: [PATCH v3 06/21] KVM: arm64: Optimize host memory aborts From: Quentin Perret To: maz@kernel.org, james.morse@arm.com, alexandru.elisei@arm.com, suzuki.poulose@arm.com, catalin.marinas@arm.com, will@kernel.org Cc: linux-arm-kernel@lists.infradead.org, kvmarm@lists.cs.columbia.edu, linux-kernel@vger.kernel.org, ardb@kernel.org, qwandor@google.com, tabba@google.com, dbrazdil@google.com, kernel-team@android.com, Quentin Perret X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20210729_062838_501413_9A568AE7 X-CRM114-Status: GOOD ( 25.45 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org The kvm_pgtable_stage2_find_range() function is used in the host memory abort path to try and look for the largest block mapping that can be used to map the faulting address. In order to do so, the function currently walks the stage-2 page-table and looks for existing incompatible mappings within the range of the largest possible block. If incompatible mappings are found, it tries the same procedure again, but using a smaller block range, and repeats until a matching range is found (potentially up to page granularity). While this approach has benefits (mostly in the fact that it proactively coalesces host stage-2 mappings), it can be slow if the ranges are fragmented, and it isn't optimized to deal with CPUs faulting on the same IPA as all of them will do all the work every time. To avoid these issues, remove kvm_pgtable_stage2_find_range(), and walk the page-table only once in the host_mem_abort() path to find the closest leaf to the input address. With this, use the corresponding range if it is invalid and not owned by another entity. If a valid leaf is found, return -EAGAIN similar to what is done in the kvm_pgtable_stage2_map() path to optimize concurrent faults. Signed-off-by: Quentin Perret Reviewed-by: Fuad Tabba --- arch/arm64/include/asm/kvm_pgtable.h | 30 ----------- arch/arm64/kvm/hyp/nvhe/mem_protect.c | 45 +++++++++++++++- arch/arm64/kvm/hyp/pgtable.c | 74 --------------------------- 3 files changed, 44 insertions(+), 105 deletions(-) diff --git a/arch/arm64/include/asm/kvm_pgtable.h b/arch/arm64/include/asm/kvm_pgtable.h index 6938eac72c1f..83c5c97d9eac 100644 --- a/arch/arm64/include/asm/kvm_pgtable.h +++ b/arch/arm64/include/asm/kvm_pgtable.h @@ -154,16 +154,6 @@ enum kvm_pgtable_prot { #define PAGE_HYP_RO (KVM_PGTABLE_PROT_R) #define PAGE_HYP_DEVICE (PAGE_HYP | KVM_PGTABLE_PROT_DEVICE) -/** - * struct kvm_mem_range - Range of Intermediate Physical Addresses - * @start: Start of the range. - * @end: End of the range. - */ -struct kvm_mem_range { - u64 start; - u64 end; -}; - /** * enum kvm_pgtable_walk_flags - Flags to control a depth-first page-table walk. * @KVM_PGTABLE_WALK_LEAF: Visit leaf entries, including invalid @@ -490,24 +480,4 @@ int kvm_pgtable_walk(struct kvm_pgtable *pgt, u64 addr, u64 size, */ int kvm_pgtable_get_leaf(struct kvm_pgtable *pgt, u64 addr, kvm_pte_t *ptep, u32 *level); - -/** - * kvm_pgtable_stage2_find_range() - Find a range of Intermediate Physical - * Addresses with compatible permission - * attributes. - * @pgt: Page-table structure initialised by kvm_pgtable_stage2_init*(). - * @addr: Address that must be covered by the range. - * @prot: Protection attributes that the range must be compatible with. - * @range: Range structure used to limit the search space at call time and - * that will hold the result. - * - * The offset of @addr within a page is ignored. An IPA is compatible with @prot - * iff its corresponding stage-2 page-table entry has default ownership and, if - * valid, is mapped with protection attributes identical to @prot. - * - * Return: 0 on success, negative error code on failure. - */ -int kvm_pgtable_stage2_find_range(struct kvm_pgtable *pgt, u64 addr, - enum kvm_pgtable_prot prot, - struct kvm_mem_range *range); #endif /* __ARM64_KVM_PGTABLE_H__ */ diff --git a/arch/arm64/kvm/hyp/nvhe/mem_protect.c b/arch/arm64/kvm/hyp/nvhe/mem_protect.c index 74280a753efb..2148d3968aa5 100644 --- a/arch/arm64/kvm/hyp/nvhe/mem_protect.c +++ b/arch/arm64/kvm/hyp/nvhe/mem_protect.c @@ -159,6 +159,11 @@ static int host_stage2_unmap_dev_all(void) return kvm_pgtable_stage2_unmap(pgt, addr, BIT(pgt->ia_bits) - addr); } +struct kvm_mem_range { + u64 start; + u64 end; +}; + static bool find_mem_range(phys_addr_t addr, struct kvm_mem_range *range) { int cur, left = 0, right = hyp_memblock_nr; @@ -227,6 +232,44 @@ static inline int __host_stage2_idmap(u64 start, u64 end, __ret; \ }) +static inline bool range_included(struct kvm_mem_range *child, + struct kvm_mem_range *parent) +{ + return parent->start <= child->start && child->end <= parent->end; +} + +static int host_stage2_adjust_range(u64 addr, struct kvm_mem_range *range) +{ + struct kvm_mem_range cur; + kvm_pte_t pte; + u32 level; + int ret; + + hyp_assert_lock_held(&host_kvm.lock); + ret = kvm_pgtable_get_leaf(&host_kvm.pgt, addr, &pte, &level); + if (ret) + return ret; + + if (kvm_pte_valid(pte)) + return -EAGAIN; + + if (pte) + return -EPERM; + + do { + u64 granule = kvm_granule_size(level); + cur.start = ALIGN_DOWN(addr, granule); + cur.end = cur.start + granule; + level++; + } while ((level < KVM_PGTABLE_MAX_LEVELS) && + !(kvm_level_supports_block_mapping(level) && + range_included(&cur, range))); + + *range = cur; + + return 0; +} + static int host_stage2_idmap(u64 addr) { enum kvm_pgtable_prot prot = KVM_PGTABLE_PROT_R | KVM_PGTABLE_PROT_W; @@ -238,7 +281,7 @@ static int host_stage2_idmap(u64 addr) prot |= KVM_PGTABLE_PROT_X; hyp_spin_lock(&host_kvm.lock); - ret = kvm_pgtable_stage2_find_range(&host_kvm.pgt, addr, prot, &range); + ret = host_stage2_adjust_range(addr, &range); if (ret) goto unlock; diff --git a/arch/arm64/kvm/hyp/pgtable.c b/arch/arm64/kvm/hyp/pgtable.c index 49d768b92997..4dff2ad39ee4 100644 --- a/arch/arm64/kvm/hyp/pgtable.c +++ b/arch/arm64/kvm/hyp/pgtable.c @@ -1102,77 +1102,3 @@ void kvm_pgtable_stage2_destroy(struct kvm_pgtable *pgt) pgt->mm_ops->free_pages_exact(pgt->pgd, pgd_sz); pgt->pgd = NULL; } - -#define KVM_PTE_LEAF_S2_COMPAT_MASK (KVM_PTE_LEAF_ATTR_S2_PERMS | \ - KVM_PTE_LEAF_ATTR_LO_S2_MEMATTR | \ - KVM_PTE_LEAF_ATTR_S2_IGNORED) - -static int stage2_check_permission_walker(u64 addr, u64 end, u32 level, - kvm_pte_t *ptep, - enum kvm_pgtable_walk_flags flag, - void * const arg) -{ - kvm_pte_t old_attr, pte = *ptep, *new_attr = arg; - - /* - * Compatible mappings are either invalid and owned by the page-table - * owner (whose id is 0), or valid with matching permission attributes. - */ - if (kvm_pte_valid(pte)) { - old_attr = pte & KVM_PTE_LEAF_S2_COMPAT_MASK; - if (old_attr != *new_attr) - return -EEXIST; - } else if (pte) { - return -EEXIST; - } - - return 0; -} - -int kvm_pgtable_stage2_find_range(struct kvm_pgtable *pgt, u64 addr, - enum kvm_pgtable_prot prot, - struct kvm_mem_range *range) -{ - kvm_pte_t attr; - struct kvm_pgtable_walker check_perm_walker = { - .cb = stage2_check_permission_walker, - .flags = KVM_PGTABLE_WALK_LEAF, - .arg = &attr, - }; - u64 granule, start, end; - u32 level; - int ret; - - ret = stage2_set_prot_attr(pgt, prot, &attr); - if (ret) - return ret; - attr &= KVM_PTE_LEAF_S2_COMPAT_MASK; - - for (level = pgt->start_level; level < KVM_PGTABLE_MAX_LEVELS; level++) { - granule = kvm_granule_size(level); - start = ALIGN_DOWN(addr, granule); - end = start + granule; - - if (!kvm_level_supports_block_mapping(level)) - continue; - - if (start < range->start || range->end < end) - continue; - - /* - * Check the presence of existing mappings with incompatible - * permissions within the current block range, and try one level - * deeper if one is found. - */ - ret = kvm_pgtable_walk(pgt, start, granule, &check_perm_walker); - if (ret != -EEXIST) - break; - } - - if (!ret) { - range->start = start; - range->end = end; - } - - return ret; -} From patchwork Thu Jul 29 13:28:04 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Quentin Perret X-Patchwork-Id: 12408727 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-17.5 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_ADSP_CUSTOM_MED,DKIM_SIGNED,DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, URIBL_BLOCKED,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id ECA06C4338F for ; Thu, 29 Jul 2021 13:33:17 +0000 (UTC) Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id B302960EBC for ; Thu, 29 Jul 2021 13:33:17 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.4.1 mail.kernel.org B302960EBC Authentication-Results: mail.kernel.org; dmarc=fail (p=reject dis=none) header.from=google.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:Cc:To:From:Subject:References: Mime-Version:Message-Id:In-Reply-To:Date:Reply-To:Content-ID: Content-Description:Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc :Resent-Message-ID:List-Owner; bh=L559Z/wHKssRk7Vuhl9WqNCgaBG2x3ToSRUWw7AZtw8=; b=3DghoWOdU+0spOEhYbw88xw8cI Wyj+Q/GzAuXfGSI9Z+4G/NYtNrpdsX4MtXIG6kPTvUirKs+62T+NbjunHs8q8m60vN9pZfQhA1xct Jh8v7byC6aQmsEUQwHYDjsIs6RkTBytyUl1Q0Z9z5sfhKpveZmHOa3KwcimcLRDA2czTcwHbBASCp U2+sOtg2FPI4Q0Qma50k4K/KwazTtaUUhvSe9vXB8c05D8SlITsp23xF9mAc/IFfnUiQmZE6CIaNY OZ8ofooFB2ncpfbU3OfEQJFtacHBwn8fLOX6RTwiuE54816i0KjfrtQ1gvXExh7ryvQPFY+PcmpOL UNDHN4pQ==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1m968Y-004Fvn-Ei; Thu, 29 Jul 2021 13:31:47 +0000 Received: from mail-ed1-x549.google.com ([2a00:1450:4864:20::549]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1m965c-004Ed3-QL for linux-arm-kernel@lists.infradead.org; Thu, 29 Jul 2021 13:28:46 +0000 Received: by mail-ed1-x549.google.com with SMTP id u25-20020aa7d8990000b02903bb6a903d90so2936490edq.17 for ; Thu, 29 Jul 2021 06:28:40 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=date:in-reply-to:message-id:mime-version:references:subject:from:to :cc; bh=rFkRNcfpvB1Ju4IVqteFXhSiHej5m8ER5DA8p5t0LAM=; b=adgE5oH5ZzQyJ5my+MCD24NaPHWhLe66wtzcPFxdI/5uGtUKivh3E2sLcxhhUi1oUH kO7wflN08UcEigT8OK5o3G9pnCptziZAF+CPeYXNsUPhGWRx9thRs8VvjUsj0wnVgVd9 FsDGXrEqaK5mNzuNKveKDQw9yNnZPbE3IwF3Nyp5jGLc24vZ6pC16gO1goRrmc6p4QeG 1pl2RVMEOAGPak5/N4DWB4EdeaFsM/1BqnR1Y+4My1F0edR0G+W0cHFgkk/nQylpkCT7 wfl1k5OCIXaC4iXp8zFS7ADwIDEW7L7ZdpJPW5wBQMrYF0zfhDiqkNWELGt0tzVlNpX3 W7MA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=rFkRNcfpvB1Ju4IVqteFXhSiHej5m8ER5DA8p5t0LAM=; b=MHpP9TLpHhRoxDsfLQ+K+A8bYE0qVhgqXGqliDYd8lCrDsGRjh2hnMMzoDHXkSIRhO 9AyZVdi+rCMDLdO2BUpprrrGDK4mQlrHtjf5+1WfA5KWzuQngKuJKXBhzaLMszYCZ8FA T2mKOHm3IMfZ3lS5DMOLhsq4DRBzdJ/BE6C4HeNo90qDUGP86eASIebrFONI07yfDLme YmSinIt4t4qdigQSI3iGWrqOkmvb6gllhDzjBN3hBwharkmUKBd83ki6moaXKElnVezd JYmgkUf0CE9NvK4fZs9XJQC0xH/1kOMjE9UQTxRl8IkPmAzCYCj4ycfzbEAWA8QKkXCI nMtw== X-Gm-Message-State: AOAM530Y2QnyZ1Olyr7bIqGRA8TA6JIQobY/CvtmEIdx4e3az/xDRn8l XRfvcAdsmsNDFL5YMFuuhPUkp/36dzhd X-Google-Smtp-Source: ABdhPJycW7HuuY9Gim/mXG1ILhZq0/6nWfqymTtbWe0EQGgl+pPQmTGRPV48lVOvv5Y7iUj6rD/h42ZYLODf X-Received: from luke.lon.corp.google.com ([2a00:79e0:d:210:293a:bc89:7514:5218]) (user=qperret job=sendgmr) by 2002:aa7:dbc3:: with SMTP id v3mr6100538edt.63.1627565318816; Thu, 29 Jul 2021 06:28:38 -0700 (PDT) Date: Thu, 29 Jul 2021 14:28:04 +0100 In-Reply-To: <20210729132818.4091769-1-qperret@google.com> Message-Id: <20210729132818.4091769-8-qperret@google.com> Mime-Version: 1.0 References: <20210729132818.4091769-1-qperret@google.com> X-Mailer: git-send-email 2.32.0.432.gabb21c7263-goog Subject: [PATCH v3 07/21] KVM: arm64: Rename KVM_PTE_LEAF_ATTR_S2_IGNORED From: Quentin Perret To: maz@kernel.org, james.morse@arm.com, alexandru.elisei@arm.com, suzuki.poulose@arm.com, catalin.marinas@arm.com, will@kernel.org Cc: linux-arm-kernel@lists.infradead.org, kvmarm@lists.cs.columbia.edu, linux-kernel@vger.kernel.org, ardb@kernel.org, qwandor@google.com, tabba@google.com, dbrazdil@google.com, kernel-team@android.com, Quentin Perret X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20210729_062844_902558_7789F946 X-CRM114-Status: GOOD ( 11.01 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org The ignored bits for both stage-1 and stage-2 page and block descriptors are in [55:58], so rename KVM_PTE_LEAF_ATTR_S2_IGNORED to make it applicable to both. And while at it, since these bits are more commonly known as 'software' bits, rename accordingly. Signed-off-by: Quentin Perret Reviewed-by: Fuad Tabba --- arch/arm64/kvm/hyp/pgtable.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/arch/arm64/kvm/hyp/pgtable.c b/arch/arm64/kvm/hyp/pgtable.c index 4dff2ad39ee4..59a394d82de3 100644 --- a/arch/arm64/kvm/hyp/pgtable.c +++ b/arch/arm64/kvm/hyp/pgtable.c @@ -36,6 +36,8 @@ #define KVM_PTE_LEAF_ATTR_HI GENMASK(63, 51) +#define KVM_PTE_LEAF_ATTR_HI_SW GENMASK(58, 55) + #define KVM_PTE_LEAF_ATTR_HI_S1_XN BIT(54) #define KVM_PTE_LEAF_ATTR_HI_S2_XN BIT(54) @@ -44,8 +46,6 @@ KVM_PTE_LEAF_ATTR_LO_S2_S2AP_W | \ KVM_PTE_LEAF_ATTR_HI_S2_XN) -#define KVM_PTE_LEAF_ATTR_S2_IGNORED GENMASK(58, 55) - #define KVM_INVALID_PTE_OWNER_MASK GENMASK(63, 56) #define KVM_MAX_OWNER_ID 1 From patchwork Thu Jul 29 13:28:05 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Quentin Perret X-Patchwork-Id: 12408725 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-17.5 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_ADSP_CUSTOM_MED,DKIM_SIGNED,DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, URIBL_BLOCKED,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 63C28C4338F for ; Thu, 29 Jul 2021 13:32:32 +0000 (UTC) Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 3261B60184 for ; Thu, 29 Jul 2021 13:32:32 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.4.1 mail.kernel.org 3261B60184 Authentication-Results: mail.kernel.org; dmarc=fail (p=reject dis=none) header.from=google.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:Cc:To:From:Subject:References: Mime-Version:Message-Id:In-Reply-To:Date:Reply-To:Content-ID: Content-Description:Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc :Resent-Message-ID:List-Owner; bh=dSBw1lh2z4TOrVaTKajO/1XisEdHP1hzKMYyIcd6B3k=; b=tCxiPiTaN0tjsPVCERBIMq9fhV y96dFkgFW4ugDj5AXmz5w1JCSrSjX3uxhXTikJacEkhBc9bqSdcTwIMoLqVNfaAMjd2EGpyt4vkwG EWncp8acT/yIAAwZJinQXJwXCxKqbrV6dfXV2fX6pqO4qFpKZ/ZcD5dDWEGOQqz/AVBRfYarqkdbI SPWm1TbQyhJFx4o6jkpTh+DVaGXWqdmwUesMwJx4keFdlnt7wA6xq5qBYkkFBF52+WEES7xDuvVS8 5OFJu8nL5SShpuSgfhNqzR5chzVMnpkhK/Jk7cKqhx39PQ1YrBLzW0r6SvTQtbFtn5mfIUvILQOwQ od1DUj3A==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1m967x-004FdO-RX; Thu, 29 Jul 2021 13:31:11 +0000 Received: from mail-wr1-x44a.google.com ([2a00:1450:4864:20::44a]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1m965b-004Edb-De for linux-arm-kernel@lists.infradead.org; Thu, 29 Jul 2021 13:28:44 +0000 Received: by mail-wr1-x44a.google.com with SMTP id f6-20020adfe9060000b0290153abe88c2dso2248230wrm.20 for ; Thu, 29 Jul 2021 06:28:42 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=date:in-reply-to:message-id:mime-version:references:subject:from:to :cc; bh=sBorrifANrV+vpyuatLE9LgyQ9DNiLt3tvLMqd3Pt6A=; b=Ut7XM1RBkTI6hICybPUi8KmJxYV9LDmJ+BO6n904mN06Lz6fMB/V/BtKm6hhsuLfZP ivNztgGio4ESMLW5q3gVJgBLVveqCae0u3Cf+OKFjx6mnhwTYrhUTBdFbeTTJYOH2Oro egeF+yk4HTHuh6YDWWxQ/DlmPg6CLxqWIwkzudNjJzvZNRClSc0OkY+jbq0xqXUU/AJb ej+MxueBK07NIyaNgEDhpAQ0F2nwLtPwY0F0IGWM8mD7iwJf9+lNClSNk/EMPe/lAOyx FXnhesCTlFKoKTNCnkpafZ7MinyOF/hCThP8HikLHozvU5iFs+Dhw8yEZKqnOzO+cPPS ViCQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=sBorrifANrV+vpyuatLE9LgyQ9DNiLt3tvLMqd3Pt6A=; b=C8vfXj0zZ7erGD+JpjtllxUQY7nMI1fLl/ABf2VZv3dUpeSrTW2ZK1kEDn8Z6EYqSL mbvTs4yqdihz655tFDWsOJcbGmHV4/OSjreRRM3EcswFYEQ1l53Rt5/UBErgwFpB5ikf J2UUqMPhxWPBQ69SkZaExyHlq2IPBJ/bJ+pri2LU2Odcnwgh7v2Gux+ickmXy/E3noxf ry0XeST2j+IbsOIqdoa4xjyJgplCOieB+9zUV3O4hJdqzoY/iZUhH360U8SLsx1RIBez 6iR3z1vBTC46dbCp5pjBMQafNXGhVlBffGfCOFFdy56jGRbHtYK1KBtnaadGo4zJaAHb eY0g== X-Gm-Message-State: AOAM532F9BQxXf2CALrtmXlAm8Z9323HbmBiHqUY8eVgUTDlEAfPlfBI f3S4ZQLL7v85W4wUgBaLENjlTITtc2cB X-Google-Smtp-Source: ABdhPJwtD68LvsEqvJLyJ+xAUhNDPqjLqDqVmLSO3BaeBLQNVsxniunIN5iAoYlgcySpAt9xikvTGJBkyHkY X-Received: from luke.lon.corp.google.com ([2a00:79e0:d:210:293a:bc89:7514:5218]) (user=qperret job=sendgmr) by 2002:a1c:980e:: with SMTP id a14mr4764799wme.123.1627565321434; Thu, 29 Jul 2021 06:28:41 -0700 (PDT) Date: Thu, 29 Jul 2021 14:28:05 +0100 In-Reply-To: <20210729132818.4091769-1-qperret@google.com> Message-Id: <20210729132818.4091769-9-qperret@google.com> Mime-Version: 1.0 References: <20210729132818.4091769-1-qperret@google.com> X-Mailer: git-send-email 2.32.0.432.gabb21c7263-goog Subject: [PATCH v3 08/21] KVM: arm64: Don't overwrite software bits with owner id From: Quentin Perret To: maz@kernel.org, james.morse@arm.com, alexandru.elisei@arm.com, suzuki.poulose@arm.com, catalin.marinas@arm.com, will@kernel.org Cc: linux-arm-kernel@lists.infradead.org, kvmarm@lists.cs.columbia.edu, linux-kernel@vger.kernel.org, ardb@kernel.org, qwandor@google.com, tabba@google.com, dbrazdil@google.com, kernel-team@android.com, Quentin Perret X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20210729_062843_530974_A076F945 X-CRM114-Status: GOOD ( 13.46 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org We will soon start annotating page-tables with new flags to track shared pages and such, and we will do so in valid mappings using software bits in the PTEs, as provided by the architecture. However, it is possible that we will need to use those flags to annotate invalid mappings as well in the future, similar to what we do to track page ownership in the host stage-2. In order to facilitate the annotation of invalid mappings with such flags, it would be preferable to re-use the same bits as for valid mappings (bits [58-55]), but these are currently used for ownership encoding. Since we have plenty of bits left to use in invalid mappings, move the ownership bits further down the PTE to avoid the conflict. Signed-off-by: Quentin Perret Reviewed-by: Fuad Tabba --- arch/arm64/kvm/hyp/pgtable.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/arch/arm64/kvm/hyp/pgtable.c b/arch/arm64/kvm/hyp/pgtable.c index 59a394d82de3..1ee1168ac32d 100644 --- a/arch/arm64/kvm/hyp/pgtable.c +++ b/arch/arm64/kvm/hyp/pgtable.c @@ -46,7 +46,7 @@ KVM_PTE_LEAF_ATTR_LO_S2_S2AP_W | \ KVM_PTE_LEAF_ATTR_HI_S2_XN) -#define KVM_INVALID_PTE_OWNER_MASK GENMASK(63, 56) +#define KVM_INVALID_PTE_OWNER_MASK GENMASK(9, 2) #define KVM_MAX_OWNER_ID 1 struct kvm_pgtable_walk_data { From patchwork Thu Jul 29 13:28:06 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Quentin Perret X-Patchwork-Id: 12408729 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-17.5 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_ADSP_CUSTOM_MED,DKIM_SIGNED,DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, URIBL_BLOCKED,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 9E412C432BE for ; Thu, 29 Jul 2021 13:34:36 +0000 (UTC) Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 67FE460F42 for ; Thu, 29 Jul 2021 13:34:36 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.4.1 mail.kernel.org 67FE460F42 Authentication-Results: mail.kernel.org; dmarc=fail (p=reject dis=none) header.from=google.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:Cc:To:From:Subject:References: Mime-Version:Message-Id:In-Reply-To:Date:Reply-To:Content-ID: Content-Description:Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc :Resent-Message-ID:List-Owner; bh=N2hV3Ty3PD+sSiM1/jojwW4idGqvIV7s4YIZmSiBJ8s=; b=x29EjY0NMawqJcA1E4vpWwYo1P 1Mgm792YrmgfECEawaiXxLpWzZZLNNsZ/TKkOittA9fhXkafLRc9/R1DKCIsfk4LMztRtRoFfcxzY xRwDdFBVCcj8m+2A/a8EoJpoznCEJtNmvdBAFzL3s1mo+NnGjPeYmYrg27tfmGhCIXbBkV9NEP92p z9qQL1XZ81wOnFHBDFw6OmnAchyOqDkMETOprADAlN4vyNCnrZTWCkg2mW63yl9gslfpqNlc1B6Ep CEWmODRZYYpRHeOm9Qq9FHI+j4Ykbzd4OinRWRDPTw12l+E6W1/Wk7kRuITCwIQp7kdD3Ja9zPEcI cmD3Z+kQ==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1m969V-004GMB-3l; Thu, 29 Jul 2021 13:32:45 +0000 Received: from mail-ed1-x54a.google.com ([2a00:1450:4864:20::54a]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1m965h-004EeC-1d for linux-arm-kernel@lists.infradead.org; Thu, 29 Jul 2021 13:28:50 +0000 Received: by mail-ed1-x54a.google.com with SMTP id s8-20020a0564020148b02903948b71f25cso2942436edu.4 for ; Thu, 29 Jul 2021 06:28:45 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=date:in-reply-to:message-id:mime-version:references:subject:from:to :cc; bh=3edu5r5MgiwqKIsFZqWR1OVvbwtz5pLkXdnxfKTgrkE=; b=kiFyFVttc3U28KQ3sfRTofJfHRrKRTa08Tr1wUv7AmJ7kmxQRQqWLqlx2vJLxnRtiB gRP5OSpTMXZ/ELbIz+ZiYYSRyHJQB4gpq308Q//Q+M8a+IkmJK4zXuePNe1iad8R6ja5 eMzYY5Z+oyjxt+f/5LtLu0Hsc68dLxTSalqaiC6MOS51NM7Vok4vOIm5S9qUlX2pNnEf OeA6/wlF2Q8cQRv34+Q8Oe3lAuIlBdBzWJOPvwJcpQE1dOMoCrTw3OPaL+7abgre+E4P F0O7kH46klsMF5HsRReEdQc0jztwb9nOkD0bFWkPhtVneMWJ37S8n/CMDzd2iYeuib5c oFJw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=3edu5r5MgiwqKIsFZqWR1OVvbwtz5pLkXdnxfKTgrkE=; b=VWFfFvOyyg8JfGWrKi7orPJNU+Ke4yP4VNqMqnRzSAPFEQFBKWkEE8uJrzl8xlgcMg o9tVYtdlV2xyc0wbJPkZ3FInQDYI9i4cAe06qcVpD72JW0GLjZdzO/Hjuud/5f/zkxAG 0GWAQLHcI0ahK+mS1psF7rmFTpo3DQ3sl3f/JiUOB4Tedc5eBw6F9PXxRO9yEKl/FqA+ DDO3qRRhXd/28OvJC+3C70VsVT3sSZ0BIdSSgHujPdb780czHJvK0luAVcijFC0vF6nk zTJo6iRtoOuYOTLjMDZWoV1VtqHvaluaJYOxQPtucwVKCJrpQSl5MifLYvFyaoUo78o/ 7C7A== X-Gm-Message-State: AOAM530AIKvHr4nm6oB3UZn3BRbN4X4TjpO7w3iIegO3Tmn88LT97fXC 3Mt+KsDmvZ8ujQn+AqD+i2iCJgIW5Znq X-Google-Smtp-Source: ABdhPJwP/AN3fa8sbZcgybFMXXlf8hjA0Q9gZJpOb0OUTytheQm4ssO6cEU1bGXuFiecy7eUiVkRiSSKUJ8i X-Received: from luke.lon.corp.google.com ([2a00:79e0:d:210:293a:bc89:7514:5218]) (user=qperret job=sendgmr) by 2002:a17:906:2bc9:: with SMTP id n9mr4537344ejg.23.1627565323698; Thu, 29 Jul 2021 06:28:43 -0700 (PDT) Date: Thu, 29 Jul 2021 14:28:06 +0100 In-Reply-To: <20210729132818.4091769-1-qperret@google.com> Message-Id: <20210729132818.4091769-10-qperret@google.com> Mime-Version: 1.0 References: <20210729132818.4091769-1-qperret@google.com> X-Mailer: git-send-email 2.32.0.432.gabb21c7263-goog Subject: [PATCH v3 09/21] KVM: arm64: Tolerate re-creating hyp mappings to set software bits From: Quentin Perret To: maz@kernel.org, james.morse@arm.com, alexandru.elisei@arm.com, suzuki.poulose@arm.com, catalin.marinas@arm.com, will@kernel.org Cc: linux-arm-kernel@lists.infradead.org, kvmarm@lists.cs.columbia.edu, linux-kernel@vger.kernel.org, ardb@kernel.org, qwandor@google.com, tabba@google.com, dbrazdil@google.com, kernel-team@android.com, Quentin Perret X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20210729_062849_144192_991AC074 X-CRM114-Status: GOOD ( 11.90 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org The current hypervisor stage-1 mapping code doesn't allow changing an existing valid mapping. Relax this condition by allowing changes that only target software bits, as that will soon be needed to annotate shared pages. Signed-off-by: Quentin Perret Reviewed-by: Fuad Tabba --- arch/arm64/kvm/hyp/pgtable.c | 18 ++++++++++++++++-- 1 file changed, 16 insertions(+), 2 deletions(-) diff --git a/arch/arm64/kvm/hyp/pgtable.c b/arch/arm64/kvm/hyp/pgtable.c index 1ee1168ac32d..2689fcb7901d 100644 --- a/arch/arm64/kvm/hyp/pgtable.c +++ b/arch/arm64/kvm/hyp/pgtable.c @@ -362,6 +362,21 @@ static int hyp_set_prot_attr(enum kvm_pgtable_prot prot, kvm_pte_t *ptep) return 0; } +static bool hyp_pte_needs_update(kvm_pte_t old, kvm_pte_t new) +{ + /* + * Tolerate KVM recreating the exact same mapping, or changing software + * bits if the existing mapping was valid. + */ + if (old == new) + return false; + + if (!kvm_pte_valid(old)) + return true; + + return !WARN_ON((old ^ new) & ~KVM_PTE_LEAF_ATTR_HI_SW); +} + static bool hyp_map_walker_try_leaf(u64 addr, u64 end, u32 level, kvm_pte_t *ptep, struct hyp_map_data *data) { @@ -371,9 +386,8 @@ static bool hyp_map_walker_try_leaf(u64 addr, u64 end, u32 level, if (!kvm_block_mapping_supported(addr, end, phys, level)) return false; - /* Tolerate KVM recreating the exact same mapping */ new = kvm_init_valid_leaf_pte(phys, data->attr, level); - if (old != new && !WARN_ON(kvm_pte_valid(old))) + if (hyp_pte_needs_update(old, new)) smp_store_release(ptep, new); data->phys += granule; From patchwork Thu Jul 29 13:28:07 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Quentin Perret X-Patchwork-Id: 12408751 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-17.5 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_ADSP_CUSTOM_MED,DKIM_SIGNED,DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, URIBL_BLOCKED,USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 119A4C4338F for ; Thu, 29 Jul 2021 13:36:53 +0000 (UTC) Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id D61D46052B for ; Thu, 29 Jul 2021 13:36:52 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.4.1 mail.kernel.org D61D46052B Authentication-Results: mail.kernel.org; dmarc=fail (p=reject dis=none) header.from=google.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:Cc:To:From:Subject:References: Mime-Version:Message-Id:In-Reply-To:Date:Reply-To:Content-ID: Content-Description:Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc :Resent-Message-ID:List-Owner; bh=fMOLL2HJ/DRIIyfftQVpN0QbCT6QrgAGD3+XvVtdw0A=; b=UXjxLOvyzGMT5PuKRBeaz+YZuD SQ/eO+i8gYL7TGbbpEgPMGkwCbC7jnanhs+39sBYbT1bWXKn+H6ICzDCxPGpOwHZrhKECkMM9FmMm RIxzIkQ0eA+u3VqxXaf7cpAyaFM9ImX6JX3lmaDUF8qaucvucXDRMGzFGdZzxqMTibWcTXb6MicIK cc0unv+9o8aNZ7ufqeEXmDUr5Tajlz+YJSrl62y943ib9lYkvfesfps+P4QhJH+PG9i30iI3HK52f 5jjMHrvwCxsmoszd+BSf0DSlwLoXgldbpTkwo+buQRWQb/OouVp3vMp84wXrOitDhILhc7eV8WtK5 LMlFWg2A==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1m96Av-004GvO-N8; Thu, 29 Jul 2021 13:34:14 +0000 Received: from mail-qt1-x849.google.com ([2607:f8b0:4864:20::849]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1m965h-004Ef4-TS for linux-arm-kernel@lists.infradead.org; Thu, 29 Jul 2021 13:28:52 +0000 Received: by mail-qt1-x849.google.com with SMTP id q10-20020a05622a04cab0290267bc0213a9so2777334qtx.0 for ; Thu, 29 Jul 2021 06:28:46 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=date:in-reply-to:message-id:mime-version:references:subject:from:to :cc; bh=CkV/kHhsqJw+nOu8v6NI/4ji+nJTLkGt2nfDjNU7Ec8=; b=dNcwPm7Tk40t8K/OOO/NAWip/HFFLWuYdE/E4M0vltqAQWc3is1NFCW2o6Lri3/KKk BUZDNVod2hkUTrQgWOZj+gd174h3ljFy8cqJK2vhuKZ02DUeyV+/ZwrYrvNJRdjiUgBr vAUtvD8mMKwE6zq2EBqxbAhJKht/AbXEUv57QBjbEKcVuAX4jd7HPazJ0ZFNfGG4UqeZ 9hpwt9omFfywTyHcOjRXtljV6w1h63zuFaj0VTVKi+pPDrN65iLNCAhgOrS5/M7rABxv 3ngJNKXFq24NJcCgLvGTqgUOGMRP9p2MTDVP+/4sluM2PBvAZKLo53BP5hQevugl9oVJ ywKA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=CkV/kHhsqJw+nOu8v6NI/4ji+nJTLkGt2nfDjNU7Ec8=; b=lNWBIQ2n5QvgeTsOK+gP56ayJqy+XTianHgfBObM4ZU7ADJRhUB7WIixUWIplc/8HE oet3mQx9EHbxD3icVLDhxeT0hVL8p1nav3yGk2rsWAvp7dhqxtllQOQzSuNzVUacSdTC E00FEiQBQgGTaHRlaQ01fG/37fd3zYOKIadGOoTU4eUVpzog83i24AuDKO1bBe9VT3fX +EMNJN3VnMSc04q65Y2tXmJWyI1hUETM+X2gHmiyETm6yvBvI4N+ugBwh10ZlFLTL8pH yJ1rgj39pJBFS3Qi2dASqZkxEz794zc2SEayIXAW2oz5NZsf0/AiT6SN5jaegQal3KO7 XkTQ== X-Gm-Message-State: AOAM531P8UzRm9Q4QcDfB/pN6CeG2dx7ad3j+R486BlXd8ghdksVn8G8 YXoCrOuUu6ZktU99ColZpP2i6wumWdQj X-Google-Smtp-Source: ABdhPJzfHm71RgU4+zf8yDrb/6TIjq5kY48fzz3KQqh8QUhkWYexNgaiLCw7letV8UWW8o4z5U7hSn//YzDq X-Received: from luke.lon.corp.google.com ([2a00:79e0:d:210:293a:bc89:7514:5218]) (user=qperret job=sendgmr) by 2002:a05:6214:a63:: with SMTP id ef3mr5301947qvb.15.1627565326131; Thu, 29 Jul 2021 06:28:46 -0700 (PDT) Date: Thu, 29 Jul 2021 14:28:07 +0100 In-Reply-To: <20210729132818.4091769-1-qperret@google.com> Message-Id: <20210729132818.4091769-11-qperret@google.com> Mime-Version: 1.0 References: <20210729132818.4091769-1-qperret@google.com> X-Mailer: git-send-email 2.32.0.432.gabb21c7263-goog Subject: [PATCH v3 10/21] KVM: arm64: Enable forcing page-level stage-2 mappings From: Quentin Perret To: maz@kernel.org, james.morse@arm.com, alexandru.elisei@arm.com, suzuki.poulose@arm.com, catalin.marinas@arm.com, will@kernel.org Cc: linux-arm-kernel@lists.infradead.org, kvmarm@lists.cs.columbia.edu, linux-kernel@vger.kernel.org, ardb@kernel.org, qwandor@google.com, tabba@google.com, dbrazdil@google.com, kernel-team@android.com, Quentin Perret X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20210729_062850_047645_3252D08B X-CRM114-Status: GOOD ( 25.49 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Much of the stage-2 manipulation logic relies on being able to destroy block mappings if e.g. installing a smaller mapping in the range. The rationale for this behaviour is that stage-2 mappings can always be re-created lazily. However, this gets more complicated when the stage-2 page-table is used to store metadata about the underlying pages. In such cases, destroying a block mapping may lead to losing part of the state, and confuse the user of those metadata (such as the hypervisor in nVHE protected mode). To avoid this, introduce a callback function in the pgtable struct which is called during all map operations to determine whether the mappings can use blocks, or should be forced to page granularity. This is used by the hypervisor when creating the host stage-2 to force page-level mappings when using non-default protection attributes. Signed-off-by: Quentin Perret --- arch/arm64/include/asm/kvm_pgtable.h | 65 ++++++++++++++++----------- arch/arm64/kvm/hyp/nvhe/mem_protect.c | 30 +++++++++++-- arch/arm64/kvm/hyp/pgtable.c | 29 +++++++++--- 3 files changed, 91 insertions(+), 33 deletions(-) diff --git a/arch/arm64/include/asm/kvm_pgtable.h b/arch/arm64/include/asm/kvm_pgtable.h index 83c5c97d9eac..ba7dcade2798 100644 --- a/arch/arm64/include/asm/kvm_pgtable.h +++ b/arch/arm64/include/asm/kvm_pgtable.h @@ -115,25 +115,6 @@ enum kvm_pgtable_stage2_flags { KVM_PGTABLE_S2_IDMAP = BIT(1), }; -/** - * struct kvm_pgtable - KVM page-table. - * @ia_bits: Maximum input address size, in bits. - * @start_level: Level at which the page-table walk starts. - * @pgd: Pointer to the first top-level entry of the page-table. - * @mm_ops: Memory management callbacks. - * @mmu: Stage-2 KVM MMU struct. Unused for stage-1 page-tables. - */ -struct kvm_pgtable { - u32 ia_bits; - u32 start_level; - kvm_pte_t *pgd; - struct kvm_pgtable_mm_ops *mm_ops; - - /* Stage-2 only */ - struct kvm_s2_mmu *mmu; - enum kvm_pgtable_stage2_flags flags; -}; - /** * enum kvm_pgtable_prot - Page-table permissions and attributes. * @KVM_PGTABLE_PROT_X: Execute permission. @@ -149,11 +130,41 @@ enum kvm_pgtable_prot { KVM_PGTABLE_PROT_DEVICE = BIT(3), }; -#define PAGE_HYP (KVM_PGTABLE_PROT_R | KVM_PGTABLE_PROT_W) +#define KVM_PGTABLE_PROT_RW (KVM_PGTABLE_PROT_R | KVM_PGTABLE_PROT_W) +#define KVM_PGTABLE_PROT_RWX (KVM_PGTABLE_PROT_RW | KVM_PGTABLE_PROT_X) + +#define PAGE_HYP KVM_PGTABLE_PROT_RW #define PAGE_HYP_EXEC (KVM_PGTABLE_PROT_R | KVM_PGTABLE_PROT_X) #define PAGE_HYP_RO (KVM_PGTABLE_PROT_R) #define PAGE_HYP_DEVICE (PAGE_HYP | KVM_PGTABLE_PROT_DEVICE) +typedef bool (*kvm_pgtable_force_pte_cb_t)(u64 addr, u64 end, + enum kvm_pgtable_prot prot); + +/** + * struct kvm_pgtable - KVM page-table. + * @ia_bits: Maximum input address size, in bits. + * @start_level: Level at which the page-table walk starts. + * @pgd: Pointer to the first top-level entry of the page-table. + * @mm_ops: Memory management callbacks. + * @mmu: Stage-2 KVM MMU struct. Unused for stage-1 page-tables. + * @flags: Stage-2 page-table flags. + * @force_pte_cb: Callback function used during map operations to decide + * whether block mappings can be used to map the given IPA + * range. + */ +struct kvm_pgtable { + u32 ia_bits; + u32 start_level; + kvm_pte_t *pgd; + struct kvm_pgtable_mm_ops *mm_ops; + + /* Stage-2 only */ + struct kvm_s2_mmu *mmu; + enum kvm_pgtable_stage2_flags flags; + kvm_pgtable_force_pte_cb_t force_pte_cb; +}; + /** * enum kvm_pgtable_walk_flags - Flags to control a depth-first page-table walk. * @KVM_PGTABLE_WALK_LEAF: Visit leaf entries, including invalid @@ -246,21 +257,25 @@ int kvm_pgtable_hyp_map(struct kvm_pgtable *pgt, u64 addr, u64 size, u64 phys, u64 kvm_get_vtcr(u64 mmfr0, u64 mmfr1, u32 phys_shift); /** - * kvm_pgtable_stage2_init_flags() - Initialise a guest stage-2 page-table. + * __kvm_pgtable_stage2_init() - Initialise a guest stage-2 page-table. * @pgt: Uninitialised page-table structure to initialise. * @arch: Arch-specific KVM structure representing the guest virtual * machine. * @mm_ops: Memory management callbacks. * @flags: Stage-2 configuration flags. + * @force_pte_cb: Callback function used during map operations to decide + * whether block mappings can be used to map the given IPA + * range. * * Return: 0 on success, negative error code on failure. */ -int kvm_pgtable_stage2_init_flags(struct kvm_pgtable *pgt, struct kvm_arch *arch, - struct kvm_pgtable_mm_ops *mm_ops, - enum kvm_pgtable_stage2_flags flags); +int __kvm_pgtable_stage2_init(struct kvm_pgtable *pgt, struct kvm_arch *arch, + struct kvm_pgtable_mm_ops *mm_ops, + enum kvm_pgtable_stage2_flags flags, + kvm_pgtable_force_pte_cb_t force_pte_cb); #define kvm_pgtable_stage2_init(pgt, arch, mm_ops) \ - kvm_pgtable_stage2_init_flags(pgt, arch, mm_ops, 0) + __kvm_pgtable_stage2_init(pgt, arch, mm_ops, 0, NULL) /** * kvm_pgtable_stage2_destroy() - Destroy an unused guest stage-2 page-table. diff --git a/arch/arm64/kvm/hyp/nvhe/mem_protect.c b/arch/arm64/kvm/hyp/nvhe/mem_protect.c index 2148d3968aa5..70c57d2c3024 100644 --- a/arch/arm64/kvm/hyp/nvhe/mem_protect.c +++ b/arch/arm64/kvm/hyp/nvhe/mem_protect.c @@ -89,6 +89,7 @@ static void prepare_host_vtcr(void) id_aa64mmfr1_el1_sys_val, phys_shift); } +static bool host_stage2_force_pte_cb(u64 addr, u64 end, enum kvm_pgtable_prot prot); int kvm_host_prepare_stage2(void *pgt_pool_base) { struct kvm_s2_mmu *mmu = &host_kvm.arch.mmu; @@ -101,8 +102,9 @@ int kvm_host_prepare_stage2(void *pgt_pool_base) if (ret) return ret; - ret = kvm_pgtable_stage2_init_flags(&host_kvm.pgt, &host_kvm.arch, - &host_kvm.mm_ops, KVM_HOST_S2_FLAGS); + ret = __kvm_pgtable_stage2_init(&host_kvm.pgt, &host_kvm.arch, + &host_kvm.mm_ops, KVM_HOST_S2_FLAGS, + host_stage2_force_pte_cb); if (ret) return ret; @@ -270,9 +272,31 @@ static int host_stage2_adjust_range(u64 addr, struct kvm_mem_range *range) return 0; } +static bool host_stage2_force_pte_cb(u64 addr, u64 end, enum kvm_pgtable_prot prot) +{ + /* + * Block mappings must be used with care in the host stage-2 as a + * kvm_pgtable_stage2_map() operation targeting a page in the range of + * an existing block will delete the block under the assumption that + * mappings in the rest of the block range can always be rebuilt lazily. + * That assumption is correct for the host stage-2 with RWX mappings + * targeting memory or RW mappings targeting MMIO ranges (see + * host_stage2_idmap() below which implements some of the host memory + * abort logic). However, this is not safe for any other mappings where + * the host stage-2 page-table is in fact the only place where this + * state is stored. In all those cases, it is safer to use page-level + * mappings, hence avoiding to lose the state because of side-effects in + * kvm_pgtable_stage2_map(). + */ + if (range_is_memory(addr, end)) + return prot != KVM_PGTABLE_PROT_RWX; + else + return prot != KVM_PGTABLE_PROT_RW; +} + static int host_stage2_idmap(u64 addr) { - enum kvm_pgtable_prot prot = KVM_PGTABLE_PROT_R | KVM_PGTABLE_PROT_W; + enum kvm_pgtable_prot prot = KVM_PGTABLE_PROT_RW; struct kvm_mem_range range; bool is_memory = find_mem_range(addr, &range); int ret; diff --git a/arch/arm64/kvm/hyp/pgtable.c b/arch/arm64/kvm/hyp/pgtable.c index 2689fcb7901d..bcc02e6e0f62 100644 --- a/arch/arm64/kvm/hyp/pgtable.c +++ b/arch/arm64/kvm/hyp/pgtable.c @@ -452,6 +452,8 @@ int kvm_pgtable_hyp_init(struct kvm_pgtable *pgt, u32 va_bits, pgt->start_level = KVM_PGTABLE_MAX_LEVELS - levels; pgt->mm_ops = mm_ops; pgt->mmu = NULL; + pgt->force_pte_cb = NULL; + return 0; } @@ -489,6 +491,9 @@ struct stage2_map_data { void *memcache; struct kvm_pgtable_mm_ops *mm_ops; + + /* Force mappings to page granularity */ + bool force_pte; }; u64 kvm_get_vtcr(u64 mmfr0, u64 mmfr1, u32 phys_shift) @@ -602,6 +607,15 @@ static bool stage2_pte_executable(kvm_pte_t pte) return !(pte & KVM_PTE_LEAF_ATTR_HI_S2_XN); } +static bool stage2_block_mapping_allowed(u64 addr, u64 end, u32 level, + struct stage2_map_data *data) +{ + if (data->force_pte && (level < (KVM_PGTABLE_MAX_LEVELS - 1))) + return false; + + return kvm_block_mapping_supported(addr, end, data->phys, level); +} + static int stage2_map_walker_try_leaf(u64 addr, u64 end, u32 level, kvm_pte_t *ptep, struct stage2_map_data *data) @@ -611,7 +625,7 @@ static int stage2_map_walker_try_leaf(u64 addr, u64 end, u32 level, struct kvm_pgtable *pgt = data->mmu->pgt; struct kvm_pgtable_mm_ops *mm_ops = data->mm_ops; - if (!kvm_block_mapping_supported(addr, end, phys, level)) + if (!stage2_block_mapping_allowed(addr, end, level, data)) return -E2BIG; if (kvm_phys_is_valid(phys)) @@ -655,7 +669,7 @@ static int stage2_map_walk_table_pre(u64 addr, u64 end, u32 level, if (data->anchor) return 0; - if (!kvm_block_mapping_supported(addr, end, data->phys, level)) + if (!stage2_block_mapping_allowed(addr, end, level, data)) return 0; data->childp = kvm_pte_follow(*ptep, data->mm_ops); @@ -785,6 +799,7 @@ int kvm_pgtable_stage2_map(struct kvm_pgtable *pgt, u64 addr, u64 size, .mmu = pgt->mmu, .memcache = mc, .mm_ops = pgt->mm_ops, + .force_pte = pgt->force_pte_cb && pgt->force_pte_cb(addr, addr + size, prot), }; struct kvm_pgtable_walker walker = { .cb = stage2_map_walker, @@ -816,6 +831,7 @@ int kvm_pgtable_stage2_set_owner(struct kvm_pgtable *pgt, u64 addr, u64 size, .memcache = mc, .mm_ops = pgt->mm_ops, .owner_id = owner_id, + .force_pte = true, }; struct kvm_pgtable_walker walker = { .cb = stage2_map_walker, @@ -1057,9 +1073,11 @@ int kvm_pgtable_stage2_flush(struct kvm_pgtable *pgt, u64 addr, u64 size) return kvm_pgtable_walk(pgt, addr, size, &walker); } -int kvm_pgtable_stage2_init_flags(struct kvm_pgtable *pgt, struct kvm_arch *arch, - struct kvm_pgtable_mm_ops *mm_ops, - enum kvm_pgtable_stage2_flags flags) + +int __kvm_pgtable_stage2_init(struct kvm_pgtable *pgt, struct kvm_arch *arch, + struct kvm_pgtable_mm_ops *mm_ops, + enum kvm_pgtable_stage2_flags flags, + kvm_pgtable_force_pte_cb_t force_pte_cb) { size_t pgd_sz; u64 vtcr = arch->vtcr; @@ -1077,6 +1095,7 @@ int kvm_pgtable_stage2_init_flags(struct kvm_pgtable *pgt, struct kvm_arch *arch pgt->mm_ops = mm_ops; pgt->mmu = &arch->mmu; pgt->flags = flags; + pgt->force_pte_cb = force_pte_cb; /* Ensure zeroed PGD pages are visible to the hardware walker */ dsb(ishst); From patchwork Thu Jul 29 13:28:08 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Quentin Perret X-Patchwork-Id: 12408749 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-17.5 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_ADSP_CUSTOM_MED,DKIM_SIGNED,DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, URIBL_BLOCKED,USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 7A355C4338F for ; Thu, 29 Jul 2021 13:35:09 +0000 (UTC) Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 4C31760F23 for ; Thu, 29 Jul 2021 13:35:09 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.4.1 mail.kernel.org 4C31760F23 Authentication-Results: mail.kernel.org; dmarc=fail (p=reject dis=none) header.from=google.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:Cc:To:From:Subject:References: Mime-Version:Message-Id:In-Reply-To:Date:Reply-To:Content-ID: Content-Description:Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc :Resent-Message-ID:List-Owner; bh=leCnVmrWuLn3lwhJHl+TfKqTlbM4vYpGfyQLPR7+cZ0=; b=LBRTJRqwGJS+Mt6fAzUrgKRhzO PELRlJSnKObgIDZwQFIJGrOXNRylK421hIJVad5a76GHZ+mISfCpr58MTEgaZ/ezjH2lORubkXhPN d2Q5efQ/Nur+qIt5U4atBYwxYhzrwDlD2XktL2NeRJ4guiv82hExvv57YdGqPCZmP592oVVmu9VKN 4I62t004WaQ99mRqZyYazb3hgtMpL+hcop/GBcssM2E1bZOjXYNf7SGe384CnMyrOD1wPRA+OOLas 2LKWKJy8mnJ4iE9klRAeEqmvEC4sYcE00029M2doZSKm4JI6hxlW05IEmGbYdoAsKMKRrOPLkfCK5 2J+vuLBg==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1m96AD-004Gdp-FL; Thu, 29 Jul 2021 13:33:30 +0000 Received: from mail-qv1-xf4a.google.com ([2607:f8b0:4864:20::f4a]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1m965h-004EfR-PK for linux-arm-kernel@lists.infradead.org; Thu, 29 Jul 2021 13:28:52 +0000 Received: by mail-qv1-xf4a.google.com with SMTP id k7-20020ad453c70000b02902f36ca6afdcso3926544qvv.7 for ; Thu, 29 Jul 2021 06:28:48 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=date:in-reply-to:message-id:mime-version:references:subject:from:to :cc; bh=K2yJYY4AwV+vn/rNuC2tG6Gq9ljpo+CbL9CfXBwYhKs=; b=QOeabQvhK+pjySXM7mdcCBpwBPGxHA2Aapue3mH3nglJQ48pdBSi1Gn+iql4C+ZGom yxB9Fd0YJyOA5p4oPp5VddH6E/ae1UId3M3sSiQLeXVEXTAyT8WEuAcHH4S3tIGW4GL7 7wL1UZpRFNTcYPQ5V4zS3cIm55suJij7Ous/EuOVId5PU1TniXtP5Aqb0dVNxvTPxtlj KiaaQa51mTF7lD9V2qlqCFAKM3pn6tq7MWusSXIEem4MarzZ67Wycb9NnvuHlTS06wLf E0b4JNz5c1tJLmgcdHVU3XSzLEhKhe3NqjOJO2gPG+nEK+YdR7Kj2Kw1hj8Xb/tq07w5 5N5A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=K2yJYY4AwV+vn/rNuC2tG6Gq9ljpo+CbL9CfXBwYhKs=; b=ayXqI1ZBwjIpfTuCF7M+nqKikueskQxeay5voXUVg5y7QQvAIU2g8IV5Qya90ZOJio 4b/zIAruU4bhMAhe5tEtcez1cZmEH6uL6TcGgae1btHhSVrUgujdUAL29kNeumjxFda8 r/LegWzxH8pAzUCbhBuQmJm/enVo7sySbT1bBxxVkkB/NVD+byo60IzhUEvwEK9yGHU5 DVq2otpYCnHurHLeoUEPThaCASq1VV0kQZAWfsI+TtMQ4dmc1+zKas2clywvMdv7OKV3 6PYzERrFqfEDXMkFrxA8aTFmzPPXJIa1TOVDTwMhFU0Tc1X/PxliOzq88fIfW9Ent/Xx XV+g== X-Gm-Message-State: AOAM533UDVGb/P9q5GsVT0YH+upoXibpdKspvBIumzGOQIPUQUsY+ien Qour8lK4VeI+n/opqht1sCqAdDQT1U3s X-Google-Smtp-Source: ABdhPJxz1R05cTjKGeAzbyffgsjdEJ6FuTW+MEeZoslgi5BQPfW/okp9D2w7vU3mDjPYNqOPOTSTUIW1Zb/l X-Received: from luke.lon.corp.google.com ([2a00:79e0:d:210:293a:bc89:7514:5218]) (user=qperret job=sendgmr) by 2002:a05:6214:10c8:: with SMTP id r8mr5270070qvs.28.1627565328196; Thu, 29 Jul 2021 06:28:48 -0700 (PDT) Date: Thu, 29 Jul 2021 14:28:08 +0100 In-Reply-To: <20210729132818.4091769-1-qperret@google.com> Message-Id: <20210729132818.4091769-12-qperret@google.com> Mime-Version: 1.0 References: <20210729132818.4091769-1-qperret@google.com> X-Mailer: git-send-email 2.32.0.432.gabb21c7263-goog Subject: [PATCH v3 11/21] KVM: arm64: Allow populating software bits From: Quentin Perret To: maz@kernel.org, james.morse@arm.com, alexandru.elisei@arm.com, suzuki.poulose@arm.com, catalin.marinas@arm.com, will@kernel.org Cc: linux-arm-kernel@lists.infradead.org, kvmarm@lists.cs.columbia.edu, linux-kernel@vger.kernel.org, ardb@kernel.org, qwandor@google.com, tabba@google.com, dbrazdil@google.com, kernel-team@android.com, Quentin Perret X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20210729_062849_866219_A4B592ED X-CRM114-Status: GOOD ( 15.97 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Introduce infrastructure allowing to manipulate software bits in stage-1 and stage-2 page-tables using additional entries in the kvm_pgtable_prot enum. This is heavily inspired by Marc's implementation of a similar feature in the NV patch series, but adapted to allow stage-1 changes as well: https://lore.kernel.org/kvmarm/20210510165920.1913477-56-maz@kernel.org/ Suggested-by: Marc Zyngier Signed-off-by: Quentin Perret --- arch/arm64/include/asm/kvm_pgtable.h | 12 +++++++++++- arch/arm64/kvm/hyp/pgtable.c | 5 +++++ 2 files changed, 16 insertions(+), 1 deletion(-) diff --git a/arch/arm64/include/asm/kvm_pgtable.h b/arch/arm64/include/asm/kvm_pgtable.h index ba7dcade2798..d5ca9b6ce241 100644 --- a/arch/arm64/include/asm/kvm_pgtable.h +++ b/arch/arm64/include/asm/kvm_pgtable.h @@ -121,6 +121,10 @@ enum kvm_pgtable_stage2_flags { * @KVM_PGTABLE_PROT_W: Write permission. * @KVM_PGTABLE_PROT_R: Read permission. * @KVM_PGTABLE_PROT_DEVICE: Device attributes. + * @KVM_PGTABLE_PROT_SW0: Software bit 0. + * @KVM_PGTABLE_PROT_SW1: Software bit 1. + * @KVM_PGTABLE_PROT_SW2: Software bit 2. + * @KVM_PGTABLE_PROT_SW3: Software bit 3. */ enum kvm_pgtable_prot { KVM_PGTABLE_PROT_X = BIT(0), @@ -128,6 +132,11 @@ enum kvm_pgtable_prot { KVM_PGTABLE_PROT_R = BIT(2), KVM_PGTABLE_PROT_DEVICE = BIT(3), + + KVM_PGTABLE_PROT_SW0 = BIT(55), + KVM_PGTABLE_PROT_SW1 = BIT(56), + KVM_PGTABLE_PROT_SW2 = BIT(57), + KVM_PGTABLE_PROT_SW3 = BIT(58), }; #define KVM_PGTABLE_PROT_RW (KVM_PGTABLE_PROT_R | KVM_PGTABLE_PROT_W) @@ -419,7 +428,8 @@ kvm_pte_t kvm_pgtable_stage2_mkold(struct kvm_pgtable *pgt, u64 addr); * If there is a valid, leaf page-table entry used to translate @addr, then * relax the permissions in that entry according to the read, write and * execute permissions specified by @prot. No permissions are removed, and - * TLB invalidation is performed after updating the entry. + * TLB invalidation is performed after updating the entry. Software bits cannot + * be set or cleared using kvm_pgtable_stage2_relax_perms(). * * Return: 0 on success, negative error code on failure. */ diff --git a/arch/arm64/kvm/hyp/pgtable.c b/arch/arm64/kvm/hyp/pgtable.c index bcc02e6e0f62..1915489bb127 100644 --- a/arch/arm64/kvm/hyp/pgtable.c +++ b/arch/arm64/kvm/hyp/pgtable.c @@ -357,6 +357,7 @@ static int hyp_set_prot_attr(enum kvm_pgtable_prot prot, kvm_pte_t *ptep) attr |= FIELD_PREP(KVM_PTE_LEAF_ATTR_LO_S1_AP, ap); attr |= FIELD_PREP(KVM_PTE_LEAF_ATTR_LO_S1_SH, sh); attr |= KVM_PTE_LEAF_ATTR_LO_S1_AF; + attr |= prot & KVM_PTE_LEAF_ATTR_HI_SW; *ptep = attr; return 0; @@ -558,6 +559,7 @@ static int stage2_set_prot_attr(struct kvm_pgtable *pgt, enum kvm_pgtable_prot p attr |= FIELD_PREP(KVM_PTE_LEAF_ATTR_LO_S2_SH, sh); attr |= KVM_PTE_LEAF_ATTR_LO_S2_AF; + attr |= prot & KVM_PTE_LEAF_ATTR_HI_SW; *ptep = attr; return 0; @@ -1025,6 +1027,9 @@ int kvm_pgtable_stage2_relax_perms(struct kvm_pgtable *pgt, u64 addr, u32 level; kvm_pte_t set = 0, clr = 0; + if (prot & KVM_PTE_LEAF_ATTR_HI_SW) + return -EINVAL; + if (prot & KVM_PGTABLE_PROT_R) set |= KVM_PTE_LEAF_ATTR_LO_S2_S2AP_R; From patchwork Thu Jul 29 13:28:09 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Quentin Perret X-Patchwork-Id: 12408753 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-17.5 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_ADSP_CUSTOM_MED,DKIM_SIGNED,DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, URIBL_BLOCKED,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 5F754C4338F for ; Thu, 29 Jul 2021 13:37:13 +0000 (UTC) Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 32C6260F46 for ; Thu, 29 Jul 2021 13:37:13 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.4.1 mail.kernel.org 32C6260F46 Authentication-Results: mail.kernel.org; dmarc=fail (p=reject dis=none) header.from=google.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:Cc:To:From:Subject:References: Mime-Version:Message-Id:In-Reply-To:Date:Reply-To:Content-ID: Content-Description:Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc :Resent-Message-ID:List-Owner; bh=D1idL08oMuq5K/e+08pujny9JbA0nUrWEFz8FMSG3fg=; b=pPUdmtUgSkO4AuY9mL/I4ilUIq WBbJ523qUBP0CCf2yyMEC8SxA4lWZ3+n1a50ozpklfP/FmDP2qbwe+tSJ3SFSfXM2xm9XIjQiJmRb cntTKkc3w15/6MMrtVN9mly1dUQ/yh0/xkGE5ElUecvcX7n4ibi+QjA6VfOPuRwLXaIwWUlR/ZsDg 1KPuMuqei9iF2fibzYFdMPmk3iYaIE7Z/E+uSvAPRekCkhtmOhT+BkhoNNOx0cunIG+uEIDkcthPJ /22zwgCtq30m1vJr+S6OF4Q1Aw4WJCs9VetTUK9IcTqRPk+5pXfsiyVpszyxZN/xCm4NzgFvswXg2 HZjADp+g==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1m96Bl-004HF2-Cw; Thu, 29 Jul 2021 13:35:06 +0000 Received: from mail-wm1-x34a.google.com ([2a00:1450:4864:20::34a]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1m965k-004Eh9-Il for linux-arm-kernel@lists.infradead.org; Thu, 29 Jul 2021 13:28:53 +0000 Received: by mail-wm1-x34a.google.com with SMTP id u14-20020a7bcb0e0000b0290248831d46e4so2003275wmj.6 for ; Thu, 29 Jul 2021 06:28:51 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=date:in-reply-to:message-id:mime-version:references:subject:from:to :cc; bh=H72uhE5VGkDx1jb1MVcWxLaZfSQPWlRzbrWGol5Bg8o=; b=HTncpCcFrfYqsq+B0XVUZSJV3OsWsCbXAq0Nsc6wlmDYOhwALGBrA3jJ41pZG0qqMI BUp7QdvrQdj2zE8BkjdDej0plVEuMnXHWUQxTorC7+RLWrL1Cl5SNZOzSPQYcIPsiBfy PrIxNokt9gtSQCGyTC4CeE1TIB4Vpi45IPhQSXcZ31S+u0qfhpB+xXZJnFGTLgqBW/FK 8j7S1I+GtzmzNFPDXkOtQ4OpR5EBk196h65xM3EPu3FNApwn8htU1h7pYOsraR/C0MHV Et3C8vRAcLob23EWSpeVAA0PxgEvB3ZxKyeGA3rhHcr1KBj7OXeo6C86lTupLKh5WO3I vGjQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=H72uhE5VGkDx1jb1MVcWxLaZfSQPWlRzbrWGol5Bg8o=; b=f74qzreaCnRacTDZNzmdgbj72BgeZVw37TAxnFneqaiEGvx+jf14pGLjunfEv+OV10 d4fHa6PLqwaRDhIgAsJiYbxOxqg43staWvIiC4IVmKxeDsfqhvCh451RXRmcghXxvjSK 7qL62UiI63g4hu6LB7mQ4z3DUrtcyrNU6war8Vvj7XA1ynuJ9uZ+5crSu50qDIS6K2H9 nuyPEx0i+HDUTrpUBeBpBFknGSU1BqxFR1xU99ZbOZ+vsbVs/5gSQDPuWkQlNQG2vyAa nNkcsnMl8+fq+VE99HjwHBxHaemhHBuOrCily8GuDkMdUnl3Po4q9vyrC4HbbkL47jZV fWVQ== X-Gm-Message-State: AOAM533RquhdGlstMvksb/vAIR95FTXvL9sUcKpGHytOmXNTEO3RnTBy BU4jvuuii6wGnMj4b9K9xcPhl9zIXNwg X-Google-Smtp-Source: ABdhPJzn1rlmoNXXKLmPJSHjhbkDAjV7ZnprHUg6PgSi5S1PmeBEhPufv9qLwdFykhgetYoUfULvmrtzd+sc X-Received: from luke.lon.corp.google.com ([2a00:79e0:d:210:293a:bc89:7514:5218]) (user=qperret job=sendgmr) by 2002:a1c:a78a:: with SMTP id q132mr4804653wme.76.1627565330622; Thu, 29 Jul 2021 06:28:50 -0700 (PDT) Date: Thu, 29 Jul 2021 14:28:09 +0100 In-Reply-To: <20210729132818.4091769-1-qperret@google.com> Message-Id: <20210729132818.4091769-13-qperret@google.com> Mime-Version: 1.0 References: <20210729132818.4091769-1-qperret@google.com> X-Mailer: git-send-email 2.32.0.432.gabb21c7263-goog Subject: [PATCH v3 12/21] KVM: arm64: Add helpers to tag shared pages in SW bits From: Quentin Perret To: maz@kernel.org, james.morse@arm.com, alexandru.elisei@arm.com, suzuki.poulose@arm.com, catalin.marinas@arm.com, will@kernel.org Cc: linux-arm-kernel@lists.infradead.org, kvmarm@lists.cs.columbia.edu, linux-kernel@vger.kernel.org, ardb@kernel.org, qwandor@google.com, tabba@google.com, dbrazdil@google.com, kernel-team@android.com, Quentin Perret X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20210729_062852_670808_C174E162 X-CRM114-Status: GOOD ( 12.17 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org We will soon start annotating shared pages in page-tables in nVHE protected mode. Define all the states in which a page can be (owned, shared and owned, shared and borrowed), and provide helpers allowing to convert this into SW bits annotations using the matching prot attributes. Signed-off-by: Quentin Perret Reviewed-by: Fuad Tabba --- arch/arm64/kvm/hyp/include/nvhe/mem_protect.h | 26 +++++++++++++++++++ 1 file changed, 26 insertions(+) diff --git a/arch/arm64/kvm/hyp/include/nvhe/mem_protect.h b/arch/arm64/kvm/hyp/include/nvhe/mem_protect.h index 9c227d87c36d..ae355bfd8c01 100644 --- a/arch/arm64/kvm/hyp/include/nvhe/mem_protect.h +++ b/arch/arm64/kvm/hyp/include/nvhe/mem_protect.h @@ -12,6 +12,32 @@ #include #include +/* + * SW bits 0-1 are reserved to track the memory ownership state of each page: + * 00: The page is owned solely by the page-table owner. + * 01: The page is owned by the page-table owner, but is shared + * with another entity. + * 10: The page is shared with, but not owned by the page-table owner. + * 11: Reserved for future use (lending). + */ +enum pkvm_page_state { + PKVM_PAGE_OWNED = 0ULL, + PKVM_PAGE_SHARED_OWNED = KVM_PGTABLE_PROT_SW0, + PKVM_PAGE_SHARED_BORROWED = KVM_PGTABLE_PROT_SW1, +}; + +#define PKVM_PAGE_STATE_PROT_MASK (KVM_PGTABLE_PROT_SW0 | KVM_PGTABLE_PROT_SW1) +static inline enum kvm_pgtable_prot pkvm_mkstate(enum kvm_pgtable_prot prot, + enum pkvm_page_state state) +{ + return (prot & ~PKVM_PAGE_STATE_PROT_MASK) | state; +} + +static inline enum pkvm_page_state pkvm_getstate(enum kvm_pgtable_prot prot) +{ + return prot & PKVM_PAGE_STATE_PROT_MASK; +} + struct host_kvm { struct kvm_arch arch; struct kvm_pgtable pgt; From patchwork Thu Jul 29 13:28:10 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Quentin Perret X-Patchwork-Id: 12408755 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-17.5 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_ADSP_CUSTOM_MED,DKIM_SIGNED,DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, URIBL_BLOCKED,USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id A6528C4338F for ; Thu, 29 Jul 2021 13:38:27 +0000 (UTC) Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 7601360F48 for ; Thu, 29 Jul 2021 13:38:27 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.4.1 mail.kernel.org 7601360F48 Authentication-Results: mail.kernel.org; dmarc=fail (p=reject dis=none) header.from=google.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:Cc:To:From:Subject:References: Mime-Version:Message-Id:In-Reply-To:Date:Reply-To:Content-ID: Content-Description:Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc :Resent-Message-ID:List-Owner; bh=TkCn5WMaP9r5Li4mweeScclGXbyoSNOZkIX1smOTaBY=; b=Me5PSS8mlaXX/pchnvpi0gwfOP va+JOKPwsDl5x4OlqEGhe7jK9iojBxFWmxVzg0s70KA1LNQks8dp3vnb/VkgI+BQ31RW4S54crd9r 4ZyG53sZxMTjJ5me2G1tMipZigp2SZIObYIXJIcq+W8WRSTHWe5/yrt11HEMe5p1nd5vAWBwTaQSV GCtZtgjpiNxtpEH78xNbfCyCBcliTEcmPtm/DmV1CvOimR3KBRgZdxYEQyRYK0TrXwWk3RbosF8b6 ANrsR9XLHxid1CsyV991Xzx+az3tq+cHx5Uz8GxDnRFKyfssUkCjPsjZ6k/DpsV9gdCAoCiKadB+D eFRa8ZwA==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1m96Cf-004HWy-TT; Thu, 29 Jul 2021 13:36:02 +0000 Received: from mail-qv1-xf4a.google.com ([2607:f8b0:4864:20::f4a]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1m965m-004Ei8-85 for linux-arm-kernel@lists.infradead.org; Thu, 29 Jul 2021 13:28:55 +0000 Received: by mail-qv1-xf4a.google.com with SMTP id m15-20020a0cbf0f0000b02902e60d75210eso3856921qvi.19 for ; Thu, 29 Jul 2021 06:28:53 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=date:in-reply-to:message-id:mime-version:references:subject:from:to :cc; bh=AlIcs6je/4eH5g1VpJMJ6MIaoeGclfxxvTq2IG6FTnE=; b=PojaLSp2J3h7SGHoj/wg3sKvQEJru11qmRDv6TPyUDNP0v2sIgCgjnQcQBfVvjJIBd yvSNbLe1o+nRvt4gE0x+yU5fyEqbr0b5rs+ag4a5W3POPPbpSKs5xgMCz0wnKXhZeNQJ Jz8fp9z+v6v263KlELujgmCZkDPY4LwdYCPTuK3m51ARcYRoysYF4I+3aFi/t2XpYMF3 LZR4+yesCGdVYduYfwUa+43g/OSLzHdoLEJ6HOnuqE+eHQa2gYpDnE7HKgtttm2Q3g6D wMuKjYgK7mmdnQTrjI/+6Qfcz2yvwA+zA0dgaLkl/O6s+UYLBYYi6gfD4pqB6YDAw10Q LWgg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=AlIcs6je/4eH5g1VpJMJ6MIaoeGclfxxvTq2IG6FTnE=; b=kPZGb10bd9DpupbujwtbSvcCLSlP4C7Yy0zrlA1FBkHv8Xbp5cKka7pwiG24HRWAxl I2TRltJSQc1ZuJDSfiolWsOS71782597Sgmnn0y5mZD0I36g8B/VBqnkhdmNMXV11WUt XHYC7rqvgJFrPlmBl1YDrpIwTwG1p2XjWPR38frQzDd+80TnwXhos4/2Niz40nCdm1t6 Dmm7dvhM11EZjzXySHmzhilai6P3HNW2pQajMkep6TSSnijy6ic4UanyXBHTDG60WRu/ 68ziX9uPoqqo3rPb94EZFYuCA3uOLPpvDhSTxfyUkqZS9CfChv8JhNkuSQ4ZgNd65n+g UybA== X-Gm-Message-State: AOAM531g+nsFzXXDweMIF9vlI0wlgkCcbnOD/wC/EiwDcLLKIRIlmqlR e/DNz8SgbiT99lq5Zfkt52Qz748/sSQU X-Google-Smtp-Source: ABdhPJwjclaLvF9NNfuY6mTQNdmoAJ3rTkidxuL1gXdxsvT/IfAhg+ijqY4w89EJuF24aa4xY4D+MGf/w1eo X-Received: from luke.lon.corp.google.com ([2a00:79e0:d:210:293a:bc89:7514:5218]) (user=qperret job=sendgmr) by 2002:a05:6214:20c8:: with SMTP id 8mr1933136qve.4.1627565332854; Thu, 29 Jul 2021 06:28:52 -0700 (PDT) Date: Thu, 29 Jul 2021 14:28:10 +0100 In-Reply-To: <20210729132818.4091769-1-qperret@google.com> Message-Id: <20210729132818.4091769-14-qperret@google.com> Mime-Version: 1.0 References: <20210729132818.4091769-1-qperret@google.com> X-Mailer: git-send-email 2.32.0.432.gabb21c7263-goog Subject: [PATCH v3 13/21] KVM: arm64: Expose host stage-2 manipulation helpers From: Quentin Perret To: maz@kernel.org, james.morse@arm.com, alexandru.elisei@arm.com, suzuki.poulose@arm.com, catalin.marinas@arm.com, will@kernel.org Cc: linux-arm-kernel@lists.infradead.org, kvmarm@lists.cs.columbia.edu, linux-kernel@vger.kernel.org, ardb@kernel.org, qwandor@google.com, tabba@google.com, dbrazdil@google.com, kernel-team@android.com, Quentin Perret X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20210729_062854_355011_2F61BE0F X-CRM114-Status: GOOD ( 12.23 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org We will need to manipulate the host stage-2 page-table from outside mem_protect.c soon. Introduce two functions allowing this, and make them usable to users of mem_protect.h. Signed-off-by: Quentin Perret --- arch/arm64/kvm/hyp/include/nvhe/mem_protect.h | 2 ++ arch/arm64/kvm/hyp/nvhe/mem_protect.c | 17 ++++++++++++++++- 2 files changed, 18 insertions(+), 1 deletion(-) diff --git a/arch/arm64/kvm/hyp/include/nvhe/mem_protect.h b/arch/arm64/kvm/hyp/include/nvhe/mem_protect.h index ae355bfd8c01..47c2a0c51612 100644 --- a/arch/arm64/kvm/hyp/include/nvhe/mem_protect.h +++ b/arch/arm64/kvm/hyp/include/nvhe/mem_protect.h @@ -49,6 +49,8 @@ extern struct host_kvm host_kvm; int __pkvm_prot_finalize(void); int __pkvm_mark_hyp(phys_addr_t start, phys_addr_t end); +int host_stage2_idmap_locked(u64 start, u64 end, enum kvm_pgtable_prot prot); +int host_stage2_set_owner_locked(u64 start, u64 end, u8 owner_id); int kvm_host_prepare_stage2(void *pgt_pool_base); void handle_host_mem_abort(struct kvm_cpu_context *host_ctxt); diff --git a/arch/arm64/kvm/hyp/nvhe/mem_protect.c b/arch/arm64/kvm/hyp/nvhe/mem_protect.c index 70c57d2c3024..a7f6134789e0 100644 --- a/arch/arm64/kvm/hyp/nvhe/mem_protect.c +++ b/arch/arm64/kvm/hyp/nvhe/mem_protect.c @@ -272,6 +272,21 @@ static int host_stage2_adjust_range(u64 addr, struct kvm_mem_range *range) return 0; } +int host_stage2_idmap_locked(u64 start, u64 end, enum kvm_pgtable_prot prot) +{ + hyp_assert_lock_held(&host_kvm.lock); + + return host_stage2_try(__host_stage2_idmap, start, end, prot); +} + +int host_stage2_set_owner_locked(u64 start, u64 end, u8 owner_id) +{ + hyp_assert_lock_held(&host_kvm.lock); + + return host_stage2_try(kvm_pgtable_stage2_set_owner, &host_kvm.pgt, + start, end - start, &host_s2_pool, owner_id); +} + static bool host_stage2_force_pte_cb(u64 addr, u64 end, enum kvm_pgtable_prot prot) { /* @@ -309,7 +324,7 @@ static int host_stage2_idmap(u64 addr) if (ret) goto unlock; - ret = host_stage2_try(__host_stage2_idmap, range.start, range.end, prot); + ret = host_stage2_idmap_locked(range.start, range.end, prot); unlock: hyp_spin_unlock(&host_kvm.lock); From patchwork Thu Jul 29 13:28:11 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Quentin Perret X-Patchwork-Id: 12408757 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-17.5 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_ADSP_CUSTOM_MED,DKIM_SIGNED,DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, URIBL_BLOCKED,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 996A8C4338F for ; Thu, 29 Jul 2021 13:39:44 +0000 (UTC) Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 565906052B for ; Thu, 29 Jul 2021 13:39:44 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.4.1 mail.kernel.org 565906052B Authentication-Results: mail.kernel.org; dmarc=fail (p=reject dis=none) header.from=google.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:Cc:To:From:Subject:References: Mime-Version:Message-Id:In-Reply-To:Date:Reply-To:Content-ID: Content-Description:Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc :Resent-Message-ID:List-Owner; bh=/hZ6B0w7M+gGHpjf0h55W0szE5eDYSNmbqQiIqzYAeQ=; b=EvtNyaCzbKgN/FlcHsFgHZhep+ ykOEsx4xQkKnWanLPEdLRRc9DvzziAdEksXA/hzAoxQOVmKlRhQ0bfledIICfdX878bb9oyWf3OIQ Gfx5C9IqweYEtbqmcfICqcQztdL9SK0Fa2AYoYl4axr6paAkURb97exkyKp9cfEa/zCtW5GQQi/LV 5va4t+o+naoF1PL1C1b5HGDeo5Ph9VVZ123l6BzNQS2hFWgBr4ifeF8vmVtd/Bkoywr6k6lyKEndO 3f9vYrnyJHNSkcz/EjzxzzwoBwmucjXaggVoClE+8U513xdd3/M6T5PLktnglrWowefDj5XqQghRh e7tDOMDQ==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1m96Dp-004Huv-P4; Thu, 29 Jul 2021 13:37:15 +0000 Received: from mail-qv1-xf4a.google.com ([2607:f8b0:4864:20::f4a]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1m965o-004Ej9-Fn for linux-arm-kernel@lists.infradead.org; Thu, 29 Jul 2021 13:28:57 +0000 Received: by mail-qv1-xf4a.google.com with SMTP id t4-20020a05621421a4b02902e2f9404330so3110894qvc.9 for ; Thu, 29 Jul 2021 06:28:55 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=date:in-reply-to:message-id:mime-version:references:subject:from:to :cc; bh=RPFJCEtaWTMugcF+NjceDUD4B85Vp1oeldRispcGqoY=; b=bCde7WqPKlxY/xHp5JNTJjJWBNhOJVIwYFfYXuBL6y9hVFcJKQjvu1Mgb3V+PuXsnX mqBCCgRgi/wqWG8KCgA+CMGVirqrgoaOVCFuzncfBsxc8XcDVkSloOQL2ifClKz0If3Y 1wGjcqXn9IBvEFZrjB/A6wvBdmg9otbA4SPJIe1YnoJj2Zoifvf6c0fZu36QDT+x1w2U MwG1JN/XJryBjI4GbZ9IGY2pInQUGfILqs/7RNeTqPYGYAb8DyV5wZCuO40ax1atjGrB w8RvMOQRQqGwRk0P2fFmfVIf+LG2HiF0+IJYdhgVjWw3xx+9XsPAm+vszSyUj/0eUuYl Xufw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=RPFJCEtaWTMugcF+NjceDUD4B85Vp1oeldRispcGqoY=; b=jluUHi7MIEUEEsnWvTKKbDB0lPqzucyYnbMl+/UTylPa85pSxlZMkB6TKbo+RGj7WX iRj/YJ5CfVU0LMaVZw0fD9kBA2tQJ1RgthAwdZbs8cirAJimKCpkSIfVi8hLsnHqEgCq wyTc2Yx6obUmmuLwbUbJEuORpbBXuMTVm26BBTNAlEseGyhNC55rVI0e8NDg9Ji1lyIM yzsPSCj2B2ZMn4JZtjPVgjfmCWxq5Fyu2+zwJHeXyjEP+z/yVWVGxhU2EcCoXkr0eDs9 jNWDhTXSC7MMi238UIMVXSb5HnR7W7tyi+ttURpY1NfFKNeCXiMtiRW5LYXIukeQ5mwX 0sow== X-Gm-Message-State: AOAM5308zI7sqXuL9NZR+bdg8pie1e2F6e99dtxwEVhQAiUmjVMzSxdS SUoSbfc3U/jPAwk71XP6uRN/V6HokSex X-Google-Smtp-Source: ABdhPJxO1maZxYuMOOkc4lCtsFIycuYzMEuf8no7IwacnRivc5nMk/nvcqWV/4Hs90Z3Rs5fFhvpcPIxnoD0 X-Received: from luke.lon.corp.google.com ([2a00:79e0:d:210:293a:bc89:7514:5218]) (user=qperret job=sendgmr) by 2002:ad4:4a8c:: with SMTP id h12mr5181240qvx.62.1627565334836; Thu, 29 Jul 2021 06:28:54 -0700 (PDT) Date: Thu, 29 Jul 2021 14:28:11 +0100 In-Reply-To: <20210729132818.4091769-1-qperret@google.com> Message-Id: <20210729132818.4091769-15-qperret@google.com> Mime-Version: 1.0 References: <20210729132818.4091769-1-qperret@google.com> X-Mailer: git-send-email 2.32.0.432.gabb21c7263-goog Subject: [PATCH v3 14/21] KVM: arm64: Expose pkvm_hyp_id From: Quentin Perret To: maz@kernel.org, james.morse@arm.com, alexandru.elisei@arm.com, suzuki.poulose@arm.com, catalin.marinas@arm.com, will@kernel.org Cc: linux-arm-kernel@lists.infradead.org, kvmarm@lists.cs.columbia.edu, linux-kernel@vger.kernel.org, ardb@kernel.org, qwandor@google.com, tabba@google.com, dbrazdil@google.com, kernel-team@android.com, Quentin Perret X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20210729_062856_584632_D3A1AE28 X-CRM114-Status: GOOD ( 11.61 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Allow references to the hypervisor's owner id from outside mem_protect.c. Signed-off-by: Quentin Perret --- arch/arm64/kvm/hyp/include/nvhe/mem_protect.h | 2 ++ arch/arm64/kvm/hyp/nvhe/mem_protect.c | 2 +- 2 files changed, 3 insertions(+), 1 deletion(-) diff --git a/arch/arm64/kvm/hyp/include/nvhe/mem_protect.h b/arch/arm64/kvm/hyp/include/nvhe/mem_protect.h index 47c2a0c51612..cc86598654b9 100644 --- a/arch/arm64/kvm/hyp/include/nvhe/mem_protect.h +++ b/arch/arm64/kvm/hyp/include/nvhe/mem_protect.h @@ -46,6 +46,8 @@ struct host_kvm { }; extern struct host_kvm host_kvm; +extern const u8 pkvm_hyp_id; + int __pkvm_prot_finalize(void); int __pkvm_mark_hyp(phys_addr_t start, phys_addr_t end); diff --git a/arch/arm64/kvm/hyp/nvhe/mem_protect.c b/arch/arm64/kvm/hyp/nvhe/mem_protect.c index a7f6134789e0..bb18940c3d12 100644 --- a/arch/arm64/kvm/hyp/nvhe/mem_protect.c +++ b/arch/arm64/kvm/hyp/nvhe/mem_protect.c @@ -31,7 +31,7 @@ static struct hyp_pool host_s2_pool; u64 id_aa64mmfr0_el1_sys_val; u64 id_aa64mmfr1_el1_sys_val; -static const u8 pkvm_hyp_id = 1; +const u8 pkvm_hyp_id = 1; static void *host_s2_zalloc_pages_exact(size_t size) { From patchwork Thu Jul 29 13:28:12 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Quentin Perret X-Patchwork-Id: 12408767 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-17.5 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_ADSP_CUSTOM_MED,DKIM_SIGNED,DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, URIBL_BLOCKED,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 602F8C4338F for ; Thu, 29 Jul 2021 13:41:44 +0000 (UTC) Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 2E59F6052B for ; Thu, 29 Jul 2021 13:41:44 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.4.1 mail.kernel.org 2E59F6052B Authentication-Results: mail.kernel.org; dmarc=fail (p=reject dis=none) header.from=google.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:Cc:To:From:Subject:References: Mime-Version:Message-Id:In-Reply-To:Date:Reply-To:Content-ID: Content-Description:Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc :Resent-Message-ID:List-Owner; bh=qA4LSS0r/TufdNXYRpLEeQ9+bP0IKio9Wt+7Fw/VzOo=; b=EGOX9Or1h4WIQWCHjlg0jKgZDC 8RLmGK1kmoGkk6r4Khv/CWeweVNEYqjYrSZIVibnMeTaM4PciZoJKS/m4r0hN2kuT9q+nlfBwfxSc LcNREeUHGhQ4fbjPkI5iBwsTBnEad4vyne6qHqrAhSkaoqKhhr4P2A5T0yF87Xf+ykPDQRmQUBYH+ iKJZwLhBDkGP8aeKv1Mgk11xB69URlTwQvF2FXdS5+qWeGo0kWMU3e0qZzkHmqxPX/ndgTD8NbHrO +ALfXsIbLTGsrq1UveQIxc+liVs2i3NC6RZYDXv7y6BHJZExyMmnUmBYIGq/ZQTEpL7GQCSPD1PF0 MkpeUV5g==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1m96Fa-004Ibs-Qk; Thu, 29 Jul 2021 13:39:04 +0000 Received: from mail-wm1-x34a.google.com ([2a00:1450:4864:20::34a]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1m965q-004Ejx-Tj for linux-arm-kernel@lists.infradead.org; Thu, 29 Jul 2021 13:29:00 +0000 Received: by mail-wm1-x34a.google.com with SMTP id n10-20020a05600c4f8ab029024eabacb065so821861wmq.2 for ; Thu, 29 Jul 2021 06:28:58 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=date:in-reply-to:message-id:mime-version:references:subject:from:to :cc; bh=OjC24vlLeAeaPgTOLWCqiGMhh2YFKJ1LYBLN6vgVBCE=; b=FUVDUhP/Rgin9C5BP59XMAHUmYRtf8wsXok+/g39vfdccCUCbbIVDOgFIm6+T7feET OomVLcyL10A24g9qFkGVaYhLGSLUP+7LPM2rYyJvJcTC/EXcS3QMcQy9Zjn2OfviE4Vv pqDLpQ9h2b8lvFh1vVNIwNeokffqlvkzEHZ/4KbVuW5P3FYo7Kz/ngnmtxczHYdDl2aY dn8R3JChCBT+YQ6mC/tGYOavge71IRXVQjA3y7zhUwAZ9lCCEAZhdKwSqg682Veyp4Op H6zjmSwxexlMkOT79IpXhHTF8vO1H6IiZx2lHKa9zI3DieJ1lud/goMYV8Lz8Fg4Utho 4szg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=OjC24vlLeAeaPgTOLWCqiGMhh2YFKJ1LYBLN6vgVBCE=; b=fiG1Iy9x0v1/9UxCaMhEqMl9NmboN7QrA1CWpyEshocf5273FYK0TyuxEXxC2zIz2R 8ioXyYeJSNFP8odenGDTV2LL/05vDULTQxsVJ/CLuR+UlUuzOtI4QHjK3+N8N871UZyT drchr6ksjE9umFdGa4v647+udTXk4r19H0ei1OygX8pLNRUQCi+PTPwctE0JoipxoRE+ S5wsTVWjDWO++LSvsQDNoesB3LuXIe0ehSdsXwGrEurwU7vvfsQlV9i8xKLlrxgrWzRp Gbb8ZLuc5V0/TFodNcRmcm13yfd7Uzn1jWqMvC9QY8lZnStGpRWKx/6Hx4hpC2ByrVgM AhtQ== X-Gm-Message-State: AOAM530rqgrgJsO//8Wp7pjv8VZzKBziSYks6IGMDA5XreznmhE9ISFz jwB6WtvTVMOAobykW7wZxaVxLA5VpJv+ X-Google-Smtp-Source: ABdhPJypNTDOJ5B4zDhDCdj5CCbjzdEdEoifMoEeuxe0LOvoyI2vi5S+2I0+5g7DpCtXnyU4MWjdIokkLn73 X-Received: from luke.lon.corp.google.com ([2a00:79e0:d:210:293a:bc89:7514:5218]) (user=qperret job=sendgmr) by 2002:a1c:1b14:: with SMTP id b20mr3965963wmb.170.1627565337101; Thu, 29 Jul 2021 06:28:57 -0700 (PDT) Date: Thu, 29 Jul 2021 14:28:12 +0100 In-Reply-To: <20210729132818.4091769-1-qperret@google.com> Message-Id: <20210729132818.4091769-16-qperret@google.com> Mime-Version: 1.0 References: <20210729132818.4091769-1-qperret@google.com> X-Mailer: git-send-email 2.32.0.432.gabb21c7263-goog Subject: [PATCH v3 15/21] KVM: arm64: Introduce addr_is_memory() From: Quentin Perret To: maz@kernel.org, james.morse@arm.com, alexandru.elisei@arm.com, suzuki.poulose@arm.com, catalin.marinas@arm.com, will@kernel.org Cc: linux-arm-kernel@lists.infradead.org, kvmarm@lists.cs.columbia.edu, linux-kernel@vger.kernel.org, ardb@kernel.org, qwandor@google.com, tabba@google.com, dbrazdil@google.com, kernel-team@android.com, Quentin Perret X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20210729_062859_062980_E04BBDA6 X-CRM114-Status: UNSURE ( 9.69 ) X-CRM114-Notice: Please train this message. X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Introduce a helper usable in nVHE protected mode to check whether a physical address is in a RAM region or not. Signed-off-by: Quentin Perret --- arch/arm64/kvm/hyp/include/nvhe/mem_protect.h | 1 + arch/arm64/kvm/hyp/nvhe/mem_protect.c | 7 +++++++ 2 files changed, 8 insertions(+) diff --git a/arch/arm64/kvm/hyp/include/nvhe/mem_protect.h b/arch/arm64/kvm/hyp/include/nvhe/mem_protect.h index cc86598654b9..5968fbbb3514 100644 --- a/arch/arm64/kvm/hyp/include/nvhe/mem_protect.h +++ b/arch/arm64/kvm/hyp/include/nvhe/mem_protect.h @@ -51,6 +51,7 @@ extern const u8 pkvm_hyp_id; int __pkvm_prot_finalize(void); int __pkvm_mark_hyp(phys_addr_t start, phys_addr_t end); +bool addr_is_memory(phys_addr_t phys); int host_stage2_idmap_locked(u64 start, u64 end, enum kvm_pgtable_prot prot); int host_stage2_set_owner_locked(u64 start, u64 end, u8 owner_id); int kvm_host_prepare_stage2(void *pgt_pool_base); diff --git a/arch/arm64/kvm/hyp/nvhe/mem_protect.c b/arch/arm64/kvm/hyp/nvhe/mem_protect.c index bb18940c3d12..4532f3d55a1a 100644 --- a/arch/arm64/kvm/hyp/nvhe/mem_protect.c +++ b/arch/arm64/kvm/hyp/nvhe/mem_protect.c @@ -196,6 +196,13 @@ static bool find_mem_range(phys_addr_t addr, struct kvm_mem_range *range) return false; } +bool addr_is_memory(phys_addr_t phys) +{ + struct kvm_mem_range range; + + return find_mem_range(phys, &range); +} + static bool range_is_memory(u64 start, u64 end) { struct kvm_mem_range r1, r2; From patchwork Thu Jul 29 13:28:13 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Quentin Perret X-Patchwork-Id: 12408769 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-17.5 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_ADSP_CUSTOM_MED,DKIM_SIGNED,DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, URIBL_BLOCKED,USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id DDF1EC4338F for ; Thu, 29 Jul 2021 13:43:12 +0000 (UTC) Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id AB9236052B for ; Thu, 29 Jul 2021 13:43:12 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.4.1 mail.kernel.org AB9236052B Authentication-Results: mail.kernel.org; dmarc=fail (p=reject dis=none) header.from=google.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:Cc:To:From:Subject:References: Mime-Version:Message-Id:In-Reply-To:Date:Reply-To:Content-ID: Content-Description:Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc :Resent-Message-ID:List-Owner; bh=fivnS/gx4ueDX42AviIl5pw5tDZ7BH+O4Gxf5yhaxtc=; b=mDR0vuK2CDOcHJi9v+SG36QMP2 aNcmnlO6iG75Rsy8knWy5jnXb/CKxrTGAy7qWyvJwOFjXYDQW4dnJCO3dgDvXIwS0E2R5f8F4t/kw nYfvO6SKTxNwAhC6ZqiUniWIGGfLsTY3QKj1nCQuYbY1fW8XL+LouWYX3T7em0ha1Kzieayt6gHoT PTKUjlM9Mynj4habUx/W/EVoMqyO6FV0v3u7Flfsq89byIS5vaD2lLbZsgKFOzsxkRlKpfod1YStF qj9z+j40fivmeUnRhv4ZfPlROjcuQiBb4r1D+Gzp4eghOBTotBd3jeLefry+YYqLZVplApZS3EC1c vIO7MjSA==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1m96HB-004JGD-Au; Thu, 29 Jul 2021 13:40:41 +0000 Received: from mail-ed1-x549.google.com ([2a00:1450:4864:20::549]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1m965t-004Ekg-IV for linux-arm-kernel@lists.infradead.org; Thu, 29 Jul 2021 13:29:03 +0000 Received: by mail-ed1-x549.google.com with SMTP id ay20-20020a0564022034b02903bc515b673dso2952863edb.3 for ; Thu, 29 Jul 2021 06:29:00 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=date:in-reply-to:message-id:mime-version:references:subject:from:to :cc; bh=NSGLd5rP0jEwEBedGkOnIxS9bTOxpLrW9yksOtiI2rw=; b=V/JC2o+aGhrHJS4UXvEuNCHDiEoLg0Y9XUZ8DbpCYMVAaV4Qp4pzYEUbN0+dCmIvz7 ug5TgqNXz91s47rFXRUdqwwXHw4JQuDC4YIRvTaES4qlC6IViW7MlohARYj5vvT0HcXy /5bS8SkpCABb135E8AaU/tE8brFIIdv5fpfYyBW71CPKxptU9BgZs+MYtCqgHg+dBstw duJUKwMMAPYhVeA9CtbAiKMEryWEmXgjNczXNbmQgmcV5olnAQbG+pcpqX2jyZgCv6/G 2PpFBHLl2j4lLRIWTBQgzXUfNbOCBlsqYaTYIFoxKrvw8JAyP3M415pQ3MBK0SqcQDRG qZow== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=NSGLd5rP0jEwEBedGkOnIxS9bTOxpLrW9yksOtiI2rw=; b=kEh2wewuP6PgY25czs+LjccHivM50JV29V36l1C3+y9DceQgfDAqNfgbKY9m1dhY3W h0AUQUaKlNjvsV78akzGGpdEknMPIqBaq2bMyuAxhueJPyvkHTO6kc+a7oinBAYeMzp4 cKvzqR/zYP7JKd6pouO7IH9nD5uemeAPQ3jgdpz3ZvSGG4HufgrpdZplje/BIXQyIq+A pE9d8EhPRFP/Wl4trML6Nd9v3qNeQFvWaIR4B5xXeatB5MSzFFejsJtvPpMpeN3DAvUc opwvcn1aR6wFtjjRiV/SfbFOVlaHt3X/MuU44X3FoJZNRcKrIsG5lUWJjphXQLZwjJSU HmCw== X-Gm-Message-State: AOAM532g1onbNV7IKPSN5MzYsat+a85FU/34cRSYj87zwA02EWOk48OK y3Inzv7A3XCcDDT2NQKDLPtkcE9lSDuR X-Google-Smtp-Source: ABdhPJzB+o9pIY4nGwgIcOv53lmv+Ol4R2FzrR+UHxpEjNw7CJIBQCY1IdszSXZqu7HKk8gJv4G9waEPi+1Y X-Received: from luke.lon.corp.google.com ([2a00:79e0:d:210:293a:bc89:7514:5218]) (user=qperret job=sendgmr) by 2002:a17:906:1701:: with SMTP id c1mr4576882eje.425.1627565339524; Thu, 29 Jul 2021 06:28:59 -0700 (PDT) Date: Thu, 29 Jul 2021 14:28:13 +0100 In-Reply-To: <20210729132818.4091769-1-qperret@google.com> Message-Id: <20210729132818.4091769-17-qperret@google.com> Mime-Version: 1.0 References: <20210729132818.4091769-1-qperret@google.com> X-Mailer: git-send-email 2.32.0.432.gabb21c7263-goog Subject: [PATCH v3 16/21] KVM: arm64: Enable retrieving protections attributes of PTEs From: Quentin Perret To: maz@kernel.org, james.morse@arm.com, alexandru.elisei@arm.com, suzuki.poulose@arm.com, catalin.marinas@arm.com, will@kernel.org Cc: linux-arm-kernel@lists.infradead.org, kvmarm@lists.cs.columbia.edu, linux-kernel@vger.kernel.org, ardb@kernel.org, qwandor@google.com, tabba@google.com, dbrazdil@google.com, kernel-team@android.com, Quentin Perret X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20210729_062901_660028_59C0C3AF X-CRM114-Status: GOOD ( 12.70 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Introduce helper functions in the KVM stage-2 and stage-1 page-table manipulation library allowing to retrieve the enum kvm_pgtable_prot of a PTE. This will be useful to implement custom walkers outside of pgtable.c. Signed-off-by: Quentin Perret --- arch/arm64/include/asm/kvm_pgtable.h | 20 +++++++++++++++ arch/arm64/kvm/hyp/pgtable.c | 37 ++++++++++++++++++++++++++++ 2 files changed, 57 insertions(+) diff --git a/arch/arm64/include/asm/kvm_pgtable.h b/arch/arm64/include/asm/kvm_pgtable.h index d5ca9b6ce241..7ff9f52239ba 100644 --- a/arch/arm64/include/asm/kvm_pgtable.h +++ b/arch/arm64/include/asm/kvm_pgtable.h @@ -505,4 +505,24 @@ int kvm_pgtable_walk(struct kvm_pgtable *pgt, u64 addr, u64 size, */ int kvm_pgtable_get_leaf(struct kvm_pgtable *pgt, u64 addr, kvm_pte_t *ptep, u32 *level); + +/** + * kvm_pgtable_stage2_pte_prot() - Retrieve the protection attributes of a + * stage-2 Page-Table Entry. + * @pte: Page-table entry + * + * Return: protection attributes of the page-table entry in the enum + * kvm_pgtable_prot format. + */ +enum kvm_pgtable_prot kvm_pgtable_stage2_pte_prot(kvm_pte_t pte); + +/** + * kvm_pgtable_hyp_pte_prot() - Retrieve the protection attributes of a stage-1 + * Page-Table Entry. + * @pte: Page-table entry + * + * Return: protection attributes of the page-table entry in the enum + * kvm_pgtable_prot format. + */ +enum kvm_pgtable_prot kvm_pgtable_hyp_pte_prot(kvm_pte_t pte); #endif /* __ARM64_KVM_PGTABLE_H__ */ diff --git a/arch/arm64/kvm/hyp/pgtable.c b/arch/arm64/kvm/hyp/pgtable.c index 1915489bb127..a6eda8f23cb6 100644 --- a/arch/arm64/kvm/hyp/pgtable.c +++ b/arch/arm64/kvm/hyp/pgtable.c @@ -363,6 +363,26 @@ static int hyp_set_prot_attr(enum kvm_pgtable_prot prot, kvm_pte_t *ptep) return 0; } +enum kvm_pgtable_prot kvm_pgtable_hyp_pte_prot(kvm_pte_t pte) +{ + enum kvm_pgtable_prot prot = pte & KVM_PTE_LEAF_ATTR_HI_SW; + u32 ap; + + if (!kvm_pte_valid(pte)) + return prot; + + if (!(pte & KVM_PTE_LEAF_ATTR_HI_S1_XN)) + prot |= KVM_PGTABLE_PROT_X; + + ap = FIELD_GET(KVM_PTE_LEAF_ATTR_LO_S1_AP, pte); + if (ap == KVM_PTE_LEAF_ATTR_LO_S1_AP_RO) + prot |= KVM_PGTABLE_PROT_R; + else if (ap == KVM_PTE_LEAF_ATTR_LO_S1_AP_RW) + prot |= KVM_PGTABLE_PROT_R | KVM_PGTABLE_PROT_W; + + return prot; +} + static bool hyp_pte_needs_update(kvm_pte_t old, kvm_pte_t new) { /* @@ -565,6 +585,23 @@ static int stage2_set_prot_attr(struct kvm_pgtable *pgt, enum kvm_pgtable_prot p return 0; } +enum kvm_pgtable_prot kvm_pgtable_stage2_pte_prot(kvm_pte_t pte) +{ + enum kvm_pgtable_prot prot = pte & KVM_PTE_LEAF_ATTR_HI_SW; + + if (!kvm_pte_valid(pte)) + return prot; + + if (pte & KVM_PTE_LEAF_ATTR_LO_S2_S2AP_R) + prot |= KVM_PGTABLE_PROT_R; + if (pte & KVM_PTE_LEAF_ATTR_LO_S2_S2AP_W) + prot |= KVM_PGTABLE_PROT_W; + if (!(pte & KVM_PTE_LEAF_ATTR_HI_S2_XN)) + prot |= KVM_PGTABLE_PROT_X; + + return prot; +} + static bool stage2_pte_needs_update(kvm_pte_t old, kvm_pte_t new) { if (!kvm_pte_valid(old) || !kvm_pte_valid(new)) From patchwork Thu Jul 29 13:28:14 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Quentin Perret X-Patchwork-Id: 12408773 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-17.5 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_ADSP_CUSTOM_MED,DKIM_SIGNED,DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, URIBL_BLOCKED,USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id A0BADC4338F for ; Thu, 29 Jul 2021 13:45:22 +0000 (UTC) Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 652C26052B for ; Thu, 29 Jul 2021 13:45:22 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.4.1 mail.kernel.org 652C26052B Authentication-Results: mail.kernel.org; dmarc=fail (p=reject dis=none) header.from=google.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:Cc:To:From:Subject:References: Mime-Version:Message-Id:In-Reply-To:Date:Reply-To:Content-ID: Content-Description:Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc :Resent-Message-ID:List-Owner; bh=jvHYtatgh8E76RfiaDQUGI8bsvIVsvXGx5iMhFiTSQM=; b=A22lzFvLmOKdLip4UEfBOp+9Kj vnCwDK2L+bbs06Ciwc/h2GYPjMGMRm5BAoV76eFpSGGxg48bTZOTG+hX3apx9JyWaTw5G1rtxVAi0 7bOLZ1+OSbkyha9FGAbwm89XWmggsnOi8WEZQiovuWvMbwxv4OI2QuX9xaJpbqTjpk3vTzkYvuMMu Pft5NICtsWulte9yjdXK4Ktkb+GkbCu0pu2mdZtSUQp/kISf+Fdw1qm8miqhKBl8sKfRJrw9yrlVw Pq7hEQy5Ms0+kACGjNdt5CLxZ8VTio1DS7DcoWbMtHLjk4HObbFBRyRgjZunkh4gjHxL/q+vmuqaw cFYACDTQ==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1m96JU-004K4B-5K; Thu, 29 Jul 2021 13:43:04 +0000 Received: from mail-qt1-x84a.google.com ([2607:f8b0:4864:20::84a]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1m965z-004ElV-8M for linux-arm-kernel@lists.infradead.org; Thu, 29 Jul 2021 13:29:08 +0000 Received: by mail-qt1-x84a.google.com with SMTP id m22-20020a05622a1196b029026549e62339so2761515qtk.1 for ; Thu, 29 Jul 2021 06:29:02 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=date:in-reply-to:message-id:mime-version:references:subject:from:to :cc; bh=t9d8PqBzyoMiv4uaDaKWmALSR0AbEM5jj3yE0ogaIjo=; b=qdH4TWs2IPN7BH6Qf64dcpE5qjfH2iCeLcZX05rnMNjDWiBnFROcDGWEe/SdMju7x/ ieDbYf7UO1Mq1ELT6YQ0XasLQyPpiRUTXjG3KEShWGZ9JFwQgDi7gkhanF3t14H2Lrym g4bXAIzrDRY7b8rE1N2wpQCdJQ3W8gXHS8E/CnjFZDjpruQc0oXYrRONUVWJjdhikTu5 ir2JnbEZXVpjouYRcMnnxiQPaEo/yNabkdYo1ggwEFlG75vu2TUW02dmBDAgZ7PyOssP 7FFDhwLvfwHe2cXVTR6aKM4U+/2gXXzCmDt3Jp5HQJXGNJ1OLuNREzhPnRzyyh3OtZkf WWqw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=t9d8PqBzyoMiv4uaDaKWmALSR0AbEM5jj3yE0ogaIjo=; b=Qnt0pZKsPh7LDSuPcBp3v4h7FetVohqJwe0e2/1SmPOYjhWiHQ3UPn4TfyxNGuXtXw sDYq1ysW4/O8UaC/njlwlb4DlQ52ScJhNM/HLHxVbkSYI4IkyZT5plt7f/MPXN5fFsj6 Mst4Dwc63la8ieJi3SUp7vrEqFEdpl2nfWRRpbz45HmCDgoRHVnKSJ60LFVT8y5qt2+E OwKhU7k6jQQNujRl8uwA0POC/jECjDeWKv/i/19n29w2/Kt6Ol1Kxo3HpUMTqpC2JrEZ ZwYd6JxKXibDFukwUkUpkNE4TKcg4tedhvz9lq0/29tT927uBdtA0c6DgywCvHkpbSTh FleQ== X-Gm-Message-State: AOAM531WFEYpVS3GUi3l5PBRZCbjIiVPq7EjEXvSFmhsCb2g0Kj4snJV rBYfhpN4HdKJvpdDZRxCdRRgF/f+5iCj X-Google-Smtp-Source: ABdhPJxqCORnrX6VEWjxQJG0CdLIdUAJQytJlymD5vtw/Ugu5Brx5yVux+z3aZpshYRqV+Kxtuu7tq0zDGIm X-Received: from luke.lon.corp.google.com ([2a00:79e0:d:210:293a:bc89:7514:5218]) (user=qperret job=sendgmr) by 2002:ad4:5965:: with SMTP id eq5mr4814986qvb.41.1627565341803; Thu, 29 Jul 2021 06:29:01 -0700 (PDT) Date: Thu, 29 Jul 2021 14:28:14 +0100 In-Reply-To: <20210729132818.4091769-1-qperret@google.com> Message-Id: <20210729132818.4091769-18-qperret@google.com> Mime-Version: 1.0 References: <20210729132818.4091769-1-qperret@google.com> X-Mailer: git-send-email 2.32.0.432.gabb21c7263-goog Subject: [PATCH v3 17/21] KVM: arm64: Mark host bss and rodata section as shared From: Quentin Perret To: maz@kernel.org, james.morse@arm.com, alexandru.elisei@arm.com, suzuki.poulose@arm.com, catalin.marinas@arm.com, will@kernel.org Cc: linux-arm-kernel@lists.infradead.org, kvmarm@lists.cs.columbia.edu, linux-kernel@vger.kernel.org, ardb@kernel.org, qwandor@google.com, tabba@google.com, dbrazdil@google.com, kernel-team@android.com, Quentin Perret X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20210729_062907_372286_60785E28 X-CRM114-Status: GOOD ( 19.28 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org As the hypervisor maps the host's .bss and .rodata sections in its stage-1, make sure to tag them as shared in hyp and host page-tables. But since the hypervisor relies on the presence of these mappings, we cannot let the host in complete control of the memory regions -- it must not unshare or donate them to another entity for example. To prevent this, let's transfer the ownership of those ranges to the hypervisor itself, and share the pages back with the host. Signed-off-by: Quentin Perret --- arch/arm64/kvm/hyp/nvhe/setup.c | 82 +++++++++++++++++++++++++++++---- 1 file changed, 74 insertions(+), 8 deletions(-) diff --git a/arch/arm64/kvm/hyp/nvhe/setup.c b/arch/arm64/kvm/hyp/nvhe/setup.c index 0b574d106519..7f557b264f62 100644 --- a/arch/arm64/kvm/hyp/nvhe/setup.c +++ b/arch/arm64/kvm/hyp/nvhe/setup.c @@ -58,6 +58,7 @@ static int recreate_hyp_mappings(phys_addr_t phys, unsigned long size, { void *start, *end, *virt = hyp_phys_to_virt(phys); unsigned long pgt_size = hyp_s1_pgtable_pages() << PAGE_SHIFT; + enum kvm_pgtable_prot prot; int ret, i; /* Recreate the hyp page-table using the early page allocator */ @@ -83,10 +84,6 @@ static int recreate_hyp_mappings(phys_addr_t phys, unsigned long size, if (ret) return ret; - ret = pkvm_create_mappings(__start_rodata, __end_rodata, PAGE_HYP_RO); - if (ret) - return ret; - ret = pkvm_create_mappings(__hyp_rodata_start, __hyp_rodata_end, PAGE_HYP_RO); if (ret) return ret; @@ -95,10 +92,6 @@ static int recreate_hyp_mappings(phys_addr_t phys, unsigned long size, if (ret) return ret; - ret = pkvm_create_mappings(__hyp_bss_end, __bss_stop, PAGE_HYP_RO); - if (ret) - return ret; - ret = pkvm_create_mappings(virt, virt + size, PAGE_HYP); if (ret) return ret; @@ -117,6 +110,24 @@ static int recreate_hyp_mappings(phys_addr_t phys, unsigned long size, return ret; } + /* + * Map the host's .bss and .rodata sections RO in the hypervisor, but + * transfer the ownerhsip from the host to the hypervisor itself to + * make sure it can't be donated or shared with another entity. + * + * The ownership transtion requires matching changes in the host + * stage-2. This will done later (see finalize_host_mappings()) once the + * hyp_vmemmap is addressable. + */ + prot = pkvm_mkstate(PAGE_HYP_RO, PKVM_PAGE_SHARED_OWNED); + ret = pkvm_create_mappings(__start_rodata, __end_rodata, prot); + if (ret) + return ret; + + ret = pkvm_create_mappings(__hyp_bss_end, __bss_stop, prot); + if (ret) + return ret; + return 0; } @@ -148,6 +159,57 @@ static void hpool_put_page(void *addr) hyp_put_page(&hpool, addr); } +static int finalize_host_mappings_walker(u64 addr, u64 end, u32 level, + kvm_pte_t *ptep, + enum kvm_pgtable_walk_flags flag, + void * const arg) +{ + enum kvm_pgtable_prot prot; + enum pkvm_page_state state; + kvm_pte_t pte = *ptep; + phys_addr_t phys; + + if (!kvm_pte_valid(pte)) + return 0; + + if (level != (KVM_PGTABLE_MAX_LEVELS - 1)) + return -EINVAL; + + phys = kvm_pte_to_phys(pte); + if (!addr_is_memory(phys)) + return 0; + + /* + * Adjust the host stage-2 mappings to match the ownership attributes + * configured in the hypervisor stage-1. + */ + state = pkvm_getstate(kvm_pgtable_hyp_pte_prot(pte)); + switch (state) { + case PKVM_PAGE_OWNED: + return host_stage2_set_owner_locked(phys, phys + PAGE_SIZE, pkvm_hyp_id); + case PKVM_PAGE_SHARED_OWNED: + prot = pkvm_mkstate(KVM_PGTABLE_PROT_RWX, PKVM_PAGE_SHARED_BORROWED); + break; + case PKVM_PAGE_SHARED_BORROWED: + prot = pkvm_mkstate(KVM_PGTABLE_PROT_RWX, PKVM_PAGE_SHARED_OWNED); + break; + default: + return -EINVAL; + } + + return host_stage2_idmap_locked(phys, phys + PAGE_SIZE, prot); +} + +static int finalize_host_mappings(void) +{ + struct kvm_pgtable_walker walker = { + .cb = finalize_host_mappings_walker, + .flags = KVM_PGTABLE_WALK_LEAF, + }; + + return kvm_pgtable_walk(&pkvm_pgtable, 0, BIT(pkvm_pgtable.ia_bits), &walker); +} + void __noreturn __pkvm_init_finalise(void) { struct kvm_host_data *host_data = this_cpu_ptr(&kvm_host_data); @@ -167,6 +229,10 @@ void __noreturn __pkvm_init_finalise(void) if (ret) goto out; + ret = finalize_host_mappings(); + if (ret) + goto out; + pkvm_pgtable_mm_ops = (struct kvm_pgtable_mm_ops) { .zalloc_page = hyp_zalloc_hyp_page, .phys_to_virt = hyp_phys_to_virt, From patchwork Thu Jul 29 13:28:15 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Quentin Perret X-Patchwork-Id: 12408771 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-17.5 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_ADSP_CUSTOM_MED,DKIM_SIGNED,DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, URIBL_BLOCKED,USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id DF8D7C4338F for ; Thu, 29 Jul 2021 13:44:27 +0000 (UTC) Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id B00646052B for ; Thu, 29 Jul 2021 13:44:27 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.4.1 mail.kernel.org B00646052B Authentication-Results: mail.kernel.org; dmarc=fail (p=reject dis=none) header.from=google.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:Cc:To:From:Subject:References: Mime-Version:Message-Id:In-Reply-To:Date:Reply-To:Content-ID: Content-Description:Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc :Resent-Message-ID:List-Owner; bh=vEOoMatxAxqELQDBzlR9IRa51weITYPWHBacDZ9JTA8=; b=iDw6gP6KVapDMsy77k5u74185R XnIYrzUaJKFtdJDg3EnSnJ9wfhZA4j+pPUU3jePthZmN9lURbCFtHpPYWc6H9G6wQ/KUpDNhL4CB+ h4aZk9r+GkIVkyFdrJe7tx3QrWo+UelrlzAm64XGVFJFuE22Y6XJXkhSkJkqw7k+ozIQyZ3gWqzzF Elg+jyOsfBk/LaiIR9A/LEnIlVdhrOa59VzFsiOqVpMIkaAsbTWqIB35gCDka4xBX3yctoFzohSa1 U1B1PvZVYKy19sxzTdd5CimHmVBuRvljVfhc6F8VtpagIIQaRCew1Xt7q4TKCY0Y6VOwtTfvetpkb HnlPUFAw==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1m96IX-004Jlv-Um; Thu, 29 Jul 2021 13:42:06 +0000 Received: from mail-qk1-x74a.google.com ([2607:f8b0:4864:20::74a]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1m965x-004Em9-JA for linux-arm-kernel@lists.infradead.org; Thu, 29 Jul 2021 13:29:07 +0000 Received: by mail-qk1-x74a.google.com with SMTP id c3-20020a05620a0ce3b02903b8eff05707so3226167qkj.5 for ; Thu, 29 Jul 2021 06:29:04 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=date:in-reply-to:message-id:mime-version:references:subject:from:to :cc; bh=A6q+k/2f/nOWcbFbtJvlk6wu0j4PPp7j4DV+/ij7+Zw=; b=U+oEmrVLnRO5V5rONRO0VKkB4yMJ0XwNC+Wadxfk9wVCif0pHoVVmUyACeRqDl5cT8 iQtuPQbYUIUYzlz/k/h7LrQi6rNfsMDLFZDTD7w1BxGSkHw2M2YPTBdizP8VYY8URChE GdIKsr9cTjF9zrFWvoVXioJeyRFqpLWSjXwOIfqsXN6g22j11Xpdy/Rtrec4jMivVFFx T9Xg2/+/lnugsqkF2REruk8Zj1inz37KO5rKfbqRo0dfJDJnCc4nWiugczgDYzxcRI6o an4yLEw/JNRfUQDFYtsJ3a0nLfJ3RJ5LuT6H38+qCUbAS9I53lQV+mIA26I33QdTRd9j asrQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=A6q+k/2f/nOWcbFbtJvlk6wu0j4PPp7j4DV+/ij7+Zw=; b=DrQYkJSfS5yqfJDlQEmt9zK7jucmiX4j/G/haVRdGrb84ZnBERekGS1vsO3OChXC4A nm4CGatuC3lT9b3SLBlx+J9ZB0QW7saDT45HxrdZXY9YSfKVt+PeNsPORgIwXopGl0Mj UWJ1l6W55a/B04OrBPynLW+8l6Rswi8ZAkPN/XtzmLEgSnjeZXuZpiRPxwh41ApS3qBd mz3XI1z6GRGw+tghcvwGF9cnevD6N+YaR9yFe93KSrUPh1492F/1sruX5B6VcfSEbWRE o3yo2rtJ4F9yqQIVnXLbwzXFreS0byzXVLbIRKUT0QZBixeIkoYk54QOmvWIusm1Qb53 TvBA== X-Gm-Message-State: AOAM533muhfcD/W1uBiZgDkvmb76BtbKi2McvCZ5gL9ch1eEu2ICP0tz VPVzaCjI8ult1oRcr77pSEJzefNH5TAi X-Google-Smtp-Source: ABdhPJwMN8hM+c5sxer4Dc5eWhx+y84XlkDTVvbjFRmJISZp4KIimDXCzxS3TTTTLggakeATNucSln2y70gM X-Received: from luke.lon.corp.google.com ([2a00:79e0:d:210:293a:bc89:7514:5218]) (user=qperret job=sendgmr) by 2002:a05:6214:d1:: with SMTP id f17mr5275922qvs.12.1627565343999; Thu, 29 Jul 2021 06:29:03 -0700 (PDT) Date: Thu, 29 Jul 2021 14:28:15 +0100 In-Reply-To: <20210729132818.4091769-1-qperret@google.com> Message-Id: <20210729132818.4091769-19-qperret@google.com> Mime-Version: 1.0 References: <20210729132818.4091769-1-qperret@google.com> X-Mailer: git-send-email 2.32.0.432.gabb21c7263-goog Subject: [PATCH v3 18/21] KVM: arm64: Remove __pkvm_mark_hyp From: Quentin Perret To: maz@kernel.org, james.morse@arm.com, alexandru.elisei@arm.com, suzuki.poulose@arm.com, catalin.marinas@arm.com, will@kernel.org Cc: linux-arm-kernel@lists.infradead.org, kvmarm@lists.cs.columbia.edu, linux-kernel@vger.kernel.org, ardb@kernel.org, qwandor@google.com, tabba@google.com, dbrazdil@google.com, kernel-team@android.com, Quentin Perret X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20210729_062905_697137_D2116AFD X-CRM114-Status: GOOD ( 15.56 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Now that we mark memory owned by the hypervisor in the host stage-2 during __pkvm_init(), we no longer need to rely on the host to explicitly mark the hyp sections later on. Remove the __pkvm_mark_hyp() hypercall altogether. Signed-off-by: Quentin Perret --- arch/arm64/include/asm/kvm_asm.h | 3 +- arch/arm64/kvm/arm.c | 46 ------------------- arch/arm64/kvm/hyp/include/nvhe/mem_protect.h | 1 - arch/arm64/kvm/hyp/nvhe/hyp-main.c | 9 ---- arch/arm64/kvm/hyp/nvhe/mem_protect.c | 19 -------- 5 files changed, 1 insertion(+), 77 deletions(-) diff --git a/arch/arm64/include/asm/kvm_asm.h b/arch/arm64/include/asm/kvm_asm.h index 9f0bf2109be7..432a9ea1f02e 100644 --- a/arch/arm64/include/asm/kvm_asm.h +++ b/arch/arm64/include/asm/kvm_asm.h @@ -63,8 +63,7 @@ #define __KVM_HOST_SMCCC_FUNC___pkvm_create_private_mapping 17 #define __KVM_HOST_SMCCC_FUNC___pkvm_cpu_set_vector 18 #define __KVM_HOST_SMCCC_FUNC___pkvm_prot_finalize 19 -#define __KVM_HOST_SMCCC_FUNC___pkvm_mark_hyp 20 -#define __KVM_HOST_SMCCC_FUNC___kvm_adjust_pc 21 +#define __KVM_HOST_SMCCC_FUNC___kvm_adjust_pc 20 #ifndef __ASSEMBLY__ diff --git a/arch/arm64/kvm/arm.c b/arch/arm64/kvm/arm.c index e9a2b8f27792..2f378482471b 100644 --- a/arch/arm64/kvm/arm.c +++ b/arch/arm64/kvm/arm.c @@ -1954,57 +1954,11 @@ static void _kvm_host_prot_finalize(void *discard) WARN_ON(kvm_call_hyp_nvhe(__pkvm_prot_finalize)); } -static inline int pkvm_mark_hyp(phys_addr_t start, phys_addr_t end) -{ - return kvm_call_hyp_nvhe(__pkvm_mark_hyp, start, end); -} - -#define pkvm_mark_hyp_section(__section) \ - pkvm_mark_hyp(__pa_symbol(__section##_start), \ - __pa_symbol(__section##_end)) - static int finalize_hyp_mode(void) { - int cpu, ret; - if (!is_protected_kvm_enabled()) return 0; - ret = pkvm_mark_hyp_section(__hyp_idmap_text); - if (ret) - return ret; - - ret = pkvm_mark_hyp_section(__hyp_text); - if (ret) - return ret; - - ret = pkvm_mark_hyp_section(__hyp_rodata); - if (ret) - return ret; - - ret = pkvm_mark_hyp_section(__hyp_bss); - if (ret) - return ret; - - ret = pkvm_mark_hyp(hyp_mem_base, hyp_mem_base + hyp_mem_size); - if (ret) - return ret; - - for_each_possible_cpu(cpu) { - phys_addr_t start = virt_to_phys((void *)kvm_arm_hyp_percpu_base[cpu]); - phys_addr_t end = start + (PAGE_SIZE << nvhe_percpu_order()); - - ret = pkvm_mark_hyp(start, end); - if (ret) - return ret; - - start = virt_to_phys((void *)per_cpu(kvm_arm_hyp_stack_page, cpu)); - end = start + PAGE_SIZE; - ret = pkvm_mark_hyp(start, end); - if (ret) - return ret; - } - /* * Flip the static key upfront as that may no longer be possible * once the host stage 2 is installed. diff --git a/arch/arm64/kvm/hyp/include/nvhe/mem_protect.h b/arch/arm64/kvm/hyp/include/nvhe/mem_protect.h index 5968fbbb3514..7ce36fbf5158 100644 --- a/arch/arm64/kvm/hyp/include/nvhe/mem_protect.h +++ b/arch/arm64/kvm/hyp/include/nvhe/mem_protect.h @@ -49,7 +49,6 @@ extern struct host_kvm host_kvm; extern const u8 pkvm_hyp_id; int __pkvm_prot_finalize(void); -int __pkvm_mark_hyp(phys_addr_t start, phys_addr_t end); bool addr_is_memory(phys_addr_t phys); int host_stage2_idmap_locked(u64 start, u64 end, enum kvm_pgtable_prot prot); diff --git a/arch/arm64/kvm/hyp/nvhe/hyp-main.c b/arch/arm64/kvm/hyp/nvhe/hyp-main.c index 1632f001f4ed..7900d5b66ba3 100644 --- a/arch/arm64/kvm/hyp/nvhe/hyp-main.c +++ b/arch/arm64/kvm/hyp/nvhe/hyp-main.c @@ -163,14 +163,6 @@ static void handle___pkvm_prot_finalize(struct kvm_cpu_context *host_ctxt) { cpu_reg(host_ctxt, 1) = __pkvm_prot_finalize(); } - -static void handle___pkvm_mark_hyp(struct kvm_cpu_context *host_ctxt) -{ - DECLARE_REG(phys_addr_t, start, host_ctxt, 1); - DECLARE_REG(phys_addr_t, end, host_ctxt, 2); - - cpu_reg(host_ctxt, 1) = __pkvm_mark_hyp(start, end); -} typedef void (*hcall_t)(struct kvm_cpu_context *); #define HANDLE_FUNC(x) [__KVM_HOST_SMCCC_FUNC_##x] = (hcall_t)handle_##x @@ -196,7 +188,6 @@ static const hcall_t host_hcall[] = { HANDLE_FUNC(__pkvm_create_mappings), HANDLE_FUNC(__pkvm_create_private_mapping), HANDLE_FUNC(__pkvm_prot_finalize), - HANDLE_FUNC(__pkvm_mark_hyp), }; static void handle_host_hcall(struct kvm_cpu_context *host_ctxt) diff --git a/arch/arm64/kvm/hyp/nvhe/mem_protect.c b/arch/arm64/kvm/hyp/nvhe/mem_protect.c index 4532f3d55a1a..0ccea58df7e0 100644 --- a/arch/arm64/kvm/hyp/nvhe/mem_protect.c +++ b/arch/arm64/kvm/hyp/nvhe/mem_protect.c @@ -338,25 +338,6 @@ static int host_stage2_idmap(u64 addr) return ret; } -int __pkvm_mark_hyp(phys_addr_t start, phys_addr_t end) -{ - int ret; - - /* - * host_stage2_unmap_dev_all() currently relies on MMIO mappings being - * non-persistent, so don't allow changing page ownership in MMIO range. - */ - if (!range_is_memory(start, end)) - return -EINVAL; - - hyp_spin_lock(&host_kvm.lock); - ret = host_stage2_try(kvm_pgtable_stage2_set_owner, &host_kvm.pgt, - start, end - start, &host_s2_pool, pkvm_hyp_id); - hyp_spin_unlock(&host_kvm.lock); - - return ret != -EAGAIN ? ret : 0; -} - void handle_host_mem_abort(struct kvm_cpu_context *host_ctxt) { struct kvm_vcpu_fault_info fault; From patchwork Thu Jul 29 13:28:16 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Quentin Perret X-Patchwork-Id: 12408775 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-17.5 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_ADSP_CUSTOM_MED,DKIM_SIGNED,DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, URIBL_BLOCKED,USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id BCAE6C4338F for ; Thu, 29 Jul 2021 13:46:29 +0000 (UTC) Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 83F5760EB5 for ; Thu, 29 Jul 2021 13:46:29 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.4.1 mail.kernel.org 83F5760EB5 Authentication-Results: mail.kernel.org; dmarc=fail (p=reject dis=none) header.from=google.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:Cc:To:From:Subject:References: Mime-Version:Message-Id:In-Reply-To:Date:Reply-To:Content-ID: Content-Description:Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc :Resent-Message-ID:List-Owner; bh=8aco4kGTWzD5MwgAKZxpS/mQGkm+eSj67kl71t+7DSY=; b=KII4dLyRPdc6tz9BMPlBzUWCIv WymdrTg0sVj7U9SOuDr8hdSq4rygk7gdEMONRsQyPcuosVXswNlX36NAR3vH+5fHh1gNxqH7s8Oof NYHxB8m4yvycBkWWf/JIXgiuF8xmwEr+/C2RA1ZnLoWOOVcJEWhc1CjPmWcWGaT9x3GOZIXVmpqFU Vd6G4153VT6bdr+oWyBhqYRQ0Jc0ehHobRMG2nORdWgvW8ISxix3eVQVSPxQ1vmXUgtPsUPpP5//F mjGSHbaqZ+gtzHivKmlcf1k7wc6uqDSlIHNE4idMmCArsN/rZZKfmQQk6DPlwk+3rhJq0Kj2adPK7 WOWGJg5w==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1m96KW-004KPV-RJ; Thu, 29 Jul 2021 13:44:09 +0000 Received: from mail-qt1-x84a.google.com ([2607:f8b0:4864:20::84a]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1m965z-004Emj-Ox for linux-arm-kernel@lists.infradead.org; Thu, 29 Jul 2021 13:29:09 +0000 Received: by mail-qt1-x84a.google.com with SMTP id t35-20020a05622a1823b02902647b518455so2758734qtc.3 for ; Thu, 29 Jul 2021 06:29:06 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=date:in-reply-to:message-id:mime-version:references:subject:from:to :cc; bh=PD2lJrK2uUAJidILwsX5pR8ZyvNUyZDv1OnaXJdJLhM=; b=mUuZZtK9nPtQzaSeRR1efvqBrUoJ0pqBZlk3FM19wBDIbLOvb8fBAK1zLHnHmRvgG4 9+m8SNZemfJ5wjNL0ad4SiPyjYjEBlLoS8Tyn4uwj2sswKXw+5b34Cpk+2cNl0zxU9i2 hb1UtR4PqkuvDCfrkQgC0MmfZumGrmMdvx63kJ33dQX7N17wUy5Vt6SMNZbQpHP5yLox bd8gesl5DHYmN7n9sV5lvWk1BJ0SWoWsa969E0R98/3453gA3Yw8Lno5XA7wFKvbI4kS Ku3g9VDc96QVgpM4vZGdkPZ0QH5KCfDWuAHB2CAaVZfb6Aalj1LyRfmpbi6Z8uylubmZ KXnA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=PD2lJrK2uUAJidILwsX5pR8ZyvNUyZDv1OnaXJdJLhM=; b=nhMRDQG+oK01WCl5Y/tmqz6ewiiaKtBCqBLQUdW7j8gHrn+f0TGHQbl8+7YxE1ITOe 8LsGb4EDSHp7hyKOJDDiQsL4Vu+lztcL2l51zl5stLwuUTiWB4TUC1ipJNIEcJxJ1eah QGhetutKRfD8hvq1QUeioXD45avlABrGpJYxPKxuVQtZVEW4NF+sYZgXwTqnztVS99L7 6lb9t9WDApXJURJgZ3aydKGS7l3EfGq0KLkA0VXj7nG+Qjp4Oq53pyN3wjCEa9HPGLpt uYJ7TGwVKOcmwOhNUcRRXnmnhsW+4rA1PcFaW6dVce/UQwMs6Vp5eAOZWzFkdIMpVgGG yWRw== X-Gm-Message-State: AOAM530/13p9ymcJde5RalCK8Z3/+LmL+jhGh55vdGCB0orxC1ZUVVrp 2rgMtRMSeej4Tc5SiCtDynfbw0XLE+fb X-Google-Smtp-Source: ABdhPJz10hpfq0SGJNMP70xRlPb0dobUvsFM7yqmK2aN+qLJctMkUjDYhRjEE2aZ3LvI5a9WzZnTi+0sHxt+ X-Received: from luke.lon.corp.google.com ([2a00:79e0:d:210:293a:bc89:7514:5218]) (user=qperret job=sendgmr) by 2002:a05:6214:10e6:: with SMTP id q6mr4966976qvt.11.1627565346131; Thu, 29 Jul 2021 06:29:06 -0700 (PDT) Date: Thu, 29 Jul 2021 14:28:16 +0100 In-Reply-To: <20210729132818.4091769-1-qperret@google.com> Message-Id: <20210729132818.4091769-20-qperret@google.com> Mime-Version: 1.0 References: <20210729132818.4091769-1-qperret@google.com> X-Mailer: git-send-email 2.32.0.432.gabb21c7263-goog Subject: [PATCH v3 19/21] KVM: arm64: Refactor protected nVHE stage-1 locking From: Quentin Perret To: maz@kernel.org, james.morse@arm.com, alexandru.elisei@arm.com, suzuki.poulose@arm.com, catalin.marinas@arm.com, will@kernel.org Cc: linux-arm-kernel@lists.infradead.org, kvmarm@lists.cs.columbia.edu, linux-kernel@vger.kernel.org, ardb@kernel.org, qwandor@google.com, tabba@google.com, dbrazdil@google.com, kernel-team@android.com, Quentin Perret X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20210729_062907_886136_FBD6A881 X-CRM114-Status: GOOD ( 13.25 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Refactor the hypervisor stage-1 locking in nVHE protected mode to expose a new pkvm_create_mappings_locked() function. This will be used in later patches to allow walking and changing the hypervisor stage-1 without releasing the lock. Signed-off-by: Quentin Perret --- arch/arm64/kvm/hyp/include/nvhe/mm.h | 1 + arch/arm64/kvm/hyp/nvhe/mm.c | 18 ++++++++++++++++-- 2 files changed, 17 insertions(+), 2 deletions(-) diff --git a/arch/arm64/kvm/hyp/include/nvhe/mm.h b/arch/arm64/kvm/hyp/include/nvhe/mm.h index 8ec3a5a7744b..c76d7136ed9b 100644 --- a/arch/arm64/kvm/hyp/include/nvhe/mm.h +++ b/arch/arm64/kvm/hyp/include/nvhe/mm.h @@ -23,6 +23,7 @@ int hyp_map_vectors(void); int hyp_back_vmemmap(phys_addr_t phys, unsigned long size, phys_addr_t back); int pkvm_cpu_set_vector(enum arm64_hyp_spectre_vector slot); int pkvm_create_mappings(void *from, void *to, enum kvm_pgtable_prot prot); +int pkvm_create_mappings_locked(void *from, void *to, enum kvm_pgtable_prot prot); int __pkvm_create_mappings(unsigned long start, unsigned long size, unsigned long phys, enum kvm_pgtable_prot prot); unsigned long __pkvm_create_private_mapping(phys_addr_t phys, size_t size, diff --git a/arch/arm64/kvm/hyp/nvhe/mm.c b/arch/arm64/kvm/hyp/nvhe/mm.c index a8efdf0f9003..6fbe8e8030f6 100644 --- a/arch/arm64/kvm/hyp/nvhe/mm.c +++ b/arch/arm64/kvm/hyp/nvhe/mm.c @@ -67,13 +67,15 @@ unsigned long __pkvm_create_private_mapping(phys_addr_t phys, size_t size, return addr; } -int pkvm_create_mappings(void *from, void *to, enum kvm_pgtable_prot prot) +int pkvm_create_mappings_locked(void *from, void *to, enum kvm_pgtable_prot prot) { unsigned long start = (unsigned long)from; unsigned long end = (unsigned long)to; unsigned long virt_addr; phys_addr_t phys; + hyp_assert_lock_held(&pkvm_pgd_lock); + start = start & PAGE_MASK; end = PAGE_ALIGN(end); @@ -81,7 +83,8 @@ int pkvm_create_mappings(void *from, void *to, enum kvm_pgtable_prot prot) int err; phys = hyp_virt_to_phys((void *)virt_addr); - err = __pkvm_create_mappings(virt_addr, PAGE_SIZE, phys, prot); + err = kvm_pgtable_hyp_map(&pkvm_pgtable, virt_addr, PAGE_SIZE, + phys, prot); if (err) return err; } @@ -89,6 +92,17 @@ int pkvm_create_mappings(void *from, void *to, enum kvm_pgtable_prot prot) return 0; } +int pkvm_create_mappings(void *from, void *to, enum kvm_pgtable_prot prot) +{ + int ret; + + hyp_spin_lock(&pkvm_pgd_lock); + ret = pkvm_create_mappings_locked(from, to, prot); + hyp_spin_unlock(&pkvm_pgd_lock); + + return ret; +} + int hyp_back_vmemmap(phys_addr_t phys, unsigned long size, phys_addr_t back) { unsigned long start, end; From patchwork Thu Jul 29 13:28:17 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Quentin Perret X-Patchwork-Id: 12408777 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-17.5 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_ADSP_CUSTOM_MED,DKIM_SIGNED,DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, URIBL_BLOCKED,USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 8D19FC4338F for ; Thu, 29 Jul 2021 13:48:13 +0000 (UTC) Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 5724660527 for ; Thu, 29 Jul 2021 13:48:13 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.4.1 mail.kernel.org 5724660527 Authentication-Results: mail.kernel.org; dmarc=fail (p=reject dis=none) header.from=google.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:Cc:To:From:Subject:References: Mime-Version:Message-Id:In-Reply-To:Date:Reply-To:Content-ID: Content-Description:Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc :Resent-Message-ID:List-Owner; bh=a5BpUeVo+rbrtSk9DIGEud7WiqF8SUlW5pOhonTlxG8=; b=QxooYAaOw96orX1vYZFWSHzs+D 0LJ5FW2eNybfFXcgwd7EtIiNfr0AYpxg0Ee/4aPdLd1HrRgwZJ9sEcGUKWPl1vbdi+a5ag6PMd/vH /9PUeXDVvDT4prJGxBtRIcti+Us0e9SKKX9fiqfPW9HmGWBYQkywo/sltiTvoSB7yTiBcNUpeetQV flsnQVoSIKoKCs2aJ6iA55tUA/FcWWIfQkYbgFMNHWWzxPAI5H99uxgNweS4TM9JPMm7P0G+344G3 sF5GbMynq4/LHWq/2gWGv6qVXEyxXjfqkXKo6t2fl+5jYjWhgG/GAxsxYgIbBFLhAwEyN+EN6n4ak kSSvluHA==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1m96Lf-004KmE-CW; Thu, 29 Jul 2021 13:45:21 +0000 Received: from mail-qk1-x749.google.com ([2607:f8b0:4864:20::749]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1m9662-004Enx-Dj for linux-arm-kernel@lists.infradead.org; Thu, 29 Jul 2021 13:29:12 +0000 Received: by mail-qk1-x749.google.com with SMTP id p123-20020a378d810000b02903ad5730c883so3779000qkd.22 for ; Thu, 29 Jul 2021 06:29:09 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=date:in-reply-to:message-id:mime-version:references:subject:from:to :cc; bh=U+etsjhdZysbfpN5f3aQijrR79iQpOJUR6haEZ5h+Dc=; b=dnpblcLgQZnSwKJDXnlhTevo3wgyttV+xg+nyZWanPpfxhC4upVCtIQV0mHz6w1CAj hjwevrpQQrl8bQgXQ2iO1FXCA6NxRw4CQBgJlfZrBujr/66uAdg7p86UebuJb+z/jtcR 3kJwj4O78VJOdidSVW7piFyY2UIxTPTAedzdpd/ZmmJtrxA5YahdOTeDBXjY4QF+Uq4t vIhBNZ0Fd/6RRMQtDl5yApYFlSAwZzdSsCWjmUbZt1uD4Wc5yfO/OYP7fFR8fRilXPdI yH1P1pAQobQVFlqYmd2axU9R/pgarGz6c70tlKUxdyKrDDJTdyWgl5qHfXMjTOTOz+0b v5AQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=U+etsjhdZysbfpN5f3aQijrR79iQpOJUR6haEZ5h+Dc=; b=AIHQy6Qk0TCsD13GmfArXNwXqu9/8wwzeyK976lPrRcKprUIS+xh5UHPJpztAZhYhK IdPhSVa19tJzBSex9qQYzhmVmcBuQSduCuLjE3NutMcUlQP34jj9M2ltvGaE3AxRaKWA zLWSq4EJwdOy/JHKTtRQQNSHb/Avkc6zukK5Mh6VGYuJV/WhAqZq8WBWaE0hsO8t7eXc ZS8sSvFJmKF7PLww5JCzWgytFT5gSgVP6rvsch9S/fLzDiXLugqis7KC+hqP/hFcZaTI rwKxo2vHS7QpH0aijmCj/WeyxJg6366ncopyRjZCjnKOWnEuCGVjyYZF6UUurz/JijlC hRUw== X-Gm-Message-State: AOAM5307O1xx7skGQy7ikzAVd7aK7ZPqh3fr6GqNyNRunJlVFqymkpVe x56rVt2zykWbO6heTRAFRCbrFUpM3YTT X-Google-Smtp-Source: ABdhPJzxZBGJGjuhPUpNeQEO1dRW0w+ioiHAo4VDnESn/2eleOJphVSIXacUC11VE1YgRvPhF+yioMYrl3eH X-Received: from luke.lon.corp.google.com ([2a00:79e0:d:210:293a:bc89:7514:5218]) (user=qperret job=sendgmr) by 2002:a0c:f14f:: with SMTP id y15mr5369567qvl.12.1627565348412; Thu, 29 Jul 2021 06:29:08 -0700 (PDT) Date: Thu, 29 Jul 2021 14:28:17 +0100 In-Reply-To: <20210729132818.4091769-1-qperret@google.com> Message-Id: <20210729132818.4091769-21-qperret@google.com> Mime-Version: 1.0 References: <20210729132818.4091769-1-qperret@google.com> X-Mailer: git-send-email 2.32.0.432.gabb21c7263-goog Subject: [PATCH v3 20/21] KVM: arm64: Restrict EL2 stage-1 changes in protected mode From: Quentin Perret To: maz@kernel.org, james.morse@arm.com, alexandru.elisei@arm.com, suzuki.poulose@arm.com, catalin.marinas@arm.com, will@kernel.org Cc: linux-arm-kernel@lists.infradead.org, kvmarm@lists.cs.columbia.edu, linux-kernel@vger.kernel.org, ardb@kernel.org, qwandor@google.com, tabba@google.com, dbrazdil@google.com, kernel-team@android.com, Quentin Perret X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20210729_062910_559675_09F688B6 X-CRM114-Status: GOOD ( 21.93 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org The host kernel is currently able to change EL2 stage-1 mappings without restrictions thanks to the __pkvm_create_mappings() hypercall. But in a world where the host is no longer part of the TCB, this clearly poses a problem. To fix this, introduce a new hypercall to allow the host to share a physical memory page with the hypervisor, and remove the __pkvm_create_mappings() variant. The new hypercall implements ownership and permission checks before allowing the sharing operation, and it annotates the shared page in the hypervisor stage-1 and host stage-2 page-tables. Signed-off-by: Quentin Perret --- arch/arm64/include/asm/kvm_asm.h | 2 +- arch/arm64/kvm/hyp/include/nvhe/mem_protect.h | 1 + arch/arm64/kvm/hyp/nvhe/hyp-main.c | 11 +-- arch/arm64/kvm/hyp/nvhe/mem_protect.c | 89 +++++++++++++++++++ arch/arm64/kvm/mmu.c | 28 +++++- 5 files changed, 119 insertions(+), 12 deletions(-) diff --git a/arch/arm64/include/asm/kvm_asm.h b/arch/arm64/include/asm/kvm_asm.h index 432a9ea1f02e..aed2aa61766a 100644 --- a/arch/arm64/include/asm/kvm_asm.h +++ b/arch/arm64/include/asm/kvm_asm.h @@ -59,7 +59,7 @@ #define __KVM_HOST_SMCCC_FUNC___vgic_v3_save_aprs 13 #define __KVM_HOST_SMCCC_FUNC___vgic_v3_restore_aprs 14 #define __KVM_HOST_SMCCC_FUNC___pkvm_init 15 -#define __KVM_HOST_SMCCC_FUNC___pkvm_create_mappings 16 +#define __KVM_HOST_SMCCC_FUNC___pkvm_host_share_hyp 16 #define __KVM_HOST_SMCCC_FUNC___pkvm_create_private_mapping 17 #define __KVM_HOST_SMCCC_FUNC___pkvm_cpu_set_vector 18 #define __KVM_HOST_SMCCC_FUNC___pkvm_prot_finalize 19 diff --git a/arch/arm64/kvm/hyp/include/nvhe/mem_protect.h b/arch/arm64/kvm/hyp/include/nvhe/mem_protect.h index 7ce36fbf5158..3d78d7782f1b 100644 --- a/arch/arm64/kvm/hyp/include/nvhe/mem_protect.h +++ b/arch/arm64/kvm/hyp/include/nvhe/mem_protect.h @@ -49,6 +49,7 @@ extern struct host_kvm host_kvm; extern const u8 pkvm_hyp_id; int __pkvm_prot_finalize(void); +int __pkvm_host_share_hyp(u64 pfn); bool addr_is_memory(phys_addr_t phys); int host_stage2_idmap_locked(u64 start, u64 end, enum kvm_pgtable_prot prot); diff --git a/arch/arm64/kvm/hyp/nvhe/hyp-main.c b/arch/arm64/kvm/hyp/nvhe/hyp-main.c index 7900d5b66ba3..2da6aa8da868 100644 --- a/arch/arm64/kvm/hyp/nvhe/hyp-main.c +++ b/arch/arm64/kvm/hyp/nvhe/hyp-main.c @@ -140,14 +140,11 @@ static void handle___pkvm_cpu_set_vector(struct kvm_cpu_context *host_ctxt) cpu_reg(host_ctxt, 1) = pkvm_cpu_set_vector(slot); } -static void handle___pkvm_create_mappings(struct kvm_cpu_context *host_ctxt) +static void handle___pkvm_host_share_hyp(struct kvm_cpu_context *host_ctxt) { - DECLARE_REG(unsigned long, start, host_ctxt, 1); - DECLARE_REG(unsigned long, size, host_ctxt, 2); - DECLARE_REG(unsigned long, phys, host_ctxt, 3); - DECLARE_REG(enum kvm_pgtable_prot, prot, host_ctxt, 4); + DECLARE_REG(u64, pfn, host_ctxt, 1); - cpu_reg(host_ctxt, 1) = __pkvm_create_mappings(start, size, phys, prot); + cpu_reg(host_ctxt, 1) = __pkvm_host_share_hyp(pfn); } static void handle___pkvm_create_private_mapping(struct kvm_cpu_context *host_ctxt) @@ -185,7 +182,7 @@ static const hcall_t host_hcall[] = { HANDLE_FUNC(__vgic_v3_restore_aprs), HANDLE_FUNC(__pkvm_init), HANDLE_FUNC(__pkvm_cpu_set_vector), - HANDLE_FUNC(__pkvm_create_mappings), + HANDLE_FUNC(__pkvm_host_share_hyp), HANDLE_FUNC(__pkvm_create_private_mapping), HANDLE_FUNC(__pkvm_prot_finalize), }; diff --git a/arch/arm64/kvm/hyp/nvhe/mem_protect.c b/arch/arm64/kvm/hyp/nvhe/mem_protect.c index 0ccea58df7e0..1b67f562b6fc 100644 --- a/arch/arm64/kvm/hyp/nvhe/mem_protect.c +++ b/arch/arm64/kvm/hyp/nvhe/mem_protect.c @@ -338,6 +338,95 @@ static int host_stage2_idmap(u64 addr) return ret; } +static inline bool check_prot(enum kvm_pgtable_prot prot, + enum kvm_pgtable_prot required, + enum kvm_pgtable_prot denied) +{ + return (prot & (required | denied)) == required; +} + +int __pkvm_host_share_hyp(u64 pfn) +{ + phys_addr_t addr = hyp_pfn_to_phys(pfn); + enum kvm_pgtable_prot prot, cur; + void *virt = __hyp_va(addr); + enum pkvm_page_state state; + kvm_pte_t pte; + u32 level; + int ret; + + if (!range_is_memory(addr, addr + PAGE_SIZE)) + return -EINVAL; + + hyp_spin_lock(&host_kvm.lock); + hyp_spin_lock(&pkvm_pgd_lock); + + ret = kvm_pgtable_get_leaf(&host_kvm.pgt, addr, &pte, &level); + if (ret) + goto unlock; + if (!pte) + goto map_shared; + + /* + * Check attributes in the host stage-2 PTE. We need the page to be: + * - mapped RWX as we're sharing memory; + * - not borrowed, as that implies absence of ownership. + * Otherwise, we can't let it got through + */ + cur = kvm_pgtable_stage2_pte_prot(pte); + prot = pkvm_mkstate(0, PKVM_PAGE_SHARED_BORROWED); + if (!check_prot(cur, KVM_PGTABLE_PROT_RWX, prot)) { + ret = -EPERM; + goto unlock; + } + + state = pkvm_getstate(cur); + if (state == PKVM_PAGE_OWNED) + goto map_shared; + + /* + * Tolerate double-sharing the same page, but this requires + * cross-checking the hypervisor stage-1. + */ + if (state != PKVM_PAGE_SHARED_OWNED) { + ret = -EPERM; + goto unlock; + } + + ret = kvm_pgtable_get_leaf(&pkvm_pgtable, (u64)virt, &pte, &level); + if (ret) + goto unlock; + + /* + * If the page has been shared with the hypervisor, it must be + * SHARED_BORROWED already. + */ + cur = kvm_pgtable_hyp_pte_prot(pte); + prot = pkvm_mkstate(PAGE_HYP, PKVM_PAGE_SHARED_BORROWED); + if (!check_prot(cur, prot, ~prot)) + ret = EPERM; + goto unlock; + +map_shared: + /* + * If the page is not yet shared, adjust mappings in both page-tables + * while both locks are held. + */ + prot = pkvm_mkstate(PAGE_HYP, PKVM_PAGE_SHARED_BORROWED); + ret = pkvm_create_mappings_locked(virt, virt + PAGE_SIZE, prot); + BUG_ON(ret); + + prot = pkvm_mkstate(KVM_PGTABLE_PROT_RWX, PKVM_PAGE_SHARED_OWNED); + ret = host_stage2_idmap_locked(addr, addr + PAGE_SIZE, prot); + BUG_ON(ret); + +unlock: + hyp_spin_unlock(&pkvm_pgd_lock); + hyp_spin_unlock(&host_kvm.lock); + + return ret; +} + void handle_host_mem_abort(struct kvm_cpu_context *host_ctxt) { struct kvm_vcpu_fault_info fault; diff --git a/arch/arm64/kvm/mmu.c b/arch/arm64/kvm/mmu.c index 0625bf2353c2..cbab146cda6a 100644 --- a/arch/arm64/kvm/mmu.c +++ b/arch/arm64/kvm/mmu.c @@ -259,10 +259,8 @@ static int __create_hyp_mappings(unsigned long start, unsigned long size, { int err; - if (!kvm_host_owns_hyp_mappings()) { - return kvm_call_hyp_nvhe(__pkvm_create_mappings, - start, size, phys, prot); - } + if (WARN_ON(!kvm_host_owns_hyp_mappings())) + return -EINVAL; mutex_lock(&kvm_hyp_pgd_mutex); err = kvm_pgtable_hyp_map(hyp_pgtable, start, size, phys, prot); @@ -282,6 +280,21 @@ static phys_addr_t kvm_kaddr_to_phys(void *kaddr) } } +static int pkvm_share_hyp(phys_addr_t start, phys_addr_t end) +{ + phys_addr_t addr; + int ret; + + for (addr = ALIGN_DOWN(start, PAGE_SIZE); addr < end; addr += PAGE_SIZE) { + ret = kvm_call_hyp_nvhe(__pkvm_host_share_hyp, + __phys_to_pfn(addr)); + if (ret) + return ret; + } + + return 0; +} + /** * create_hyp_mappings - duplicate a kernel virtual address range in Hyp mode * @from: The virtual kernel start address of the range @@ -302,6 +315,13 @@ int create_hyp_mappings(void *from, void *to, enum kvm_pgtable_prot prot) if (is_kernel_in_hyp_mode()) return 0; + if (!kvm_host_owns_hyp_mappings()) { + if (WARN_ON(prot != PAGE_HYP)) + return -EPERM; + return pkvm_share_hyp(kvm_kaddr_to_phys(from), + kvm_kaddr_to_phys(to)); + } + start = start & PAGE_MASK; end = PAGE_ALIGN(end); From patchwork Thu Jul 29 13:28:18 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Quentin Perret X-Patchwork-Id: 12408779 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-17.5 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_ADSP_CUSTOM_MED,DKIM_SIGNED,DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, URIBL_BLOCKED,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id AC2FAC4338F for ; Thu, 29 Jul 2021 13:48:57 +0000 (UTC) Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 7AC556023F for ; Thu, 29 Jul 2021 13:48:57 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.4.1 mail.kernel.org 7AC556023F Authentication-Results: mail.kernel.org; dmarc=fail (p=reject dis=none) header.from=google.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:Cc:To:From:Subject:References: Mime-Version:Message-Id:In-Reply-To:Date:Reply-To:Content-ID: Content-Description:Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc :Resent-Message-ID:List-Owner; bh=EcEBg71o0zFzh4WBFXFVsjpAe0risgMkETNji8aYshI=; b=oJABoWpynqa2OaYASAKmF27mZK NBG+ljMYGkQv/q8367yLLEzGjPJcm4+WcBjeGfmwI6+CKzIhg5SmoGu+BDgx/QWUsgYzOdtUVorbL tGfvLvOuax/0NfR1gR2wtm2g7kts9lil4ohw/GGZrB09irNQPxnw6HA2tSfKcytuwGeM5UE0uT32t VJF+13DNZ/Gfh0V6n8co2cTBH3IpIVQR+YVt0kv3BDt0SBWPghmr10OI9msbuC42E9ABMNG6Xz3XG pyH6vTnmFlfVccZ12hXZXQfeB3WSOheD41lSd9XXKH/Cf1bEiovAD3y2K6w2vzP5gt46jTdi7MEic CuldzGeA==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1m96Mh-004L93-B8; Thu, 29 Jul 2021 13:46:24 +0000 Received: from mail-qv1-xf4a.google.com ([2607:f8b0:4864:20::f4a]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1m9664-004Eov-2Y for linux-arm-kernel@lists.infradead.org; Thu, 29 Jul 2021 13:29:13 +0000 Received: by mail-qv1-xf4a.google.com with SMTP id a2-20020a0562141302b02903303839b843so3916001qvv.13 for ; Thu, 29 Jul 2021 06:29:11 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=date:in-reply-to:message-id:mime-version:references:subject:from:to :cc; bh=vQtosGJFAOrUOiBl4clG9WDMUZRQZWton/qwil1k0+Q=; b=dXygFS4MzHesaeMqbwJ0+cZ0afNQ4/u5vTF/RgsqUpPCSvO1FUb9yuOtobHsreJWqr dUqlIAu7Fvz+I43aDHKZyp8VKGAJcvdmKnnkeHIjZqt8EhOzaHANhcXh8RtV0Ilxlnt9 4Pl/Gc+kDSrSP77EuPnHl91zLmkSkct+Gl67dPohs9YPPdMJ8ph+HGp8BjrsrKxaXVEy OoSuN3lBXk7s1yZPSllQnVPCpAnlYGI91eJ1m/mdcSbcCE+Z2qNsccfX2HfuY/+rJJcM noOMmdViiMNvT23lNHKkf8AZYFmTtkC55+7rWH4afynTCCzcjLC8ILshwjJH8cojfSWr oXXg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=vQtosGJFAOrUOiBl4clG9WDMUZRQZWton/qwil1k0+Q=; b=YI5pxhY/kFocnfb3MVyum9V7AH9d56JwueIElgnWcc/mj5wSw/wNJkgZs7a6d5f2v5 iU/ifugkazIMmV5RdK/hT3+Gb4qYXEaOqD79VMoZ8TI/ku7tqrtTs9NUoorUilirHGrO VGaFGCdHKWfTV7uulqfbq7kpsJgC4TOWjjjBavPIpAFmGf6qiwchFkKfI58Nlm5y+fmy MQ07++jgbaP35WqZXJihbnz5YSe3WUXF/N+plO27j65c0tHO4RkTa+JEnblLDzYKoov1 S6U1WQomR0PKtSLDTInV9I+681ZRsy2BTb5xs3wOVHTap88SzGCj953KRELqgo42aidp yP9w== X-Gm-Message-State: AOAM531rb1RiB2sPsDwjWatv+2lKdjcfyCr8p07046TMWMjovkgdOBsG fCZGGd9MW2dc5tvnAa2jgJMnacHHXVTy X-Google-Smtp-Source: ABdhPJzMM++UNzruvJmyLbB/7mURxGlTSkGbvn6rPkDW6SXHchtq4/8ma0IIMxLOdpKM8oJyuuW51EsCRzfk X-Received: from luke.lon.corp.google.com ([2a00:79e0:d:210:293a:bc89:7514:5218]) (user=qperret job=sendgmr) by 2002:a05:6214:104b:: with SMTP id l11mr5321061qvr.40.1627565350559; Thu, 29 Jul 2021 06:29:10 -0700 (PDT) Date: Thu, 29 Jul 2021 14:28:18 +0100 In-Reply-To: <20210729132818.4091769-1-qperret@google.com> Message-Id: <20210729132818.4091769-22-qperret@google.com> Mime-Version: 1.0 References: <20210729132818.4091769-1-qperret@google.com> X-Mailer: git-send-email 2.32.0.432.gabb21c7263-goog Subject: [PATCH v3 21/21] KVM: arm64: Make __pkvm_create_mappings static From: Quentin Perret To: maz@kernel.org, james.morse@arm.com, alexandru.elisei@arm.com, suzuki.poulose@arm.com, catalin.marinas@arm.com, will@kernel.org Cc: linux-arm-kernel@lists.infradead.org, kvmarm@lists.cs.columbia.edu, linux-kernel@vger.kernel.org, ardb@kernel.org, qwandor@google.com, tabba@google.com, dbrazdil@google.com, kernel-team@android.com, Quentin Perret X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20210729_062912_183944_18344A0C X-CRM114-Status: GOOD ( 10.72 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org The __pkvm_create_mappings() function is no longer used outside of nvhe/mm.c, make it static. Signed-off-by: Quentin Perret --- arch/arm64/kvm/hyp/include/nvhe/mm.h | 2 -- arch/arm64/kvm/hyp/nvhe/mm.c | 4 ++-- 2 files changed, 2 insertions(+), 4 deletions(-) diff --git a/arch/arm64/kvm/hyp/include/nvhe/mm.h b/arch/arm64/kvm/hyp/include/nvhe/mm.h index c76d7136ed9b..c9a8f535212e 100644 --- a/arch/arm64/kvm/hyp/include/nvhe/mm.h +++ b/arch/arm64/kvm/hyp/include/nvhe/mm.h @@ -24,8 +24,6 @@ int hyp_back_vmemmap(phys_addr_t phys, unsigned long size, phys_addr_t back); int pkvm_cpu_set_vector(enum arm64_hyp_spectre_vector slot); int pkvm_create_mappings(void *from, void *to, enum kvm_pgtable_prot prot); int pkvm_create_mappings_locked(void *from, void *to, enum kvm_pgtable_prot prot); -int __pkvm_create_mappings(unsigned long start, unsigned long size, - unsigned long phys, enum kvm_pgtable_prot prot); unsigned long __pkvm_create_private_mapping(phys_addr_t phys, size_t size, enum kvm_pgtable_prot prot); diff --git a/arch/arm64/kvm/hyp/nvhe/mm.c b/arch/arm64/kvm/hyp/nvhe/mm.c index 6fbe8e8030f6..2fabeceb889a 100644 --- a/arch/arm64/kvm/hyp/nvhe/mm.c +++ b/arch/arm64/kvm/hyp/nvhe/mm.c @@ -23,8 +23,8 @@ u64 __io_map_base; struct memblock_region hyp_memory[HYP_MEMBLOCK_REGIONS]; unsigned int hyp_memblock_nr; -int __pkvm_create_mappings(unsigned long start, unsigned long size, - unsigned long phys, enum kvm_pgtable_prot prot) +static int __pkvm_create_mappings(unsigned long start, unsigned long size, + unsigned long phys, enum kvm_pgtable_prot prot) { int err;