From patchwork Wed Dec 1 17:03:55 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Quentin Perret X-Patchwork-Id: 12694348 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 06114C433EF for ; Wed, 1 Dec 2021 17:06:08 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:Cc:To:From:Subject:References: Mime-Version:Message-Id:In-Reply-To:Date:Reply-To:Content-ID: Content-Description:Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc :Resent-Message-ID:List-Owner; bh=O2Tbht+T7nnXOsfHT2s1JPNm7N68XYtfeWDsbiiIqUM=; b=d+SWBM6Vv0K2wlp5adexP1IS9I 3ON4bujCIY/FPH5TviwL1wSCt2PejZDQy6nusTrps4FBUxaQZ4kFjxT/Xv++d3vB+PcbQCghCRPh6 QP8ayHw9kullZfgS0tnO9WGwdjaoCdXTXB05IM/WVbahn/YOip/h61KcsRkHHWgiuyKTQRyoIYONQ tieXEyaBGgMbT420O9zA5GemTYFLd1BwRKsSHLge6buFtwVD1Tbktu5KtPvNp0YaM3+w+Dw2fodVz dicGQYMWlClSlnIaQ6SJhLwdv7CH9ZBxQHthqU1+gxBtYn2nAWl5eFTfy1M0hMmEfwK8PWCWpo2XJ WDYfX+NQ==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1msT1z-009Rrg-Kp; Wed, 01 Dec 2021 17:04:31 +0000 Received: from mail-wm1-x349.google.com ([2a00:1450:4864:20::349]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1msT1o-009RoP-Nf for linux-arm-kernel@lists.infradead.org; Wed, 01 Dec 2021 17:04:21 +0000 Received: by mail-wm1-x349.google.com with SMTP id m18-20020a05600c3b1200b0033283ea5facso610941wms.1 for ; Wed, 01 Dec 2021 09:04:18 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20210112; h=date:in-reply-to:message-id:mime-version:references:subject:from:to :cc; bh=To0SNnHVfI+H6lAX6lBPI0gbbL9kvAmYwmWxoThbEmQ=; b=c3N+Hsz63lDc/AuLe7OwcJn+QlFGAiiPAK/v+uveeQj4J/xIMFLUVreGUkLB/OhWe7 xxQExo5sN2tjJX/K0t22Pyedrge8bHw8PQX9ioa+ZyPcJXTNVTqwnzaN4wJmEAY0G3WF qePOIM36BNZrzbzHLF499Ie9giwIeOEhK0DjLg4A4aO0pyneUPX9V3J3QAgZhnAalmGq GgU2j5rGVQ+/ywMfsyzLjUCf/yIn9JUsfAZrM4tEXfekoTu7F7CPl0GECKg5acnJo0gD ldlS2SRcWRjiOuZ7JvgkDuWdek0FI/pSl01nLYU8GyAhvbnuRN8Ufpu4D57DZce5aeNr wU9Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=To0SNnHVfI+H6lAX6lBPI0gbbL9kvAmYwmWxoThbEmQ=; b=LlJ4seYuuPCz9kHUJ/rGt51Wa9sgxXFY1mFF7JGg/igEkDL63lTOA3Yrn5hqfEDsLl etU06YsjPiuBottqu7fQ7QF7wBlAXpfDeYUxWKBVA9OLBDUSIqhu8EVYimsG/oecNle+ TEi7S9VxmgvtdeTfRlOWiq/eXIO0aMrByqJnAmQVTlZLxUC9aqUsDOgl5ExkyoMJ7gF4 Ie8ztdo9d9wj54VSQRXmNz7OD+zxrMRHZfyISRY+xxzYWO+aX+KPpwfzt1Y/M/k89Te0 yDB1A8nYi8msnQqkbWtQeBTrTei+G+CSs6YjNxNbBYA/0TeJo9cOGQuLOBDq8tF8UfWB iDqw== X-Gm-Message-State: AOAM530wvLu06WConAleWEz3Y4gOJRsRUR/gTbBkzDBDM6sYq56vXT7D 71IwAEGYhqskQO/eY6unNL0sDHu5acEI X-Google-Smtp-Source: ABdhPJyFcaB2RvMSztkS941w8QnL4bS9fgV6ZFzqQ3PoBgP+qzxkDtb1aqs2DI+obM4eDv5SPs/p5zCTtLuI X-Received: from luke.lon.corp.google.com ([2a00:79e0:d:210:1cab:1a20:2ef1:ddde]) (user=qperret job=sendgmr) by 2002:a05:600c:198f:: with SMTP id t15mr8655534wmq.94.1638378257676; Wed, 01 Dec 2021 09:04:17 -0800 (PST) Date: Wed, 1 Dec 2021 17:03:55 +0000 In-Reply-To: <20211201170411.1561936-1-qperret@google.com> Message-Id: <20211201170411.1561936-2-qperret@google.com> Mime-Version: 1.0 References: <20211201170411.1561936-1-qperret@google.com> X-Mailer: git-send-email 2.34.0.rc2.393.gf8c9666880-goog Subject: [PATCH v3 01/15] KVM: arm64: Check if running in VHE from kvm_host_owns_hyp_mappings() From: Quentin Perret To: Marc Zyngier , James Morse , Alexandru Elisei , Suzuki K Poulose , Catalin Marinas , Will Deacon Cc: linux-arm-kernel@lists.infradead.org, kvmarm@lists.cs.columbia.edu, linux-kernel@vger.kernel.org, kernel-team@android.com X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20211201_090420_793289_9C61517D X-CRM114-Status: GOOD ( 12.30 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org The kvm_host_owns_hyp_mappings() function should return true if and only if the host kernel is responsible for creating the hypervisor stage-1 mappings. That is only possible in standard non-VHE mode, or during boot in protected nVHE mode. But either way, non of this makes sense in VHE, so make sure to catch this case as well, hence making the function return sensible values in any context (VHE or not). Suggested-by: Marc Zyngier Signed-off-by: Quentin Perret --- arch/arm64/kvm/mmu.c | 3 +++ 1 file changed, 3 insertions(+) diff --git a/arch/arm64/kvm/mmu.c b/arch/arm64/kvm/mmu.c index 326cdfec74a1..f8f1096a297f 100644 --- a/arch/arm64/kvm/mmu.c +++ b/arch/arm64/kvm/mmu.c @@ -239,6 +239,9 @@ void free_hyp_pgds(void) static bool kvm_host_owns_hyp_mappings(void) { + if (is_kernel_in_hyp_mode()) + return false; + if (static_branch_likely(&kvm_protected_mode_initialized)) return false; From patchwork Wed Dec 1 17:03:56 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Quentin Perret X-Patchwork-Id: 12694349 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 84214C433F5 for ; Wed, 1 Dec 2021 17:06:12 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:Cc:To:From:Subject:References: Mime-Version:Message-Id:In-Reply-To:Date:Reply-To:Content-ID: Content-Description:Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc :Resent-Message-ID:List-Owner; bh=eSDJ5Uv/XYhjE0iCM30uRR86W0rgJH6yUyqn7k1HE1g=; b=1EUCX7gusRr3zDR3E7iQJt8iDg ZS24L2Rn7+X7vGY61z2XWxPZaHN6ha+LBqIE9C1FkWnmhhrev+WbbsUJVqaLThW91ZFhAlw3tvjJZ bL+fbKQYU5BJLgllRgVGfGInckPRh3Q+ddGNf8CeIRMEkI48lvDsLYS31q16nNYrCA1bxjwLVq3N0 64a0imjuJiqSucbtuyjTeSovYX88toZhxIFxqGWoIZXBdY/KbZZVgWX6zDvL9k+OQFRGXnvorkjg3 i7VAOehCXIZQWGU1Z71eD84ObIIVKK+5Jdyq/2Jjmi3mCSoG8S+3CPAoapgWT38jRjvgsMM9+2UXd L0sU8oQg==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1msT2C-009Rum-A5; Wed, 01 Dec 2021 17:04:44 +0000 Received: from mail-wm1-x349.google.com ([2a00:1450:4864:20::349]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1msT1r-009RpF-6Y for linux-arm-kernel@lists.infradead.org; Wed, 01 Dec 2021 17:04:24 +0000 Received: by mail-wm1-x349.google.com with SMTP id r129-20020a1c4487000000b00333629ed22dso132989wma.6 for ; Wed, 01 Dec 2021 09:04:22 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20210112; h=date:in-reply-to:message-id:mime-version:references:subject:from:to :cc; bh=5u4bcd8p5lWhOh9Twmw6ckk6rlz/H+k7Di86Nv+W7y0=; b=c2jCqyYJWi6j0q+TdDuz84phld5+VtSpY4m7fhUMjwZLascGKc4969abcgbluEB/Ze h/sY9sRt4zD8i/w/dPj//QCFbZ71q4VJxjSfwZNtAByXbmjpRHJ+yOLVXEcfb6s9kZIH NnaaFE1z0Ehl4VNNXPMVWuvl102KADwuQxFHzKv2d2JeQSOYpIVIs1BlfbuuZmb857DQ r30VAksuJc/Cbj3iGu3aZe7i20L+ENRdWzTjRz5xtASwajdp145hE153kJaS7HGrgjxj 8SIhH82RSlvwXFa2KohYPvS1mvjQCVQPZ51hqOt61AfOjEwOtzTvnuVJWbZIxCVcSEw1 GHXg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=5u4bcd8p5lWhOh9Twmw6ckk6rlz/H+k7Di86Nv+W7y0=; b=u7QZkMIMHz6wquGSkmyUaToMNk5sECiScID9CjWtaLnZeWfS3ye/+dP8UFAs9H1F1+ VGc0rcNXsmFOhnJ8LNhBpRst5M2UqoQeqjywHdaCKIu76MG1EKklG57QbfaPriqchEfI Hr7NUu7kmrbVPYmxbRZXvKhlcjlZviraep3DqYkNCwVd/n/+yOjsmww/VXVhIQNaajAU 2vYiiGPdILkaIgTfRcIvBxAvXIRqlYgDVIOfXZ6E0crBXVlw8/w2Np9jGqvEnY7DaRvL /UcUvqk2TyRz06B6kJMRAB5Lok3bmRy7f23NPo60swWzFO58D5tcUpLVByvhl4zOwrBo HwJQ== X-Gm-Message-State: AOAM532Mem0rg3xQ7BQLhbkGv3e6+1LDJsB1jrzXNT2NnNL3jjttNub8 /gJ30sy+5JS/+lSRK1j1ZA6cBFHPZM8C X-Google-Smtp-Source: ABdhPJz8KqUSlppvt5PWeoeu8pXK06/96X8A4lVCiTuTfHGPzga9ENc8lxXkM7E2YtzgjN+WEGl+8JdtGsxP X-Received: from luke.lon.corp.google.com ([2a00:79e0:d:210:1cab:1a20:2ef1:ddde]) (user=qperret job=sendgmr) by 2002:a05:6000:23a:: with SMTP id l26mr8091897wrz.215.1638378261551; Wed, 01 Dec 2021 09:04:21 -0800 (PST) Date: Wed, 1 Dec 2021 17:03:56 +0000 In-Reply-To: <20211201170411.1561936-1-qperret@google.com> Message-Id: <20211201170411.1561936-3-qperret@google.com> Mime-Version: 1.0 References: <20211201170411.1561936-1-qperret@google.com> X-Mailer: git-send-email 2.34.0.rc2.393.gf8c9666880-goog Subject: [PATCH v3 02/15] KVM: arm64: Provide {get, put}_page() stubs for early hyp allocator From: Quentin Perret To: Marc Zyngier , James Morse , Alexandru Elisei , Suzuki K Poulose , Catalin Marinas , Will Deacon Cc: linux-arm-kernel@lists.infradead.org, kvmarm@lists.cs.columbia.edu, linux-kernel@vger.kernel.org, kernel-team@android.com X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20211201_090423_266494_9D9364FD X-CRM114-Status: GOOD ( 11.98 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org In nVHE protected mode, the EL2 code uses a temporary allocator during boot while re-creating its stage-1 page-table. Unfortunately, the hyp_vmmemap is not ready to use at this stage, so refcounting pages is not possible. That is not currently a problem because hyp stage-1 mappings are never removed, which implies refcounting of page-table pages is unnecessary. In preparation for allowing hypervisor stage-1 mappings to be removed, provide stub implementations for {get,put}_page() in the early allocator. Signed-off-by: Quentin Perret Acked-by: Will Deacon --- arch/arm64/kvm/hyp/nvhe/early_alloc.c | 5 +++++ 1 file changed, 5 insertions(+) diff --git a/arch/arm64/kvm/hyp/nvhe/early_alloc.c b/arch/arm64/kvm/hyp/nvhe/early_alloc.c index 1306c430ab87..00de04153cc6 100644 --- a/arch/arm64/kvm/hyp/nvhe/early_alloc.c +++ b/arch/arm64/kvm/hyp/nvhe/early_alloc.c @@ -43,6 +43,9 @@ void *hyp_early_alloc_page(void *arg) return hyp_early_alloc_contig(1); } +static void hyp_early_alloc_get_page(void *addr) { } +static void hyp_early_alloc_put_page(void *addr) { } + void hyp_early_alloc_init(void *virt, unsigned long size) { base = cur = (unsigned long)virt; @@ -51,4 +54,6 @@ void hyp_early_alloc_init(void *virt, unsigned long size) hyp_early_alloc_mm_ops.zalloc_page = hyp_early_alloc_page; hyp_early_alloc_mm_ops.phys_to_virt = hyp_phys_to_virt; hyp_early_alloc_mm_ops.virt_to_phys = hyp_virt_to_phys; + hyp_early_alloc_mm_ops.get_page = hyp_early_alloc_get_page; + hyp_early_alloc_mm_ops.put_page = hyp_early_alloc_put_page; } From patchwork Wed Dec 1 17:03:57 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Quentin Perret X-Patchwork-Id: 12694350 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 51FB3C433F5 for ; Wed, 1 Dec 2021 17:06:42 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:Cc:To:From:Subject:References: Mime-Version:Message-Id:In-Reply-To:Date:Reply-To:Content-ID: Content-Description:Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc :Resent-Message-ID:List-Owner; bh=FvF5onZwwitvB0GJDXUDEFer13+6BzmVcw8rC3JoLes=; b=hk1H2ux9KjSmGIiu8vSoR0BPZG rf7KKFdlcaWDPXk0Qp2ZBO8G9w/ldaAyQHfRarkmCwQRKljomGCJgGBLiNgpAUPdO15/qBrTXXUEB nv2PL5NOI3AvCpgJEqrabhJ2l1JejgXcGIztbqgre+r7jRxDs8KcKFSljHFNzhMt9jhbMwuEIFbxZ 9Eu1mwAaqm849YDBTY/9E4Cb7pzW10QUWf4AV368YEwhfNAFgw4ru0dk+hfqh5kvQdCPS6+osW66W cDfrBHWbvRWdL8g+p3dfCwtUX84uDgj0jfpI0fCkzNkQ/HjmEf3/wKoTPPZrGnEJss9iEWEpVGoMr G6RIUJNQ==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1msT2O-009S0P-Na; Wed, 01 Dec 2021 17:04:56 +0000 Received: from mail-wm1-x34a.google.com ([2a00:1450:4864:20::34a]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1msT1v-009RqY-QH for linux-arm-kernel@lists.infradead.org; Wed, 01 Dec 2021 17:04:29 +0000 Received: by mail-wm1-x34a.google.com with SMTP id i131-20020a1c3b89000000b00337f92384e0so135690wma.5 for ; Wed, 01 Dec 2021 09:04:26 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20210112; h=date:in-reply-to:message-id:mime-version:references:subject:from:to :cc; bh=A7hr35xG6SzEHwDet74XtvHzJsI/X+Zg/yL5/WYnOz0=; b=ah12j+uUrkgUrk9b1HPkNYJu1wVkvH0SX1GjxWlFdsGE3bEQ46fAXvn0aJfcBwEo+8 6LrYpwJQVU32AtFiqZZFehFhMRCpiksII11x5No7K2l4MzTVhOGCcKSwW4VsaKYB/KwZ STISVRv3cmicA+IFYacwY4r+EdcvE6uDIJ6X9F/HMudy2oqDeRgFfx9EYMTh5ZhcNkDE y+NXXLWJwOH5boJw7lqmcJwIiiAh7SITqADYhwT/9haBjdb8CHxhJXGD/TIgVldyEoGn uw+wuq1gmGXOB6VUh00f2IDCloa5CVB3P1CMZ5640YlWHcMYBtJVsIffUl4qaumK2x0t XUbw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=A7hr35xG6SzEHwDet74XtvHzJsI/X+Zg/yL5/WYnOz0=; b=SfGFy4CXMCr7uFU7apj8DvX4vKpqK8OedZ15sc08cMUT7Cp/JuuOfCFw2bnhHLu2nz J2DnVJt8iOXfs6rs2U//Ak8O/3EMVBPWaphVfBnB3t5B1WO1LtecxYkFiYsyYcRPD43L jp7bZFv+qzo9QkdbTWqRzI2UMb11CKQj0SoBcHAUuOW6VAQElPeV9AAgEys6Dfd2zQ8q 23UD7z4OIw1fLqx13vmjznrs03sRL9kRt3mOy+66/5Tb8fds09XHNMCAGm7yBaZjM3v8 MdgnIFt0e1aB0i2pAFEDEEKjCQqnOK1AZRUwM6MYdzpnyOla77QEErEMBXnBNx6M/ZN3 mkRQ== X-Gm-Message-State: AOAM533LQpTpkGaKzAyTG773Ft3kIadCrNzY3co7K8e9breZXemwZLp/ lKda0ggLwQ2abz6xuehwZZlKtRyvAbHu X-Google-Smtp-Source: ABdhPJxp5L6PtF860OZ96JvJzz+R53mWZq1H92zAhum23/hQpBFE1gQt/KAt2QyLRQi8U4J5dXjZ/F1KEhLb X-Received: from luke.lon.corp.google.com ([2a00:79e0:d:210:1cab:1a20:2ef1:ddde]) (user=qperret job=sendgmr) by 2002:a5d:40cf:: with SMTP id b15mr8314262wrq.161.1638378265569; Wed, 01 Dec 2021 09:04:25 -0800 (PST) Date: Wed, 1 Dec 2021 17:03:57 +0000 In-Reply-To: <20211201170411.1561936-1-qperret@google.com> Message-Id: <20211201170411.1561936-4-qperret@google.com> Mime-Version: 1.0 References: <20211201170411.1561936-1-qperret@google.com> X-Mailer: git-send-email 2.34.0.rc2.393.gf8c9666880-goog Subject: [PATCH v3 03/15] KVM: arm64: Refcount hyp stage-1 pgtable pages From: Quentin Perret To: Marc Zyngier , James Morse , Alexandru Elisei , Suzuki K Poulose , Catalin Marinas , Will Deacon Cc: linux-arm-kernel@lists.infradead.org, kvmarm@lists.cs.columbia.edu, linux-kernel@vger.kernel.org, kernel-team@android.com X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20211201_090427_879243_CAB0E606 X-CRM114-Status: GOOD ( 12.86 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org To prepare the ground for allowing hyp stage-1 mappings to be removed at run-time, update the KVM page-table code to maintain a correct refcount using the ->{get,put}_page() function callbacks. Signed-off-by: Quentin Perret --- arch/arm64/kvm/hyp/pgtable.c | 17 ++++++++++++++--- 1 file changed, 14 insertions(+), 3 deletions(-) diff --git a/arch/arm64/kvm/hyp/pgtable.c b/arch/arm64/kvm/hyp/pgtable.c index f8ceebe4982e..768a58835153 100644 --- a/arch/arm64/kvm/hyp/pgtable.c +++ b/arch/arm64/kvm/hyp/pgtable.c @@ -408,8 +408,10 @@ static bool hyp_map_walker_try_leaf(u64 addr, u64 end, u32 level, return false; new = kvm_init_valid_leaf_pte(phys, data->attr, level); - if (hyp_pte_needs_update(old, new)) + if (hyp_pte_needs_update(old, new)) { smp_store_release(ptep, new); + data->mm_ops->get_page(ptep); + } data->phys += granule; return true; @@ -433,6 +435,7 @@ static int hyp_map_walker(u64 addr, u64 end, u32 level, kvm_pte_t *ptep, return -ENOMEM; kvm_set_table_pte(ptep, childp, mm_ops); + mm_ops->get_page(ptep); return 0; } @@ -482,8 +485,16 @@ static int hyp_free_walker(u64 addr, u64 end, u32 level, kvm_pte_t *ptep, enum kvm_pgtable_walk_flags flag, void * const arg) { struct kvm_pgtable_mm_ops *mm_ops = arg; + kvm_pte_t pte = *ptep; + + if (!kvm_pte_valid(pte)) + return 0; + + mm_ops->put_page(ptep); + + if (kvm_pte_table(pte, level)) + mm_ops->put_page(kvm_pte_follow(pte, mm_ops)); - mm_ops->put_page((void *)kvm_pte_follow(*ptep, mm_ops)); return 0; } @@ -491,7 +502,7 @@ void kvm_pgtable_hyp_destroy(struct kvm_pgtable *pgt) { struct kvm_pgtable_walker walker = { .cb = hyp_free_walker, - .flags = KVM_PGTABLE_WALK_TABLE_POST, + .flags = KVM_PGTABLE_WALK_LEAF | KVM_PGTABLE_WALK_TABLE_POST, .arg = pgt->mm_ops, }; From patchwork Wed Dec 1 17:03:58 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Quentin Perret X-Patchwork-Id: 12694351 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id AED1FC433F5 for ; Wed, 1 Dec 2021 17:06:56 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:Cc:To:From:Subject:References: Mime-Version:Message-Id:In-Reply-To:Date:Reply-To:Content-ID: Content-Description:Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc :Resent-Message-ID:List-Owner; bh=/DZ72Hg1A7x+udsw5YjDhLp/7eKqfLLKP6GZRmBmnkU=; b=nrixfuSX/nk2+XCkSVnrpRqmbk +TmlvwzcRlcM406PWExl10XwSn+qP+DylKxjdszhymYx0l5ENEZGwVnsj31qp33h44d94EEE4fi/6 qCMPKdiXNEwoZerLoE9/2qV/zFbx47SnGoyX/azYrjoNu5jlN7cdPQ39+vYEKmJQZ5h5SlAi8p4tB XmnNQEXXGI/G1wCk4KBBdz/IFFMxApXEk0POgBaAs51dWfERkZ1c+qE01JoiUvgzsjFbaqwU7IwxR AM7p6Q2N0BxTBhtOwYg29lBtpOEf/cxS2k+OJbAgijg9dFgcLlpsUQer2j2unAg0UK3iI4viElWz/ 2XneB/2g==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1msT2e-009S8H-3W; Wed, 01 Dec 2021 17:05:12 +0000 Received: from mail-wr1-x449.google.com ([2a00:1450:4864:20::449]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1msT20-009RrY-Hb for linux-arm-kernel@lists.infradead.org; Wed, 01 Dec 2021 17:04:33 +0000 Received: by mail-wr1-x449.google.com with SMTP id p17-20020adff211000000b0017b902a7701so4459617wro.19 for ; Wed, 01 Dec 2021 09:04:30 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20210112; h=date:in-reply-to:message-id:mime-version:references:subject:from:to :cc; bh=9uBUypl+EXPtiyqLggdiPCeBqKTFxHsKtVyJKG8pgmY=; b=Vv1c7CHoktHXpAkMllAbrV9Qh8WJZnyTA3AZU15KbH4HMkRzFi9p8jcrrt9SYvw/vm XiJfozl/oFlTHNfLRWY3HBA3gFa9GG27TT0SAeeuzCVqz8nF3/MFgRTE1JTlFJUP24Ec BzDBGevNwW/zf0YB+zazdFhJ4kiNaPq5iHmleo2R5XUDtGWhB+8awKuhqPooz9Z/mSNG gs4ndbhCyBpRHS3gb21jCowKbTdnE1ozwPsjKll5mD7/DVNd1EvOyQ72bfzphCiSXYhN gFAldNGJnBPpUdYyDkMsuJXxN+63Mq6uHhqZqhCmWw9MzrD9lAxNSXUtKzeOMkHVbfX8 pvlg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=9uBUypl+EXPtiyqLggdiPCeBqKTFxHsKtVyJKG8pgmY=; b=Qtzpd3OQo/qlRgiSBC41B4GegBKbqUF/OjA/FH2NDBKlukZ4g6WtILSUyxJe5GBUdr EjPAeRMHpZG2Qg6qPUXMrxxTQNdHTtr8+nWGgDSKO0+LyFzul2kqqgg5yl3lgZI7Qeff k8hCF3DWqn/dP60Q1MLkSAZyKXiK0vXVuG9DnRMBuz0RueCpm33PPftiw3CojTRE6ME5 lhQWOTf1IN3ZVA4E9b/N5gzZUCosNS/K2Ty4gjdn5GgbHrL0PxzT+gUHt+LA/qx3LGZt AbBJs5B2Co2wnusNJlsugVZiPzAAUf3nvjw+SwYlBQ50uyv4kWcHx9f0/xhjjg4R9lbt HMqg== X-Gm-Message-State: AOAM53075G5kzXAfvVRWmioPg1WOYRZdLMzvEp2EHHBaXq8kw8VMhwxV nsdz0hpWtDrJ1DaWVZAI+yp7crpgyGc6 X-Google-Smtp-Source: ABdhPJynMlhVluzA+1W+geGVMD660u2IXflLbXQYLj5MfYI1LEe60lA76lE/vavpHK8Z8gHIBVjRNs1KfRK+ X-Received: from luke.lon.corp.google.com ([2a00:79e0:d:210:1cab:1a20:2ef1:ddde]) (user=qperret job=sendgmr) by 2002:a05:600c:1d1b:: with SMTP id l27mr113174wms.1.1638378269441; Wed, 01 Dec 2021 09:04:29 -0800 (PST) Date: Wed, 1 Dec 2021 17:03:58 +0000 In-Reply-To: <20211201170411.1561936-1-qperret@google.com> Message-Id: <20211201170411.1561936-5-qperret@google.com> Mime-Version: 1.0 References: <20211201170411.1561936-1-qperret@google.com> X-Mailer: git-send-email 2.34.0.rc2.393.gf8c9666880-goog Subject: [PATCH v3 04/15] KVM: arm64: Fixup hyp stage-1 refcount From: Quentin Perret To: Marc Zyngier , James Morse , Alexandru Elisei , Suzuki K Poulose , Catalin Marinas , Will Deacon Cc: linux-arm-kernel@lists.infradead.org, kvmarm@lists.cs.columbia.edu, linux-kernel@vger.kernel.org, kernel-team@android.com X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20211201_090432_605599_C0E3E065 X-CRM114-Status: GOOD ( 16.07 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org In nVHE-protected mode, the hyp stage-1 page-table refcount is broken due to the lack of refcount support in the early allocator. Fix-up the refcount in the finalize walker, once the 'hyp_vmemmap' is up and running. Signed-off-by: Quentin Perret Acked-by: Will Deacon --- arch/arm64/kvm/hyp/nvhe/setup.c | 21 ++++++++++++++++----- 1 file changed, 16 insertions(+), 5 deletions(-) diff --git a/arch/arm64/kvm/hyp/nvhe/setup.c b/arch/arm64/kvm/hyp/nvhe/setup.c index 578f71798c2e..875b5174342f 100644 --- a/arch/arm64/kvm/hyp/nvhe/setup.c +++ b/arch/arm64/kvm/hyp/nvhe/setup.c @@ -165,6 +165,7 @@ static int finalize_host_mappings_walker(u64 addr, u64 end, u32 level, enum kvm_pgtable_walk_flags flag, void * const arg) { + struct kvm_pgtable_mm_ops *mm_ops = arg; enum kvm_pgtable_prot prot; enum pkvm_page_state state; kvm_pte_t pte = *ptep; @@ -173,6 +174,15 @@ static int finalize_host_mappings_walker(u64 addr, u64 end, u32 level, if (!kvm_pte_valid(pte)) return 0; + /* + * Fix-up the refcount for the page-table pages as the early allocator + * was unable to access the hyp_vmemmap and so the buddy allocator has + * initialised the refcount to '1'. + */ + mm_ops->get_page(ptep); + if (flag != KVM_PGTABLE_WALK_LEAF) + return 0; + if (level != (KVM_PGTABLE_MAX_LEVELS - 1)) return -EINVAL; @@ -205,7 +215,8 @@ static int finalize_host_mappings(void) { struct kvm_pgtable_walker walker = { .cb = finalize_host_mappings_walker, - .flags = KVM_PGTABLE_WALK_LEAF, + .flags = KVM_PGTABLE_WALK_LEAF | KVM_PGTABLE_WALK_TABLE_POST, + .arg = pkvm_pgtable.mm_ops, }; int i, ret; @@ -240,10 +251,6 @@ void __noreturn __pkvm_init_finalise(void) if (ret) goto out; - ret = finalize_host_mappings(); - if (ret) - goto out; - pkvm_pgtable_mm_ops = (struct kvm_pgtable_mm_ops) { .zalloc_page = hyp_zalloc_hyp_page, .phys_to_virt = hyp_phys_to_virt, @@ -253,6 +260,10 @@ void __noreturn __pkvm_init_finalise(void) }; pkvm_pgtable.mm_ops = &pkvm_pgtable_mm_ops; + ret = finalize_host_mappings(); + if (ret) + goto out; + out: /* * We tail-called to here from handle___pkvm_init() and will not return, From patchwork Wed Dec 1 17:03:59 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Quentin Perret X-Patchwork-Id: 12694352 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id C2114C433F5 for ; Wed, 1 Dec 2021 17:07:12 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:Cc:To:From:Subject:References: Mime-Version:Message-Id:In-Reply-To:Date:Reply-To:Content-ID: Content-Description:Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc :Resent-Message-ID:List-Owner; bh=L99OajAd1Wsvq8/yrYoXE260B9SPvCqucUnCLxSpqA4=; b=p79f/DQNwYtKU7nDEO96tiTYkP rzejDuJRG8zEo449+zFOG43bzwKQO4JmRVWTGC5sOMC/q7TruSOS5qFgAfdyPsZMLbwgcYiiHR1KT 5FRi6qGhO6G9NT10hInPZTemLWS0/YgRJ3f/9ozbdfslrQX4MjBS7yfmWjZxYWtA28tIGi2mzuaKI 29FoB1rWKxisEBmtXBw3drmw1cYA8ZBMhdHThzn0Dmn3XPzONuYv4BdqpTi/jtNg5K+9e1wbnszgT SeE5pZLB2yTKgYTl8j6SbOwBDh2RtV2srn5Hj4JsAS5mrIBNYmOIVnOeyTy1HZQa3qYWhxIoThQE1 dR472DMg==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1msT2s-009SEZ-E2; Wed, 01 Dec 2021 17:05:26 +0000 Received: from mail-wm1-x34a.google.com ([2a00:1450:4864:20::34a]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1msT23-009Rsh-7k for linux-arm-kernel@lists.infradead.org; Wed, 01 Dec 2021 17:04:36 +0000 Received: by mail-wm1-x34a.google.com with SMTP id g11-20020a1c200b000000b003320d092d08so12564646wmg.9 for ; Wed, 01 Dec 2021 09:04:34 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20210112; h=date:in-reply-to:message-id:mime-version:references:subject:from:to :cc; bh=R4aAsZuEycX7jTv8i7SIMGlzcxlZQ+O2BggkYaTuEjA=; b=plQ5ysBljc0xG5cfba5Qiw3VOO5L5djBwH2KOMSeq2bRcSiUiqowbYb2+TW5TZ1dcR cZWPWJlgzX7jmmLaQ4ByFFTPnDp+JrwbYmQ0pJu3qL0LL5Eo+xr1+5IlSuQo/beLH3qI JiGvDzHNZtcTCeCYXNMRt2Eu49Ss2ylezV25kVj0ijXktXiPryN7T1mPUMkDLUYRtoQ9 g5ziLntuM1OC00Zp1xER7Mdxpwh5cZbTWq+fg+F47gsCCyqUZ4vYXt6mWjWGzETLnuhs TFuCujebbXw0EIIYAgEOWZZVUNMrvnHiz1yf1Alt6UryT32jt7vGzOKDYq+Al4/E+S/J tosQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=R4aAsZuEycX7jTv8i7SIMGlzcxlZQ+O2BggkYaTuEjA=; b=BcGKgYzs+yyrCNGEAFZkkl/BiT5TozUyaLQCR9kiooZQAKMFGaaDgG7/mfbFYh1zAY oKwYVLCTOWkF+xuD9PiHV6UgPWR2JNtsDYkksDtefLpExFgzRxg+J/+73T7rY4ah9wlK VeTPgWIRBfRcX2HDeMIHt5oi48NNTSxWHXBlHpz/5C7qKEX5Ua0e2DDBNAOotyzaZ1Db SyzBDMQ0fL0Vvcv4HBGHAHAMYbrODIAukX8wlCSahodACSLqjGivJBW+sub8Ydy/rX/s NgouYehF58o6Zk3olXNGFNFQil4l7EpTPQZb5DHE0OE1+e9Xf13gvAQCLkzusrtJjeJ9 U+DQ== X-Gm-Message-State: AOAM531dqOzeDuGFYqaG1gJe14+n5ZgF6atjskzGzHbiBRLyCHlVJkuY Xz0GVHyESGmJ65Y27XY5vRju2kvZ64tC X-Google-Smtp-Source: ABdhPJzQigV4hbaNpX4iucPZvGwYvAo/hfN1TJ3QONjwQcFMPn8kkHe0dIUY1NGAIghVVqO5zssHeXhuINPH X-Received: from luke.lon.corp.google.com ([2a00:79e0:d:210:1cab:1a20:2ef1:ddde]) (user=qperret job=sendgmr) by 2002:a05:600c:4e02:: with SMTP id b2mr8493849wmq.105.1638378273705; Wed, 01 Dec 2021 09:04:33 -0800 (PST) Date: Wed, 1 Dec 2021 17:03:59 +0000 In-Reply-To: <20211201170411.1561936-1-qperret@google.com> Message-Id: <20211201170411.1561936-6-qperret@google.com> Mime-Version: 1.0 References: <20211201170411.1561936-1-qperret@google.com> X-Mailer: git-send-email 2.34.0.rc2.393.gf8c9666880-goog Subject: [PATCH v3 05/15] KVM: arm64: Hook up ->page_count() for hypervisor stage-1 page-table From: Quentin Perret To: Marc Zyngier , James Morse , Alexandru Elisei , Suzuki K Poulose , Catalin Marinas , Will Deacon Cc: linux-arm-kernel@lists.infradead.org, kvmarm@lists.cs.columbia.edu, linux-kernel@vger.kernel.org, kernel-team@android.com X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20211201_090435_314771_52FB6EB1 X-CRM114-Status: UNSURE ( 9.62 ) X-CRM114-Notice: Please train this message. X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org From: Will Deacon kvm_pgtable_hyp_unmap() relies on the ->page_count() function callback being provided by the memory-management operations for the page-table. Wire up this callback for the hypervisor stage-1 page-table. Signed-off-by: Will Deacon Signed-off-by: Quentin Perret --- arch/arm64/kvm/hyp/nvhe/setup.c | 1 + 1 file changed, 1 insertion(+) diff --git a/arch/arm64/kvm/hyp/nvhe/setup.c b/arch/arm64/kvm/hyp/nvhe/setup.c index 875b5174342f..855a19056627 100644 --- a/arch/arm64/kvm/hyp/nvhe/setup.c +++ b/arch/arm64/kvm/hyp/nvhe/setup.c @@ -257,6 +257,7 @@ void __noreturn __pkvm_init_finalise(void) .virt_to_phys = hyp_virt_to_phys, .get_page = hpool_get_page, .put_page = hpool_put_page, + .page_count = hyp_page_count, }; pkvm_pgtable.mm_ops = &pkvm_pgtable_mm_ops; From patchwork Wed Dec 1 17:04:00 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Quentin Perret X-Patchwork-Id: 12694353 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 6974CC433EF for ; Wed, 1 Dec 2021 17:07:36 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:Cc:To:From:Subject:References: Mime-Version:Message-Id:In-Reply-To:Date:Reply-To:Content-ID: Content-Description:Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc :Resent-Message-ID:List-Owner; bh=VIPohqmq37+VheAETCzob/ivt4nnO8vUYyLv0Sn4ysY=; b=t4Skr2kQcEbKeachZy7oV1hzo/ urpmx227GGVeP1sv9BytiOI0DAaVXxJ0Ppqconyw0DC/8Dj+q9//40rb61ivh/yaN6JG+lEU0uIN+ KDAZN3Yi6Me0C+nzTeBtUuTaS9poiul8xp9DwPRVr2IULJ9U5/4RgShDhESe41y1mlDViZSJxP2b1 pNqPz5B9LxTSqRKzuQo2n6jPBbFx3A1KKJ1G/E7AN/RguAe62eeq8Z3ATiq8t06apDcSNkmziNrej T7MRVS75NfZ8C8CfeSRjSfhDj/EGk051yRz8Gzui093XzfaNd32pRxdq9PrJxXLtm48Om68obWDP/ BwVF96QQ==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1msT39-009SMi-At; Wed, 01 Dec 2021 17:05:43 +0000 Received: from mail-wm1-x349.google.com ([2a00:1450:4864:20::349]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1msT27-009RtY-4T for linux-arm-kernel@lists.infradead.org; Wed, 01 Dec 2021 17:04:40 +0000 Received: by mail-wm1-x349.google.com with SMTP id g11-20020a1c200b000000b003320d092d08so12564772wmg.9 for ; Wed, 01 Dec 2021 09:04:38 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20210112; h=date:in-reply-to:message-id:mime-version:references:subject:from:to :cc; bh=3c2dJ4eKewKUxW9OPMDLajs9hR1P2X0mGpCHqHvqVFY=; b=bTlclf9xBTOAAuzZZJMyWEcfwP/C9oNdfGpG6mOtQuG9DYEudsF4CZsBALV4mKD9OQ tm/+AnZCNgyyfyWjfX3UBQmxCpc8ey7ulsU/5UowNt5+l4+NDSaCNXlsQbs+swv6jO22 sXQPKUzDX1/Q9of6po7MiBEZQqbr41FP1RjZPiiJz/s+/QpJiQGWOQico2yMAS7FpstP wPHbU9oYik16ZQEZj0XMi4J8Ws6a0pndJ2SNbXKM7Bjr1gw/d4MDyYcgXhN4mobC8WgB 4+kWCXhwKgoYsubsQQk/BD2yL1G/AbyOgLPmOwahvq2X6xHNjRBPDk8RxR0nvHzeDQRr dpkw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=3c2dJ4eKewKUxW9OPMDLajs9hR1P2X0mGpCHqHvqVFY=; b=UMjn0MnbrunhwaN5kdkd32h1SCZH5t2o53mxjHoKwBzGBjrSM3K3O+9dnEMTHY08Eo PpsxNTZM3WoajGk/lTUW+QbOfsbtfS9ULD7y2Ct1MNnXqJD5WG/h5ZEA2U+yVG9lwfMx IGINnLnZtk5zMFG5TOaSgdykF65OKZ6ps/QJuu4ESXl5/3wg3N8uZW2GDQxWAujZthiF D+Kaiz3YMhuOAfh6CWd/dhgEzfWZHfECKpIYEYzbf2x/STWDLJqTnpmP+N+Thh5ovuSk uFVeEtL+JRxPklZc7eCkxl6c0aW0ZPP2HSIB8sEmO7n5bRy/IgCntyNug8gye+pMg2Z3 I+yQ== X-Gm-Message-State: AOAM532klbJKkLlcPCO5rAy/3gCwrA1+TWACS+TqtVYPU+vOseDhcaF1 dC/6N8cI5xZT0BBXUyvgvkpwBJq8aspI X-Google-Smtp-Source: ABdhPJxYpvAHsdmsXbB1y3zMWxiK1ZKFb+wxEi407hM/usX/+75gzO/p82iTF7zix/y32s/89ZoHtckD3SQL X-Received: from luke.lon.corp.google.com ([2a00:79e0:d:210:1cab:1a20:2ef1:ddde]) (user=qperret job=sendgmr) by 2002:a05:600c:34c2:: with SMTP id d2mr8964734wmq.102.1638378277555; Wed, 01 Dec 2021 09:04:37 -0800 (PST) Date: Wed, 1 Dec 2021 17:04:00 +0000 In-Reply-To: <20211201170411.1561936-1-qperret@google.com> Message-Id: <20211201170411.1561936-7-qperret@google.com> Mime-Version: 1.0 References: <20211201170411.1561936-1-qperret@google.com> X-Mailer: git-send-email 2.34.0.rc2.393.gf8c9666880-goog Subject: [PATCH v3 06/15] KVM: arm64: Implement kvm_pgtable_hyp_unmap() at EL2 From: Quentin Perret To: Marc Zyngier , James Morse , Alexandru Elisei , Suzuki K Poulose , Catalin Marinas , Will Deacon Cc: linux-arm-kernel@lists.infradead.org, kvmarm@lists.cs.columbia.edu, linux-kernel@vger.kernel.org, kernel-team@android.com X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20211201_090439_202162_A489048A X-CRM114-Status: GOOD ( 16.00 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org From: Will Deacon Implement kvm_pgtable_hyp_unmap() which can be used to remove hypervisor stage-1 mappings at EL2. Signed-off-by: Will Deacon Signed-off-by: Quentin Perret --- arch/arm64/include/asm/kvm_pgtable.h | 21 ++++++++++ arch/arm64/kvm/hyp/pgtable.c | 63 ++++++++++++++++++++++++++++ 2 files changed, 84 insertions(+) diff --git a/arch/arm64/include/asm/kvm_pgtable.h b/arch/arm64/include/asm/kvm_pgtable.h index 027783829584..9d076f36401d 100644 --- a/arch/arm64/include/asm/kvm_pgtable.h +++ b/arch/arm64/include/asm/kvm_pgtable.h @@ -251,6 +251,27 @@ void kvm_pgtable_hyp_destroy(struct kvm_pgtable *pgt); int kvm_pgtable_hyp_map(struct kvm_pgtable *pgt, u64 addr, u64 size, u64 phys, enum kvm_pgtable_prot prot); +/** + * kvm_pgtable_hyp_unmap() - Remove a mapping from a hypervisor stage-1 page-table. + * @pgt: Page-table structure initialised by kvm_pgtable_hyp_init(). + * @addr: Virtual address from which to remove the mapping. + * @size: Size of the mapping. + * + * The offset of @addr within a page is ignored, @size is rounded-up to + * the next page boundary and @phys is rounded-down to the previous page + * boundary. + * + * TLB invalidation is performed for each page-table entry cleared during the + * unmapping operation and the reference count for the page-table page + * containing the cleared entry is decremented, with unreferenced pages being + * freed. The unmapping operation will stop early if it encounters either an + * invalid page-table entry or a valid block mapping which maps beyond the range + * being unmapped. + * + * Return: Number of bytes unmapped, which may be 0. + */ +u64 kvm_pgtable_hyp_unmap(struct kvm_pgtable *pgt, u64 addr, u64 size); + /** * kvm_get_vtcr() - Helper to construct VTCR_EL2 * @mmfr0: Sanitized value of SYS_ID_AA64MMFR0_EL1 register. diff --git a/arch/arm64/kvm/hyp/pgtable.c b/arch/arm64/kvm/hyp/pgtable.c index 768a58835153..6ad4cb2d6947 100644 --- a/arch/arm64/kvm/hyp/pgtable.c +++ b/arch/arm64/kvm/hyp/pgtable.c @@ -463,6 +463,69 @@ int kvm_pgtable_hyp_map(struct kvm_pgtable *pgt, u64 addr, u64 size, u64 phys, return ret; } +struct hyp_unmap_data { + u64 unmapped; + struct kvm_pgtable_mm_ops *mm_ops; +}; + +static int hyp_unmap_walker(u64 addr, u64 end, u32 level, kvm_pte_t *ptep, + enum kvm_pgtable_walk_flags flag, void * const arg) +{ + kvm_pte_t pte = *ptep, *childp = NULL; + u64 granule = kvm_granule_size(level); + struct hyp_unmap_data *data = arg; + struct kvm_pgtable_mm_ops *mm_ops = data->mm_ops; + + if (!kvm_pte_valid(pte)) + return -EINVAL; + + if (kvm_pte_table(pte, level)) { + childp = kvm_pte_follow(pte, mm_ops); + + if (mm_ops->page_count(childp) != 1) + return 0; + + kvm_clear_pte(ptep); + dsb(ishst); + __tlbi_level(vae2is, __TLBI_VADDR(addr, 0), level); + } else { + if (end - addr < granule) + return -EINVAL; + + kvm_clear_pte(ptep); + dsb(ishst); + __tlbi_level(vale2is, __TLBI_VADDR(addr, 0), level); + data->unmapped += granule; + } + + dsb(ish); + isb(); + mm_ops->put_page(ptep); + + if (childp) + mm_ops->put_page(childp); + + return 0; +} + +u64 kvm_pgtable_hyp_unmap(struct kvm_pgtable *pgt, u64 addr, u64 size) +{ + struct hyp_unmap_data unmap_data = { + .mm_ops = pgt->mm_ops, + }; + struct kvm_pgtable_walker walker = { + .cb = hyp_unmap_walker, + .arg = &unmap_data, + .flags = KVM_PGTABLE_WALK_LEAF | KVM_PGTABLE_WALK_TABLE_POST, + }; + + if (!pgt->mm_ops->page_count) + return 0; + + kvm_pgtable_walk(pgt, addr, size, &walker); + return unmap_data.unmapped; +} + int kvm_pgtable_hyp_init(struct kvm_pgtable *pgt, u32 va_bits, struct kvm_pgtable_mm_ops *mm_ops) { From patchwork Wed Dec 1 17:04:01 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Quentin Perret X-Patchwork-Id: 12694354 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id DFD08C433EF for ; Wed, 1 Dec 2021 17:08:03 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:Cc:To:From:Subject:References: Mime-Version:Message-Id:In-Reply-To:Date:Reply-To:Content-ID: Content-Description:Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc :Resent-Message-ID:List-Owner; bh=P9wFHqRk7dnliVJl+T+lNSFCmkxtjQeB3xCc0+wwqws=; b=k9rUnKClWfsVqBE7iOCk8q5i/1 G5PjuzGX19HsMtKg/zO+3YBpkIjv5kOPLrWDZ38c25qzQrP95OkyUNAdZOnDdyjY5bcYoUFx+x4Bi 9UZdKnBzqrQ02jiDEIWXfUbYDMT13sK4qVmegbQkk17VjrmrM7VS+r45d42RkRDF2odCMIL5mcuF8 toRdg7IR0B14iw30cvWvtWDy9UIrvoFnUG/W2eutlFRoaC9oH7Fs6JLua06/6+O5+REvAgok/8kWc 0Pv5zi9h+WAgqVM/sdcPoRn8X8xrn+heNvv4IgqmEZv4p86mjZbMVvidN20msq5G5XAHLEfydBVMU 7CslOTdA==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1msT3g-009Sde-Qq; Wed, 01 Dec 2021 17:06:17 +0000 Received: from mail-wm1-x34a.google.com ([2a00:1450:4864:20::34a]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1msT2B-009RuY-6f for linux-arm-kernel@lists.infradead.org; Wed, 01 Dec 2021 17:04:44 +0000 Received: by mail-wm1-x34a.google.com with SMTP id l6-20020a05600c4f0600b0033321934a39so1191331wmq.9 for ; Wed, 01 Dec 2021 09:04:42 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20210112; h=date:in-reply-to:message-id:mime-version:references:subject:from:to :cc; bh=NJuegLyFty4h+k4BvUryfHfsyVJ26LHh21kFNoYVPms=; b=pjEZYfEFH2f9DZZLkJPc+DJKonqmGOFU+/IndvDwaAD0Mc38KaGhN+e2+ThY9Xkoie LyXXIL01+VTsQKZGzSG7BqPQx8sGPd7M0yPwve2btEpLEMaXtEoXnnm461+DFbQ1JuSF hVKeslduzUKxJvDgBe+SxkbHjOmG/IPTUQcWOBwoZm9RXnnQTefZq8EKI/qpAGZte/ig lFFXkNlytDGtJUhJPWkYUsGKCHLOFSLpEsdcuwsG6GsTNDLC1cXDgdJHgOrcZKjbIFVC 5UvpJAycHv+RaqfhaI2HTgGTD/ahlaWvkRYDC9DdisOszJzGcDRE3EN9f/dfRsXCxmde DbRw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=NJuegLyFty4h+k4BvUryfHfsyVJ26LHh21kFNoYVPms=; b=sPKf70aanSuRwbi2TxU8I1cspdPV7zLwQsjj3iLSKaDjkzN2uuLgYdbyiFw5Xt8qjR DoKQQPTvbA881HxiCT3oQJ2quU6ZVpf/tMv4lX26Z1OstO1Ak4iUI6aEusLgTiVO6VkP dSWt7Bi/DC6BoCFCPirZ+/+do17P231EJmmIwd2qod+OaZR74ZnqRzWJPEFs5ruTy/+U AV7FmzlTqNGNQPOcauJf0ALtrTpjCcykLEB2b4N4KRcoZi/6e0U5J5O0hVtwlSztC7Rn INa7Ixe+JtjUn4fUqpkU0tqn4rmwCfXuMEtTE5AuXETbE6gYzZnlOvQAMcKhcK+Su17M bEHA== X-Gm-Message-State: AOAM533CCc5awMZfGoo8yykQCEBUQ91ZV69UHaIDbP2e4DEGhmmXE7q5 Vonq9bHgoigHfkEHOqwfx7NjUppxfivw X-Google-Smtp-Source: ABdhPJxCJlvPNxfX1Vbthcz7eOfsb4kNC+DIdOG4XHH/CTzEgT2hYZqSk1komsbwuqasr1MGCR4/qh/O+mWH X-Received: from luke.lon.corp.google.com ([2a00:79e0:d:210:1cab:1a20:2ef1:ddde]) (user=qperret job=sendgmr) by 2002:a1c:a592:: with SMTP id o140mr8609740wme.10.1638378281534; Wed, 01 Dec 2021 09:04:41 -0800 (PST) Date: Wed, 1 Dec 2021 17:04:01 +0000 In-Reply-To: <20211201170411.1561936-1-qperret@google.com> Message-Id: <20211201170411.1561936-8-qperret@google.com> Mime-Version: 1.0 References: <20211201170411.1561936-1-qperret@google.com> X-Mailer: git-send-email 2.34.0.rc2.393.gf8c9666880-goog Subject: [PATCH v3 07/15] KVM: arm64: Introduce kvm_share_hyp() From: Quentin Perret To: Marc Zyngier , James Morse , Alexandru Elisei , Suzuki K Poulose , Catalin Marinas , Will Deacon Cc: linux-arm-kernel@lists.infradead.org, kvmarm@lists.cs.columbia.edu, linux-kernel@vger.kernel.org, kernel-team@android.com X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20211201_090443_331058_0BABA951 X-CRM114-Status: GOOD ( 19.89 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org The create_hyp_mappings() function can currently be called at any point in time. However, its behaviour in protected mode changes widely depending on when it is being called. Prior to KVM init, it is used to create the temporary page-table used to bring-up the hypervisor, and later on it is transparently turned into a 'share' hypercall when the kernel has lost control over the hypervisor stage-1. In order to prepare the ground for also unsharing pages with the hypervisor during guest teardown, introduce a kvm_share_hyp() function to make it clear in which places a share hypercall should be expected, as we will soon need a matching unshare hypercall in all those places. Signed-off-by: Quentin Perret --- arch/arm64/include/asm/kvm_mmu.h | 1 + arch/arm64/kvm/arm.c | 4 ++-- arch/arm64/kvm/fpsimd.c | 2 +- arch/arm64/kvm/mmu.c | 27 +++++++++++++++++++++------ arch/arm64/kvm/reset.c | 2 +- 5 files changed, 26 insertions(+), 10 deletions(-) diff --git a/arch/arm64/include/asm/kvm_mmu.h b/arch/arm64/include/asm/kvm_mmu.h index 02d378887743..185d0f62b724 100644 --- a/arch/arm64/include/asm/kvm_mmu.h +++ b/arch/arm64/include/asm/kvm_mmu.h @@ -150,6 +150,7 @@ static __always_inline unsigned long __kern_hyp_va(unsigned long v) #include #include +int kvm_share_hyp(void *from, void *to); int create_hyp_mappings(void *from, void *to, enum kvm_pgtable_prot prot); int create_hyp_io_mappings(phys_addr_t phys_addr, size_t size, void __iomem **kaddr, diff --git a/arch/arm64/kvm/arm.c b/arch/arm64/kvm/arm.c index 9b745d2bc89a..c202abb448b1 100644 --- a/arch/arm64/kvm/arm.c +++ b/arch/arm64/kvm/arm.c @@ -146,7 +146,7 @@ int kvm_arch_init_vm(struct kvm *kvm, unsigned long type) if (ret) return ret; - ret = create_hyp_mappings(kvm, kvm + 1, PAGE_HYP); + ret = kvm_share_hyp(kvm, kvm + 1); if (ret) goto out_free_stage2_pgd; @@ -342,7 +342,7 @@ int kvm_arch_vcpu_create(struct kvm_vcpu *vcpu) if (err) return err; - return create_hyp_mappings(vcpu, vcpu + 1, PAGE_HYP); + return kvm_share_hyp(vcpu, vcpu + 1); } void kvm_arch_vcpu_postcreate(struct kvm_vcpu *vcpu) diff --git a/arch/arm64/kvm/fpsimd.c b/arch/arm64/kvm/fpsimd.c index 5526d79c7b47..86899d3aa9a9 100644 --- a/arch/arm64/kvm/fpsimd.c +++ b/arch/arm64/kvm/fpsimd.c @@ -30,7 +30,7 @@ int kvm_arch_vcpu_run_map_fp(struct kvm_vcpu *vcpu) struct user_fpsimd_state *fpsimd = ¤t->thread.uw.fpsimd_state; /* Make sure the host task fpsimd state is visible to hyp: */ - ret = create_hyp_mappings(fpsimd, fpsimd + 1, PAGE_HYP); + ret = kvm_share_hyp(fpsimd, fpsimd + 1); if (!ret) vcpu->arch.host_fpsimd_state = kern_hyp_va(fpsimd); diff --git a/arch/arm64/kvm/mmu.c b/arch/arm64/kvm/mmu.c index f8f1096a297f..fd868fb9d922 100644 --- a/arch/arm64/kvm/mmu.c +++ b/arch/arm64/kvm/mmu.c @@ -299,6 +299,25 @@ static int pkvm_share_hyp(phys_addr_t start, phys_addr_t end) return 0; } +int kvm_share_hyp(void *from, void *to) +{ + if (is_kernel_in_hyp_mode()) + return 0; + + /* + * The share hcall maps things in the 'fixed-offset' region of the hyp + * VA space, so we can only share physically contiguous data-structures + * for now. + */ + if (is_vmalloc_addr(from) || is_vmalloc_addr(to)) + return -EINVAL; + + if (kvm_host_owns_hyp_mappings()) + return create_hyp_mappings(from, to, PAGE_HYP); + + return pkvm_share_hyp(__pa(from), __pa(to)); +} + /** * create_hyp_mappings - duplicate a kernel virtual address range in Hyp mode * @from: The virtual kernel start address of the range @@ -319,12 +338,8 @@ int create_hyp_mappings(void *from, void *to, enum kvm_pgtable_prot prot) if (is_kernel_in_hyp_mode()) return 0; - if (!kvm_host_owns_hyp_mappings()) { - if (WARN_ON(prot != PAGE_HYP)) - return -EPERM; - return pkvm_share_hyp(kvm_kaddr_to_phys(from), - kvm_kaddr_to_phys(to)); - } + if (!kvm_host_owns_hyp_mappings()) + return -EPERM; start = start & PAGE_MASK; end = PAGE_ALIGN(end); diff --git a/arch/arm64/kvm/reset.c b/arch/arm64/kvm/reset.c index c7a0249df840..e3e2a79fbd75 100644 --- a/arch/arm64/kvm/reset.c +++ b/arch/arm64/kvm/reset.c @@ -113,7 +113,7 @@ static int kvm_vcpu_finalize_sve(struct kvm_vcpu *vcpu) if (!buf) return -ENOMEM; - ret = create_hyp_mappings(buf, buf + reg_sz, PAGE_HYP); + ret = kvm_share_hyp(buf, buf + reg_sz); if (ret) { kfree(buf); return ret; From patchwork Wed Dec 1 17:04:02 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Quentin Perret X-Patchwork-Id: 12694355 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 0B052C433F5 for ; Wed, 1 Dec 2021 17:08:31 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:Cc:To:From:Subject:References: Mime-Version:Message-Id:In-Reply-To:Date:Reply-To:Content-ID: Content-Description:Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc :Resent-Message-ID:List-Owner; bh=VAWtH2iuqic3+nePYqwh80KFL0Oc9sIti2b7cm3QV0c=; b=d1E9/umUj7GvDTgyk87J4RMSOm MZ7w3tBvkdwVGR3S6L/yqw9Tl9dTog4FVFrHB+DI82Y8xER4GJk7fofW2NbW6jTBnXHz7pofbdJtZ 8W29jIKoOAzoH5hQ9Qjn7O9g2maEsyRhY9diPkbxXDUngss0Eg7SIIJapjsWuCeN1V8NEH2i1i7hy qVN5bmBNT8TiDR+kBW/gdkxIaAAaXq0JMn8nlJLHWvXuiq6zAcxSu69+hMh/lVfFxlQFCkav5tqZd 80AI22xjfanNGO8nm4chDpHgzqyqA0JIIa6Wg5DHndLRdBeS2fsf8nwWZau1sQt7sAgdrNaczU+VI o/EFMsrg==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1msT4I-009T2S-K0; Wed, 01 Dec 2021 17:06:54 +0000 Received: from mail-wm1-x349.google.com ([2a00:1450:4864:20::349]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1msT2G-009RwE-1C for linux-arm-kernel@lists.infradead.org; Wed, 01 Dec 2021 17:04:49 +0000 Received: by mail-wm1-x349.google.com with SMTP id m18-20020a05600c3b1200b0033283ea5facso611274wms.1 for ; Wed, 01 Dec 2021 09:04:46 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20210112; h=date:in-reply-to:message-id:mime-version:references:subject:from:to :cc; bh=3kGqtrkDX/7YFDJqNxF3cw4ExJXKG5GDRcsjJYQ/JVo=; b=TMO4hW6gBvmBsrzVeqH/Hwa2gs9Xbv3N2Qp7sh2MQz/vWRh6mkuEatIkrGrTZf3Z4I J4ZT0n9fQuNxYUa41Q/bfrGeXJfXoAxp0fVLJYsUlcz4fJ9elw13634GMg/GtfXBD9gS HgsZuqhwLHubwI2YpiTsQ5gPYmF4oiPV0t0LYGbZQsqW5tsVfYeH3pZdLC20OCtG9SBk hhssObLi/zjJz/dfhhWUJZkRS15K+o5PXPkzjL6lAnmts21mdTmxYXoGDWcbisE6lvjw Zdp4uo94fV72wWfRF6SQkDM9TrvSF/jxNwMbnGAP2S4QVSSagWKieTx6L10xi1OZqvWY FH7g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=3kGqtrkDX/7YFDJqNxF3cw4ExJXKG5GDRcsjJYQ/JVo=; b=D1X71rE5wJCAHa5/T9Bg8xU7f7ifNtnwDAySLCl5c+aahEmUiefL7RlQSqF/C3uVtu YEALHv9jTpkomliBDBZV4qrG8l/rWmjXtuC9olnQw97UfyT49sUDQOjMtumYlg0ufad2 kneUzjfmoDMG/NHECg1LyzjjxIETJuBWaI+SWRPuafPAqqHPwtsijRDZrfRCn39CULKC pixPtP+idMhjulKQ/9rlGEtHKs88IRq056K3pMW+1skJHC8Slbq6aE8mmL/anWzfVlmW MUOroE01/Z+jueEu1aa3qcuu9pokjKlHXKGL+Nm8dIgpxkRA+2lK/NDHnSOzB6mVd5YZ Zg7g== X-Gm-Message-State: AOAM532Mz+Cr2VVohpr9CNZVQ6LQ6fTITO3s4/RXWbNt1pDfwBWWUVyX YFiZ9E27MVDgVLfpkSzPUfsUXcsDJjOB X-Google-Smtp-Source: ABdhPJzzYr8OrnKjxfmfCT1wD/gpPb9rlHIvVwT+T1gmq0Ql8fSZ2Xu5j+Ura+iJuS9E70AhlNnb1YCGJ043 X-Received: from luke.lon.corp.google.com ([2a00:79e0:d:210:1cab:1a20:2ef1:ddde]) (user=qperret job=sendgmr) by 2002:a05:600c:4e07:: with SMTP id b7mr8305933wmq.16.1638378285742; Wed, 01 Dec 2021 09:04:45 -0800 (PST) Date: Wed, 1 Dec 2021 17:04:02 +0000 In-Reply-To: <20211201170411.1561936-1-qperret@google.com> Message-Id: <20211201170411.1561936-9-qperret@google.com> Mime-Version: 1.0 References: <20211201170411.1561936-1-qperret@google.com> X-Mailer: git-send-email 2.34.0.rc2.393.gf8c9666880-goog Subject: [PATCH v3 08/15] KVM: arm64: pkvm: Refcount the pages shared with EL2 From: Quentin Perret To: Marc Zyngier , James Morse , Alexandru Elisei , Suzuki K Poulose , Catalin Marinas , Will Deacon Cc: linux-arm-kernel@lists.infradead.org, kvmarm@lists.cs.columbia.edu, linux-kernel@vger.kernel.org, kernel-team@android.com X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20211201_090448_113499_3949A67F X-CRM114-Status: GOOD ( 16.22 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org In order to simplify the page tracking infrastructure at EL2 in nVHE protected mode, move the responsibility of refcounting pages that are shared multiple times on the host. In order to do so, let's create a red-black tree tracking all the PFNs that have been shared, along with a refcount. Signed-off-by: Quentin Perret Acked-by: Will Deacon --- arch/arm64/kvm/mmu.c | 78 ++++++++++++++++++++++++++++++++++++++------ 1 file changed, 68 insertions(+), 10 deletions(-) diff --git a/arch/arm64/kvm/mmu.c b/arch/arm64/kvm/mmu.c index fd868fb9d922..d72566896755 100644 --- a/arch/arm64/kvm/mmu.c +++ b/arch/arm64/kvm/mmu.c @@ -284,23 +284,72 @@ static phys_addr_t kvm_kaddr_to_phys(void *kaddr) } } -static int pkvm_share_hyp(phys_addr_t start, phys_addr_t end) +struct hyp_shared_pfn { + u64 pfn; + int count; + struct rb_node node; +}; + +static DEFINE_MUTEX(hyp_shared_pfns_lock); +static struct rb_root hyp_shared_pfns = RB_ROOT; + +static struct hyp_shared_pfn *find_shared_pfn(u64 pfn, struct rb_node ***node, + struct rb_node **parent) { - phys_addr_t addr; - int ret; + struct hyp_shared_pfn *this; + + *node = &hyp_shared_pfns.rb_node; + *parent = NULL; + while (**node) { + this = container_of(**node, struct hyp_shared_pfn, node); + *parent = **node; + if (this->pfn < pfn) + *node = &((**node)->rb_left); + else if (this->pfn > pfn) + *node = &((**node)->rb_right); + else + return this; + } - for (addr = ALIGN_DOWN(start, PAGE_SIZE); addr < end; addr += PAGE_SIZE) { - ret = kvm_call_hyp_nvhe(__pkvm_host_share_hyp, - __phys_to_pfn(addr)); - if (ret) - return ret; + return NULL; +} + +static int share_pfn_hyp(u64 pfn) +{ + struct rb_node **node, *parent; + struct hyp_shared_pfn *this; + int ret = 0; + + mutex_lock(&hyp_shared_pfns_lock); + this = find_shared_pfn(pfn, &node, &parent); + if (this) { + this->count++; + goto unlock; } - return 0; + this = kzalloc(sizeof(*this), GFP_KERNEL); + if (!this) { + ret = -ENOMEM; + goto unlock; + } + + this->pfn = pfn; + this->count = 1; + rb_link_node(&this->node, parent, node); + rb_insert_color(&this->node, &hyp_shared_pfns); + ret = kvm_call_hyp_nvhe(__pkvm_host_share_hyp, pfn, 1); +unlock: + mutex_unlock(&hyp_shared_pfns_lock); + + return ret; } int kvm_share_hyp(void *from, void *to) { + phys_addr_t start, end, cur; + u64 pfn; + int ret; + if (is_kernel_in_hyp_mode()) return 0; @@ -315,7 +364,16 @@ int kvm_share_hyp(void *from, void *to) if (kvm_host_owns_hyp_mappings()) return create_hyp_mappings(from, to, PAGE_HYP); - return pkvm_share_hyp(__pa(from), __pa(to)); + start = ALIGN_DOWN(__pa(from), PAGE_SIZE); + end = PAGE_ALIGN(__pa(to)); + for (cur = start; cur < end; cur += PAGE_SIZE) { + pfn = __phys_to_pfn(cur); + ret = share_pfn_hyp(pfn); + if (ret) + return ret; + } + + return 0; } /** From patchwork Wed Dec 1 17:04:03 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Quentin Perret X-Patchwork-Id: 12694356 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 4945BC433F5 for ; Wed, 1 Dec 2021 17:09:03 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:Cc:To:From:Subject:References: Mime-Version:Message-Id:In-Reply-To:Date:Reply-To:Content-ID: Content-Description:Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc :Resent-Message-ID:List-Owner; bh=MdFkmrOatTo01WhLcL5nqtjoIAh2uykTaVXPVeCZln8=; b=hBItg0hEuV6exUggK/c5HqonWk xVDDc9wc7RvSdfAyRcx3ebOnnNcXgx2LUvf3M6ab+BInXl6TdTGOfARdOTHpzNVbChYYeI+VZUnA7 d86bJ3+Sd0yTOT1deMrcvO7iY0wUYRD7+yHBz1cw9IO3aUgKcoaSJdf1e104oUUwyLkbnipB2RVmW FQakT5JOWfMsLoy72USulthKuMVjXFck/3/7EZbD+IOOkvx1W+NPQBRnjPcnlD/vXJ9n73dkdjY9r Bf+2kZNtGDfke1NVosE0Wdq7Jk5/6QB5bekZEmcQRx6+7p+5E0jL5voUyNte5SmCktpkYnpIKwkIG sbuw6DLQ==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1msT4o-009THs-3i; Wed, 01 Dec 2021 17:07:26 +0000 Received: from mail-wr1-x44a.google.com ([2a00:1450:4864:20::44a]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1msT2K-009RxW-5e for linux-arm-kernel@lists.infradead.org; Wed, 01 Dec 2021 17:04:53 +0000 Received: by mail-wr1-x44a.google.com with SMTP id q5-20020a5d5745000000b00178abb72486so4461467wrw.9 for ; Wed, 01 Dec 2021 09:04:51 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20210112; h=date:in-reply-to:message-id:mime-version:references:subject:from:to :cc; bh=gP7Fa7oF1mk2VL/C80BdOcZ11ucSfvyvrK5tI2TJOao=; b=YIFEpSMiiphVp00kT4SqQiVFlCZjWeW7pHUFlPbzlz6LipVw2aWy7Rd6tb3JUzJM+3 QKIISnMlREKuRhzWfT+DUQt86OHAlAejeH8a+q5aOiFs7nF9U2+bewCuES5TKaRshu2s qH0lsFmh+oO1cw1jDqXHNIc8iZWQ2FkeeVNPRVURvz8RBrMKrSNdu28aFetakrGmqP// ult2AtCbkCkC9p4XjEbc6jcacWZURDLmr7Iv9xCzICqAYxduRH8LXhXBrpLmELev6Puf 5SG+5mVlcaSiNM5tx0UO0lC2C2y3/Zrd77xHXrAIHeEj5kfiRwqdzDjzXaSZxaueQr4u SamA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=gP7Fa7oF1mk2VL/C80BdOcZ11ucSfvyvrK5tI2TJOao=; b=Gv0mtXxNmdzoEcLn1pR6OK4WSU/WjWp5TGt8vf5s1UZj7/mDD0Mrl6aCQsrNGchiU/ Mn6T88Blz/mYQq6WEBMNlmVAEELKkqU2lkyipqIJ38E9TJvu8CKqT/8dRqJHxswPDWLH Wg3vyDOJpeuqDVDyMJ37D+w14+1qWzKMOqWFZctxDu99RdjpwgRSQjstpwPv7DOF8QBL pbqjKB4rOaUmU0bQ8J9ysZbLnzOPYWVTD1RtnZejpR4ondAouBcO0m/fpbOpjHQiwyEr xyZa5UBtWorIGRO1JipvzTNir0vlrFfiI4D9ZmmV63T1OdoSWt0B5g4NiuAXtR2zieyV n40w== X-Gm-Message-State: AOAM532aOurTAhe9iaotbsLNEp6d7QEsgmeUppZeLxLBVU8xD7EZvyEl d3sj2SKK07BJ8zbLKwERgZ2Hh9BZKKW+ X-Google-Smtp-Source: ABdhPJxYTjRVBjX0Rb9ACwgvUv8SYqRjTELJGdvUGk2ZT9j9VAZxKEYlzDVuho8VN54x8a7VK7k6aXTc82QW X-Received: from luke.lon.corp.google.com ([2a00:79e0:d:210:1cab:1a20:2ef1:ddde]) (user=qperret job=sendgmr) by 2002:a05:600c:354f:: with SMTP id i15mr8597282wmq.59.1638378289941; Wed, 01 Dec 2021 09:04:49 -0800 (PST) Date: Wed, 1 Dec 2021 17:04:03 +0000 In-Reply-To: <20211201170411.1561936-1-qperret@google.com> Message-Id: <20211201170411.1561936-10-qperret@google.com> Mime-Version: 1.0 References: <20211201170411.1561936-1-qperret@google.com> X-Mailer: git-send-email 2.34.0.rc2.393.gf8c9666880-goog Subject: [PATCH v3 09/15] KVM: arm64: Extend pkvm_page_state enumeration to handle absent pages From: Quentin Perret To: Marc Zyngier , James Morse , Alexandru Elisei , Suzuki K Poulose , Catalin Marinas , Will Deacon Cc: linux-arm-kernel@lists.infradead.org, kvmarm@lists.cs.columbia.edu, linux-kernel@vger.kernel.org, kernel-team@android.com X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20211201_090452_240649_82960807 X-CRM114-Status: GOOD ( 12.27 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org From: Will Deacon Explicitly name the combination of SW0 | SW1 as reserved in the pte and introduce a new PKVM_NOPAGE meta-state which, although not directly stored in the software bits of the pte, can be used to represent an entry for which there is no underlying page. This is distinct from an invalid pte, as stage-2 identity mappings for the host are created lazily and so an invalid pte there is the same as a valid mapping for the purposes of ownership information. This state will be used for permission checking during page transitions in later patches. Signed-off-by: Will Deacon Signed-off-by: Quentin Perret Reviewed-by: Andrew Walbran --- arch/arm64/kvm/hyp/include/nvhe/mem_protect.h | 5 +++++ 1 file changed, 5 insertions(+) diff --git a/arch/arm64/kvm/hyp/include/nvhe/mem_protect.h b/arch/arm64/kvm/hyp/include/nvhe/mem_protect.h index b58c910babaf..56445586c755 100644 --- a/arch/arm64/kvm/hyp/include/nvhe/mem_protect.h +++ b/arch/arm64/kvm/hyp/include/nvhe/mem_protect.h @@ -24,6 +24,11 @@ enum pkvm_page_state { PKVM_PAGE_OWNED = 0ULL, PKVM_PAGE_SHARED_OWNED = KVM_PGTABLE_PROT_SW0, PKVM_PAGE_SHARED_BORROWED = KVM_PGTABLE_PROT_SW1, + __PKVM_PAGE_RESERVED = KVM_PGTABLE_PROT_SW0 | + KVM_PGTABLE_PROT_SW1, + + /* Meta-states which aren't encoded directly in the PTE's SW bits */ + PKVM_NOPAGE, }; #define PKVM_PAGE_STATE_PROT_MASK (KVM_PGTABLE_PROT_SW0 | KVM_PGTABLE_PROT_SW1) From patchwork Wed Dec 1 17:04:04 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Quentin Perret X-Patchwork-Id: 12694357 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 3D4DEC433EF for ; Wed, 1 Dec 2021 17:09:43 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:Cc:To:From:Subject:References: Mime-Version:Message-Id:In-Reply-To:Date:Reply-To:Content-ID: Content-Description:Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc :Resent-Message-ID:List-Owner; bh=Gi+RS31tgR+uVr6egEVZ1GPdiwBZ/X6dJdnEvDWCta4=; b=S5ton3rNacggEN/5FUvlrU4zpt RXroeFpG0cqPUPXZd2/SGIB+CYM7FBzYKFyAYLFBCmn43cP3K4j+5WNrILi4duyr+UQn3Kf9PEqNc 2TMJbErsPf80aTGu1k7V17PEhjBAqwIV6MGP2LG+TzCnvXXIpGBkb1hvliuWjXUwRJuu5ctRISJdG rR6ZYv57JTftnkm/ypnj5yLGpoeVavcKqk+b6EmhXwzraP+qwsTResCsugWAeVNU47i2hQrPlMVyd QmM6aLbmt3SZ6BJldVhR1OL0WdX6WiONt/VsDXa3W2PGD2Q4yjkKTrMjfeJd4AUyKB3o6wVEtUYPN PP3IXd2A==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1msT5R-009TeM-A8; Wed, 01 Dec 2021 17:08:05 +0000 Received: from mail-wr1-x44a.google.com ([2a00:1450:4864:20::44a]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1msT2N-009Rzk-Jt for linux-arm-kernel@lists.infradead.org; Wed, 01 Dec 2021 17:04:57 +0000 Received: by mail-wr1-x44a.google.com with SMTP id r2-20020adfe682000000b00198af042b0dso4462675wrm.23 for ; Wed, 01 Dec 2021 09:04:54 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20210112; h=date:in-reply-to:message-id:mime-version:references:subject:from:to :cc; bh=T0JsrzG68YfPigMGVPlPrd/xCSmb5pbfUw0MQLeIq18=; b=FxwXE/GBzpALB+WDGk4xxS/qk7AEHYIAQOFqBsax91XFh54Oe4XTkZKSH5mjhhiGwi xojuzI07IO1NX0heyd2/dn4DPfxq0MBzpigfJwCCo9EWFe/7k5cjD6K87Hr1bnzMr75n LPvlTZ5UVckhBdhWY1tA5gp6BooYMA6vk6UqpvQCQKxmxXijzAAUxOLQHmq3W/UT5K76 LLOzTHeRcv7HG31p0iL095jbrbR4CWaw0qd/Lm3WvIAX+mRlmQSBbLi4bERT3mmAXGm9 giXC8vRUXTgmyMn6BJyiPcxZQDAjsrAMk92/xCu0oed2MKOEUxmlhKYTnIEcRcGVQi/j d5Rg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=T0JsrzG68YfPigMGVPlPrd/xCSmb5pbfUw0MQLeIq18=; b=6uUkiWtjs/5pmkCKAzQS7pB70nnFWlimMgd3pUA05y6yO4bUp5wqUOVS1ZfNfkuClP rBzDgiO5X1syrjxnltvRMZDD7I++lbsT9qiFTi/3sCwyPysFe9nN/us/qUdcptLOsGW4 EAPy39GZo57FOlheVymwCyBerF01HB5On+T0qMy7hTHqkgvYnZDJfcORu24y73iihWyL lHz36SqbkOf7DkaZKyUiKh9EErS5AZfB4Ct2Rrek2i+By/I4QrOqf7Qm82s39+Wcuxqb QsPjZtxoVnKgrksqGT16zfNa46eNnqXwPmVPLbchWyUHjq8frLR8141TLIlzzUvhDT7k UfSQ== X-Gm-Message-State: AOAM532MNlMqj56YE1dZ1K4fhALBJLK0oUi+tUgO/qxqGa8pjtUSP4IF t7Y2Ogq1odQw0Sa8cU7GfnqWCDSx0YKJ X-Google-Smtp-Source: ABdhPJzxm+jj5EWvAYdKtX2YJJnnLDNfrgC/7WKMyffToJZDquq3DYO27TbqsoOCOfqh0AkV6YTZ7zDrqd13 X-Received: from luke.lon.corp.google.com ([2a00:79e0:d:210:1cab:1a20:2ef1:ddde]) (user=qperret job=sendgmr) by 2002:adf:fb09:: with SMTP id c9mr8021924wrr.223.1638378293678; Wed, 01 Dec 2021 09:04:53 -0800 (PST) Date: Wed, 1 Dec 2021 17:04:04 +0000 In-Reply-To: <20211201170411.1561936-1-qperret@google.com> Message-Id: <20211201170411.1561936-11-qperret@google.com> Mime-Version: 1.0 References: <20211201170411.1561936-1-qperret@google.com> X-Mailer: git-send-email 2.34.0.rc2.393.gf8c9666880-goog Subject: [PATCH v3 10/15] KVM: arm64: Introduce wrappers for host and hyp spin lock accessors From: Quentin Perret To: Marc Zyngier , James Morse , Alexandru Elisei , Suzuki K Poulose , Catalin Marinas , Will Deacon Cc: linux-arm-kernel@lists.infradead.org, kvmarm@lists.cs.columbia.edu, linux-kernel@vger.kernel.org, kernel-team@android.com X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20211201_090455_720839_6375A326 X-CRM114-Status: GOOD ( 13.20 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org From: Will Deacon In preparation for adding additional locked sections for manipulating page-tables at EL2, introduce some simple wrappers around the host and hypervisor locks so that it's a bit easier to read and bit more difficult to take the wrong lock (or even take them in the wrong order). Signed-off-by: Will Deacon Signed-off-by: Quentin Perret --- arch/arm64/kvm/hyp/nvhe/mem_protect.c | 32 ++++++++++++++++++++++----- 1 file changed, 26 insertions(+), 6 deletions(-) diff --git a/arch/arm64/kvm/hyp/nvhe/mem_protect.c b/arch/arm64/kvm/hyp/nvhe/mem_protect.c index c1a90dd022b8..757dfefe3aeb 100644 --- a/arch/arm64/kvm/hyp/nvhe/mem_protect.c +++ b/arch/arm64/kvm/hyp/nvhe/mem_protect.c @@ -27,6 +27,26 @@ static struct hyp_pool host_s2_pool; const u8 pkvm_hyp_id = 1; +static void host_lock_component(void) +{ + hyp_spin_lock(&host_kvm.lock); +} + +static void host_unlock_component(void) +{ + hyp_spin_unlock(&host_kvm.lock); +} + +static void hyp_lock_component(void) +{ + hyp_spin_lock(&pkvm_pgd_lock); +} + +static void hyp_unlock_component(void) +{ + hyp_spin_unlock(&pkvm_pgd_lock); +} + static void *host_s2_zalloc_pages_exact(size_t size) { void *addr = hyp_alloc_pages(&host_s2_pool, get_order(size)); @@ -338,14 +358,14 @@ static int host_stage2_idmap(u64 addr) prot = is_memory ? PKVM_HOST_MEM_PROT : PKVM_HOST_MMIO_PROT; - hyp_spin_lock(&host_kvm.lock); + host_lock_component(); ret = host_stage2_adjust_range(addr, &range); if (ret) goto unlock; ret = host_stage2_idmap_locked(range.start, range.end - range.start, prot); unlock: - hyp_spin_unlock(&host_kvm.lock); + host_unlock_component(); return ret; } @@ -369,8 +389,8 @@ int __pkvm_host_share_hyp(u64 pfn) if (!addr_is_memory(addr)) return -EINVAL; - hyp_spin_lock(&host_kvm.lock); - hyp_spin_lock(&pkvm_pgd_lock); + host_lock_component(); + hyp_lock_component(); ret = kvm_pgtable_get_leaf(&host_kvm.pgt, addr, &pte, NULL); if (ret) @@ -432,8 +452,8 @@ int __pkvm_host_share_hyp(u64 pfn) BUG_ON(ret); unlock: - hyp_spin_unlock(&pkvm_pgd_lock); - hyp_spin_unlock(&host_kvm.lock); + hyp_unlock_component(); + host_unlock_component(); return ret; } From patchwork Wed Dec 1 17:04:05 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Quentin Perret X-Patchwork-Id: 12694358 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 8974DC433F5 for ; Wed, 1 Dec 2021 17:10:30 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:Cc:To:From:Subject:References: Mime-Version:Message-Id:In-Reply-To:Date:Reply-To:Content-ID: Content-Description:Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc :Resent-Message-ID:List-Owner; bh=NTuYAAq/lugGk3qmlb8krJIkGU+5G6ifyqPXgQauWio=; b=VBYz3lYKk1WCPyU10IwnLjeIUJ C/dWYDmgG1K7bXUWdVANPXDOp12z2eDweCsL9lvcHJ2oMONFiWxcx+IVZKQyavUNhE1xHMJN+SsTB u6fRNDz49V1uSEes3JClZ/5RjMgktZOMMxGlVzKMIO9Rr7QjNs7hMt1hn7kZ/S28Q51Z/oD82zClc PhnwFA70GfD2kapPSlda2fzFoAHEIlFheKLPJVJIxHJxAZyxU5UU7I/sduIVUwc1KJo1lm6wI4vHz 2k+yWTUxrf7wDhEj/eq/+EQtQMWTpDeGxvvHE56SumyaG5RtecxG72Ya/FfpyAjmHlzk0/ytj+7nZ uDQU7ILw==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1msT66-009Twk-3Z; Wed, 01 Dec 2021 17:08:46 +0000 Received: from mail-wm1-x349.google.com ([2a00:1450:4864:20::349]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1msT2R-009S2I-LF for linux-arm-kernel@lists.infradead.org; Wed, 01 Dec 2021 17:05:01 +0000 Received: by mail-wm1-x349.google.com with SMTP id j25-20020a05600c1c1900b00332372c252dso1208607wms.1 for ; Wed, 01 Dec 2021 09:04:59 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20210112; h=date:in-reply-to:message-id:mime-version:references:subject:from:to :cc; bh=cppIKzBLivpZhTWcdBQA9iBtJc2y/Q/PH/HIg10Qqik=; b=jue7MMjEyJR/NSAOBXbh6RRCgkAT7LIskDpH3d6p1qSg1N6dZmSISrPvzzp3P6S4OX C7Fuy9mt2Oi7OQRJX6Vmn14Sd1ALG7bsYYAHqoQg1jHu8JWsWafrMiwGijIQ4hfFXrX3 nFhudB62JkYGJ7ujHx2cvdyxgNJ0Z8nxeLhdSdLRJo4WTk/25O7z/CxGwycteeoLXVsk hqf0RVLMjRKzw4PWvg/3lhj4DDOLvJBijKv5opZgLbYKdpw+eKwrLZuGeHDCTdi6YRzB Ep/X/H0s/L6cBle+o/sHThsBB8nlEnjkfiO3hOKxVCHjDGeUyxnZU/uKQ0XdziEg99xu 12oQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=cppIKzBLivpZhTWcdBQA9iBtJc2y/Q/PH/HIg10Qqik=; b=veZCjtvAba/v3ClPV730G58VkLOEANF84mHbuV5y9/p28fzJB6OOhVmDIYr4K5dhHT Rm64IU/ZqUVu/BaCvGYlE49B04WaqMK1n25Aad46pgx/s2/VW238M8cj/OQ3lB5IaY09 nwqcEQuG0m8T/U2k7EtwrgnT4m5vz8hT6/eFKwZioaySSQT8bmASyMLaJjLlom01blaO j93q+AgD66rJmHrWWm3GrWVQcnlKIEcWh1GXgVkSLIGpJ8DrBgBOAQFWzY0YjFd4MdUd EmkPuuxCu5FPdzEy1848Ph4R1utApJqB8gw8nnKi887nEQC8VOC/M0d/UM07a933a6hg nnyg== X-Gm-Message-State: AOAM530fpLUFFmrkiIecS5laec4M6g1dV3ZECOPb90KUDs2lE3szMnQQ ubRx2R2RtY5yDrqCFlYd+Lo6N7Qf0BbO X-Google-Smtp-Source: ABdhPJz2JxagZOtuzHEfphfl/KUPvg8AaYMdPF6mH04O7NOiowXuH0F1OwswHFbG+B3odTGAwzouRDQlOmSe X-Received: from luke.lon.corp.google.com ([2a00:79e0:d:210:1cab:1a20:2ef1:ddde]) (user=qperret job=sendgmr) by 2002:adf:f589:: with SMTP id f9mr8004315wro.505.1638378297567; Wed, 01 Dec 2021 09:04:57 -0800 (PST) Date: Wed, 1 Dec 2021 17:04:05 +0000 In-Reply-To: <20211201170411.1561936-1-qperret@google.com> Message-Id: <20211201170411.1561936-12-qperret@google.com> Mime-Version: 1.0 References: <20211201170411.1561936-1-qperret@google.com> X-Mailer: git-send-email 2.34.0.rc2.393.gf8c9666880-goog Subject: [PATCH v3 11/15] KVM: arm64: Implement do_share() helper for sharing memory From: Quentin Perret To: Marc Zyngier , James Morse , Alexandru Elisei , Suzuki K Poulose , Catalin Marinas , Will Deacon Cc: linux-arm-kernel@lists.infradead.org, kvmarm@lists.cs.columbia.edu, linux-kernel@vger.kernel.org, kernel-team@android.com X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20211201_090459_794440_0573916C X-CRM114-Status: GOOD ( 15.44 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org From: Will Deacon By default, protected KVM isolates memory pages so that they are accessible only to their owner: be it the host kernel, the hypervisor at EL2 or (in future) the guest. Establishing shared-memory regions between these components therefore involves a transition for each page so that the owner can share memory with a borrower under a certain set of permissions. Introduce a do_share() helper for safely sharing a memory region between two components. Currently, only host-to-hyp sharing is implemented, but the code is easily extended to handle other combinations and the permission checks for each component are reusable. Signed-off-by: Will Deacon Signed-off-by: Quentin Perret Reviewed-by: Andrew Walbran --- arch/arm64/kvm/hyp/nvhe/mem_protect.c | 237 ++++++++++++++++++++++++++ 1 file changed, 237 insertions(+) diff --git a/arch/arm64/kvm/hyp/nvhe/mem_protect.c b/arch/arm64/kvm/hyp/nvhe/mem_protect.c index 757dfefe3aeb..74ca4043b08a 100644 --- a/arch/arm64/kvm/hyp/nvhe/mem_protect.c +++ b/arch/arm64/kvm/hyp/nvhe/mem_protect.c @@ -471,3 +471,240 @@ void handle_host_mem_abort(struct kvm_cpu_context *host_ctxt) ret = host_stage2_idmap(addr); BUG_ON(ret && ret != -EAGAIN); } + +/* This corresponds to locking order */ +enum pkvm_component_id { + PKVM_ID_HOST, + PKVM_ID_HYP, +}; + +struct pkvm_mem_transition { + u64 nr_pages; + + struct { + enum pkvm_component_id id; + /* Address in the initiator's address space */ + u64 addr; + + union { + struct { + /* Address in the completer's address space */ + u64 completer_addr; + } host; + }; + } initiator; + + struct { + enum pkvm_component_id id; + } completer; +}; + +struct pkvm_mem_share { + const struct pkvm_mem_transition tx; + const enum kvm_pgtable_prot prot; +}; + +struct check_walk_data { + enum pkvm_page_state desired; + enum pkvm_page_state (*get_page_state)(kvm_pte_t pte); +}; + +static int __check_page_state_visitor(u64 addr, u64 end, u32 level, + kvm_pte_t *ptep, + enum kvm_pgtable_walk_flags flag, + void * const arg) +{ + struct check_walk_data *d = arg; + kvm_pte_t pte = *ptep; + + if (kvm_pte_valid(pte) && !addr_is_memory(kvm_pte_to_phys(pte))) + return -EINVAL; + + return d->get_page_state(pte) == d->desired ? 0 : -EPERM; +} + +static int check_page_state_range(struct kvm_pgtable *pgt, u64 addr, u64 size, + struct check_walk_data *data) +{ + struct kvm_pgtable_walker walker = { + .cb = __check_page_state_visitor, + .arg = data, + .flags = KVM_PGTABLE_WALK_LEAF, + }; + + return kvm_pgtable_walk(pgt, addr, size, &walker); +} + +static enum pkvm_page_state host_get_page_state(kvm_pte_t pte) +{ + if (!kvm_pte_valid(pte) && pte) + return PKVM_NOPAGE; + + return pkvm_getstate(kvm_pgtable_stage2_pte_prot(pte)); +} + +static int __host_check_page_state_range(u64 addr, u64 size, + enum pkvm_page_state state) +{ + struct check_walk_data d = { + .desired = state, + .get_page_state = host_get_page_state, + }; + + hyp_assert_lock_held(&host_kvm.lock); + return check_page_state_range(&host_kvm.pgt, addr, size, &d); +} + +static int __host_set_page_state_range(u64 addr, u64 size, + enum pkvm_page_state state) +{ + enum kvm_pgtable_prot prot = pkvm_mkstate(PKVM_HOST_MEM_PROT, state); + + return host_stage2_idmap_locked(addr, size, prot); +} + +static int host_request_owned_transition(u64 *completer_addr, + const struct pkvm_mem_transition *tx) +{ + u64 size = tx->nr_pages * PAGE_SIZE; + u64 addr = tx->initiator.addr; + + *completer_addr = tx->initiator.host.completer_addr; + return __host_check_page_state_range(addr, size, PKVM_PAGE_OWNED); +} + +static int host_initiate_share(u64 *completer_addr, + const struct pkvm_mem_transition *tx) +{ + u64 size = tx->nr_pages * PAGE_SIZE; + u64 addr = tx->initiator.addr; + + *completer_addr = tx->initiator.host.completer_addr; + return __host_set_page_state_range(addr, size, PKVM_PAGE_SHARED_OWNED); +} + +static enum pkvm_page_state hyp_get_page_state(kvm_pte_t pte) +{ + if (!kvm_pte_valid(pte)) + return PKVM_NOPAGE; + + return pkvm_getstate(kvm_pgtable_stage2_pte_prot(pte)); +} + +static int __hyp_check_page_state_range(u64 addr, u64 size, + enum pkvm_page_state state) +{ + struct check_walk_data d = { + .desired = state, + .get_page_state = hyp_get_page_state, + }; + + hyp_assert_lock_held(&pkvm_pgd_lock); + return check_page_state_range(&pkvm_pgtable, addr, size, &d); +} + +static bool __hyp_ack_skip_pgtable_check(const struct pkvm_mem_transition *tx) +{ + return !(IS_ENABLED(CONFIG_NVHE_EL2_DEBUG) || + tx->initiator.id != PKVM_ID_HOST); +} + +static int hyp_ack_share(u64 addr, const struct pkvm_mem_transition *tx, + enum kvm_pgtable_prot perms) +{ + u64 size = tx->nr_pages * PAGE_SIZE; + + if (perms != PAGE_HYP) + return -EPERM; + + if (__hyp_ack_skip_pgtable_check(tx)) + return 0; + + return __hyp_check_page_state_range(addr, size, PKVM_NOPAGE); +} + +static int hyp_complete_share(u64 addr, const struct pkvm_mem_transition *tx, + enum kvm_pgtable_prot perms) +{ + void *start = (void *)addr, *end = start + (tx->nr_pages * PAGE_SIZE); + enum kvm_pgtable_prot prot; + + prot = pkvm_mkstate(perms, PKVM_PAGE_SHARED_BORROWED); + return pkvm_create_mappings_locked(start, end, prot); +} + +static int check_share(struct pkvm_mem_share *share) +{ + const struct pkvm_mem_transition *tx = &share->tx; + u64 completer_addr; + int ret; + + switch (tx->initiator.id) { + case PKVM_ID_HOST: + ret = host_request_owned_transition(&completer_addr, tx); + break; + default: + ret = -EINVAL; + } + + if (ret) + return ret; + + switch (tx->completer.id) { + case PKVM_ID_HYP: + ret = hyp_ack_share(completer_addr, tx, share->prot); + break; + default: + ret = -EINVAL; + } + + return ret; +} + +static int __do_share(struct pkvm_mem_share *share) +{ + const struct pkvm_mem_transition *tx = &share->tx; + u64 completer_addr; + int ret; + + switch (tx->initiator.id) { + case PKVM_ID_HOST: + ret = host_initiate_share(&completer_addr, tx); + break; + default: + ret = -EINVAL; + } + + if (ret) + return ret; + + switch (tx->completer.id) { + case PKVM_ID_HYP: + ret = hyp_complete_share(completer_addr, tx, share->prot); + break; + default: + ret = -EINVAL; + } + + return ret; +} + +/* + * do_share(): + * + * The page owner grants access to another component with a given set + * of permissions. + * + * Initiator: OWNED => SHARED_OWNED + * Completer: NOPAGE => SHARED_BORROWED + */ +static int do_share(struct pkvm_mem_share *share) +{ + int ret; + + ret = check_share(share); + if (ret) + return ret; + + return WARN_ON(__do_share(share)); +} From patchwork Wed Dec 1 17:04:06 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Quentin Perret X-Patchwork-Id: 12694359 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 3E0CEC433EF for ; Wed, 1 Dec 2021 17:11:22 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:Cc:To:From:Subject:References: Mime-Version:Message-Id:In-Reply-To:Date:Reply-To:Content-ID: Content-Description:Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc :Resent-Message-ID:List-Owner; bh=qFyanYuuAFcAd/9K5QxnCl2MN1b9drC2xtT32vlkZ+s=; b=VTgwNh7Z/2juLuFeHrmTchasMI /aJlS3ASNpd5RbARqilIPEz4vMrayZbYOSw2y9fSaNP13N8/9iyHNt5JY79Kd1OLFXHEA8RWKJfhB HjSfGHpyyQvZc8UhC+aqDFb+fQWygRlA0fJ0+MHODMRuzqRheKArRYl/zJh8DPw4LusR3rmqzQLmJ Gw0PIrBvGadHR+b+8QCVjHq4GTiZlDtlNz/iU/dO7W1Il161l8YRgl42HmpM+6BwOx0SijQDDD/xj 0xzwZyyAZQzka9g1oic6Tb53TgEZnPum/walIeMOesz6eMwGVUTRakwiNOLa1KNJO2sL2Ao8B3m8w Ib0xC7pQ==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1msT6n-009UGq-Ka; Wed, 01 Dec 2021 17:09:30 +0000 Received: from mail-wr1-x44a.google.com ([2a00:1450:4864:20::44a]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1msT2V-009S4W-D9 for linux-arm-kernel@lists.infradead.org; Wed, 01 Dec 2021 17:05:04 +0000 Received: by mail-wr1-x44a.google.com with SMTP id p3-20020a056000018300b00186b195d4ddso4461493wrx.15 for ; Wed, 01 Dec 2021 09:05:02 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20210112; h=date:in-reply-to:message-id:mime-version:references:subject:from:to :cc; bh=5wiVz7SPfRWryEm8NS78CviZAnf1vSptd9MDigwGN/4=; b=dCzRG28Wf9Hc1Tp8I8w9ve3Vw14cFJY4sy6ylndkDIvYcMBINBhYfEJQuT0z5jDNv1 tGtQIA4ScW62n9Y8NpIUEIAJPgcSiROzIIIoCmndHzXPxCyEyLTWbrESkPqQWrUF0K3p 9QOXgt39AuSkMUAbxmR0DvTI8l7eczc4wPKt+PgNh5XCms6IJXMamETMAzof/W4pXV0T wnKviJ2j0cYtPfs9cxEqyXVW4TErhwlx3FItlrJ0wzbTGP5qUqcJTqBUJ9hz1JfBV9P4 ym+hRGvYhdZTese4GXjjv7hVrbAqdUmjbu/PvO4zHWFnvI9MLZYIxoYlqL/MNIHhDMLB fBTQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=5wiVz7SPfRWryEm8NS78CviZAnf1vSptd9MDigwGN/4=; b=bbmRhaBdoGoELQg4Id4zIHTe9wZk0DegtSsoXn4zGrP9TbLcfpbsC1eHAX9iF8ura5 lYwOK2ITQv20RoNDZ2OzwFj2P2scsoY+CDvi3gHUFxpi/hIvklkeq4AwbZKcducvaZQF CweI83hXPDUr8KoNH+Yk+Tl5x15c7gS/6dJE6toVyM0Urf/RiNxorf05Qc76cMYtfq9H MpJ1T6idlen1TbKRDZSy8hVTN55uy2m4NMMLF6sdXpAjUSw/A//zRZkvdnzNFadk1Sh5 MpPE3oWDq2o1eEXG3GKIwgDQliU1r91CZubP/PhotcIQht7Fv+d5JCa0f7Qs2hKx00uT A0JQ== X-Gm-Message-State: AOAM530tzU2W2tJoLbHlp1PwN5NhXF3HHs4Fka5hefeJ/yPzOtDv/i8X FMOJbJ4pUctyt4kubT8cgd2aKAnd91dp X-Google-Smtp-Source: ABdhPJwwAeEl/fBZkORhX4MxHC0CwsUHrjrNI+Tq88kiP5Hx0+90GQIUxvIJUedAX1Syxs87ImOEfCacsfMj X-Received: from luke.lon.corp.google.com ([2a00:79e0:d:210:1cab:1a20:2ef1:ddde]) (user=qperret job=sendgmr) by 2002:adf:ee04:: with SMTP id y4mr8657629wrn.0.1638378301567; Wed, 01 Dec 2021 09:05:01 -0800 (PST) Date: Wed, 1 Dec 2021 17:04:06 +0000 In-Reply-To: <20211201170411.1561936-1-qperret@google.com> Message-Id: <20211201170411.1561936-13-qperret@google.com> Mime-Version: 1.0 References: <20211201170411.1561936-1-qperret@google.com> X-Mailer: git-send-email 2.34.0.rc2.393.gf8c9666880-goog Subject: [PATCH v3 12/15] KVM: arm64: Implement __pkvm_host_share_hyp() using do_share() From: Quentin Perret To: Marc Zyngier , James Morse , Alexandru Elisei , Suzuki K Poulose , Catalin Marinas , Will Deacon Cc: linux-arm-kernel@lists.infradead.org, kvmarm@lists.cs.columbia.edu, linux-kernel@vger.kernel.org, kernel-team@android.com X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20211201_090503_514849_EA6B745C X-CRM114-Status: GOOD ( 16.80 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org From: Will Deacon __pkvm_host_share_hyp() shares memory between the host and the hypervisor so implement it as an invocation of the new do_share() mechanism. Note that double-sharing is no longer permitted (as this allows us to reduce the number of page-table walks significantly), but is thankfully no longer relied upon by the host. Signed-off-by: Will Deacon Signed-off-by: Quentin Perret --- arch/arm64/kvm/hyp/nvhe/mem_protect.c | 121 +++++++------------------- 1 file changed, 33 insertions(+), 88 deletions(-) diff --git a/arch/arm64/kvm/hyp/nvhe/mem_protect.c b/arch/arm64/kvm/hyp/nvhe/mem_protect.c index 74ca4043b08a..1282cbd6b9b3 100644 --- a/arch/arm64/kvm/hyp/nvhe/mem_protect.c +++ b/arch/arm64/kvm/hyp/nvhe/mem_protect.c @@ -370,94 +370,6 @@ static int host_stage2_idmap(u64 addr) return ret; } -static inline bool check_prot(enum kvm_pgtable_prot prot, - enum kvm_pgtable_prot required, - enum kvm_pgtable_prot denied) -{ - return (prot & (required | denied)) == required; -} - -int __pkvm_host_share_hyp(u64 pfn) -{ - phys_addr_t addr = hyp_pfn_to_phys(pfn); - enum kvm_pgtable_prot prot, cur; - void *virt = __hyp_va(addr); - enum pkvm_page_state state; - kvm_pte_t pte; - int ret; - - if (!addr_is_memory(addr)) - return -EINVAL; - - host_lock_component(); - hyp_lock_component(); - - ret = kvm_pgtable_get_leaf(&host_kvm.pgt, addr, &pte, NULL); - if (ret) - goto unlock; - if (!pte) - goto map_shared; - - /* - * Check attributes in the host stage-2 PTE. We need the page to be: - * - mapped RWX as we're sharing memory; - * - not borrowed, as that implies absence of ownership. - * Otherwise, we can't let it got through - */ - cur = kvm_pgtable_stage2_pte_prot(pte); - prot = pkvm_mkstate(0, PKVM_PAGE_SHARED_BORROWED); - if (!check_prot(cur, PKVM_HOST_MEM_PROT, prot)) { - ret = -EPERM; - goto unlock; - } - - state = pkvm_getstate(cur); - if (state == PKVM_PAGE_OWNED) - goto map_shared; - - /* - * Tolerate double-sharing the same page, but this requires - * cross-checking the hypervisor stage-1. - */ - if (state != PKVM_PAGE_SHARED_OWNED) { - ret = -EPERM; - goto unlock; - } - - ret = kvm_pgtable_get_leaf(&pkvm_pgtable, (u64)virt, &pte, NULL); - if (ret) - goto unlock; - - /* - * If the page has been shared with the hypervisor, it must be - * already mapped as SHARED_BORROWED in its stage-1. - */ - cur = kvm_pgtable_hyp_pte_prot(pte); - prot = pkvm_mkstate(PAGE_HYP, PKVM_PAGE_SHARED_BORROWED); - if (!check_prot(cur, prot, ~prot)) - ret = -EPERM; - goto unlock; - -map_shared: - /* - * If the page is not yet shared, adjust mappings in both page-tables - * while both locks are held. - */ - prot = pkvm_mkstate(PAGE_HYP, PKVM_PAGE_SHARED_BORROWED); - ret = pkvm_create_mappings_locked(virt, virt + PAGE_SIZE, prot); - BUG_ON(ret); - - prot = pkvm_mkstate(PKVM_HOST_MEM_PROT, PKVM_PAGE_SHARED_OWNED); - ret = host_stage2_idmap_locked(addr, PAGE_SIZE, prot); - BUG_ON(ret); - -unlock: - hyp_unlock_component(); - host_unlock_component(); - - return ret; -} - void handle_host_mem_abort(struct kvm_cpu_context *host_ctxt) { struct kvm_vcpu_fault_info fault; @@ -708,3 +620,36 @@ static int do_share(struct pkvm_mem_share *share) return WARN_ON(__do_share(share)); } + +int __pkvm_host_share_hyp(u64 pfn) +{ + int ret; + u64 host_addr = hyp_pfn_to_phys(pfn); + u64 hyp_addr = (u64)__hyp_va(host_addr); + struct pkvm_mem_share share = { + .tx = { + .nr_pages = 1, + .initiator = { + .id = PKVM_ID_HOST, + .addr = host_addr, + .host = { + .completer_addr = hyp_addr, + }, + }, + .completer = { + .id = PKVM_ID_HYP, + }, + }, + .prot = PAGE_HYP, + }; + + host_lock_component(); + hyp_lock_component(); + + ret = do_share(&share); + + hyp_unlock_component(); + host_unlock_component(); + + return ret; +} From patchwork Wed Dec 1 17:04:07 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Quentin Perret X-Patchwork-Id: 12694360 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 45BCCC433EF for ; Wed, 1 Dec 2021 17:12:09 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:Cc:To:From:Subject:References: Mime-Version:Message-Id:In-Reply-To:Date:Reply-To:Content-ID: Content-Description:Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc :Resent-Message-ID:List-Owner; bh=2xkIkjhi0CHVBtcFlZRX5R/zYBbay+XLw1Lm/pewpo8=; b=Cm9+UKf9UMxb02kM0npU5gAq3M dC9lL/DpOJ19hzGG+fw+kIuFZxjPZdJeXvSY+TNKG9rE8BVac/f+kakXeB/MGpg1SDXT347lIMrXf /uZgZZZ/YL9PDkoifJ7LM6FRPCpClb9+GqmumV0nUQJ2x+9CyV41UdHMtAXHMQ38xQ+2iTg8Dv+q1 DE0IxzDcz7I561uAberZqNU3nsH2CHjDBjR4F1JOOORSeglcIc+wAY6YkCelAAS3lYdUlmDMYPsEA 5hYQd0qci6rr+Thav6ALrJYcQUdypBjnIKVDCUHBe5oJ6oAbfaH4NKbn6BTyQls03RBuA8SdfhH6H ukp2BpIA==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1msT7T-009UaW-13; Wed, 01 Dec 2021 17:10:11 +0000 Received: from mail-wm1-x34a.google.com ([2a00:1450:4864:20::34a]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1msT2Z-009S6V-Ko for linux-arm-kernel@lists.infradead.org; Wed, 01 Dec 2021 17:05:09 +0000 Received: by mail-wm1-x34a.google.com with SMTP id a64-20020a1c7f43000000b003335e5dc26bso12559959wmd.8 for ; Wed, 01 Dec 2021 09:05:06 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20210112; h=date:in-reply-to:message-id:mime-version:references:subject:from:to :cc; bh=Q/onK/NKlbcaX4KhfxRAjbAZkoN+SiOcre92D8hLf9o=; b=aul30UamSYt4Ckyai5kRTFmEJuXbGut32orKNWwnc9AayiXqn+TxLOd7v1BpFQ1BkV GbCmN3XKpHNrtpSiY4CcZt0+PMOPn9pIlCl5IYKNx/AzLVDxuTWDGyHkUEIHTm4F+7cP cfs5tvOShjkY1ghkmPC8ee+EsApXcRTCfMeKKWXX/PXQg+tWXSxLdryzJzl8FhLuN8Hk y6S4a8zmzBAclLt3d+Vt4GfxGLQIz3KoJinZGxhw7iraE71AsYv3QtwpJOmgwWU/nFDT kk6AcCiotX0F6MR0Ngy6Cts7whuy5SakHBFyAf4/uFCGznBRUGtBNTXqhMkf8eot6Ri6 r4eQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=Q/onK/NKlbcaX4KhfxRAjbAZkoN+SiOcre92D8hLf9o=; b=irsw4qj0RAe0OBHS/nsnfS+uiot295YsJRfbDQ1076elqCp8hFfT7xEarwdBRqPMMH vstMk+WdiF2stvXEdlwnZhShMO7Dbzlac2JPcHdL4D4NzyFOCWlffQtjm8D9zHdmxaRS NpOJEWNWkQqy9wqHmDSZqe/Hm7euH8dAqUCAb6MSZeO8hKRPvJf4phAklrrfn9fzzIgG N2kK8YeGcdpJHqMcwjjkF+0/XLXholejLnk9TRtWnK7qshsvjtDslBURNQPuowhu6/Ag s/fJyPMW9nNxmBJ0QfH3FNZ/9YXPWS2pqUwS29LJmhhSS9FSdl3TY1n5It2V8w2LutNf eH4A== X-Gm-Message-State: AOAM530ZoKcWGNiuo85ItcKGVek3oDj4WWaxGIRdw+ei6CnhToXqSIzW n9xhlrdyQVpvFNstsJD6aGHvBlVzAZo6 X-Google-Smtp-Source: ABdhPJy8c9GuXRQczl6Iol0RhaVFPr754ddoPaIqPgP6tAiFNldL0Xn8h+g/trLyF7Bg2zGWhojVvIGkL+73 X-Received: from luke.lon.corp.google.com ([2a00:79e0:d:210:1cab:1a20:2ef1:ddde]) (user=qperret job=sendgmr) by 2002:a05:600c:500a:: with SMTP id n10mr9113665wmr.136.1638378305616; Wed, 01 Dec 2021 09:05:05 -0800 (PST) Date: Wed, 1 Dec 2021 17:04:07 +0000 In-Reply-To: <20211201170411.1561936-1-qperret@google.com> Message-Id: <20211201170411.1561936-14-qperret@google.com> Mime-Version: 1.0 References: <20211201170411.1561936-1-qperret@google.com> X-Mailer: git-send-email 2.34.0.rc2.393.gf8c9666880-goog Subject: [PATCH v3 13/15] KVM: arm64: Implement do_unshare() helper for unsharing memory From: Quentin Perret To: Marc Zyngier , James Morse , Alexandru Elisei , Suzuki K Poulose , Catalin Marinas , Will Deacon Cc: linux-arm-kernel@lists.infradead.org, kvmarm@lists.cs.columbia.edu, linux-kernel@vger.kernel.org, kernel-team@android.com X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20211201_090507_733465_20618E1D X-CRM114-Status: GOOD ( 13.13 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org From: Will Deacon Tearing down a previously shared memory region results in the borrower losing access to the underlying pages and returning them to the "owned" state in the owner. Implement a do_unshare() helper, along the same lines as do_share(), to provide this functionality for the host-to-hyp case. Signed-off-by: Will Deacon Signed-off-by: Quentin Perret Reviewed-by: Andrew Walbran --- arch/arm64/kvm/hyp/nvhe/mem_protect.c | 115 ++++++++++++++++++++++++++ 1 file changed, 115 insertions(+) diff --git a/arch/arm64/kvm/hyp/nvhe/mem_protect.c b/arch/arm64/kvm/hyp/nvhe/mem_protect.c index 1282cbd6b9b3..43b25e2de780 100644 --- a/arch/arm64/kvm/hyp/nvhe/mem_protect.c +++ b/arch/arm64/kvm/hyp/nvhe/mem_protect.c @@ -485,6 +485,16 @@ static int host_request_owned_transition(u64 *completer_addr, return __host_check_page_state_range(addr, size, PKVM_PAGE_OWNED); } +static int host_request_unshare(u64 *completer_addr, + const struct pkvm_mem_transition *tx) +{ + u64 size = tx->nr_pages * PAGE_SIZE; + u64 addr = tx->initiator.addr; + + *completer_addr = tx->initiator.host.completer_addr; + return __host_check_page_state_range(addr, size, PKVM_PAGE_SHARED_OWNED); +} + static int host_initiate_share(u64 *completer_addr, const struct pkvm_mem_transition *tx) { @@ -495,6 +505,16 @@ static int host_initiate_share(u64 *completer_addr, return __host_set_page_state_range(addr, size, PKVM_PAGE_SHARED_OWNED); } +static int host_initiate_unshare(u64 *completer_addr, + const struct pkvm_mem_transition *tx) +{ + u64 size = tx->nr_pages * PAGE_SIZE; + u64 addr = tx->initiator.addr; + + *completer_addr = tx->initiator.host.completer_addr; + return __host_set_page_state_range(addr, size, PKVM_PAGE_OWNED); +} + static enum pkvm_page_state hyp_get_page_state(kvm_pte_t pte) { if (!kvm_pte_valid(pte)) @@ -535,6 +555,17 @@ static int hyp_ack_share(u64 addr, const struct pkvm_mem_transition *tx, return __hyp_check_page_state_range(addr, size, PKVM_NOPAGE); } +static int hyp_ack_unshare(u64 addr, const struct pkvm_mem_transition *tx) +{ + u64 size = tx->nr_pages * PAGE_SIZE; + + if (__hyp_ack_skip_pgtable_check(tx)) + return 0; + + return __hyp_check_page_state_range(addr, size, + PKVM_PAGE_SHARED_BORROWED); +} + static int hyp_complete_share(u64 addr, const struct pkvm_mem_transition *tx, enum kvm_pgtable_prot perms) { @@ -545,6 +576,14 @@ static int hyp_complete_share(u64 addr, const struct pkvm_mem_transition *tx, return pkvm_create_mappings_locked(start, end, prot); } +static int hyp_complete_unshare(u64 addr, const struct pkvm_mem_transition *tx) +{ + u64 size = tx->nr_pages * PAGE_SIZE; + int ret = kvm_pgtable_hyp_unmap(&pkvm_pgtable, addr, size); + + return (ret != size) ? -EFAULT : 0; +} + static int check_share(struct pkvm_mem_share *share) { const struct pkvm_mem_transition *tx = &share->tx; @@ -621,6 +660,82 @@ static int do_share(struct pkvm_mem_share *share) return WARN_ON(__do_share(share)); } +static int check_unshare(struct pkvm_mem_share *share) +{ + const struct pkvm_mem_transition *tx = &share->tx; + u64 completer_addr; + int ret; + + switch (tx->initiator.id) { + case PKVM_ID_HOST: + ret = host_request_unshare(&completer_addr, tx); + break; + default: + ret = -EINVAL; + } + + if (ret) + return ret; + + switch (tx->completer.id) { + case PKVM_ID_HYP: + ret = hyp_ack_unshare(completer_addr, tx); + break; + default: + ret = -EINVAL; + } + + return ret; +} + +static int __do_unshare(struct pkvm_mem_share *share) +{ + const struct pkvm_mem_transition *tx = &share->tx; + u64 completer_addr; + int ret; + + switch (tx->initiator.id) { + case PKVM_ID_HOST: + ret = host_initiate_unshare(&completer_addr, tx); + break; + default: + ret = -EINVAL; + } + + if (ret) + return ret; + + switch (tx->completer.id) { + case PKVM_ID_HYP: + ret = hyp_complete_unshare(completer_addr, tx); + break; + default: + ret = -EINVAL; + } + + return ret; +} + +/* + * do_unshare(): + * + * The page owner revokes access from another component for a range of + * pages which were previously shared using do_share(). + * + * Initiator: SHARED_OWNED => OWNED + * Completer: SHARED_BORROWED => NOPAGE + */ +static int do_unshare(struct pkvm_mem_share *share) +{ + int ret; + + ret = check_unshare(share); + if (ret) + return ret; + + return WARN_ON(__do_unshare(share)); +} + int __pkvm_host_share_hyp(u64 pfn) { int ret; From patchwork Wed Dec 1 17:04:08 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Quentin Perret X-Patchwork-Id: 12694361 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 6779DC433F5 for ; Wed, 1 Dec 2021 17:13:16 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:Cc:To:From:Subject:References: Mime-Version:Message-Id:In-Reply-To:Date:Reply-To:Content-ID: Content-Description:Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc :Resent-Message-ID:List-Owner; bh=1ypQd5yALl911JVzvBO2BBLyFh02hhTRXIiV9pKYVC4=; b=urDScT32Q696Z/Itz3GoGBlvm/ F/X+Eab7UFNttY/QGclyncSh5UxCRLBesv5CGHg8hiKdFkOPPtJsrk5iHKv4Y/LwcitN67CAiyg2s AVPIuVlELBXbr08jIuJXR8zOS0MfcjfHJE297JdJ9Qct8P7ttAtegkIkuBj8tS4zK+dSmrB93Tw18 oeWfi3QG6TefzpacLYKQn7AfBUIsVJwe90F9x61vDWQ9xbL5Sc1hTUDmebPTRzFlJ0rMeaMjcZHYm yOAedKPfLH+EnPHIOpLhCoET2WhnPpg1MKIQpTgNYlscYBVOQWNQA2N1nTBsfFavDKfXAwHC9M6dZ Uv6a47Zg==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1msT8V-009V1D-5u; Wed, 01 Dec 2021 17:11:16 +0000 Received: from mail-wr1-x449.google.com ([2a00:1450:4864:20::449]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1msT2d-009S86-I3 for linux-arm-kernel@lists.infradead.org; Wed, 01 Dec 2021 17:05:12 +0000 Received: by mail-wr1-x449.google.com with SMTP id f3-20020a5d50c3000000b00183ce1379feso4464077wrt.5 for ; Wed, 01 Dec 2021 09:05:10 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20210112; h=date:in-reply-to:message-id:mime-version:references:subject:from:to :cc; bh=HbFxjGh3zL5KJap6QW7mkZZ6VIZaiwXKEnlk6BAwVfU=; b=JtzM0gEDraZHkDaYQXTg0ZV8V0cXC3mNBHZ97J5W0AM9joQ+Tu6jEW6c3fBIQkM6Ib GwAR01pqvQylN5LeKRH5Q7RYcav9puXAvqGGEcceHhNVJtu2fZHSGJ5I0rAaGQRSIYb2 um0QpT9MkVgcbMmR6Fe5Gy1YhwIaojgSu8kpYp6Fc99EcuSk/ZUsB6cbr7CyUb/dupU5 Gz5vsL1JQx95KAsJyQQyYmsRFPNuLYANb7uJyyiLv4lMWxabM0Cwl9zjdJN6qvIeviEy 3CnwNybnqLkVpaWiFzwQXR0drk5xVz195jgBHNzbI5LpKat6JDQIqFPJtbQdyW+JwiqW 0uQQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=HbFxjGh3zL5KJap6QW7mkZZ6VIZaiwXKEnlk6BAwVfU=; b=OKWuJN6EZfgn4pyXv7LbGDPqCLGSViQZgFS4b2q63aOy1/yXp3QQvpSjpGJBSo+TtV zw+Md1Y/RKGssh9C3h5sPD8PV7piM5kDl9SdhDxln6AWkA9gujZt+4RIx7bGs2y5X2Dp JP+kMUVZzLSh3w2WlBt9onZMhpqJmOaqj5Wf8qH3Y6t+2L9GHknnUqbQM7WjJAlT8CTs OjeNE5WS3Rl8J/tDyFTnFdEwS5flvjGec3846bDG9TOgQ4Ec2Wbf/yiIcJrNp6Nzn42J kCQfCa53ClswztmdkDijekpd1wrZqdkkrXK+hDxlQ2PMIcJBYSEygdCoF3l4jetqIPNn pTCQ== X-Gm-Message-State: AOAM531MzTzguZeoSxA51/jlmm5gPTtKOVh7JJBKPRIxBFbk2Y8oAZ+P 0Pz15yTpTDwQTocV1VDixds0OW/z5gzf X-Google-Smtp-Source: ABdhPJwxaQUpa6nGrPntWg01UvZg5ANqj35b7DgPpex7mqDVaKYEMPj2lT/WlosMtvdaA5BJp8zHD3MNtwZE X-Received: from luke.lon.corp.google.com ([2a00:79e0:d:210:1cab:1a20:2ef1:ddde]) (user=qperret job=sendgmr) by 2002:a5d:508d:: with SMTP id a13mr8350562wrt.41.1638378309381; Wed, 01 Dec 2021 09:05:09 -0800 (PST) Date: Wed, 1 Dec 2021 17:04:08 +0000 In-Reply-To: <20211201170411.1561936-1-qperret@google.com> Message-Id: <20211201170411.1561936-15-qperret@google.com> Mime-Version: 1.0 References: <20211201170411.1561936-1-qperret@google.com> X-Mailer: git-send-email 2.34.0.rc2.393.gf8c9666880-goog Subject: [PATCH v3 14/15] KVM: arm64: Expose unshare hypercall to the host From: Quentin Perret To: Marc Zyngier , James Morse , Alexandru Elisei , Suzuki K Poulose , Catalin Marinas , Will Deacon Cc: linux-arm-kernel@lists.infradead.org, kvmarm@lists.cs.columbia.edu, linux-kernel@vger.kernel.org, kernel-team@android.com X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20211201_090511_632520_DC6B417B X-CRM114-Status: GOOD ( 13.31 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org From: Will Deacon Introduce an unshare hypercall which can be used to unmap memory from the hypervisor stage-1 in nVHE protected mode. This will be useful to update the EL2 ownership state of pages during guest teardown, and avoids keeping dangling mappings to unreferenced portions of memory. Signed-off-by: Will Deacon Signed-off-by: Quentin Perret --- arch/arm64/include/asm/kvm_asm.h | 1 + arch/arm64/kvm/hyp/include/nvhe/mem_protect.h | 1 + arch/arm64/kvm/hyp/nvhe/hyp-main.c | 8 +++++ arch/arm64/kvm/hyp/nvhe/mem_protect.c | 33 +++++++++++++++++++ 4 files changed, 43 insertions(+) diff --git a/arch/arm64/include/asm/kvm_asm.h b/arch/arm64/include/asm/kvm_asm.h index 50d5e4de244c..d5b0386ef765 100644 --- a/arch/arm64/include/asm/kvm_asm.h +++ b/arch/arm64/include/asm/kvm_asm.h @@ -63,6 +63,7 @@ enum __kvm_host_smccc_func { /* Hypercalls available after pKVM finalisation */ __KVM_HOST_SMCCC_FUNC___pkvm_host_share_hyp, + __KVM_HOST_SMCCC_FUNC___pkvm_host_unshare_hyp, __KVM_HOST_SMCCC_FUNC___kvm_adjust_pc, __KVM_HOST_SMCCC_FUNC___kvm_vcpu_run, __KVM_HOST_SMCCC_FUNC___kvm_flush_vm_context, diff --git a/arch/arm64/kvm/hyp/include/nvhe/mem_protect.h b/arch/arm64/kvm/hyp/include/nvhe/mem_protect.h index 56445586c755..80e99836eac7 100644 --- a/arch/arm64/kvm/hyp/include/nvhe/mem_protect.h +++ b/arch/arm64/kvm/hyp/include/nvhe/mem_protect.h @@ -55,6 +55,7 @@ extern const u8 pkvm_hyp_id; int __pkvm_prot_finalize(void); int __pkvm_host_share_hyp(u64 pfn); +int __pkvm_host_unshare_hyp(u64 pfn); bool addr_is_memory(phys_addr_t phys); int host_stage2_idmap_locked(phys_addr_t addr, u64 size, enum kvm_pgtable_prot prot); diff --git a/arch/arm64/kvm/hyp/nvhe/hyp-main.c b/arch/arm64/kvm/hyp/nvhe/hyp-main.c index b096bf009144..5e2197db0d32 100644 --- a/arch/arm64/kvm/hyp/nvhe/hyp-main.c +++ b/arch/arm64/kvm/hyp/nvhe/hyp-main.c @@ -147,6 +147,13 @@ static void handle___pkvm_host_share_hyp(struct kvm_cpu_context *host_ctxt) cpu_reg(host_ctxt, 1) = __pkvm_host_share_hyp(pfn); } +static void handle___pkvm_host_unshare_hyp(struct kvm_cpu_context *host_ctxt) +{ + DECLARE_REG(u64, pfn, host_ctxt, 1); + + cpu_reg(host_ctxt, 1) = __pkvm_host_unshare_hyp(pfn); +} + static void handle___pkvm_create_private_mapping(struct kvm_cpu_context *host_ctxt) { DECLARE_REG(phys_addr_t, phys, host_ctxt, 1); @@ -184,6 +191,7 @@ static const hcall_t host_hcall[] = { HANDLE_FUNC(__pkvm_prot_finalize), HANDLE_FUNC(__pkvm_host_share_hyp), + HANDLE_FUNC(__pkvm_host_unshare_hyp), HANDLE_FUNC(__kvm_adjust_pc), HANDLE_FUNC(__kvm_vcpu_run), HANDLE_FUNC(__kvm_flush_vm_context), diff --git a/arch/arm64/kvm/hyp/nvhe/mem_protect.c b/arch/arm64/kvm/hyp/nvhe/mem_protect.c index 43b25e2de780..640234d3896a 100644 --- a/arch/arm64/kvm/hyp/nvhe/mem_protect.c +++ b/arch/arm64/kvm/hyp/nvhe/mem_protect.c @@ -768,3 +768,36 @@ int __pkvm_host_share_hyp(u64 pfn) return ret; } + +int __pkvm_host_unshare_hyp(u64 pfn) +{ + int ret; + u64 host_addr = hyp_pfn_to_phys(pfn); + u64 hyp_addr = (u64)__hyp_va(host_addr); + struct pkvm_mem_share share = { + .tx = { + .nr_pages = 1, + .initiator = { + .id = PKVM_ID_HOST, + .addr = host_addr, + .host = { + .completer_addr = hyp_addr, + }, + }, + .completer = { + .id = PKVM_ID_HYP, + }, + }, + .prot = PAGE_HYP, + }; + + host_lock_component(); + hyp_lock_component(); + + ret = do_unshare(&share); + + hyp_unlock_component(); + host_unlock_component(); + + return ret; +} From patchwork Wed Dec 1 17:04:09 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Quentin Perret X-Patchwork-Id: 12694362 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 2AF78C433EF for ; Wed, 1 Dec 2021 17:14:18 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:Cc:To:From:Subject:References: Mime-Version:Message-Id:In-Reply-To:Date:Reply-To:Content-ID: Content-Description:Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc :Resent-Message-ID:List-Owner; bh=qWwpRk8EyxM96R/Gwv/ubDra41Fwuj1FZtpFioddLBY=; b=xMJKukwppJOPQsqxb+UHus687/ 0Ksm0qsmH5sC0k1eyTZYFHwA0ub1ECEKtT9Vk+Qsrq1oI3agbPGR4oR4+ucorTgOj2hRlNcMpX8BF om0NXc8a3RcCi5jfUElf8aZlg5Be2YSVed6vZ7FJMFA+d1TBS1kUBTv7CGL64M7Z93OgU+/f6ya0N QzRoQ8lpelSXyxm31tdfQqFoTJKIM+gFoQyR3+S6cY6H0HND797NnDOP6bncW0w5DpoYakYv6gVul 5IIoyjhyZ7iSpKpNxyEtSjPSv6L58NYeYrCoZ1zjAY6lLYmszLFn+xGlycx1Iyi0MPVNK1oVIwmuW ojLSmO5w==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1msT9Z-009VQW-7W; Wed, 01 Dec 2021 17:12:22 +0000 Received: from mail-wm1-x34a.google.com ([2a00:1450:4864:20::34a]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1msT2h-009S9v-C5 for linux-arm-kernel@lists.infradead.org; Wed, 01 Dec 2021 17:05:17 +0000 Received: by mail-wm1-x34a.google.com with SMTP id p12-20020a05600c1d8c00b0033a22e48203so1198142wms.6 for ; Wed, 01 Dec 2021 09:05:14 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20210112; h=date:in-reply-to:message-id:mime-version:references:subject:from:to :cc; bh=94lgfXtWTVzPjMgUB8UON2q/QY1bS12ewvWubTStSBY=; b=byjaspXWj3Emirw/SYW07K50hbbJRqfTTb/c+PdXpZ0s9t6fSMlrX8UA6jozLKWJT/ FzbGPWt4tNhwSGSlojLMXTJG5BJZLCGiIimZ+laxo/7TdLeglhvGLKDQJioEzGJg1dVx j7YVB6jfQsQlTpZoq9A27qDCvWfQHMd2DwTBjzu6tg0vHrPL2BGv551S3HM377d/kbWt X+GNVh9o9oIUYDsoOXowzMyK6+WpNO3mKqSvKHfAGpagL+J5RU5Ml5NA/5H0Ue9pW75X qzNE12E3WOdHLnzof4efNf1OjChZkivDFFknbJxjtvTRGyDI6UdwHsNvz1rL8cTvayu/ U2oQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=94lgfXtWTVzPjMgUB8UON2q/QY1bS12ewvWubTStSBY=; b=6l8aGADyY4wXngOgVdEIq1yIJFwnVzlBoeziX+Ql14yUT3RxsfTLfaaz2IlTMQ1me+ wkn5Svci5JKWZ2UHcqe0a3wse6he8UAwHTmlpKA6rrgN2lay+SFd8T5lbnE0WpWa4a1U B6wyH9g8ksFYk/GB/+ed3J1S8KIJ0ZJU3fVP+mtwBztydEjwKt94oXaxDSCiKKiVQlW1 5ifhTVs6oukkSAOCrxbog1nSpf73IqofK9DDxAvYQKbDxSCC/XMy5FTr1bFp6w0dpSet pjfGCPJF+2XH6KEYxDZAXrTqhwBMnrB/IDVZypjEmt3eWWLAr8egS6Ko9oZNvQkSD9kK ndDA== X-Gm-Message-State: AOAM530bF4GY1LXDHHkS0BiekENtx7vbRPDbXsigOF+dPYYmAy6AY4vx T7CjX6B8zAW/AsArsqD5O0imJP++t6jp X-Google-Smtp-Source: ABdhPJxbxvLDyUj4CZ5Ql0OoMQ3x1gNpb3V5A/9auOAcSgkEsXxfrFva++4+GqLbfoV75ix1zNS22MKmonO8 X-Received: from luke.lon.corp.google.com ([2a00:79e0:d:210:1cab:1a20:2ef1:ddde]) (user=qperret job=sendgmr) by 2002:a05:600c:1d97:: with SMTP id p23mr8674163wms.186.1638378313175; Wed, 01 Dec 2021 09:05:13 -0800 (PST) Date: Wed, 1 Dec 2021 17:04:09 +0000 In-Reply-To: <20211201170411.1561936-1-qperret@google.com> Message-Id: <20211201170411.1561936-16-qperret@google.com> Mime-Version: 1.0 References: <20211201170411.1561936-1-qperret@google.com> X-Mailer: git-send-email 2.34.0.rc2.393.gf8c9666880-goog Subject: [PATCH v3 15/15] KVM: arm64: pkvm: Unshare guest structs during teardown From: Quentin Perret To: Marc Zyngier , James Morse , Alexandru Elisei , Suzuki K Poulose , Catalin Marinas , Will Deacon Cc: linux-arm-kernel@lists.infradead.org, kvmarm@lists.cs.columbia.edu, linux-kernel@vger.kernel.org, kernel-team@android.com X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20211201_090515_450828_EAC1B51D X-CRM114-Status: GOOD ( 22.66 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Make use of the newly introduced unshare hypercall during guest teardown to unmap guest-related data structures from the hyp stage-1. Signed-off-by: Quentin Perret --- arch/arm64/include/asm/kvm_host.h | 2 ++ arch/arm64/include/asm/kvm_mmu.h | 1 + arch/arm64/kvm/arm.c | 2 ++ arch/arm64/kvm/fpsimd.c | 34 ++++++++++++++++++++++--- arch/arm64/kvm/mmu.c | 42 +++++++++++++++++++++++++++++++ arch/arm64/kvm/reset.c | 8 +++++- 6 files changed, 85 insertions(+), 4 deletions(-) diff --git a/arch/arm64/include/asm/kvm_host.h b/arch/arm64/include/asm/kvm_host.h index cf858a7e3533..9360a2804df1 100644 --- a/arch/arm64/include/asm/kvm_host.h +++ b/arch/arm64/include/asm/kvm_host.h @@ -321,6 +321,7 @@ struct kvm_vcpu_arch { struct kvm_guest_debug_arch external_debug_state; struct user_fpsimd_state *host_fpsimd_state; /* hyp VA */ + struct task_struct *parent_task; struct { /* {Break,watch}point registers */ @@ -737,6 +738,7 @@ void kvm_arch_vcpu_load_fp(struct kvm_vcpu *vcpu); void kvm_arch_vcpu_ctxflush_fp(struct kvm_vcpu *vcpu); void kvm_arch_vcpu_ctxsync_fp(struct kvm_vcpu *vcpu); void kvm_arch_vcpu_put_fp(struct kvm_vcpu *vcpu); +void kvm_vcpu_unshare_task_fp(struct kvm_vcpu *vcpu); static inline bool kvm_pmu_counter_deferred(struct perf_event_attr *attr) { diff --git a/arch/arm64/include/asm/kvm_mmu.h b/arch/arm64/include/asm/kvm_mmu.h index 185d0f62b724..81839e9a8a24 100644 --- a/arch/arm64/include/asm/kvm_mmu.h +++ b/arch/arm64/include/asm/kvm_mmu.h @@ -151,6 +151,7 @@ static __always_inline unsigned long __kern_hyp_va(unsigned long v) #include int kvm_share_hyp(void *from, void *to); +void kvm_unshare_hyp(void *from, void *to); int create_hyp_mappings(void *from, void *to, enum kvm_pgtable_prot prot); int create_hyp_io_mappings(phys_addr_t phys_addr, size_t size, void __iomem **kaddr, diff --git a/arch/arm64/kvm/arm.c b/arch/arm64/kvm/arm.c index c202abb448b1..6057f3c5aafe 100644 --- a/arch/arm64/kvm/arm.c +++ b/arch/arm64/kvm/arm.c @@ -188,6 +188,8 @@ void kvm_arch_destroy_vm(struct kvm *kvm) } } atomic_set(&kvm->online_vcpus, 0); + + kvm_unshare_hyp(kvm, kvm + 1); } int kvm_vm_ioctl_check_extension(struct kvm *kvm, long ext) diff --git a/arch/arm64/kvm/fpsimd.c b/arch/arm64/kvm/fpsimd.c index 86899d3aa9a9..2f48fd362a8c 100644 --- a/arch/arm64/kvm/fpsimd.c +++ b/arch/arm64/kvm/fpsimd.c @@ -14,6 +14,19 @@ #include #include +void kvm_vcpu_unshare_task_fp(struct kvm_vcpu *vcpu) +{ + struct task_struct *p = vcpu->arch.parent_task; + struct user_fpsimd_state *fpsimd; + + if (!is_protected_kvm_enabled() || !p) + return; + + fpsimd = &p->thread.uw.fpsimd_state; + kvm_unshare_hyp(fpsimd, fpsimd + 1); + put_task_struct(p); +} + /* * Called on entry to KVM_RUN unless this vcpu previously ran at least * once and the most recent prior KVM_RUN for this vcpu was called from @@ -29,12 +42,27 @@ int kvm_arch_vcpu_run_map_fp(struct kvm_vcpu *vcpu) struct user_fpsimd_state *fpsimd = ¤t->thread.uw.fpsimd_state; + kvm_vcpu_unshare_task_fp(vcpu); + /* Make sure the host task fpsimd state is visible to hyp: */ ret = kvm_share_hyp(fpsimd, fpsimd + 1); - if (!ret) - vcpu->arch.host_fpsimd_state = kern_hyp_va(fpsimd); + if (ret) + return ret; + + vcpu->arch.host_fpsimd_state = kern_hyp_va(fpsimd); + + /* + * We need to keep current's task_struct pinned until its data has been + * unshared with the hypervisor to make sure it is not re-used by the + * kernel and donated to someone else while already shared -- see + * kvm_vcpu_unshare_task_fp() for the matching put_task_struct(). + */ + if (is_protected_kvm_enabled()) { + get_task_struct(current); + vcpu->arch.parent_task = current; + } - return ret; + return 0; } /* diff --git a/arch/arm64/kvm/mmu.c b/arch/arm64/kvm/mmu.c index d72566896755..8e506ba8988e 100644 --- a/arch/arm64/kvm/mmu.c +++ b/arch/arm64/kvm/mmu.c @@ -344,6 +344,32 @@ static int share_pfn_hyp(u64 pfn) return ret; } +static int unshare_pfn_hyp(u64 pfn) +{ + struct rb_node **node, *parent; + struct hyp_shared_pfn *this; + int ret = 0; + + mutex_lock(&hyp_shared_pfns_lock); + this = find_shared_pfn(pfn, &node, &parent); + if (WARN_ON(!this)) { + ret = -EINVAL; + goto unlock; + } + + this->count--; + if (this->count) + goto unlock; + + rb_erase(&this->node, &hyp_shared_pfns); + kfree(this); + ret = kvm_call_hyp_nvhe(__pkvm_host_unshare_hyp, pfn, 1); +unlock: + mutex_unlock(&hyp_shared_pfns_lock); + + return ret; +} + int kvm_share_hyp(void *from, void *to) { phys_addr_t start, end, cur; @@ -376,6 +402,22 @@ int kvm_share_hyp(void *from, void *to) return 0; } +void kvm_unshare_hyp(void *from, void *to) +{ + phys_addr_t start, end, cur; + u64 pfn; + + if (is_kernel_in_hyp_mode() || kvm_host_owns_hyp_mappings() || !from) + return; + + start = ALIGN_DOWN(__pa(from), PAGE_SIZE); + end = PAGE_ALIGN(__pa(to)); + for (cur = start; cur < end; cur += PAGE_SIZE) { + pfn = __phys_to_pfn(cur); + WARN_ON(unshare_pfn_hyp(pfn)); + } +} + /** * create_hyp_mappings - duplicate a kernel virtual address range in Hyp mode * @from: The virtual kernel start address of the range diff --git a/arch/arm64/kvm/reset.c b/arch/arm64/kvm/reset.c index e3e2a79fbd75..798a84eddbde 100644 --- a/arch/arm64/kvm/reset.c +++ b/arch/arm64/kvm/reset.c @@ -150,7 +150,13 @@ bool kvm_arm_vcpu_is_finalized(struct kvm_vcpu *vcpu) void kvm_arm_vcpu_destroy(struct kvm_vcpu *vcpu) { - kfree(vcpu->arch.sve_state); + void *sve_state = vcpu->arch.sve_state; + + kvm_vcpu_unshare_task_fp(vcpu); + kvm_unshare_hyp(vcpu, vcpu + 1); + if (sve_state) + kvm_unshare_hyp(sve_state, sve_state + vcpu_sve_state_size(vcpu)); + kfree(sve_state); } static void kvm_vcpu_reset_sve(struct kvm_vcpu *vcpu)