From patchwork Tue Oct 19 12:12:50 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Quentin Perret X-Patchwork-Id: 12569723 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id DD1CEC433F5 for ; Tue, 19 Oct 2021 12:15:02 +0000 (UTC) Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id AB84C6115B for ; Tue, 19 Oct 2021 12:15:02 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.4.1 mail.kernel.org AB84C6115B Authentication-Results: mail.kernel.org; dmarc=fail (p=reject dis=none) header.from=google.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:Cc:To:From:Subject:References: Mime-Version:Message-Id:In-Reply-To:Date:Reply-To:Content-ID: Content-Description:Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc :Resent-Message-ID:List-Owner; bh=moXD0sJnnraOARpXWhIbj2T98jftAxxp4rqXGLG+PlA=; b=rwd4fbaeFeiw8DFaA+YXOAc+cV IxWnoStfJnETWwwoulxMyGGeaP+4MUqGY4mspywL/+W2cZB9LuPON78B5xM7+q4aTmrqEiZolLpEL CGjQjgkARJiJUGgXLZ/LP751kRXZMH0Jts9xNl9J/LOHiYEpxMpRwm6S/EGOmUnc53zydDEa5KkcB 5Y5pbLcp/+ENd4pgIfzgvR74VaGDu8J2tZItwEg1Xv46JBeGf9n5676I7yA1TXe+UbXtYLH/EbXHU rz09gggrSMEH7Nxgxv9zFg3ZcsrlEH1ge7cz4wWJVH+elIa6hiqZ9j0y6YD4m0qxv3+rwjH1kNdNE si8aOKNg==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1mcnzj-0015H3-FE; Tue, 19 Oct 2021 12:13:27 +0000 Received: from mail-wr1-x44a.google.com ([2a00:1450:4864:20::44a]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1mcnzU-00159B-Hz for linux-arm-kernel@lists.infradead.org; Tue, 19 Oct 2021 12:13:13 +0000 Received: by mail-wr1-x44a.google.com with SMTP id l8-20020a5d6d88000000b001611b5de796so10085720wrs.10 for ; Tue, 19 Oct 2021 05:13:10 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20210112; h=date:in-reply-to:message-id:mime-version:references:subject:from:to :cc; bh=tdsCErEUABqyyQ1mbx3uMSi/aufrxWWc3XxvUpZyi6c=; b=agqompm2B2MZmrGmB/gxpWpG6vgaVYlKU4CIg03TGpGC+4aalw5NfoiD3nd16AtPKC +DDl3VZfg72c4W3xGXNnVhohTo/s1mh5Cyu5/QSadO0RvwtMUqZBiqtMAx328Fh9wqjq O0Nnn+TUupK1jSkoQ6zAv3/qn9uizgYDbldXMVCRDlXhC3lCi4AOj3ogrxk1mN+ucv3t cBTjLmdxquTpk/VsYC9VrDb0/yJLDYC7h9Lyd93revkEa0NK2jXBGNPbXzqtFp+2iCfn x3H7XuI7FYVAgUe1wh0Gh6d7Wleobtrh8ZGRAvmytblQRuG07xpaCyWW6Wf03jkJTJVX x5Zw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=tdsCErEUABqyyQ1mbx3uMSi/aufrxWWc3XxvUpZyi6c=; b=FOLJZW1Sb5Nt7ucqZYTxIaesKnAoHap80zr5MRZyMdL2mgneRiv0LOW/WiPnb+qM55 YsYmSCmvdKNQzsiY7FFvRH21jtrMCoIKC1o/Uq6E+k1LDOEGpLRx+65HkVT7HHINjelc ZtcpxrQbvKEpUjPSYPs7bb+Jamq10RDlvj23IRQKBeLBQpwoaWlrD90guswU8GFSpx6t ChNH8vKFqiIubDGq995pDT6mvyOy86EsjS/iEaiBkUrkBsSJcc1D/Qi4QamMvfMeHQcn 9zYiQgpBT1PNsyfp4Mq7IErdQuuKaGL81cmJrRlRMb++CSl2tP4tTy+SypMmAqWCgBJg yprw== X-Gm-Message-State: AOAM533nvGV3N+jmXV3tN/Ad1UyAGCCD8I4F03s83zoMa0SxNKfP7pFL B18yKC9JhqEkvhC1okzA2MfW8n1tleSD X-Google-Smtp-Source: ABdhPJwSSTIWtOO8ktVy4rHrNPU+CIvJsJy3oLDvfo1jOaII6PMNjG5KAfWEfB/Wt+zdj/T+fCLv/8OQmHxo X-Received: from luke.lon.corp.google.com ([2a00:79e0:d:210:59ca:401f:83a8:de6d]) (user=qperret job=sendgmr) by 2002:a1c:5417:: with SMTP id i23mr5564552wmb.17.1634645589172; Tue, 19 Oct 2021 05:13:09 -0700 (PDT) Date: Tue, 19 Oct 2021 13:12:50 +0100 In-Reply-To: <20211019121304.2732332-1-qperret@google.com> Message-Id: <20211019121304.2732332-2-qperret@google.com> Mime-Version: 1.0 References: <20211019121304.2732332-1-qperret@google.com> X-Mailer: git-send-email 2.33.0.1079.g6e70778dc9-goog Subject: [PATCH v2 01/15] KVM: arm64: Check if running in VHE from kvm_host_owns_hyp_mappings() From: Quentin Perret To: Marc Zyngier , James Morse , Alexandru Elisei , Suzuki K Poulose , Catalin Marinas , Will Deacon , Fuad Tabba , David Brazdil , Andrew Walbran Cc: linux-arm-kernel@lists.infradead.org, kvmarm@lists.cs.columbia.edu, linux-kernel@vger.kernel.org, kernel-team@android.com, qperret@google.com X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20211019_051312_628478_F3AEF33B X-CRM114-Status: GOOD ( 12.52 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org The kvm_host_owns_hyp_mappings() function should return true if and only if the host kernel is responsible for creating the hypervisor stage-1 mappings. That is only possible in standard non-VHE mode, or during boot in protected nVHE mode. But either way, non of this makes sense in VHE, so make sure to catch this case as well, hence making the function return sensible values in any context (VHE or not). Suggested-by: Marc Zyngier Signed-off-by: Quentin Perret --- arch/arm64/kvm/mmu.c | 3 +++ 1 file changed, 3 insertions(+) diff --git a/arch/arm64/kvm/mmu.c b/arch/arm64/kvm/mmu.c index da10996dcdf1..0019b2309f70 100644 --- a/arch/arm64/kvm/mmu.c +++ b/arch/arm64/kvm/mmu.c @@ -239,6 +239,9 @@ void free_hyp_pgds(void) static bool kvm_host_owns_hyp_mappings(void) { + if (is_kernel_in_hyp_mode()) + return false; + if (static_branch_likely(&kvm_protected_mode_initialized)) return false; From patchwork Tue Oct 19 12:12:51 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Quentin Perret X-Patchwork-Id: 12569725 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 2FC78C433EF for ; Tue, 19 Oct 2021 12:15:26 +0000 (UTC) Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id E624D6115B for ; Tue, 19 Oct 2021 12:15:25 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.4.1 mail.kernel.org E624D6115B Authentication-Results: mail.kernel.org; dmarc=fail (p=reject dis=none) header.from=google.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:Cc:To:From:Subject:References: Mime-Version:Message-Id:In-Reply-To:Date:Reply-To:Content-ID: Content-Description:Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc :Resent-Message-ID:List-Owner; bh=7M2s4VxuS4uwwvYwVlStK8PrxHzV1Zss214F3XitBa8=; b=17NapY8Vzfk6SoW5ogFiKrAVqF zLtEjFJmj7LCdujhtGZLk3fwmdAhN4M9ErSHlsAxdx1FqrAbq52K6Mtfyox6LgQWIkDMEqb6Qn0ge LnFo1Qr907JUu3ib+IahFyKms6YDBBTmBH42vWQFgNQVD4VbgsOmMlxA4aeBroky1iX/3AqHtCZDP 9NjxTCmccBCkpl+YEwYv+dB4xCpMH8tF8JIzOyo0mhnnz7wUYipPCjmdgvre+XNdKn/zlPOoPmSlF AJ94+aokylmMSwtUrUvf9tk82a6x69d4pYF3r4DXtTKpUdtRvYpg0zfwTXlgxF3nZohSwIR6JzgVI 9FwmXv7g==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1mco07-0015Sy-Nf; Tue, 19 Oct 2021 12:13:52 +0000 Received: from mail-wm1-x349.google.com ([2a00:1450:4864:20::349]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1mcnzW-00159v-6x for linux-arm-kernel@lists.infradead.org; Tue, 19 Oct 2021 12:13:15 +0000 Received: by mail-wm1-x349.google.com with SMTP id a20-20020a1c9814000000b0032311c7fc54so170844wme.1 for ; Tue, 19 Oct 2021 05:13:12 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20210112; h=date:in-reply-to:message-id:mime-version:references:subject:from:to :cc; bh=9qioxcWNTdM29KTskJovFvpNouj7jwaQhn/w48JNYr0=; b=RCVF+kFj8+Q1ZIvAw2o1YEfdbyV/QvERANN2zOL3vp6KFp4sOKvq50TuiuJVXjk8CI sJ8+ANxFKAZIB5XW2WXqgfxqioUa8/FqpJDAAlHYk4VwJzPsMiTKfyXjGsWp6nYM2egs 6ve8/RASgPFUMVKruWnMc+7Dq7xO467GfVDxse/ta3etQdRPTINc1Tn1t35gFQUffSa4 I17Na6p96zob6zezj90RVS7pxK8HHoy186iRxq1vgOzGonEHYihEMN8mRi5acuaAqYmN 0uVSCaCv9OaDCrMFHRsAD+gEn2IzZEReHw70B6v4JZTVux2bKXVrx5uW9Gfu3BqI6frT y8QQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=9qioxcWNTdM29KTskJovFvpNouj7jwaQhn/w48JNYr0=; b=clNH5zowNLfYa+oeGv3EiWz2+qZ7NGRzKYze5KHN7l0oi3nS7QeLmXrIK8rqftksOh OcXeQQfZD+QJF8Wcs1hfMI8DiezGP1MXTHvaQv+qFCzwWiCIdlRc0hmxtkBjyWUUpxDE zPa3EQ6lgs+d0Jw6GZtMgY72rdui1UmNHfLHGRqKFMSqK4QVnBUz2FSWLIkwhM3Yy9Or rc3DdUEAGRNgq8YYBKmwXHWdK2oWFU22pftb82AtmkoEp9c1C1BAF3bEo9lygalPtkI7 BEiI9SFa+GqqIteqPBJukeGaKv285r8OQEnZtAa46dcNRizrVQocdgpPAF0h8tBPtINa 26NQ== X-Gm-Message-State: AOAM533olIL8oIcqtXNLDDESVwBSUtDrpQkzzp7kCgYQbUrMF3aReTUO c53w321HvMUQD+V6oKqBoQQFTcJCl0gW X-Google-Smtp-Source: ABdhPJwlXJ0DM6qyO/wxtqG0399vc1RBGDWqJP0qqKuvfhECekIT7zJoTt4K3Fn+DO1gRLB/9VBwvtDmOWwC X-Received: from luke.lon.corp.google.com ([2a00:79e0:d:210:59ca:401f:83a8:de6d]) (user=qperret job=sendgmr) by 2002:a05:6000:50:: with SMTP id k16mr42638419wrx.32.1634645591442; Tue, 19 Oct 2021 05:13:11 -0700 (PDT) Date: Tue, 19 Oct 2021 13:12:51 +0100 In-Reply-To: <20211019121304.2732332-1-qperret@google.com> Message-Id: <20211019121304.2732332-3-qperret@google.com> Mime-Version: 1.0 References: <20211019121304.2732332-1-qperret@google.com> X-Mailer: git-send-email 2.33.0.1079.g6e70778dc9-goog Subject: [PATCH v2 02/15] KVM: arm64: Provide {get, put}_page() stubs for early hyp allocator From: Quentin Perret To: Marc Zyngier , James Morse , Alexandru Elisei , Suzuki K Poulose , Catalin Marinas , Will Deacon , Fuad Tabba , David Brazdil , Andrew Walbran Cc: linux-arm-kernel@lists.infradead.org, kvmarm@lists.cs.columbia.edu, linux-kernel@vger.kernel.org, kernel-team@android.com, qperret@google.com X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20211019_051314_308801_5D3E472D X-CRM114-Status: GOOD ( 12.28 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org In nVHE protected mode, the EL2 code uses a temporary allocator during boot while re-creating its stage-1 page-table. Unfortunately, the hyp_vmmemap is not ready to use at this stage, so refcounting pages is not possible. That is not currently a problem because hyp stage-1 mappings are never removed, which implies refcounting of page-table pages is unnecessary. In preparation for allowing hypervisor stage-1 mappings to be removed, provide stub implementations for {get,put}_page() in the early allocator. Signed-off-by: Quentin Perret --- arch/arm64/kvm/hyp/nvhe/early_alloc.c | 5 +++++ 1 file changed, 5 insertions(+) diff --git a/arch/arm64/kvm/hyp/nvhe/early_alloc.c b/arch/arm64/kvm/hyp/nvhe/early_alloc.c index 1306c430ab87..00de04153cc6 100644 --- a/arch/arm64/kvm/hyp/nvhe/early_alloc.c +++ b/arch/arm64/kvm/hyp/nvhe/early_alloc.c @@ -43,6 +43,9 @@ void *hyp_early_alloc_page(void *arg) return hyp_early_alloc_contig(1); } +static void hyp_early_alloc_get_page(void *addr) { } +static void hyp_early_alloc_put_page(void *addr) { } + void hyp_early_alloc_init(void *virt, unsigned long size) { base = cur = (unsigned long)virt; @@ -51,4 +54,6 @@ void hyp_early_alloc_init(void *virt, unsigned long size) hyp_early_alloc_mm_ops.zalloc_page = hyp_early_alloc_page; hyp_early_alloc_mm_ops.phys_to_virt = hyp_phys_to_virt; hyp_early_alloc_mm_ops.virt_to_phys = hyp_virt_to_phys; + hyp_early_alloc_mm_ops.get_page = hyp_early_alloc_get_page; + hyp_early_alloc_mm_ops.put_page = hyp_early_alloc_put_page; } From patchwork Tue Oct 19 12:12:52 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Quentin Perret X-Patchwork-Id: 12569727 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 189A4C433F5 for ; Tue, 19 Oct 2021 12:15:55 +0000 (UTC) Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id D729661260 for ; Tue, 19 Oct 2021 12:15:54 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.4.1 mail.kernel.org D729661260 Authentication-Results: mail.kernel.org; dmarc=fail (p=reject dis=none) header.from=google.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:Cc:To:From:Subject:References: Mime-Version:Message-Id:In-Reply-To:Date:Reply-To:Content-ID: Content-Description:Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc :Resent-Message-ID:List-Owner; bh=Kf6u755yJHL1/Q8SC92xAUhbZ21ogN22/SCmqzIzfDE=; b=o+QCjGmgDfpzv1dWEdCsoTNJu/ eFNj6eMHaTEuiNIFGOFCTPIGxo1JGfo2d+o5OWSEdK5sKFh4yEtT/Gkw95hGBseFpqg7ojUOg/tZX FCQOzZ0ILUS9h3uyNXTnn3d6Dv4kLt9N3UItZiJm+VE4iW4gHBSD1H+iSZ28JtR5VtbRGSurR9LyI gJTmcrslmy0+crIPH6B+qmjt6qvIzNVYhdLRcFWBYkqmlG3kuhwm0nOmcu7oo1DYCOBEok4/cLaKv khm/yvSa1OrAx/5GKzOi0PpL52yML5ZjeiFilXAri8dz4PhxZVjg8FuEl21/RxospZmIB3Nqlp+b3 QuXCulmA==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1mco0e-0015jW-MO; Tue, 19 Oct 2021 12:14:25 +0000 Received: from mail-wr1-x449.google.com ([2a00:1450:4864:20::449]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1mcnzY-0015BY-MC for linux-arm-kernel@lists.infradead.org; Tue, 19 Oct 2021 12:13:18 +0000 Received: by mail-wr1-x449.google.com with SMTP id a15-20020a056000188f00b00161068d8461so10129598wri.11 for ; Tue, 19 Oct 2021 05:13:15 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20210112; h=date:in-reply-to:message-id:mime-version:references:subject:from:to :cc; bh=TYk1LPnI99k/ggVKO8Mj9uEvzzDQ3GdDPOmXLwpCDNI=; b=O8yTSrO0Uca5wm5K4vU0ejt4H7paio27+Bqs4+KFIFEGBAiFkCCrrUrGNwkEt/Fefe 28wzo1Ssyx+9lQZLUUAw2vVund8vhlY/9UwK8rovsHyNg4yw2Rn3gBGVepSL7Eq2w3yF hOEEM+UMiw4N/19fDLtDtMDjzcuOLCQZrQwnn8kjya9y0T0dnD7uEF3AAv8UWU/4yrKi S6OImPM+oD/vG3KMqhrA+Gyqt8kAYJpzoj+A1WyLb90HutKsJMU8ddw6cAouTDJTBNKC PQvZvate6zTr45VFMsJe5Czj+08pSBtCAh4K0m25lr/XPX0osBrElQ5U3Cd9eTV8m49b ZmUw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=TYk1LPnI99k/ggVKO8Mj9uEvzzDQ3GdDPOmXLwpCDNI=; b=NK2XvoYuVExRbKUoNRFw7nyOicQIxFHly46ovTnO7v90WyUf14iC2uDcu8+k23j3Up k0kovBtVjypxRkqQFIEEm48irP1d3qRaBnwI7t5+sg/L04GlAw2AC8o0UkTantetjtVI j4XZ5kpwi8nY8YejMQ6pPEnSCEcbrkh7bsstqZAMpX3yYZkUDAqf+fGbvPBuxFxYs75g g3qu22dEbx4KyDy0y8NI/gimbHdEm4EdxsMO9TyecGE/IcFtxkmFzsuSPDp9F++I7/fU goQ/1Bbd6/Ojrd51ByiQdZvvdTUUxOnj4qXPHkRuqfK1J/mfM/pTTHxhlYSRrd0G4yw6 I+Zg== X-Gm-Message-State: AOAM533FvjUkC2CkljDCsdu5pxFprhxPe08apsc9Zi6gRcgj55Qz5XcY k4dt5gtfKIXZBvLcm+5rXFOcaz+GecYe X-Google-Smtp-Source: ABdhPJzFSw1s7wfnFX8qmHfRSKmq59QIp1f27cUPH/WdPiT9CqXbF6yjtS78WRcCH6lpSA4IMEtzt5LuNzVT X-Received: from luke.lon.corp.google.com ([2a00:79e0:d:210:59ca:401f:83a8:de6d]) (user=qperret job=sendgmr) by 2002:a1c:2b04:: with SMTP id r4mr5781071wmr.48.1634645593915; Tue, 19 Oct 2021 05:13:13 -0700 (PDT) Date: Tue, 19 Oct 2021 13:12:52 +0100 In-Reply-To: <20211019121304.2732332-1-qperret@google.com> Message-Id: <20211019121304.2732332-4-qperret@google.com> Mime-Version: 1.0 References: <20211019121304.2732332-1-qperret@google.com> X-Mailer: git-send-email 2.33.0.1079.g6e70778dc9-goog Subject: [PATCH v2 03/15] KVM: arm64: Refcount hyp stage-1 pgtable pages From: Quentin Perret To: Marc Zyngier , James Morse , Alexandru Elisei , Suzuki K Poulose , Catalin Marinas , Will Deacon , Fuad Tabba , David Brazdil , Andrew Walbran Cc: linux-arm-kernel@lists.infradead.org, kvmarm@lists.cs.columbia.edu, linux-kernel@vger.kernel.org, kernel-team@android.com, qperret@google.com X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20211019_051316_760138_2540FF57 X-CRM114-Status: GOOD ( 13.25 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org To prepare the ground for allowing hyp stage-1 mappings to be removed at run-time, update the KVM page-table code to maintain a correct refcount using the ->{get,put}_page() function callbacks. Signed-off-by: Quentin Perret --- arch/arm64/kvm/hyp/pgtable.c | 17 ++++++++++++++--- 1 file changed, 14 insertions(+), 3 deletions(-) diff --git a/arch/arm64/kvm/hyp/pgtable.c b/arch/arm64/kvm/hyp/pgtable.c index f8ceebe4982e..768a58835153 100644 --- a/arch/arm64/kvm/hyp/pgtable.c +++ b/arch/arm64/kvm/hyp/pgtable.c @@ -408,8 +408,10 @@ static bool hyp_map_walker_try_leaf(u64 addr, u64 end, u32 level, return false; new = kvm_init_valid_leaf_pte(phys, data->attr, level); - if (hyp_pte_needs_update(old, new)) + if (hyp_pte_needs_update(old, new)) { smp_store_release(ptep, new); + data->mm_ops->get_page(ptep); + } data->phys += granule; return true; @@ -433,6 +435,7 @@ static int hyp_map_walker(u64 addr, u64 end, u32 level, kvm_pte_t *ptep, return -ENOMEM; kvm_set_table_pte(ptep, childp, mm_ops); + mm_ops->get_page(ptep); return 0; } @@ -482,8 +485,16 @@ static int hyp_free_walker(u64 addr, u64 end, u32 level, kvm_pte_t *ptep, enum kvm_pgtable_walk_flags flag, void * const arg) { struct kvm_pgtable_mm_ops *mm_ops = arg; + kvm_pte_t pte = *ptep; + + if (!kvm_pte_valid(pte)) + return 0; + + mm_ops->put_page(ptep); + + if (kvm_pte_table(pte, level)) + mm_ops->put_page(kvm_pte_follow(pte, mm_ops)); - mm_ops->put_page((void *)kvm_pte_follow(*ptep, mm_ops)); return 0; } @@ -491,7 +502,7 @@ void kvm_pgtable_hyp_destroy(struct kvm_pgtable *pgt) { struct kvm_pgtable_walker walker = { .cb = hyp_free_walker, - .flags = KVM_PGTABLE_WALK_TABLE_POST, + .flags = KVM_PGTABLE_WALK_LEAF | KVM_PGTABLE_WALK_TABLE_POST, .arg = pgt->mm_ops, }; From patchwork Tue Oct 19 12:12:53 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Quentin Perret X-Patchwork-Id: 12569729 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 1A34FC433EF for ; Tue, 19 Oct 2021 12:16:41 +0000 (UTC) Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id E0BB461260 for ; Tue, 19 Oct 2021 12:16:40 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.4.1 mail.kernel.org E0BB461260 Authentication-Results: mail.kernel.org; dmarc=fail (p=reject dis=none) header.from=google.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:Cc:To:From:Subject:References: Mime-Version:Message-Id:In-Reply-To:Date:Reply-To:Content-ID: Content-Description:Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc :Resent-Message-ID:List-Owner; bh=If517BwXQ3v7xqdTD9l44zExTEvkh8RscnUC8zvhj+U=; b=DYSKRRHNnbVnvfqFFJVfauf/KP 9Hu/jFtjI5P1bDF/CUk/CtFwkQQCvxU6/Rjj0GRW9hIuzHQ/lMzri7ktZnI3L3i/lBAI5tcx45/Qd ZEC2KKu3g+yW/pKLS5JIKUEMqdTrflBXiGY3BGN7XTW/3fQN0kRdSSATewozEegN/zIhZoQBnNx1R 0qqkK2fobArYEgQybYrs+onvDzHIFoYH17ikXndsx3CgLRwqXvNc5FGsLO72L/lpM2AcXRmH+YEiH /ID4eL9wtfdLhKkJDUD7fvr0k95rgpYlPqXYnFRIqIG+G8yaURdkXWeWxprlCmr9QwjT3A4CcHDsb Lf6/q3oQ==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1mco1B-00161s-2T; Tue, 19 Oct 2021 12:14:57 +0000 Received: from mail-wr1-x449.google.com ([2a00:1450:4864:20::449]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1mcnza-0015CK-7S for linux-arm-kernel@lists.infradead.org; Tue, 19 Oct 2021 12:13:19 +0000 Received: by mail-wr1-x449.google.com with SMTP id h99-20020adf906c000000b001644add8925so6903824wrh.0 for ; Tue, 19 Oct 2021 05:13:17 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20210112; h=date:in-reply-to:message-id:mime-version:references:subject:from:to :cc; bh=++xVX7cq9dri2+SFZW5WIc0fF/Kl7wWBtgdSK2pMHEw=; b=YfeNcGEPdATddU+z3YnUUEeQ58+n/htJaUr7pcFV0pLA8xf8N3BummwMBtbl3EkXsd 5k5yC2YGf2i6gvmE+RzgT0nkae5+wdy/qpTCYJ2RFzC+k0trTK9gggZpZpt8KAEIh0Fa ewhY3rYjID7F6W3V0jeBklt1ck6TqnAD2uCqaZ2EjRpP12OowyDJSEC/xSOu6LMRgjju 4IcUs8iFs92golKvI4zYNg+BTy+L8nK5+HfpFxA9z/rdh4Gg4waWf2oAdnC8KUoxp8N9 LhGqQkkyjCCBEo5f856TkfoeG0dIh52NHjH5Jedm2jt+9zDWLyDLl2hgbw60pjmTfn+n 0NYA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=++xVX7cq9dri2+SFZW5WIc0fF/Kl7wWBtgdSK2pMHEw=; b=qKO/as9bfVgNlv0GPsIPEm6FbGQaMOIawku9VzMyOsECsxe78N4ovLnzQYJn2DTVMD 0xKbtCsqazWP+CFt/XJ+3Uw1DlGr8XVvqPJP7ktv7wzZaqUGHGjvlkeC9xCeIEEt53Aj HEwWt56SaFLXPo+MvOM9cMoC2tqHNBwTI5hwkGlb6pDJ/vluwvCQ8zp6p9HCoavdlNh7 kBa6eM//1a3dgIZvUozKBQv7N2znSA+TKPEo+oPkbMzkxh394GGmEOwTDcvefT9NOnsk E9ee1snpLSaJqF8KIfzIsi3eouS30Bo6MZPcM7eWZyhxpJRei7xShyA3fnClVb5Erqxo a/WA== X-Gm-Message-State: AOAM533ys/hkq6BJGibgmN98VUw80e5FQIXV9ATm9RMSKeFWPluef4Xg 39cwlZDbN9jPqwliIgCWTdM6CDM3JKRG X-Google-Smtp-Source: ABdhPJyUp2THf2vqTs2t6aOxbzWJ1ZyfKwGpN2QbKlKxK+/isqHvkyIyOmDTVZPJdG4S62oIcZFlzqHeLYBK X-Received: from luke.lon.corp.google.com ([2a00:79e0:d:210:59ca:401f:83a8:de6d]) (user=qperret job=sendgmr) by 2002:adf:b1d4:: with SMTP id r20mr43489909wra.308.1634645596136; Tue, 19 Oct 2021 05:13:16 -0700 (PDT) Date: Tue, 19 Oct 2021 13:12:53 +0100 In-Reply-To: <20211019121304.2732332-1-qperret@google.com> Message-Id: <20211019121304.2732332-5-qperret@google.com> Mime-Version: 1.0 References: <20211019121304.2732332-1-qperret@google.com> X-Mailer: git-send-email 2.33.0.1079.g6e70778dc9-goog Subject: [PATCH v2 04/15] KVM: arm64: Fixup hyp stage-1 refcount From: Quentin Perret To: Marc Zyngier , James Morse , Alexandru Elisei , Suzuki K Poulose , Catalin Marinas , Will Deacon , Fuad Tabba , David Brazdil , Andrew Walbran Cc: linux-arm-kernel@lists.infradead.org, kvmarm@lists.cs.columbia.edu, linux-kernel@vger.kernel.org, kernel-team@android.com, qperret@google.com X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20211019_051318_310376_C0A79B78 X-CRM114-Status: GOOD ( 15.41 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org In nVHE-protected mode, the hyp stage-1 page-table refcount is broken due to the lack of refcount support in the early allocator. Fix-up the refcount in the finalize walker, once the 'hyp_vmemmap' is up and running. Signed-off-by: Quentin Perret --- arch/arm64/kvm/hyp/nvhe/setup.c | 31 +++++++++++++++++++++---------- 1 file changed, 21 insertions(+), 10 deletions(-) diff --git a/arch/arm64/kvm/hyp/nvhe/setup.c b/arch/arm64/kvm/hyp/nvhe/setup.c index 862c7b514e20..9e5c92237a51 100644 --- a/arch/arm64/kvm/hyp/nvhe/setup.c +++ b/arch/arm64/kvm/hyp/nvhe/setup.c @@ -167,12 +167,22 @@ static int finalize_host_mappings_walker(u64 addr, u64 end, u32 level, { enum kvm_pgtable_prot prot; enum pkvm_page_state state; + struct kvm_pgtable_mm_ops *mm_ops = arg; kvm_pte_t pte = *ptep; phys_addr_t phys; if (!kvm_pte_valid(pte)) return 0; + /* + * Fix-up the refcount for the page-table pages as the early allocator + * was unable to access the hyp_vmemmap and so the buddy allocator has + * initialised the refcount to '1'. + */ + mm_ops->get_page(ptep); + if (flag != KVM_PGTABLE_WALK_LEAF) + return 0; + if (level != (KVM_PGTABLE_MAX_LEVELS - 1)) return -EINVAL; @@ -205,7 +215,8 @@ static int finalize_host_mappings(void) { struct kvm_pgtable_walker walker = { .cb = finalize_host_mappings_walker, - .flags = KVM_PGTABLE_WALK_LEAF, + .flags = KVM_PGTABLE_WALK_LEAF | KVM_PGTABLE_WALK_TABLE_POST, + .arg = pkvm_pgtable.mm_ops, }; return kvm_pgtable_walk(&pkvm_pgtable, 0, BIT(pkvm_pgtable.ia_bits), &walker); @@ -230,19 +241,19 @@ void __noreturn __pkvm_init_finalise(void) if (ret) goto out; - ret = finalize_host_mappings(); - if (ret) - goto out; - pkvm_pgtable_mm_ops = (struct kvm_pgtable_mm_ops) { - .zalloc_page = hyp_zalloc_hyp_page, - .phys_to_virt = hyp_phys_to_virt, - .virt_to_phys = hyp_virt_to_phys, - .get_page = hpool_get_page, - .put_page = hpool_put_page, + .zalloc_page = hyp_zalloc_hyp_page, + .phys_to_virt = hyp_phys_to_virt, + .virt_to_phys = hyp_virt_to_phys, + .get_page = hpool_get_page, + .put_page = hpool_put_page, }; pkvm_pgtable.mm_ops = &pkvm_pgtable_mm_ops; + ret = finalize_host_mappings(); + if (ret) + goto out; + out: /* * We tail-called to here from handle___pkvm_init() and will not return, From patchwork Tue Oct 19 12:12:54 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Quentin Perret X-Patchwork-Id: 12569731 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 16599C433EF for ; Tue, 19 Oct 2021 12:16:48 +0000 (UTC) Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id DC1C56115B for ; Tue, 19 Oct 2021 12:16:47 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.4.1 mail.kernel.org DC1C56115B Authentication-Results: mail.kernel.org; dmarc=fail (p=reject dis=none) header.from=google.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:Cc:To:From:Subject:References: Mime-Version:Message-Id:In-Reply-To:Date:Reply-To:Content-ID: Content-Description:Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc :Resent-Message-ID:List-Owner; bh=s7PNgvxI0+F0URFQ1tY7XHPVtCEVFKmxnuCnOH0p2Zs=; b=J6q0g/GIWe5BtkF8A5Rt38I6Gh v+GppXzqN7ihLXfJtmmCvxXQkcD8zSZVOPUT13CPZB5zfSmMkSGwxOoZ3JSqnWKZbgzB9WEcnB+Jk pc4QZxNVJ1TbMQbP5wmi4gM4CSQwL1imjS9o9DM4OFH9qnZfPCibBEHCC3hV5VJqBCHJ1KbzxpPin hi1RpZcBuGI+P9QJ7LQws7ojORygZULKZqFO8FSZ6/dPMwEFJ7m9a+FfZu9FAlNE3gnw5a+BR+zYw y1J3e7u0s7wJ2U95hgovcJuQGGSzQr9gAjdQGnYkno/YpK/m975XMZqeRsT6YBzz4uZEhFKXoS9jl i5C4ZECA==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1mco1V-0016Cr-M8; Tue, 19 Oct 2021 12:15:17 +0000 Received: from mail-ed1-x549.google.com ([2a00:1450:4864:20::549]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1mcnzd-0015Dy-Bl for linux-arm-kernel@lists.infradead.org; Tue, 19 Oct 2021 12:13:22 +0000 Received: by mail-ed1-x549.google.com with SMTP id x5-20020a50f185000000b003db0f796903so17387211edl.18 for ; Tue, 19 Oct 2021 05:13:19 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20210112; h=date:in-reply-to:message-id:mime-version:references:subject:from:to :cc; bh=Xom4QB7AOMzMrKQIokAKkuFuVfZT5vZhj8rSYX3sNOQ=; b=UEBfi9pZv9JWQ9n6/h/iOXEYKZp+KAlUh2DKJbB4tCbbbpQt6Q9qFpuPdsiYcGSHx4 LX6u3LKvK+ZLLiBEXefMFZZuJL/m57dP0wNf92XMwUS2Vc4dHj+r3kRAM+CL+lqpMimX 354GwtJhsJ/BwC6r/0XdfiJNr9uoaPCzbdcNTOih4LRbc2iBl63nuNe9XPomDGTu4R52 5h9hahfNMP9P3o1jfQpOzoMJ43rFPXjB2Uz7vART5kBtHnDArZemdhxKKysPgIU8UxyD IXWd2hd/An+PUfJ97+ttKWnIHwUIuxY1/PzKFQseQtOWrSYNG0vc08yZilX4NoOhQn46 fInA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=Xom4QB7AOMzMrKQIokAKkuFuVfZT5vZhj8rSYX3sNOQ=; b=UVrzrwyeaksqbxwUKInN8CXD4m9PuRBkitAhXgQZDsxbWvbECiJpvhf5FNvNePI8ix Z/lovHZaqhe1ltzSXgykOzJ/CS6PC+eeQU6c4WgjuV3zRjDBMPPDwHDX0QrG9G03WaoY aPHpi2JouwizwF9sRZmcGXjZ9cv9cdoKX9hVpk3d4PjIwxh/62zu/nJLk5dTqU8Ssf58 F0Sy3O9kWDhi8N1cCtQ2VWAsQPkTf2ecyrnAfsKpONR3fIuAEYYDMkFxTeNk/GlRja8V KT61F5F1UW/YADHas/Ps8dM/6PjAA1x/gWkGxshSF6hXhZeHHMtTNLQ8b2DxyWs+SULK ADBQ== X-Gm-Message-State: AOAM532+bqvKQ8IYdcMzQwp8yHRMh9Oi1woKlXW44SBv1f9Qdc3N3HUP BBay85ygtTjy5Eu30FfpW8NHHvITsHGo X-Google-Smtp-Source: ABdhPJxpv/+Qrz1w29XpbnCupLR0QNbk4QaH0nM4qXe1dCgEnX/Y+92sdhWJIqq0OnYUXAbOKpiNqDTYPy5k X-Received: from luke.lon.corp.google.com ([2a00:79e0:d:210:59ca:401f:83a8:de6d]) (user=qperret job=sendgmr) by 2002:aa7:dbcf:: with SMTP id v15mr53721659edt.243.1634645598342; Tue, 19 Oct 2021 05:13:18 -0700 (PDT) Date: Tue, 19 Oct 2021 13:12:54 +0100 In-Reply-To: <20211019121304.2732332-1-qperret@google.com> Message-Id: <20211019121304.2732332-6-qperret@google.com> Mime-Version: 1.0 References: <20211019121304.2732332-1-qperret@google.com> X-Mailer: git-send-email 2.33.0.1079.g6e70778dc9-goog Subject: [PATCH v2 05/15] KVM: arm64: Hook up ->page_count() for hypervisor stage-1 page-table From: Quentin Perret To: Marc Zyngier , James Morse , Alexandru Elisei , Suzuki K Poulose , Catalin Marinas , Will Deacon , Fuad Tabba , David Brazdil , Andrew Walbran Cc: linux-arm-kernel@lists.infradead.org, kvmarm@lists.cs.columbia.edu, linux-kernel@vger.kernel.org, kernel-team@android.com, qperret@google.com X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20211019_051321_438134_90EB032D X-CRM114-Status: UNSURE ( 9.64 ) X-CRM114-Notice: Please train this message. X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org From: Will Deacon kvm_pgtable_hyp_unmap() relies on the ->page_count() function callback being provided by the memory-management operations for the page-table. Wire up this callback for the hypervisor stage-1 page-table. Signed-off-by: Will Deacon Signed-off-by: Quentin Perret --- arch/arm64/kvm/hyp/nvhe/setup.c | 1 + 1 file changed, 1 insertion(+) diff --git a/arch/arm64/kvm/hyp/nvhe/setup.c b/arch/arm64/kvm/hyp/nvhe/setup.c index 9e5c92237a51..56343b1bb3a9 100644 --- a/arch/arm64/kvm/hyp/nvhe/setup.c +++ b/arch/arm64/kvm/hyp/nvhe/setup.c @@ -247,6 +247,7 @@ void __noreturn __pkvm_init_finalise(void) .virt_to_phys = hyp_virt_to_phys, .get_page = hpool_get_page, .put_page = hpool_put_page, + .page_count = hyp_page_count, }; pkvm_pgtable.mm_ops = &pkvm_pgtable_mm_ops; From patchwork Tue Oct 19 12:12:55 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Quentin Perret X-Patchwork-Id: 12569733 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id DBD83C433F5 for ; Tue, 19 Oct 2021 12:17:22 +0000 (UTC) Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 99A9661260 for ; Tue, 19 Oct 2021 12:17:22 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.4.1 mail.kernel.org 99A9661260 Authentication-Results: mail.kernel.org; dmarc=fail (p=reject dis=none) header.from=google.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:Cc:To:From:Subject:References: Mime-Version:Message-Id:In-Reply-To:Date:Reply-To:Content-ID: Content-Description:Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc :Resent-Message-ID:List-Owner; bh=WpXfCwFURsmM4cakSVLo9nFO7dt+uAaw9hdOg92hJK0=; b=SGQvpj/op5W4pyFU7fkDhJbKP5 AVd8uPzvSEdf7zVibRHcn9PNKC+ZL9OWU1NbRwnBPjXuXecIp4yCeivmBfuywLsxyIIOIVBD/mP7c PPZS7EDVd/1BvDrByJVH5zlsuGZe781n6TYaeek8eamfHT829Vhr9sWIJnCqV0s4s0Y1J7Z7z/p08 vOerTwZ2q6pqdkxhq39QdNuxmy4xB0kGU8V/CVs4w5n92bpm37CM2M8EmqTjar06cucoaPSkduvXu AI4Ue+1U1uMO4MJD8hdAWTDs4azGs24hn8w8Q8klIPHevT4T6r+TBurIzxgTKkVo46dpef+gN2Zb8 HDotFVqQ==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1mco1r-0016MU-97; Tue, 19 Oct 2021 12:15:40 +0000 Received: from mail-wr1-x44a.google.com ([2a00:1450:4864:20::44a]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1mcnze-0015Em-RC for linux-arm-kernel@lists.infradead.org; Tue, 19 Oct 2021 12:13:24 +0000 Received: by mail-wr1-x44a.google.com with SMTP id 75-20020adf82d1000000b00160cbb0f800so10084353wrc.22 for ; Tue, 19 Oct 2021 05:13:22 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20210112; h=date:in-reply-to:message-id:mime-version:references:subject:from:to :cc; bh=lCXO97acIcTVNhdFrwlNIxSBr90mPvzDNoClgWHJCM0=; b=lLI82bpsEIkmjoY5bXVC+fOWz3tBbzQRnGXROzrfoL0Hsv0Kg91MrtueOcbv3Emc/D qaQT0oCkTvnG0PCKOTLDzY6v56OT7Xcsj19OR9UJSMXrfkR9x8ClnzTDgv+H6isG1hCM tFShnrWPTZoIA+XbRBnJGPdh49+d/LzmInXPzxoXDPmstwOFRmG/JKhXFV5i6uSWJVcI 9RX4o1KhQiK2xrPYekIxETgAMU85Q52iJ43Rosf3Bg+xTnx67n6CpMqukh034pJa10wc wLC72mXLvthBUu7yCdluNZmPKe6Y6/szco4QpoMjuBJ3QODkWdoutMHmV/S4TRmiPCZY 9ocw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=lCXO97acIcTVNhdFrwlNIxSBr90mPvzDNoClgWHJCM0=; b=RgyzxV5UYeso3TVIDUdbwggLkv6LOlfQiuOes+KrT66e1QxI6zay0McpkjA6CGNiW2 gsaKms48xLiLaqz5p7RINARIhR+4b131ClM+FA4PVhUL2ER1Lk4N6aD4W9ooBWNhjlay /UkZWeKzWbgNVmyOWKxXAQV7Q+u58OHTtNbjJG6SAXouZ4jT08sv6+ffsSUpEMVg5QDC Qum9Kvexxv5GZ9ZvWutJxt7g2nS/z/mxmlg0/oAt8MqVc4q0uWvUeLt0+Cb9FihZ9isf prSASsaAMlHtVs6LcTexkh3HGRy81NPKRDa7Y1GIESi2e3IGcwp3FiBz70tIREpJL1jx aP6w== X-Gm-Message-State: AOAM532ubUU8fhSxrEpQ3xauttsMNC8dyZIuQ2WTaVz9+NZUEbios5gS LHv0uKigX1CwOHH1R4f9KkVdrY/wfQJ6 X-Google-Smtp-Source: ABdhPJye5OMqyc5G8hN4J/2112S8j3HbKdiZz7xBn4/qs578wVcSY9YucExD5V8rQ+B6hGbkCyayZWZROE/d X-Received: from luke.lon.corp.google.com ([2a00:79e0:d:210:59ca:401f:83a8:de6d]) (user=qperret job=sendgmr) by 2002:a1c:770b:: with SMTP id t11mr5737869wmi.19.1634645600892; Tue, 19 Oct 2021 05:13:20 -0700 (PDT) Date: Tue, 19 Oct 2021 13:12:55 +0100 In-Reply-To: <20211019121304.2732332-1-qperret@google.com> Message-Id: <20211019121304.2732332-7-qperret@google.com> Mime-Version: 1.0 References: <20211019121304.2732332-1-qperret@google.com> X-Mailer: git-send-email 2.33.0.1079.g6e70778dc9-goog Subject: [PATCH v2 06/15] KVM: arm64: Implement kvm_pgtable_hyp_unmap() at EL2 From: Quentin Perret To: Marc Zyngier , James Morse , Alexandru Elisei , Suzuki K Poulose , Catalin Marinas , Will Deacon , Fuad Tabba , David Brazdil , Andrew Walbran Cc: linux-arm-kernel@lists.infradead.org, kvmarm@lists.cs.columbia.edu, linux-kernel@vger.kernel.org, kernel-team@android.com, qperret@google.com X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20211019_051322_925176_BAEA70D0 X-CRM114-Status: GOOD ( 16.31 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org From: Will Deacon Implement kvm_pgtable_hyp_unmap() which can be used to remove hypervisor stage-1 mappings at EL2. Signed-off-by: Will Deacon Signed-off-by: Quentin Perret --- arch/arm64/include/asm/kvm_pgtable.h | 21 ++++++++++ arch/arm64/kvm/hyp/pgtable.c | 63 ++++++++++++++++++++++++++++ 2 files changed, 84 insertions(+) diff --git a/arch/arm64/include/asm/kvm_pgtable.h b/arch/arm64/include/asm/kvm_pgtable.h index 027783829584..9d076f36401d 100644 --- a/arch/arm64/include/asm/kvm_pgtable.h +++ b/arch/arm64/include/asm/kvm_pgtable.h @@ -251,6 +251,27 @@ void kvm_pgtable_hyp_destroy(struct kvm_pgtable *pgt); int kvm_pgtable_hyp_map(struct kvm_pgtable *pgt, u64 addr, u64 size, u64 phys, enum kvm_pgtable_prot prot); +/** + * kvm_pgtable_hyp_unmap() - Remove a mapping from a hypervisor stage-1 page-table. + * @pgt: Page-table structure initialised by kvm_pgtable_hyp_init(). + * @addr: Virtual address from which to remove the mapping. + * @size: Size of the mapping. + * + * The offset of @addr within a page is ignored, @size is rounded-up to + * the next page boundary and @phys is rounded-down to the previous page + * boundary. + * + * TLB invalidation is performed for each page-table entry cleared during the + * unmapping operation and the reference count for the page-table page + * containing the cleared entry is decremented, with unreferenced pages being + * freed. The unmapping operation will stop early if it encounters either an + * invalid page-table entry or a valid block mapping which maps beyond the range + * being unmapped. + * + * Return: Number of bytes unmapped, which may be 0. + */ +u64 kvm_pgtable_hyp_unmap(struct kvm_pgtable *pgt, u64 addr, u64 size); + /** * kvm_get_vtcr() - Helper to construct VTCR_EL2 * @mmfr0: Sanitized value of SYS_ID_AA64MMFR0_EL1 register. diff --git a/arch/arm64/kvm/hyp/pgtable.c b/arch/arm64/kvm/hyp/pgtable.c index 768a58835153..6ad4cb2d6947 100644 --- a/arch/arm64/kvm/hyp/pgtable.c +++ b/arch/arm64/kvm/hyp/pgtable.c @@ -463,6 +463,69 @@ int kvm_pgtable_hyp_map(struct kvm_pgtable *pgt, u64 addr, u64 size, u64 phys, return ret; } +struct hyp_unmap_data { + u64 unmapped; + struct kvm_pgtable_mm_ops *mm_ops; +}; + +static int hyp_unmap_walker(u64 addr, u64 end, u32 level, kvm_pte_t *ptep, + enum kvm_pgtable_walk_flags flag, void * const arg) +{ + kvm_pte_t pte = *ptep, *childp = NULL; + u64 granule = kvm_granule_size(level); + struct hyp_unmap_data *data = arg; + struct kvm_pgtable_mm_ops *mm_ops = data->mm_ops; + + if (!kvm_pte_valid(pte)) + return -EINVAL; + + if (kvm_pte_table(pte, level)) { + childp = kvm_pte_follow(pte, mm_ops); + + if (mm_ops->page_count(childp) != 1) + return 0; + + kvm_clear_pte(ptep); + dsb(ishst); + __tlbi_level(vae2is, __TLBI_VADDR(addr, 0), level); + } else { + if (end - addr < granule) + return -EINVAL; + + kvm_clear_pte(ptep); + dsb(ishst); + __tlbi_level(vale2is, __TLBI_VADDR(addr, 0), level); + data->unmapped += granule; + } + + dsb(ish); + isb(); + mm_ops->put_page(ptep); + + if (childp) + mm_ops->put_page(childp); + + return 0; +} + +u64 kvm_pgtable_hyp_unmap(struct kvm_pgtable *pgt, u64 addr, u64 size) +{ + struct hyp_unmap_data unmap_data = { + .mm_ops = pgt->mm_ops, + }; + struct kvm_pgtable_walker walker = { + .cb = hyp_unmap_walker, + .arg = &unmap_data, + .flags = KVM_PGTABLE_WALK_LEAF | KVM_PGTABLE_WALK_TABLE_POST, + }; + + if (!pgt->mm_ops->page_count) + return 0; + + kvm_pgtable_walk(pgt, addr, size, &walker); + return unmap_data.unmapped; +} + int kvm_pgtable_hyp_init(struct kvm_pgtable *pgt, u32 va_bits, struct kvm_pgtable_mm_ops *mm_ops) { From patchwork Tue Oct 19 12:12:56 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Quentin Perret X-Patchwork-Id: 12569735 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 6C813C433EF for ; Tue, 19 Oct 2021 12:17:48 +0000 (UTC) Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 3784A6135E for ; Tue, 19 Oct 2021 12:17:48 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.4.1 mail.kernel.org 3784A6135E Authentication-Results: mail.kernel.org; dmarc=fail (p=reject dis=none) header.from=google.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:Cc:To:From:Subject:References: Mime-Version:Message-Id:In-Reply-To:Date:Reply-To:Content-ID: Content-Description:Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc :Resent-Message-ID:List-Owner; bh=Qg8H02eq1q5vtg+HVmS+INLweITDKhwj8eWCoY5E8bY=; b=KP7QpXmyxio4sDVWgqdZnkQ31Z hsWbz9DMDPl2hlVyNulrfuJXSpzmDwY0C2pqnRr7d2NhjvNqUs+KvikBZ/lwm1v4fUlBEIoyrB63Q X9N1kM2GA3CaFcZC4wKEsIFeVJjLraMERYGAkBTUnpWc5F8KW0qA08AS5d30AbiOguP1Krrb7boU6 Q0xNtFvrnrT9nuyCdHGGaodrSAm7v6ZwM33o/a0tFMgJd6mdmOp04C6mNTgg53/4Y6sogrUdUYFia nq7hXIPQ/DXlca6KD4oiCSOjHRnpG6uLgx98eP7/4HISh6uwkQyGfKMCv7uhxcdQklYXh4mZc/soo m0kve8Ag==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1mco2K-0016bA-Fq; Tue, 19 Oct 2021 12:16:08 +0000 Received: from mail-wm1-x349.google.com ([2a00:1450:4864:20::349]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1mcnzh-0015GH-9M for linux-arm-kernel@lists.infradead.org; Tue, 19 Oct 2021 12:13:26 +0000 Received: by mail-wm1-x349.google.com with SMTP id l39-20020a05600c1d2700b0030dba1dc6eeso1068303wms.7 for ; Tue, 19 Oct 2021 05:13:24 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20210112; h=date:in-reply-to:message-id:mime-version:references:subject:from:to :cc; bh=ty9iYiagMI0gH20pET1Xndb3iV+CDb06dji5ebe9K8w=; b=SpQpP/yRjdUQxoyHPN7C8cqP88mZaVeLPJ8bNLz3pg1pqcyqLiVJsbc0K2Vjc2jhuQ 2PGXAA+SI+BFXG8WCVUEkCOMHE+olMIAfl/O7rw9/CpMKQ3ycyZbyiqIcqOGev1xjX0N PEgg90LPzx39+XnWldfpg31ucLbcIg1wmtCGmYpSObuVrI1qzD3SzeruVAF1hfcCy/+8 Ij2iODxEL4liU9qZb81s8tlJZ9rIM548MRcoLUE2LPbZmNK/kzfgWJDQYll1z/vSSj11 aZZZauk73uQCoiVzVIT69awzlL+qtPyGpzwKXNXuMBJn77bMEeYOM1FBhCrmd0l57ide dvEQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=ty9iYiagMI0gH20pET1Xndb3iV+CDb06dji5ebe9K8w=; b=WHbLs3wFbG22+QFeyaIHK4CPN7WyWN8dVQrJ3ExzeAlgKjf1kbViW/tfPi1FA8I7qk l541+yc9QWuhTqhLajwqHoAqFoGI8HdtKKGTJnqLJ4uRoEK6PtNc8KYc4ZwXLHV/12u6 CIEyqGxZjcy0VtSuXSy+RP4O3NwkutJLAuuJeWs7wA+XgEHHLpKVuNn+VO2MQQ8WaO69 cgc1CAVxLNXSg/oxM5S/PFrdgedjy/v+5sQBKRzhluvTcu8qlVpWOqSIAzkls7yUF4sA EOevpDHiUSgapHUNEIdopkTzrfVmOqQPh6D0kHtxNwfL1CdwYjp8l97AYtckBMN/rcfx 9a3Q== X-Gm-Message-State: AOAM530P7V8UoZW7808HFxDrWvFcZ8VPWf4p90E0EGjh6DmPNXRDPREI GpOe/7SfQ1nrGKnV4oPJlWloPcMYB6Bz X-Google-Smtp-Source: ABdhPJwYQAu9x65aeAB6QkD0CZV58F2ousV/0NfyOsnnHv4pwVmmNZyY52hWDmpVRmvp0zkLy2sxMRWr0PHi X-Received: from luke.lon.corp.google.com ([2a00:79e0:d:210:59ca:401f:83a8:de6d]) (user=qperret job=sendgmr) by 2002:a7b:c442:: with SMTP id l2mr5662234wmi.131.1634645603276; Tue, 19 Oct 2021 05:13:23 -0700 (PDT) Date: Tue, 19 Oct 2021 13:12:56 +0100 In-Reply-To: <20211019121304.2732332-1-qperret@google.com> Message-Id: <20211019121304.2732332-8-qperret@google.com> Mime-Version: 1.0 References: <20211019121304.2732332-1-qperret@google.com> X-Mailer: git-send-email 2.33.0.1079.g6e70778dc9-goog Subject: [PATCH v2 07/15] KVM: arm64: Introduce kvm_share_hyp() From: Quentin Perret To: Marc Zyngier , James Morse , Alexandru Elisei , Suzuki K Poulose , Catalin Marinas , Will Deacon , Fuad Tabba , David Brazdil , Andrew Walbran Cc: linux-arm-kernel@lists.infradead.org, kvmarm@lists.cs.columbia.edu, linux-kernel@vger.kernel.org, kernel-team@android.com, qperret@google.com X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20211019_051325_363520_031E8F65 X-CRM114-Status: GOOD ( 18.41 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org The create_hyp_mappings() function can currently be called at any point in time. However, its behaviour in protected mode changes widely depending on when it is being called. Prior to KVM init, it is used to create the temporary page-table used to bring-up the hypervisor, and later on it is transparently turned into a 'share' hypercall when the kernel has lost control over the hypervisor stage-1. In order to prepare the ground for also unsharing pages with the hypervisor during guest teardown, introduce a kvm_share_hyp() function to make it clear in which places a share hypercall should be expected, as we will soon need a matching unshare hypercall in all those places. Signed-off-by: Quentin Perret --- arch/arm64/include/asm/kvm_mmu.h | 1 + arch/arm64/kvm/arm.c | 4 ++-- arch/arm64/kvm/fpsimd.c | 4 ++-- arch/arm64/kvm/mmu.c | 19 +++++++++++++------ arch/arm64/kvm/reset.c | 2 +- 5 files changed, 19 insertions(+), 11 deletions(-) diff --git a/arch/arm64/include/asm/kvm_mmu.h b/arch/arm64/include/asm/kvm_mmu.h index 02d378887743..185d0f62b724 100644 --- a/arch/arm64/include/asm/kvm_mmu.h +++ b/arch/arm64/include/asm/kvm_mmu.h @@ -150,6 +150,7 @@ static __always_inline unsigned long __kern_hyp_va(unsigned long v) #include #include +int kvm_share_hyp(void *from, void *to); int create_hyp_mappings(void *from, void *to, enum kvm_pgtable_prot prot); int create_hyp_io_mappings(phys_addr_t phys_addr, size_t size, void __iomem **kaddr, diff --git a/arch/arm64/kvm/arm.c b/arch/arm64/kvm/arm.c index e1cc3f8560e7..5312d5a2ef69 100644 --- a/arch/arm64/kvm/arm.c +++ b/arch/arm64/kvm/arm.c @@ -146,7 +146,7 @@ int kvm_arch_init_vm(struct kvm *kvm, unsigned long type) if (ret) return ret; - ret = create_hyp_mappings(kvm, kvm + 1, PAGE_HYP); + ret = kvm_share_hyp(kvm, kvm + 1); if (ret) goto out_free_stage2_pgd; @@ -343,7 +343,7 @@ int kvm_arch_vcpu_create(struct kvm_vcpu *vcpu) if (err) return err; - return create_hyp_mappings(vcpu, vcpu + 1, PAGE_HYP); + return kvm_share_hyp(vcpu, vcpu + 1); } void kvm_arch_vcpu_postcreate(struct kvm_vcpu *vcpu) diff --git a/arch/arm64/kvm/fpsimd.c b/arch/arm64/kvm/fpsimd.c index 62c0d78da7be..2fe1128d9f3d 100644 --- a/arch/arm64/kvm/fpsimd.c +++ b/arch/arm64/kvm/fpsimd.c @@ -35,11 +35,11 @@ int kvm_arch_vcpu_run_map_fp(struct kvm_vcpu *vcpu) * Make sure the host task thread flags and fpsimd state are * visible to hyp: */ - ret = create_hyp_mappings(ti, ti + 1, PAGE_HYP); + ret = kvm_share_hyp(ti, ti + 1); if (ret) goto error; - ret = create_hyp_mappings(fpsimd, fpsimd + 1, PAGE_HYP); + ret = kvm_share_hyp(fpsimd, fpsimd + 1); if (ret) goto error; diff --git a/arch/arm64/kvm/mmu.c b/arch/arm64/kvm/mmu.c index 0019b2309f70..0cc4b295e525 100644 --- a/arch/arm64/kvm/mmu.c +++ b/arch/arm64/kvm/mmu.c @@ -299,6 +299,17 @@ static int pkvm_share_hyp(phys_addr_t start, phys_addr_t end) return 0; } +int kvm_share_hyp(void *from, void *to) +{ + if (is_kernel_in_hyp_mode()) + return 0; + + if (kvm_host_owns_hyp_mappings()) + return create_hyp_mappings(from, to, PAGE_HYP); + + return pkvm_share_hyp(kvm_kaddr_to_phys(from), kvm_kaddr_to_phys(to)); +} + /** * create_hyp_mappings - duplicate a kernel virtual address range in Hyp mode * @from: The virtual kernel start address of the range @@ -319,12 +330,8 @@ int create_hyp_mappings(void *from, void *to, enum kvm_pgtable_prot prot) if (is_kernel_in_hyp_mode()) return 0; - if (!kvm_host_owns_hyp_mappings()) { - if (WARN_ON(prot != PAGE_HYP)) - return -EPERM; - return pkvm_share_hyp(kvm_kaddr_to_phys(from), - kvm_kaddr_to_phys(to)); - } + if (!kvm_host_owns_hyp_mappings()) + return -EPERM; start = start & PAGE_MASK; end = PAGE_ALIGN(end); diff --git a/arch/arm64/kvm/reset.c b/arch/arm64/kvm/reset.c index 59e4c6e819e9..cf781ec9212a 100644 --- a/arch/arm64/kvm/reset.c +++ b/arch/arm64/kvm/reset.c @@ -113,7 +113,7 @@ static int kvm_vcpu_finalize_sve(struct kvm_vcpu *vcpu) if (!buf) return -ENOMEM; - ret = create_hyp_mappings(buf, buf + reg_sz, PAGE_HYP); + ret = kvm_share_hyp(buf, buf + reg_sz); if (ret) { kfree(buf); return ret; From patchwork Tue Oct 19 12:12:57 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Quentin Perret X-Patchwork-Id: 12569747 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 52E25C433F5 for ; Tue, 19 Oct 2021 12:18:16 +0000 (UTC) Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 225D061260 for ; Tue, 19 Oct 2021 12:18:16 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.4.1 mail.kernel.org 225D061260 Authentication-Results: mail.kernel.org; dmarc=fail (p=reject dis=none) header.from=google.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:Cc:To:From:Subject:References: Mime-Version:Message-Id:In-Reply-To:Date:Reply-To:Content-ID: Content-Description:Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc :Resent-Message-ID:List-Owner; bh=05vMcv582X3cxTD/ljdspUW1NV9h5PXPW5ZAgRS1Ixs=; b=S+ogqfFz75YIX0UMIjtHuMoRgy Etbg0u84yF3kx9by69gXSd7gcoyWPTiIasOwEZi8SPzWCH29r9KqIJk2zgLgqHFDaEEf+wJS4abnN 5tLrMpWgOjsul3pxJy40Zfi5/KdZ04nWc7yF9B15nB990PX7DGEBcsufAyD74CLV8Cr55K0nDtF0M oIagWlmv9zROwPPIaat3utiPw4/MRdvnysOkt12nzxQYq3ksUfDm3mQAtNOIimKz/OhglYxPhyGfE 4ZXFlbhtt66vxIoiRKGp04xYljq7CtxDQY7E1Bpqet4rDZUp++51amLnkFrsrgRurrQ5JVWpff+gG 3g35OOxA==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1mco2v-0016r7-7Q; Tue, 19 Oct 2021 12:16:45 +0000 Received: from mail-wm1-x34a.google.com ([2a00:1450:4864:20::34a]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1mcnzj-0015H1-G1 for linux-arm-kernel@lists.infradead.org; Tue, 19 Oct 2021 12:13:28 +0000 Received: by mail-wm1-x34a.google.com with SMTP id v18-20020a7bcb52000000b00322fea1d5b7so1065151wmj.9 for ; Tue, 19 Oct 2021 05:13:26 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20210112; h=date:in-reply-to:message-id:mime-version:references:subject:from:to :cc; bh=FltoTPFX6jcmVGDt6TNVLWqV+gbyvD26Qibq15L3bkE=; b=jlG1oEJu0XALf3UMY9dDzZLKa57GGILI3CeQZbwtRWFXxRmxyJ4hrufsS2cJNR8YIr JNvbgekimwjNmNstdhzrPkwKZqgPJPeSo0J71SXvKrgiUXiuiHLmruUHs8rcc/indAiX 26CLj+txDLSji5YnWMA6yzm7AtzfIVGTatEphI+y5c26xa90hXLAlmYFL//1dTQQOyPw HMCbzOhB4xv2VomYIlo14xWYxcD4vbtMZhrxk2yL7qWC5KwsBpwln8lsNwynvPrxIQHB Xiss9bsk7TSpL1SH0P53UFrzTXqsorbicgZEDet4/UyMoolHEu3NjoWnHFqaYu3ykP1r VGRA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=FltoTPFX6jcmVGDt6TNVLWqV+gbyvD26Qibq15L3bkE=; b=srg/3SayQJerVn0Ds/wNHtMHCLSOD9gOxQqAPBhIVmFnlC+fyUVwIq5hKlMYB+kqzn doZdAACSNn1WzYPFtUvCEM+SYSUlSEc/tZMnRaS94NA2Te+Y7Nf6TrKsOs9HFls4UIPT 150BGvjTpS1lkMs3m5klwTQjkf/Aa6hCg2eV6Em9VpIY3Jr9b1qtFV02vv5dOu2L7SO1 68wTJ0MtsBpNTR9uK6Sd6zHBHy76MpDtImm1YLYVsTvpPq3dVd+fRuTrD1WOtMP9r5Fh FhvDNjELhImd3GzzMWbqvYiQKZ2Uoksnrbnhpg1elb0WN7mZrmNrUEFwDXpyyWETGDOf 8hfA== X-Gm-Message-State: AOAM533PKSEYboNJJvc7AtaPkQ6CLbFZoVmM9Ii4fIGP/dneuq/apXa9 toJ4rmfHHAVeQctZASRj9oUcM+beCoa9 X-Google-Smtp-Source: ABdhPJzHTfoE95BxQfr07fXGEheE3lVwrZ1OysKwSSC2rt9N5CO+2igZJDqU/WMlnTkRLHPLM8olUAyj+sVU X-Received: from luke.lon.corp.google.com ([2a00:79e0:d:210:59ca:401f:83a8:de6d]) (user=qperret job=sendgmr) by 2002:a1c:2309:: with SMTP id j9mr5432829wmj.189.1634645605661; Tue, 19 Oct 2021 05:13:25 -0700 (PDT) Date: Tue, 19 Oct 2021 13:12:57 +0100 In-Reply-To: <20211019121304.2732332-1-qperret@google.com> Message-Id: <20211019121304.2732332-9-qperret@google.com> Mime-Version: 1.0 References: <20211019121304.2732332-1-qperret@google.com> X-Mailer: git-send-email 2.33.0.1079.g6e70778dc9-goog Subject: [PATCH v2 08/15] KVM: arm64: pkvm: Refcount the pages shared with EL2 From: Quentin Perret To: Marc Zyngier , James Morse , Alexandru Elisei , Suzuki K Poulose , Catalin Marinas , Will Deacon , Fuad Tabba , David Brazdil , Andrew Walbran Cc: linux-arm-kernel@lists.infradead.org, kvmarm@lists.cs.columbia.edu, linux-kernel@vger.kernel.org, kernel-team@android.com, qperret@google.com X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20211019_051327_580620_C2F19D92 X-CRM114-Status: GOOD ( 16.25 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org In order to simplify the page tracking infrastructure at EL2 in nVHE protected mode, move the responsibility of refcounting pages that are shared multiple times on the host. In order to do so, let's create a red-black tree tracking all the PFNs that have been shared, along with a refcount. Signed-off-by: Quentin Perret --- arch/arm64/kvm/mmu.c | 78 ++++++++++++++++++++++++++++++++++++++------ 1 file changed, 68 insertions(+), 10 deletions(-) diff --git a/arch/arm64/kvm/mmu.c b/arch/arm64/kvm/mmu.c index 0cc4b295e525..f0c16ed95974 100644 --- a/arch/arm64/kvm/mmu.c +++ b/arch/arm64/kvm/mmu.c @@ -284,30 +284,88 @@ static phys_addr_t kvm_kaddr_to_phys(void *kaddr) } } -static int pkvm_share_hyp(phys_addr_t start, phys_addr_t end) +struct hyp_shared_pfn { + u64 pfn; + int count; + struct rb_node node; +}; + +static DEFINE_MUTEX(hyp_shared_pfns_lock); +static struct rb_root hyp_shared_pfns = RB_ROOT; + +static struct hyp_shared_pfn *find_shared_pfn(u64 pfn, struct rb_node ***node, + struct rb_node **parent) { - phys_addr_t addr; - int ret; + struct hyp_shared_pfn *this; + + *node = &hyp_shared_pfns.rb_node; + *parent = NULL; + while (**node) { + this = container_of(**node, struct hyp_shared_pfn, node); + *parent = **node; + if (this->pfn < pfn) + *node = &((**node)->rb_left); + else if (this->pfn > pfn) + *node = &((**node)->rb_right); + else + return this; + } - for (addr = ALIGN_DOWN(start, PAGE_SIZE); addr < end; addr += PAGE_SIZE) { - ret = kvm_call_hyp_nvhe(__pkvm_host_share_hyp, - __phys_to_pfn(addr)); - if (ret) - return ret; + return NULL; +} + +static int share_pfn_hyp(u64 pfn) +{ + struct rb_node **node, *parent; + struct hyp_shared_pfn *this; + int ret = 0; + + mutex_lock(&hyp_shared_pfns_lock); + this = find_shared_pfn(pfn, &node, &parent); + if (this) { + this->count++; + goto unlock; } - return 0; + this = kzalloc(sizeof(*this), GFP_KERNEL); + if (!this) { + ret = -ENOMEM; + goto unlock; + } + + this->pfn = pfn; + this->count = 1; + rb_link_node(&this->node, parent, node); + rb_insert_color(&this->node, &hyp_shared_pfns); + ret = kvm_call_hyp_nvhe(__pkvm_host_share_hyp, pfn, 1); +unlock: + mutex_unlock(&hyp_shared_pfns_lock); + + return ret; } int kvm_share_hyp(void *from, void *to) { + phys_addr_t start, end, cur; + u64 pfn; + int ret; + if (is_kernel_in_hyp_mode()) return 0; if (kvm_host_owns_hyp_mappings()) return create_hyp_mappings(from, to, PAGE_HYP); - return pkvm_share_hyp(kvm_kaddr_to_phys(from), kvm_kaddr_to_phys(to)); + start = ALIGN_DOWN(kvm_kaddr_to_phys(from), PAGE_SIZE); + end = PAGE_ALIGN(kvm_kaddr_to_phys(to)); + for (cur = start; cur < end; cur += PAGE_SIZE) { + pfn = __phys_to_pfn(cur); + ret = share_pfn_hyp(pfn); + if (ret) + return ret; + } + + return 0; } /** From patchwork Tue Oct 19 12:12:58 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Quentin Perret X-Patchwork-Id: 12569749 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id A3191C433EF for ; Tue, 19 Oct 2021 12:19:03 +0000 (UTC) Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 761FF61360 for ; Tue, 19 Oct 2021 12:19:03 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.4.1 mail.kernel.org 761FF61360 Authentication-Results: mail.kernel.org; dmarc=fail (p=reject dis=none) header.from=google.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:Cc:To:From:Subject:References: Mime-Version:Message-Id:In-Reply-To:Date:Reply-To:Content-ID: Content-Description:Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc :Resent-Message-ID:List-Owner; bh=T/kXQFGsRTRKr/Nc9clUCKzQDcIzHv+BEdIb18pGZYI=; b=JYR8wMo0OrqChIRTesTwh6LBKf stDx/d6rEu0fmdCGItK/IdWJq3ANEWr8mFMV2up1i9o//67znGzzxar5x4RkpheCqIOcM5bz2NChf 3AA6Pt9pKvAXMxwi7//wj5wjMFCcouYD7RXGGHegMNScY5oHIQhAd8zUHFD/ILS5BLTO3ZqeaIyAC olV8j72VMd7lBnxezXv24F2RVDsqWpXtB+mh6tKQyVDf73CHK07+Xjn5bIxDJAVBwzI3iCq70upQz URGPcrDhqRJGYBEyS9FqibzA+snf+sxDkIrb7EKjRTnmKDR9seewyPXVIv2uVnPvWJzwDjlgxqGTC pp+jx+kQ==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1mco3V-00179m-JX; Tue, 19 Oct 2021 12:17:22 +0000 Received: from mail-wr1-x449.google.com ([2a00:1450:4864:20::449]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1mcnzl-0015Hw-Rp for linux-arm-kernel@lists.infradead.org; Tue, 19 Oct 2021 12:13:31 +0000 Received: by mail-wr1-x449.google.com with SMTP id r16-20020adfbb10000000b00160958ed8acso10048963wrg.16 for ; Tue, 19 Oct 2021 05:13:29 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20210112; h=date:in-reply-to:message-id:mime-version:references:subject:from:to :cc; bh=lFnO3V1vcCaxCQp+VktMJcKCn5jbqprlvSckwnNsC2I=; b=UesM75tSQOr45py6fmGldxnFZ9qhEpvDJtfc4apxLzl+vaSpskjZwXcD+vvH4lZb71 As8F+7i1zzMOpoNT7YUGxMUVpiB8c0NDcbRvtlAROdU3IQYsujSlMzJLH9nvz+6DLVHK vJRzQS7wBFHQO3YkY6Xec9Pwo6xYZSj30VNWKHbaOabamzLvVtuZDneBjv1pLT8mGCYX DcoRFa8C8Iwqa1uNt0xjG6wxks4HeuMis1JBBY2eEu5k9YvLWHzHgOOhKtYnx5KcDzGR +xuNnU1Xky1kYEkg3hTbTL4NVyJOzh3NB457XGrqSEC0sX/d9IlZuLvuuYTnTBcm4Y1m ZRwA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=lFnO3V1vcCaxCQp+VktMJcKCn5jbqprlvSckwnNsC2I=; b=D0z9KovJkQCxeDmlnPwMLq9LfC1cWBA3HUz0E5QZwya6UxZlME/c//nKE7qYS/MWMB t/2EDUSaOsy1+Xju7UsALZEx956dTEF/K94U4JFopu+v6kasXz2IIHTmUFIWjcnO4DY8 T0l6xWvoP6fgVWBph1DSQ8rCsF824nDKSV2AEj2Wb+iGTln8+yhPnuL/EaO6d9Y+H9r1 28FiQ9hN9ZgG7g+iuzHjmFDrcIzCpTw87ozHj2zWoLYEqWVbtNc3UtOtxLTlgo6oVYFU 7c+JIWL/pBX8V6RmE8rEMG1kOAc3SYq3eP+77Uy8+KlccgR1Mw/IJ7FX88VbVWG3DGPr NqvA== X-Gm-Message-State: AOAM532xfy6dXYRkQUgdIMp4ZAmU/MElYoSaKBmuVlWOF7ZaNRENmjqC RSEg248ANA0IU0ty4yTb3RlG7ml81dDV X-Google-Smtp-Source: ABdhPJzM8Yi1zkltqsZIknAWVlvpJd+76GDP3lU7clYj1DZGyxNzgbxFRtOLXsCPKmvrYy6VEEMSHUHhKXXR X-Received: from luke.lon.corp.google.com ([2a00:79e0:d:210:59ca:401f:83a8:de6d]) (user=qperret job=sendgmr) by 2002:a7b:c441:: with SMTP id l1mr5486888wmi.69.1634645607924; Tue, 19 Oct 2021 05:13:27 -0700 (PDT) Date: Tue, 19 Oct 2021 13:12:58 +0100 In-Reply-To: <20211019121304.2732332-1-qperret@google.com> Message-Id: <20211019121304.2732332-10-qperret@google.com> Mime-Version: 1.0 References: <20211019121304.2732332-1-qperret@google.com> X-Mailer: git-send-email 2.33.0.1079.g6e70778dc9-goog Subject: [PATCH v2 09/15] KVM: arm64: Extend pkvm_page_state enumeration to handle absent pages From: Quentin Perret To: Marc Zyngier , James Morse , Alexandru Elisei , Suzuki K Poulose , Catalin Marinas , Will Deacon , Fuad Tabba , David Brazdil , Andrew Walbran Cc: linux-arm-kernel@lists.infradead.org, kvmarm@lists.cs.columbia.edu, linux-kernel@vger.kernel.org, kernel-team@android.com, qperret@google.com X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20211019_051329_944663_F5FA4722 X-CRM114-Status: GOOD ( 12.58 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org From: Will Deacon Explicitly name the combination of SW0 | SW1 as reserved in the pte and introduce a new PKVM_NOPAGE meta-state which, although not directly stored in the software bits of the pte, can be used to represent an entry for which there is no underlying page. This is distinct from an invalid pte, as stage-2 identity mappings for the host are created lazily and so an invalid pte there is the same as a valid mapping for the purposes of ownership information. This state will be used for permission checking during page transitions in later patches. Signed-off-by: Will Deacon Signed-off-by: Quentin Perret --- arch/arm64/kvm/hyp/include/nvhe/mem_protect.h | 5 +++++ 1 file changed, 5 insertions(+) diff --git a/arch/arm64/kvm/hyp/include/nvhe/mem_protect.h b/arch/arm64/kvm/hyp/include/nvhe/mem_protect.h index b58c910babaf..56445586c755 100644 --- a/arch/arm64/kvm/hyp/include/nvhe/mem_protect.h +++ b/arch/arm64/kvm/hyp/include/nvhe/mem_protect.h @@ -24,6 +24,11 @@ enum pkvm_page_state { PKVM_PAGE_OWNED = 0ULL, PKVM_PAGE_SHARED_OWNED = KVM_PGTABLE_PROT_SW0, PKVM_PAGE_SHARED_BORROWED = KVM_PGTABLE_PROT_SW1, + __PKVM_PAGE_RESERVED = KVM_PGTABLE_PROT_SW0 | + KVM_PGTABLE_PROT_SW1, + + /* Meta-states which aren't encoded directly in the PTE's SW bits */ + PKVM_NOPAGE, }; #define PKVM_PAGE_STATE_PROT_MASK (KVM_PGTABLE_PROT_SW0 | KVM_PGTABLE_PROT_SW1) From patchwork Tue Oct 19 12:12:59 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Quentin Perret X-Patchwork-Id: 12569751 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id ADDE7C433EF for ; Tue, 19 Oct 2021 12:19:43 +0000 (UTC) Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 73CA861260 for ; Tue, 19 Oct 2021 12:19:43 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.4.1 mail.kernel.org 73CA861260 Authentication-Results: mail.kernel.org; dmarc=fail (p=reject dis=none) header.from=google.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:Cc:To:From:Subject:References: Mime-Version:Message-Id:In-Reply-To:Date:Reply-To:Content-ID: Content-Description:Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc :Resent-Message-ID:List-Owner; bh=7m4wmqFpgrDtljMnClRcIS5N7J3uWVcpmIAYqQwN3nQ=; b=gU+GOJAc8ABwJgZxPBckWU2JCX AQxT3EP1VGvy6HE5oREGNBiahO2hQhCfU8h77X0lhV0B2r8tXGNkmatFNoMJGegDyrY76RAC0/P90 uqFG6M76pQrNWXYlb2GqAv3Lo5uZ1IXv8MpxVe4PGdArmdb7hzILFtpN6TXYEMY3Cg2sIstSrZqbX Q3Q2Pr4f4xtVQC2X+sUc937kvdU1KyjXyWGVnHtDqwWvkKdvfB/SdCNdTuQDCZY8YC3K+DpbecgKV eCR7LcTq06kVGRpVe2cPhm6ZG8RNr6WiWGOl1T4AneP12MbFopK16faBOxU7pLgiDBLOo733xvlkz tG+UHILg==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1mco47-0017Sk-PR; Tue, 19 Oct 2021 12:18:00 +0000 Received: from mail-wr1-x44a.google.com ([2a00:1450:4864:20::44a]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1mcnzo-0015Jg-8C for linux-arm-kernel@lists.infradead.org; Tue, 19 Oct 2021 12:13:33 +0000 Received: by mail-wr1-x44a.google.com with SMTP id 10-20020a5d47aa000000b001610cbda93dso10036934wrb.23 for ; Tue, 19 Oct 2021 05:13:31 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20210112; h=date:in-reply-to:message-id:mime-version:references:subject:from:to :cc; bh=XSGm2ZIKVrWUlcjyigReTRCR+F+26tmq3jrTuKF0cGY=; b=Ff2j/4N+CqKMpFaKt5pCbLJb0KMSeXSN5oHh9p1pg7HOlQUt4WVieG3ZBh7ENNwFm5 viyVQrMK1sbcJirVOPfonTx/8M/aJhkeJ5B0u39TrugRP37ewl3yR7JqNBkgGs5haXuK 2fi9U7qJQGDInMSacShFQe6bjTZGdOvZEJjC2u9S3JYEC9A/zFpkeAbhlKyXkFOd6lwC oL4KmJI4uy8trhkrcZEQPwkbiQ5Bl3dvWof9sI+QQEALuNIQmcBzTtZ0TfprSuIPENHy ufCkQwqZGkNRKzejuz/j5PitlWgZX9pmE79oY8xDJrop9UJlUpRgizFQwE0bT83ExOBQ UJSA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=XSGm2ZIKVrWUlcjyigReTRCR+F+26tmq3jrTuKF0cGY=; b=FAp8FUnjOqGVlFdCyOXwfJYrqu+nF3RLRZPJ0AEVt2Odk8HhXvY4h5ijD8y12o6oIB 6YkCKBqOH9wWHgdABSmZ6ArB+wI8Xn7ciRZlw01R5+09LFROyzu/uA1fSvB9UoM3QHSF CJMm3BHdZL4ZVx7QdgYU5m1IjFoHXXwtLaz3NG5K4QBDFqOmixrg3oTxoUhAr7aRwGhJ yojYguu7JQYaEWnIph+Wtip1ul06bOLu/+mNx1UCFXH7VM5X3jTzCKpdaNj/3mqjYap6 Qf2xVeOGOc5EuldBMOzEapW0aCPWcGVyKZ5B34AWlP7zSfFQkj34mhcYsP5V+UYCWe0P vwPA== X-Gm-Message-State: AOAM532qagqvXPktK2T3xtaMOrLc+9K/EMuBIzAlmNsH4iKY++79BzJm I2Hb7UOew58I+UuRMTgHvSgoNw5o0Wmc X-Google-Smtp-Source: ABdhPJwm42LEsA7dym14pML9UYuqWVeZUfAiAtE2QfcZAOVwsec8LgosKo8zwZK3COAZq6Dpj3+/8mX1AvV+ X-Received: from luke.lon.corp.google.com ([2a00:79e0:d:210:59ca:401f:83a8:de6d]) (user=qperret job=sendgmr) by 2002:adf:b302:: with SMTP id j2mr44597972wrd.285.1634645610149; Tue, 19 Oct 2021 05:13:30 -0700 (PDT) Date: Tue, 19 Oct 2021 13:12:59 +0100 In-Reply-To: <20211019121304.2732332-1-qperret@google.com> Message-Id: <20211019121304.2732332-11-qperret@google.com> Mime-Version: 1.0 References: <20211019121304.2732332-1-qperret@google.com> X-Mailer: git-send-email 2.33.0.1079.g6e70778dc9-goog Subject: [PATCH v2 10/15] KVM: arm64: Introduce wrappers for host and hyp spin lock accessors From: Quentin Perret To: Marc Zyngier , James Morse , Alexandru Elisei , Suzuki K Poulose , Catalin Marinas , Will Deacon , Fuad Tabba , David Brazdil , Andrew Walbran Cc: linux-arm-kernel@lists.infradead.org, kvmarm@lists.cs.columbia.edu, linux-kernel@vger.kernel.org, kernel-team@android.com, qperret@google.com X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20211019_051332_360242_39FC6E7F X-CRM114-Status: GOOD ( 13.30 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org From: Will Deacon In preparation for adding additional locked sections for manipulating page-tables at EL2, introduce some simple wrappers around the host and hypervisor locks so that it's a bit easier to read and bit more difficult to take the wrong lock (or even take them in the wrong order). Signed-off-by: Will Deacon Signed-off-by: Quentin Perret --- arch/arm64/kvm/hyp/nvhe/mem_protect.c | 32 ++++++++++++++++++++++----- 1 file changed, 26 insertions(+), 6 deletions(-) diff --git a/arch/arm64/kvm/hyp/nvhe/mem_protect.c b/arch/arm64/kvm/hyp/nvhe/mem_protect.c index 1f0fb0780622..c111e0fcfa0a 100644 --- a/arch/arm64/kvm/hyp/nvhe/mem_protect.c +++ b/arch/arm64/kvm/hyp/nvhe/mem_protect.c @@ -27,6 +27,26 @@ static struct hyp_pool host_s2_pool; const u8 pkvm_hyp_id = 1; +static void host_lock_component(void) +{ + hyp_spin_lock(&host_kvm.lock); +} + +static void host_unlock_component(void) +{ + hyp_spin_unlock(&host_kvm.lock); +} + +static void hyp_lock_component(void) +{ + hyp_spin_lock(&pkvm_pgd_lock); +} + +static void hyp_unlock_component(void) +{ + hyp_spin_unlock(&pkvm_pgd_lock); +} + static void *host_s2_zalloc_pages_exact(size_t size) { return hyp_alloc_pages(&host_s2_pool, get_order(size)); @@ -327,14 +347,14 @@ static int host_stage2_idmap(u64 addr) prot = is_memory ? PKVM_HOST_MEM_PROT : PKVM_HOST_MMIO_PROT; - hyp_spin_lock(&host_kvm.lock); + host_lock_component(); ret = host_stage2_adjust_range(addr, &range); if (ret) goto unlock; ret = host_stage2_idmap_locked(range.start, range.end - range.start, prot); unlock: - hyp_spin_unlock(&host_kvm.lock); + host_unlock_component(); return ret; } @@ -358,8 +378,8 @@ int __pkvm_host_share_hyp(u64 pfn) if (!addr_is_memory(addr)) return -EINVAL; - hyp_spin_lock(&host_kvm.lock); - hyp_spin_lock(&pkvm_pgd_lock); + host_lock_component(); + hyp_lock_component(); ret = kvm_pgtable_get_leaf(&host_kvm.pgt, addr, &pte, NULL); if (ret) @@ -421,8 +441,8 @@ int __pkvm_host_share_hyp(u64 pfn) BUG_ON(ret); unlock: - hyp_spin_unlock(&pkvm_pgd_lock); - hyp_spin_unlock(&host_kvm.lock); + hyp_unlock_component(); + host_unlock_component(); return ret; } From patchwork Tue Oct 19 12:13:00 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Quentin Perret X-Patchwork-Id: 12569753 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 3748FC433EF for ; Tue, 19 Oct 2021 12:20:30 +0000 (UTC) Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id F238760F25 for ; Tue, 19 Oct 2021 12:20:29 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.4.1 mail.kernel.org F238760F25 Authentication-Results: mail.kernel.org; dmarc=fail (p=reject dis=none) header.from=google.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:Cc:To:From:Subject:References: Mime-Version:Message-Id:In-Reply-To:Date:Reply-To:Content-ID: Content-Description:Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc :Resent-Message-ID:List-Owner; bh=C7iy3gn2PNR+SOSlVJaFpeH5QmIOzR6tT7P8qUIUQV8=; b=Pg7Fp5Y0TJXsBBHzl2zIMKqX5y xSwtE2gpJL8+DV/F59tuAFnZuF0wj9FPOKKPknTzgNf2X3Srs5+1/d7JrSCwvQnweBdA7WRgdL+g6 G58BP3bRAefmYiL26TvkFTszdqKpb3M09Z1eJZ1TfsO/FBBg/lCy23hs5AW5sIzr2VB9RozGkZ4WJ fTapmH7bKFVUfuK8zPcpKyXRoldHddkgpO4vKCffobPiGTJNWA5PcorUvQ5nAwSu7CAqciOiqbETO NgJDFUrSJY/fxhoAgLEs8oUj+hWPNsRyC2Y4p+Yny9m09RH+Y/4AAqeCqntfvi6F1N7uND7YBJuGN N1qrO2iA==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1mco4n-0017lF-0s; Tue, 19 Oct 2021 12:18:42 +0000 Received: from mail-wm1-x34a.google.com ([2a00:1450:4864:20::34a]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1mcnzq-0015Kx-Ji for linux-arm-kernel@lists.infradead.org; Tue, 19 Oct 2021 12:13:36 +0000 Received: by mail-wm1-x34a.google.com with SMTP id d7-20020a1c7307000000b0030d6982305bso662370wmb.5 for ; Tue, 19 Oct 2021 05:13:33 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20210112; h=date:in-reply-to:message-id:mime-version:references:subject:from:to :cc; bh=ujj+7B+0nuZuoSuPZCEqJilB3KVDDbovLbdFeyFEdl4=; b=Z36MuVbabgtLcQJafM4rih1H9ZRXiETiB3WImm4G8LI3SOhi/roAv2h0LV2C7/DSQl nmrSsgXGZW+pxC5jEUB/4n4FWkFXLRAan+kNQWF6VlX08HYf3v7IclApBg61Cgwz/WZT a+eOLK9mz2Zy+hTo27dvPDSxhTNljl0ZH2SB+xcskO+vVkBntf7xW/RxP9uppeP9QIsf 2phhSYPRu2DbI41PCsQFHA06lrqm4QDaf6DSwSqacBpfRVUSb9+haZEx3DCn3otJIDDH TACCyDDp2WEY/GE6fkQbvLNpXgkkjDsGai+UPTXHl24rQ2+SbbheB87qi/bAIHG7QEtT rPlg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=ujj+7B+0nuZuoSuPZCEqJilB3KVDDbovLbdFeyFEdl4=; b=Brj/s1s/HxxTRXwPxuWrVzuiQN87BinVYiHwuRE8QMUmjwQlbmFSHDgkRWX9p3n+bf ZCj28t5ii4tdmAI7kfUuGUPG8YA89IKDIUBW+HKw8gfTEMH3cjrffFg2Z0tGnQWSWLMh VeBDuAqX4/pKS4VEM22bJt3TzfCdaGKtLUYj1sslKtiPQ8hkSnM1jYhxZ7xmbrt8VBp2 0ogc+JYE38BjnzVWm2loJFJAgwuVuOGieTTERy75esgQ1BNxplsuOxWhlWYr/A3SWySc wKvMVXqT7Fz0khcbs8V9BPvhqpLuixA+MyS6Ye06IkNA5ufbTEXiOR0LdbTfHF/y62Mw yiNQ== X-Gm-Message-State: AOAM531Mlq0Uky1y7EeKuYgCyRdVe/TYDK6F+M0AOdCWLU1X+w2UOW16 v3kQdT+ZYdmtm6IRSd41zWzIvyrFefmP X-Google-Smtp-Source: ABdhPJzPSTaE5jzLKB1zddUidDr+xxPerq5HIbOVn52bHTZu5cFEMbPaBO1n4f6RCVVoKM6rqb4L5boRccfl X-Received: from luke.lon.corp.google.com ([2a00:79e0:d:210:59ca:401f:83a8:de6d]) (user=qperret job=sendgmr) by 2002:a05:600c:154b:: with SMTP id f11mr5592079wmg.143.1634645612364; Tue, 19 Oct 2021 05:13:32 -0700 (PDT) Date: Tue, 19 Oct 2021 13:13:00 +0100 In-Reply-To: <20211019121304.2732332-1-qperret@google.com> Message-Id: <20211019121304.2732332-12-qperret@google.com> Mime-Version: 1.0 References: <20211019121304.2732332-1-qperret@google.com> X-Mailer: git-send-email 2.33.0.1079.g6e70778dc9-goog Subject: [PATCH v2 11/15] KVM: arm64: Implement do_share() helper for sharing memory From: Quentin Perret To: Marc Zyngier , James Morse , Alexandru Elisei , Suzuki K Poulose , Catalin Marinas , Will Deacon , Fuad Tabba , David Brazdil , Andrew Walbran Cc: linux-arm-kernel@lists.infradead.org, kvmarm@lists.cs.columbia.edu, linux-kernel@vger.kernel.org, kernel-team@android.com, qperret@google.com X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20211019_051334_716641_6B3D77B3 X-CRM114-Status: GOOD ( 15.82 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org From: Will Deacon By default, protected KVM isolates memory pages so that they are accessible only to their owner: be it the host kernel, the hypervisor at EL2 or (in future) the guest. Establishing shared-memory regions between these components therefore involves a transition for each page so that the owner can share memory with a borrower under a certain set of permissions. Introduce a do_share() helper for safely sharing a memory region between two components. Currently, only host-to-hyp sharing is implemented, but the code is easily extended to handle other combinations and the permission checks for each component are reusable. Signed-off-by: Will Deacon Signed-off-by: Quentin Perret --- arch/arm64/kvm/hyp/nvhe/mem_protect.c | 237 ++++++++++++++++++++++++++ 1 file changed, 237 insertions(+) diff --git a/arch/arm64/kvm/hyp/nvhe/mem_protect.c b/arch/arm64/kvm/hyp/nvhe/mem_protect.c index c111e0fcfa0a..d87cdd53dd21 100644 --- a/arch/arm64/kvm/hyp/nvhe/mem_protect.c +++ b/arch/arm64/kvm/hyp/nvhe/mem_protect.c @@ -460,3 +460,240 @@ void handle_host_mem_abort(struct kvm_cpu_context *host_ctxt) ret = host_stage2_idmap(addr); BUG_ON(ret && ret != -EAGAIN); } + +/* This corresponds to locking order */ +enum pkvm_component_id { + PKVM_ID_HOST, + PKVM_ID_HYP, +}; + +struct pkvm_mem_transition { + u64 nr_pages; + + struct { + enum pkvm_component_id id; + /* Address in the initiator's address space */ + u64 addr; + + union { + struct { + /* Address in the completer's address space */ + u64 completer_addr; + } host; + }; + } initiator; + + struct { + enum pkvm_component_id id; + } completer; +}; + +struct pkvm_mem_share { + const struct pkvm_mem_transition tx; + const enum kvm_pgtable_prot prot; +}; + +struct check_walk_data { + enum pkvm_page_state desired; + enum pkvm_page_state (*get_page_state)(kvm_pte_t pte); +}; + +static int __check_page_state_visitor(u64 addr, u64 end, u32 level, + kvm_pte_t *ptep, + enum kvm_pgtable_walk_flags flag, + void * const arg) +{ + struct check_walk_data *d = arg; + kvm_pte_t pte = *ptep; + + if (kvm_pte_valid(pte) && !addr_is_memory(kvm_pte_to_phys(pte))) + return -EINVAL; + + return d->get_page_state(pte) == d->desired ? 0 : -EPERM; +} + +static int check_page_state_range(struct kvm_pgtable *pgt, u64 addr, u64 size, + struct check_walk_data *data) +{ + struct kvm_pgtable_walker walker = { + .cb = __check_page_state_visitor, + .arg = data, + .flags = KVM_PGTABLE_WALK_LEAF, + }; + + return kvm_pgtable_walk(pgt, addr, size, &walker); +} + +static enum pkvm_page_state host_get_page_state(kvm_pte_t pte) +{ + if (!kvm_pte_valid(pte) && pte) + return PKVM_NOPAGE; + + return pkvm_getstate(kvm_pgtable_stage2_pte_prot(pte)); +} + +static int __host_check_page_state_range(u64 addr, u64 size, + enum pkvm_page_state state) +{ + struct check_walk_data d = { + .desired = state, + .get_page_state = host_get_page_state, + }; + + hyp_assert_lock_held(&host_kvm.lock); + return check_page_state_range(&host_kvm.pgt, addr, size, &d); +} + +static int __host_set_page_state_range(u64 addr, u64 size, + enum pkvm_page_state state) +{ + enum kvm_pgtable_prot prot = pkvm_mkstate(PKVM_HOST_MEM_PROT, state); + + return host_stage2_idmap_locked(addr, size, prot); +} + +static int host_request_owned_transition(u64 *completer_addr, + const struct pkvm_mem_transition *tx) +{ + u64 size = tx->nr_pages * PAGE_SIZE; + u64 addr = tx->initiator.addr; + + *completer_addr = tx->initiator.host.completer_addr; + return __host_check_page_state_range(addr, size, PKVM_PAGE_OWNED); +} + +static int host_initiate_share(u64 *completer_addr, + const struct pkvm_mem_transition *tx) +{ + u64 size = tx->nr_pages * PAGE_SIZE; + u64 addr = tx->initiator.addr; + + *completer_addr = tx->initiator.host.completer_addr; + return __host_set_page_state_range(addr, size, PKVM_PAGE_SHARED_OWNED); +} + +static enum pkvm_page_state hyp_get_page_state(kvm_pte_t pte) +{ + if (!kvm_pte_valid(pte)) + return PKVM_NOPAGE; + + return pkvm_getstate(kvm_pgtable_stage2_pte_prot(pte)); +} + +static int __hyp_check_page_state_range(u64 addr, u64 size, + enum pkvm_page_state state) +{ + struct check_walk_data d = { + .desired = state, + .get_page_state = hyp_get_page_state, + }; + + hyp_assert_lock_held(&pkvm_pgd_lock); + return check_page_state_range(&pkvm_pgtable, addr, size, &d); +} + +static bool __hyp_ack_skip_pgtable_check(const struct pkvm_mem_transition *tx) +{ + return !(IS_ENABLED(CONFIG_NVHE_EL2_DEBUG) || + tx->initiator.id != PKVM_ID_HOST); +} + +static int hyp_ack_share(u64 addr, const struct pkvm_mem_transition *tx, + enum kvm_pgtable_prot perms) +{ + u64 size = tx->nr_pages * PAGE_SIZE; + + if (perms != PAGE_HYP) + return -EPERM; + + if (__hyp_ack_skip_pgtable_check(tx)) + return 0; + + return __hyp_check_page_state_range(addr, size, PKVM_NOPAGE); +} + +static int hyp_complete_share(u64 addr, const struct pkvm_mem_transition *tx, + enum kvm_pgtable_prot perms) +{ + void *start = (void *)addr, *end = start + (tx->nr_pages * PAGE_SIZE); + enum kvm_pgtable_prot prot; + + prot = pkvm_mkstate(perms, PKVM_PAGE_SHARED_BORROWED); + return pkvm_create_mappings_locked(start, end, prot); +} + +static int check_share(struct pkvm_mem_share *share) +{ + const struct pkvm_mem_transition *tx = &share->tx; + u64 completer_addr; + int ret; + + switch (tx->initiator.id) { + case PKVM_ID_HOST: + ret = host_request_owned_transition(&completer_addr, tx); + break; + default: + ret = -EINVAL; + } + + if (ret) + return ret; + + switch (tx->completer.id) { + case PKVM_ID_HYP: + ret = hyp_ack_share(completer_addr, tx, share->prot); + break; + default: + ret = -EINVAL; + } + + return ret; +} + +static int __do_share(struct pkvm_mem_share *share) +{ + const struct pkvm_mem_transition *tx = &share->tx; + u64 completer_addr; + int ret; + + switch (tx->initiator.id) { + case PKVM_ID_HOST: + ret = host_initiate_share(&completer_addr, tx); + break; + default: + ret = -EINVAL; + } + + if (ret) + return ret; + + switch (tx->completer.id) { + case PKVM_ID_HYP: + ret = hyp_complete_share(completer_addr, tx, share->prot); + break; + default: + ret = -EINVAL; + } + + return ret; +} + +/* + * do_share(): + * + * The page owner grants access to another component with a given set + * of permissions. + * + * Initiator: OWNED => SHARED_OWNED + * Completer: NOPAGE => SHARED_BORROWED + */ +static int do_share(struct pkvm_mem_share *share) +{ + int ret; + + ret = check_share(share); + if (ret) + return ret; + + return WARN_ON(__do_share(share)); +} From patchwork Tue Oct 19 12:13:01 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Quentin Perret X-Patchwork-Id: 12569755 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id C3146C433F5 for ; Tue, 19 Oct 2021 12:21:12 +0000 (UTC) Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 8F9BD60F25 for ; Tue, 19 Oct 2021 12:21:12 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.4.1 mail.kernel.org 8F9BD60F25 Authentication-Results: mail.kernel.org; dmarc=fail (p=reject dis=none) header.from=google.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:Cc:To:From:Subject:References: Mime-Version:Message-Id:In-Reply-To:Date:Reply-To:Content-ID: Content-Description:Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc :Resent-Message-ID:List-Owner; bh=ia7sxkGcZxQHLXXhmzJ+uSmaWv35uAXZ3u4iy2NQQmo=; b=w4MK9mIgGIg4Y/xMbf7s7wapyc GiaqEIHfP7avcH9Be0nqMN0VvM1l8QpJ/7gbnWr+r1S7KTJyFSkBd3of8N/j/L7WELSzv/XwcwbMz 1xyV6z4w/ZpcIjFtFzWWGqLunFIRrsW4zIkWb+Qrl+mKcR0T3k1H1BowwT1P0jO8XQlL+U+d6ptA+ uFYQeDZ97p0weL866WxmC2T9+o9Md3BVSQWg3nol9JOSLu4fEsNI2J69t3ou+mqnuMFvlCNqb1YFA G7HjxbZrov/BJMVTy2EBxwjttn+rfcFxK26o9LxWAsNCwZRj1WmiH1StRxdoWowsRCOPWEFnQ+tpC aRNvV15A==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1mco5a-00186D-6o; Tue, 19 Oct 2021 12:19:31 +0000 Received: from mail-wr1-x449.google.com ([2a00:1450:4864:20::449]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1mcnzs-0015M0-K3 for linux-arm-kernel@lists.infradead.org; Tue, 19 Oct 2021 12:13:38 +0000 Received: by mail-wr1-x449.google.com with SMTP id 41-20020adf812c000000b00160dfbfe1a2so10117927wrm.3 for ; Tue, 19 Oct 2021 05:13:35 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20210112; h=date:in-reply-to:message-id:mime-version:references:subject:from:to :cc; bh=Ck8KqbWWMae7AkcYUcDJVvLhTKwYixox75Gde7aqwnI=; b=N1X0+fMmX2ELYVHNfvnYshgrGY1QPYHtzxD2UrOgV3PkeCujsVtLhyaC7O9FoW3Ppl P8iizGRJTkEPDqs+zvQodXLZP/nhkcLJMA3ySJ5DA+FpQO+nginrBfDiSj8Rlk/nN/fM N4LAy5E8anFDzGfCbnMUmYZ6lfUnSJ/2ZfVreeRN2N+0ISdPuQVD+D+UKKuv1JBaTw5r XqUgHq8AR/wKJDtRW06u51EP+Kn92ZF/THtMU+7kA83TRHin40ZyxcHcjiHA+Cc+vxIv gBtq+0o5RQNM5lgPUZIBuKx8nZQ+bZEZoDmWh2yHB1Z7soqQzthwhUMjoF9Re2+qw2Ha 2PXg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=Ck8KqbWWMae7AkcYUcDJVvLhTKwYixox75Gde7aqwnI=; b=b/2KHP09M1sxlYLDuN1NCsOLgI+ayYuhf4RhodGRz9/xowdQbeXy9b2htXW9uQ0Jt5 OlW8/z3eki1/iExTPObZtsrBIhZnO8PincnHm9UjcHCwP7KLvKm2+Dr9hEsmLLHHElLH AkS7ujXoXWLdMI3qE3KVNgDfFdWObhRqxxebzSJiM3eBGCrW3Lh4T8VIOYnZ2Vrtf7OB DI5G6/4JjaL1ivRSPuyk7QIw3Ltq5DIIrgKGvuYGNOXvUGUqze2uoXbuQQhwTolFr66b KJxtPZdqDYut+/TTRf52CKYp/Fw/SxX2mNobi1NE/HglfNwZKSFYF1lNX+sbWtemH4mE QYVw== X-Gm-Message-State: AOAM533z0BZOnOp0bfY1GUq1odmlgo4arKhERXvVspV4Ty2D9xsYJMpv 4ZIMHQC4O1AJCdqX+Yi6FKn4P/Y923cg X-Google-Smtp-Source: ABdhPJwuoB1vbPKcvGKMvoxjhGJXSCqivwYwRjiSn+hagjA4deHJDWh9elWq7/kmk5iywJnzibKQqL+7r11H X-Received: from luke.lon.corp.google.com ([2a00:79e0:d:210:59ca:401f:83a8:de6d]) (user=qperret job=sendgmr) by 2002:adf:f10c:: with SMTP id r12mr31956400wro.298.1634645614634; Tue, 19 Oct 2021 05:13:34 -0700 (PDT) Date: Tue, 19 Oct 2021 13:13:01 +0100 In-Reply-To: <20211019121304.2732332-1-qperret@google.com> Message-Id: <20211019121304.2732332-13-qperret@google.com> Mime-Version: 1.0 References: <20211019121304.2732332-1-qperret@google.com> X-Mailer: git-send-email 2.33.0.1079.g6e70778dc9-goog Subject: [PATCH v2 12/15] KVM: arm64: Implement __pkvm_host_share_hyp() using do_share() From: Quentin Perret To: Marc Zyngier , James Morse , Alexandru Elisei , Suzuki K Poulose , Catalin Marinas , Will Deacon , Fuad Tabba , David Brazdil , Andrew Walbran Cc: linux-arm-kernel@lists.infradead.org, kvmarm@lists.cs.columbia.edu, linux-kernel@vger.kernel.org, kernel-team@android.com, qperret@google.com X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20211019_051336_716834_67304A15 X-CRM114-Status: GOOD ( 17.10 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org From: Will Deacon __pkvm_host_share_hyp() shares memory between the host and the hypervisor so implement it as an invocation of the new do_share() mechanism. Note that double-sharing is no longer permitted (as this allows us to reduce the number of page-table walks significantly), but is thankfully no longer relied upon by the host. Signed-off-by: Will Deacon Signed-off-by: Quentin Perret --- arch/arm64/kvm/hyp/nvhe/mem_protect.c | 121 +++++++------------------- 1 file changed, 33 insertions(+), 88 deletions(-) diff --git a/arch/arm64/kvm/hyp/nvhe/mem_protect.c b/arch/arm64/kvm/hyp/nvhe/mem_protect.c index d87cdd53dd21..ab1732c38b3b 100644 --- a/arch/arm64/kvm/hyp/nvhe/mem_protect.c +++ b/arch/arm64/kvm/hyp/nvhe/mem_protect.c @@ -359,94 +359,6 @@ static int host_stage2_idmap(u64 addr) return ret; } -static inline bool check_prot(enum kvm_pgtable_prot prot, - enum kvm_pgtable_prot required, - enum kvm_pgtable_prot denied) -{ - return (prot & (required | denied)) == required; -} - -int __pkvm_host_share_hyp(u64 pfn) -{ - phys_addr_t addr = hyp_pfn_to_phys(pfn); - enum kvm_pgtable_prot prot, cur; - void *virt = __hyp_va(addr); - enum pkvm_page_state state; - kvm_pte_t pte; - int ret; - - if (!addr_is_memory(addr)) - return -EINVAL; - - host_lock_component(); - hyp_lock_component(); - - ret = kvm_pgtable_get_leaf(&host_kvm.pgt, addr, &pte, NULL); - if (ret) - goto unlock; - if (!pte) - goto map_shared; - - /* - * Check attributes in the host stage-2 PTE. We need the page to be: - * - mapped RWX as we're sharing memory; - * - not borrowed, as that implies absence of ownership. - * Otherwise, we can't let it got through - */ - cur = kvm_pgtable_stage2_pte_prot(pte); - prot = pkvm_mkstate(0, PKVM_PAGE_SHARED_BORROWED); - if (!check_prot(cur, PKVM_HOST_MEM_PROT, prot)) { - ret = -EPERM; - goto unlock; - } - - state = pkvm_getstate(cur); - if (state == PKVM_PAGE_OWNED) - goto map_shared; - - /* - * Tolerate double-sharing the same page, but this requires - * cross-checking the hypervisor stage-1. - */ - if (state != PKVM_PAGE_SHARED_OWNED) { - ret = -EPERM; - goto unlock; - } - - ret = kvm_pgtable_get_leaf(&pkvm_pgtable, (u64)virt, &pte, NULL); - if (ret) - goto unlock; - - /* - * If the page has been shared with the hypervisor, it must be - * already mapped as SHARED_BORROWED in its stage-1. - */ - cur = kvm_pgtable_hyp_pte_prot(pte); - prot = pkvm_mkstate(PAGE_HYP, PKVM_PAGE_SHARED_BORROWED); - if (!check_prot(cur, prot, ~prot)) - ret = -EPERM; - goto unlock; - -map_shared: - /* - * If the page is not yet shared, adjust mappings in both page-tables - * while both locks are held. - */ - prot = pkvm_mkstate(PAGE_HYP, PKVM_PAGE_SHARED_BORROWED); - ret = pkvm_create_mappings_locked(virt, virt + PAGE_SIZE, prot); - BUG_ON(ret); - - prot = pkvm_mkstate(PKVM_HOST_MEM_PROT, PKVM_PAGE_SHARED_OWNED); - ret = host_stage2_idmap_locked(addr, PAGE_SIZE, prot); - BUG_ON(ret); - -unlock: - hyp_unlock_component(); - host_unlock_component(); - - return ret; -} - void handle_host_mem_abort(struct kvm_cpu_context *host_ctxt) { struct kvm_vcpu_fault_info fault; @@ -697,3 +609,36 @@ static int do_share(struct pkvm_mem_share *share) return WARN_ON(__do_share(share)); } + +int __pkvm_host_share_hyp(u64 pfn) +{ + int ret; + u64 host_addr = hyp_pfn_to_phys(pfn); + u64 hyp_addr = (u64)__hyp_va(host_addr); + struct pkvm_mem_share share = { + .tx = { + .nr_pages = 1, + .initiator = { + .id = PKVM_ID_HOST, + .addr = host_addr, + .host = { + .completer_addr = hyp_addr, + }, + }, + .completer = { + .id = PKVM_ID_HYP, + }, + }, + .prot = PAGE_HYP, + }; + + host_lock_component(); + hyp_lock_component(); + + ret = do_share(&share); + + hyp_unlock_component(); + host_unlock_component(); + + return ret; +} From patchwork Tue Oct 19 12:13:02 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Quentin Perret X-Patchwork-Id: 12569757 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 7747FC433EF for ; Tue, 19 Oct 2021 12:22:21 +0000 (UTC) Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 3CD6260FDA for ; Tue, 19 Oct 2021 12:22:21 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.4.1 mail.kernel.org 3CD6260FDA Authentication-Results: mail.kernel.org; dmarc=fail (p=reject dis=none) header.from=google.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:Cc:To:From:Subject:References: Mime-Version:Message-Id:In-Reply-To:Date:Reply-To:Content-ID: Content-Description:Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc :Resent-Message-ID:List-Owner; bh=5axwvHDqgYC4M7OWgDZvINSW4LQwD483Rpwj8v5MeJ0=; b=xQ+ME9OljQwMrr46ZgkkGLWR4F nOj+BIr46TSndo8jvMuQ6bxzYCnz6Db+X3AinlYOLFH+1dwsGaucWH2jYLZzhRKw9ixdG+xlXzLbs 9qCi9hhmyDTw4AJdp6TTKOQKjMat99CjzJrOf8L8wIq2wbky9lv18WisR+o6wMkImzmIch7b3OLab NTzlhygvWtfpjipSOEUy18sdTL7q/i4p7YQr90wh3Z0maLsQl08WqCxyYrv7yvZn1f7GNpggMInke N4EFXWp2WnhTDPrwk4b+NJkGcmUIhbTfo5agUR+JqQvYzhkqYXeIe0TTBpd+1aGhOSZfb4fOFa72I LxsMOYzA==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1mco6N-0018UH-Cg; Tue, 19 Oct 2021 12:20:20 +0000 Received: from mail-lj1-x249.google.com ([2a00:1450:4864:20::249]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1mcnzw-0015Nu-WD for linux-arm-kernel@lists.infradead.org; Tue, 19 Oct 2021 12:13:42 +0000 Received: by mail-lj1-x249.google.com with SMTP id j8-20020a2e3c08000000b00210e2890d61so812766lja.9 for ; Tue, 19 Oct 2021 05:13:38 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20210112; h=date:in-reply-to:message-id:mime-version:references:subject:from:to :cc; bh=ISeivE4VJePhwCDFvsBtzOVIerKBvyMpPq/U4suJ8Dg=; b=pMTyZ0Vbl+u7Dlf64iUAHAL8iII5oExKlGMnTMHZaH1N3KOh6pszIEURsP71VOA8zm YmAfLbs73BpmIWiQFsAwuQdZch7d1aBX0VE1dqPVlo4MR+Iv3SEIvK1dzqA44YUBqokQ 3BWhGEojgqF2KRv1G62YRc39Vn8Nnjta2ATq8LApIpEt7DGix0abtEW9pGr56bx76kdj 22z6U+fFymB5GMOz4WZX6L7B8vnwJ+yGDZM1gxOHNZEdOyXICg/04hJWqSYDUUhm5VGZ 8XPgn1dhN12ntFAgc27i+8EKZxW4j4Rv26/Vjwmr2QvI0pNoKwfj6Pef3gBLMRK7blQh duHQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=ISeivE4VJePhwCDFvsBtzOVIerKBvyMpPq/U4suJ8Dg=; b=K9nrhC8naorM7SJM0ZoLkemNGFBCd7qDbojrSFo3b9TFIa+sa2kDOTPMGSUTdiCb+h UlOFYuTl6wT8CHUsXsTM/N1qrItHo15tbyZ9Kq1Rs2f7750lYbTXr83KujO3j4WKvxdt TlY8ig7gFIwifULW5fx4FGSuwc48azRrwO/wjFgu4QQtiubB5QUYkROKvCbOPDdC96GV 4gIdDXVictW9Jh0TNRFPm17EOgpcH8/ZrxO7VzdX1zXp01/tpTyqmm7F2PwRY9kwtDeO xvfWIxmAkyRXNRT+Dv53W74CtUGCLrwVqAq5cVb8miA1BOVBw1EtTes7+67Y8GPxbnbH qRrg== X-Gm-Message-State: AOAM530Aly1lRC/HbigNV8brFDxb3mpCxxaHqRiN/hZkhNpV7IZFu7gC 5Dcue2zL13kKrNXEx90mTAwN5JtiPDhn X-Google-Smtp-Source: ABdhPJwgKkmhxiB5XvxU66pG49kdREnEBCBJUkS8jWIoTxJjPaFPdF/auvzHlo/liyhgODAIlIOug/KJN2Yd X-Received: from luke.lon.corp.google.com ([2a00:79e0:d:210:59ca:401f:83a8:de6d]) (user=qperret job=sendgmr) by 2002:a2e:7816:: with SMTP id t22mr6884979ljc.133.1634645617143; Tue, 19 Oct 2021 05:13:37 -0700 (PDT) Date: Tue, 19 Oct 2021 13:13:02 +0100 In-Reply-To: <20211019121304.2732332-1-qperret@google.com> Message-Id: <20211019121304.2732332-14-qperret@google.com> Mime-Version: 1.0 References: <20211019121304.2732332-1-qperret@google.com> X-Mailer: git-send-email 2.33.0.1079.g6e70778dc9-goog Subject: [PATCH v2 13/15] KVM: arm64: Implement do_unshare() helper for unsharing memory From: Quentin Perret To: Marc Zyngier , James Morse , Alexandru Elisei , Suzuki K Poulose , Catalin Marinas , Will Deacon , Fuad Tabba , David Brazdil , Andrew Walbran Cc: linux-arm-kernel@lists.infradead.org, kvmarm@lists.cs.columbia.edu, linux-kernel@vger.kernel.org, kernel-team@android.com, qperret@google.com X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20211019_051341_111975_50A9B0AA X-CRM114-Status: GOOD ( 13.43 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org From: Will Deacon Tearing down a previously shared memory region results in the borrower losing access to the underlying pages and returning them to the "owned" state in the owner. Implement a do_unshare() helper, along the same lines as do_share(), to provide this functionality for the host-to-hyp case. Signed-off-by: Will Deacon Signed-off-by: Quentin Perret --- arch/arm64/kvm/hyp/nvhe/mem_protect.c | 115 ++++++++++++++++++++++++++ 1 file changed, 115 insertions(+) diff --git a/arch/arm64/kvm/hyp/nvhe/mem_protect.c b/arch/arm64/kvm/hyp/nvhe/mem_protect.c index ab1732c38b3b..205616c9f107 100644 --- a/arch/arm64/kvm/hyp/nvhe/mem_protect.c +++ b/arch/arm64/kvm/hyp/nvhe/mem_protect.c @@ -474,6 +474,16 @@ static int host_request_owned_transition(u64 *completer_addr, return __host_check_page_state_range(addr, size, PKVM_PAGE_OWNED); } +static int host_request_unshare(u64 *completer_addr, + const struct pkvm_mem_transition *tx) +{ + u64 size = tx->nr_pages * PAGE_SIZE; + u64 addr = tx->initiator.addr; + + *completer_addr = tx->initiator.host.completer_addr; + return __host_check_page_state_range(addr, size, PKVM_PAGE_SHARED_OWNED); +} + static int host_initiate_share(u64 *completer_addr, const struct pkvm_mem_transition *tx) { @@ -484,6 +494,16 @@ static int host_initiate_share(u64 *completer_addr, return __host_set_page_state_range(addr, size, PKVM_PAGE_SHARED_OWNED); } +static int host_initiate_unshare(u64 *completer_addr, + const struct pkvm_mem_transition *tx) +{ + u64 size = tx->nr_pages * PAGE_SIZE; + u64 addr = tx->initiator.addr; + + *completer_addr = tx->initiator.host.completer_addr; + return __host_set_page_state_range(addr, size, PKVM_PAGE_OWNED); +} + static enum pkvm_page_state hyp_get_page_state(kvm_pte_t pte) { if (!kvm_pte_valid(pte)) @@ -524,6 +544,17 @@ static int hyp_ack_share(u64 addr, const struct pkvm_mem_transition *tx, return __hyp_check_page_state_range(addr, size, PKVM_NOPAGE); } +static int hyp_ack_unshare(u64 addr, const struct pkvm_mem_transition *tx) +{ + u64 size = tx->nr_pages * PAGE_SIZE; + + if (__hyp_ack_skip_pgtable_check(tx)) + return 0; + + return __hyp_check_page_state_range(addr, size, + PKVM_PAGE_SHARED_BORROWED); +} + static int hyp_complete_share(u64 addr, const struct pkvm_mem_transition *tx, enum kvm_pgtable_prot perms) { @@ -534,6 +565,14 @@ static int hyp_complete_share(u64 addr, const struct pkvm_mem_transition *tx, return pkvm_create_mappings_locked(start, end, prot); } +static int hyp_complete_unshare(u64 addr, const struct pkvm_mem_transition *tx) +{ + u64 size = tx->nr_pages * PAGE_SIZE; + int ret = kvm_pgtable_hyp_unmap(&pkvm_pgtable, addr, size); + + return (ret != size) ? -EFAULT : 0; +} + static int check_share(struct pkvm_mem_share *share) { const struct pkvm_mem_transition *tx = &share->tx; @@ -610,6 +649,82 @@ static int do_share(struct pkvm_mem_share *share) return WARN_ON(__do_share(share)); } +static int check_unshare(struct pkvm_mem_share *share) +{ + const struct pkvm_mem_transition *tx = &share->tx; + u64 completer_addr; + int ret; + + switch (tx->initiator.id) { + case PKVM_ID_HOST: + ret = host_request_unshare(&completer_addr, tx); + break; + default: + ret = -EINVAL; + } + + if (ret) + return ret; + + switch (tx->completer.id) { + case PKVM_ID_HYP: + ret = hyp_ack_unshare(completer_addr, tx); + break; + default: + ret = -EINVAL; + } + + return ret; +} + +static int __do_unshare(struct pkvm_mem_share *share) +{ + const struct pkvm_mem_transition *tx = &share->tx; + u64 completer_addr; + int ret; + + switch (tx->initiator.id) { + case PKVM_ID_HOST: + ret = host_initiate_unshare(&completer_addr, tx); + break; + default: + ret = -EINVAL; + } + + if (ret) + return ret; + + switch (tx->completer.id) { + case PKVM_ID_HYP: + ret = hyp_complete_unshare(completer_addr, tx); + break; + default: + ret = -EINVAL; + } + + return ret; +} + +/* + * do_unshare(): + * + * The page owner revokes access from another component for a range of + * pages which were previously shared using do_share(). + * + * Initiator: SHARED_OWNED => OWNED + * Completer: SHARED_BORROWED => NOPAGE + */ +static int do_unshare(struct pkvm_mem_share *share) +{ + int ret; + + ret = check_unshare(share); + if (ret) + return ret; + + return WARN_ON(__do_unshare(share)); +} + int __pkvm_host_share_hyp(u64 pfn) { int ret; From patchwork Tue Oct 19 12:13:03 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Quentin Perret X-Patchwork-Id: 12569759 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 68CDAC433EF for ; Tue, 19 Oct 2021 12:22:59 +0000 (UTC) Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 35B2861260 for ; Tue, 19 Oct 2021 12:22:59 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.4.1 mail.kernel.org 35B2861260 Authentication-Results: mail.kernel.org; dmarc=fail (p=reject dis=none) header.from=google.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:Cc:To:From:Subject:References: Mime-Version:Message-Id:In-Reply-To:Date:Reply-To:Content-ID: Content-Description:Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc :Resent-Message-ID:List-Owner; bh=WNVFwsZQicZ1LGKU7fcLr7xK8lEeYcSF6yh1NqE3Ksg=; b=rEMDt4lU6ucSihxYhL9Hpnb8oW nMhKBS/O7ooaCAWOGL/Ex7J3JCQViStkSoEnl1WXseoelzs99UF9eVGI7ZU5B3Giuj379KZO4NOgG 3W/axq9y+gjwM5ZllOK+xH1riSZm0YMjCJ+f8J4IW3YIgP5h0IzcYxl8YMk4NED7Ihyv0FuoBOiEz ZuW/UgLOpbt2wRxAgdqgfP29fy1Jn2+BM/hB88HrVtvLwcYZBzae2Xhfihr6IUgkrJPiB4F4NOPsy VCXO7KYejqWQYy0fK+JgmaMlO0PVgssE24C4na5AIy0fclOdfKaJexlslQIr1S378dAzWapJL7STG izM6XypQ==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1mco7B-0018nV-1a; Tue, 19 Oct 2021 12:21:09 +0000 Received: from mail-wm1-x34a.google.com ([2a00:1450:4864:20::34a]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1mcnzx-0015PG-NF for linux-arm-kernel@lists.infradead.org; Tue, 19 Oct 2021 12:13:44 +0000 Received: by mail-wm1-x34a.google.com with SMTP id k20-20020a7bc414000000b0030dcd454771so1066072wmi.8 for ; Tue, 19 Oct 2021 05:13:40 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20210112; h=date:in-reply-to:message-id:mime-version:references:subject:from:to :cc; bh=DHnuaOy9p1LD/61fjq4CO1t0/yktzi5R0H6G7O3QKgQ=; b=lk3KhNsTMuoV5fphW2OJrEAgxIGGD12sA+tP3VzSZuG0/N9jed9FhMYky4JoMvaFrN +x4sMMoqdzzFC43SWfYVVOpAsfikOD+9NnhKBjeye7tX/ZodpmTn/c4ee+4kxccZ51VU 7va8DyRJN6maWNmNRzWTTREvIK23n/rM0URBAPnlrK0XimQkCduqHwnwewzNrwCLTc8U /d2MFLhpRxS6H2KjHdkW96tYBgsNJCHtXcPSgDaHpasRIVddeFfv7qhNkneW+IYJwO5v 9dWgaGj9LZiK33eYs49Nu3b/ZtrJaznu+sfhEVhtXyx/3MGq2Zqpt9Bl76zcGctcJ6gT ycaw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=DHnuaOy9p1LD/61fjq4CO1t0/yktzi5R0H6G7O3QKgQ=; b=Utz/mvwnZLlKKhBrVNOVNq26SZJc5chMv68g7oRUoXkLZZmK94iTYtxha2uOXmU4/2 ac5aFuVXS0CLlVhvA5N6je0hWLvvxEZyJGYZYQm++Mde4teEBmobo90AdU3Ki9Esfu+4 uyqq01aUnsq/4dTtkbSAHLCHZozxlzMM1b+18m40DfV7a56yLcKSNG6Oi3VGsJsECQbO QCEBgBvO/WbPCIMNZVbCv5ywFOSt1nOj4KoJ9kBQ+3qEkiwLdD+Yf/UagmBYY680GC+Q sDSmZlL64OE1NX3ZUXfWgyBvFHy/7KpVAAHA2SJZYZd5cVsSBQQUe3sWc7O4nWZGShUE 58EQ== X-Gm-Message-State: AOAM531dcENWtw89xKRSFLBGHZYStfeKFgE4EMqr1Orv7NqkQUi98pZq kY8yD3GnOWDYno4Aflm7S8iWiL4+dqDu X-Google-Smtp-Source: ABdhPJziYoIjlxnulRRJqnFfUHmz1k8SNbs7d5KuqNS+gY2QUofneE5ubXRJhX+TuE7cLXglafWHIUoL6D1m X-Received: from luke.lon.corp.google.com ([2a00:79e0:d:210:59ca:401f:83a8:de6d]) (user=qperret job=sendgmr) by 2002:a05:600c:17ca:: with SMTP id y10mr5762129wmo.62.1634645619541; Tue, 19 Oct 2021 05:13:39 -0700 (PDT) Date: Tue, 19 Oct 2021 13:13:03 +0100 In-Reply-To: <20211019121304.2732332-1-qperret@google.com> Message-Id: <20211019121304.2732332-15-qperret@google.com> Mime-Version: 1.0 References: <20211019121304.2732332-1-qperret@google.com> X-Mailer: git-send-email 2.33.0.1079.g6e70778dc9-goog Subject: [PATCH v2 14/15] KVM: arm64: Expose unshare hypercall to the host From: Quentin Perret To: Marc Zyngier , James Morse , Alexandru Elisei , Suzuki K Poulose , Catalin Marinas , Will Deacon , Fuad Tabba , David Brazdil , Andrew Walbran Cc: linux-arm-kernel@lists.infradead.org, kvmarm@lists.cs.columbia.edu, linux-kernel@vger.kernel.org, kernel-team@android.com, qperret@google.com X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20211019_051341_828582_04D1E2CF X-CRM114-Status: GOOD ( 13.53 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org From: Will Deacon Introduce an unshare hypercall which can be used to unmap memory from the hypervisor stage-1 in nVHE protected mode. This will be useful to update the EL2 ownership state of pages during guest teardown, and avoids keeping dangling mappings to unreferenced portions of memory. Signed-off-by: Will Deacon Signed-off-by: Quentin Perret --- arch/arm64/include/asm/kvm_asm.h | 1 + arch/arm64/kvm/hyp/include/nvhe/mem_protect.h | 1 + arch/arm64/kvm/hyp/nvhe/hyp-main.c | 8 +++++ arch/arm64/kvm/hyp/nvhe/mem_protect.c | 33 +++++++++++++++++++ 4 files changed, 43 insertions(+) diff --git a/arch/arm64/include/asm/kvm_asm.h b/arch/arm64/include/asm/kvm_asm.h index 0167f3702c61..59c55aedb0ae 100644 --- a/arch/arm64/include/asm/kvm_asm.h +++ b/arch/arm64/include/asm/kvm_asm.h @@ -63,6 +63,7 @@ enum __kvm_host_smccc_func { /* Hypercalls available after pKVM finalisation */ __KVM_HOST_SMCCC_FUNC___pkvm_host_share_hyp, + __KVM_HOST_SMCCC_FUNC___pkvm_host_unshare_hyp, __KVM_HOST_SMCCC_FUNC___kvm_adjust_pc, __KVM_HOST_SMCCC_FUNC___kvm_vcpu_run, __KVM_HOST_SMCCC_FUNC___kvm_flush_vm_context, diff --git a/arch/arm64/kvm/hyp/include/nvhe/mem_protect.h b/arch/arm64/kvm/hyp/include/nvhe/mem_protect.h index 56445586c755..80e99836eac7 100644 --- a/arch/arm64/kvm/hyp/include/nvhe/mem_protect.h +++ b/arch/arm64/kvm/hyp/include/nvhe/mem_protect.h @@ -55,6 +55,7 @@ extern const u8 pkvm_hyp_id; int __pkvm_prot_finalize(void); int __pkvm_host_share_hyp(u64 pfn); +int __pkvm_host_unshare_hyp(u64 pfn); bool addr_is_memory(phys_addr_t phys); int host_stage2_idmap_locked(phys_addr_t addr, u64 size, enum kvm_pgtable_prot prot); diff --git a/arch/arm64/kvm/hyp/nvhe/hyp-main.c b/arch/arm64/kvm/hyp/nvhe/hyp-main.c index b096bf009144..5e2197db0d32 100644 --- a/arch/arm64/kvm/hyp/nvhe/hyp-main.c +++ b/arch/arm64/kvm/hyp/nvhe/hyp-main.c @@ -147,6 +147,13 @@ static void handle___pkvm_host_share_hyp(struct kvm_cpu_context *host_ctxt) cpu_reg(host_ctxt, 1) = __pkvm_host_share_hyp(pfn); } +static void handle___pkvm_host_unshare_hyp(struct kvm_cpu_context *host_ctxt) +{ + DECLARE_REG(u64, pfn, host_ctxt, 1); + + cpu_reg(host_ctxt, 1) = __pkvm_host_unshare_hyp(pfn); +} + static void handle___pkvm_create_private_mapping(struct kvm_cpu_context *host_ctxt) { DECLARE_REG(phys_addr_t, phys, host_ctxt, 1); @@ -184,6 +191,7 @@ static const hcall_t host_hcall[] = { HANDLE_FUNC(__pkvm_prot_finalize), HANDLE_FUNC(__pkvm_host_share_hyp), + HANDLE_FUNC(__pkvm_host_unshare_hyp), HANDLE_FUNC(__kvm_adjust_pc), HANDLE_FUNC(__kvm_vcpu_run), HANDLE_FUNC(__kvm_flush_vm_context), diff --git a/arch/arm64/kvm/hyp/nvhe/mem_protect.c b/arch/arm64/kvm/hyp/nvhe/mem_protect.c index 205616c9f107..fc542585a964 100644 --- a/arch/arm64/kvm/hyp/nvhe/mem_protect.c +++ b/arch/arm64/kvm/hyp/nvhe/mem_protect.c @@ -757,3 +757,36 @@ int __pkvm_host_share_hyp(u64 pfn) return ret; } + +int __pkvm_host_unshare_hyp(u64 pfn) +{ + int ret; + u64 host_addr = hyp_pfn_to_phys(pfn); + u64 hyp_addr = (u64)__hyp_va(host_addr); + struct pkvm_mem_share share = { + .tx = { + .nr_pages = 1, + .initiator = { + .id = PKVM_ID_HOST, + .addr = host_addr, + .host = { + .completer_addr = hyp_addr, + }, + }, + .completer = { + .id = PKVM_ID_HYP, + }, + }, + .prot = PAGE_HYP, + }; + + host_lock_component(); + hyp_lock_component(); + + ret = do_unshare(&share); + + hyp_unlock_component(); + host_unlock_component(); + + return ret; +} From patchwork Tue Oct 19 12:13:04 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Quentin Perret X-Patchwork-Id: 12569763 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id B480CC433EF for ; Tue, 19 Oct 2021 12:24:14 +0000 (UTC) Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 757D661372 for ; Tue, 19 Oct 2021 12:24:13 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.4.1 mail.kernel.org 757D661372 Authentication-Results: mail.kernel.org; dmarc=fail (p=reject dis=none) header.from=google.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:Cc:To:From:Subject:References: Mime-Version:Message-Id:In-Reply-To:Date:Reply-To:Content-ID: Content-Description:Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc :Resent-Message-ID:List-Owner; bh=4sfUM5TUPNZsqzhfympFqH0c5OuLxvpm+Nk/y5RVWd0=; b=3PNAJ0L8nEOfmn5rlRIYOV9Qt5 ykcdpanDjvVzioo3dhCuleQeMuO/dplmwrDoFlwpmaf9EHEBAGJmXlY0m85/Jo6qaI4QoNJBczrCy OWTDGCqp+FF0CsdtGo/hsx5DfwTPe8br5kalGomDPEVmR4eNtwQB3c/9WOxVxFHjhU3DqDd/FvMe3 RRKKWPsoK7Kq2II3xZGRMCOl86JOKETmHojGPw5R37uMJaZZ1rrQOQ2ejn8A8fmZWm9fXxoYYznjf ixP9P71UzR28J5xUb/dwx3sNTP75IcDvyYEVb/K1xmMDcMrWCCkbiwqfXRlggZCTIFefs3hBBfMMd 0k8JDdCQ==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1mco84-0019AE-OD; Tue, 19 Oct 2021 12:22:05 +0000 Received: from mail-wr1-x449.google.com ([2a00:1450:4864:20::449]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1mcnzz-0015Ql-UM for linux-arm-kernel@lists.infradead.org; Tue, 19 Oct 2021 12:13:45 +0000 Received: by mail-wr1-x449.google.com with SMTP id l6-20020adfa386000000b00160c4c1866eso10092987wrb.4 for ; Tue, 19 Oct 2021 05:13:42 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20210112; h=date:in-reply-to:message-id:mime-version:references:subject:from:to :cc; bh=VqYPavyR2JRTUWvE5kjjzlUp50pAuHmuHcARAYk4cwk=; b=fGi7IVyHWwfKQS2WbRt2o+yRVdxkv/7OeCdfA8Py2fFF0DRYQaLM7bUAn4tRvOCb6L UYC9SRTQ42xvB8Xoqub8ks9s52dpmbfkm+nh04FUXNyyBbLGkbdTtw3MW8S/ct0VuF1a hatlEFd42G71G8EgIcsabJ2RwHJb+puTGYm+uiRBCH5+mK8nCmY59E2VMpa+tfRufGpu Uyux/TC1Du3dMEK8gNu4keJ5UFr673qhIq57qoEayFQBYpe7vg6F+mJBhZHawxAHDBcN If1bGrUtd8wJRrN6RNtohM6Jmv2jaGMrZ/5+CnlgLSAkSU8VUMwN/vtvLWctXfMQ/0jH T5Jw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=VqYPavyR2JRTUWvE5kjjzlUp50pAuHmuHcARAYk4cwk=; b=BVoDez+3GoxbeGzUhtV2reAdmZIM4K1dygl9VIDrZkTEr/b1QZEG/zQjqAjSMYjxew Wllrb0M7kaYqqN65B4vw1PMBKIVSbvYjf4LTVwuL90MO4j+KeopMBCxXh/2Iid2qPKLC 5Y9e/Y6KuhAIUWw9kn+bKmlpI+8ddV+6jYOHqTLWOpHEmEINgSKVsyPrzegMFIeFfJsh g9wjg+aqgL5gebstXzeMiWFRkJ3HhtK5B2QZFD3sfF1VMpm0wXUzPouHSFcDlZdLxmer OYCV+yCBtFKc35Q0OqROIq2yMfj3ewm0wN8odHM5FUgkC7CddBTwprNu8SIyh7H0wpli zXPA== X-Gm-Message-State: AOAM531hieajJs+gAXLV7fqZ70M/f8lAZQeorS2UHqDEhjJpeorq3I8K /82jBH6CapHlX4esrnY7cqrkvftdelsM X-Google-Smtp-Source: ABdhPJydGp5bAuju3SKMKeRNA5buAgHVc9pJMxXIPwxc98YN65zoWKEuWvhMGvpeXA+5NqfJbVAcywe3r4Wf X-Received: from luke.lon.corp.google.com ([2a00:79e0:d:210:59ca:401f:83a8:de6d]) (user=qperret job=sendgmr) by 2002:a05:600c:4e94:: with SMTP id f20mr5727676wmq.52.1634645621773; Tue, 19 Oct 2021 05:13:41 -0700 (PDT) Date: Tue, 19 Oct 2021 13:13:04 +0100 In-Reply-To: <20211019121304.2732332-1-qperret@google.com> Message-Id: <20211019121304.2732332-16-qperret@google.com> Mime-Version: 1.0 References: <20211019121304.2732332-1-qperret@google.com> X-Mailer: git-send-email 2.33.0.1079.g6e70778dc9-goog Subject: [PATCH v2 15/15] KVM: arm64: pkvm: Unshare guest structs during teardown From: Quentin Perret To: Marc Zyngier , James Morse , Alexandru Elisei , Suzuki K Poulose , Catalin Marinas , Will Deacon , Fuad Tabba , David Brazdil , Andrew Walbran Cc: linux-arm-kernel@lists.infradead.org, kvmarm@lists.cs.columbia.edu, linux-kernel@vger.kernel.org, kernel-team@android.com, qperret@google.com X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20211019_051344_042158_8AB7A6B5 X-CRM114-Status: GOOD ( 23.15 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Make use of the newly introduced unshare hypercall during guest teardown to unmap guest-related data structures from the hyp stage-1. Signed-off-by: Quentin Perret --- arch/arm64/include/asm/kvm_host.h | 2 ++ arch/arm64/include/asm/kvm_mmu.h | 1 + arch/arm64/kvm/arm.c | 2 ++ arch/arm64/kvm/fpsimd.c | 29 +++++++++++++++++++++ arch/arm64/kvm/mmu.c | 42 +++++++++++++++++++++++++++++++ arch/arm64/kvm/reset.c | 8 +++++- 6 files changed, 83 insertions(+), 1 deletion(-) diff --git a/arch/arm64/include/asm/kvm_host.h b/arch/arm64/include/asm/kvm_host.h index d1451c8a768b..9f70e82c7436 100644 --- a/arch/arm64/include/asm/kvm_host.h +++ b/arch/arm64/include/asm/kvm_host.h @@ -323,6 +323,7 @@ struct kvm_vcpu_arch { struct thread_info *host_thread_info; /* hyp VA */ struct user_fpsimd_state *host_fpsimd_state; /* hyp VA */ + struct task_struct *parent_task; struct { /* {Break,watch}point registers */ @@ -738,6 +739,7 @@ int kvm_arch_vcpu_run_map_fp(struct kvm_vcpu *vcpu); void kvm_arch_vcpu_load_fp(struct kvm_vcpu *vcpu); void kvm_arch_vcpu_ctxsync_fp(struct kvm_vcpu *vcpu); void kvm_arch_vcpu_put_fp(struct kvm_vcpu *vcpu); +void kvm_vcpu_unshare_task_fp(struct kvm_vcpu *vcpu); static inline bool kvm_pmu_counter_deferred(struct perf_event_attr *attr) { diff --git a/arch/arm64/include/asm/kvm_mmu.h b/arch/arm64/include/asm/kvm_mmu.h index 185d0f62b724..81839e9a8a24 100644 --- a/arch/arm64/include/asm/kvm_mmu.h +++ b/arch/arm64/include/asm/kvm_mmu.h @@ -151,6 +151,7 @@ static __always_inline unsigned long __kern_hyp_va(unsigned long v) #include int kvm_share_hyp(void *from, void *to); +void kvm_unshare_hyp(void *from, void *to); int create_hyp_mappings(void *from, void *to, enum kvm_pgtable_prot prot); int create_hyp_io_mappings(phys_addr_t phys_addr, size_t size, void __iomem **kaddr, diff --git a/arch/arm64/kvm/arm.c b/arch/arm64/kvm/arm.c index 5312d5a2ef69..ebab9529b736 100644 --- a/arch/arm64/kvm/arm.c +++ b/arch/arm64/kvm/arm.c @@ -188,6 +188,8 @@ void kvm_arch_destroy_vm(struct kvm *kvm) } } atomic_set(&kvm->online_vcpus, 0); + + kvm_unshare_hyp(kvm, kvm + 1); } int kvm_vm_ioctl_check_extension(struct kvm *kvm, long ext) diff --git a/arch/arm64/kvm/fpsimd.c b/arch/arm64/kvm/fpsimd.c index 2fe1128d9f3d..ace5ebef8aab 100644 --- a/arch/arm64/kvm/fpsimd.c +++ b/arch/arm64/kvm/fpsimd.c @@ -15,6 +15,22 @@ #include #include +void kvm_vcpu_unshare_task_fp(struct kvm_vcpu *vcpu) +{ + struct task_struct *p = vcpu->arch.parent_task; + struct user_fpsimd_state *fpsimd; + struct thread_info *ti; + + if (!is_protected_kvm_enabled() || !p) + return; + + ti = &p->thread_info; + kvm_unshare_hyp(ti, ti + 1); + fpsimd = &p->thread.uw.fpsimd_state; + kvm_unshare_hyp(fpsimd, fpsimd + 1); + put_task_struct(p); +} + /* * Called on entry to KVM_RUN unless this vcpu previously ran at least * once and the most recent prior KVM_RUN for this vcpu was called from @@ -31,6 +47,8 @@ int kvm_arch_vcpu_run_map_fp(struct kvm_vcpu *vcpu) struct thread_info *ti = ¤t->thread_info; struct user_fpsimd_state *fpsimd = ¤t->thread.uw.fpsimd_state; + kvm_vcpu_unshare_task_fp(vcpu); + /* * Make sure the host task thread flags and fpsimd state are * visible to hyp: @@ -45,6 +63,17 @@ int kvm_arch_vcpu_run_map_fp(struct kvm_vcpu *vcpu) vcpu->arch.host_thread_info = kern_hyp_va(ti); vcpu->arch.host_fpsimd_state = kern_hyp_va(fpsimd); + + /* + * We need to keep current's task_struct pinned until its data has been + * unshared with the hypervisor to make sure it is not re-used by the + * kernel and donated to someone else while already shared -- see + * kvm_vcpu_unshare_task_fp() for the matching put_task_struct(). + */ + if (is_protected_kvm_enabled()) { + get_task_struct(current); + vcpu->arch.parent_task = current; + } error: return ret; } diff --git a/arch/arm64/kvm/mmu.c b/arch/arm64/kvm/mmu.c index f0c16ed95974..d548141da2ea 100644 --- a/arch/arm64/kvm/mmu.c +++ b/arch/arm64/kvm/mmu.c @@ -344,6 +344,32 @@ static int share_pfn_hyp(u64 pfn) return ret; } +static int unshare_pfn_hyp(u64 pfn) +{ + struct rb_node **node, *parent; + struct hyp_shared_pfn *this; + int ret = 0; + + mutex_lock(&hyp_shared_pfns_lock); + this = find_shared_pfn(pfn, &node, &parent); + if (WARN_ON(!this)) { + ret = -EINVAL; + goto unlock; + } + + this->count--; + if (this->count) + goto unlock; + + rb_erase(&this->node, &hyp_shared_pfns); + kfree(this); + ret = kvm_call_hyp_nvhe(__pkvm_host_unshare_hyp, pfn, 1); +unlock: + mutex_unlock(&hyp_shared_pfns_lock); + + return ret; +} + int kvm_share_hyp(void *from, void *to) { phys_addr_t start, end, cur; @@ -368,6 +394,22 @@ int kvm_share_hyp(void *from, void *to) return 0; } +void kvm_unshare_hyp(void *from, void *to) +{ + phys_addr_t start, end, cur; + u64 pfn; + + if (is_kernel_in_hyp_mode() || kvm_host_owns_hyp_mappings() || !from) + return; + + start = ALIGN_DOWN(kvm_kaddr_to_phys(from), PAGE_SIZE); + end = PAGE_ALIGN(kvm_kaddr_to_phys(to)); + for (cur = start; cur < end; cur += PAGE_SIZE) { + pfn = __phys_to_pfn(cur); + WARN_ON(unshare_pfn_hyp(pfn)); + } +} + /** * create_hyp_mappings - duplicate a kernel virtual address range in Hyp mode * @from: The virtual kernel start address of the range diff --git a/arch/arm64/kvm/reset.c b/arch/arm64/kvm/reset.c index cf781ec9212a..bf15e9de4123 100644 --- a/arch/arm64/kvm/reset.c +++ b/arch/arm64/kvm/reset.c @@ -150,7 +150,13 @@ bool kvm_arm_vcpu_is_finalized(struct kvm_vcpu *vcpu) void kvm_arm_vcpu_destroy(struct kvm_vcpu *vcpu) { - kfree(vcpu->arch.sve_state); + void *sve_state = vcpu->arch.sve_state; + + kvm_vcpu_unshare_task_fp(vcpu); + kvm_unshare_hyp(vcpu, vcpu + 1); + if (sve_state) + kvm_unshare_hyp(sve_state, sve_state + vcpu_sve_state_size(vcpu)); + kfree(sve_state); } static void kvm_vcpu_reset_sve(struct kvm_vcpu *vcpu)