From patchwork Fri Apr 15 21:58:52 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Oliver Upton X-Patchwork-Id: 12815451 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id E5DBFC433EF for ; Fri, 15 Apr 2022 22:02:39 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:Cc:To:From:Subject:References: Mime-Version:Message-Id:In-Reply-To:Date:Reply-To:Content-ID: Content-Description:Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc :Resent-Message-ID:List-Owner; bh=euK9vK48hsV1lVjdTZ4ZTw3BPKShUEDQTxTUdxAxv+M=; b=yJzTKlBSIN8QobNB0zjhL+2oZ3 kne5FxxgQkWV0iENjUSpeLVyynIFXP+mU4jaHbFdDjcyHq0RQJGiF3Uj4LSkNOEHoniOveLlAqC5L FzVEy8bcYkqDPfAOcGLyKisXixGEOy9DC9Xfq80CTQYi9mniy2ZoUC/TlfBYt+HWI+l4QFyeXPqWZ FRKOXt8+0GuMt/erjcA72BuQU4tFyGy4Sc9N+nBM4CvWy47dY2PJ6bsWoR+kX8gWl7bnCShDx9kzq CFJ/EhDpZpIga/E084a2eERf9o999usqOwdvsdnXJUISWm+ItFzJvsqqRHNxTu86TC4LPIa1mPQNX My4fPdnA==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1nfU0O-00BT0W-3M; Fri, 15 Apr 2022 22:01:28 +0000 Received: from mail-yw1-x114a.google.com ([2607:f8b0:4864:20::114a]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1nfTyI-00BS3F-1o for linux-arm-kernel@lists.infradead.org; Fri, 15 Apr 2022 21:59:19 +0000 Received: by mail-yw1-x114a.google.com with SMTP id 00721157ae682-2ec12272fb2so75111417b3.6 for ; Fri, 15 Apr 2022 14:59:16 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20210112; h=date:in-reply-to:message-id:mime-version:references:subject:from:to :cc; bh=B6406tYui+5lJgRvZIHJ2dZYS6AUdUd0eyrsU1t7VRI=; b=RzttCkqDWkMAl2afGr2S9DJJm5Ry89zm9fK8vSWXxjh+Br5xy5GQOart2FDuwkrzu8 +3Iq4y2goOAEc/FMm6DlVztE2ytFr3oiz9PTrVTxIm5m3Agm5OTfDy0Zs4ZMjS6SjcTP Zb1tTD9KujpWLSKkx4Mqs5RJ6LI4Z/u/gq513Flk/dllCKb2U4eZ8URwcntAGL1GuOmq ufWltbnns/Dj/9u47iW81VO53GtXf0bBfExXn11hVKS6QGf6GMMShHapP0dxeuiycC4J XphrZoAoqeoctTsXEHQSr5ZLlJzNIHeMt45VK8ODHWJivQe27pVAh1K+W85kwYhJyLS2 rKpg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=B6406tYui+5lJgRvZIHJ2dZYS6AUdUd0eyrsU1t7VRI=; b=LvctxYKjNEXow3UjLQfu+ePR+Nhy/+m719TyUrbG6CL3Ca5vPn4JIB1t+vVNGnaWlG hZvhi8/2yoF5RaqdqZ5vPZJIv/UyQ/Y2UjplAvebMmasdkTrfsgI+f3RE/999CHVdvre 7VEpHNiUAZ7qsGdLYWaqe7hxAVE0M9mY/uJCzBpIACkesSEiMcptkEKohF5lK7djaML0 dgS57zEHcqwhSf2ZLFwdBA03VEoaIZU7WGK+I+1WcZXsn5vq+fkRzEVtXomPEyLStmCW 0qbc52RlFrYEYP7hZXg8pHb0fk81cVIjV30+AczAnJ22FtsEYuZMc+VHF0a/unHeVVMs qp8w== X-Gm-Message-State: AOAM5331obwSgBixoCwXP3aj6ut000SZrfN3P1sHzz7+vXxLNLWm0VCZ mS1H31J8UlVXDKoq3ncKsK1BGRt4K18= X-Google-Smtp-Source: ABdhPJyC6al0eeORHQcVaP84DLwWwiUHL9eCDBoXdDwFa7r/iAh2U4p6UOyrCV4QCls9mT6W0yMv3xIYId8= X-Received: from oupton.c.googlers.com ([fda3:e722:ac3:cc00:2b:ff92:c0a8:404]) (user=oupton job=sendgmr) by 2002:a81:4989:0:b0:2f1:47b9:8ac2 with SMTP id w131-20020a814989000000b002f147b98ac2mr935486ywa.346.1650059955713; Fri, 15 Apr 2022 14:59:15 -0700 (PDT) Date: Fri, 15 Apr 2022 21:58:52 +0000 In-Reply-To: <20220415215901.1737897-1-oupton@google.com> Message-Id: <20220415215901.1737897-9-oupton@google.com> Mime-Version: 1.0 References: <20220415215901.1737897-1-oupton@google.com> X-Mailer: git-send-email 2.36.0.rc0.470.gd361397f0d-goog Subject: [RFC PATCH 08/17] KVM: arm64: Spin off helper for initializing table pte From: Oliver Upton To: kvmarm@lists.cs.columbia.edu Cc: kvm@vger.kernel.org, Marc Zyngier , James Morse , Alexandru Elisei , Suzuki K Poulose , linux-arm-kernel@lists.infradead.org, Peter Shier , Ricardo Koller , Reiji Watanabe , Paolo Bonzini , Sean Christopherson , Ben Gardon , David Matlack , Oliver Upton X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20220415_145918_170646_89FB57B4 X-CRM114-Status: GOOD ( 12.01 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org With parallel table walks there is no guarantee that KVM reads back the same pte that was written. Spin off a helper that creates a pte value, thereby allowing the visitor callback to return the next table without reading the ptep again. Signed-off-by: Oliver Upton --- arch/arm64/kvm/hyp/pgtable.c | 23 ++++++++++++++++------- 1 file changed, 16 insertions(+), 7 deletions(-) diff --git a/arch/arm64/kvm/hyp/pgtable.c b/arch/arm64/kvm/hyp/pgtable.c index ff6f14755d0c..ffdfd5ee9642 100644 --- a/arch/arm64/kvm/hyp/pgtable.c +++ b/arch/arm64/kvm/hyp/pgtable.c @@ -167,14 +167,23 @@ static void kvm_clear_pte(kvm_pte_t *ptep) WRITE_ONCE(*ptep, 0); } -static void kvm_set_table_pte(kvm_pte_t *ptep, kvm_pte_t *childp, - struct kvm_pgtable_mm_ops *mm_ops) +static kvm_pte_t kvm_init_table_pte(kvm_pte_t *childp, struct kvm_pgtable_mm_ops *mm_ops) { - kvm_pte_t old = *ptep, pte = kvm_phys_to_pte(mm_ops->virt_to_phys(childp)); + kvm_pte_t pte = kvm_phys_to_pte(mm_ops->virt_to_phys(childp)); pte |= FIELD_PREP(KVM_PTE_TYPE, KVM_PTE_TYPE_TABLE); pte |= KVM_PTE_VALID; + return pte; +} + +static void kvm_set_table_pte(kvm_pte_t *ptep, kvm_pte_t *childp, + struct kvm_pgtable_mm_ops *mm_ops) +{ + kvm_pte_t pte, old = *ptep; + + pte = kvm_init_table_pte(childp, mm_ops); + WARN_ON(kvm_pte_valid(old)); smp_store_release(ptep, pte); } @@ -931,7 +940,7 @@ static int stage2_map_walk_leaf(u64 addr, u64 end, u32 level, kvm_pte_t *ptep, kvm_pte_t *old, struct stage2_map_data *data, bool shared) { struct kvm_pgtable_mm_ops *mm_ops = data->mm_ops; - kvm_pte_t *childp; + kvm_pte_t *childp, pte; int ret; if (data->anchor) { @@ -969,9 +978,9 @@ static int stage2_map_walk_leaf(u64 addr, u64 end, u32 level, kvm_pte_t *ptep, * a table. Accesses beyond 'end' that fall within the new table * will be mapped lazily. */ - kvm_set_table_pte(ptep, childp, mm_ops); - mm_ops->get_page(ptep); - *old = *ptep; + pte = kvm_init_table_pte(childp, mm_ops); + stage2_make_pte(ptep, pte, data->mm_ops); + *old = pte; return 0; }