From patchwork Fri Dec 2 18:51:51 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Oliver Upton X-Patchwork-Id: 13063126 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id C1A79C4332F for ; Fri, 2 Dec 2022 18:53:26 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Message-Id:Date:Subject:Cc:To:From:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=e8ccvYFE1H26bf6HpuIOSD8T1YWZ8gdhwmtd22f21sA=; b=0Y0c86+C1fNCLA pYcETEvvnL/cS2QdAHoxMhAnEDhuiA+cilmjzXdfZ5IKbwIrXBWp4SnQ8whBn5Y8omw4MjdCiYqmf V6MwGQKC831z8xZuVJx+E5viEHRVPinXwEQTnKsoFh9koSiE2jpYrTezgredDCPfdd8jn1WZh5ooB 4Fgo4QrHCTqxikPvXqx6TwYueJGtROgVDB1sFugWYMhruGxseY4HODHFf9Wv5/KRWXNTdXFpl0e/D DShcEKJSK8hlKS85ZwFu+LV23RdBC1EEFCpvEoPl7z+P6lju0bl/pJ1vc3ZsRmjR2H+JEHlunRdnJ tkLOxEx3CBJ88uGwtSYQ==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1p1B95-000loO-2m; Fri, 02 Dec 2022 18:52:23 +0000 Received: from out-184.mta0.migadu.com ([91.218.175.184]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1p1B92-000lkq-0o for linux-arm-kernel@lists.infradead.org; Fri, 02 Dec 2022 18:52:21 +0000 X-Report-Abuse: Please report any abuse attempt to abuse@migadu.com and include these headers. DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.dev; s=key1; t=1670007134; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=HSI8GaEsrat3YCFbYitHgJjBxasqUXZCSku/OSEtqlI=; b=rn85i92GvmxIPofIBwzmf1lgVTJuWw0qPq1mAfrFEoJGvXJqdyrlRYn1kHDeuX+rlP4Vx/ tmJ5gmeMa66ZXMnADmpVdaE5YinkQjY/RVPLx5K/leof84T2RL7O+6UkRik8oNep32Mcye z1yTKtG3yErw1oBHC6vUD1maCGLrZ00= From: Oliver Upton To: Marc Zyngier , James Morse , Alexandru Elisei , Suzuki K Poulose , Oliver Upton , Catalin Marinas , Will Deacon Cc: linux-arm-kernel@lists.infradead.org, kvmarm@lists.cs.columbia.edu, kvm@vger.kernel.org, kvmarm@lists.linux.dev, Ricardo Koller , linux-kernel@vger.kernel.org Subject: [PATCH v2 1/6] KVM: arm64: Use KVM's pte type/helpers in handle_access_fault() Date: Fri, 2 Dec 2022 18:51:51 +0000 Message-Id: <20221202185156.696189-2-oliver.upton@linux.dev> In-Reply-To: <20221202185156.696189-1-oliver.upton@linux.dev> References: <20221202185156.696189-1-oliver.upton@linux.dev> MIME-Version: 1.0 X-Migadu-Flow: FLOW_OUT X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20221202_105220_253286_9378713D X-CRM114-Status: UNSURE ( 9.83 ) X-CRM114-Notice: Please train this message. X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Consistently use KVM's own pte types and helpers in handle_access_fault(). No functional change intended. Signed-off-by: Oliver Upton --- arch/arm64/include/asm/kvm_pgtable.h | 5 +++++ arch/arm64/kvm/mmu.c | 10 ++++------ 2 files changed, 9 insertions(+), 6 deletions(-) diff --git a/arch/arm64/include/asm/kvm_pgtable.h b/arch/arm64/include/asm/kvm_pgtable.h index 63f81b27a4e3..192f33b88dc1 100644 --- a/arch/arm64/include/asm/kvm_pgtable.h +++ b/arch/arm64/include/asm/kvm_pgtable.h @@ -71,6 +71,11 @@ static inline kvm_pte_t kvm_phys_to_pte(u64 pa) return pte; } +static inline kvm_pfn_t kvm_pte_to_pfn(kvm_pte_t pte) +{ + return __phys_to_pfn(kvm_pte_to_phys(pte)); +} + static inline u64 kvm_granule_shift(u32 level) { /* Assumes KVM_PGTABLE_MAX_LEVELS is 4 */ diff --git a/arch/arm64/kvm/mmu.c b/arch/arm64/kvm/mmu.c index a3c71b5172cd..886ad5ee767a 100644 --- a/arch/arm64/kvm/mmu.c +++ b/arch/arm64/kvm/mmu.c @@ -1399,20 +1399,18 @@ static int user_mem_abort(struct kvm_vcpu *vcpu, phys_addr_t fault_ipa, /* Resolve the access fault by making the page young again. */ static void handle_access_fault(struct kvm_vcpu *vcpu, phys_addr_t fault_ipa) { - pte_t pte; - kvm_pte_t kpte; + kvm_pte_t pte; struct kvm_s2_mmu *mmu; trace_kvm_access_fault(fault_ipa); write_lock(&vcpu->kvm->mmu_lock); mmu = vcpu->arch.hw_mmu; - kpte = kvm_pgtable_stage2_mkyoung(mmu->pgt, fault_ipa); + pte = kvm_pgtable_stage2_mkyoung(mmu->pgt, fault_ipa); write_unlock(&vcpu->kvm->mmu_lock); - pte = __pte(kpte); - if (pte_valid(pte)) - kvm_set_pfn_accessed(pte_pfn(pte)); + if (kvm_pte_valid(pte)) + kvm_set_pfn_accessed(kvm_pte_to_pfn(pte)); } /** From patchwork Fri Dec 2 18:51:52 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Oliver Upton X-Patchwork-Id: 13063128 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id B338FC4332F for ; Fri, 2 Dec 2022 18:53:41 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Message-Id:Date:Subject:Cc:To:From:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=cqOapxgGrqtJ9dKexiCRBMqVuih9wIiQLzb0en52VQY=; b=PNfjraEX4NzUWB 0ul22QFjgbmype7vkYYIHHFC5XM6vUU0dmmpI2xHysQd0kQQev9Xg845nWrJ7Ku0hOI6zTo+5Jn6p cPJUnBPVxePViN9F2NGw/TRfD1O3sbgiPoCsdi6DLZVvO6mdnvNS6tDYQCHFLEnU0jvtdtT008iLV 7O6h/DWfE+T8DLUiTH1raCDqt3/mDWXj9ylo6l+z2uZpMtQJdAo0kyqaEhaGgaA7LOhaJrbqPJNp6 PV0Eg5tlsgJQ7zxY+uvS4k/+l4D37xXG6NpNOE2n/iTpt8Oq8GvMgc19u/zml8FYZk6+buA0AftW5 OiYuyPvlmvT9Mc3acpZQ==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1p1B9M-000lvK-R8; Fri, 02 Dec 2022 18:52:40 +0000 Received: from out-230.mta0.migadu.com ([91.218.175.230]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1p1B92-000lm0-6u for linux-arm-kernel@lists.infradead.org; Fri, 02 Dec 2022 18:52:23 +0000 X-Report-Abuse: Please report any abuse attempt to abuse@migadu.com and include these headers. DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.dev; s=key1; t=1670007137; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=DotXXATh2xdtC3iIVT5NnRWrGz5Cdan4kCRiJ+W1n+E=; b=MzpkHqPQ3VskkGqUcudj5NRlAggncjx0SdgO8O/+0tAeK0c/ddUKQDLutnJu+G9j8VivWd 7gY7H+zXwq74Dcvw3YxHkIqf+VUo+UbCQh+605YPvLVeore39m4c+9bYvNplDUb7IoyI+s 1KSMKc/XU9K87xYkbvsz2hBEtFxDs9o= From: Oliver Upton To: Marc Zyngier , James Morse , Alexandru Elisei , Suzuki K Poulose , Oliver Upton , Catalin Marinas , Will Deacon Cc: linux-arm-kernel@lists.infradead.org, kvmarm@lists.cs.columbia.edu, kvm@vger.kernel.org, kvmarm@lists.linux.dev, Ricardo Koller , linux-kernel@vger.kernel.org Subject: [PATCH v2 2/6] KVM: arm64: Ignore EAGAIN for walks outside of a fault Date: Fri, 2 Dec 2022 18:51:52 +0000 Message-Id: <20221202185156.696189-3-oliver.upton@linux.dev> In-Reply-To: <20221202185156.696189-1-oliver.upton@linux.dev> References: <20221202185156.696189-1-oliver.upton@linux.dev> MIME-Version: 1.0 X-Migadu-Flow: FLOW_OUT X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20221202_105220_562729_3A8D4460 X-CRM114-Status: GOOD ( 17.68 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org The page table walkers are invoked outside fault handling paths, such as write protecting a range of memory. EAGAIN is generally used by the walkers to retry execution due to races on a particular PTE, like taking an access fault on a PTE being invalidated from another thread. This early return behavior is undesirable for walkers that operate outside a fault handler. Suppress EAGAIN and continue the walk if operating outside a fault handler. Signed-off-by: Oliver Upton --- arch/arm64/include/asm/kvm_pgtable.h | 3 +++ arch/arm64/kvm/hyp/pgtable.c | 30 +++++++++++++++++++++++++--- arch/arm64/kvm/mmu.c | 4 +++- 3 files changed, 33 insertions(+), 4 deletions(-) diff --git a/arch/arm64/include/asm/kvm_pgtable.h b/arch/arm64/include/asm/kvm_pgtable.h index 192f33b88dc1..4cd6762bda80 100644 --- a/arch/arm64/include/asm/kvm_pgtable.h +++ b/arch/arm64/include/asm/kvm_pgtable.h @@ -193,12 +193,15 @@ typedef bool (*kvm_pgtable_force_pte_cb_t)(u64 addr, u64 end, * children. * @KVM_PGTABLE_WALK_SHARED: Indicates the page-tables may be shared * with other software walkers. + * @KVM_PGTABLE_WALK_HANDLE_FAULT: Indicates the page-table walk was + * invoked from a fault handler. */ enum kvm_pgtable_walk_flags { KVM_PGTABLE_WALK_LEAF = BIT(0), KVM_PGTABLE_WALK_TABLE_PRE = BIT(1), KVM_PGTABLE_WALK_TABLE_POST = BIT(2), KVM_PGTABLE_WALK_SHARED = BIT(3), + KVM_PGTABLE_WALK_HANDLE_FAULT = BIT(4), }; struct kvm_pgtable_visit_ctx { diff --git a/arch/arm64/kvm/hyp/pgtable.c b/arch/arm64/kvm/hyp/pgtable.c index b11cf2c618a6..98818214a479 100644 --- a/arch/arm64/kvm/hyp/pgtable.c +++ b/arch/arm64/kvm/hyp/pgtable.c @@ -168,6 +168,25 @@ static int kvm_pgtable_visitor_cb(struct kvm_pgtable_walk_data *data, return walker->cb(ctx, visit); } +static bool kvm_pgtable_walk_continue(const struct kvm_pgtable_walker *walker, + int r) +{ + /* + * Visitor callbacks return EAGAIN when the conditions that led to a + * fault are no longer reflected in the page tables due to a race to + * update a PTE. In the context of a fault handler this is interpreted + * as a signal to retry guest execution. + * + * Ignore the return code altogether for walkers outside a fault handler + * (e.g. write protecting a range of memory) and chug along with the + * page table walk. + */ + if (r == -EAGAIN) + return !(walker->flags & KVM_PGTABLE_WALK_HANDLE_FAULT); + + return !r; +} + static int __kvm_pgtable_walk(struct kvm_pgtable_walk_data *data, struct kvm_pgtable_mm_ops *mm_ops, kvm_pteref_t pgtable, u32 level); @@ -200,7 +219,7 @@ static inline int __kvm_pgtable_visit(struct kvm_pgtable_walk_data *data, table = kvm_pte_table(ctx.old, level); } - if (ret) + if (!kvm_pgtable_walk_continue(data->walker, ret)) goto out; if (!table) { @@ -211,13 +230,16 @@ static inline int __kvm_pgtable_visit(struct kvm_pgtable_walk_data *data, childp = (kvm_pteref_t)kvm_pte_follow(ctx.old, mm_ops); ret = __kvm_pgtable_walk(data, mm_ops, childp, level + 1); - if (ret) + if (!kvm_pgtable_walk_continue(data->walker, ret)) goto out; if (ctx.flags & KVM_PGTABLE_WALK_TABLE_POST) ret = kvm_pgtable_visitor_cb(data, &ctx, KVM_PGTABLE_WALK_TABLE_POST); out: + if (kvm_pgtable_walk_continue(data->walker, ret)) + return 0; + return ret; } @@ -1095,7 +1117,8 @@ kvm_pte_t kvm_pgtable_stage2_mkyoung(struct kvm_pgtable *pgt, u64 addr) { kvm_pte_t pte = 0; stage2_update_leaf_attrs(pgt, addr, 1, KVM_PTE_LEAF_ATTR_LO_S2_AF, 0, - &pte, NULL, 0); + &pte, NULL, + KVM_PGTABLE_WALK_HANDLE_FAULT); dsb(ishst); return pte; } @@ -1141,6 +1164,7 @@ int kvm_pgtable_stage2_relax_perms(struct kvm_pgtable *pgt, u64 addr, clr |= KVM_PTE_LEAF_ATTR_HI_S2_XN; ret = stage2_update_leaf_attrs(pgt, addr, 1, set, clr, NULL, &level, + KVM_PGTABLE_WALK_HANDLE_FAULT | KVM_PGTABLE_WALK_SHARED); if (!ret) kvm_call_hyp(__kvm_tlb_flush_vmid_ipa, pgt->mmu, addr, level); diff --git a/arch/arm64/kvm/mmu.c b/arch/arm64/kvm/mmu.c index 886ad5ee767a..dd8c715f0775 100644 --- a/arch/arm64/kvm/mmu.c +++ b/arch/arm64/kvm/mmu.c @@ -1381,7 +1381,9 @@ static int user_mem_abort(struct kvm_vcpu *vcpu, phys_addr_t fault_ipa, else ret = kvm_pgtable_stage2_map(pgt, fault_ipa, vma_pagesize, __pfn_to_phys(pfn), prot, - memcache, KVM_PGTABLE_WALK_SHARED); + memcache, + KVM_PGTABLE_WALK_HANDLE_FAULT | + KVM_PGTABLE_WALK_SHARED); /* Mark the page dirty only if the fault is handled successfully */ if (writable && !ret) { From patchwork Fri Dec 2 18:51:53 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Oliver Upton X-Patchwork-Id: 13063129 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 4DE39C4332F for ; Fri, 2 Dec 2022 18:53:46 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Message-Id:Date:Subject:Cc:To:From:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=wsh4FSVeL9gy11UqMO5sSbzxH4E75hg4stDiIqu/I+s=; b=c0mZ2zJ85ZpmX3 gzwMAyaz3JWIJeS3mf1eiRFtSgn3ZwUq80+Xy0EPgwLgxrQ+VTwAI2/i0obs8BO0Q1sK2bXeZGnfB H+zx8ZwAYbfFpEt7bcxDFjYA3R2nga1HxDB99n3PstO4jFS5+kJ3E8y+1tJk9im5fyGe9YOFeukSI W4/6GUVaAtFpBajgGilIHhJTlHCCfPXU6om3odcH4dZ7W/PCbV3M/OvIYVpzzxNmw2ET5NKnFg56L 00Oq0SAT4JrisLbnr/OT4xvvqQ8q5LDxcbhWYRvc+ZGtT8UeuMOgaQJNorZiOrrNvt1CU0Q+HEQ75 BduHnFPxfQrpxXnvsrvw==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1p1B9V-000lzK-To; Fri, 02 Dec 2022 18:52:50 +0000 Received: from out-160.mta0.migadu.com ([2001:41d0:1004:224b::a0]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1p1B9A-000lnN-0l for linux-arm-kernel@lists.infradead.org; Fri, 02 Dec 2022 18:52:29 +0000 X-Report-Abuse: Please report any abuse attempt to abuse@migadu.com and include these headers. DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.dev; s=key1; t=1670007140; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=p0G2k7kswoLiP2SpEr1E5NK57Rotg3EVnFBgR0I4F3I=; b=eCd/6YTIdgNLrnwSwiWJGcww3hgEmBKfKAvGafrp8UOmQ8M7RWmBMHLU7/9MoNFNay6G8S yVt6FFkHbzpvEXwXFLXVaUFyS2MFoJgmVsptp5ChP1IWoXkjs0rfzAUfwQcr+ReyjY+0v2 sdb951ZVTYiLT9uRYhjOv9fbV9xgyvo= From: Oliver Upton To: Marc Zyngier , James Morse , Alexandru Elisei , Suzuki K Poulose , Oliver Upton , Catalin Marinas , Will Deacon Cc: linux-arm-kernel@lists.infradead.org, kvmarm@lists.cs.columbia.edu, kvm@vger.kernel.org, kvmarm@lists.linux.dev, Ricardo Koller , linux-kernel@vger.kernel.org Subject: [PATCH v2 3/6] KVM: arm64: Return EAGAIN for invalid PTE in attr walker Date: Fri, 2 Dec 2022 18:51:53 +0000 Message-Id: <20221202185156.696189-4-oliver.upton@linux.dev> In-Reply-To: <20221202185156.696189-1-oliver.upton@linux.dev> References: <20221202185156.696189-1-oliver.upton@linux.dev> MIME-Version: 1.0 X-Migadu-Flow: FLOW_OUT X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20221202_105228_232505_A0C21CF6 X-CRM114-Status: GOOD ( 10.22 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Return EAGAIN for invalid PTEs in the attr walker, signaling to the caller that any serialization and/or invalidation can be elided. Signed-off-by: Oliver Upton --- arch/arm64/kvm/hyp/pgtable.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/arch/arm64/kvm/hyp/pgtable.c b/arch/arm64/kvm/hyp/pgtable.c index 98818214a479..204e59e05674 100644 --- a/arch/arm64/kvm/hyp/pgtable.c +++ b/arch/arm64/kvm/hyp/pgtable.c @@ -1048,7 +1048,7 @@ static int stage2_attr_walker(const struct kvm_pgtable_visit_ctx *ctx, struct kvm_pgtable_mm_ops *mm_ops = ctx->mm_ops; if (!kvm_pte_valid(ctx->old)) - return 0; + return -EAGAIN; data->level = ctx->level; data->pte = pte; From patchwork Fri Dec 2 18:51:54 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Oliver Upton X-Patchwork-Id: 13063130 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id D9890C4332F for ; Fri, 2 Dec 2022 18:53:56 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Message-Id:Date:Subject:Cc:To:From:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=J5JiSjJ/7WGhw4t/s9Mz+nrv8h9r827Xy8gSzRmE7n4=; b=eeWl0EwZJWk7rz KzTCNnn8yr4yw0nbF1jQothGoYJwQd1OSi2xzzqIK/3BWqjlY0UUUI1ylSSWAjRFNaNxFm8J6Qvgq kvIkXim/WHszGchOe6Ypl92PhFAJQ0v518sERf6dqg1b4BgyjmTQtsq0/OCF822XajNDmasbHm7Mt zCji6KJ9KJcGjY/v+erkvGHVfh+nfjxhKabAxkPuW2HPH4lIb2cw9hzI6ZrnhEr2OJQhxY7LxwJzT QuzOFKD/7tJhqwQr5piJG8316XcyY0hCIlKAAKKmkCXxO24W7C5E7upu0ClOP5V6om3UjcBX3BWbs wIXJkaQbskJSxURxg2hg==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1p1B9k-000m50-0E; Fri, 02 Dec 2022 18:53:04 +0000 Received: from out-114.mta0.migadu.com ([91.218.175.114]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1p1B9A-000low-6y for linux-arm-kernel@lists.infradead.org; Fri, 02 Dec 2022 18:52:29 +0000 X-Report-Abuse: Please report any abuse attempt to abuse@migadu.com and include these headers. DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.dev; s=key1; t=1670007143; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=9ZCh4EK+ILrdNiTkDxgoQos8GcG756fNB3ErlzEuT8I=; b=oIRtW/qtFE+f4BRGQbkhz9LDxzzx4jHeyI9Kr0i7Q8X237Bcpai7BotPaX/3hUG5tesQIk cEyEUI31AfdEY2A3AHbp0YGJ75fHka73HJXRL0JgsS4pFl9SRkWSiwXdZxTV0TuRmHmuEf YhFScCjKQgwpM8fNDDxvtEDM8wq+Q8E= From: Oliver Upton To: Marc Zyngier , James Morse , Alexandru Elisei , Suzuki K Poulose , Oliver Upton , Catalin Marinas , Will Deacon Cc: linux-arm-kernel@lists.infradead.org, kvmarm@lists.cs.columbia.edu, kvm@vger.kernel.org, kvmarm@lists.linux.dev, Ricardo Koller , linux-kernel@vger.kernel.org Subject: [PATCH v2 4/6] KVM: arm64: Don't serialize if the access flag isn't set Date: Fri, 2 Dec 2022 18:51:54 +0000 Message-Id: <20221202185156.696189-5-oliver.upton@linux.dev> In-Reply-To: <20221202185156.696189-1-oliver.upton@linux.dev> References: <20221202185156.696189-1-oliver.upton@linux.dev> MIME-Version: 1.0 X-Migadu-Flow: FLOW_OUT X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20221202_105228_452187_91598BB3 X-CRM114-Status: UNSURE ( 9.41 ) X-CRM114-Notice: Please train this message. X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Of course, if the PTE wasn't changed then there are absolutely no serialization requirements. Skip the DSB for an unsuccessful update to the access flag. Signed-off-by: Oliver Upton --- arch/arm64/kvm/hyp/pgtable.c | 12 ++++++++---- 1 file changed, 8 insertions(+), 4 deletions(-) diff --git a/arch/arm64/kvm/hyp/pgtable.c b/arch/arm64/kvm/hyp/pgtable.c index 204e59e05674..aa36d896bd8c 100644 --- a/arch/arm64/kvm/hyp/pgtable.c +++ b/arch/arm64/kvm/hyp/pgtable.c @@ -1116,10 +1116,14 @@ int kvm_pgtable_stage2_wrprotect(struct kvm_pgtable *pgt, u64 addr, u64 size) kvm_pte_t kvm_pgtable_stage2_mkyoung(struct kvm_pgtable *pgt, u64 addr) { kvm_pte_t pte = 0; - stage2_update_leaf_attrs(pgt, addr, 1, KVM_PTE_LEAF_ATTR_LO_S2_AF, 0, - &pte, NULL, - KVM_PGTABLE_WALK_HANDLE_FAULT); - dsb(ishst); + int ret; + + ret = stage2_update_leaf_attrs(pgt, addr, 1, KVM_PTE_LEAF_ATTR_LO_S2_AF, 0, + &pte, NULL, + KVM_PGTABLE_WALK_HANDLE_FAULT); + if (!ret) + dsb(ishst); + return pte; } From patchwork Fri Dec 2 18:51:55 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Oliver Upton X-Patchwork-Id: 13063131 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 9A07DC4332F for ; Fri, 2 Dec 2022 18:54:26 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Message-Id:Date:Subject:Cc:To:From:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=4PgSLiefhMjWbHqPB25vATbPuvLNTY8g9wrOUssYwNs=; b=qb3JigT5ZjbS35 JLymH6TL5ToIlWI4fH9u7tNmLOz4KgMOEzDxwEikU+oNHGYWG5sLZoUMPxXUGMm4ihGxXYM+ns921 Ta282Vdh2QlVDafat2ON9vGFW8FTZbRphjVmSPq5K86PxZyxQF4W8H2l3Sk9UlJ23y1JMLwvqWrk5 QP3DMk1c4EgjOhFi4ZH0VGmAiEI3jhHryuhgzwTijpWI8g3y1RrFKGlfEPuXNserzLDGdv7l9MsgI J8vSPGHEMHvRkQHyrSd7CbjP5zBZDBkFP9SDLMnCNFHztHQAeFSRvHKMe3T6vecnhqtfmDp/kGTsG lgaOhcTaZO7g349yJF2w==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1p1BA6-000mGa-Pa; Fri, 02 Dec 2022 18:53:26 +0000 Received: from out-59.mta0.migadu.com ([91.218.175.59]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1p1B9B-000lqY-IY for linux-arm-kernel@lists.infradead.org; Fri, 02 Dec 2022 18:52:31 +0000 X-Report-Abuse: Please report any abuse attempt to abuse@migadu.com and include these headers. DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.dev; s=key1; t=1670007146; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=36q34GkAdZNcp02VfFJ/fMceb+GlTAxgwL6t8UJ/nCU=; b=LAPfFRXkfCOfssjRADm7/jdYZ7LQl2uW9dy14PfWeo2914srk2DSSW7OAI1GrB7JEscgbY DxllYbdQAX26FIFaFFLzCntc4Uhua5fSy2WeUOjvTWcv81z6joZgp064csO8ICW67kHF+9 cSAdicod8mmQR59+MNnYmUOZe79b8Wc= From: Oliver Upton To: Marc Zyngier , James Morse , Alexandru Elisei , Suzuki K Poulose , Oliver Upton , Catalin Marinas , Will Deacon Cc: linux-arm-kernel@lists.infradead.org, kvmarm@lists.cs.columbia.edu, kvm@vger.kernel.org, kvmarm@lists.linux.dev, Ricardo Koller , linux-kernel@vger.kernel.org Subject: [PATCH v2 5/6] KVM: arm64: Handle access faults behind the read lock Date: Fri, 2 Dec 2022 18:51:55 +0000 Message-Id: <20221202185156.696189-6-oliver.upton@linux.dev> In-Reply-To: <20221202185156.696189-1-oliver.upton@linux.dev> References: <20221202185156.696189-1-oliver.upton@linux.dev> MIME-Version: 1.0 X-Migadu-Flow: FLOW_OUT X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20221202_105229_794398_D23E8FA7 X-CRM114-Status: UNSURE ( 9.20 ) X-CRM114-Notice: Please train this message. X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org As the underlying software walkers are able to traverse and update stage-2 in parallel there is no need to serialize access faults. Only take the read lock when handling an access fault. Signed-off-by: Oliver Upton --- arch/arm64/kvm/hyp/pgtable.c | 3 ++- arch/arm64/kvm/mmu.c | 4 ++-- 2 files changed, 4 insertions(+), 3 deletions(-) diff --git a/arch/arm64/kvm/hyp/pgtable.c b/arch/arm64/kvm/hyp/pgtable.c index aa36d896bd8c..30575b5f5dcd 100644 --- a/arch/arm64/kvm/hyp/pgtable.c +++ b/arch/arm64/kvm/hyp/pgtable.c @@ -1120,7 +1120,8 @@ kvm_pte_t kvm_pgtable_stage2_mkyoung(struct kvm_pgtable *pgt, u64 addr) ret = stage2_update_leaf_attrs(pgt, addr, 1, KVM_PTE_LEAF_ATTR_LO_S2_AF, 0, &pte, NULL, - KVM_PGTABLE_WALK_HANDLE_FAULT); + KVM_PGTABLE_WALK_HANDLE_FAULT | + KVM_PGTABLE_WALK_SHARED); if (!ret) dsb(ishst); diff --git a/arch/arm64/kvm/mmu.c b/arch/arm64/kvm/mmu.c index dd8c715f0775..25e18e1d9a15 100644 --- a/arch/arm64/kvm/mmu.c +++ b/arch/arm64/kvm/mmu.c @@ -1406,10 +1406,10 @@ static void handle_access_fault(struct kvm_vcpu *vcpu, phys_addr_t fault_ipa) trace_kvm_access_fault(fault_ipa); - write_lock(&vcpu->kvm->mmu_lock); + read_lock(&vcpu->kvm->mmu_lock); mmu = vcpu->arch.hw_mmu; pte = kvm_pgtable_stage2_mkyoung(mmu->pgt, fault_ipa); - write_unlock(&vcpu->kvm->mmu_lock); + read_unlock(&vcpu->kvm->mmu_lock); if (kvm_pte_valid(pte)) kvm_set_pfn_accessed(kvm_pte_to_pfn(pte)); From patchwork Fri Dec 2 18:51:56 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Oliver Upton X-Patchwork-Id: 13063132 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 3EEA4C47088 for ; Fri, 2 Dec 2022 18:54:41 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Message-Id:Date:Subject:Cc:To:From:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=QdqcAlfPBb7I6OjiOB0BO+LUCg5Ctzs7YAJ9u3cNn7g=; b=wVzwF9TPRdbQel RJfr0Np3CEjyCeK3h0B8xuLCVnPVQscUzjFgI1rlKfUhQmFoO6Er/UG2DTpkt6maox8Wnv/6NYgI+ Clajzp1Tw6b4tZGRbkmqvzrziXnd3D5lZaQFRBFyy4Lb9vNBYSW+DotlMYuFXC/pB99ANEJ1+XHeh IWGxF3HAjhYr4WBNKK0XLq84jy7dpsxavr3es0IBmgFxp1pw3O8Q7CJuFUkQwvU3IKenl+t8f5j8L tot81oQ28RykZ2IecsDXFoAuCnyw0ANiifxoj0ZNdV1dWksl9gjvZK5lobxtEGRRGIY7p3rUzYD5J RmKf8yz6j3fKhYMqvcBQ==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1p1BAP-000mY9-Ni; Fri, 02 Dec 2022 18:53:45 +0000 Received: from out-100.mta0.migadu.com ([91.218.175.100]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1p1B9E-000lsH-Fv for linux-arm-kernel@lists.infradead.org; Fri, 02 Dec 2022 18:52:33 +0000 X-Report-Abuse: Please report any abuse attempt to abuse@migadu.com and include these headers. DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.dev; s=key1; t=1670007149; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=m5/0ZswNwgSJxP0pUMVY0+4NluAB70GQeaBLsW+AvoM=; b=BWVsnn8YRvzHQkRDkssOV+GGDtLI9N13an/VllrSUa7NSf5DViJNMvV5pDptDQuC0BvQvd v5RHoXV/YeZPcoDowAdvpr83VkUHJbAS76GbVtHvLA4FQ5kHZniQLDT1A0KGtcZewy4uaq rqVIQYOMD+MlfNaOJoXftx57B6fc2KU= From: Oliver Upton To: Marc Zyngier , James Morse , Alexandru Elisei , Suzuki K Poulose , Oliver Upton , Catalin Marinas , Will Deacon Cc: linux-arm-kernel@lists.infradead.org, kvmarm@lists.cs.columbia.edu, kvm@vger.kernel.org, kvmarm@lists.linux.dev, Ricardo Koller , linux-kernel@vger.kernel.org Subject: [PATCH v2 6/6] KVM: arm64: Condition HW AF updates on config option Date: Fri, 2 Dec 2022 18:51:56 +0000 Message-Id: <20221202185156.696189-7-oliver.upton@linux.dev> In-Reply-To: <20221202185156.696189-1-oliver.upton@linux.dev> References: <20221202185156.696189-1-oliver.upton@linux.dev> MIME-Version: 1.0 X-Migadu-Flow: FLOW_OUT X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20221202_105232_693403_C4C403F3 X-CRM114-Status: UNSURE ( 9.05 ) X-CRM114-Notice: Please train this message. X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org As it currently stands, KVM makes use of FEAT_HAFDBS unconditionally. Use of the feature in the rest of the kernel is guarded by an associated Kconfig option. Align KVM with the rest of the kernel and only enable VTCR_HA when ARM64_HW_AFDBM is enabled. This can be helpful for testing changes to the stage-2 access fault path on Armv8.1+ implementations. Signed-off-by: Oliver Upton --- arch/arm64/kvm/hyp/pgtable.c | 2 ++ 1 file changed, 2 insertions(+) diff --git a/arch/arm64/kvm/hyp/pgtable.c b/arch/arm64/kvm/hyp/pgtable.c index 30575b5f5dcd..3d61bd3e591d 100644 --- a/arch/arm64/kvm/hyp/pgtable.c +++ b/arch/arm64/kvm/hyp/pgtable.c @@ -606,12 +606,14 @@ u64 kvm_get_vtcr(u64 mmfr0, u64 mmfr1, u32 phys_shift) lvls = 2; vtcr |= VTCR_EL2_LVLS_TO_SL0(lvls); +#ifdef CONFIG_ARM64_HW_AFDBM /* * Enable the Hardware Access Flag management, unconditionally * on all CPUs. The features is RES0 on CPUs without the support * and must be ignored by the CPUs. */ vtcr |= VTCR_EL2_HA; +#endif /* CONFIG_ARM64_HW_AFDBM */ /* Set the vmid bits */ vtcr |= (get_vmid_bits(mmfr1) == 16) ?