From patchwork Mon May 22 10:32:58 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Fuad Tabba X-Patchwork-Id: 13250055 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 0B8A8C77B75 for ; Mon, 22 May 2023 10:33:33 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:Cc:To:From:Subject:Message-ID: Mime-Version:Date:Reply-To:Content-ID:Content-Description:Resent-Date: Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:In-Reply-To: References:List-Owner; bh=gjHhzlvoCAkUAEbKbFq1NtO2rodYUqltFJPaXXoKfn4=; b=oTc 10aAXt+jP1gL6Gm678ZawtkS3kvDZsvJo7Gi1CjXQ0p2Eo6+ImRi6oL2ihGGSjsoPHucgnLUlEWM8 uqx0A290v+Bp6U+J4LcfC4OGfHNM1ONNeh7+k8cfaLSsjvvcTT/Pr78ssjfeCdgtI1A2w25CwUgO/ gYdPHGySQ/T86WIbYxBC4m5myfowXBB2j1AgemkdkhbadsDIasDchhoeuC44ivtXHFCy+tAr1gDaW 2tMx8/Ig0lysQHNzwo6SrxWeMvJyMwgw3HT5IV/pHOfzBqG8mq3VUH1pCuNMupMbmTkysoBuUA1jx sLEWHILdZsMf5NuHEwUuCzlTy2gXkEg==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.96 #2 (Red Hat Linux)) id 1q12qg-005z3H-0u; Mon, 22 May 2023 10:33:06 +0000 Received: from mail-wr1-x449.google.com ([2a00:1450:4864:20::449]) by bombadil.infradead.org with esmtps (Exim 4.96 #2 (Red Hat Linux)) id 1q12qd-005z2D-2Y for linux-arm-kernel@lists.infradead.org; Mon, 22 May 2023 10:33:05 +0000 Received: by mail-wr1-x449.google.com with SMTP id ffacd0b85a97d-3094e661decso1473264f8f.2 for ; Mon, 22 May 2023 03:33:01 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20221208; t=1684751580; x=1687343580; h=cc:to:from:subject:message-id:mime-version:date:from:to:cc:subject :date:message-id:reply-to; bh=OuYPfEC3lTolcZHxyOkV/uA5U2PRVKfanaHh5bZLyhs=; b=CARegYa+KCOwleIKli38zoCxH29KqzVeIJg1egkQJ+MmU9Q9rqahxU5AGUpAG1Cxfh kXqXxKBsgC6wpqzv14W6j+lfAEsvYiDZURa7TvYu/zu9DUqVKUdnspgckdn4ZaQ+b7KT TTYLeoqeo/RVcOlcOHZDdfzvmX+3AwXp0o7ofYIhm//AQRw81EH7ZvcgrjrWRX07EQI8 EM56X1uuSY81dDT9atIJDkj9Xjxb86o/rEOmrc55qjD/QYWhF+8TjTwL6VRKlCzB0n+M Lydh2WeIc/4k5YSNCPo7V0zoaBkljTCRijOwVpfQ33g8ZPfB2KE7+bt4tYJroHzBmSvX q3Wg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1684751580; x=1687343580; h=cc:to:from:subject:message-id:mime-version:date:x-gm-message-state :from:to:cc:subject:date:message-id:reply-to; bh=OuYPfEC3lTolcZHxyOkV/uA5U2PRVKfanaHh5bZLyhs=; b=C0H5DoBOtPe0sU+UbmVMwbG14k9q1i4ew3sTz5n1MTukumHraRCFBjaPNwVCrBb5MV ZnciwWCjqMFouatCbOujuaen6jk3c8eh+iDHwpR2f66fxXWSmGvvhbkxAEgSPj7HAkHL D0vzieizsq6VCYVkECaHsFIr02JCPg71oWQbt1/4JwFzBWNAzYHZSCXE3eD+kDpqm/h5 QAPIE6IyHs1akpfkBnAdqVbPfF33aFa1EAYVXItkYd90N8H/QLHnsYgL9DNb5oNEXt/J 19HDbjxC9OtIsYX+RAPAWM16uXe38ySX167/yQwtf1bUMcFREdwffGyZ2TiDVoapzUdy Y7sQ== X-Gm-Message-State: AC+VfDxKeG3ExxVKA/hvbOGZEZvOLNm0lba8neNhfiyMKhwE7ZHUXDQX i/CcmYtW+7SqHl9V7taXmoIsO/HDyA== X-Google-Smtp-Source: ACHHUZ7tbe/scTwu6g/5zpaMTnOczY/wXe97y7NmSgNxXaS0NalPf4tWe6vF9ztRICdSstWe82RuI721SQ== X-Received: from fuad.c.googlers.com ([fda3:e722:ac3:cc00:28:9cb1:c0a8:1613]) (user=tabba job=sendgmr) by 2002:adf:dcca:0:b0:309:4424:9ce with SMTP id x10-20020adfdcca000000b00309442409cemr1191431wrm.5.1684751580720; Mon, 22 May 2023 03:33:00 -0700 (PDT) Date: Mon, 22 May 2023 11:32:58 +0100 Mime-Version: 1.0 X-Mailer: git-send-email 2.40.1.698.g37aff9b760-goog Message-ID: <20230522103258.402272-1-tabba@google.com> Subject: [PATCH] KVM: arm64: Reload PTE after invoking walker callback on preorder traversal From: Fuad Tabba To: kvmarm@lists.linux.dev Cc: linux-arm-kernel@lists.infradead.org, maz@kernel.org, james.morse@arm.com, catalin.marinas@arm.com, alexandru.elisei@arm.com, oliver.upton@linux.dev, suzuki.poulose@arm.com, yuzenghui@huawei.com, will@kernel.org, reijiw@google.com, ricarkol@google.com, dmatlack@google.com, qperret@google.com, bgardon@google.com, gshan@redhat.com, peterx@redhat.com, seanjc@google.com, tabba@google.com X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20230522_033303_852087_F205CD79 X-CRM114-Status: GOOD ( 19.91 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org The preorder callback on the kvm_pgtable_stage2_map() path can replace a table with a block, then recursively free the detached table. The higher-level walking logic stashes the old page table entry and then walks the freed table, invoking the leaf callback and potentially freeing pgtable pages prematurely. In normal operation, the call to tear down the detached stage-2 is indirected and uses an RCU callback to trigger the freeing. RCU is not available to pKVM, which is where this bug is triggered. Change the behavior of the walker to reload the page table entry after invoking the walker callback on preorder traversal, as it does for leaf entries. Tested on Pixel 6. Fixes: 5c359cca1faf ("KVM: arm64: Tear down unlinked stage-2 subtree after break-before-make") Suggested-by: Oliver Upton Signed-off-by: Fuad Tabba --- Based on: f1fcbaa18b28 (6.4-rc2) The bug can be triggered by applying Will's FFA series [1] to android mainline [2] and booting a Pixel 6 in protected mode (pKVM). [1] 20230419122051.1341-1-will@kernel.org [2] https://android.googlesource.com/kernel/common/+/refs/tags/android-mainline-6.3 --- arch/arm64/include/asm/kvm_pgtable.h | 6 +++--- arch/arm64/kvm/hyp/pgtable.c | 14 +++++++++++++- 2 files changed, 16 insertions(+), 4 deletions(-) base-commit: f1fcbaa18b28dec10281551dfe6ed3a3ed80e3d6 diff --git a/arch/arm64/include/asm/kvm_pgtable.h b/arch/arm64/include/asm/kvm_pgtable.h index 4cd6762bda80..3664f1d85ce6 100644 --- a/arch/arm64/include/asm/kvm_pgtable.h +++ b/arch/arm64/include/asm/kvm_pgtable.h @@ -631,9 +631,9 @@ int kvm_pgtable_stage2_flush(struct kvm_pgtable *pgt, u64 addr, u64 size); * * The walker will walk the page-table entries corresponding to the input * address range specified, visiting entries according to the walker flags. - * Invalid entries are treated as leaf entries. Leaf entries are reloaded - * after invoking the walker callback, allowing the walker to descend into - * a newly installed table. + * Invalid entries are treated as leaf entries. The visited page table entry is + * reloaded after invoking the walker callback, allowing the walker to descend + * into a newly installed table. * * Returning a negative error code from the walker callback function will * terminate the walk immediately with the same error code. diff --git a/arch/arm64/kvm/hyp/pgtable.c b/arch/arm64/kvm/hyp/pgtable.c index 3d61bd3e591d..120c49d52ca0 100644 --- a/arch/arm64/kvm/hyp/pgtable.c +++ b/arch/arm64/kvm/hyp/pgtable.c @@ -207,14 +207,26 @@ static inline int __kvm_pgtable_visit(struct kvm_pgtable_walk_data *data, .flags = flags, }; int ret = 0; + bool reload = false; kvm_pteref_t childp; bool table = kvm_pte_table(ctx.old, level); - if (table && (ctx.flags & KVM_PGTABLE_WALK_TABLE_PRE)) + if (table && (ctx.flags & KVM_PGTABLE_WALK_TABLE_PRE)) { ret = kvm_pgtable_visitor_cb(data, &ctx, KVM_PGTABLE_WALK_TABLE_PRE); + reload = true; + } if (!table && (ctx.flags & KVM_PGTABLE_WALK_LEAF)) { ret = kvm_pgtable_visitor_cb(data, &ctx, KVM_PGTABLE_WALK_LEAF); + reload = true; + } + + /* + * Reload the page table after invoking the walker callback for leaf + * entries or after pre-order traversal, to allow the walker to descend + * into a newly installed or replaced table. + */ + if (reload) { ctx.old = READ_ONCE(*ptep); table = kvm_pte_table(ctx.old, level); }