From patchwork Fri Apr 15 21:58:48 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Oliver Upton X-Patchwork-Id: 12815447 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id E0BEDC433EF for ; Fri, 15 Apr 2022 22:01:04 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:Cc:To:From:Subject:References: Mime-Version:Message-Id:In-Reply-To:Date:Reply-To:Content-ID: Content-Description:Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc :Resent-Message-ID:List-Owner; bh=qtP92N6YqsjCU1iwE5vI1KgzL+Es+RThlLfO3VIBTC8=; b=uVdyxY8mnI43MH8kOW2O1xpYe9 ONWGrwXLDtVW+jeFn/IC/tuebJVQH2wjGr3seyXMy/lFyWtmOkE4XAvh8z9hrrpxQJvDUVfwf1HOS snki9flKaeUhmZ0lQ9U1MO3wMt8t7cD4wvHueWYAKyCRnTjD3BC+YSweIl2jg2pbmMtgo3JqGE6DZ w2eZYKwi5AnBUhC8L4k5dvBF+7n2Ba7ywvC0uNOtBXjJ4LR2eFPxC5n2ETXV2QBZ8dageXMHCDUNB PU5f7I8FGk796GYAXsqwXjR786j3GR8tZ+oVGdCSfNipKOSMJtfy4rPoYMOTcRJ0N2CBpie7LtPFV f65TU1vw==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1nfTyy-00BSOO-Mo; Fri, 15 Apr 2022 22:00:00 +0000 Received: from mail-io1-xd4a.google.com ([2607:f8b0:4864:20::d4a]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1nfTyE-00BS0Y-No for linux-arm-kernel@lists.infradead.org; Fri, 15 Apr 2022 21:59:17 +0000 Received: by mail-io1-xd4a.google.com with SMTP id z23-20020a6b0a17000000b00649f13ea3a7so5404304ioi.23 for ; Fri, 15 Apr 2022 14:59:13 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20210112; h=date:in-reply-to:message-id:mime-version:references:subject:from:to :cc; bh=tskkletjI1xqy2hmf+NUDTVo6kgDSU4EL5qdYTa3BLw=; b=QA94h9r9pTA8GiRYxYPRCc5lnTzXtNJwZmjCbvYN9o6CVFPh0a2tzWI0hkn2GGLweA Txpvs213Jksx0vyr2ngPBysyiEw8KC95EbN0Ju8DT94akH1bQ62gDiZBnBpVmOCl0cU/ aVzTHXbtTPOJrXqnXT7zTKFXplRLdZ7urB7/NYg8iYE+D1B5NQXQp/3h+iCMDQQGzAYs +i1QI9nvkwP3sApqy92wxAC2ykVI1oGcIkQRiYST9T63+AoQWeZ9+xAr1yQAoB7YwpSw L5Y0d4II8kHY6IRbv/oLjeQ9jnR7XErbOljztHLeXABbdjtB3zDvW2Om/oADovnpl9yM AUkQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=tskkletjI1xqy2hmf+NUDTVo6kgDSU4EL5qdYTa3BLw=; b=nZzyx299/n6kJClTS6oovyqQSylB/dHvilkA6nuZ+NorAFUhF6K+j31e2RBlyldOOw eBgtlAXO2M18GlOm/alu1nN12DoT9+IpUCfC1SLdHmtm8PQ1uaFh55WuhwIdMoJ3cBg3 KtFYOaPPL4qaFB9Ah2AWaFDSCEDO04oWfaWCl0fKvWoEaOWhDpe4nsSJq0HPvqVMipCo D2by2fSBaQrJFZ1MXP7lYVzhYcpgdvsng5WBuGn8MCys8+0CI0/NaCdiQhcmqn8V92F4 2x+rj+XnkKlNoeLAmUIpBtLe2vusXPdk4lVwwfIAZt65jpk8ATrmmrDGQJsXobPYVjm8 rnCA== X-Gm-Message-State: AOAM530XHeNh1+uzkUvmxM6dVS5bJi8tRVaEBz0dt2oPXMdAOhx/pckR qayJabDAz0b4xzrDFUVF55hn3kPkZrI= X-Google-Smtp-Source: ABdhPJyVhtFMabjcfJB2grqI91o53/8kDGiTwPio15sftKb54bR0cux4tb8br2mZT5w8v0L5arzr0CpVpX8= X-Received: from oupton.c.googlers.com ([fda3:e722:ac3:cc00:2b:ff92:c0a8:404]) (user=oupton job=sendgmr) by 2002:a05:6e02:20cd:b0:2ca:c074:4ebb with SMTP id 13-20020a056e0220cd00b002cac0744ebbmr283965ilq.73.1650059952284; Fri, 15 Apr 2022 14:59:12 -0700 (PDT) Date: Fri, 15 Apr 2022 21:58:48 +0000 In-Reply-To: <20220415215901.1737897-1-oupton@google.com> Message-Id: <20220415215901.1737897-5-oupton@google.com> Mime-Version: 1.0 References: <20220415215901.1737897-1-oupton@google.com> X-Mailer: git-send-email 2.36.0.rc0.470.gd361397f0d-goog Subject: [RFC PATCH 04/17] KVM: arm64: Protect page table traversal with RCU From: Oliver Upton To: kvmarm@lists.cs.columbia.edu Cc: kvm@vger.kernel.org, Marc Zyngier , James Morse , Alexandru Elisei , Suzuki K Poulose , linux-arm-kernel@lists.infradead.org, Peter Shier , Ricardo Koller , Reiji Watanabe , Paolo Bonzini , Sean Christopherson , Ben Gardon , David Matlack , Oliver Upton X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20220415_145914_838127_64B3F0DE X-CRM114-Status: GOOD ( 10.85 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Use RCU to safely traverse the page tables in parallel; the tables themselves will only be freed from an RCU synchronized context. Don't even bother with adding support to hyp, and instead just assume exclusive access of the page tables. Signed-off-by: Oliver Upton --- arch/arm64/kvm/hyp/pgtable.c | 23 ++++++++++++++++++++++- 1 file changed, 22 insertions(+), 1 deletion(-) diff --git a/arch/arm64/kvm/hyp/pgtable.c b/arch/arm64/kvm/hyp/pgtable.c index 5b64fbca8a93..d4699f698d6e 100644 --- a/arch/arm64/kvm/hyp/pgtable.c +++ b/arch/arm64/kvm/hyp/pgtable.c @@ -132,9 +132,28 @@ static kvm_pte_t kvm_phys_to_pte(u64 pa) return pte; } + +#if defined(__KVM_NVHE_HYPERVISOR__) +static inline void kvm_pgtable_walk_begin(void) +{} + +static inline void kvm_pgtable_walk_end(void) +{} + +#define kvm_dereference_ptep rcu_dereference_raw +#else +#define kvm_pgtable_walk_begin rcu_read_lock + +#define kvm_pgtable_walk_end rcu_read_unlock + +#define kvm_dereference_ptep rcu_dereference +#endif + static kvm_pte_t *kvm_pte_follow(kvm_pte_t pte, struct kvm_pgtable_mm_ops *mm_ops) { - return mm_ops->phys_to_virt(kvm_pte_to_phys(pte)); + kvm_pte_t __rcu *ptep = mm_ops->phys_to_virt(kvm_pte_to_phys(pte)); + + return kvm_dereference_ptep(ptep); } static void kvm_clear_pte(kvm_pte_t *ptep) @@ -288,7 +307,9 @@ int kvm_pgtable_walk(struct kvm_pgtable *pgt, u64 addr, u64 size, .walker = walker, }; + kvm_pgtable_walk_begin(); return _kvm_pgtable_walk(&walk_data); + kvm_pgtable_walk_end(); } struct leaf_walk_data {