From patchwork Wed Jan 31 09:34:55 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Christoffer Dall X-Patchwork-Id: 10193649 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork.web.codeaurora.org (Postfix) with ESMTP id B6F5360595 for ; Wed, 31 Jan 2018 09:38:18 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 99EE3201BC for ; Wed, 31 Jan 2018 09:38:18 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 8DAF62860F; Wed, 31 Jan 2018 09:38:18 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-4.2 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,RCVD_IN_DNSWL_MED autolearn=ham version=3.3.1 Received: from bombadil.infradead.org (bombadil.infradead.org [65.50.211.133]) (using TLSv1.2 with cipher AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.wl.linuxfoundation.org (Postfix) with ESMTPS id 1E6CD201BC for ; Wed, 31 Jan 2018 09:38:18 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20170209; h=Sender: Content-Transfer-Encoding:Content-Type:MIME-Version:Cc:List-Subscribe: List-Help:List-Post:List-Archive:List-Unsubscribe:List-Id:References: In-Reply-To:Message-Id:Date:Subject:To:From:Reply-To:Content-ID: Content-Description:Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc :Resent-Message-ID:List-Owner; bh=JmAxMLlrrhARQEgE+0GSNExTuvqpOxa/O9R931xEsU4=; b=epLMzjN6gDRnfPdDmINBAw+aRt YD/3zIx+HkRopFv/ILgOeWN6cESVhhMpwf4DV4giS+OHR+IGYiqCGJgk4OoSjRJ+cW9riYM716KUz u89Frmi4e4SrTA0tC3Qsq7dd4QgIpSc6erRLDymglMOu5bfKoCszqWR3v1o4LXY5353fkNo7gZ6ui nZ7uVDtnuXhYvn+8wkFa6yFCtMbu2NhWq//P20Wb5DaqcacgrxZ+skqYZt/yBAKvDn1fp+VTPnje6 rMuvEBASlEZH+uyJbAIxDWwvDLN80DicBvVahJWBzeC96GBnXDPzDwPdxF/aIQ66r3J0tcybEjoqO xP0EtEbw==; Received: from localhost ([127.0.0.1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.89 #1 (Red Hat Linux)) id 1egoqN-0008Vd-Az; Wed, 31 Jan 2018 09:38:15 +0000 Received: from mail-wm0-x243.google.com ([2a00:1450:400c:c09::243]) by bombadil.infradead.org with esmtps (Exim 4.89 #1 (Red Hat Linux)) id 1egonq-0005hc-78 for linux-arm-kernel@lists.infradead.org; Wed, 31 Jan 2018 09:35:44 +0000 Received: by mail-wm0-x243.google.com with SMTP id 141so6540444wme.3 for ; Wed, 31 Jan 2018 01:35:37 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=FxKmIRk9GblIlKatHD11QTVQAHZwhcLEQSnK4W0qITs=; b=bnoDbQ1hXEcb1uGIGNvz/pMP4KX47QaVpM2N5CIaZ3lGjyrA9x4+DqT/TQIdPuxvKV NuC7pIXRwJMeU7OWyHqi6kTKkN7rExEbxmqIJqfCz/IZSKxgyPZ6dqDQNf6J4VZrZVxO Ia/lbQraqIegE2j0lHNySI4dYc6KTK+mLyl5M= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=FxKmIRk9GblIlKatHD11QTVQAHZwhcLEQSnK4W0qITs=; b=OkiBz5XVjRm69K+sTfTgPG6Isr5JajTfQu3iZ3hJfVDka2qpcdNboFlIUfqm7WzG/Q JDfzJJ47ayL6zrgDeiGJOfQel8FLoV6yA2iY4alqDPpgE61mLRV1Pylzn5F5Ifah0D8B OITsfeLNpRNOgMetfOAa5HOcnihCFby7HrMdRAF1lbBlgfmlDaQh4KdDln+fhHu5EXKr NVnqFdEUN8yyZWeSduDjjXXAUxZ4vUhvdsqk3YwjliMBLWQAKI/VLlal/Cdi1caktSCi qaB4gUzvE2TrrTkx+9HxpnZXnoOEz6/adNocOtqwDk80/etoH9+POgP0pFGS3yk0nuaw 8G3w== X-Gm-Message-State: AKwxytemiNPIAGDVf+VhvD5U84EwrchYx0r6VVxoLMEZL7yGqb8tfoaU j+C3ovcwQr6K5EE9tWc2WPjAkA== X-Google-Smtp-Source: AH8x2240+KrApEo1Cl0wOfOdcTUVjpyZQh7Bg45OnGpXly3P8DGIqlwaRajLpEONIbbukHW1Ttliiw== X-Received: by 10.80.144.92 with SMTP id z28mr22903339edz.235.1517391336315; Wed, 31 Jan 2018 01:35:36 -0800 (PST) Received: from localhost.localdomain (x50d2404e.cust.hiper.dk. [80.210.64.78]) by smtp.gmail.com with ESMTPSA id z2sm10247837edc.50.2018.01.31.01.35.34 (version=TLS1_2 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Wed, 31 Jan 2018 01:35:35 -0800 (PST) From: Christoffer Dall To: Paolo Bonzini , =?UTF-8?q?Radim=20Kr=C4=8Dm=C3=A1=C5=99?= Subject: [PULL 16/28] arm: KVM: Add optimized PIPT icache flushing Date: Wed, 31 Jan 2018 10:34:55 +0100 Message-Id: <20180131093507.22219-17-christoffer.dall@linaro.org> X-Mailer: git-send-email 2.14.2 In-Reply-To: <20180131093507.22219-1-christoffer.dall@linaro.org> References: <20180131093507.22219-1-christoffer.dall@linaro.org> X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Marc Zyngier , Christoffer Dall , kvmarm@lists.cs.columbia.edu, linux-arm-kernel@lists.infradead.org, kvm@vger.kernel.org MIME-Version: 1.0 Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+patchwork-linux-arm=patchwork.kernel.org@lists.infradead.org X-Virus-Scanned: ClamAV using ClamSMTP From: Marc Zyngier Calling __cpuc_coherent_user_range to invalidate the icache on a PIPT icache machine has some pointless overhead, as it starts by cleaning the dcache to the PoU, while we're guaranteed to have already cleaned it to the PoC. As KVM is the only user of such a feature, let's implement some ad-hoc cache flushing in kvm_mmu.h. Should it become useful to other subsystems, it can be moved to a more global location. Reviewed-by: Christoffer Dall Signed-off-by: Marc Zyngier Signed-off-by: Christoffer Dall --- arch/arm/include/asm/kvm_hyp.h | 2 ++ arch/arm/include/asm/kvm_mmu.h | 32 +++++++++++++++++++++++++++++--- 2 files changed, 31 insertions(+), 3 deletions(-) diff --git a/arch/arm/include/asm/kvm_hyp.h b/arch/arm/include/asm/kvm_hyp.h index 76368de7237b..1ab8329e9ff7 100644 --- a/arch/arm/include/asm/kvm_hyp.h +++ b/arch/arm/include/asm/kvm_hyp.h @@ -68,6 +68,8 @@ #define HIFAR __ACCESS_CP15(c6, 4, c0, 2) #define HPFAR __ACCESS_CP15(c6, 4, c0, 4) #define ICIALLUIS __ACCESS_CP15(c7, 0, c1, 0) +#define BPIALLIS __ACCESS_CP15(c7, 0, c1, 6) +#define ICIMVAU __ACCESS_CP15(c7, 0, c5, 1) #define ATS1CPR __ACCESS_CP15(c7, 0, c8, 0) #define TLBIALLIS __ACCESS_CP15(c8, 0, c3, 0) #define TLBIALL __ACCESS_CP15(c8, 0, c7, 0) diff --git a/arch/arm/include/asm/kvm_mmu.h b/arch/arm/include/asm/kvm_mmu.h index 9fa4b2520974..bc8d21e76637 100644 --- a/arch/arm/include/asm/kvm_mmu.h +++ b/arch/arm/include/asm/kvm_mmu.h @@ -37,6 +37,8 @@ #include #include +#include +#include #include #include @@ -157,6 +159,8 @@ static inline void __invalidate_icache_guest_page(struct kvm_vcpu *vcpu, kvm_pfn_t pfn, unsigned long size) { + u32 iclsz; + /* * If we are going to insert an instruction page and the icache is * either VIPT or PIPT, there is a potential problem where the host @@ -181,18 +185,40 @@ static inline void __invalidate_icache_guest_page(struct kvm_vcpu *vcpu, return; } - /* PIPT cache. As for the d-side, use a temporary kernel mapping. */ + /* + * CTR IminLine contains Log2 of the number of words in the + * cache line, so we can get the number of words as + * 2 << (IminLine - 1). To get the number of bytes, we + * multiply by 4 (the number of bytes in a 32-bit word), and + * get 4 << (IminLine). + */ + iclsz = 4 << (read_cpuid(CPUID_CACHETYPE) & 0xf); + while (size) { void *va = kmap_atomic_pfn(pfn); + void *end = va + PAGE_SIZE; + void *addr = va; - __cpuc_coherent_user_range((unsigned long)va, - (unsigned long)va + PAGE_SIZE); + do { + write_sysreg(addr, ICIMVAU); + addr += iclsz; + } while (addr < end); + + dsb(ishst); + isb(); size -= PAGE_SIZE; pfn++; kunmap_atomic(va); } + + /* Check if we need to invalidate the BTB */ + if ((read_cpuid_ext(CPUID_EXT_MMFR1) >> 28) != 4) { + write_sysreg(0, BPIALLIS); + dsb(ishst); + isb(); + } } static inline void __kvm_flush_dcache_pte(pte_t pte)