From patchwork Mon May 12 09:56:00 2014 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: ??? X-Patchwork-Id: 4158161 Return-Path: X-Original-To: patchwork-linux-samsung-soc@patchwork.kernel.org Delivered-To: patchwork-parsemail@patchwork2.web.kernel.org Received: from mail.kernel.org (mail.kernel.org [198.145.19.201]) by patchwork2.web.kernel.org (Postfix) with ESMTP id 0BBA3BFF02 for ; Mon, 12 May 2014 09:56:11 +0000 (UTC) Received: from mail.kernel.org (localhost [127.0.0.1]) by mail.kernel.org (Postfix) with ESMTP id F028B201E4 for ; Mon, 12 May 2014 09:56:09 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id 75E1D200BE for ; Mon, 12 May 2014 09:56:08 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1752685AbaELJ4H (ORCPT ); Mon, 12 May 2014 05:56:07 -0400 Received: from mailout3.samsung.com ([203.254.224.33]:20624 "EHLO mailout3.samsung.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1752319AbaELJ4G (ORCPT ); Mon, 12 May 2014 05:56:06 -0400 Received: from epcpsbgr3.samsung.com (u143.gpu120.samsung.co.kr [203.254.230.143]) by mailout3.samsung.com (Oracle Communications Messaging Server 7u4-24.01 (7.0.4.24.0) 64bit (built Nov 17 2011)) with ESMTP id <0N5G00BAJGXCJE50@mailout3.samsung.com>; Mon, 12 May 2014 18:56:01 +0900 (KST) Received: from epcpsbgm2.samsung.com ( [203.254.230.48]) by epcpsbgr3.samsung.com (EPCPMTA) with SMTP id E7.EC.11120.0BA90735; Mon, 12 May 2014 18:56:00 +0900 (KST) X-AuditID: cbfee68f-b7eff6d000002b70-fc-53709ab0332c Received: from epmmp1.local.host ( [203.254.227.16]) by epcpsbgm2.samsung.com (EPCPMTA) with SMTP id 39.1E.25708.0BA90735; Mon, 12 May 2014 18:56:00 +0900 (KST) Received: from DOJAYSLEE01 ([12.36.166.151]) by mmp1.samsung.com (Oracle Communications Messaging Server 7u4-24.01 (7.0.4.24.0) 64bit (built Nov 17 2011)) with ESMTPA id <0N5G00JBUGXC86A0@mmp1.samsung.com>; Mon, 12 May 2014 18:56:00 +0900 (KST) From: Jungseok Lee To: linux-arm-kernel@lists.infradead.org, kvmarm@lists.cs.columbia.edu, Catalin.Marinas@arm.com, Marc Zyngier , Christoffer Dall Cc: linux-kernel@vger.kernel.org, linux-samsung-soc , steve.capper@linaro.org, sungjinn.chung@samsung.com, Arnd Bergmann , kgene.kim@samsung.com, ilho215.lee@samsung.com Subject: [RESEND PATCH v6 7/7] arm64: KVM: Implement 4 levels of translation tables for HYP and stage2 Date: Mon, 12 May 2014 18:56:00 +0900 Message-id: <000001cf6dc8$60eaa170$22bfe450$@samsung.com> MIME-version: 1.0 Content-type: text/plain; charset=us-ascii Content-transfer-encoding: 7bit X-Mailer: Microsoft Outlook 14.0 Thread-index: Ac9tyFCHQNZTaB1QSWyd4/FOBBsilQ== Content-language: ko X-Brightmail-Tracker: H4sIAAAAAAAAA+NgFnrPIsWRmVeSWpSXmKPExsVy+t8zA90NswqCDQ4+0rP4O+kYu8X7ZT2M Fi9e/2O0OPpvIaNF74KrbBYfTx1nt9j0+BqrxeVdc9gsZpzfx2Tx984/NosV85axWXyYsZLR gcdjzbw1jB6/f01i9LhzbQ+bx/lNa5g9Ni+p9+jbsorR4/MmuQD2KC6blNSczLLUIn27BK6M O5umsRQsdKm48biTqYFxj1kXIyeHhICJxK8vLSwQtpjEhXvr2boYuTiEBJYxSlz5toAFpuhV 5zx2iMQiRonfN++xQDh/gJz/j9hBqtgENCUe3e0BqxIR2MEoMXntIlYQh1ngIaPEz7f7mUGq hAUyJLb9aWYEsVkEVCWaZi1gBbF5BSwlPs/rh7IFJX5Mvge2m1lAS2L9zuNMELa8xOY1b5kh blKQ2HH2NdgcEQE9ice/dzJD1IhI7HvxjhFksYRAK4fEjy2n2CCWCUh8m3wIaCgHUEJWYtMB qDmSEgdX3GCZwCg2C8nqWUhWz0KyehaSFQsYWVYxiqYWJBcUJ6UXGesVJ+YWl+al6yXn525i hER2/w7GuwesDzEmA62fyCwlmpwPTAx5JfGGxmZGFqYmpsZG5pZmpAkrifPef5gUJCSQnliS mp2aWpBaFF9UmpNafIiRiYNTqoFRK8TLfbqClGnKZTldUZbYBR/WsR6+tNq4c3WHu+kFtk43 aytmJmOuW2qmOsXBcTnmXzQM4jy/r14WGpOZOzWY/WdORuOTk7EbZvx499ly+bWOh73XWzh2 BEz6tGxvDPu+xzbXXygJeOlN3ix9TS/GJXsR0xINdYWADQ0vjzyU0Zsjpn1yQocSS3FGoqEW c1FxIgDRT4TEAgMAAA== X-Brightmail-Tracker: H4sIAAAAAAAAA+NgFjrDKsWRmVeSWpSXmKPExsVy+t9jAd0NswqCDV5M1LD4O+kYu8X7ZT2M Fi9e/2O0OPpvIaNF74KrbBYfTx1nt9j0+BqrxeVdc9gsZpzfx2Tx984/NosV85axWXyYsZLR gcdjzbw1jB6/f01i9LhzbQ+bx/lNa5g9Ni+p9+jbsorR4/MmuQD2qAZGm4zUxJTUIoXUvOT8 lMy8dFsl7+B453hTMwNDXUNLC3MlhbzE3FRbJRefAF23zBygS5UUyhJzSoFCAYnFxUr6dpgm hIa46VrANEbo+oYEwfUYGaCBhHWMGXc2TWMpWOhSceNxJ1MD4x6zLkZODgkBE4lXnfPYIWwx iQv31rN1MXJxCAksYpT4ffMeC4TzB8j5/wisik1AU+LR3R52kISIwA5GiclrF7GCOMwCDxkl fr7dzwxSJSyQIbHtTzMjiM0ioCrRNGsBK4jNK2Ap8XleP5QtKPFjMsgKTqBmLYn1O48zQdjy EpvXvGWGuElBYsfZ12BzRAT0JB7/3skMUSMise/FO8YJjAKzkIyahWTULCSjZiFpWcDIsopR NLUguaA4KT3XSK84Mbe4NC9dLzk/dxMjOG08k97BuKrB4hCjAAejEg/vB4aCYCHWxLLiytxD jBIczEoivCcnAoV4UxIrq1KL8uOLSnNSiw8xJgN9OpFZSjQ5H5jS8kriDY1NzIwsjcwsjEzM zUkTVhLnPdhqHSgkkJ5YkpqdmlqQWgSzhYmDU6qBsW1p550YpfylVmqNi/0Pl2z7onfrbc29 42VMSZef8vKsNvKaP+3MtRNndjXGzl6e234pp/BXXWCXwi/m1nSGfUu4y0NsPutfSfk3I47j X/WBT17z1Az3/uxMqHo/weqgU/LfV5e036Tv5LomJLsr/nTLZXGdurj1TFLLlxsn1nmu+jbx Z7iBqBJLcUaioRZzUXEiALq/iGZfAwAA DLP-Filter: Pass X-MTR: 20000000000000000@CPGS X-CFilter-Loop: Reflected Sender: linux-samsung-soc-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-samsung-soc@vger.kernel.org X-Spam-Status: No, score=-7.5 required=5.0 tests=BAYES_00, RCVD_IN_DNSWL_HI, RP_MATCHES_RCVD, UNPARSEABLE_RELAY autolearn=unavailable version=3.3.1 X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on mail.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP This patch adds 4 levels of translation tables implementation for both HYP and stage2. Both symmetric and asymmetric configurations for page size and translation levels are are validated on Fast Models: 1) 4KB + 3 levels guest on 4KB + 4 levels host 2) 4KB + 4 levels guest on 4KB + 4 levels host 3) 64KB + 2 levels guest on 4KB + 4 levels host 4) 4KB + 3 levels guest on 64KB + 2 levels host 5) 4KB + 4 levels guest on 64KB + 2 levels host 6) 64KB + 2 levels guest on 64KB + 2 levels host Cc: Marc Zyngier Cc: Christoffer Dall Signed-off-by: Jungseok Lee Reviewed-by: Sungjinn Chung Acked-by: Kukjin Kim --- Please ignore the previous patch since it has a critical error on KVM_MMU_CACHE_MIN_PAGES. I'm really sorry about noise. My email client terminated unexpectedly, so some words are dropped :( --- arch/arm/include/asm/kvm_mmu.h | 10 +++++ arch/arm/kvm/arm.c | 8 ++++ arch/arm/kvm/mmu.c | 77 ++++++++++++++++++++++++++++++++------ arch/arm64/include/asm/kvm_arm.h | 12 ++++++ arch/arm64/include/asm/kvm_mmu.h | 12 ++++++ 5 files changed, 108 insertions(+), 11 deletions(-) diff --git a/arch/arm/include/asm/kvm_mmu.h b/arch/arm/include/asm/kvm_mmu.h index 5c7aa3c..d319ef6 100644 --- a/arch/arm/include/asm/kvm_mmu.h +++ b/arch/arm/include/asm/kvm_mmu.h @@ -37,6 +37,11 @@ */ #define TRAMPOLINE_VA UL(CONFIG_VECTORS_BASE) +/* + * KVM_MMU_CACHE_MIN_PAGES is the number of stage2 page table translation levels. + */ +#define KVM_MMU_CACHE_MIN_PAGES 2 + #ifndef __ASSEMBLY__ #include @@ -94,6 +99,11 @@ static inline void kvm_clean_pgd(pgd_t *pgd) clean_dcache_area(pgd, PTRS_PER_S2_PGD * sizeof(pgd_t)); } +static inline void kvm_clean_pmd(pmd_t *pmd) +{ + clean_dcache_area(pmd, PTRS_PER_PMD * sizeof(pmd_t)); +} + static inline void kvm_clean_pmd_entry(pmd_t *pmd) { clean_pmd_entry(pmd); diff --git a/arch/arm/kvm/arm.c b/arch/arm/kvm/arm.c index 9f19f2c..0785291 100644 --- a/arch/arm/kvm/arm.c +++ b/arch/arm/kvm/arm.c @@ -473,11 +473,19 @@ static int set_vttbr_baddr_mask(void) * 21 <= T0SZ <= 30 is valid under 3 level of translation tables * 30 <= T0SZ <= 39 is valid under 2 level of translation tables */ +#ifdef CONFIG_ARM64_3_LEVELS if (t0sz <= 20) { kvm_err("Cannot support %d-bit address space\n", 64 - t0sz); return -EINVAL; } vttbr_x = 37 - t0sz; +#else + if (t0sz <= 15) { + kvm_err("Cannot support %d-bit address space\n", 64 - t0sz); + return -EINVAL; + } + vttbr_x = 28 - t0sz; +#endif #endif vttbr_baddr_mask = (((1LLU << (48 - vttbr_x)) - 1) << (vttbr_x - 1)); #endif diff --git a/arch/arm/kvm/mmu.c b/arch/arm/kvm/mmu.c index 16f8049..6e2a0b0 100644 --- a/arch/arm/kvm/mmu.c +++ b/arch/arm/kvm/mmu.c @@ -390,13 +390,44 @@ static int create_hyp_pmd_mappings(pud_t *pud, unsigned long start, return 0; } +static int create_hyp_pud_mappings(pgd_t *pgd, unsigned long start, + unsigned long end, unsigned long pfn, + pgprot_t prot) +{ + pud_t *pud; + pmd_t *pmd; + unsigned long addr, next; + + addr = start; + do { + pud = pud_offset(pgd, addr); + + if (pud_none_or_clear_bad(pud)) { + pmd = pmd_alloc_one(NULL, addr); + if (!pmd) { + kvm_err("Cannot allocate Hyp pmd\n"); + return -ENOMEM; + } + pud_populate(NULL, pud, pmd); + get_page(virt_to_page(pud)); + kvm_flush_dcache_to_poc(pud, sizeof(*pud)); + } + + next = pud_addr_end(addr, end); + + create_hyp_pmd_mappings(pud, addr, next, pfn, prot); + pfn += (next - addr) >> PAGE_SHIFT; + } while (addr = next, addr != end); + + return 0; +} + static int __create_hyp_mappings(pgd_t *pgdp, unsigned long start, unsigned long end, unsigned long pfn, pgprot_t prot) { pgd_t *pgd; pud_t *pud; - pmd_t *pmd; unsigned long addr, next; int err = 0; @@ -405,22 +436,21 @@ static int __create_hyp_mappings(pgd_t *pgdp, end = PAGE_ALIGN(end); do { pgd = pgdp + pgd_index(addr); - pud = pud_offset(pgd, addr); - if (pud_none_or_clear_bad(pud)) { - pmd = pmd_alloc_one(NULL, addr); - if (!pmd) { - kvm_err("Cannot allocate Hyp pmd\n"); + if (pgd_none(*pgd)) { + pud = pud_alloc_one(NULL, addr); + if (!pud) { + kvm_err("Cannot allocate Hyp pud\n"); err = -ENOMEM; goto out; } - pud_populate(NULL, pud, pmd); - get_page(virt_to_page(pud)); - kvm_flush_dcache_to_poc(pud, sizeof(*pud)); + pgd_populate(NULL, pgd, pud); + get_page(virt_to_page(pgd)); + kvm_flush_dcache_to_poc(pgd, sizeof(*pgd)); } next = pgd_addr_end(addr, end); - err = create_hyp_pmd_mappings(pud, addr, next, pfn, prot); + err = create_hyp_pud_mappings(pgd, addr, next, pfn, prot); if (err) goto out; pfn += (next - addr) >> PAGE_SHIFT; @@ -565,6 +595,24 @@ void kvm_free_stage2_pgd(struct kvm *kvm) kvm->arch.pgd = NULL; } +static pud_t *stage2_get_pud(struct kvm *kvm, struct kvm_mmu_memory_cache *cache, + phys_addr_t addr) +{ + pgd_t *pgd; + pud_t *pud; + + pgd = kvm->arch.pgd + pgd_index(addr); + if (pgd_none(*pgd)) { + if (!cache) + return NULL; + pud = mmu_memory_cache_alloc(cache); + pgd_populate(NULL, pgd, pud); + get_page(virt_to_page(pgd)); + } + + return pud_offset(pgd, addr); +} + static pmd_t *stage2_get_pmd(struct kvm *kvm, struct kvm_mmu_memory_cache *cache, phys_addr_t addr) { @@ -616,9 +664,15 @@ static int stage2_set_pmd_huge(struct kvm *kvm, struct kvm_mmu_memory_cache static int stage2_set_pte(struct kvm *kvm, struct kvm_mmu_memory_cache *cache, phys_addr_t addr, const pte_t *new_pte, bool iomap) { + pud_t *pud; pmd_t *pmd; pte_t *pte, old_pte; + /* Create stage-2 page table mapping - Level 0 */ + pud = stage2_get_pud(kvm, cache, addr); + if (!pud) + return 0; + /* Create stage-2 page table mapping - Level 1 */ pmd = stage2_get_pmd(kvm, cache, addr); if (!pmd) { @@ -677,7 +731,8 @@ int kvm_phys_addr_ioremap(struct kvm *kvm, phys_addr_t guest_ipa, for (addr = guest_ipa; addr < end; addr += PAGE_SIZE) { pte_t pte = pfn_pte(pfn, PAGE_S2_DEVICE); - ret = mmu_topup_memory_cache(&cache, 2, 2); + ret = mmu_topup_memory_cache(&cache, KVM_MMU_CACHE_MIN_PAGES, + KVM_MMU_CACHE_MIN_PAGES); if (ret) goto out; spin_lock(&kvm->mmu_lock); diff --git a/arch/arm64/include/asm/kvm_arm.h b/arch/arm64/include/asm/kvm_arm.h index 8dbef70..ac796d0 100644 --- a/arch/arm64/include/asm/kvm_arm.h +++ b/arch/arm64/include/asm/kvm_arm.h @@ -114,6 +114,7 @@ #define VTCR_EL2_IRGN0_MASK (3 << 8) #define VTCR_EL2_IRGN0_WBWA (1 << 8) #define VTCR_EL2_SL0_MASK (3 << 6) +#define VTCR_EL2_SL0_LVL0 (2 << 6) #define VTCR_EL2_SL0_LVL1 (1 << 6) #define VTCR_EL2_T0SZ_MASK 0x3f #define VTCR_EL2_T0SZ(bits) (64 - (bits)) @@ -128,6 +129,7 @@ VTCR_EL2_ORGN0_WBWA | VTCR_EL2_IRGN0_WBWA | \ VTCR_EL2_SL0_LVL1) #else +#ifdef CONFIG_ARM64_3_LEVELS /* * Stage2 translation configuration: * 4kB pages (TG0 = 0) @@ -136,6 +138,16 @@ #define VTCR_EL2_FLAGS (VTCR_EL2_TG0_4K | VTCR_EL2_SH0_INNER | \ VTCR_EL2_ORGN0_WBWA | VTCR_EL2_IRGN0_WBWA | \ VTCR_EL2_SL0_LVL1) +#else +/* + * Stage2 translation configuration: + * 4kB pages (TG0 = 0) + * 4 level page tables (SL = 2) + */ +#define VTCR_EL2_FLAGS (VTCR_EL2_TG0_4K | VTCR_EL2_SH0_INNER | \ + VTCR_EL2_ORGN0_WBWA | VTCR_EL2_IRGN0_WBWA | \ + VTCR_EL2_SL0_LVL0) +#endif #endif #define VTTBR_VMID_SHIFT (48LLU) diff --git a/arch/arm64/include/asm/kvm_mmu.h b/arch/arm64/include/asm/kvm_mmu.h index 7d29847..778bf42 100644 --- a/arch/arm64/include/asm/kvm_mmu.h +++ b/arch/arm64/include/asm/kvm_mmu.h @@ -41,6 +41,17 @@ */ #define TRAMPOLINE_VA (HYP_PAGE_OFFSET_MASK & PAGE_MASK) +/* + * KVM_MMU_CACHE_MIN_PAGES is the number of stage2 page table translation levels. + */ +#ifdef CONFIG_ARM64_2_LEVELS +#define KVM_MMU_CACHE_MIN_PAGES 1 +#elif defined(CONFIG_ARM64_3_LEVELS) +#define KVM_MMU_CACHE_MIN_PAGES 2 +#else +#define KVM_MMU_CACHE_MIN_PAGES 3 +#endif + #ifdef __ASSEMBLY__ /* @@ -107,6 +118,7 @@ static inline bool kvm_is_write_fault(unsigned long esr) } static inline void kvm_clean_pgd(pgd_t *pgd) {} +static inline void kvm_clean_pmd(pmd_t *pmd) {} static inline void kvm_clean_pmd_entry(pmd_t *pmd) {} static inline void kvm_clean_pte(pte_t *pte) {} static inline void kvm_clean_pte_entry(pte_t *pte) {}