From patchwork Fri Apr 21 00:50:01 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Matlack X-Patchwork-Id: 9691539 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork.web.codeaurora.org (Postfix) with ESMTP id 2D7AB6038D for ; Fri, 21 Apr 2017 00:51:00 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 1E7672847B for ; Fri, 21 Apr 2017 00:51:00 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 1279A28480; Fri, 21 Apr 2017 00:51:00 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-6.5 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID, DKIM_VALID_AU, RCVD_IN_DNSWL_HI, RCVD_IN_SORBS_SPAM autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 966252847B for ; Fri, 21 Apr 2017 00:50:59 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1034281AbdDUAu5 (ORCPT ); Thu, 20 Apr 2017 20:50:57 -0400 Received: from mail-io0-f177.google.com ([209.85.223.177]:33446 "EHLO mail-io0-f177.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1034194AbdDUAui (ORCPT ); Thu, 20 Apr 2017 20:50:38 -0400 Received: by mail-io0-f177.google.com with SMTP id k87so95057801ioi.0 for ; Thu, 20 Apr 2017 17:50:38 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=P3zLZp5PofcOMN0f6SUwgpwTRA8Pt51UJHlxhC9EWSY=; b=di0Uhh5s9nY9HYZrCuBzZA1xvpQbrwVEIKDndbe/VANKKTPhHsXmLWfXhz3bhQioW9 CZ5Q31XY3Sw3HcfJyPzDy0JGACI+hOYJekvtkeGqzB8sYG1Hi/+zVawsT2jR/HbhOdE8 0fubqJ0HQ/ZqDRBwQh/isM1i5GjifXF1zCiqil8LdZZbRTu+1xxrd+FZOJwV+oWJVGiO tymmYerK2Re7EioZjmDE9mj39q8mlbLpFUlIoPQtSoG2gnGYlovRGJZebW/p1Es4xxWu NXgWMDNmBOAPIDx/UiGhafKdRLAInQrN3HgkzabMBLmdf2jLyK1cgJdIBb+sNd381XQm ZPzA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=P3zLZp5PofcOMN0f6SUwgpwTRA8Pt51UJHlxhC9EWSY=; b=YwZbkn1X5biu9UjbIexJt9s5dfdw+J6strRBtEMQWdJCyp1AN1eScaX3rUmzdMNqWV V+RAREi0iAe5ntxTzaCQF3LaBXolxG0FPkqrmTQCq9lt+VcJJbzavvodZxD1Mr4k9YjH zJ18vSUrlfPh7UQ/BlvcaX9zPAADc2Vz060js9q4KG0VXuorTfN/WiRVd9yeZkSW/5fE wpyDqsw8Y7l/e6TDvAxlrHQxstY/RXF10WiIN0WQBYb9/R+lLqM68MnEjK8/V55EVNiQ o/v+tvG/v+KGPrON4tdulXS+Dg6JmrnPtLz0W+L0NjI40yvGUcosKI63Yww/v6MtmEY2 9erw== X-Gm-Message-State: AN3rC/6T9i3PMiOW+srxV/1LsqtvoItoZK2w51CGQn8kAnfuEvLkUUst S4Tcjs4VH9tKEgFB X-Received: by 10.84.237.1 with SMTP id s1mr12392163plk.157.1492735837411; Thu, 20 Apr 2017 17:50:37 -0700 (PDT) Received: from dmatlack.sea.corp.google.com ([100.100.206.82]) by smtp.gmail.com with ESMTPSA id e13sm12466486pfb.30.2017.04.20.17.50.36 (version=TLS1_2 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Thu, 20 Apr 2017 17:50:36 -0700 (PDT) From: David Matlack To: kvm@vger.kernel.org Cc: Peter Feiner , David Matlack Subject: [kvm-unit-tests PATCH 29/32] lib: x86: page table utilities Date: Thu, 20 Apr 2017 17:50:01 -0700 Message-Id: <20170421005004.137260-30-dmatlack@google.com> X-Mailer: git-send-email 2.12.2.816.g2cccc81164-goog In-Reply-To: <20170421005004.137260-1-dmatlack@google.com> References: <20170421005004.137260-1-dmatlack@google.com> Sender: kvm-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP From: Peter Feiner Convenience functions for manipulating and querying page tables. Signed-off-by: Peter Feiner Signed-off-by: David Matlack --- lib/x86/vm.c | 99 +++++++++++++++++++++++++++++++++++++++++++++++++----------- lib/x86/vm.h | 27 +++++++++++++++++ 2 files changed, 108 insertions(+), 18 deletions(-) diff --git a/lib/x86/vm.c b/lib/x86/vm.c index 438ab9c70ffa..d52bb53fcf5e 100644 --- a/lib/x86/vm.c +++ b/lib/x86/vm.c @@ -139,23 +139,62 @@ unsigned long *install_pte(unsigned long *cr3, return &pt[offset]; } +/* + * Finds last PTE in the mapping of @virt that's at or above @lowest_level. The + * returned PTE isn't necessarily present, but its parent is. + */ +struct pte_search find_pte_level(unsigned long *cr3, void *virt, + int lowest_level) +{ + unsigned long *pt = cr3, pte; + unsigned offset; + unsigned long shift; + struct pte_search r; + + assert(lowest_level >= 1 && lowest_level <= PAGE_LEVEL); + + for (r.level = PAGE_LEVEL;; --r.level) { + shift = (r.level - 1) * PGDIR_WIDTH + 12; + offset = ((unsigned long)virt >> shift) & PGDIR_MASK; + r.pte = &pt[offset]; + pte = *r.pte; + + if (!(pte & PT_PRESENT_MASK)) + return r; + + if ((r.level == 2 || r.level == 3) && (pte & PT_PAGE_SIZE_MASK)) + return r; + + if (r.level == lowest_level) + return r; + + pt = phys_to_virt(pte & 0xffffffffff000ull); + } +} + +/* + * Returns the leaf PTE in the mapping of @virt (i.e., 4K PTE or a present huge + * PTE). Returns NULL if no leaf PTE exists. + */ unsigned long *get_pte(unsigned long *cr3, void *virt) { - int level; - unsigned long *pt = cr3, pte; - unsigned offset; + struct pte_search search; - for (level = PAGE_LEVEL; level > 1; --level) { - offset = ((unsigned long)virt >> (((level-1) * PGDIR_WIDTH) + 12)) & PGDIR_MASK; - pte = pt[offset]; - if (!(pte & PT_PRESENT_MASK)) - return NULL; - if (level == 2 && (pte & PT_PAGE_SIZE_MASK)) - return &pt[offset]; - pt = phys_to_virt(pte & PT_ADDR_MASK); - } - offset = ((unsigned long)virt >> (((level-1) * PGDIR_WIDTH) + 12)) & PGDIR_MASK; - return &pt[offset]; + search = find_pte_level(cr3, virt, 1); + return found_leaf_pte(search) ? search.pte : NULL; +} + +/* + * Returns the PTE in the mapping of @virt at the given level @pte_level. + * Returns NULL if the PT at @pte_level isn't present (i.e., the mapping at + * @pte_level - 1 isn't present). + */ +unsigned long *get_pte_level(unsigned long *cr3, void *virt, int pte_level) +{ + struct pte_search search; + + search = find_pte_level(cr3, virt, pte_level); + return search.level == pte_level ? search.pte : NULL; } unsigned long *install_large_page(unsigned long *cr3, @@ -173,6 +212,33 @@ unsigned long *install_page(unsigned long *cr3, return install_pte(cr3, 1, virt, phys | PT_PRESENT_MASK | PT_WRITABLE_MASK | PT_USER_MASK, 0); } +void install_pages(unsigned long *cr3, unsigned long phys, unsigned long len, + void *virt) +{ + unsigned long max = (u64)len + (u64)phys; + assert(phys % PAGE_SIZE == 0); + assert((unsigned long) virt % PAGE_SIZE == 0); + assert(len % PAGE_SIZE == 0); + + while (phys + PAGE_SIZE <= max) { + install_page(cr3, phys, virt); + phys += PAGE_SIZE; + virt = (char *) virt + PAGE_SIZE; + } +} + +bool any_present_pages(unsigned long *cr3, void *virt, unsigned long len) +{ + unsigned long max = (unsigned long) virt + len; + unsigned long curr; + + for (curr = (unsigned long) virt; curr < max; curr += PAGE_SIZE) { + unsigned long *ptep = get_pte(cr3, (void *) curr); + if (ptep && (*ptep & PT_PRESENT_MASK)) + return true; + } + return false; +} static void setup_mmu_range(unsigned long *cr3, unsigned long start, unsigned long len) @@ -184,10 +250,7 @@ static void setup_mmu_range(unsigned long *cr3, unsigned long start, install_large_page(cr3, phys, (void *)(ulong)phys); phys += LARGE_PAGE_SIZE; } - while (phys + PAGE_SIZE <= max) { - install_page(cr3, phys, (void *)(ulong)phys); - phys += PAGE_SIZE; - } + install_pages(cr3, phys, max - phys, (void *)(ulong)phys); } static void setup_mmu(unsigned long len) diff --git a/lib/x86/vm.h b/lib/x86/vm.h index 64e4b566333f..3522ba8b99b0 100644 --- a/lib/x86/vm.h +++ b/lib/x86/vm.h @@ -14,7 +14,27 @@ void *alloc_vpage(void); void *alloc_vpages(ulong nr); uint64_t virt_to_phys_cr3(void *mem); +struct pte_search { + int level; + unsigned long *pte; +}; + +static inline bool found_huge_pte(struct pte_search search) +{ + return (search.level == 2 || search.level == 3) && + (*search.pte & PT_PRESENT_MASK) && + (*search.pte & PT_PAGE_SIZE_MASK); +} + +static inline bool found_leaf_pte(struct pte_search search) +{ + return search.level == 1 || found_huge_pte(search); +} + +struct pte_search find_pte_level(unsigned long *cr3, void *virt, + int lowest_level); unsigned long *get_pte(unsigned long *cr3, void *virt); +unsigned long *get_pte_level(unsigned long *cr3, void *virt, int pte_level); unsigned long *install_pte(unsigned long *cr3, int pte_level, void *virt, @@ -28,5 +48,12 @@ void free_page(void *page); unsigned long *install_large_page(unsigned long *cr3,unsigned long phys, void *virt); unsigned long *install_page(unsigned long *cr3, unsigned long phys, void *virt); +void install_pages(unsigned long *cr3, unsigned long phys, unsigned long len, + void *virt); +bool any_present_pages(unsigned long *cr3, void *virt, unsigned long len); +static inline void *current_page_table(void) +{ + return phys_to_virt(read_cr3()); +} #endif