From patchwork Wed Jul 22 10:30:33 2015 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrey Ryabinin X-Patchwork-Id: 6842001 Return-Path: X-Original-To: patchwork-linux-arm@patchwork.kernel.org Delivered-To: patchwork-parsemail@patchwork2.web.kernel.org Received: from mail.kernel.org (mail.kernel.org [198.145.29.136]) by patchwork2.web.kernel.org (Postfix) with ESMTP id 55183C05AC for ; Wed, 22 Jul 2015 10:34:04 +0000 (UTC) Received: from mail.kernel.org (localhost [127.0.0.1]) by mail.kernel.org (Postfix) with ESMTP id 13610206B6 for ; Wed, 22 Jul 2015 10:34:03 +0000 (UTC) Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.9]) (using TLSv1.2 with cipher AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 0924A2065D for ; Wed, 22 Jul 2015 10:34:02 +0000 (UTC) Received: from localhost ([127.0.0.1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.80.1 #2 (Red Hat Linux)) id 1ZHrJj-0000zj-9m; Wed, 22 Jul 2015 10:32:03 +0000 Received: from mailout1.w1.samsung.com ([210.118.77.11]) by bombadil.infradead.org with esmtps (Exim 4.80.1 #2 (Red Hat Linux)) id 1ZHrIz-0000gN-MN for linux-arm-kernel@lists.infradead.org; Wed, 22 Jul 2015 10:31:20 +0000 Received: from eucpsbgm1.samsung.com (unknown [203.254.199.244]) by mailout1.w1.samsung.com (Oracle Communications Messaging Server 7.0.5.31.0 64bit (built May 5 2014)) with ESMTP id <0NRV00ARHX7CEP50@mailout1.w1.samsung.com> for linux-arm-kernel@lists.infradead.org; Wed, 22 Jul 2015 11:30:48 +0100 (BST) X-AuditID: cbfec7f4-f79c56d0000012ee-63-55af70d771df Received: from eusync4.samsung.com ( [203.254.199.214]) by eucpsbgm1.samsung.com (EUCPMTA) with SMTP id 79.A7.04846.7D07FA55; Wed, 22 Jul 2015 11:30:47 +0100 (BST) Received: from localhost.localdomain ([106.109.129.143]) by eusync4.samsung.com (Oracle Communications Messaging Server 7.0.5.31.0 64bit (built May 5 2014)) with ESMTPA id <0NRV00ML7X769090@eusync4.samsung.com>; Wed, 22 Jul 2015 11:30:47 +0100 (BST) From: Andrey Ryabinin To: Catalin Marinas , Will Deacon , linux-arm-kernel@lists.infradead.org Subject: [PATCH v3 1/5] mm: kasan: introduce generic kasan_populate_zero_shadow() Date: Wed, 22 Jul 2015 13:30:33 +0300 Message-id: <1437561037-31995-2-git-send-email-a.ryabinin@samsung.com> X-Mailer: git-send-email 2.4.5 In-reply-to: <1437561037-31995-1-git-send-email-a.ryabinin@samsung.com> References: <1437561037-31995-1-git-send-email-a.ryabinin@samsung.com> X-Brightmail-Tracker: H4sIAAAAAAAAA+NgFnrJLMWRmVeSWpSXmKPExsVy+t/xa7rXC9aHGkx/YGmx7dcjNos569ew WfyddIzd4v2yHkaL3ZeeMVtMeNjGbtH+cS+zxbSN4hZT/ixnstj0+BqrxeVdc9gs7q35z2px 6cACJovNm6YyW7z8eILF4seGx6wOAh5r5q1h9Pj9axKjx+W+XiaPBZtKPTat6mTz2PRpErvH nWt72DzenTvH7nFixm8Wj81L6j3e77vK5tG3ZRWjx+dNch4nWr6wBvBFcdmkpOZklqUW6dsl cGU0nFnLUrDFtOLXtbtsDYyntbsYOTgkBEwkrt+L6GLkBDLFJC7cW8/WxcjFISSwlFFi7tVe FginiUni4uVTjCBVbAJ6Ev9mbWcDsUUEKiSuXp3ACFLELHCGWeL2pBdMIAlhgWCJ168OgTWw CKhKrLxwgxFkG6+Am0TzLkOIxXISS5trQSo4Bdwlzp/dzwxiCwFV/P/0gWkCI+8CRoZVjKKp pckFxUnpuYZ6xYm5xaV56XrJ+bmbGCER8GUH4+JjVocYBTgYlXh4JxxdFyrEmlhWXJl7iFGC g1lJhPdF1vpQId6UxMqq1KL8+KLSnNTiQ4zSHCxK4rxzd70PERJITyxJzU5NLUgtgskycXBK NTBO5FeNiLPv8e+/wRR//HvL7KbsI5y6GT1XbkZcijVVPvk/rdd0/cleB70VVzf2HFzW3xI+ 5b/95stFey9KzsifX7Tp7HRHbuurJbXbSqbzh/sd/3CgMEC2bmt4vohf3zUftkfNDdOPe2mY Nsdr58ZtuJMba/u3/lpLVr2qp+oFAaX5PJk7lyixFGckGmoxFxUnAgBmwb3LfAIAAA== X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20150722_033117_928541_4F0CE7C4 X-CRM114-Status: GOOD ( 21.44 ) X-Spam-Score: -8.1 (--------) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Arnd Bergmann , linux-mm@kvack.org, Andrey Ryabinin , Linus Walleij , x86@kernel.org, linux-kernel@vger.kernel.org, David Keitel , Ingo Molnar , Alexander Potapenko , "H. Peter Anvin" , Andrew Morton , Thomas Gleixner , Dmitry Vyukov MIME-Version: 1.0 Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+patchwork-linux-arm=patchwork.kernel.org@lists.infradead.org X-Spam-Status: No, score=-5.4 required=5.0 tests=BAYES_00, RCVD_IN_DNSWL_MED, RP_MATCHES_RCVD, UNPARSEABLE_RELAY autolearn=unavailable version=3.3.1 X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on mail.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP Introduce generic kasan_populate_zero_shadow(start, end). This function maps kasan_zero_page to the [start, end] addresses. In follow on patches it will be used for ARMv8 (and maybe other architectures) and will replace x86_64 specific populate_zero_shadow(). Signed-off-by: Andrey Ryabinin --- arch/x86/mm/kasan_init_64.c | 8 +-- include/linux/kasan.h | 8 +++ mm/kasan/Makefile | 2 +- mm/kasan/kasan_init.c | 142 ++++++++++++++++++++++++++++++++++++++++++++ 4 files changed, 155 insertions(+), 5 deletions(-) create mode 100644 mm/kasan/kasan_init.c diff --git a/arch/x86/mm/kasan_init_64.c b/arch/x86/mm/kasan_init_64.c index e1840f3..2390dba 100644 --- a/arch/x86/mm/kasan_init_64.c +++ b/arch/x86/mm/kasan_init_64.c @@ -12,9 +12,9 @@ extern pgd_t early_level4_pgt[PTRS_PER_PGD]; extern struct range pfn_mapped[E820_X_MAX]; -static pud_t kasan_zero_pud[PTRS_PER_PUD] __page_aligned_bss; -static pmd_t kasan_zero_pmd[PTRS_PER_PMD] __page_aligned_bss; -static pte_t kasan_zero_pte[PTRS_PER_PTE] __page_aligned_bss; +pud_t kasan_zero_pud[PTRS_PER_PUD] __page_aligned_bss; +pmd_t kasan_zero_pmd[PTRS_PER_PMD] __page_aligned_bss; +pte_t kasan_zero_pte[PTRS_PER_PTE] __page_aligned_bss; /* * This page used as early shadow. We don't use empty_zero_page @@ -24,7 +24,7 @@ static pte_t kasan_zero_pte[PTRS_PER_PTE] __page_aligned_bss; * that allowed to access, but not instrumented by kasan * (vmalloc/vmemmap ...). */ -static unsigned char kasan_zero_page[PAGE_SIZE] __page_aligned_bss; +unsigned char kasan_zero_page[PAGE_SIZE] __page_aligned_bss; static int __init map_range(struct range *range) { diff --git a/include/linux/kasan.h b/include/linux/kasan.h index 5486d77..5ef3925 100644 --- a/include/linux/kasan.h +++ b/include/linux/kasan.h @@ -13,8 +13,16 @@ struct vm_struct; #define KASAN_SHADOW_OFFSET _AC(CONFIG_KASAN_SHADOW_OFFSET, UL) #include +#include #include +extern unsigned char kasan_zero_page[PAGE_SIZE]; +extern pte_t kasan_zero_pte[PTRS_PER_PTE]; +extern pmd_t kasan_zero_pmd[PTRS_PER_PMD]; +extern pud_t kasan_zero_pud[PTRS_PER_PUD]; + +void kasan_populate_zero_shadow(const void *start, const void *end); + static inline void *kasan_mem_to_shadow(const void *addr) { return (void *)((unsigned long)addr >> KASAN_SHADOW_SCALE_SHIFT) diff --git a/mm/kasan/Makefile b/mm/kasan/Makefile index bd837b8..6471014 100644 --- a/mm/kasan/Makefile +++ b/mm/kasan/Makefile @@ -5,4 +5,4 @@ CFLAGS_REMOVE_kasan.o = -pg # see: https://gcc.gnu.org/bugzilla/show_bug.cgi?id=63533 CFLAGS_kasan.o := $(call cc-option, -fno-conserve-stack -fno-stack-protector) -obj-y := kasan.o report.o +obj-y := kasan.o report.o kasan_init.o diff --git a/mm/kasan/kasan_init.c b/mm/kasan/kasan_init.c new file mode 100644 index 0000000..37fb46a --- /dev/null +++ b/mm/kasan/kasan_init.c @@ -0,0 +1,142 @@ +#include +#include +#include +#include +#include +#include + +#include +#include + +static __init void *early_alloc(size_t size, int node) +{ + return memblock_virt_alloc_try_nid(size, size, __pa(MAX_DMA_ADDRESS), + BOOTMEM_ALLOC_ACCESSIBLE, node); +} + +static int __init zero_pte_populate(pmd_t *pmd, unsigned long addr, + unsigned long end) +{ + pte_t *pte = pte_offset_kernel(pmd, addr); + pte_t zero_pte; + + zero_pte = pfn_pte(PFN_DOWN(__pa(kasan_zero_page)), PAGE_KERNEL); + zero_pte = pte_wrprotect(zero_pte); + + while (addr + PAGE_SIZE <= end) { + set_pte_at(&init_mm, addr, pte, zero_pte); + addr += PAGE_SIZE; + pte = pte_offset_kernel(pmd, addr); + } + return 0; +} + +static int __init zero_pmd_populate(pud_t *pud, unsigned long addr, + unsigned long end) +{ + int ret = 0; + pmd_t *pmd = pmd_offset(pud, addr); + unsigned long next; + + do { + next = pmd_addr_end(addr, end); + + if (IS_ALIGNED(addr, PMD_SIZE) && end - addr >= PMD_SIZE) { + pmd_populate_kernel(&init_mm, pmd, kasan_zero_pte); + continue; + } + + if (pmd_none(*pmd)) { + void *p = early_alloc(PAGE_SIZE, NUMA_NO_NODE); + if (!p) + return -ENOMEM; + pmd_populate_kernel(&init_mm, pmd, p); + } + zero_pte_populate(pmd, addr, pmd_addr_end(addr, end)); + } while (pmd++, addr = next, addr != end); + + return ret; +} + +static int __init zero_pud_populate(pgd_t *pgd, unsigned long addr, + unsigned long end) +{ + int ret = 0; + pud_t *pud = pud_offset(pgd, addr); + unsigned long next; + + do { + next = pud_addr_end(addr, end); + if (IS_ALIGNED(addr, PUD_SIZE) && end - addr >= PUD_SIZE) { + pmd_t *pmd; + + pud_populate(&init_mm, pud, kasan_zero_pmd); + pmd = pmd_offset(pud, addr); + pmd_populate_kernel(&init_mm, pmd, kasan_zero_pte); + continue; + } + + if (pud_none(*pud)) { + void *p = early_alloc(PAGE_SIZE, NUMA_NO_NODE); + if (!p) + return -ENOMEM; + pud_populate(&init_mm, pud, p); + } + zero_pmd_populate(pud, addr, pud_addr_end(addr, end)); + } while (pud++, addr = next, addr != end); + + return ret; +} + +static int __init zero_pgd_populate(unsigned long addr, unsigned long end) +{ + int ret = 0; + pgd_t *pgd = pgd_offset_k(addr); + unsigned long next; + + do { + next = pgd_addr_end(addr, end); + + if (IS_ALIGNED(addr, PGDIR_SIZE) && end - addr >= PGDIR_SIZE) { + pud_t *pud; + pmd_t *pmd; + + /* + * kasan_zero_pud should be populated with pmds + * at this moment. + * [pud,pmd]_populate*() bellow needed only for + * 3,2 - level page tables where we don't have + * puds,pmds, so pgd_populate(), pud_populate() + * is noops. + */ + pgd_populate(&init_mm, pgd, kasan_zero_pud); + pud = pud_offset(pgd, addr); + pud_populate(&init_mm, pud, kasan_zero_pmd); + pmd = pmd_offset(pud, addr); + pmd_populate_kernel(&init_mm, pmd, kasan_zero_pte); + continue; + } + + if (pgd_none(*pgd)) { + void *p = early_alloc(PAGE_SIZE, NUMA_NO_NODE); + if (!p) + return -ENOMEM; + pgd_populate(&init_mm, pgd, p); + } + zero_pud_populate(pgd, addr, next); + } while (pgd++, addr = next, addr != end); + + return ret; +} + +/** + * kasan_populate_zero_shadow - populate shadow memory region with + * kasan_zero_page + * @start - start of the memory range to populate + * @end - end of the memory range to populate + */ +void __init kasan_populate_zero_shadow(const void *start, const void *end) +{ + if (zero_pgd_populate((unsigned long)start, (unsigned long)end)) + panic("kasan: unable to map zero shadow!"); +}