From patchwork Fri Sep 13 16:32:38 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jia He X-Patchwork-Id: 11145013 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 515CE14DB for ; Fri, 13 Sep 2019 16:33:15 +0000 (UTC) Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 3029020693 for ; Fri, 13 Sep 2019 16:33:15 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=lists.infradead.org header.i=@lists.infradead.org header.b="IV2WwTMd" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 3029020693 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=arm.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+patchwork-linux-arm=patchwork.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20170209; h=Sender: Content-Transfer-Encoding:Content-Type:MIME-Version:Cc:List-Subscribe: List-Help:List-Post:List-Archive:List-Unsubscribe:List-Id:References: In-Reply-To:Message-Id:Date:Subject:To:From:Reply-To:Content-ID: Content-Description:Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc :Resent-Message-ID:List-Owner; bh=FXZ9NuqHTyN27QsybV14Gh+NuXjhLKpeGpqo6hVKdsM=; b=IV2WwTMdjqtXUHaXKmHxWsZ3UO QbSZYNIGUtMhxGmHzfl9jNiCCLhfNY1GR45V7/ZTErGsO7ToYDy1r05YcbQ/w6DCo5KVYQN3vingG +XKdA5csqfMpyGS7Nk/Mp30851XWxOa/SsEvAqAUxWD5ClGAg8Lu/ybFbR5fLHQxHC2E3Z80NiHJP DuHlIwS5ZVMf8l5Gn34dFMEs+1hq/9LEDiXEKaY55TlI00pxQ+AJI++23X1tEGMnJ7Tty6TgeuSfN vSfPf5fZJt3qwNaDIceV1Iq9dr6nKhBDLhPp47P4qz2Vf/vjBYaYjy/WhbpbgmqQew4a2iA/XzNjJ Uhbg7P7Q==; Received: from localhost ([127.0.0.1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.92.2 #3 (Red Hat Linux)) id 1i8oVR-0005J3-Tw; Fri, 13 Sep 2019 16:33:10 +0000 Received: from foss.arm.com ([217.140.110.172]) by bombadil.infradead.org with esmtp (Exim 4.92.2 #3 (Red Hat Linux)) id 1i8oVF-00057T-PT for linux-arm-kernel@lists.infradead.org; Fri, 13 Sep 2019 16:32:59 +0000 Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id DD7C91570; Fri, 13 Sep 2019 09:32:56 -0700 (PDT) Received: from localhost.localdomain (entos-thunderx2-02.shanghai.arm.com [10.169.40.54]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPA id C82FE3F67D; Fri, 13 Sep 2019 09:32:52 -0700 (PDT) From: Jia He To: Catalin Marinas , Will Deacon , Mark Rutland , James Morse , Marc Zyngier , Matthew Wilcox , "Kirill A. Shutemov" , linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, linux-mm@kvack.org Subject: [PATCH v3 1/2] arm64: mm: implement arch_faults_on_old_pte() on arm64 Date: Sat, 14 Sep 2019 00:32:38 +0800 Message-Id: <20190913163239.125108-2-justin.he@arm.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20190913163239.125108-1-justin.he@arm.com> References: <20190913163239.125108-1-justin.he@arm.com> X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20190913_093257_870128_7D629AC9 X-CRM114-Status: GOOD ( 10.91 ) X-Spam-Score: 0.0 (/) X-Spam-Report: SpamAssassin version 3.4.2 on bombadil.infradead.org summary: Content analysis details: (0.0 points) pts rule name description ---- ---------------------- -------------------------------------------------- 0.0 SPF_HELO_NONE SPF: HELO does not publish an SPF Record -0.0 SPF_PASS SPF: sender matches SPF record X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Ralph Campbell , Jia He , Andrew Morton , Anshuman Khandual , Jun Yao , =?utf-8?b?SsOpcsO0bWUgR2xpc3Nl?= , Punit Agrawal , hejianet@gmail.com, Thomas Gleixner , Robin Murphy , Alex Van Brunt MIME-Version: 1.0 Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+patchwork-linux-arm=patchwork.kernel.org@lists.infradead.org On arm64 without hardware Access Flag, copying fromuser will fail because the pte is old and cannot be marked young. So we always end up with zeroed page after fork() + CoW for pfn mappings. we don't always have a hardware-managed access flag on arm64. Hence implement arch_faults_on_old_pte on arm64 to indicate that it might cause page fault when accessing old pte. Signed-off-by: Jia He --- arch/arm64/include/asm/pgtable.h | 12 ++++++++++++ 1 file changed, 12 insertions(+) diff --git a/arch/arm64/include/asm/pgtable.h b/arch/arm64/include/asm/pgtable.h index e09760ece844..b41399d758df 100644 --- a/arch/arm64/include/asm/pgtable.h +++ b/arch/arm64/include/asm/pgtable.h @@ -868,6 +868,18 @@ static inline void update_mmu_cache(struct vm_area_struct *vma, #define phys_to_ttbr(addr) (addr) #endif +/* + * On arm64 without hardware Access Flag, copying fromuser will fail because + * the pte is old and cannot be marked young. So we always end up with zeroed + * page after fork() + CoW for pfn mappings. we don't always have a + * hardware-managed access flag on arm64. + */ +static inline bool arch_faults_on_old_pte(void) +{ + return true; +} +#define arch_faults_on_old_pte arch_faults_on_old_pte + #endif /* !__ASSEMBLY__ */ #endif /* __ASM_PGTABLE_H */ From patchwork Fri Sep 13 16:32:39 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jia He X-Patchwork-Id: 11145015 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 4D01F14DB for ; Fri, 13 Sep 2019 16:33:29 +0000 (UTC) Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 2B52C20693 for ; Fri, 13 Sep 2019 16:33:29 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=lists.infradead.org header.i=@lists.infradead.org header.b="G40jXRuA" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 2B52C20693 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=arm.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+patchwork-linux-arm=patchwork.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20170209; h=Sender: Content-Transfer-Encoding:Content-Type:MIME-Version:Cc:List-Subscribe: List-Help:List-Post:List-Archive:List-Unsubscribe:List-Id:References: In-Reply-To:Message-Id:Date:Subject:To:From:Reply-To:Content-ID: Content-Description:Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc :Resent-Message-ID:List-Owner; bh=lW1jaeNiKcxj99JZMxdJb2+aNNThNpNVdOZ6BjJFFqs=; b=G40jXRuAa8fbwdaMYaLWGwYfI7 bzJukgSADK0bwUCizsz9O0MrLiiCGcwt+kTgHegg5Gg+WIvZ226IoHKEs2ibKXxUnWMAtRYIcibfa RAhufxkCtiHyntwfXeorMk7kDVOkwwz51w22eMNOidJz+9gXKxxpIERvxuRo0mYGpTkqruqBviBv0 X8Go1oQiLDM/NHmzQUli7ncSSp4qa3jd5wHKqGKQbeuUW6yjoJ6FrrVJbeGBWEUb56HEO1hG6FUW/ hxrkUHvPQiw67lazeZcE4+T+m4TLA6OxEJkVFmMFBJ2m5fR4JHmt7TI2bStkuFzNRpaJEiW4d+Nbg ac43mlkg==; Received: from localhost ([127.0.0.1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.92.2 #3 (Red Hat Linux)) id 1i8oVk-0005Xj-0e; Fri, 13 Sep 2019 16:33:28 +0000 Received: from foss.arm.com ([217.140.110.172]) by bombadil.infradead.org with esmtp (Exim 4.92.2 #3 (Red Hat Linux)) id 1i8oVJ-0005Bb-Op for linux-arm-kernel@lists.infradead.org; Fri, 13 Sep 2019 16:33:03 +0000 Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 5F46315A2; Fri, 13 Sep 2019 09:33:01 -0700 (PDT) Received: from localhost.localdomain (entos-thunderx2-02.shanghai.arm.com [10.169.40.54]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPA id 400293F67D; Fri, 13 Sep 2019 09:32:57 -0700 (PDT) From: Jia He To: Catalin Marinas , Will Deacon , Mark Rutland , James Morse , Marc Zyngier , Matthew Wilcox , "Kirill A. Shutemov" , linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, linux-mm@kvack.org Subject: [PATCH v3 2/2] mm: fix double page fault on arm64 if PTE_AF is cleared Date: Sat, 14 Sep 2019 00:32:39 +0800 Message-Id: <20190913163239.125108-3-justin.he@arm.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20190913163239.125108-1-justin.he@arm.com> References: <20190913163239.125108-1-justin.he@arm.com> X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20190913_093301_980622_79B2C194 X-CRM114-Status: GOOD ( 15.60 ) X-Spam-Score: 0.0 (/) X-Spam-Report: SpamAssassin version 3.4.2 on bombadil.infradead.org summary: Content analysis details: (0.0 points) pts rule name description ---- ---------------------- -------------------------------------------------- 0.0 SPF_HELO_NONE SPF: HELO does not publish an SPF Record -0.0 SPF_PASS SPF: sender matches SPF record X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Ralph Campbell , Jia He , Andrew Morton , Anshuman Khandual , Jun Yao , =?utf-8?b?SsOpcsO0bWUgR2xpc3Nl?= , Punit Agrawal , hejianet@gmail.com, Thomas Gleixner , Robin Murphy , Alex Van Brunt MIME-Version: 1.0 Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+patchwork-linux-arm=patchwork.kernel.org@lists.infradead.org When we tested pmdk unit test [1] vmmalloc_fork TEST1 in arm64 guest, there will be a double page fault in __copy_from_user_inatomic of cow_user_page. Below call trace is from arm64 do_page_fault for debugging purpose [ 110.016195] Call trace: [ 110.016826] do_page_fault+0x5a4/0x690 [ 110.017812] do_mem_abort+0x50/0xb0 [ 110.018726] el1_da+0x20/0xc4 [ 110.019492] __arch_copy_from_user+0x180/0x280 [ 110.020646] do_wp_page+0xb0/0x860 [ 110.021517] __handle_mm_fault+0x994/0x1338 [ 110.022606] handle_mm_fault+0xe8/0x180 [ 110.023584] do_page_fault+0x240/0x690 [ 110.024535] do_mem_abort+0x50/0xb0 [ 110.025423] el0_da+0x20/0x24 The pte info before __copy_from_user_inatomic is (PTE_AF is cleared): [ffff9b007000] pgd=000000023d4f8003, pud=000000023da9b003, pmd=000000023d4b3003, pte=360000298607bd3 As told by Catalin: "On arm64 without hardware Access Flag, copying from user will fail because the pte is old and cannot be marked young. So we always end up with zeroed page after fork() + CoW for pfn mappings. we don't always have a hardware-managed access flag on arm64." This patch fix it by calling pte_mkyoung. Also, the parameter is changed because vmf should be passed to cow_user_page() [1] https://github.com/pmem/pmdk/tree/master/src/test/vmmalloc_fork Reported-by: Yibo Cai Signed-off-by: Jia He --- mm/memory.c | 30 +++++++++++++++++++++++++----- 1 file changed, 25 insertions(+), 5 deletions(-) diff --git a/mm/memory.c b/mm/memory.c index e2bb51b6242e..a64af6495f71 100644 --- a/mm/memory.c +++ b/mm/memory.c @@ -118,6 +118,13 @@ int randomize_va_space __read_mostly = 2; #endif +#ifndef arch_faults_on_old_pte +static inline bool arch_faults_on_old_pte(void) +{ + return false; +} +#endif + static int __init disable_randmaps(char *s) { randomize_va_space = 0; @@ -2140,7 +2147,8 @@ static inline int pte_unmap_same(struct mm_struct *mm, pmd_t *pmd, return same; } -static inline void cow_user_page(struct page *dst, struct page *src, unsigned long va, struct vm_area_struct *vma) +static inline void cow_user_page(struct page *dst, struct page *src, + struct vm_fault *vmf) { debug_dma_assert_idle(src); @@ -2152,20 +2160,32 @@ static inline void cow_user_page(struct page *dst, struct page *src, unsigned lo */ if (unlikely(!src)) { void *kaddr = kmap_atomic(dst); - void __user *uaddr = (void __user *)(va & PAGE_MASK); + void __user *uaddr = (void __user *)(vmf->address & PAGE_MASK); + pte_t entry; /* * This really shouldn't fail, because the page is there * in the page tables. But it might just be unreadable, * in which case we just give up and fill the result with - * zeroes. + * zeroes. If PTE_AF is cleared on arm64, it might + * cause double page fault. So makes pte young here */ + if (arch_faults_on_old_pte() && !pte_young(vmf->orig_pte)) { + spin_lock(vmf->ptl); + entry = pte_mkyoung(vmf->orig_pte); + if (ptep_set_access_flags(vmf->vma, vmf->address, + vmf->pte, entry, 0)) + update_mmu_cache(vmf->vma, vmf->address, + vmf->pte); + spin_unlock(vmf->ptl); + } + if (__copy_from_user_inatomic(kaddr, uaddr, PAGE_SIZE)) clear_page(kaddr); kunmap_atomic(kaddr); flush_dcache_page(dst); } else - copy_user_highpage(dst, src, va, vma); + copy_user_highpage(dst, src, vmf->address, vmf->vma); } static gfp_t __get_fault_gfp_mask(struct vm_area_struct *vma) @@ -2318,7 +2338,7 @@ static vm_fault_t wp_page_copy(struct vm_fault *vmf) vmf->address); if (!new_page) goto oom; - cow_user_page(new_page, old_page, vmf->address, vma); + cow_user_page(new_page, old_page, vmf); } if (mem_cgroup_try_charge_delay(new_page, mm, GFP_KERNEL, &memcg, false))