From patchwork Fri Jan 15 17:08:59 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Peter Xu X-Patchwork-Id: 12023353 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.6 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 00721C433DB for ; Fri, 15 Jan 2021 17:10:14 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 9D9532333E for ; Fri, 15 Jan 2021 17:10:13 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 9D9532333E Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=redhat.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 3AF3D8D01B0; Fri, 15 Jan 2021 12:09:58 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 2C0288D01AE; Fri, 15 Jan 2021 12:09:58 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 1AF188D01B0; Fri, 15 Jan 2021 12:09:58 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0210.hostedemail.com [216.40.44.210]) by kanga.kvack.org (Postfix) with ESMTP id F31A78D01AE for ; Fri, 15 Jan 2021 12:09:57 -0500 (EST) Received: from smtpin07.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay04.hostedemail.com (Postfix) with ESMTP id B4C87876D for ; Fri, 15 Jan 2021 17:09:57 +0000 (UTC) X-FDA: 77708646834.07.wheel50_3e10dc927531 Received: from filter.hostedemail.com (10.5.16.251.rfc1918.com [10.5.16.251]) by smtpin07.hostedemail.com (Postfix) with ESMTP id 924FD1818FCD3 for ; Fri, 15 Jan 2021 17:09:57 +0000 (UTC) X-HE-Tag: wheel50_3e10dc927531 X-Filterd-Recvd-Size: 8368 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [216.205.24.124]) by imf18.hostedemail.com (Postfix) with ESMTP for ; Fri, 15 Jan 2021 17:09:56 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1610730596; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=ez9VaM0jxqRxRHrokBxOLqFRU+5IpsC9OSJDJDSHKdU=; b=J/Ozmtq4RRfqpvgwigfUJony8GrqnlEGWqtX/gRdtFLqKFbkfKQR9b8AuCOJNkGwvyeK2q l9GTktVkSoDRU0zrtkAfUYHRbcv9viJPvay++teCl5NU/PToPkyHY0GILR37u3DE43zzgR ytxXEe1pB0theULiNiQ2pXCMgOFyojY= Received: from mail-qt1-f197.google.com (mail-qt1-f197.google.com [209.85.160.197]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-501-pxG-Z7XCNmu5aaxsE1IJXg-1; Fri, 15 Jan 2021 12:09:55 -0500 X-MC-Unique: pxG-Z7XCNmu5aaxsE1IJXg-1 Received: by mail-qt1-f197.google.com with SMTP id b8so7861553qtr.18 for ; Fri, 15 Jan 2021 09:09:54 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=ez9VaM0jxqRxRHrokBxOLqFRU+5IpsC9OSJDJDSHKdU=; b=VdILaMvJaMUQBjZGbwFybFLB8C/vvlo9vNn7K9je2RqgaSKEwToyUBRWer2vNjtdai 3qVJsAtAMedXoZ34puOqFWWj29WunfX4D3uiuJiHVuR6tvyss3YkIVK7Ewo8UEI7RZnD VTX5ITEwUU4UIFn9BvmyTjs5O9l5CiiCn+6JMfPPbYXZE4q9gvqtv0Wm4Z0Xn/GNY3jl Nn4u6tfHCAVwroawHyAcYL2BVohANGntvL4VIRybqYaYe8R8sisI28HV/yAuRBe3i4Lf 6hPBwKjB+gq3XI2+95820hdsdMH1WQEYEcr+Z81zMo8h+rKXV7Fdls8zvD+MjAfFQbTc Bpgw== X-Gm-Message-State: AOAM532nZT6/K5dDZHB49BVOspjb28siEe7friCmd7fM8esgbY8Q9bTf pVnc9IoHdXO3TbjNjl8+Rnkj6OCynDfqtRstJnNyPHDtrZqicFOgFTNgTSbx2Akq9vFjhQxkvXn Obm0EqrfHgGI= X-Received: by 2002:a37:64c4:: with SMTP id y187mr13101249qkb.371.1610730593519; Fri, 15 Jan 2021 09:09:53 -0800 (PST) X-Google-Smtp-Source: ABdhPJwyfJ5XAHeQaZql7ziGHlK2PdUYiLPmxsrv1KSOsOkkqMGGZaMw9BF6kuEjwLDe+dcyq4yjJA== X-Received: by 2002:a37:64c4:: with SMTP id y187mr13101223qkb.371.1610730593288; Fri, 15 Jan 2021 09:09:53 -0800 (PST) Received: from localhost.localdomain ([142.126.83.202]) by smtp.gmail.com with ESMTPSA id d123sm5187840qke.95.2021.01.15.09.09.51 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 15 Jan 2021 09:09:52 -0800 (PST) From: Peter Xu To: linux-kernel@vger.kernel.org, linux-mm@kvack.org Cc: Mike Rapoport , Mike Kravetz , peterx@redhat.com, Jerome Glisse , "Kirill A . Shutemov" , Hugh Dickins , Axel Rasmussen , Matthew Wilcox , Andrew Morton , Andrea Arcangeli , Nadav Amit Subject: [PATCH RFC 22/30] hugetlb/userfaultfd: Forbid huge pmd sharing when uffd enabled Date: Fri, 15 Jan 2021 12:08:59 -0500 Message-Id: <20210115170907.24498-23-peterx@redhat.com> X-Mailer: git-send-email 2.26.2 In-Reply-To: <20210115170907.24498-1-peterx@redhat.com> References: <20210115170907.24498-1-peterx@redhat.com> MIME-Version: 1.0 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=peterx@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Huge pmd sharing could bring problem to userfaultfd. The thing is that userfaultfd is running its logic based on the special bits on page table entries, however the huge pmd sharing could potentially share page table entries for different address ranges. That could cause issues on either: - When sharing huge pmd page tables for an uffd write protected range, the newly mapped huge pmd range will also be write protected unexpectedly, or, - When we try to write protect a range of huge pmd shared range, we'll first do huge_pmd_unshare() in hugetlb_change_protection(), however that also means the UFFDIO_WRITEPROTECT could be silently skipped for the shared region, which could lead to data loss. Since at it, a few other things are done altogether: - Move want_pmd_share() from mm/hugetlb.c into linux/hugetlb.h, because that's definitely something that arch code would like to use too - ARM64 currently directly check against CONFIG_ARCH_WANT_HUGE_PMD_SHARE when trying to share huge pmd. Switch to the want_pmd_share() helper. Signed-off-by: Peter Xu --- arch/arm64/mm/hugetlbpage.c | 3 +-- include/linux/hugetlb.h | 12 ++++++++++++ include/linux/userfaultfd_k.h | 9 +++++++++ mm/hugetlb.c | 5 ++--- 4 files changed, 24 insertions(+), 5 deletions(-) diff --git a/arch/arm64/mm/hugetlbpage.c b/arch/arm64/mm/hugetlbpage.c index 5b32ec888698..1a8ce0facfe8 100644 --- a/arch/arm64/mm/hugetlbpage.c +++ b/arch/arm64/mm/hugetlbpage.c @@ -284,8 +284,7 @@ pte_t *huge_pte_alloc(struct mm_struct *mm, struct vm_area_struct *vma, */ ptep = pte_alloc_map(mm, pmdp, addr); } else if (sz == PMD_SIZE) { - if (IS_ENABLED(CONFIG_ARCH_WANT_HUGE_PMD_SHARE) && - pud_none(READ_ONCE(*pudp))) + if (want_pmd_share(vma) && pud_none(READ_ONCE(*pudp))) ptep = huge_pmd_share(mm, addr, pudp); else ptep = (pte_t *)pmd_alloc(mm, pudp, addr); diff --git a/include/linux/hugetlb.h b/include/linux/hugetlb.h index 7d4c5669e118..27ada597a8e6 100644 --- a/include/linux/hugetlb.h +++ b/include/linux/hugetlb.h @@ -11,6 +11,7 @@ #include #include #include +#include struct ctl_table; struct user_struct; @@ -951,4 +952,15 @@ static inline __init void hugetlb_cma_check(void) } #endif +static inline bool want_pmd_share(struct vm_area_struct *vma) +{ +#ifdef CONFIG_ARCH_WANT_HUGE_PMD_SHARE + if (uffd_disable_huge_pmd_share(vma)) + return false; + return true; +#else + return false; +#endif +} + #endif /* _LINUX_HUGETLB_H */ diff --git a/include/linux/userfaultfd_k.h b/include/linux/userfaultfd_k.h index 7d6071a65ded..7d14444862d4 100644 --- a/include/linux/userfaultfd_k.h +++ b/include/linux/userfaultfd_k.h @@ -52,6 +52,15 @@ static inline bool is_mergeable_vm_userfaultfd_ctx(struct vm_area_struct *vma, return vma->vm_userfaultfd_ctx.ctx == vm_ctx.ctx; } +/* + * Never enable huge pmd sharing on uffd-wp registered vmas, because uffd-wp + * protect information is per pgtable entry. + */ +static inline bool uffd_disable_huge_pmd_share(struct vm_area_struct *vma) +{ + return vma->vm_flags & VM_UFFD_WP; +} + static inline bool userfaultfd_missing(struct vm_area_struct *vma) { return vma->vm_flags & VM_UFFD_MISSING; diff --git a/mm/hugetlb.c b/mm/hugetlb.c index eb7cd0c7d6d2..dd2acb8b3f0f 100644 --- a/mm/hugetlb.c +++ b/mm/hugetlb.c @@ -5386,7 +5386,7 @@ int huge_pmd_unshare(struct mm_struct *mm, struct vm_area_struct *vma, *addr = ALIGN(*addr, HPAGE_SIZE * PTRS_PER_PTE) - HPAGE_SIZE; return 1; } -#define want_pmd_share() (1) + #else /* !CONFIG_ARCH_WANT_HUGE_PMD_SHARE */ pte_t *huge_pmd_share(struct mm_struct *mm, unsigned long addr, pud_t *pud) { @@ -5403,7 +5403,6 @@ void adjust_range_if_pmd_sharing_possible(struct vm_area_struct *vma, unsigned long *start, unsigned long *end) { } -#define want_pmd_share() (0) #endif /* CONFIG_ARCH_WANT_HUGE_PMD_SHARE */ #ifdef CONFIG_ARCH_WANT_GENERAL_HUGETLB @@ -5425,7 +5424,7 @@ pte_t *huge_pte_alloc(struct mm_struct *mm, struct vm_area_struct *vma, pte = (pte_t *)pud; } else { BUG_ON(sz != PMD_SIZE); - if (want_pmd_share() && pud_none(*pud)) + if (want_pmd_share(vma) && pud_none(*pud)) pte = huge_pmd_share(mm, addr, pud); else pte = (pte_t *)pmd_alloc(mm, pud, addr);