From patchwork Fri Nov 15 11:58:08 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: =?utf-8?q?Thomas_Hellstr=C3=B6m_=28Intel=29?= X-Patchwork-Id: 11246155 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 2DBA413B2 for ; Fri, 15 Nov 2019 11:58:24 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id EC0BC20732 for ; Fri, 15 Nov 2019 11:58:23 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (1024-bit key) header.d=shipmail.org header.i=@shipmail.org header.b="ent4YHLc" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org EC0BC20732 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=shipmail.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 1D3446B0008; Fri, 15 Nov 2019 06:58:23 -0500 (EST) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 183256B0007; Fri, 15 Nov 2019 06:58:23 -0500 (EST) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 0721D6B000C; Fri, 15 Nov 2019 06:58:23 -0500 (EST) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0156.hostedemail.com [216.40.44.156]) by kanga.kvack.org (Postfix) with ESMTP id E38426B0007 for ; Fri, 15 Nov 2019 06:58:22 -0500 (EST) Received: from smtpin09.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay03.hostedemail.com (Postfix) with SMTP id AA90A8249980 for ; Fri, 15 Nov 2019 11:58:22 +0000 (UTC) X-FDA: 76158364044.09.bird79_eadfbf19480b X-Spam-Summary: 2,0,0,f19649f738817521,d41d8cd98f00b204,thomas_os@shipmail.org,:linux-kernel@vger.kernel.org::thellstrom@vmware.com:arnd@arndb.de:akpm@linux-foundation.org:kirill.shutemov@linux.intel.com:willy@infradead.org,RULES_HIT:41:355:379:541:800:960:973:988:989:1260:1261:1311:1314:1345:1359:1431:1437:1515:1535:1543:1676:1711:1730:1747:1777:1792:1801:1981:2194:2199:2393:2559:2562:2693:3138:3139:3140:3141:3142:3353:3865:3867:3868:3870:3871:3874:4321:4605:5007:6119:6261:6653:7576:7903:10004:11026:11473:11658:11914:12043:12291:12297:12438:12517:12519:12555:12679:12895:13894:14093:14096:14181:14394:14721:21080:21451:21627:30003:30054:30070,0,RBL:79.136.2.42:@shipmail.org:.lbl8.mailshell.net-62.14.203.100 64.201.201.201,CacheIP:none,Bayesian:0.5,0.5,0.5,Netcheck:none,DomainCache:0,MSF:not bulk,SPF:fp,MSBL:0,DNSBL:none,Custom_rules:0:0:0,LFtime:65,LUA_SUMMARY:none X-HE-Tag: bird79_eadfbf19480b X-Filterd-Recvd-Size: 5606 Received: from pio-pvt-msa3.bahnhof.se (pio-pvt-msa3.bahnhof.se [79.136.2.42]) by imf08.hostedemail.com (Postfix) with ESMTP for ; Fri, 15 Nov 2019 11:58:20 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by pio-pvt-msa3.bahnhof.se (Postfix) with ESMTP id 203053F603; Fri, 15 Nov 2019 12:58:19 +0100 (CET) Authentication-Results: pio-pvt-msa3.bahnhof.se; dkim=pass (1024-bit key; unprotected) header.d=shipmail.org header.i=@shipmail.org header.b=ent4YHLc; dkim-atps=neutral X-Virus-Scanned: Debian amavisd-new at bahnhof.se X-Spam-Flag: NO X-Spam-Score: -2.099 X-Spam-Level: X-Spam-Status: No, score=-2.099 tagged_above=-999 required=6.31 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, URIBL_BLOCKED=0.001] autolearn=ham autolearn_force=no Received: from pio-pvt-msa3.bahnhof.se ([127.0.0.1]) by localhost (pio-pvt-msa3.bahnhof.se [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id lt7waE584qmj; Fri, 15 Nov 2019 12:58:18 +0100 (CET) Received: from mail1.shipmail.org (h-205-35.A357.priv.bahnhof.se [155.4.205.35]) (Authenticated sender: mb878879) by pio-pvt-msa3.bahnhof.se (Postfix) with ESMTPA id E08D73F52C; Fri, 15 Nov 2019 12:58:16 +0100 (CET) Received: from localhost.localdomain.localdomain (h-205-35.A357.priv.bahnhof.se [155.4.205.35]) by mail1.shipmail.org (Postfix) with ESMTPSA id 3FD1E360325; Fri, 15 Nov 2019 12:58:16 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=shipmail.org; s=mail; t=1573819096; bh=xVuLL/1vTl5ZKgjhLNPEJeY++MAr80mNmQBa2Hc4ZbU=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=ent4YHLcAnF69PXuJXOvYqTEWm0aLp7iTbKKDC4UgAZZ7kACAaxtdDLwiQ1Q4BcmR TTkQkqf6Qnc52bHVgtIH5iahU56ptBz5rWliz9LYAVhqhoJopVmln3kuADITeD0LQK s23gNUPyK5QcNcq5M5k1Qu2ldhPYMspyB/mxnvFo= From: =?utf-8?q?Thomas_Hellstr=C3=B6m_=28VMware=29?= To: linux-kernel@vger.kernel.org, linux-mm@kvack.org Cc: Thomas Hellstrom , Arnd Bergmann , Andrew Morton , "Kirill A. Shutemov" , Matthew Wilcox Subject: [PATCH 2/2] mm: Fix a huge pud insertion race during faulting Date: Fri, 15 Nov 2019 12:58:08 +0100 Message-Id: <20191115115808.21181-2-thomas_os@shipmail.org> X-Mailer: git-send-email 2.21.0 In-Reply-To: <20191115115808.21181-1-thomas_os@shipmail.org> References: <20191115115808.21181-1-thomas_os@shipmail.org> MIME-Version: 1.0 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Thomas Hellstrom A huge pud page can theoretically be faulted in racing with pmd_alloc() in __handle_mm_fault(). That will lead to pmd_alloc() returning an invalid pmd pointer. Fix this by adding a pud_trans_unstable() function similar to pmd_trans_unstable() and check whether the pud is really stable before using the pmd pointer. Race: Thread 1: Thread 2: Comment create_huge_pud() Fallback - not taken. create_huge_pud() Taken. pmd_alloc() Returns an invalid pointer. Cc: Arnd Bergmann Cc: Andrew Morton Cc: "Kirill A. Shutemov" Cc: Matthew Wilcox Fixes: a00cc7d9dd93 ("mm, x86: add support for PUD-sized transparent hugepages") Signed-off-by: Thomas Hellstrom Acked-by: Kirill A. Shutemov --- include/asm-generic/pgtable.h | 25 +++++++++++++++++++++++++ mm/memory.c | 6 ++++++ 2 files changed, 31 insertions(+) diff --git a/include/asm-generic/pgtable.h b/include/asm-generic/pgtable.h index 8efa45580fd0..c40a0ced53bd 100644 --- a/include/asm-generic/pgtable.h +++ b/include/asm-generic/pgtable.h @@ -927,6 +927,31 @@ static inline int pud_trans_huge(pud_t pud) } #endif +/* See pmd_none_or_trans_huge_or_clear_bad for discussion. */ +static inline int pud_none_or_trans_huge_or_dev_or_clear_bad(pud_t *pud) +{ + pud_t pudval = READ_ONCE(*pud); + + if (pud_none(pudval) || pud_trans_huge(pudval) || pud_devmap(pudval)) + return 1; + if (unlikely(pud_bad(pudval))) { + pud_clear_bad(pud); + return 1; + } + return 0; +} + +/* See pmd_trans_unstable for discussion. */ +static inline int pud_trans_unstable(pud_t *pud) +{ +#if defined(CONFIG_TRANSPARENT_HUGEPAGE) && \ + defined(CONFIG_HAVE_ARCH_TRANSPARENT_HUGEPAGE_PUD) + return pud_none_or_trans_huge_or_dev_or_clear_bad(pud); +#else + return 0; +#endif +} + #ifndef pmd_read_atomic static inline pmd_t pmd_read_atomic(pmd_t *pmdp) { diff --git a/mm/memory.c b/mm/memory.c index b1ca51a079f2..43ff372f4f07 100644 --- a/mm/memory.c +++ b/mm/memory.c @@ -3914,6 +3914,7 @@ static vm_fault_t __handle_mm_fault(struct vm_area_struct *vma, vmf.pud = pud_alloc(mm, p4d, address); if (!vmf.pud) return VM_FAULT_OOM; +retry_pud: if (pud_none(*vmf.pud) && __transparent_hugepage_enabled(vma)) { ret = create_huge_pud(&vmf); if (!(ret & VM_FAULT_FALLBACK)) @@ -3940,6 +3941,11 @@ static vm_fault_t __handle_mm_fault(struct vm_area_struct *vma, vmf.pmd = pmd_alloc(mm, vmf.pud, address); if (!vmf.pmd) return VM_FAULT_OOM; + + /* Huge pud page fault raced with pmd_alloc? */ + if (pud_trans_unstable(vmf.pud)) + goto retry_pud; + if (pmd_none(*vmf.pmd) && __transparent_hugepage_enabled(vma)) { ret = create_huge_pmd(&vmf); if (!(ret & VM_FAULT_FALLBACK))