From patchwork Tue Jan 14 00:29:10 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 11331105 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 2E8B41820 for ; Tue, 14 Jan 2020 00:29:14 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id E28F42084D for ; Tue, 14 Jan 2020 00:29:13 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=kernel.org header.i=@kernel.org header.b="F9daBbTe" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org E28F42084D Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 030528E0008; Mon, 13 Jan 2020 19:29:13 -0500 (EST) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id EFBE18E0003; Mon, 13 Jan 2020 19:29:12 -0500 (EST) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id DC3DF8E0008; Mon, 13 Jan 2020 19:29:12 -0500 (EST) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0176.hostedemail.com [216.40.44.176]) by kanga.kvack.org (Postfix) with ESMTP id C4BBD8E0003 for ; Mon, 13 Jan 2020 19:29:12 -0500 (EST) Received: from smtpin11.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay04.hostedemail.com (Postfix) with SMTP id 67B492464 for ; Tue, 14 Jan 2020 00:29:12 +0000 (UTC) X-FDA: 76374355344.11.north63_6d7148559a455 X-Spam-Summary: 2,0,0,f478630a290265c5,d41d8cd98f00b204,akpm@linux-foundation.org,:thomas.willhalm@intel.com:stable@vger.kernel.org:otto.g.bruggeman@intel.com:kirill.shutemov@linux.intel.com:dan.j.williams@intel.com:aneesh.kumar@linux.vnet.ibm.com:kirill@shutemov.name:akpm@linux-foundation.org::mm-commits@vger.kernel.org:torvalds@linux-foundation.org,RULES_HIT:41:69:355:379:800:960:967:973:988:989:1260:1263:1345:1359:1381:1431:1437:1535:1544:1605:1711:1730:1747:1777:1792:2393:2525:2559:2563:2682:2685:2693:2859:2890:2892:2899:2902:2933:2937:2939:2942:2945:2947:2951:2954:3022:3138:3139:3140:3141:3142:3865:3866:3867:3868:3870:3871:3872:3874:3934:3936:3938:3941:3944:3947:3950:3953:3956:3959:4042:4117:4250:4321:4605:5007:6119:6120:6261:6653:6737:7576:7875:7901:7903:7904:8599:8603:9025:9036:9545:9592:10004:10913:11026:11473:11658:11914:12043:12048:12291:12296:12297:12438:12517:12519:12555:12679:12683:12783:12986:13141:13161:13184:13191:13192:13221:13229:13230:13255:14096:14181:1472 1:14849: X-HE-Tag: north63_6d7148559a455 X-Filterd-Recvd-Size: 6060 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf30.hostedemail.com (Postfix) with ESMTP for ; Tue, 14 Jan 2020 00:29:11 +0000 (UTC) Received: from akpm3.svl.corp.google.com (unknown [104.133.8.65]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPSA id EA2E92080D; Tue, 14 Jan 2020 00:29:10 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=default; t=1578961751; bh=YI2CR8TD42NI+UiZNILuPqhvV3jhtV6hqkOe1BZvkvk=; h=Date:From:To:Subject:In-Reply-To:From; b=F9daBbTegDbBNOmSsgqsKJ5CTiOx5biB1BimM6sJai64VycJPZ4MWSzUiqkEginKe GiIzSpobqJ2twgFHJfpUx0k0sJnqSEoW+1hPjXGHu0m8FbQhX7vX98GEzOLIjweru3 fvEjkDJEHbgtTkUA0KsxpfUZteumroN/R8XjbKYY= Date: Mon, 13 Jan 2020 16:29:10 -0800 From: Andrew Morton To: thomas.willhalm@intel.com, stable@vger.kernel.org, otto.g.bruggeman@intel.com, kirill.shutemov@linux.intel.com, dan.j.williams@intel.com, aneesh.kumar@linux.vnet.ibm.com, kirill@shutemov.name, akpm@linux-foundation.org, linux-mm@kvack.org, mm-commits@vger.kernel.org, torvalds@linux-foundation.org Subject: [patch 03/11] mm/huge_memory.c: thp: fix conflict of above-47bit hint address and PMD alignment Message-ID: <20200114002910.rzSOD%akpm@linux-foundation.org> In-Reply-To: <20200113162831.f7d69e11e9e673c40005c9b0@linux-foundation.org> User-Agent: s-nail v14.9.15 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: "Kirill A. Shutemov" Subject: mm/huge_memory.c: thp: fix conflict of above-47bit hint address and PMD alignment Patch series "Fix two above-47bit hint address vs. THP bugs". The two get_unmapped_area() implementations have to be fixed to provide THP-friendly mappings if above-47bit hint address is specified. This patch (of 2): Filesystems use thp_get_unmapped_area() to provide THP-friendly mappings. For DAX in particular. Normally, the kernel doesn't create userspace mappings above 47-bit, even if the machine allows this (such as with 5-level paging on x86-64). Not all user space is ready to handle wide addresses. It's known that at least some JIT compilers use higher bits in pointers to encode their information. Userspace can ask for allocation from full address space by specifying hint address (with or without MAP_FIXED) above 47-bits. If the application doesn't need a particular address, but wants to allocate from whole address space it can specify -1 as a hint address. Unfortunately, this trick breaks thp_get_unmapped_area(): the function would not try to allocate PMD-aligned area if *any* hint address specified. Modify the routine to handle it correctly: - Try to allocate the space at the specified hint address with length padding required for PMD alignment. - If failed, retry without length padding (but with the same hint address); - If the returned address matches the hint address return it. - Otherwise, align the address as required for THP and return. The user specified hint address is passed down to get_unmapped_area() so above-47bit hint address will be taken into account without breaking alignment requirements. Link: http://lkml.kernel.org/r/20191220142548.7118-2-kirill.shutemov@linux.intel.com Fixes: b569bab78d8d ("x86/mm: Prepare to expose larger address space to userspace") Signed-off-by: Kirill A. Shutemov Reported-by: Thomas Willhalm Tested-by: Dan Williams Cc: "Aneesh Kumar K . V" Cc: "Bruggeman, Otto G" Cc: Signed-off-by: Andrew Morton --- mm/huge_memory.c | 38 ++++++++++++++++++++++++-------------- 1 file changed, 24 insertions(+), 14 deletions(-) --- a/mm/huge_memory.c~thp-fix-conflict-of-above-47bit-hint-address-and-pmd-alignment +++ a/mm/huge_memory.c @@ -527,13 +527,13 @@ void prep_transhuge_page(struct page *pa set_compound_page_dtor(page, TRANSHUGE_PAGE_DTOR); } -static unsigned long __thp_get_unmapped_area(struct file *filp, unsigned long len, +static unsigned long __thp_get_unmapped_area(struct file *filp, + unsigned long addr, unsigned long len, loff_t off, unsigned long flags, unsigned long size) { - unsigned long addr; loff_t off_end = off + len; loff_t off_align = round_up(off, size); - unsigned long len_pad; + unsigned long len_pad, ret; if (off_end <= off_align || (off_end - off_align) < size) return 0; @@ -542,30 +542,40 @@ static unsigned long __thp_get_unmapped_ if (len_pad < len || (off + len_pad) < off) return 0; - addr = current->mm->get_unmapped_area(filp, 0, len_pad, + ret = current->mm->get_unmapped_area(filp, addr, len_pad, off >> PAGE_SHIFT, flags); - if (IS_ERR_VALUE(addr)) + + /* + * The failure might be due to length padding. The caller will retry + * without the padding. + */ + if (IS_ERR_VALUE(ret)) return 0; - addr += (off - addr) & (size - 1); - return addr; + /* + * Do not try to align to THP boundary if allocation at the address + * hint succeeds. + */ + if (ret == addr) + return addr; + + ret += (off - ret) & (size - 1); + return ret; } unsigned long thp_get_unmapped_area(struct file *filp, unsigned long addr, unsigned long len, unsigned long pgoff, unsigned long flags) { + unsigned long ret; loff_t off = (loff_t)pgoff << PAGE_SHIFT; - if (addr) - goto out; if (!IS_DAX(filp->f_mapping->host) || !IS_ENABLED(CONFIG_FS_DAX_PMD)) goto out; - addr = __thp_get_unmapped_area(filp, len, off, flags, PMD_SIZE); - if (addr) - return addr; - - out: + ret = __thp_get_unmapped_area(filp, addr, len, off, flags, PMD_SIZE); + if (ret) + return ret; +out: return current->mm->get_unmapped_area(filp, addr, len, pgoff, flags); } EXPORT_SYMBOL_GPL(thp_get_unmapped_area);