From patchwork Fri May 20 21:15:59 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yang Shi X-Patchwork-Id: 12857449 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id C1B06C433EF for ; Fri, 20 May 2022 21:16:15 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 5A6DE6B0072; Fri, 20 May 2022 17:16:15 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 554D66B0073; Fri, 20 May 2022 17:16:15 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 41FD26B0074; Fri, 20 May 2022 17:16:15 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0010.hostedemail.com [216.40.44.10]) by kanga.kvack.org (Postfix) with ESMTP id 236BE6B0072 for ; Fri, 20 May 2022 17:16:15 -0400 (EDT) Received: from smtpin06.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay08.hostedemail.com (Postfix) with ESMTP id E944E20AEE for ; Fri, 20 May 2022 21:16:14 +0000 (UTC) X-FDA: 79487379468.06.AC16FE2 Received: from mail-pf1-f180.google.com (mail-pf1-f180.google.com [209.85.210.180]) by imf23.hostedemail.com (Postfix) with ESMTP id C6228140019 for ; Fri, 20 May 2022 21:15:55 +0000 (UTC) Received: by mail-pf1-f180.google.com with SMTP id 202so1730443pfu.0 for ; Fri, 20 May 2022 14:16:14 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=1QLKDkRDVEEGrLLKtQdSS38U6TUL0EcDbwYp3Or4BIw=; b=NYhrj7RyelVXOqRq9pojKSUdPKLFF1GIynVR/ldLf8JQqCX/3ntxNjQlV8L/A0DZMc vZ3c2YpPQjeoSfj1bIIeoFSY9OrsZKsihZwdZVL9hNitrGjSLb+nkpG3pDB8zJalWliK z7pCIOXi+JA7utglSAMGkRg9wmUgLkKmGLZ45dCi0HvrjH9NyUYg12kTdp+RiMT92spe 5kAh9NdQ+7XItGxLIKc7LOvbW3UGvk3GT6Y2UnsuvKQVidxG9rsjG8OvEtzQi7RQ64/R vKvAj2RGL5JUkp3/a7pW0lPJ3ubI9yqBORm5hJxGErWGBNVxb5CSgIczrHxiy0pRnpBs 0HHw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=1QLKDkRDVEEGrLLKtQdSS38U6TUL0EcDbwYp3Or4BIw=; b=4+TMIXZ9BGDFpSDxbfXfm3a/XnFMTvNTsyF6DE+Kw/31ebJExNpV+40Mfnr8buFDp2 fWMV7h7s6Sa//Rf+o7dW+EOQSyFfaXcQul0BPluyqAMt7GvmBC1ES0tbeek+jUvVoFa3 3R99oQ+sti0DB9u3adD6UhLan+u9EUVQGdeHM9uBoQfcyLiJ4jxa5x86xb4CrcwKoBUh iVRz+1IDepBV/cvCGVrBINGYlCh005zFZBzyiaTzpEyasVqhL4WpRG7PTzUDL00Ir9sP OKEShwrOWuUfOAyoY45eAPZkcUQzrbHus59SEv1oPehmBmdokZyv260op1XDeVsOhDxU wJ1w== X-Gm-Message-State: AOAM5311YKk8XgIuXO+m/JzzXnAYw958rFbXAmGDlggPiqbeKQflMbf2 kURBmr6ES06tUv4+3OrA9w8= X-Google-Smtp-Source: ABdhPJzjqV91/dbDxBd+KTR/6WHbquHR0Y+DY4HXucFpGAZnlfdhBfKsWiMXYk3Dbuqk5G7NZrTGMQ== X-Received: by 2002:a05:6a00:1251:b0:50d:f2c2:7f4e with SMTP id u17-20020a056a00125100b0050df2c27f4emr11912367pfi.72.1653081373606; Fri, 20 May 2022 14:16:13 -0700 (PDT) Received: from localhost.localdomain (c-67-174-241-145.hsd1.ca.comcast.net. [67.174.241.145]) by smtp.gmail.com with ESMTPSA id h1-20020a170902748100b0015e8d4eb2c2sm167620pll.268.2022.05.20.14.16.12 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 20 May 2022 14:16:12 -0700 (PDT) From: Yang Shi To: vbabka@suse.cz, kirill.shutemov@linux.intel.com, akpm@linux-foundation.org Cc: shy828301@gmail.com, linux-mm@kvack.org, linux-kernel@vger.kernel.org Subject: [PATCH 1/7] mm: khugepaged: check THP flag in hugepage_vma_check() Date: Fri, 20 May 2022 14:15:59 -0700 Message-Id: <20220520211605.51473-2-shy828301@gmail.com> X-Mailer: git-send-email 2.26.3 In-Reply-To: <20220520211605.51473-1-shy828301@gmail.com> References: <20220520211605.51473-1-shy828301@gmail.com> MIME-Version: 1.0 X-Rspamd-Queue-Id: C6228140019 X-Stat-Signature: xu49ore87kxazn9xrf9r8ujri7oqrnzx Authentication-Results: imf23.hostedemail.com; dkim=pass header.d=gmail.com header.s=20210112 header.b=NYhrj7Ry; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (imf23.hostedemail.com: domain of shy828301@gmail.com designates 209.85.210.180 as permitted sender) smtp.mailfrom=shy828301@gmail.com X-Rspam-User: X-Rspamd-Server: rspam06 X-HE-Tag: 1653081355-219826 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Currently the THP flag check in hugepage_vma_check() will fallthrough if the flag is NEVER and VM_HUGEPAGE is set. This is not a problem for now since all the callers have the flag checked before or can't be invoked if the flag is NEVER. However, the following patch will call hugepage_vma_check() in more places, for example, page fault, so this flag must be checked in hugepge_vma_check(). Signed-off-by: Yang Shi --- mm/khugepaged.c | 3 +++ 1 file changed, 3 insertions(+) diff --git a/mm/khugepaged.c b/mm/khugepaged.c index 637bfecd6bf5..92e91c08d96a 100644 --- a/mm/khugepaged.c +++ b/mm/khugepaged.c @@ -458,6 +458,9 @@ bool hugepage_vma_check(struct vm_area_struct *vma, if (shmem_file(vma->vm_file)) return shmem_huge_enabled(vma); + if (!khugepaged_enabled()) + return false; + /* THP settings require madvise. */ if (!(vm_flags & VM_HUGEPAGE) && !khugepaged_always()) return false; From patchwork Fri May 20 21:16:00 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yang Shi X-Patchwork-Id: 12857450 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 4ECFCC433F5 for ; Fri, 20 May 2022 21:16:17 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id D6D726B0073; Fri, 20 May 2022 17:16:16 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id D1E666B0074; Fri, 20 May 2022 17:16:16 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id B97376B0075; Fri, 20 May 2022 17:16:16 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0014.hostedemail.com [216.40.44.14]) by kanga.kvack.org (Postfix) with ESMTP id AB7FE6B0073 for ; Fri, 20 May 2022 17:16:16 -0400 (EDT) Received: from smtpin28.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay09.hostedemail.com (Postfix) with ESMTP id 7D3B4342B8 for ; Fri, 20 May 2022 21:16:16 +0000 (UTC) X-FDA: 79487379552.28.1F11736 Received: from mail-pg1-f171.google.com (mail-pg1-f171.google.com [209.85.215.171]) by imf10.hostedemail.com (Postfix) with ESMTP id C2C3BC0019 for ; Fri, 20 May 2022 21:15:43 +0000 (UTC) Received: by mail-pg1-f171.google.com with SMTP id 31so8724698pgp.8 for ; Fri, 20 May 2022 14:16:15 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=gg6tNe85KQQ/g3LzRoRENfwwXGulqQh5OTFWqrUU8No=; b=PgTK3glRRfd5JkX+9f6IiYFOmD9Z9dNofGK/OipD+mpgQCHsKjBPp/AQFjZRf0O3qR hKge+wGyWYzGvM+XyongdCHlTqQcHcJKsxqSNcz7crYk+saXUcgUuxTUw2w0bZuQj1N9 Ajipf+rSlrmD0+qBqy5vme1F//nPQIIke6CoQSWdFlkX8NfeJlJqmN9bj5lDVLMt/kZx RJAn3mEOXG65s4VtYcaNT6jb+Kl6w42xRHQhZ2EpOng4nvcb026P/FE8v/RncZcKC9xv zV5b4U2kzG890iKByk0lsfUKoZgZ02xxnRFgP+WURkTaZdFSvAuz1kSFbiv7EZUYt79+ auLA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=gg6tNe85KQQ/g3LzRoRENfwwXGulqQh5OTFWqrUU8No=; b=04YBX1M0x4bkxSInDYZvwI8l61XArBIr8tB3FxmvlhsxQCbQuGpTmqY15xqxNSjsxi 8OTmb5HfH3Jw7IgdaoEZGZjQL8vEGZB5xyj4gudeSY7tCxWcqcqyXEwMa6ddF77i/3l9 VPsmHUC5whgDekxQ6e3xotTsKpYCxb31Pi9mHoPryDJbTbC9J11bjg4w71/aS9xh52R9 tQk81p09Y9OyH9NERMtinMmW4EdRGTpYiXGnyceTiKbkbvBNL1AlFiS40GBI8MyWqpPL aMtxNDfMiwVBO/NruhUNtk68mC6W64s/HrfT3ZQWSy49DOoc+XAyfMGLiiYm0aTo+d+9 wXGg== X-Gm-Message-State: AOAM533aob0KD9+eV5OKk5+IovxA5eP/eCH9c0yQ+eZzrKDm8tHPPDPf e7J9E2YODXnem3D55aKhdaw= X-Google-Smtp-Source: ABdhPJzSPHAO+qKZMPiyyvZY1u0P5E9VsnEg+dfk2XO7c6Ka5FUtpo8JOyZCJt+tyv57RY71VKVndw== X-Received: by 2002:a63:2002:0:b0:3c6:ae77:1869 with SMTP id g2-20020a632002000000b003c6ae771869mr9888206pgg.71.1653081375058; Fri, 20 May 2022 14:16:15 -0700 (PDT) Received: from localhost.localdomain (c-67-174-241-145.hsd1.ca.comcast.net. [67.174.241.145]) by smtp.gmail.com with ESMTPSA id h1-20020a170902748100b0015e8d4eb2c2sm167620pll.268.2022.05.20.14.16.13 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 20 May 2022 14:16:14 -0700 (PDT) From: Yang Shi To: vbabka@suse.cz, kirill.shutemov@linux.intel.com, akpm@linux-foundation.org Cc: shy828301@gmail.com, linux-mm@kvack.org, linux-kernel@vger.kernel.org Subject: [PATCH 2/7] mm: thp: introduce transhuge_vma_size_ok() helper Date: Fri, 20 May 2022 14:16:00 -0700 Message-Id: <20220520211605.51473-3-shy828301@gmail.com> X-Mailer: git-send-email 2.26.3 In-Reply-To: <20220520211605.51473-1-shy828301@gmail.com> References: <20220520211605.51473-1-shy828301@gmail.com> MIME-Version: 1.0 X-Stat-Signature: tyemt1ouh9s9fbne8cagttb4dti16px7 X-Rspam-User: Authentication-Results: imf10.hostedemail.com; dkim=pass header.d=gmail.com header.s=20210112 header.b=PgTK3glR; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (imf10.hostedemail.com: domain of shy828301@gmail.com designates 209.85.215.171 as permitted sender) smtp.mailfrom=shy828301@gmail.com X-Rspamd-Server: rspam09 X-Rspamd-Queue-Id: C2C3BC0019 X-HE-Tag: 1653081343-111848 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: There are couple of places that check whether the vma size is ok for THP or not, they are open coded and duplicate, introduce transhuge_vma_size_ok() helper to do the job. Signed-off-by: Yang Shi --- include/linux/huge_mm.h | 17 +++++++++++++++++ mm/huge_memory.c | 5 +---- mm/khugepaged.c | 12 ++++++------ 3 files changed, 24 insertions(+), 10 deletions(-) diff --git a/include/linux/huge_mm.h b/include/linux/huge_mm.h index 648cb3ce7099..a8f61db47f2a 100644 --- a/include/linux/huge_mm.h +++ b/include/linux/huge_mm.h @@ -116,6 +116,18 @@ extern struct kobj_attribute shmem_enabled_attr; extern unsigned long transparent_hugepage_flags; +/* + * The vma size has to be large enough to hold an aligned HPAGE_PMD_SIZE area. + */ +static inline bool transhuge_vma_size_ok(struct vm_area_struct *vma) +{ + if (round_up(vma->vm_start, HPAGE_PMD_SIZE) < + (vma->vm_end & HPAGE_PMD_MASK)) + return true; + + return false; +} + static inline bool transhuge_vma_suitable(struct vm_area_struct *vma, unsigned long addr) { @@ -345,6 +357,11 @@ static inline bool transparent_hugepage_active(struct vm_area_struct *vma) return false; } +static inline bool transhuge_vma_size_ok(struct vm_area_struct *vma) +{ + return false; +} + static inline bool transhuge_vma_suitable(struct vm_area_struct *vma, unsigned long addr) { diff --git a/mm/huge_memory.c b/mm/huge_memory.c index 80e8b58b4f39..d633f97452c1 100644 --- a/mm/huge_memory.c +++ b/mm/huge_memory.c @@ -71,10 +71,7 @@ unsigned long huge_zero_pfn __read_mostly = ~0UL; bool transparent_hugepage_active(struct vm_area_struct *vma) { - /* The addr is used to check if the vma size fits */ - unsigned long addr = (vma->vm_end & HPAGE_PMD_MASK) - HPAGE_PMD_SIZE; - - if (!transhuge_vma_suitable(vma, addr)) + if (!transhuge_vma_size_ok(vma)) return false; if (vma_is_anonymous(vma)) return __transparent_hugepage_enabled(vma); diff --git a/mm/khugepaged.c b/mm/khugepaged.c index 92e91c08d96a..5bdb30a76f05 100644 --- a/mm/khugepaged.c +++ b/mm/khugepaged.c @@ -454,6 +454,9 @@ bool hugepage_vma_check(struct vm_area_struct *vma, vma->vm_pgoff, HPAGE_PMD_NR)) return false; + if (!transhuge_vma_size_ok(vma)) + return false; + /* Enabled via shmem mount options or sysfs settings. */ if (shmem_file(vma->vm_file)) return shmem_huge_enabled(vma); @@ -512,9 +515,7 @@ void khugepaged_enter_vma(struct vm_area_struct *vma, unsigned long vm_flags) { if (!test_bit(MMF_VM_HUGEPAGE, &vma->vm_mm->flags) && - khugepaged_enabled() && - (((vma->vm_start + ~HPAGE_PMD_MASK) & HPAGE_PMD_MASK) < - (vma->vm_end & HPAGE_PMD_MASK))) { + khugepaged_enabled()) { if (hugepage_vma_check(vma, vm_flags)) __khugepaged_enter(vma->vm_mm); } @@ -2144,10 +2145,9 @@ static unsigned int khugepaged_scan_mm_slot(unsigned int pages, progress++; continue; } - hstart = (vma->vm_start + ~HPAGE_PMD_MASK) & HPAGE_PMD_MASK; + + hstart = round_up(vma->vm_start, HPAGE_PMD_SIZE); hend = vma->vm_end & HPAGE_PMD_MASK; - if (hstart >= hend) - goto skip; if (khugepaged_scan.address > hend) goto skip; if (khugepaged_scan.address < hstart) From patchwork Fri May 20 21:16:01 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yang Shi X-Patchwork-Id: 12857451 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 03F48C433EF for ; Fri, 20 May 2022 21:16:19 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 5F53D6B0074; Fri, 20 May 2022 17:16:19 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 5A5DE6B0075; Fri, 20 May 2022 17:16:19 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 444746B0078; Fri, 20 May 2022 17:16:19 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0011.hostedemail.com [216.40.44.11]) by kanga.kvack.org (Postfix) with ESMTP id 346216B0074 for ; Fri, 20 May 2022 17:16:19 -0400 (EDT) Received: from smtpin12.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay13.hostedemail.com (Postfix) with ESMTP id 0FF086011C for ; Fri, 20 May 2022 21:16:19 +0000 (UTC) X-FDA: 79487379678.12.485651D Received: from mail-pg1-f180.google.com (mail-pg1-f180.google.com [209.85.215.180]) by imf25.hostedemail.com (Postfix) with ESMTP id 724FDA0017 for ; Fri, 20 May 2022 21:15:53 +0000 (UTC) Received: by mail-pg1-f180.google.com with SMTP id t28so1130919pga.6 for ; Fri, 20 May 2022 14:16:18 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=CWTSuBzkkT6cGFJIErCii42a5cTy6mj6tu0MEYBpYuw=; b=iGY0yXmtNOFzXqS3vyE8ZWNBhizNBWAbJa2rbH/Vkpzz7T3dDYF9Ol/pbh9S0aT/QO XAqSSTx79FNGM4luPdcTV/Iog4IgdOeHuLsBRaztYmtCMFvj43q1Ue3PA3TBcvLNo5/b vrpVPdbVgaIWH2tZoEFXRgwC8mh2vx6y6OO60N5j6CIvFX1tPqUnOEKRfunPOQr3b0qx FYZjBYPOpKz0LwrsaPFfj7T6567Q+Uik2rhKsrGy8XHJ9+2YtpcJW74Hzu54nURexFHm Bw55LeG5STs7fGwFdmdV3yvrAj0nfoJXzQgEZ/nVABaILAVfFB7HsmDEHYaYhfUATFlZ oyVg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=CWTSuBzkkT6cGFJIErCii42a5cTy6mj6tu0MEYBpYuw=; b=Dnej8gKYOq8Y6wXIrljekqCP1zSOLg5zhdTcLUcx2EiV6l0SzGzv+wo1ugqT92ut6T 9WMyjB3jvvLTwcLjYHqIW48iNkOgk7PtYzw78oEjE13ju+z3BYPONdnfWKz2R2iayjR/ Opx1o23qjIvKvpZzegdVkGfCxurZAC4Y4GYyU/Jat3lxV2XVdIHayxdZFh9yV3md8Jst oBzJK+HAdZQQnYalDQ9X+j5TNPZB+loBgNtb5/xgjdqweR2/fG6pdAAK8xAbckYI9PYW YQk40n5gPEdPShDWvDnKThYuiqtUHh+bwRzh4Wi0Vn70q4hrw9R5sBJw7xnE8i3Zmua6 jxNA== X-Gm-Message-State: AOAM532GgpVuDWpCqv2GAEUprbNaWnG6Bqs4vnRfuyyjzyD6CRqXka/6 6TGO5FDx3iEpVOyMlN7o8BCpojL8uIs= X-Google-Smtp-Source: ABdhPJzR7VPOsIaHO4OZTJQpRymr7nffns8nLDPr1XGwHxgbnuzrVAXgs8g9ePKP8oLouCW6W3jqvA== X-Received: by 2002:a05:6a00:1c76:b0:510:8b76:93b5 with SMTP id s54-20020a056a001c7600b005108b7693b5mr12027301pfw.44.1653081376489; Fri, 20 May 2022 14:16:16 -0700 (PDT) Received: from localhost.localdomain (c-67-174-241-145.hsd1.ca.comcast.net. [67.174.241.145]) by smtp.gmail.com with ESMTPSA id h1-20020a170902748100b0015e8d4eb2c2sm167620pll.268.2022.05.20.14.16.15 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 20 May 2022 14:16:15 -0700 (PDT) From: Yang Shi To: vbabka@suse.cz, kirill.shutemov@linux.intel.com, akpm@linux-foundation.org Cc: shy828301@gmail.com, linux-mm@kvack.org, linux-kernel@vger.kernel.org Subject: [PATCH 3/7] mm: khugepaged: remove the redundant anon vma check Date: Fri, 20 May 2022 14:16:01 -0700 Message-Id: <20220520211605.51473-4-shy828301@gmail.com> X-Mailer: git-send-email 2.26.3 In-Reply-To: <20220520211605.51473-1-shy828301@gmail.com> References: <20220520211605.51473-1-shy828301@gmail.com> MIME-Version: 1.0 X-Rspamd-Server: rspam08 X-Rspamd-Queue-Id: 724FDA0017 X-Stat-Signature: 73kahoihognjnarnmp6jdyca86k9iphs Authentication-Results: imf25.hostedemail.com; dkim=pass header.d=gmail.com header.s=20210112 header.b=iGY0yXmt; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (imf25.hostedemail.com: domain of shy828301@gmail.com designates 209.85.215.180 as permitted sender) smtp.mailfrom=shy828301@gmail.com X-Rspam-User: X-HE-Tag: 1653081353-883352 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: The hugepage_vma_check() already checked it, so remove the redundant check. Signed-off-by: Yang Shi --- mm/khugepaged.c | 3 --- 1 file changed, 3 deletions(-) diff --git a/mm/khugepaged.c b/mm/khugepaged.c index 5bdb30a76f05..4fe1dd3d8ef4 100644 --- a/mm/khugepaged.c +++ b/mm/khugepaged.c @@ -966,9 +966,6 @@ static int hugepage_vma_revalidate(struct mm_struct *mm, unsigned long address, return SCAN_ADDRESS_RANGE; if (!hugepage_vma_check(vma, vma->vm_flags)) return SCAN_VMA_CHECK; - /* Anon VMA expected */ - if (!vma->anon_vma || !vma_is_anonymous(vma)) - return SCAN_VMA_CHECK; return 0; } From patchwork Fri May 20 21:16:02 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yang Shi X-Patchwork-Id: 12857452 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id E548EC433FE for ; Fri, 20 May 2022 21:16:20 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id C38076B0075; Fri, 20 May 2022 17:16:19 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id BE82B6B0078; Fri, 20 May 2022 17:16:19 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id A37146B007D; Fri, 20 May 2022 17:16:19 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0011.hostedemail.com [216.40.44.11]) by kanga.kvack.org (Postfix) with ESMTP id 96C6B6B0075 for ; Fri, 20 May 2022 17:16:19 -0400 (EDT) Received: from smtpin14.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay07.hostedemail.com (Postfix) with ESMTP id 68E2820935 for ; Fri, 20 May 2022 21:16:19 +0000 (UTC) X-FDA: 79487379678.14.C7DA844 Received: from mail-pf1-f170.google.com (mail-pf1-f170.google.com [209.85.210.170]) by imf28.hostedemail.com (Postfix) with ESMTP id 8CE1FC001B for ; Fri, 20 May 2022 21:15:50 +0000 (UTC) Received: by mail-pf1-f170.google.com with SMTP id u15so8688683pfi.3 for ; Fri, 20 May 2022 14:16:18 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=FWCxIN+M68b8uiRtqF0Q6BeafTmXXsLbaGp3KwE5XtE=; b=MG5vVMB1O1NSDsob7g9pjDp5FCqlGuxDo+ahWdni+xykPUyYHOm+0D8noRAq74+JQ3 ymSgqr4s8HXyKD75hSX82thQ/A9aHCbx1yaH+uEPfkYDwW8qXoJmJOdc3k4PxTJ0l/x/ zem0fbbcKw3CLdBC5NhXLdocPf/BD4U7TApI0nu3S0/wZGBWPblHmC+20JV2qpH86OIq +fNUten4IZo85SyB0S0PDX93GJNkybmS5QUO2lt8NCZryroleEsgnJFw6M0khfq2iGMV 0wvG2C0aQJWNUL0fReMVyC4l1ULdpPVNgBy7a0TGl84OT/JETPlFJ1pjAL87O/I32R5A aHMw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=FWCxIN+M68b8uiRtqF0Q6BeafTmXXsLbaGp3KwE5XtE=; b=HdqdkWUSPFJDVEcYdXavJXiEMXqsu8s/elyp5y9hgsl10DZeOQPGwxLoOx2Sk1tCYB GtSq6jFzlyfo6Eoh+RETZIdj+xpGzgds9kM56s0lTErXc6y3sSEXsVkb7uKIMqxMrqiN 8ad2NbtgG6FiHPbOpqDP1UM5d10s1h/Q2mg3rHZ8FAUAQHDf3F9s3VtlfEbfhko37KQN zAEiMvVg4L0LkE7yf/7wQrzbvUSrIQpEUxg7ZUkp3sKD64y8ciXg38A7H7yCKeSOsnHb P9gj5wMXlYy+7FeHBKfcdU5eHREqGy36ij8BDjqYmvgEFVb5OgFTPUvMN1FhSXsaFG7m 87rg== X-Gm-Message-State: AOAM533FOqRgyVzI5V9nq0M4wNAHQeEhyUZQnkwRJNwvzQbOn45Feg2Y k5q20+vbk1Xr5G+v0jkAlz8= X-Google-Smtp-Source: ABdhPJyMccpJT5QHUtC5TElaI9zRj5Wzpv1ff3zBCJTMwc9y4u0+s7GN6DjhmisDKbpeVNwQvxjWew== X-Received: by 2002:a62:6410:0:b0:4f3:9654:266d with SMTP id y16-20020a626410000000b004f39654266dmr11682191pfb.59.1653081377797; Fri, 20 May 2022 14:16:17 -0700 (PDT) Received: from localhost.localdomain (c-67-174-241-145.hsd1.ca.comcast.net. [67.174.241.145]) by smtp.gmail.com with ESMTPSA id h1-20020a170902748100b0015e8d4eb2c2sm167620pll.268.2022.05.20.14.16.16 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 20 May 2022 14:16:17 -0700 (PDT) From: Yang Shi To: vbabka@suse.cz, kirill.shutemov@linux.intel.com, akpm@linux-foundation.org Cc: shy828301@gmail.com, linux-mm@kvack.org, linux-kernel@vger.kernel.org Subject: [PATCH 4/7] mm: khugepaged: use transhuge_vma_suitable replace open-code Date: Fri, 20 May 2022 14:16:02 -0700 Message-Id: <20220520211605.51473-5-shy828301@gmail.com> X-Mailer: git-send-email 2.26.3 In-Reply-To: <20220520211605.51473-1-shy828301@gmail.com> References: <20220520211605.51473-1-shy828301@gmail.com> MIME-Version: 1.0 X-Rspamd-Server: rspam08 X-Rspamd-Queue-Id: 8CE1FC001B X-Stat-Signature: c6c63oc8bgycwxbx5dc9x847o9ir6mpt Authentication-Results: imf28.hostedemail.com; dkim=pass header.d=gmail.com header.s=20210112 header.b=MG5vVMB1; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (imf28.hostedemail.com: domain of shy828301@gmail.com designates 209.85.210.170 as permitted sender) smtp.mailfrom=shy828301@gmail.com X-Rspam-User: X-HE-Tag: 1653081350-515429 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: The hugepage_vma_revalidate() needs to check if the address is still in the aligned HPAGE_PMD_SIZE area of the vma when reacquiring mmap_lock, but it was open-coded, use transhuge_vma_suitable() to do the job. And add proper comments for transhuge_vma_suitable(). Signed-off-by: Yang Shi --- include/linux/huge_mm.h | 6 ++++++ mm/khugepaged.c | 5 +---- 2 files changed, 7 insertions(+), 4 deletions(-) diff --git a/include/linux/huge_mm.h b/include/linux/huge_mm.h index a8f61db47f2a..79d5919beb83 100644 --- a/include/linux/huge_mm.h +++ b/include/linux/huge_mm.h @@ -128,6 +128,12 @@ static inline bool transhuge_vma_size_ok(struct vm_area_struct *vma) return false; } +/* + * Do the below checks: + * - For non-anon vma, check if the vm_pgoff is HPAGE_PMD_NR aligned. + * - For all vmas, check if the haddr is in an aligned HPAGE_PMD_SIZE + * area. + */ static inline bool transhuge_vma_suitable(struct vm_area_struct *vma, unsigned long addr) { diff --git a/mm/khugepaged.c b/mm/khugepaged.c index 4fe1dd3d8ef4..51f0e6ea3977 100644 --- a/mm/khugepaged.c +++ b/mm/khugepaged.c @@ -951,7 +951,6 @@ static int hugepage_vma_revalidate(struct mm_struct *mm, unsigned long address, struct vm_area_struct **vmap) { struct vm_area_struct *vma; - unsigned long hstart, hend; if (unlikely(khugepaged_test_exit(mm))) return SCAN_ANY_PROCESS; @@ -960,9 +959,7 @@ static int hugepage_vma_revalidate(struct mm_struct *mm, unsigned long address, if (!vma) return SCAN_VMA_NULL; - hstart = (vma->vm_start + ~HPAGE_PMD_MASK) & HPAGE_PMD_MASK; - hend = vma->vm_end & HPAGE_PMD_MASK; - if (address < hstart || address + HPAGE_PMD_SIZE > hend) + if (!transhuge_vma_suitable(vma, address)) return SCAN_ADDRESS_RANGE; if (!hugepage_vma_check(vma, vma->vm_flags)) return SCAN_VMA_CHECK; From patchwork Fri May 20 21:16:03 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yang Shi X-Patchwork-Id: 12857453 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 69888C433F5 for ; Fri, 20 May 2022 21:16:22 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id B9ADA6B0078; Fri, 20 May 2022 17:16:21 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id B4D0B6B007D; Fri, 20 May 2022 17:16:21 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 9D0226B007E; Fri, 20 May 2022 17:16:21 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0010.hostedemail.com [216.40.44.10]) by kanga.kvack.org (Postfix) with ESMTP id 8EE886B0078 for ; Fri, 20 May 2022 17:16:21 -0400 (EDT) Received: from smtpin11.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay10.hostedemail.com (Postfix) with ESMTP id 6F978735 for ; Fri, 20 May 2022 21:16:21 +0000 (UTC) X-FDA: 79487379762.11.128993B Received: from mail-pg1-f182.google.com (mail-pg1-f182.google.com [209.85.215.182]) by imf15.hostedemail.com (Postfix) with ESMTP id C287CA0015 for ; Fri, 20 May 2022 21:16:03 +0000 (UTC) Received: by mail-pg1-f182.google.com with SMTP id 137so8739341pgb.5 for ; Fri, 20 May 2022 14:16:20 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=uZ2MKo3NmwofqEpjSOVwGsJSdmq+ezPANYKqYBOCivQ=; b=GlUHTdvFzA+MiS/hUNdv6efIELq4i8AG5YNwOF3pCD/x8eHQ3mO/nO9rzy2mnR7XH1 ValbE4N4DtXGnshp8xRyTVvJzq2qACTDgi0ajm9hyH7DMsj5u4C2Jg8reA4QlehsVHbR be5ed75Yfa6fp38jcJq+/yfWqPbe2ETtPdPr+IVTBft67Os6XqydSX+tMIGtCeZUUOtp KiuB1bGfYS8zV6EUxtSrDxCoMRTFSmk3fUy67kQv2L/Eti7oN2DIRQTTx8mc0mLSl5Wg 2ppQD4KDMrosfKKb8KfPX1P4BH2GMZ9hcmKAcs2McyocWqVKB4rdSI04zdLEy3XJpuIN 76Cg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=uZ2MKo3NmwofqEpjSOVwGsJSdmq+ezPANYKqYBOCivQ=; b=Mudv/Kg1BLg75xNEXN887B4Sz75z+BbsRGhnko2+9JP22soVH2iWKImxZ0PxeNaWcY ScoOQvM6NnaeW4keJbgBLstk+cnk4ADRheRhJKXlRY3CdYrkloXdwy6x8C0VaSesFCTm sR60W0AAMqeAIh7Bye7WQInFS1KRevWmyroKuNFEu8EODKohisPTjVevsKHdMkVzJTle 0ZEaX2u0r4VoKNxwSvRZbMjG0N/eil+9Gf6dE+0JjPQaHk71FR7WrG91V6OL741ZvlSF 7bMVcUdc0Jo1rv1mO0xdO2EqdqHr/MaqcMjdbSxfwC3N6L7jHIfaOmf2P6p0kKpFbe3T UObA== X-Gm-Message-State: AOAM531zEzbRpqPDYO4nRZKXTua9SNkUKNbzA0hPXxnZaOhwCt7rAiN6 /vfCtDZx2Ojrf1uJx2RZAFE= X-Google-Smtp-Source: ABdhPJxHWgmmuDTl0VVnULJFQSbEJPMKkPtl2dOD6vFWj6vfs9p9296+xsaK0tTFIq3fQZmISbfDAQ== X-Received: by 2002:a62:b515:0:b0:50a:3d51:671e with SMTP id y21-20020a62b515000000b0050a3d51671emr11785398pfe.48.1653081379866; Fri, 20 May 2022 14:16:19 -0700 (PDT) Received: from localhost.localdomain (c-67-174-241-145.hsd1.ca.comcast.net. [67.174.241.145]) by smtp.gmail.com with ESMTPSA id h1-20020a170902748100b0015e8d4eb2c2sm167620pll.268.2022.05.20.14.16.17 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 20 May 2022 14:16:19 -0700 (PDT) From: Yang Shi To: vbabka@suse.cz, kirill.shutemov@linux.intel.com, akpm@linux-foundation.org Cc: shy828301@gmail.com, linux-mm@kvack.org, linux-kernel@vger.kernel.org Subject: [PATCH 5/7] mm: thp: kill transparent_hugepage_active() Date: Fri, 20 May 2022 14:16:03 -0700 Message-Id: <20220520211605.51473-6-shy828301@gmail.com> X-Mailer: git-send-email 2.26.3 In-Reply-To: <20220520211605.51473-1-shy828301@gmail.com> References: <20220520211605.51473-1-shy828301@gmail.com> MIME-Version: 1.0 Authentication-Results: imf15.hostedemail.com; dkim=pass header.d=gmail.com header.s=20210112 header.b=GlUHTdvF; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (imf15.hostedemail.com: domain of shy828301@gmail.com designates 209.85.215.182 as permitted sender) smtp.mailfrom=shy828301@gmail.com X-Rspam-User: X-Rspamd-Server: rspam02 X-Rspamd-Queue-Id: C287CA0015 X-Stat-Signature: mahwtqi5jhq1xfwjed4tu3eyrrsckcgb X-HE-Tag: 1653081363-469940 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: The transparent_hugepage_active() was introduced to show THP eligibility bit in smaps in proc, smaps is the only user. But it actually does the similar check as hugepage_vma_check() which is used by khugepaged. We definitely don't have to maintain two similar checks, so kill transparent_hugepage_active(). Also move hugepage_vma_check() to huge_memory.c and huge_mm.h since it is not only for khugepaged anymore. Signed-off-by: Yang Shi --- fs/proc/task_mmu.c | 2 +- include/linux/huge_mm.h | 14 +++++++------ include/linux/khugepaged.h | 2 -- mm/huge_memory.c | 41 ++++++++++++++++++++++++++++++------ mm/khugepaged.c | 43 -------------------------------------- 5 files changed, 44 insertions(+), 58 deletions(-) diff --git a/fs/proc/task_mmu.c b/fs/proc/task_mmu.c index f9c9abb50bb7..ab4849d07a1f 100644 --- a/fs/proc/task_mmu.c +++ b/fs/proc/task_mmu.c @@ -863,7 +863,7 @@ static int show_smap(struct seq_file *m, void *v) __show_smap(m, &mss, false); seq_printf(m, "THPeligible: %d\n", - transparent_hugepage_active(vma)); + hugepage_vma_check(vma, vma->vm_flags)); if (arch_pkeys_enabled()) seq_printf(m, "ProtectionKey: %8u\n", vma_pkey(vma)); diff --git a/include/linux/huge_mm.h b/include/linux/huge_mm.h index 79d5919beb83..2da52ca8fd52 100644 --- a/include/linux/huge_mm.h +++ b/include/linux/huge_mm.h @@ -209,7 +209,8 @@ static inline bool file_thp_enabled(struct vm_area_struct *vma) !inode_is_open_for_write(inode) && S_ISREG(inode->i_mode); } -bool transparent_hugepage_active(struct vm_area_struct *vma); +extern bool hugepage_vma_check(struct vm_area_struct *vma, + unsigned long vm_flags); #define transparent_hugepage_use_zero_page() \ (transparent_hugepage_flags & \ @@ -358,11 +359,6 @@ static inline bool __transparent_hugepage_enabled(struct vm_area_struct *vma) return false; } -static inline bool transparent_hugepage_active(struct vm_area_struct *vma) -{ - return false; -} - static inline bool transhuge_vma_size_ok(struct vm_area_struct *vma) { return false; @@ -380,6 +376,12 @@ static inline bool transhuge_vma_enabled(struct vm_area_struct *vma, return false; } +static inline bool hugepage_vma_check(struct vm_area_struct *vma, + unsigned long vm_flags) +{ + return false; +} + static inline void prep_transhuge_page(struct page *page) {} #define transparent_hugepage_flags 0UL diff --git a/include/linux/khugepaged.h b/include/linux/khugepaged.h index 392d34c3c59a..795c8d2170b0 100644 --- a/include/linux/khugepaged.h +++ b/include/linux/khugepaged.h @@ -10,8 +10,6 @@ extern struct attribute_group khugepaged_attr_group; extern int khugepaged_init(void); extern void khugepaged_destroy(void); extern int start_stop_khugepaged(void); -extern bool hugepage_vma_check(struct vm_area_struct *vma, - unsigned long vm_flags); extern void __khugepaged_enter(struct mm_struct *mm); extern void __khugepaged_exit(struct mm_struct *mm); extern void khugepaged_enter_vma(struct vm_area_struct *vma, diff --git a/mm/huge_memory.c b/mm/huge_memory.c index d633f97452c1..d9ec82892b05 100644 --- a/mm/huge_memory.c +++ b/mm/huge_memory.c @@ -69,18 +69,47 @@ static atomic_t huge_zero_refcount; struct page *huge_zero_page __read_mostly; unsigned long huge_zero_pfn __read_mostly = ~0UL; -bool transparent_hugepage_active(struct vm_area_struct *vma) +bool hugepage_vma_check(struct vm_area_struct *vma, + unsigned long vm_flags) { + if (!transhuge_vma_enabled(vma, vm_flags)) + return false; + + if (vm_flags & VM_NO_KHUGEPAGED) + return false; + + /* Don't run khugepaged against DAX vma */ + if (vma_is_dax(vma)) + return false; + + if (vma->vm_file && !IS_ALIGNED((vma->vm_start >> PAGE_SHIFT) - + vma->vm_pgoff, HPAGE_PMD_NR)) + return false; + if (!transhuge_vma_size_ok(vma)) return false; - if (vma_is_anonymous(vma)) - return __transparent_hugepage_enabled(vma); - if (vma_is_shmem(vma)) + + /* Enabled via shmem mount options or sysfs settings. */ + if (shmem_file(vma->vm_file)) return shmem_huge_enabled(vma); - if (transhuge_vma_enabled(vma, vma->vm_flags) && file_thp_enabled(vma)) + + if (!khugepaged_enabled()) + return false; + + /* THP settings require madvise. */ + if (!(vm_flags & VM_HUGEPAGE) && !khugepaged_always()) + return false; + + /* Only regular file is valid */ + if (file_thp_enabled(vma)) return true; - return false; + if (!vma->anon_vma || !vma_is_anonymous(vma)) + return false; + if (vma_is_temporary_stack(vma)) + return false; + + return true; } static bool get_huge_zero_page(void) diff --git a/mm/khugepaged.c b/mm/khugepaged.c index 51f0e6ea3977..a7ac1ae418f8 100644 --- a/mm/khugepaged.c +++ b/mm/khugepaged.c @@ -437,49 +437,6 @@ static inline int khugepaged_test_exit(struct mm_struct *mm) return atomic_read(&mm->mm_users) == 0; } -bool hugepage_vma_check(struct vm_area_struct *vma, - unsigned long vm_flags) -{ - if (!transhuge_vma_enabled(vma, vm_flags)) - return false; - - if (vm_flags & VM_NO_KHUGEPAGED) - return false; - - /* Don't run khugepaged against DAX vma */ - if (vma_is_dax(vma)) - return false; - - if (vma->vm_file && !IS_ALIGNED((vma->vm_start >> PAGE_SHIFT) - - vma->vm_pgoff, HPAGE_PMD_NR)) - return false; - - if (!transhuge_vma_size_ok(vma)) - return false; - - /* Enabled via shmem mount options or sysfs settings. */ - if (shmem_file(vma->vm_file)) - return shmem_huge_enabled(vma); - - if (!khugepaged_enabled()) - return false; - - /* THP settings require madvise. */ - if (!(vm_flags & VM_HUGEPAGE) && !khugepaged_always()) - return false; - - /* Only regular file is valid */ - if (file_thp_enabled(vma)) - return true; - - if (!vma->anon_vma || !vma_is_anonymous(vma)) - return false; - if (vma_is_temporary_stack(vma)) - return false; - - return true; -} - void __khugepaged_enter(struct mm_struct *mm) { struct mm_slot *mm_slot; From patchwork Fri May 20 21:16:04 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yang Shi X-Patchwork-Id: 12857454 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id D455BC433FE for ; Fri, 20 May 2022 21:16:23 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 642DD6B007D; Fri, 20 May 2022 17:16:23 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 5F48C6B007E; Fri, 20 May 2022 17:16:23 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 380D16B0080; Fri, 20 May 2022 17:16:23 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0016.hostedemail.com [216.40.44.16]) by kanga.kvack.org (Postfix) with ESMTP id 294386B007D for ; Fri, 20 May 2022 17:16:23 -0400 (EDT) Received: from smtpin06.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay08.hostedemail.com (Postfix) with ESMTP id 02E2F20AFA for ; Fri, 20 May 2022 21:16:23 +0000 (UTC) X-FDA: 79487379846.06.70A9FB1 Received: from mail-pl1-f181.google.com (mail-pl1-f181.google.com [209.85.214.181]) by imf15.hostedemail.com (Postfix) with ESMTP id 73EC8A0010 for ; Fri, 20 May 2022 21:16:05 +0000 (UTC) Received: by mail-pl1-f181.google.com with SMTP id n8so8348173plh.1 for ; Fri, 20 May 2022 14:16:22 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=TZGbrX4QM92lpEadFI2/iSQf+FDJdRaQWbUBPCSmje8=; b=UDHxs7Tka/6dGhK1B10RCJNyooHkQLV2mOuI+Tnf9llsxd5DSAeIb0RfTrKjghW7Wu /K1eGBERoI3QiWpcVu0Aj1n5DQvxi6qIoY825v+wpc1gGXTm0U/gZmQzAlTPjbPvdZ89 31sD4OtYTOVJXwEd1rJNyhQxmTqAgYvsL9+JAfsEf0ICkwVrvrBqf0125qAobUEl3f6w //F/ZalYdnOKXF/H8KP5tDveq2wW7joL27gehvWZOoZ4IRGP5kq1/LWqMwR7ZfDsMIqL /vOhfTPWXb9vfWKC5MMIaskT0tyMDMy30YStSBxt9sJaBQKF8bFMPIMAQLCDM3kPy6kG yrEg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=TZGbrX4QM92lpEadFI2/iSQf+FDJdRaQWbUBPCSmje8=; b=SFqyNsOxdETdHBOy9XDKvjQfM8ueigSxT95AH2MMQTHBHl3hT4qlLW/VeDHrENBnwo AJAdSli5LDCmsFm+cNuNg35N0YiJbvJjTLDF5fCY5RMa36KSWdTDP2LKYw9VfbNt4Mjg uTnvbljCkGbE5T8Tp/C4b6GeWlgJXxMGjcjtvgaGanhipaxh5Di5hMZBKDR4LJRtlGjl XyW17h+RMJE+Ui7BK8ws/nhB1uyIyVogsmWh8LpTbnHdgWLmgqV5JrRp9uic4XvKKfmN FDylEIPY1YWu57goD1RCaCban1APqTnetWLXCIRPmm1ysg/IWapCtGnr1LV0aVX1yulE N9SQ== X-Gm-Message-State: AOAM532avyMUq435iRSDTmXWo7squIrpY+p6pvmLKTH3k3UKY+RxvBQj HWufoKhjD9I3/91yrL5xMsQ= X-Google-Smtp-Source: ABdhPJxu9xr7EW6hkAt6iaVTmS4PakGrCNCaum6wmHT0fFwAs3shwvx4OWMk38pErjAtuqxeDTpfbQ== X-Received: by 2002:a17:90b:4b8a:b0:1df:75f2:d2f4 with SMTP id lr10-20020a17090b4b8a00b001df75f2d2f4mr13096306pjb.109.1653081381644; Fri, 20 May 2022 14:16:21 -0700 (PDT) Received: from localhost.localdomain (c-67-174-241-145.hsd1.ca.comcast.net. [67.174.241.145]) by smtp.gmail.com with ESMTPSA id h1-20020a170902748100b0015e8d4eb2c2sm167620pll.268.2022.05.20.14.16.20 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 20 May 2022 14:16:20 -0700 (PDT) From: Yang Shi To: vbabka@suse.cz, kirill.shutemov@linux.intel.com, akpm@linux-foundation.org Cc: shy828301@gmail.com, linux-mm@kvack.org, linux-kernel@vger.kernel.org Subject: [PATCH 6/7] mm: thp: kill __transhuge_page_enabled() Date: Fri, 20 May 2022 14:16:04 -0700 Message-Id: <20220520211605.51473-7-shy828301@gmail.com> X-Mailer: git-send-email 2.26.3 In-Reply-To: <20220520211605.51473-1-shy828301@gmail.com> References: <20220520211605.51473-1-shy828301@gmail.com> MIME-Version: 1.0 X-Rspamd-Server: rspam07 X-Rspamd-Queue-Id: 73EC8A0010 X-Stat-Signature: erke99tumcbjonjxznwbd3agcixj5igm X-Rspam-User: Authentication-Results: imf15.hostedemail.com; dkim=pass header.d=gmail.com header.s=20210112 header.b=UDHxs7Tk; spf=pass (imf15.hostedemail.com: domain of shy828301@gmail.com designates 209.85.214.181 as permitted sender) smtp.mailfrom=shy828301@gmail.com; dmarc=pass (policy=none) header.from=gmail.com X-HE-Tag: 1653081365-381391 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: The page fault path checks THP eligibility with __transhuge_page_enabled() which does the similar thing as hugepage_vma_check(), so use hugepage_vma_check() instead. However page fault allows DAX and !anon_vma cases, so added a new flag, in_pf, to hugepage_vma_check() to make page fault work correctly. The in_pf flag is also used to skip shmem and file THP for page fault since shmem handles THP in its own shmem_fault() and file THP allocation on fault is not supported yet. Also remove hugepage_vma_enabled() since hugepage_vma_check() is the only caller now, it is not necessary to have a helper function. Signed-off-by: Yang Shi --- fs/proc/task_mmu.c | 2 +- include/linux/huge_mm.h | 57 ++------------------------------------ include/linux/khugepaged.h | 2 +- mm/huge_memory.c | 24 +++++++++++----- mm/khugepaged.c | 8 +++--- mm/memory.c | 6 ++-- 6 files changed, 29 insertions(+), 70 deletions(-) diff --git a/fs/proc/task_mmu.c b/fs/proc/task_mmu.c index ab4849d07a1f..28f472a2eb51 100644 --- a/fs/proc/task_mmu.c +++ b/fs/proc/task_mmu.c @@ -863,7 +863,7 @@ static int show_smap(struct seq_file *m, void *v) __show_smap(m, &mss, false); seq_printf(m, "THPeligible: %d\n", - hugepage_vma_check(vma, vma->vm_flags)); + hugepage_vma_check(vma, vma->vm_flags, false)); if (arch_pkeys_enabled()) seq_printf(m, "ProtectionKey: %8u\n", vma_pkey(vma)); diff --git a/include/linux/huge_mm.h b/include/linux/huge_mm.h index 2da52ca8fd52..67c913e66e60 100644 --- a/include/linux/huge_mm.h +++ b/include/linux/huge_mm.h @@ -153,48 +153,6 @@ static inline bool transhuge_vma_suitable(struct vm_area_struct *vma, return true; } -static inline bool transhuge_vma_enabled(struct vm_area_struct *vma, - unsigned long vm_flags) -{ - /* Explicitly disabled through madvise. */ - if ((vm_flags & VM_NOHUGEPAGE) || - test_bit(MMF_DISABLE_THP, &vma->vm_mm->flags)) - return false; - return true; -} - -/* - * to be used on vmas which are known to support THP. - * Use transparent_hugepage_active otherwise - */ -static inline bool __transparent_hugepage_enabled(struct vm_area_struct *vma) -{ - - /* - * If the hardware/firmware marked hugepage support disabled. - */ - if (transparent_hugepage_flags & (1 << TRANSPARENT_HUGEPAGE_NEVER_DAX)) - return false; - - if (!transhuge_vma_enabled(vma, vma->vm_flags)) - return false; - - if (vma_is_temporary_stack(vma)) - return false; - - if (transparent_hugepage_flags & (1 << TRANSPARENT_HUGEPAGE_FLAG)) - return true; - - if (vma_is_dax(vma)) - return true; - - if (transparent_hugepage_flags & - (1 << TRANSPARENT_HUGEPAGE_REQ_MADV_FLAG)) - return !!(vma->vm_flags & VM_HUGEPAGE); - - return false; -} - static inline bool file_thp_enabled(struct vm_area_struct *vma) { struct inode *inode; @@ -210,7 +168,7 @@ static inline bool file_thp_enabled(struct vm_area_struct *vma) } extern bool hugepage_vma_check(struct vm_area_struct *vma, - unsigned long vm_flags); + unsigned long vm_flags, bool in_pf); #define transparent_hugepage_use_zero_page() \ (transparent_hugepage_flags & \ @@ -354,11 +312,6 @@ static inline bool folio_test_pmd_mappable(struct folio *folio) return false; } -static inline bool __transparent_hugepage_enabled(struct vm_area_struct *vma) -{ - return false; -} - static inline bool transhuge_vma_size_ok(struct vm_area_struct *vma) { return false; @@ -370,14 +323,8 @@ static inline bool transhuge_vma_suitable(struct vm_area_struct *vma, return false; } -static inline bool transhuge_vma_enabled(struct vm_area_struct *vma, - unsigned long vm_flags) -{ - return false; -} - static inline bool hugepage_vma_check(struct vm_area_struct *vma, - unsigned long vm_flags) + unsigned long vm_flags, bool in_pf) { return false; } diff --git a/include/linux/khugepaged.h b/include/linux/khugepaged.h index 795c8d2170b0..8a6452e089ca 100644 --- a/include/linux/khugepaged.h +++ b/include/linux/khugepaged.h @@ -55,7 +55,7 @@ static inline void khugepaged_enter(struct vm_area_struct *vma, { if (!test_bit(MMF_VM_HUGEPAGE, &vma->vm_mm->flags) && khugepaged_enabled()) { - if (hugepage_vma_check(vma, vm_flags)) + if (hugepage_vma_check(vma, vm_flags, false)) __khugepaged_enter(vma->vm_mm); } } diff --git a/mm/huge_memory.c b/mm/huge_memory.c index d9ec82892b05..b4cab3aa2534 100644 --- a/mm/huge_memory.c +++ b/mm/huge_memory.c @@ -70,17 +70,26 @@ struct page *huge_zero_page __read_mostly; unsigned long huge_zero_pfn __read_mostly = ~0UL; bool hugepage_vma_check(struct vm_area_struct *vma, - unsigned long vm_flags) + unsigned long vm_flags, bool in_pf) { - if (!transhuge_vma_enabled(vma, vm_flags)) + /* Explicitly disabled through madvise or prctl. */ + if ((vm_flags & VM_NOHUGEPAGE) || + test_bit(MMF_DISABLE_THP, &vma->vm_mm->flags)) return false; + /* + * If the hardware/firmware marked hugepage support disabled. + */ + if (transparent_hugepage_flags & (1 << TRANSPARENT_HUGEPAGE_NEVER_DAX)) + return false; + + /* Special VMA and hugetlb VMA */ if (vm_flags & VM_NO_KHUGEPAGED) return false; - /* Don't run khugepaged against DAX vma */ + /* khugepaged doesn't collapse DAX vma, but page fault is fine. */ if (vma_is_dax(vma)) - return false; + return in_pf; if (vma->vm_file && !IS_ALIGNED((vma->vm_start >> PAGE_SHIFT) - vma->vm_pgoff, HPAGE_PMD_NR)) @@ -90,7 +99,7 @@ bool hugepage_vma_check(struct vm_area_struct *vma, return false; /* Enabled via shmem mount options or sysfs settings. */ - if (shmem_file(vma->vm_file)) + if (!in_pf && shmem_file(vma->vm_file)) return shmem_huge_enabled(vma); if (!khugepaged_enabled()) @@ -101,10 +110,11 @@ bool hugepage_vma_check(struct vm_area_struct *vma, return false; /* Only regular file is valid */ - if (file_thp_enabled(vma)) + if (!in_pf && file_thp_enabled(vma)) return true; - if (!vma->anon_vma || !vma_is_anonymous(vma)) + /* anon_vma may be not prepared before the first page fault */ + if ((!in_pf && !vma->anon_vma) || !vma_is_anonymous(vma)) return false; if (vma_is_temporary_stack(vma)) return false; diff --git a/mm/khugepaged.c b/mm/khugepaged.c index a7ac1ae418f8..292ed9c0db1e 100644 --- a/mm/khugepaged.c +++ b/mm/khugepaged.c @@ -473,7 +473,7 @@ void khugepaged_enter_vma(struct vm_area_struct *vma, { if (!test_bit(MMF_VM_HUGEPAGE, &vma->vm_mm->flags) && khugepaged_enabled()) { - if (hugepage_vma_check(vma, vm_flags)) + if (hugepage_vma_check(vma, vm_flags, false)) __khugepaged_enter(vma->vm_mm); } } @@ -918,7 +918,7 @@ static int hugepage_vma_revalidate(struct mm_struct *mm, unsigned long address, if (!transhuge_vma_suitable(vma, address)) return SCAN_ADDRESS_RANGE; - if (!hugepage_vma_check(vma, vma->vm_flags)) + if (!hugepage_vma_check(vma, vma->vm_flags, false)) return SCAN_VMA_CHECK; return 0; } @@ -1399,7 +1399,7 @@ void collapse_pte_mapped_thp(struct mm_struct *mm, unsigned long addr) * the valid THP. Add extra VM_HUGEPAGE so hugepage_vma_check() * will not fail the vma for missing VM_HUGEPAGE */ - if (!hugepage_vma_check(vma, vma->vm_flags | VM_HUGEPAGE)) + if (!hugepage_vma_check(vma, vma->vm_flags | VM_HUGEPAGE, false)) return; /* Keep pmd pgtable for uffd-wp; see comment in retract_page_tables() */ @@ -2091,7 +2091,7 @@ static unsigned int khugepaged_scan_mm_slot(unsigned int pages, progress++; break; } - if (!hugepage_vma_check(vma, vma->vm_flags)) { + if (!hugepage_vma_check(vma, vma->vm_flags, false)) { skip: progress++; continue; diff --git a/mm/memory.c b/mm/memory.c index 95735bee3876..0850b3e38dbf 100644 --- a/mm/memory.c +++ b/mm/memory.c @@ -4979,7 +4979,8 @@ static vm_fault_t __handle_mm_fault(struct vm_area_struct *vma, if (!vmf.pud) return VM_FAULT_OOM; retry_pud: - if (pud_none(*vmf.pud) && __transparent_hugepage_enabled(vma)) { + if (pud_none(*vmf.pud) && + hugepage_vma_check(vma, vma->vm_flags, true)) { ret = create_huge_pud(&vmf); if (!(ret & VM_FAULT_FALLBACK)) return ret; @@ -5012,7 +5013,8 @@ static vm_fault_t __handle_mm_fault(struct vm_area_struct *vma, if (pud_trans_unstable(vmf.pud)) goto retry_pud; - if (pmd_none(*vmf.pmd) && __transparent_hugepage_enabled(vma)) { + if (pmd_none(*vmf.pmd) && + hugepage_vma_check(vma, vma->vm_flags, true)) { ret = create_huge_pmd(&vmf); if (!(ret & VM_FAULT_FALLBACK)) return ret; From patchwork Fri May 20 21:16:05 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yang Shi X-Patchwork-Id: 12857455 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 4AD94C433EF for ; Fri, 20 May 2022 21:16:26 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id CF9A66B007E; Fri, 20 May 2022 17:16:25 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id CAAF06B0080; Fri, 20 May 2022 17:16:25 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id A60896B0081; Fri, 20 May 2022 17:16:25 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0017.hostedemail.com [216.40.44.17]) by kanga.kvack.org (Postfix) with ESMTP id 958F16B007E for ; Fri, 20 May 2022 17:16:25 -0400 (EDT) Received: from smtpin05.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay02.hostedemail.com (Postfix) with ESMTP id 7E1F431D75 for ; Fri, 20 May 2022 21:16:25 +0000 (UTC) X-FDA: 79487379930.05.61F06BE Received: from mail-pl1-f177.google.com (mail-pl1-f177.google.com [209.85.214.177]) by imf12.hostedemail.com (Postfix) with ESMTP id AC3114000B for ; Fri, 20 May 2022 21:15:53 +0000 (UTC) Received: by mail-pl1-f177.google.com with SMTP id c2so8347429plh.2 for ; Fri, 20 May 2022 14:16:24 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=j4XmJAWPYD2D8xrDs4B9doFxdeAqtS3ydjsiwVqFNQE=; b=Ptl/vJ+4tr0wo8YH9eqwLAvaSJxi7Oycnq92HxfoZk+QdfTfA9ENKktv7+IZjg4dQA Tnji/iUJmJpcFvSWDbCv4VN85SuKnLwbT0huZV6bIyCoRD/7bhs0irQMtF8Lnc7ZY5Q7 wOe9j+hlD+5P7UxV/dOtjIyR381ynQL1SqKIiUsdjoTXtM7kP+Wyw+cZdIglDsx1pDeK 9+HigMz9haP+HCDkKNneVDaB0D2N0tyMbU+HA5dEbgRdmoCqplw1qxf0V8I6+wunGFDl 5lfbF7srBIkNk8zjstYMnHqPez+C1RfCdxiUC3Izu8Owg3I4cjReT4lCRDb/or6V4+zu +/Hw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=j4XmJAWPYD2D8xrDs4B9doFxdeAqtS3ydjsiwVqFNQE=; b=553TZpnWgm+1Y1W6aIXs/pl5+oANvwlhc7wi79AYSrvIxka8ZkeuyLrfYyqWZGc+a9 JnG20eml7nSZYueTF2ZuPPikxNTSgkVxykyE4CAGDXZ2l+4QS9PivG7M5qtt2wn+5LRn SI1CCeiXGbLmBOA94e0hrRMkAyboqYL9Ve8JIzdBtNEy14nMfJ/k82XM4OB4//1SYkt6 Ehs6uEEp32LJ8S2JdtkPTcUFeR7tiqxfESuiGVE/ZCAAtxS6Pq8dO4VMbUUgZpT7iEvv IhD3ySKvjkIHNqkkrGMI6zINuPIU6nMmnhp26n7fmfrILxT2WRYT9xrrZagpLRqPBv6F iqqw== X-Gm-Message-State: AOAM532BrOcgmuLmgcDnN4n+y7OyXmB9deYbKoZUur4lwqib72+jrj2V Gz5C1yXWROV4Lg9gcYgmjaIHy4AmPfs= X-Google-Smtp-Source: ABdhPJy9LubXKnw2CwDr67vvJdDrVydqGO6/z63N55tXZRS8KRReOt/HeiFZIFFyfXfdOhEcnhSYdQ== X-Received: by 2002:a17:902:f551:b0:15e:999a:2425 with SMTP id h17-20020a170902f55100b0015e999a2425mr11172213plf.3.1653081383403; Fri, 20 May 2022 14:16:23 -0700 (PDT) Received: from localhost.localdomain (c-67-174-241-145.hsd1.ca.comcast.net. [67.174.241.145]) by smtp.gmail.com with ESMTPSA id h1-20020a170902748100b0015e8d4eb2c2sm167620pll.268.2022.05.20.14.16.21 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 20 May 2022 14:16:22 -0700 (PDT) From: Yang Shi To: vbabka@suse.cz, kirill.shutemov@linux.intel.com, akpm@linux-foundation.org Cc: shy828301@gmail.com, linux-mm@kvack.org, linux-kernel@vger.kernel.org Subject: [PATCH 7/7] mm: khugepaged: reorg some khugepaged helpers Date: Fri, 20 May 2022 14:16:05 -0700 Message-Id: <20220520211605.51473-8-shy828301@gmail.com> X-Mailer: git-send-email 2.26.3 In-Reply-To: <20220520211605.51473-1-shy828301@gmail.com> References: <20220520211605.51473-1-shy828301@gmail.com> MIME-Version: 1.0 X-Rspam-User: X-Rspamd-Queue-Id: AC3114000B X-Stat-Signature: dipczihus1zypae99guibwi86gp11d9x Authentication-Results: imf12.hostedemail.com; dkim=pass header.d=gmail.com header.s=20210112 header.b="Ptl/vJ+4"; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (imf12.hostedemail.com: domain of shy828301@gmail.com designates 209.85.214.177 as permitted sender) smtp.mailfrom=shy828301@gmail.com X-Rspamd-Server: rspam04 X-HE-Tag: 1653081353-756672 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: The khugepaged_{enabled|always|req_madv} are not khugepaged only anymore, move them to huge_mm.h and rename to hugepage_flags_xxx, and remove khugepaged_req_madv due to no users. Also move khugepaged_defrag to khugepaged.c since its only caller is in that file, it doesn't have to be in a header file. Signed-off-by: Yang Shi --- include/linux/huge_mm.h | 8 ++++++++ include/linux/khugepaged.h | 17 +---------------- mm/huge_memory.c | 4 ++-- mm/khugepaged.c | 18 +++++++++++------- 4 files changed, 22 insertions(+), 25 deletions(-) diff --git a/include/linux/huge_mm.h b/include/linux/huge_mm.h index 67c913e66e60..e6f89d8ad5f8 100644 --- a/include/linux/huge_mm.h +++ b/include/linux/huge_mm.h @@ -116,6 +116,14 @@ extern struct kobj_attribute shmem_enabled_attr; extern unsigned long transparent_hugepage_flags; +#define hugepage_flags_enabled() \ + (transparent_hugepage_flags & \ + ((1<flags)) @@ -53,8 +39,7 @@ static inline void khugepaged_exit(struct mm_struct *mm) static inline void khugepaged_enter(struct vm_area_struct *vma, unsigned long vm_flags) { - if (!test_bit(MMF_VM_HUGEPAGE, &vma->vm_mm->flags) && - khugepaged_enabled()) { + if (!test_bit(MMF_VM_HUGEPAGE, &vma->vm_mm->flags)) { if (hugepage_vma_check(vma, vm_flags, false)) __khugepaged_enter(vma->vm_mm); } diff --git a/mm/huge_memory.c b/mm/huge_memory.c index b4cab3aa2534..f4d7b6378b83 100644 --- a/mm/huge_memory.c +++ b/mm/huge_memory.c @@ -102,11 +102,11 @@ bool hugepage_vma_check(struct vm_area_struct *vma, if (!in_pf && shmem_file(vma->vm_file)) return shmem_huge_enabled(vma); - if (!khugepaged_enabled()) + if (!hugepage_flags_enabled()) return false; /* THP settings require madvise. */ - if (!(vm_flags & VM_HUGEPAGE) && !khugepaged_always()) + if (!(vm_flags & VM_HUGEPAGE) && !hugepage_flags_always()) return false; /* Only regular file is valid */ diff --git a/mm/khugepaged.c b/mm/khugepaged.c index 292ed9c0db1e..37ea5a5396cd 100644 --- a/mm/khugepaged.c +++ b/mm/khugepaged.c @@ -472,7 +472,7 @@ void khugepaged_enter_vma(struct vm_area_struct *vma, unsigned long vm_flags) { if (!test_bit(MMF_VM_HUGEPAGE, &vma->vm_mm->flags) && - khugepaged_enabled()) { + hugepage_flags_enabled()) { if (hugepage_vma_check(vma, vm_flags, false)) __khugepaged_enter(vma->vm_mm); } @@ -763,6 +763,10 @@ static bool khugepaged_scan_abort(int nid) return false; } +#define khugepaged_defrag() \ + (transparent_hugepage_flags & \ + (1<