From patchwork Thu Jun 2 18:21:53 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yang Shi X-Patchwork-Id: 12868042 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 71544C433EF for ; Thu, 2 Jun 2022 18:22:07 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 0A15E6B0073; Thu, 2 Jun 2022 14:22:07 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 0066A6B0074; Thu, 2 Jun 2022 14:22:06 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id DBD286B0075; Thu, 2 Jun 2022 14:22:06 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0014.hostedemail.com [216.40.44.14]) by kanga.kvack.org (Postfix) with ESMTP id CDFE36B0073 for ; Thu, 2 Jun 2022 14:22:06 -0400 (EDT) Received: from smtpin16.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay11.hostedemail.com (Postfix) with ESMTP id 9CECA813E2 for ; Thu, 2 Jun 2022 18:22:06 +0000 (UTC) X-FDA: 79534115052.16.24204AF Received: from mail-pl1-f172.google.com (mail-pl1-f172.google.com [209.85.214.172]) by imf02.hostedemail.com (Postfix) with ESMTP id 795A680076 for ; Thu, 2 Jun 2022 18:22:00 +0000 (UTC) Received: by mail-pl1-f172.google.com with SMTP id q18so5139162pln.12 for ; Thu, 02 Jun 2022 11:22:05 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=SWErXY/21cRT+JuPcizRjGDN5I2BQSMNvqCbtDuuEgI=; b=OoXAfZEZ4MNYzgwa1mxiiNLYILQhuzvKYToutAR1jjUYvKsN/qDaq0gORFJAw9/cMc Swt3hbAIAH81YoqZPRe0tQShdki9kmu9D8ux7uQVtefn1LynwDEYYY4W7FQXp9cZiw6z RbDJFDdvFyH8GP+u4N4gMypVi6x6+gxi0ewd3+HgSj8ymzFH8MzRlbC/kzV13iEMGwmH Lp8clnYaxcOFH1CqANZ7k/LKzCNXs1qiY+fd9Yu1zJ1EjGGrcosgC5Xzvp2gBS0EHOpE 5JOBXFQqT2zKdvyP2JZh6k9aG87muIeU40osHXul4peFUx4vhf4iBwzGpqaX2iKMR8AS Nraw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=SWErXY/21cRT+JuPcizRjGDN5I2BQSMNvqCbtDuuEgI=; b=ZlCW/ywRhKK1TBpgdx7kyvkn+auPwGsQe8E0hrOO7HDTg3wLlZxmDM3DUFR9WSpzcv JXVZuXej4WgcUoMUXqOQQ80U2aQ3L2cMBiJW6d24XYaJ27Pp143IRX7Q5CpuSwNLUzPZ sHlrrJQsF2O/C+PHLPvHnBeVw9uEDl9Nvrr74GtSP0rOdS8mNE6NxIlqVeqPbW2f6/Io 35Lt4JDlLDaQQKCWBjwuFs/NXAiEJZE4JqUcR8kve6bBl6xurjhO2s0uRx1oTmdTbCDx ycvzRpX1UI4CFQovPag7z4RXY+b27aDsn/55+r2C2nOFgtjTAeU4Mp9RdGurKbCfJjsH H1qw== X-Gm-Message-State: AOAM532ywzSomiq2YyPjfDbvvl0kEW+PAfxF2jBMJJOb0rdZPiAp149j h3CDXs0vWKKhzAxCD6d7eD0= X-Google-Smtp-Source: ABdhPJzC7OcR42WCXQLvw7aP2hx+qZcdKxnU7F7jgc1gw7m3PVF8cbDihuLNflGU/XhxlYAkH0vxSQ== X-Received: by 2002:a17:903:1108:b0:156:73a7:7c1 with SMTP id n8-20020a170903110800b0015673a707c1mr6113799plh.101.1654194125146; Thu, 02 Jun 2022 11:22:05 -0700 (PDT) Received: from localhost.localdomain (c-67-174-241-145.hsd1.ca.comcast.net. [67.174.241.145]) by smtp.gmail.com with ESMTPSA id t10-20020a1709027fca00b0015f2d549b46sm3866779plb.237.2022.06.02.11.22.04 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 02 Jun 2022 11:22:04 -0700 (PDT) From: Yang Shi To: vbabka@suse.cz, kirill.shutemov@linux.intel.com, akpm@linux-foundation.org Cc: shy828301@gmail.com, linux-mm@kvack.org, linux-kernel@vger.kernel.org Subject: [v2 PATCH 1/7] mm: khugepaged: check THP flag in hugepage_vma_check() Date: Thu, 2 Jun 2022 11:21:53 -0700 Message-Id: <20220602182159.248387-2-shy828301@gmail.com> X-Mailer: git-send-email 2.26.3 In-Reply-To: <20220602182159.248387-1-shy828301@gmail.com> References: <20220602182159.248387-1-shy828301@gmail.com> MIME-Version: 1.0 X-Rspam-User: X-Rspamd-Server: rspam02 X-Rspamd-Queue-Id: 795A680076 X-Stat-Signature: 8tc83z1aye3nqfaaa5fcqikqecnte1j6 Authentication-Results: imf02.hostedemail.com; dkim=pass header.d=gmail.com header.s=20210112 header.b=OoXAfZEZ; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (imf02.hostedemail.com: domain of shy828301@gmail.com designates 209.85.214.172 as permitted sender) smtp.mailfrom=shy828301@gmail.com X-HE-Tag: 1654194120-515004 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Currently the THP flag check in hugepage_vma_check() will fallthrough if the flag is NEVER and VM_HUGEPAGE is set. This is not a problem for now since all the callers have the flag checked before or can't be invoked if the flag is NEVER. However, the following patch will call hugepage_vma_check() in more places, for example, page fault, so this flag must be checked in hugepge_vma_check(). Signed-off-by: Yang Shi --- mm/khugepaged.c | 3 +++ 1 file changed, 3 insertions(+) diff --git a/mm/khugepaged.c b/mm/khugepaged.c index 671ac7800e53..84b9cf4b9be9 100644 --- a/mm/khugepaged.c +++ b/mm/khugepaged.c @@ -458,6 +458,9 @@ bool hugepage_vma_check(struct vm_area_struct *vma, if (shmem_file(vma->vm_file)) return shmem_huge_enabled(vma); + if (!khugepaged_enabled()) + return false; + /* THP settings require madvise. */ if (!(vm_flags & VM_HUGEPAGE) && !khugepaged_always()) return false; From patchwork Thu Jun 2 18:21:54 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yang Shi X-Patchwork-Id: 12868043 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id E44FEC43334 for ; Thu, 2 Jun 2022 18:22:08 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 645896B0074; Thu, 2 Jun 2022 14:22:08 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 5A7B06B0075; Thu, 2 Jun 2022 14:22:08 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 41AD06B0078; Thu, 2 Jun 2022 14:22:08 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0011.hostedemail.com [216.40.44.11]) by kanga.kvack.org (Postfix) with ESMTP id 2C2E46B0074 for ; Thu, 2 Jun 2022 14:22:08 -0400 (EDT) Received: from smtpin21.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay11.hostedemail.com (Postfix) with ESMTP id 0A589813E0 for ; Thu, 2 Jun 2022 18:22:08 +0000 (UTC) X-FDA: 79534115136.21.D7B130B Received: from mail-pl1-f175.google.com (mail-pl1-f175.google.com [209.85.214.175]) by imf29.hostedemail.com (Postfix) with ESMTP id 15B00120057 for ; Thu, 2 Jun 2022 18:21:53 +0000 (UTC) Received: by mail-pl1-f175.google.com with SMTP id n18so5170098plg.5 for ; Thu, 02 Jun 2022 11:22:07 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=D/C1dubPl+rj/RbGuMLBKXaMg54FT3nuKqdm5O29Xsg=; b=FRrND4uki4knPZ0gSWutNMQKlOfS5LJi4uIMfHYUiv+1a8lr0Xem3mWuTw7DfDlSY3 9tRImdO5zSIr4nPyQ8HcKICzyQCFdpFFWdXtHalaE6GGfEw4F98wdnuF73lw2uk6p0xH nr9CoqMB8CGi9J52LmD84U6h8P08cVzicro/KUR77FT60PiVBDVCTuWBX4zI70ktKx+N R6wDi3C47n7AWxDnJ1DtU604BHF7nH8hFZsXeJ9ieLzroSjoD+x/6sG7zHK5ArGZulnD axyJD0uVhxDY1agFrwfOHh8N38BgADp3plNxnDatVLAmFHHdJ0no0cbEE1Sw7JXJcci9 Ap/A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=D/C1dubPl+rj/RbGuMLBKXaMg54FT3nuKqdm5O29Xsg=; b=iE1a16CnElBr8q2UE319RPUBXhAinJeRKw9AE7a0bR+MPrHVOPoB2Z60cU90iqQYDE lJpHkQeHIuvXApZnlwatFLxcsdly1hcTqBAs+o072ZUrqtRn9AGGrP/VkijIT1LU0SeJ BlUbeWmXaA94L2Mhh/o86swfioDoGxb0mL0Rgf4t/pkvi1B6Y983ZfQkZHUCJHnKbyFO dj5MeA8fl3eZz0tqsfiRqp+PQVypqagAvXpyBDihw+pl/t4KUj7v5Ho42fLWrLxocuXm 7T6XSd8KspqT6zHJqO+SS8x+Vjc0rUaEJLsylnmuKZHy85X16mo8PRk8+r0Cb/ZhiTr6 zUig== X-Gm-Message-State: AOAM531TDC0N52GSi5bjjmR1Fb0+JZBPbXRuCJOVA16zymBEwmTA8lCW 7Q3vUKMxz/dX+pb5b6R1rrg= X-Google-Smtp-Source: ABdhPJzy/X13hVsWfZKyg71A7q6Qroq+HiX2lAOxkS2DH/C0df1y1IyvvBp0ImlM7TMfCrMNVybNWw== X-Received: by 2002:a17:902:7597:b0:15e:b5d3:267d with SMTP id j23-20020a170902759700b0015eb5d3267dmr6206049pll.55.1654194126432; Thu, 02 Jun 2022 11:22:06 -0700 (PDT) Received: from localhost.localdomain (c-67-174-241-145.hsd1.ca.comcast.net. [67.174.241.145]) by smtp.gmail.com with ESMTPSA id t10-20020a1709027fca00b0015f2d549b46sm3866779plb.237.2022.06.02.11.22.05 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 02 Jun 2022 11:22:05 -0700 (PDT) From: Yang Shi To: vbabka@suse.cz, kirill.shutemov@linux.intel.com, akpm@linux-foundation.org Cc: shy828301@gmail.com, linux-mm@kvack.org, linux-kernel@vger.kernel.org Subject: [v2 PATCH 2/7] mm: thp: introduce transhuge_vma_size_ok() helper Date: Thu, 2 Jun 2022 11:21:54 -0700 Message-Id: <20220602182159.248387-3-shy828301@gmail.com> X-Mailer: git-send-email 2.26.3 In-Reply-To: <20220602182159.248387-1-shy828301@gmail.com> References: <20220602182159.248387-1-shy828301@gmail.com> MIME-Version: 1.0 X-Rspamd-Server: rspam01 X-Rspamd-Queue-Id: 15B00120057 Authentication-Results: imf29.hostedemail.com; dkim=pass header.d=gmail.com header.s=20210112 header.b=FRrND4uk; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (imf29.hostedemail.com: domain of shy828301@gmail.com designates 209.85.214.175 as permitted sender) smtp.mailfrom=shy828301@gmail.com X-Stat-Signature: 1a7az64focx743z9zns98pokn5sj6o1w X-Rspam-User: X-HE-Tag: 1654194113-889162 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: There are couple of places that check whether the vma size is ok for THP or not, they are open coded and duplicate, introduce transhuge_vma_size_ok() helper to do the job. Signed-off-by: Yang Shi --- include/linux/huge_mm.h | 17 +++++++++++++++++ mm/huge_memory.c | 5 +---- mm/khugepaged.c | 12 ++++++------ 3 files changed, 24 insertions(+), 10 deletions(-) diff --git a/include/linux/huge_mm.h b/include/linux/huge_mm.h index 648cb3ce7099..a8f61db47f2a 100644 --- a/include/linux/huge_mm.h +++ b/include/linux/huge_mm.h @@ -116,6 +116,18 @@ extern struct kobj_attribute shmem_enabled_attr; extern unsigned long transparent_hugepage_flags; +/* + * The vma size has to be large enough to hold an aligned HPAGE_PMD_SIZE area. + */ +static inline bool transhuge_vma_size_ok(struct vm_area_struct *vma) +{ + if (round_up(vma->vm_start, HPAGE_PMD_SIZE) < + (vma->vm_end & HPAGE_PMD_MASK)) + return true; + + return false; +} + static inline bool transhuge_vma_suitable(struct vm_area_struct *vma, unsigned long addr) { @@ -345,6 +357,11 @@ static inline bool transparent_hugepage_active(struct vm_area_struct *vma) return false; } +static inline bool transhuge_vma_size_ok(struct vm_area_struct *vma) +{ + return false; +} + static inline bool transhuge_vma_suitable(struct vm_area_struct *vma, unsigned long addr) { diff --git a/mm/huge_memory.c b/mm/huge_memory.c index 48182c8fe151..36ada544e494 100644 --- a/mm/huge_memory.c +++ b/mm/huge_memory.c @@ -71,10 +71,7 @@ unsigned long huge_zero_pfn __read_mostly = ~0UL; bool transparent_hugepage_active(struct vm_area_struct *vma) { - /* The addr is used to check if the vma size fits */ - unsigned long addr = (vma->vm_end & HPAGE_PMD_MASK) - HPAGE_PMD_SIZE; - - if (!transhuge_vma_suitable(vma, addr)) + if (!transhuge_vma_size_ok(vma)) return false; if (vma_is_anonymous(vma)) return __transparent_hugepage_enabled(vma); diff --git a/mm/khugepaged.c b/mm/khugepaged.c index 84b9cf4b9be9..d0f8020164fc 100644 --- a/mm/khugepaged.c +++ b/mm/khugepaged.c @@ -454,6 +454,9 @@ bool hugepage_vma_check(struct vm_area_struct *vma, vma->vm_pgoff, HPAGE_PMD_NR)) return false; + if (!transhuge_vma_size_ok(vma)) + return false; + /* Enabled via shmem mount options or sysfs settings. */ if (shmem_file(vma->vm_file)) return shmem_huge_enabled(vma); @@ -512,9 +515,7 @@ void khugepaged_enter_vma(struct vm_area_struct *vma, unsigned long vm_flags) { if (!test_bit(MMF_VM_HUGEPAGE, &vma->vm_mm->flags) && - khugepaged_enabled() && - (((vma->vm_start + ~HPAGE_PMD_MASK) & HPAGE_PMD_MASK) < - (vma->vm_end & HPAGE_PMD_MASK))) { + khugepaged_enabled()) { if (hugepage_vma_check(vma, vm_flags)) __khugepaged_enter(vma->vm_mm); } @@ -2142,10 +2143,9 @@ static unsigned int khugepaged_scan_mm_slot(unsigned int pages, progress++; continue; } - hstart = (vma->vm_start + ~HPAGE_PMD_MASK) & HPAGE_PMD_MASK; + + hstart = round_up(vma->vm_start, HPAGE_PMD_SIZE); hend = vma->vm_end & HPAGE_PMD_MASK; - if (hstart >= hend) - goto skip; if (khugepaged_scan.address > hend) goto skip; if (khugepaged_scan.address < hstart) From patchwork Thu Jun 2 18:21:55 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yang Shi X-Patchwork-Id: 12868044 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 92B91CCA47B for ; Thu, 2 Jun 2022 18:22:10 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id C5D956B0075; Thu, 2 Jun 2022 14:22:09 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id B47C86B0078; Thu, 2 Jun 2022 14:22:09 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 9E6176B007B; Thu, 2 Jun 2022 14:22:09 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0013.hostedemail.com [216.40.44.13]) by kanga.kvack.org (Postfix) with ESMTP id 884BE6B0075 for ; Thu, 2 Jun 2022 14:22:09 -0400 (EDT) Received: from smtpin15.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay11.hostedemail.com (Postfix) with ESMTP id 3E225813D5 for ; Thu, 2 Jun 2022 18:22:09 +0000 (UTC) X-FDA: 79534115178.15.CF4358A Received: from mail-pj1-f43.google.com (mail-pj1-f43.google.com [209.85.216.43]) by imf03.hostedemail.com (Postfix) with ESMTP id 8BDA420056 for ; Thu, 2 Jun 2022 18:21:53 +0000 (UTC) Received: by mail-pj1-f43.google.com with SMTP id w2-20020a17090ac98200b001e0519fe5a8so5492898pjt.4 for ; Thu, 02 Jun 2022 11:22:08 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=TeR+saO9bGVSVDYLkT6whuJkztc98Y7uqJRw9Zwr6CA=; b=lox+mE5DUQIYDSINDMGLvp+f+4p6bG1alTdnY/p5yaPQT5MhcSCKJI+8s/kk/vYrgz FmcQS7G4CMRDlFwl2en89XVTfwF/hAIbYQkKoMHAdDWhFIaHkqtdpSXvA6PBc63CpxCI QdpyQZOXfOltvUHMi4sMlFeaVN4KN8g5IAllCh/D2rckkxATCgwcej4OmeqCVYOwgUG6 Hp6eQFXuIu0CX2Z+YMXbCI3f+zfr9qllgXYuEBQxqBuLFGxngc/1cMy4KaQscxiRzwMg gUYcE+aF5Qyi06JBD45s6qomtBFmAr3AdtDn3TQFbQkrcOfkcvp4efLZuD1aC1dcrnxH AqdA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=TeR+saO9bGVSVDYLkT6whuJkztc98Y7uqJRw9Zwr6CA=; b=5zBznr2SFm6Agt/55uHAXx5wXLk+7CQVFWoi9vm6kFjZKVA5AZbES9QuILRd7PVdFA vc3GFoyEixkN3+VZ/+rmaW+GHjw908AXJ0r02zjFobBxxzZSipZaOoVYgk2LUKJ11MFd iyWiy+3ZpiNCgSSWciDwFm2XG2mrQjNqJmiQL58S+X6Qz+6NyU3FhrtElX3zxRRxHNGX LxoQS2MYd5AehDncRISKMbftJiwQ2UsRyoZcPZOlzcpSAoKSMBYiK9Bj5IorzwxgM9k+ V0qJgxCBER6tIIujZU0Y4vmn5PCC0rmylr38ngFPyCTlSHR/Twqrkw0LIIzLGxoCTVtQ U6gg== X-Gm-Message-State: AOAM5326v70JvrJoPEcRuDyQcvwFEuKZ2/IoogusCL/j4Bd3IP9duIKB gr1kL5ZYzLAF3oyOJaHeCvo= X-Google-Smtp-Source: ABdhPJwUloxNPQ1Kt3swhvWRsLalZE4j17qsysUEVMeRb96vd1+i3T7NbIkG0a3KYeFbHoyp7u9MwA== X-Received: by 2002:a17:90b:3b4c:b0:1e0:3354:5e5d with SMTP id ot12-20020a17090b3b4c00b001e033545e5dmr41812977pjb.239.1654194127979; Thu, 02 Jun 2022 11:22:07 -0700 (PDT) Received: from localhost.localdomain (c-67-174-241-145.hsd1.ca.comcast.net. [67.174.241.145]) by smtp.gmail.com with ESMTPSA id t10-20020a1709027fca00b0015f2d549b46sm3866779plb.237.2022.06.02.11.22.06 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 02 Jun 2022 11:22:07 -0700 (PDT) From: Yang Shi To: vbabka@suse.cz, kirill.shutemov@linux.intel.com, akpm@linux-foundation.org Cc: shy828301@gmail.com, linux-mm@kvack.org, linux-kernel@vger.kernel.org Subject: [v2 PATCH 3/7] mm: khugepaged: remove the redundant anon vma check Date: Thu, 2 Jun 2022 11:21:55 -0700 Message-Id: <20220602182159.248387-4-shy828301@gmail.com> X-Mailer: git-send-email 2.26.3 In-Reply-To: <20220602182159.248387-1-shy828301@gmail.com> References: <20220602182159.248387-1-shy828301@gmail.com> MIME-Version: 1.0 Authentication-Results: imf03.hostedemail.com; dkim=pass header.d=gmail.com header.s=20210112 header.b=lox+mE5D; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (imf03.hostedemail.com: domain of shy828301@gmail.com designates 209.85.216.43 as permitted sender) smtp.mailfrom=shy828301@gmail.com X-Rspamd-Server: rspam03 X-Rspamd-Queue-Id: 8BDA420056 X-Rspam-User: X-Stat-Signature: ctwxshcy51739ua7oxmxk36rs6hu5bue X-HE-Tag: 1654194113-418909 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: The hugepage_vma_check() already checked it, so remove the redundant check. Signed-off-by: Yang Shi --- mm/khugepaged.c | 3 --- 1 file changed, 3 deletions(-) diff --git a/mm/khugepaged.c b/mm/khugepaged.c index d0f8020164fc..7a5d1c1a1833 100644 --- a/mm/khugepaged.c +++ b/mm/khugepaged.c @@ -966,9 +966,6 @@ static int hugepage_vma_revalidate(struct mm_struct *mm, unsigned long address, return SCAN_ADDRESS_RANGE; if (!hugepage_vma_check(vma, vma->vm_flags)) return SCAN_VMA_CHECK; - /* Anon VMA expected */ - if (!vma->anon_vma || !vma_is_anonymous(vma)) - return SCAN_VMA_CHECK; return 0; } From patchwork Thu Jun 2 18:21:56 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yang Shi X-Patchwork-Id: 12868045 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 21172C43334 for ; Thu, 2 Jun 2022 18:22:12 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 3CD516B0078; Thu, 2 Jun 2022 14:22:11 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 32CC36B007B; Thu, 2 Jun 2022 14:22:11 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 1C9696B007D; Thu, 2 Jun 2022 14:22:11 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0015.hostedemail.com [216.40.44.15]) by kanga.kvack.org (Postfix) with ESMTP id 01A2B6B0078 for ; Thu, 2 Jun 2022 14:22:10 -0400 (EDT) Received: from smtpin29.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay13.hostedemail.com (Postfix) with ESMTP id B77F4613E6 for ; Thu, 2 Jun 2022 18:22:10 +0000 (UTC) X-FDA: 79534115220.29.2234805 Received: from mail-pf1-f175.google.com (mail-pf1-f175.google.com [209.85.210.175]) by imf30.hostedemail.com (Postfix) with ESMTP id C234A80060 for ; Thu, 2 Jun 2022 18:21:34 +0000 (UTC) Received: by mail-pf1-f175.google.com with SMTP id c196so5438667pfb.1 for ; Thu, 02 Jun 2022 11:22:10 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=Ij7n7UZO/IK0zKEVoAHKDokBQE/6XWgjwPeS6gPTqcc=; b=IpMzaIeFOsyZY5RirkJatHmncMuZPPqDAIcHJBZPIVp7G09X1sx9pTdtZ7RdyauoYu CXIXH82Up2DBdc5z5bS621MpSvaQAGhEYPODX5fWKNzCB4RQkpwyu2MPYjFnd6ml7115 Nwn/88YW4mU+XngygZGDRSvug40didfLWbcdd349F5mSLAuviIyjwBV6PRONP4z47m0x FPA/CgZSc2y/sQh3LXmJ9s9TVq2jz2mvYgIZe77eWdrLNR2mbFcu5BYQ9WCsR1D/zQfL 5PPDk7dZs5AT4EQhjDnYpA8iC6vJaBbnXsV2xL/9ZDOr3dh5+4P3HEM8j12kvAV/h3wN 5ysA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=Ij7n7UZO/IK0zKEVoAHKDokBQE/6XWgjwPeS6gPTqcc=; b=fAayFfC6aJwMnkaD1ATzEa5zt2yVdFMVIFPFhgCFfGAy5RjdwzvEXV7ed+ftelNxSf 63W+liLOQxvhJOCthvMavDaHW/VVhpZlPsykzfDSn3FME7r8dgn2gpYozgRmwfPFnfjr dNP73cl9DPvqukh30KLYNbMcH4D+LFNVt4tQIZf315Zle0200kdxAZIY6SEF4ZyW25wc l59b08MzPhaD+DN99Bv9S/h4xM4ng0Ylw1mJEUCb0XDY7zQ+4TqzkhvXjOzS6YPDDDb1 cVhYbJwCftg+YvWQ4vFhY7y2+ZDJk54eHTtGyzZ63oFi+6uhprIvufp/5QAEwdd0iwJk kBLg== X-Gm-Message-State: AOAM533xm55Ww/EMyMtyEfVqskvR+Iwu36XkbAfZFwx6TPuJ2/aqWocz lcyV5QmGzVzpE9spmHPfHzk= X-Google-Smtp-Source: ABdhPJyLtH0blR5AqKJCr/dkM/hu+mh+rj2O4bSVPpE5tGGO/EHu3X2wtkjB9koaOEl7+HMO4L+bWQ== X-Received: by 2002:a05:6a00:179e:b0:518:9e1d:1cbd with SMTP id s30-20020a056a00179e00b005189e1d1cbdmr54800149pfg.12.1654194129509; Thu, 02 Jun 2022 11:22:09 -0700 (PDT) Received: from localhost.localdomain (c-67-174-241-145.hsd1.ca.comcast.net. [67.174.241.145]) by smtp.gmail.com with ESMTPSA id t10-20020a1709027fca00b0015f2d549b46sm3866779plb.237.2022.06.02.11.22.08 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 02 Jun 2022 11:22:08 -0700 (PDT) From: Yang Shi To: vbabka@suse.cz, kirill.shutemov@linux.intel.com, akpm@linux-foundation.org Cc: shy828301@gmail.com, linux-mm@kvack.org, linux-kernel@vger.kernel.org Subject: [v2 PATCH 4/7] mm: khugepaged: use transhuge_vma_suitable replace open-code Date: Thu, 2 Jun 2022 11:21:56 -0700 Message-Id: <20220602182159.248387-5-shy828301@gmail.com> X-Mailer: git-send-email 2.26.3 In-Reply-To: <20220602182159.248387-1-shy828301@gmail.com> References: <20220602182159.248387-1-shy828301@gmail.com> MIME-Version: 1.0 X-Stat-Signature: zccydj9d5pk746canhcbugxhs7fzrets X-Rspam-User: Authentication-Results: imf30.hostedemail.com; dkim=pass header.d=gmail.com header.s=20210112 header.b=IpMzaIeF; spf=pass (imf30.hostedemail.com: domain of shy828301@gmail.com designates 209.85.210.175 as permitted sender) smtp.mailfrom=shy828301@gmail.com; dmarc=pass (policy=none) header.from=gmail.com X-Rspamd-Server: rspam12 X-Rspamd-Queue-Id: C234A80060 X-HE-Tag: 1654194094-449109 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: The hugepage_vma_revalidate() needs to check if the address is still in the aligned HPAGE_PMD_SIZE area of the vma when reacquiring mmap_lock, but it was open-coded, use transhuge_vma_suitable() to do the job. And add proper comments for transhuge_vma_suitable(). Signed-off-by: Yang Shi --- include/linux/huge_mm.h | 6 ++++++ mm/khugepaged.c | 5 +---- 2 files changed, 7 insertions(+), 4 deletions(-) diff --git a/include/linux/huge_mm.h b/include/linux/huge_mm.h index a8f61db47f2a..79d5919beb83 100644 --- a/include/linux/huge_mm.h +++ b/include/linux/huge_mm.h @@ -128,6 +128,12 @@ static inline bool transhuge_vma_size_ok(struct vm_area_struct *vma) return false; } +/* + * Do the below checks: + * - For non-anon vma, check if the vm_pgoff is HPAGE_PMD_NR aligned. + * - For all vmas, check if the haddr is in an aligned HPAGE_PMD_SIZE + * area. + */ static inline bool transhuge_vma_suitable(struct vm_area_struct *vma, unsigned long addr) { diff --git a/mm/khugepaged.c b/mm/khugepaged.c index 7a5d1c1a1833..ca1754d3a827 100644 --- a/mm/khugepaged.c +++ b/mm/khugepaged.c @@ -951,7 +951,6 @@ static int hugepage_vma_revalidate(struct mm_struct *mm, unsigned long address, struct vm_area_struct **vmap) { struct vm_area_struct *vma; - unsigned long hstart, hend; if (unlikely(khugepaged_test_exit(mm))) return SCAN_ANY_PROCESS; @@ -960,9 +959,7 @@ static int hugepage_vma_revalidate(struct mm_struct *mm, unsigned long address, if (!vma) return SCAN_VMA_NULL; - hstart = (vma->vm_start + ~HPAGE_PMD_MASK) & HPAGE_PMD_MASK; - hend = vma->vm_end & HPAGE_PMD_MASK; - if (address < hstart || address + HPAGE_PMD_SIZE > hend) + if (!transhuge_vma_suitable(vma, address)) return SCAN_ADDRESS_RANGE; if (!hugepage_vma_check(vma, vma->vm_flags)) return SCAN_VMA_CHECK; From patchwork Thu Jun 2 18:21:57 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yang Shi X-Patchwork-Id: 12868046 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 93F28CCA47E for ; Thu, 2 Jun 2022 18:22:13 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 9E5D36B007B; Thu, 2 Jun 2022 14:22:12 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 944546B007D; Thu, 2 Jun 2022 14:22:12 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 78F656B007E; Thu, 2 Jun 2022 14:22:12 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0015.hostedemail.com [216.40.44.15]) by kanga.kvack.org (Postfix) with ESMTP id 699A96B007B for ; Thu, 2 Jun 2022 14:22:12 -0400 (EDT) Received: from smtpin24.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay12.hostedemail.com (Postfix) with ESMTP id 46E71121359 for ; Thu, 2 Jun 2022 18:22:12 +0000 (UTC) X-FDA: 79534115304.24.D372F1F Received: from mail-pj1-f54.google.com (mail-pj1-f54.google.com [209.85.216.54]) by imf27.hostedemail.com (Postfix) with ESMTP id DE2EC4005C for ; Thu, 2 Jun 2022 18:22:07 +0000 (UTC) Received: by mail-pj1-f54.google.com with SMTP id u12-20020a17090a1d4c00b001df78c7c209so10201802pju.1 for ; Thu, 02 Jun 2022 11:22:11 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=3YLj2fl29PQ2nWLBeslxp0V88dJtNL8lfA8pD6oU8q8=; b=Rf/b23jKN1mypu5L/BY5+gkApgLOkO60VjFloJqDSSOi3Zp7C9ySYekBmK9b+UhJN5 3wP3DcExFTbzWgAPhJ6F5nMjsgYO1nGOIqU4X/r6K4WYiPi55lHjxU5j2eZopyZ5pYT4 tluPHDRvLNEzwN3PvPDeTqp58jkBh6aQfHVphPNX5Dtt1C6JC5ZQyOyRzfPQjzD3GTug 5TvTWnw0L6/M4dx3JmMLOgRIvNWbJgzWE4tPhp1aGVGl2boPcGIYYRoIfwf6uLA19nvb bGEgYAdJyApHIoBPJWMwj/4MvL7hZPg1WhL6q2S8NGzXd0p0RpgA5wow27UJkPtcz3Hy 3YSw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=3YLj2fl29PQ2nWLBeslxp0V88dJtNL8lfA8pD6oU8q8=; b=WhAUtkxcpPUuENTMRf94yUnAcx2APPKbIXWb8idn2NVpm8AAdv0obSVK8ZGU0Wzymi 63q85uWUW+NvisGdMnlOfJmKwccn0SzrbtkaHKJopHbnc76iFWk0kmsdawTWnfqzVOdf eUDO7TDQMyMKGeP3qQFO4BeTT6q4t+bY6OfPTNaEa9vqh6v96GSO7dWwKN3bVZ9xjz5g yUx0/YBFNRIWvr5D4VLlf+BQiUv9cd9ijDt+0wW7IzfauTf5tudX8qVY2RO+MlU6isSL mwiqSsMhsH/6N3hR+Ea5wGK2MJrizisMt4Pgv0r9I87CX3gkJN8WU9pJoqt+8dS9RP2q /IKw== X-Gm-Message-State: AOAM531RFc2hb/26WC6RN8rotGBHBIwiYbfO7CpLvi+1F3czvDjkR1qy dwPwgGsFkcZMVEF003iSthwFw1SRXrw= X-Google-Smtp-Source: ABdhPJyclp/ibM+hVcPzkUg9jkHC7l2XJyrVQn117jPC5u/X7fUVVnVA5aK3MtI5vqtPAMF4aWzcsw== X-Received: by 2002:a17:90b:3841:b0:1e2:f16a:a117 with SMTP id nl1-20020a17090b384100b001e2f16aa117mr25691608pjb.130.1654194130914; Thu, 02 Jun 2022 11:22:10 -0700 (PDT) Received: from localhost.localdomain (c-67-174-241-145.hsd1.ca.comcast.net. [67.174.241.145]) by smtp.gmail.com with ESMTPSA id t10-20020a1709027fca00b0015f2d549b46sm3866779plb.237.2022.06.02.11.22.09 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 02 Jun 2022 11:22:10 -0700 (PDT) From: Yang Shi To: vbabka@suse.cz, kirill.shutemov@linux.intel.com, akpm@linux-foundation.org Cc: shy828301@gmail.com, linux-mm@kvack.org, linux-kernel@vger.kernel.org Subject: [v2 PATCH 5/7] mm: thp: kill transparent_hugepage_active() Date: Thu, 2 Jun 2022 11:21:57 -0700 Message-Id: <20220602182159.248387-6-shy828301@gmail.com> X-Mailer: git-send-email 2.26.3 In-Reply-To: <20220602182159.248387-1-shy828301@gmail.com> References: <20220602182159.248387-1-shy828301@gmail.com> MIME-Version: 1.0 Authentication-Results: imf27.hostedemail.com; dkim=pass header.d=gmail.com header.s=20210112 header.b="Rf/b23jK"; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (imf27.hostedemail.com: domain of shy828301@gmail.com designates 209.85.216.54 as permitted sender) smtp.mailfrom=shy828301@gmail.com X-Rspamd-Server: rspam03 X-Rspamd-Queue-Id: DE2EC4005C X-Rspam-User: X-Stat-Signature: i69zw8bkryhddy7rsh9fgi9cbx41tx5z X-HE-Tag: 1654194127-666026 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: The transparent_hugepage_active() was introduced to show THP eligibility bit in smaps in proc, smaps is the only user. But it actually does the similar check as hugepage_vma_check() which is used by khugepaged. We definitely don't have to maintain two similar checks, so kill transparent_hugepage_active(). Also move hugepage_vma_check() to huge_memory.c and huge_mm.h since it is not only for khugepaged anymore. Signed-off-by: Yang Shi --- fs/proc/task_mmu.c | 2 +- include/linux/huge_mm.h | 16 +++++++----- include/linux/khugepaged.h | 4 +-- mm/huge_memory.c | 50 ++++++++++++++++++++++++++++++++----- mm/khugepaged.c | 51 +++----------------------------------- 5 files changed, 60 insertions(+), 63 deletions(-) diff --git a/fs/proc/task_mmu.c b/fs/proc/task_mmu.c index 2dd8c8a66924..fd79566e204c 100644 --- a/fs/proc/task_mmu.c +++ b/fs/proc/task_mmu.c @@ -860,7 +860,7 @@ static int show_smap(struct seq_file *m, void *v) __show_smap(m, &mss, false); seq_printf(m, "THPeligible: %d\n", - transparent_hugepage_active(vma)); + hugepage_vma_check(vma, vma->vm_flags, true)); if (arch_pkeys_enabled()) seq_printf(m, "ProtectionKey: %8u\n", vma_pkey(vma)); diff --git a/include/linux/huge_mm.h b/include/linux/huge_mm.h index 79d5919beb83..f315c70d5256 100644 --- a/include/linux/huge_mm.h +++ b/include/linux/huge_mm.h @@ -209,7 +209,9 @@ static inline bool file_thp_enabled(struct vm_area_struct *vma) !inode_is_open_for_write(inode) && S_ISREG(inode->i_mode); } -bool transparent_hugepage_active(struct vm_area_struct *vma); +extern bool hugepage_vma_check(struct vm_area_struct *vma, + unsigned long vm_flags, + bool smaps); #define transparent_hugepage_use_zero_page() \ (transparent_hugepage_flags & \ @@ -358,11 +360,6 @@ static inline bool __transparent_hugepage_enabled(struct vm_area_struct *vma) return false; } -static inline bool transparent_hugepage_active(struct vm_area_struct *vma) -{ - return false; -} - static inline bool transhuge_vma_size_ok(struct vm_area_struct *vma) { return false; @@ -380,6 +377,13 @@ static inline bool transhuge_vma_enabled(struct vm_area_struct *vma, return false; } +static inline bool hugepage_vma_check(struct vm_area_struct *vma, + unsigned long vm_flags, + bool smaps) +{ + return false; +} + static inline void prep_transhuge_page(struct page *page) {} #define transparent_hugepage_flags 0UL diff --git a/include/linux/khugepaged.h b/include/linux/khugepaged.h index 392d34c3c59a..8a6452e089ca 100644 --- a/include/linux/khugepaged.h +++ b/include/linux/khugepaged.h @@ -10,8 +10,6 @@ extern struct attribute_group khugepaged_attr_group; extern int khugepaged_init(void); extern void khugepaged_destroy(void); extern int start_stop_khugepaged(void); -extern bool hugepage_vma_check(struct vm_area_struct *vma, - unsigned long vm_flags); extern void __khugepaged_enter(struct mm_struct *mm); extern void __khugepaged_exit(struct mm_struct *mm); extern void khugepaged_enter_vma(struct vm_area_struct *vma, @@ -57,7 +55,7 @@ static inline void khugepaged_enter(struct vm_area_struct *vma, { if (!test_bit(MMF_VM_HUGEPAGE, &vma->vm_mm->flags) && khugepaged_enabled()) { - if (hugepage_vma_check(vma, vm_flags)) + if (hugepage_vma_check(vma, vm_flags, false)) __khugepaged_enter(vma->vm_mm); } } diff --git a/mm/huge_memory.c b/mm/huge_memory.c index 36ada544e494..bc8370856e85 100644 --- a/mm/huge_memory.c +++ b/mm/huge_memory.c @@ -69,18 +69,56 @@ static atomic_t huge_zero_refcount; struct page *huge_zero_page __read_mostly; unsigned long huge_zero_pfn __read_mostly = ~0UL; -bool transparent_hugepage_active(struct vm_area_struct *vma) +bool hugepage_vma_check(struct vm_area_struct *vma, + unsigned long vm_flags, + bool smaps) { + if (!transhuge_vma_enabled(vma, vm_flags)) + return false; + + if (vm_flags & VM_NO_KHUGEPAGED) + return false; + + /* Don't run khugepaged against DAX vma */ + if (vma_is_dax(vma)) + return false; + + if (vma->vm_file && !IS_ALIGNED((vma->vm_start >> PAGE_SHIFT) - + vma->vm_pgoff, HPAGE_PMD_NR)) + return false; + if (!transhuge_vma_size_ok(vma)) return false; - if (vma_is_anonymous(vma)) - return __transparent_hugepage_enabled(vma); - if (vma_is_shmem(vma)) + + /* Enabled via shmem mount options or sysfs settings. */ + if (shmem_file(vma->vm_file)) return shmem_huge_enabled(vma); - if (transhuge_vma_enabled(vma, vma->vm_flags) && file_thp_enabled(vma)) + + if (!khugepaged_enabled()) + return false; + + /* THP settings require madvise. */ + if (!(vm_flags & VM_HUGEPAGE) && !khugepaged_always()) + return false; + + /* Only regular file is valid */ + if (file_thp_enabled(vma)) return true; - return false; + if (!vma_is_anonymous(vma)) + return false; + + if (vma_is_temporary_stack(vma)) + return false; + + /* + * THPeligible bit of smaps should show 1 for proper VMAs even + * though anon_vma is not initialized yet. + */ + if (!vma->anon_vma) + return smaps; + + return true; } static bool get_huge_zero_page(void) diff --git a/mm/khugepaged.c b/mm/khugepaged.c index ca1754d3a827..aa0769e3b0d9 100644 --- a/mm/khugepaged.c +++ b/mm/khugepaged.c @@ -437,49 +437,6 @@ static inline int khugepaged_test_exit(struct mm_struct *mm) return atomic_read(&mm->mm_users) == 0; } -bool hugepage_vma_check(struct vm_area_struct *vma, - unsigned long vm_flags) -{ - if (!transhuge_vma_enabled(vma, vm_flags)) - return false; - - if (vm_flags & VM_NO_KHUGEPAGED) - return false; - - /* Don't run khugepaged against DAX vma */ - if (vma_is_dax(vma)) - return false; - - if (vma->vm_file && !IS_ALIGNED((vma->vm_start >> PAGE_SHIFT) - - vma->vm_pgoff, HPAGE_PMD_NR)) - return false; - - if (!transhuge_vma_size_ok(vma)) - return false; - - /* Enabled via shmem mount options or sysfs settings. */ - if (shmem_file(vma->vm_file)) - return shmem_huge_enabled(vma); - - if (!khugepaged_enabled()) - return false; - - /* THP settings require madvise. */ - if (!(vm_flags & VM_HUGEPAGE) && !khugepaged_always()) - return false; - - /* Only regular file is valid */ - if (file_thp_enabled(vma)) - return true; - - if (!vma->anon_vma || !vma_is_anonymous(vma)) - return false; - if (vma_is_temporary_stack(vma)) - return false; - - return true; -} - void __khugepaged_enter(struct mm_struct *mm) { struct mm_slot *mm_slot; @@ -516,7 +473,7 @@ void khugepaged_enter_vma(struct vm_area_struct *vma, { if (!test_bit(MMF_VM_HUGEPAGE, &vma->vm_mm->flags) && khugepaged_enabled()) { - if (hugepage_vma_check(vma, vm_flags)) + if (hugepage_vma_check(vma, vm_flags, false)) __khugepaged_enter(vma->vm_mm); } } @@ -961,7 +918,7 @@ static int hugepage_vma_revalidate(struct mm_struct *mm, unsigned long address, if (!transhuge_vma_suitable(vma, address)) return SCAN_ADDRESS_RANGE; - if (!hugepage_vma_check(vma, vma->vm_flags)) + if (!hugepage_vma_check(vma, vma->vm_flags, false)) return SCAN_VMA_CHECK; return 0; } @@ -1442,7 +1399,7 @@ void collapse_pte_mapped_thp(struct mm_struct *mm, unsigned long addr) * the valid THP. Add extra VM_HUGEPAGE so hugepage_vma_check() * will not fail the vma for missing VM_HUGEPAGE */ - if (!hugepage_vma_check(vma, vma->vm_flags | VM_HUGEPAGE)) + if (!hugepage_vma_check(vma, vma->vm_flags | VM_HUGEPAGE, false)) return; /* Keep pmd pgtable for uffd-wp; see comment in retract_page_tables() */ @@ -2132,7 +2089,7 @@ static unsigned int khugepaged_scan_mm_slot(unsigned int pages, progress++; break; } - if (!hugepage_vma_check(vma, vma->vm_flags)) { + if (!hugepage_vma_check(vma, vma->vm_flags, false)) { skip: progress++; continue; From patchwork Thu Jun 2 18:21:58 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yang Shi X-Patchwork-Id: 12868047 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 34C2AC43334 for ; Thu, 2 Jun 2022 18:22:15 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 4B30F6B007D; Thu, 2 Jun 2022 14:22:14 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 440046B007E; Thu, 2 Jun 2022 14:22:14 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 213506B0080; Thu, 2 Jun 2022 14:22:14 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0016.hostedemail.com [216.40.44.16]) by kanga.kvack.org (Postfix) with ESMTP id 12D196B007D for ; Thu, 2 Jun 2022 14:22:14 -0400 (EDT) Received: from smtpin24.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay02.hostedemail.com (Postfix) with ESMTP id DB3D334BD5 for ; Thu, 2 Jun 2022 18:22:13 +0000 (UTC) X-FDA: 79534115346.24.1FB528C Received: from mail-pg1-f181.google.com (mail-pg1-f181.google.com [209.85.215.181]) by imf02.hostedemail.com (Postfix) with ESMTP id DAD9C80076 for ; Thu, 2 Jun 2022 18:22:07 +0000 (UTC) Received: by mail-pg1-f181.google.com with SMTP id 7so4619963pga.12 for ; Thu, 02 Jun 2022 11:22:13 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=nGO/cFv0tsR5sAJeDSH8YmjUaRsEQ2kTqkOPuJivkVQ=; b=SFgoyvHkd6A4TqKD6C5Qehm4SuIE20sr4w2oqpsvxqyZc+KCK/EtTt6xkwXnAJ6K+Z 1p2Dodm4Qf6zeRQOeX27oDjr+vB6FZJG6LiZM5++0Xpzo5bMjr1OoB7MBBkAF+5z3qq0 tuV7y8rlyV9VeIaw5mkTIyXEP68Iyzmb973gmshX3UtiCTYZS2nqoQFKav5pshPfRDzO uiklltM2pQ60Cj6GEbC1sOkZ9AH9Xrk+WacNJt9lMB0CVH7KmycefvEkicwETvM806rH tFpShS0wv/Wdp9v18xcnwP/meTmLhwMe56l/rUTePWwIrnUMB+k4Ld9Cp9XEYZ+HA5Jl T1rA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=nGO/cFv0tsR5sAJeDSH8YmjUaRsEQ2kTqkOPuJivkVQ=; b=3hJOSD9CAn0IfnJpb64IiybofR6PZJ7BkWJd6Wnz4nm4WU9et4bv7D11BCImq9HEnh 6mrIs1vhVy8o+ai0weCDxeq/cycEWeHS1IIilItLW37F28q6zTe6f99fKF82+ZUq6P6M 7Q5Kyi+GuZyQPnV2wf7FEsKitekMnCwexjaBonlO0APQ6SGyngb9AZuTaz8vJEoXFvV2 OWNCmLZA4bTYSV3QppBaPVJzRPZxx90LVa3EwP7q+JxAoaYtOlKbNtL1Gz5tS5R0VbiP /IocBwDfMkYaK8a2rsU2cAIZ++eLav6+d6afDQFvja440LomEgV+Za/nPnx6BuOfcUmJ /+rw== X-Gm-Message-State: AOAM531N3JD2ZUFpm3lHFuBPEf917xGg5g6L6BE9YdTRuC63m6jtpuEP Qws1zTSETPGXVhIvDSIp4BkjWAOd7a4= X-Google-Smtp-Source: ABdhPJx/7D10UVwBR5vy5WQND7kM5n2ip0J8Zi1RKgOtVn//SpUYNVTYuwgcv5lFt5uBKe891Dz/Vg== X-Received: by 2002:a63:4d5d:0:b0:3fc:159c:8159 with SMTP id n29-20020a634d5d000000b003fc159c8159mr5337595pgl.106.1654194132485; Thu, 02 Jun 2022 11:22:12 -0700 (PDT) Received: from localhost.localdomain (c-67-174-241-145.hsd1.ca.comcast.net. [67.174.241.145]) by smtp.gmail.com with ESMTPSA id t10-20020a1709027fca00b0015f2d549b46sm3866779plb.237.2022.06.02.11.22.11 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 02 Jun 2022 11:22:11 -0700 (PDT) From: Yang Shi To: vbabka@suse.cz, kirill.shutemov@linux.intel.com, akpm@linux-foundation.org Cc: shy828301@gmail.com, linux-mm@kvack.org, linux-kernel@vger.kernel.org Subject: [v2 PATCH 6/7] mm: thp: kill __transhuge_page_enabled() Date: Thu, 2 Jun 2022 11:21:58 -0700 Message-Id: <20220602182159.248387-7-shy828301@gmail.com> X-Mailer: git-send-email 2.26.3 In-Reply-To: <20220602182159.248387-1-shy828301@gmail.com> References: <20220602182159.248387-1-shy828301@gmail.com> MIME-Version: 1.0 X-Rspamd-Server: rspam06 X-Rspamd-Queue-Id: DAD9C80076 X-Stat-Signature: o8k5z1cyspw46hk3ff6un7xom96gsrxw X-Rspam-User: Authentication-Results: imf02.hostedemail.com; dkim=pass header.d=gmail.com header.s=20210112 header.b=SFgoyvHk; spf=pass (imf02.hostedemail.com: domain of shy828301@gmail.com designates 209.85.215.181 as permitted sender) smtp.mailfrom=shy828301@gmail.com; dmarc=pass (policy=none) header.from=gmail.com X-HE-Tag: 1654194127-157931 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: The page fault path checks THP eligibility with __transhuge_page_enabled() which does the similar thing as hugepage_vma_check(), so use hugepage_vma_check() instead. However page fault allows DAX and !anon_vma cases, so added a new flag, in_pf, to hugepage_vma_check() to make page fault work correctly. The in_pf flag is also used to skip shmem and file THP for page fault since shmem handles THP in its own shmem_fault() and file THP allocation on fault is not supported yet. Also remove hugepage_vma_enabled() since hugepage_vma_check() is the only caller now, it is not necessary to have a helper function. Signed-off-by: Yang Shi --- fs/proc/task_mmu.c | 2 +- include/linux/huge_mm.h | 57 ++------------------------------------ include/linux/khugepaged.h | 2 +- mm/huge_memory.c | 25 ++++++++++++----- mm/khugepaged.c | 8 +++--- mm/memory.c | 7 +++-- 6 files changed, 31 insertions(+), 70 deletions(-) diff --git a/fs/proc/task_mmu.c b/fs/proc/task_mmu.c index fd79566e204c..a0850303baec 100644 --- a/fs/proc/task_mmu.c +++ b/fs/proc/task_mmu.c @@ -860,7 +860,7 @@ static int show_smap(struct seq_file *m, void *v) __show_smap(m, &mss, false); seq_printf(m, "THPeligible: %d\n", - hugepage_vma_check(vma, vma->vm_flags, true)); + hugepage_vma_check(vma, vma->vm_flags, true, false)); if (arch_pkeys_enabled()) seq_printf(m, "ProtectionKey: %8u\n", vma_pkey(vma)); diff --git a/include/linux/huge_mm.h b/include/linux/huge_mm.h index f315c70d5256..bbbaf3eba30a 100644 --- a/include/linux/huge_mm.h +++ b/include/linux/huge_mm.h @@ -153,48 +153,6 @@ static inline bool transhuge_vma_suitable(struct vm_area_struct *vma, return true; } -static inline bool transhuge_vma_enabled(struct vm_area_struct *vma, - unsigned long vm_flags) -{ - /* Explicitly disabled through madvise. */ - if ((vm_flags & VM_NOHUGEPAGE) || - test_bit(MMF_DISABLE_THP, &vma->vm_mm->flags)) - return false; - return true; -} - -/* - * to be used on vmas which are known to support THP. - * Use transparent_hugepage_active otherwise - */ -static inline bool __transparent_hugepage_enabled(struct vm_area_struct *vma) -{ - - /* - * If the hardware/firmware marked hugepage support disabled. - */ - if (transparent_hugepage_flags & (1 << TRANSPARENT_HUGEPAGE_NEVER_DAX)) - return false; - - if (!transhuge_vma_enabled(vma, vma->vm_flags)) - return false; - - if (vma_is_temporary_stack(vma)) - return false; - - if (transparent_hugepage_flags & (1 << TRANSPARENT_HUGEPAGE_FLAG)) - return true; - - if (vma_is_dax(vma)) - return true; - - if (transparent_hugepage_flags & - (1 << TRANSPARENT_HUGEPAGE_REQ_MADV_FLAG)) - return !!(vma->vm_flags & VM_HUGEPAGE); - - return false; -} - static inline bool file_thp_enabled(struct vm_area_struct *vma) { struct inode *inode; @@ -211,7 +169,7 @@ static inline bool file_thp_enabled(struct vm_area_struct *vma) extern bool hugepage_vma_check(struct vm_area_struct *vma, unsigned long vm_flags, - bool smaps); + bool smaps, bool in_pf); #define transparent_hugepage_use_zero_page() \ (transparent_hugepage_flags & \ @@ -355,11 +313,6 @@ static inline bool folio_test_pmd_mappable(struct folio *folio) return false; } -static inline bool __transparent_hugepage_enabled(struct vm_area_struct *vma) -{ - return false; -} - static inline bool transhuge_vma_size_ok(struct vm_area_struct *vma) { return false; @@ -371,15 +324,9 @@ static inline bool transhuge_vma_suitable(struct vm_area_struct *vma, return false; } -static inline bool transhuge_vma_enabled(struct vm_area_struct *vma, - unsigned long vm_flags) -{ - return false; -} - static inline bool hugepage_vma_check(struct vm_area_struct *vma, unsigned long vm_flags, - bool smaps) + bool smaps, bool in_pf) { return false; } diff --git a/include/linux/khugepaged.h b/include/linux/khugepaged.h index 8a6452e089ca..e047be601268 100644 --- a/include/linux/khugepaged.h +++ b/include/linux/khugepaged.h @@ -55,7 +55,7 @@ static inline void khugepaged_enter(struct vm_area_struct *vma, { if (!test_bit(MMF_VM_HUGEPAGE, &vma->vm_mm->flags) && khugepaged_enabled()) { - if (hugepage_vma_check(vma, vm_flags, false)) + if (hugepage_vma_check(vma, vm_flags, false, false)) __khugepaged_enter(vma->vm_mm); } } diff --git a/mm/huge_memory.c b/mm/huge_memory.c index bc8370856e85..b95786ada466 100644 --- a/mm/huge_memory.c +++ b/mm/huge_memory.c @@ -71,17 +71,25 @@ unsigned long huge_zero_pfn __read_mostly = ~0UL; bool hugepage_vma_check(struct vm_area_struct *vma, unsigned long vm_flags, - bool smaps) + bool smaps, bool in_pf) { - if (!transhuge_vma_enabled(vma, vm_flags)) + /* Explicitly disabled through madvise or prctl. */ + if ((vm_flags & VM_NOHUGEPAGE) || + test_bit(MMF_DISABLE_THP, &vma->vm_mm->flags)) + return false; + /* + * If the hardware/firmware marked hugepage support disabled. + */ + if (transparent_hugepage_flags & (1 << TRANSPARENT_HUGEPAGE_NEVER_DAX)) return false; + /* Special VMA and hugetlb VMA */ if (vm_flags & VM_NO_KHUGEPAGED) return false; - /* Don't run khugepaged against DAX vma */ + /* khugepaged doesn't collapse DAX vma, but page fault is fine. */ if (vma_is_dax(vma)) - return false; + return in_pf; if (vma->vm_file && !IS_ALIGNED((vma->vm_start >> PAGE_SHIFT) - vma->vm_pgoff, HPAGE_PMD_NR)) @@ -91,7 +99,7 @@ bool hugepage_vma_check(struct vm_area_struct *vma, return false; /* Enabled via shmem mount options or sysfs settings. */ - if (shmem_file(vma->vm_file)) + if (!in_pf && shmem_file(vma->vm_file)) return shmem_huge_enabled(vma); if (!khugepaged_enabled()) @@ -102,7 +110,7 @@ bool hugepage_vma_check(struct vm_area_struct *vma, return false; /* Only regular file is valid */ - if (file_thp_enabled(vma)) + if (!in_pf && file_thp_enabled(vma)) return true; if (!vma_is_anonymous(vma)) @@ -114,9 +122,12 @@ bool hugepage_vma_check(struct vm_area_struct *vma, /* * THPeligible bit of smaps should show 1 for proper VMAs even * though anon_vma is not initialized yet. + * + * Allow page fault since anon_vma may be not initialized until + * the first page fault. */ if (!vma->anon_vma) - return smaps; + return (smaps || in_pf); return true; } diff --git a/mm/khugepaged.c b/mm/khugepaged.c index aa0769e3b0d9..ab6183c5489f 100644 --- a/mm/khugepaged.c +++ b/mm/khugepaged.c @@ -473,7 +473,7 @@ void khugepaged_enter_vma(struct vm_area_struct *vma, { if (!test_bit(MMF_VM_HUGEPAGE, &vma->vm_mm->flags) && khugepaged_enabled()) { - if (hugepage_vma_check(vma, vm_flags, false)) + if (hugepage_vma_check(vma, vm_flags, false, false)) __khugepaged_enter(vma->vm_mm); } } @@ -918,7 +918,7 @@ static int hugepage_vma_revalidate(struct mm_struct *mm, unsigned long address, if (!transhuge_vma_suitable(vma, address)) return SCAN_ADDRESS_RANGE; - if (!hugepage_vma_check(vma, vma->vm_flags, false)) + if (!hugepage_vma_check(vma, vma->vm_flags, false, false)) return SCAN_VMA_CHECK; return 0; } @@ -1399,7 +1399,7 @@ void collapse_pte_mapped_thp(struct mm_struct *mm, unsigned long addr) * the valid THP. Add extra VM_HUGEPAGE so hugepage_vma_check() * will not fail the vma for missing VM_HUGEPAGE */ - if (!hugepage_vma_check(vma, vma->vm_flags | VM_HUGEPAGE, false)) + if (!hugepage_vma_check(vma, vma->vm_flags | VM_HUGEPAGE, false, false)) return; /* Keep pmd pgtable for uffd-wp; see comment in retract_page_tables() */ @@ -2089,7 +2089,7 @@ static unsigned int khugepaged_scan_mm_slot(unsigned int pages, progress++; break; } - if (!hugepage_vma_check(vma, vma->vm_flags, false)) { + if (!hugepage_vma_check(vma, vma->vm_flags, false, false)) { skip: progress++; continue; diff --git a/mm/memory.c b/mm/memory.c index bc5d40eec5d5..673f7561a30a 100644 --- a/mm/memory.c +++ b/mm/memory.c @@ -4962,6 +4962,7 @@ static vm_fault_t __handle_mm_fault(struct vm_area_struct *vma, .gfp_mask = __get_fault_gfp_mask(vma), }; struct mm_struct *mm = vma->vm_mm; + unsigned long vm_flags = vma->vm_flags; pgd_t *pgd; p4d_t *p4d; vm_fault_t ret; @@ -4975,7 +4976,8 @@ static vm_fault_t __handle_mm_fault(struct vm_area_struct *vma, if (!vmf.pud) return VM_FAULT_OOM; retry_pud: - if (pud_none(*vmf.pud) && __transparent_hugepage_enabled(vma)) { + if (pud_none(*vmf.pud) && + hugepage_vma_check(vma, vm_flags, false, true)) { ret = create_huge_pud(&vmf); if (!(ret & VM_FAULT_FALLBACK)) return ret; @@ -5008,7 +5010,8 @@ static vm_fault_t __handle_mm_fault(struct vm_area_struct *vma, if (pud_trans_unstable(vmf.pud)) goto retry_pud; - if (pmd_none(*vmf.pmd) && __transparent_hugepage_enabled(vma)) { + if (pmd_none(*vmf.pmd) && + hugepage_vma_check(vma, vm_flags, false, true)) { ret = create_huge_pmd(&vmf); if (!(ret & VM_FAULT_FALLBACK)) return ret; From patchwork Thu Jun 2 18:21:59 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yang Shi X-Patchwork-Id: 12868048 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id CF2E1CCA47E for ; Thu, 2 Jun 2022 18:22:16 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 82B6D6B007E; Thu, 2 Jun 2022 14:22:15 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 7DF946B0080; Thu, 2 Jun 2022 14:22:15 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 67E276B0081; Thu, 2 Jun 2022 14:22:15 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0016.hostedemail.com [216.40.44.16]) by kanga.kvack.org (Postfix) with ESMTP id 584546B007E for ; Thu, 2 Jun 2022 14:22:15 -0400 (EDT) Received: from smtpin09.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay01.hostedemail.com (Postfix) with ESMTP id 35FFD61549 for ; Thu, 2 Jun 2022 18:22:15 +0000 (UTC) X-FDA: 79534115430.09.ACE8AF8 Received: from mail-pj1-f43.google.com (mail-pj1-f43.google.com [209.85.216.43]) by imf03.hostedemail.com (Postfix) with ESMTP id 7798D20055 for ; Thu, 2 Jun 2022 18:21:59 +0000 (UTC) Received: by mail-pj1-f43.google.com with SMTP id w2-20020a17090ac98200b001e0519fe5a8so5492898pjt.4 for ; Thu, 02 Jun 2022 11:22:14 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=hSlU6Lbj1xiMbj9OJuSwVIYBVljgpyUnPXL4SbncscE=; b=lGZfQS2dawbRWYfKai5mohgnC6yt8NU/fBuHreLbnhxS97F65F2IH6yIfmwsgNBetc uFdybSvcBfa+1M9DRHLj17ljNMUCHlZ6Xk7AG7WWycs4dAoYD5s/K4rdpgBIL3UCUB9G HH4CoA17dtS6p+Vd8cfKk30Eu9iclYqbz1S0R41UohcGIvBsQi5lMbCZQmpA8hdUW5IG aVQoazoRZZpG8c1P2qx8COdlJb8mN/ADUEM21hHpay14hLGhve2iRFQO0hr5NggEsvut /PF6h+6XgH8ZsPuvCKZvUvacDbVy4LqiyBF5x7hoX0X4zq2qaCroPWD88S79nZKNrc9O ujWg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=hSlU6Lbj1xiMbj9OJuSwVIYBVljgpyUnPXL4SbncscE=; b=QQCky5xwLVrt72aYTOtQVIm5+5tkrPpUvs3Sg3d/expP/KvnC+XYMLU6BTSwgWZAdi ZF01Semp8NhXq4w7FWhrY+wNA09GyhG1AOSaZqEhBks/Q1EwBRn0/njvK1LPiRCB5ZvU 4CB7wr9l4F3iuw/DuUGrLDf0SbB/P2YrRLK4t7MGk6D26zbwJr6etkIsaznxQgKRKqe9 gdLwNtprF4+sHiNFCuWCkMns6dmkrPeoNE29SfcV+k/FfAxwE6MrGz9lRE/WwSnr6l/y Sc058pH6YaBKlRrm9DIMcymjcW51PYJqc+/vhUWqYom0pHBoIk7M3xB0OwomU8pRMfBZ QzNA== X-Gm-Message-State: AOAM532ASVqYaFRPzL2+IKUMvUT5PIfsZuxhnegtlipzbBRuSsvVyDN6 jLcCbEIEvwLHdNOxj0265tk= X-Google-Smtp-Source: ABdhPJxLT/aEEZENsrdalshxpkuYcAThzVkBKsWYUCQpfppCvGax4dxmv0zOoGgMtYqcV1ArZ2nVxA== X-Received: by 2002:a17:90b:4d0a:b0:1e2:c0b4:8bb8 with SMTP id mw10-20020a17090b4d0a00b001e2c0b48bb8mr6724889pjb.94.1654194134287; Thu, 02 Jun 2022 11:22:14 -0700 (PDT) Received: from localhost.localdomain (c-67-174-241-145.hsd1.ca.comcast.net. [67.174.241.145]) by smtp.gmail.com with ESMTPSA id t10-20020a1709027fca00b0015f2d549b46sm3866779plb.237.2022.06.02.11.22.12 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 02 Jun 2022 11:22:13 -0700 (PDT) From: Yang Shi To: vbabka@suse.cz, kirill.shutemov@linux.intel.com, akpm@linux-foundation.org Cc: shy828301@gmail.com, linux-mm@kvack.org, linux-kernel@vger.kernel.org Subject: [v2 PATCH 7/7] mm: khugepaged: reorg some khugepaged helpers Date: Thu, 2 Jun 2022 11:21:59 -0700 Message-Id: <20220602182159.248387-8-shy828301@gmail.com> X-Mailer: git-send-email 2.26.3 In-Reply-To: <20220602182159.248387-1-shy828301@gmail.com> References: <20220602182159.248387-1-shy828301@gmail.com> MIME-Version: 1.0 Authentication-Results: imf03.hostedemail.com; dkim=pass header.d=gmail.com header.s=20210112 header.b=lGZfQS2d; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (imf03.hostedemail.com: domain of shy828301@gmail.com designates 209.85.216.43 as permitted sender) smtp.mailfrom=shy828301@gmail.com X-Rspamd-Server: rspam03 X-Rspamd-Queue-Id: 7798D20055 X-Rspam-User: X-Stat-Signature: o5fyujhz4qayx9cmfjmz1qzodin8ji4b X-HE-Tag: 1654194119-254652 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: The khugepaged_{enabled|always|req_madv} are not khugepaged only anymore, move them to huge_mm.h and rename to hugepage_flags_xxx, and remove khugepaged_req_madv due to no users. Also move khugepaged_defrag to khugepaged.c since its only caller is in that file, it doesn't have to be in a header file. Signed-off-by: Yang Shi --- include/linux/huge_mm.h | 8 ++++++++ include/linux/khugepaged.h | 17 +---------------- mm/huge_memory.c | 4 ++-- mm/khugepaged.c | 18 +++++++++++------- 4 files changed, 22 insertions(+), 25 deletions(-) diff --git a/include/linux/huge_mm.h b/include/linux/huge_mm.h index bbbaf3eba30a..452eaef24948 100644 --- a/include/linux/huge_mm.h +++ b/include/linux/huge_mm.h @@ -116,6 +116,14 @@ extern struct kobj_attribute shmem_enabled_attr; extern unsigned long transparent_hugepage_flags; +#define hugepage_flags_enabled() \ + (transparent_hugepage_flags & \ + ((1<flags)) @@ -53,8 +39,7 @@ static inline void khugepaged_exit(struct mm_struct *mm) static inline void khugepaged_enter(struct vm_area_struct *vma, unsigned long vm_flags) { - if (!test_bit(MMF_VM_HUGEPAGE, &vma->vm_mm->flags) && - khugepaged_enabled()) { + if (!test_bit(MMF_VM_HUGEPAGE, &vma->vm_mm->flags)) { if (hugepage_vma_check(vma, vm_flags, false, false)) __khugepaged_enter(vma->vm_mm); } diff --git a/mm/huge_memory.c b/mm/huge_memory.c index b95786ada466..866b98a39496 100644 --- a/mm/huge_memory.c +++ b/mm/huge_memory.c @@ -102,11 +102,11 @@ bool hugepage_vma_check(struct vm_area_struct *vma, if (!in_pf && shmem_file(vma->vm_file)) return shmem_huge_enabled(vma); - if (!khugepaged_enabled()) + if (!hugepage_flags_enabled()) return false; /* THP settings require madvise. */ - if (!(vm_flags & VM_HUGEPAGE) && !khugepaged_always()) + if (!(vm_flags & VM_HUGEPAGE) && !hugepage_flags_always()) return false; /* Only regular file is valid */ diff --git a/mm/khugepaged.c b/mm/khugepaged.c index ab6183c5489f..2523c085625a 100644 --- a/mm/khugepaged.c +++ b/mm/khugepaged.c @@ -472,7 +472,7 @@ void khugepaged_enter_vma(struct vm_area_struct *vma, unsigned long vm_flags) { if (!test_bit(MMF_VM_HUGEPAGE, &vma->vm_mm->flags) && - khugepaged_enabled()) { + hugepage_flags_enabled()) { if (hugepage_vma_check(vma, vm_flags, false, false)) __khugepaged_enter(vma->vm_mm); } @@ -763,6 +763,10 @@ static bool khugepaged_scan_abort(int nid) return false; } +#define khugepaged_defrag() \ + (transparent_hugepage_flags & \ + (1<