From patchwork Mon Jun 6 21:44:08 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yang Shi X-Patchwork-Id: 12871104 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id D28B3C433EF for ; Mon, 6 Jun 2022 21:44:22 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 574996B0073; Mon, 6 Jun 2022 17:44:22 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 524266B0074; Mon, 6 Jun 2022 17:44:22 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 378316B0075; Mon, 6 Jun 2022 17:44:22 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0010.hostedemail.com [216.40.44.10]) by kanga.kvack.org (Postfix) with ESMTP id 2649D6B0073 for ; Mon, 6 Jun 2022 17:44:22 -0400 (EDT) Received: from smtpin02.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay12.hostedemail.com (Postfix) with ESMTP id 06A43120A9E for ; Mon, 6 Jun 2022 21:44:22 +0000 (UTC) X-FDA: 79549139964.02.CA52F17 Received: from mail-pj1-f47.google.com (mail-pj1-f47.google.com [209.85.216.47]) by imf31.hostedemail.com (Postfix) with ESMTP id DB77A20020 for ; Mon, 6 Jun 2022 21:43:35 +0000 (UTC) Received: by mail-pj1-f47.google.com with SMTP id cx11so13918782pjb.1 for ; Mon, 06 Jun 2022 14:44:21 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=SWErXY/21cRT+JuPcizRjGDN5I2BQSMNvqCbtDuuEgI=; b=I5APtot2v61wKkN0XCT4995ZIiul394ZNakXAWx3CnTxQvkSfrURRzUJ0YUHHBDmoN n7WES+tRmlehNiQ+3ho3huUDvxWLUQKDPTnFK3bHGNIBaYMyTK43rUlQCWrS8jBRXYqw asEC1jsCThymiE5h5B8duobPkjqKD79mgUEVqb3+uLy+sG21Xr8hCgblpLAIDX9GBdY+ YnoLjc3K3sv/qGe2DmVNbu56MCjX9ZwGKXZc3UA8b6ez1d+JTPISyuuAZIt5r+pAMldH A3XSVAeQxTWsGtKRdSh1OHuMOvFPqiIUvWqLjyZT3XWfA2X2Q+VmkOMkXtoUx/+j9lcl C7dQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=SWErXY/21cRT+JuPcizRjGDN5I2BQSMNvqCbtDuuEgI=; b=7ys/8gQt7apsfvp74Nh8TvZ7mj9ctonrBGwRuHcfnFik0dX9ErOiKy2OTDgONaqwgB LPGjruv3GkZC+ie3IG+VLPdtJQmayk5qBvMPzD93R9DN5n8WvKSFAaGReWBx3oHTX0Vp ivSqc0rMKE67Oy5P2FreLcIxXo2SvTmJcT/88tQU5937Y8qk5ChyXnxbhMQAU2/WQRt5 jYrpNwQuBle4SnvV8odpwk+uk6qUMr00KA+hqLp0Q3yrXospTcsvq8uFVycBJaCGVzP/ bqL+fRA76M1hfrW2HyHPCQ9LHG5l3z3ruOH+okfB064q8fTu0Qo0G++2lV0iBJYAYJAx Y7sQ== X-Gm-Message-State: AOAM533h4yW65DWMplt6PV2CWr2bNImZjbS+R6psALWVr+hOJMD/UCPu rKC0UtAEnT8Q44uwNa0gnao= X-Google-Smtp-Source: ABdhPJxxmkvlq3kb5ZvctnDvImOgaVYAMOt1gHJ3eNFjJyFulmHv+ybHJWXnTeDs5PmIyxjHCkv45Q== X-Received: by 2002:a17:902:b282:b0:163:ffe6:2cf9 with SMTP id u2-20020a170902b28200b00163ffe62cf9mr26504023plr.76.1654551860779; Mon, 06 Jun 2022 14:44:20 -0700 (PDT) Received: from localhost.biz ([8.25.197.27]) by smtp.gmail.com with ESMTPSA id a4-20020a170903100400b0016397da033csm10881675plb.62.2022.06.06.14.44.19 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 06 Jun 2022 14:44:20 -0700 (PDT) From: Yang Shi To: vbabka@suse.cz, kirill.shutemov@linux.intel.com, willy@infradead.org, akpm@linux-foundation.org Cc: shy828301@gmail.com, linux-mm@kvack.org, linux-kernel@vger.kernel.org Subject: [v3 PATCH 1/7] mm: khugepaged: check THP flag in hugepage_vma_check() Date: Mon, 6 Jun 2022 14:44:08 -0700 Message-Id: <20220606214414.736109-2-shy828301@gmail.com> X-Mailer: git-send-email 2.26.3 In-Reply-To: <20220606214414.736109-1-shy828301@gmail.com> References: <20220606214414.736109-1-shy828301@gmail.com> MIME-Version: 1.0 X-Rspamd-Server: rspam07 X-Rspamd-Queue-Id: DB77A20020 X-Stat-Signature: w99obnb5gtc45cns3o5cmdup8re1oeaw X-Rspam-User: Authentication-Results: imf31.hostedemail.com; dkim=pass header.d=gmail.com header.s=20210112 header.b=I5APtot2; spf=pass (imf31.hostedemail.com: domain of shy828301@gmail.com designates 209.85.216.47 as permitted sender) smtp.mailfrom=shy828301@gmail.com; dmarc=pass (policy=none) header.from=gmail.com X-HE-Tag: 1654551815-417927 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Currently the THP flag check in hugepage_vma_check() will fallthrough if the flag is NEVER and VM_HUGEPAGE is set. This is not a problem for now since all the callers have the flag checked before or can't be invoked if the flag is NEVER. However, the following patch will call hugepage_vma_check() in more places, for example, page fault, so this flag must be checked in hugepge_vma_check(). Signed-off-by: Yang Shi Reviewed-by: Zach O'Keefe Reviewed-by: Miaohe Lin --- mm/khugepaged.c | 3 +++ 1 file changed, 3 insertions(+) diff --git a/mm/khugepaged.c b/mm/khugepaged.c index 671ac7800e53..84b9cf4b9be9 100644 --- a/mm/khugepaged.c +++ b/mm/khugepaged.c @@ -458,6 +458,9 @@ bool hugepage_vma_check(struct vm_area_struct *vma, if (shmem_file(vma->vm_file)) return shmem_huge_enabled(vma); + if (!khugepaged_enabled()) + return false; + /* THP settings require madvise. */ if (!(vm_flags & VM_HUGEPAGE) && !khugepaged_always()) return false; From patchwork Mon Jun 6 21:44:09 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yang Shi X-Patchwork-Id: 12871105 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 5A97EC43334 for ; Mon, 6 Jun 2022 21:44:24 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id F14B46B0074; Mon, 6 Jun 2022 17:44:23 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id EC1DC8D0001; Mon, 6 Jun 2022 17:44:23 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id CEDA36B0078; Mon, 6 Jun 2022 17:44:23 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0011.hostedemail.com [216.40.44.11]) by kanga.kvack.org (Postfix) with ESMTP id C04556B0074 for ; Mon, 6 Jun 2022 17:44:23 -0400 (EDT) Received: from smtpin12.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay12.hostedemail.com (Postfix) with ESMTP id A17C1120B10 for ; Mon, 6 Jun 2022 21:44:23 +0000 (UTC) X-FDA: 79549140006.12.3680838 Received: from mail-pl1-f177.google.com (mail-pl1-f177.google.com [209.85.214.177]) by imf31.hostedemail.com (Postfix) with ESMTP id 7FB1C20045 for ; Mon, 6 Jun 2022 21:43:37 +0000 (UTC) Received: by mail-pl1-f177.google.com with SMTP id i1so13178270plg.7 for ; Mon, 06 Jun 2022 14:44:22 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=D/C1dubPl+rj/RbGuMLBKXaMg54FT3nuKqdm5O29Xsg=; b=YRS9DKv1jCdO/JdsfD2Zdx2c6vo3ureR7Qm46tf+wcDQKy1FeN7iCjcyusFTyim11S d0j0ZZpoAaKyN+IaCjtJMSz9RrTX8Q3XcRUCrbEwBT8IazrwYVwuLwUQZAPvljDjYQFO ldHcD5wfIiPmSXhaTPA9LHw9ZUhMTbUSlZ40NzcPhON+Pr3zxvPwDSODHzYSHcjjhK9g Z38Uhow01G4f8nz97ErvS2BY6kUS+gFldseEqN0iI1/pBd1K9x5LG5XvmW3+WTctCrdR mr/60PMYJyB7y+cU2eD3G2ZfqOZ9jzYJM4+mZZ6eRiqmevTRTU534YjwsFgh0MJpVknB Gprg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=D/C1dubPl+rj/RbGuMLBKXaMg54FT3nuKqdm5O29Xsg=; b=0ldNwvQFXCltohzK798WjdnBZrL8hqEA5ssjgMVfv12tTKZr+X+4Kizozyv/vzvru3 FjZVc1uAJe+7bI6GsyocX20VLk+k1jnu1/Yv1ihDU9E3NOB7c7jpY3x1LFWIWhVcIP7O STcUP9QIzcJDn/7lqoEHAUOqikPcc9FLkPLqvqTNnf1DNXLUeNOS+8OLMa/2RuK7b71t X7X2/5/1lZP7pD+URV4ECPpvpbKldk7lakq7vtS6XS5hPUZo/RADeSDi2LhGWeGRch4E i2ul3bT2jr0OeAU/odO2trw7u+HWe4WGC54KjdfvzW8LUwfQRPJepfU0T7G9a5crmMpg AkUg== X-Gm-Message-State: AOAM533m/3auwSMtFTXAONwZu2ZZtuK4wiCMsZADOB4RGW4BF0+Cf5Fb 7Fv7MBY8oF0T81XXnh36TcU= X-Google-Smtp-Source: ABdhPJzikW8LtgDjXM0E0yc+iosNsTNVfvkgGIPBuUaO7oCFPC7sJOjOY3z/ay2XBf6L9aDLJBVqBA== X-Received: by 2002:a17:90a:1f4c:b0:1e6:6f77:c573 with SMTP id y12-20020a17090a1f4c00b001e66f77c573mr30435048pjy.17.1654551862332; Mon, 06 Jun 2022 14:44:22 -0700 (PDT) Received: from localhost.biz ([8.25.197.27]) by smtp.gmail.com with ESMTPSA id a4-20020a170903100400b0016397da033csm10881675plb.62.2022.06.06.14.44.21 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 06 Jun 2022 14:44:21 -0700 (PDT) From: Yang Shi To: vbabka@suse.cz, kirill.shutemov@linux.intel.com, willy@infradead.org, akpm@linux-foundation.org Cc: shy828301@gmail.com, linux-mm@kvack.org, linux-kernel@vger.kernel.org Subject: [v3 PATCH 2/7] mm: thp: introduce transhuge_vma_size_ok() helper Date: Mon, 6 Jun 2022 14:44:09 -0700 Message-Id: <20220606214414.736109-3-shy828301@gmail.com> X-Mailer: git-send-email 2.26.3 In-Reply-To: <20220606214414.736109-1-shy828301@gmail.com> References: <20220606214414.736109-1-shy828301@gmail.com> MIME-Version: 1.0 X-Rspam-User: Authentication-Results: imf31.hostedemail.com; dkim=pass header.d=gmail.com header.s=20210112 header.b=YRS9DKv1; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (imf31.hostedemail.com: domain of shy828301@gmail.com designates 209.85.214.177 as permitted sender) smtp.mailfrom=shy828301@gmail.com X-Rspamd-Server: rspam10 X-Rspamd-Queue-Id: 7FB1C20045 X-Stat-Signature: 7xzrkkkb13od5zjhy4rb7ip7hzw8ubjn X-HE-Tag: 1654551817-528157 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: There are couple of places that check whether the vma size is ok for THP or not, they are open coded and duplicate, introduce transhuge_vma_size_ok() helper to do the job. Signed-off-by: Yang Shi --- include/linux/huge_mm.h | 17 +++++++++++++++++ mm/huge_memory.c | 5 +---- mm/khugepaged.c | 12 ++++++------ 3 files changed, 24 insertions(+), 10 deletions(-) diff --git a/include/linux/huge_mm.h b/include/linux/huge_mm.h index 648cb3ce7099..a8f61db47f2a 100644 --- a/include/linux/huge_mm.h +++ b/include/linux/huge_mm.h @@ -116,6 +116,18 @@ extern struct kobj_attribute shmem_enabled_attr; extern unsigned long transparent_hugepage_flags; +/* + * The vma size has to be large enough to hold an aligned HPAGE_PMD_SIZE area. + */ +static inline bool transhuge_vma_size_ok(struct vm_area_struct *vma) +{ + if (round_up(vma->vm_start, HPAGE_PMD_SIZE) < + (vma->vm_end & HPAGE_PMD_MASK)) + return true; + + return false; +} + static inline bool transhuge_vma_suitable(struct vm_area_struct *vma, unsigned long addr) { @@ -345,6 +357,11 @@ static inline bool transparent_hugepage_active(struct vm_area_struct *vma) return false; } +static inline bool transhuge_vma_size_ok(struct vm_area_struct *vma) +{ + return false; +} + static inline bool transhuge_vma_suitable(struct vm_area_struct *vma, unsigned long addr) { diff --git a/mm/huge_memory.c b/mm/huge_memory.c index 48182c8fe151..36ada544e494 100644 --- a/mm/huge_memory.c +++ b/mm/huge_memory.c @@ -71,10 +71,7 @@ unsigned long huge_zero_pfn __read_mostly = ~0UL; bool transparent_hugepage_active(struct vm_area_struct *vma) { - /* The addr is used to check if the vma size fits */ - unsigned long addr = (vma->vm_end & HPAGE_PMD_MASK) - HPAGE_PMD_SIZE; - - if (!transhuge_vma_suitable(vma, addr)) + if (!transhuge_vma_size_ok(vma)) return false; if (vma_is_anonymous(vma)) return __transparent_hugepage_enabled(vma); diff --git a/mm/khugepaged.c b/mm/khugepaged.c index 84b9cf4b9be9..d0f8020164fc 100644 --- a/mm/khugepaged.c +++ b/mm/khugepaged.c @@ -454,6 +454,9 @@ bool hugepage_vma_check(struct vm_area_struct *vma, vma->vm_pgoff, HPAGE_PMD_NR)) return false; + if (!transhuge_vma_size_ok(vma)) + return false; + /* Enabled via shmem mount options or sysfs settings. */ if (shmem_file(vma->vm_file)) return shmem_huge_enabled(vma); @@ -512,9 +515,7 @@ void khugepaged_enter_vma(struct vm_area_struct *vma, unsigned long vm_flags) { if (!test_bit(MMF_VM_HUGEPAGE, &vma->vm_mm->flags) && - khugepaged_enabled() && - (((vma->vm_start + ~HPAGE_PMD_MASK) & HPAGE_PMD_MASK) < - (vma->vm_end & HPAGE_PMD_MASK))) { + khugepaged_enabled()) { if (hugepage_vma_check(vma, vm_flags)) __khugepaged_enter(vma->vm_mm); } @@ -2142,10 +2143,9 @@ static unsigned int khugepaged_scan_mm_slot(unsigned int pages, progress++; continue; } - hstart = (vma->vm_start + ~HPAGE_PMD_MASK) & HPAGE_PMD_MASK; + + hstart = round_up(vma->vm_start, HPAGE_PMD_SIZE); hend = vma->vm_end & HPAGE_PMD_MASK; - if (hstart >= hend) - goto skip; if (khugepaged_scan.address > hend) goto skip; if (khugepaged_scan.address < hstart) From patchwork Mon Jun 6 21:44:10 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yang Shi X-Patchwork-Id: 12871106 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 08DCAC433EF for ; Mon, 6 Jun 2022 21:44:26 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 995636B0075; Mon, 6 Jun 2022 17:44:25 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 8F81D6B0078; Mon, 6 Jun 2022 17:44:25 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 6AC456B007B; Mon, 6 Jun 2022 17:44:25 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0014.hostedemail.com [216.40.44.14]) by kanga.kvack.org (Postfix) with ESMTP id 5AB8C6B0075 for ; Mon, 6 Jun 2022 17:44:25 -0400 (EDT) Received: from smtpin25.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay10.hostedemail.com (Postfix) with ESMTP id 2FD22B8E for ; Mon, 6 Jun 2022 21:44:25 +0000 (UTC) X-FDA: 79549140090.25.18DF528 Received: from mail-pj1-f46.google.com (mail-pj1-f46.google.com [209.85.216.46]) by imf27.hostedemail.com (Postfix) with ESMTP id 6644F40063 for ; Mon, 6 Jun 2022 21:44:20 +0000 (UTC) Received: by mail-pj1-f46.google.com with SMTP id o6-20020a17090a0a0600b001e2c6566046so19047287pjo.0 for ; Mon, 06 Jun 2022 14:44:24 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=TeR+saO9bGVSVDYLkT6whuJkztc98Y7uqJRw9Zwr6CA=; b=HKhirssn3eLksqebeSYrk+jXcfBxshfBpkTxjBPO9398Kqa18Cm3e71kgcZu26dIvY o6XXpciQWvJwXzV5TU9EluS5IVMmvxncZ5Mi/Vz+lQNfHrPN3rzLvi/l00C5t+Za+YHj YSPmh2kgWJ07k6J+1bq9EH3HdkCP54M7O6Vm/tRIEY34h4UfvDTYtD/vhvyQzncqUZA5 lIVQRmQhGGRj5O6xWtvnzee96vGcj+iwnnmw/OVFSsrJmnCfV4t/Ft+pb31IkLeX0FcM 9gByGWOdP6wH6PFe0kKeAl3/kjT4wP6BDTfJq8QgNMLS5uVGvXEfYFP0xTXUn4EZHlGi 7T5Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=TeR+saO9bGVSVDYLkT6whuJkztc98Y7uqJRw9Zwr6CA=; b=B66EVWSQ3n7ny78JD1Ag0u0kKNPectjbtDR6arivF7UCunecsOKLDR/Oienv6oHxpS WA4lxqC5zVQZ1pRhfylf9bdpLbMRd7K7pDIN/s21prLiGlPvcfolKV1PfUrOb8vbnujE jM0A7xZArs6bRc7w04pOftduLr6tLk1E1s+JmkgPRDg/ZbPf5/9zzPEosusg7BRLtqqQ 1xa0BwWq+ONFsLmNtnp2R0HBZXx1Neos2F4DpRFMFezwOEBgkZXoJvWPnDofcKn2jv/u TLpi2hW/cKSEZnnDdsGgbErxaS6sh3s8CqpCvCzKBZOJXXTtvyKmAX6NESe5vNo0VLrM TWnQ== X-Gm-Message-State: AOAM532Xw+9PUnDptIZWOiAw63He0RK5Zfiy9jaq4KOnHwvYcDl0JBtE 0ZsjgcT1eYSFCnkbcJjRYTk= X-Google-Smtp-Source: ABdhPJx+QgadWDwD5L0qXpqeZx6oasMAo4joDNR6AQwapxgpVmCIYIAdm45+392KvIhA5+qYQ6m75A== X-Received: by 2002:a17:90b:4a90:b0:1e8:626c:e1b2 with SMTP id lp16-20020a17090b4a9000b001e8626ce1b2mr13173095pjb.141.1654551863738; Mon, 06 Jun 2022 14:44:23 -0700 (PDT) Received: from localhost.biz ([8.25.197.27]) by smtp.gmail.com with ESMTPSA id a4-20020a170903100400b0016397da033csm10881675plb.62.2022.06.06.14.44.22 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 06 Jun 2022 14:44:23 -0700 (PDT) From: Yang Shi To: vbabka@suse.cz, kirill.shutemov@linux.intel.com, willy@infradead.org, akpm@linux-foundation.org Cc: shy828301@gmail.com, linux-mm@kvack.org, linux-kernel@vger.kernel.org Subject: [v3 PATCH 3/7] mm: khugepaged: remove the redundant anon vma check Date: Mon, 6 Jun 2022 14:44:10 -0700 Message-Id: <20220606214414.736109-4-shy828301@gmail.com> X-Mailer: git-send-email 2.26.3 In-Reply-To: <20220606214414.736109-1-shy828301@gmail.com> References: <20220606214414.736109-1-shy828301@gmail.com> MIME-Version: 1.0 X-Rspam-User: Authentication-Results: imf27.hostedemail.com; dkim=pass header.d=gmail.com header.s=20210112 header.b=HKhirssn; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (imf27.hostedemail.com: domain of shy828301@gmail.com designates 209.85.216.46 as permitted sender) smtp.mailfrom=shy828301@gmail.com X-Rspamd-Server: rspam10 X-Rspamd-Queue-Id: 6644F40063 X-Stat-Signature: bbfu69kq19sgo8k651jt4zyz9htyunb5 X-HE-Tag: 1654551860-624881 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: The hugepage_vma_check() already checked it, so remove the redundant check. Signed-off-by: Yang Shi --- mm/khugepaged.c | 3 --- 1 file changed, 3 deletions(-) diff --git a/mm/khugepaged.c b/mm/khugepaged.c index d0f8020164fc..7a5d1c1a1833 100644 --- a/mm/khugepaged.c +++ b/mm/khugepaged.c @@ -966,9 +966,6 @@ static int hugepage_vma_revalidate(struct mm_struct *mm, unsigned long address, return SCAN_ADDRESS_RANGE; if (!hugepage_vma_check(vma, vma->vm_flags)) return SCAN_VMA_CHECK; - /* Anon VMA expected */ - if (!vma->anon_vma || !vma_is_anonymous(vma)) - return SCAN_VMA_CHECK; return 0; } From patchwork Mon Jun 6 21:44:11 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yang Shi X-Patchwork-Id: 12871108 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 19F31C43334 for ; Mon, 6 Jun 2022 21:44:32 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id D3D506B007B; Mon, 6 Jun 2022 17:44:30 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id CECBC6B007D; Mon, 6 Jun 2022 17:44:30 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id BDBF18D0001; Mon, 6 Jun 2022 17:44:30 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0010.hostedemail.com [216.40.44.10]) by kanga.kvack.org (Postfix) with ESMTP id AEB626B007B for ; Mon, 6 Jun 2022 17:44:30 -0400 (EDT) Received: from smtpin18.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay08.hostedemail.com (Postfix) with ESMTP id 7933420B80 for ; Mon, 6 Jun 2022 21:44:30 +0000 (UTC) X-FDA: 79549140300.18.CDDC6D8 Received: from mail-pf1-f178.google.com (mail-pf1-f178.google.com [209.85.210.178]) by imf30.hostedemail.com (Postfix) with ESMTP id 336958002B for ; Mon, 6 Jun 2022 21:43:47 +0000 (UTC) Received: by mail-pf1-f178.google.com with SMTP id b135so13746445pfb.12 for ; Mon, 06 Jun 2022 14:44:25 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=Ij7n7UZO/IK0zKEVoAHKDokBQE/6XWgjwPeS6gPTqcc=; b=mXJ/hahxbI7vbEPKM3HNI31LOwv12MOVMykwZIF/4WcDnhIo7yCNxtwhCyA+9ateh8 /dsaYwTgy+aYi1g7b3kCczzNJ+WEAy6PcjMcJ5J0IiDz9N71++VS+nAfh/2zpRkZHRfa vYvtioaXpeUvymXxTzMCRSVMrRPyXiSIYuL1uN7AZNHMXqfLKTCTvXDqHwZWqahHfM2L WCDWc/QOyZDpHyCDtkPuRhYal0AaWB8/j7KaKcBGmN9/a5lpVSIJnaHwWII2HPckBuW7 rl6/VYaSW5bRJ2ZinxJHxIyok/ClC8cfp/QXquPk6mOoO7jd8h/jlQpaGDZwA+AlTVPK vFDQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=Ij7n7UZO/IK0zKEVoAHKDokBQE/6XWgjwPeS6gPTqcc=; b=uhPORWCsdcgUtm3Oh8Ny696IEVhjg0M91uKcyKMi5e2lTEsX54HQ3WK89lB/1StdqO NOgAmA5aTu2+ozLXCwYKyUzvxguM3eNf42CrqTCQdhQ9IzhcrCMOvg428ro+uCIAg8rm 1zCyQoxYHl664NCxnOkZhRI1zmSvMtoWKc2M7FemB5HFN3ntHsHqdYk7xHS2xDtJ4Q3d Suha7ULRwfeHuKDhOT9QM/sFE8o4apt+KvnDXaIhCfTHJKpfGKBjH84Sj5pth7u8NUZU f2eh85rkUd0km3UxYySQpJKzTDL5/ZL6LV8FYaiJftaxnzlpSsd0futEMKmHQuQQ+RXe 8KpA== X-Gm-Message-State: AOAM53052igJBSOziEjRCS7tkSZNpy2uICQswD5iBgBAdFuDlzLWGSv/ 5QRL/K1ICT+1+R1U08c+/18= X-Google-Smtp-Source: ABdhPJzMLSIncjP6QHixKNI/DAPcwxVzDs8Kf+dHw2xn4cBNcjVPbSTe/tA+bImKxTtxSNxqvHYUlA== X-Received: by 2002:a65:4907:0:b0:3fd:bc3e:fb0a with SMTP id p7-20020a654907000000b003fdbc3efb0amr6869220pgs.123.1654551865142; Mon, 06 Jun 2022 14:44:25 -0700 (PDT) Received: from localhost.biz ([8.25.197.27]) by smtp.gmail.com with ESMTPSA id a4-20020a170903100400b0016397da033csm10881675plb.62.2022.06.06.14.44.23 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 06 Jun 2022 14:44:24 -0700 (PDT) From: Yang Shi To: vbabka@suse.cz, kirill.shutemov@linux.intel.com, willy@infradead.org, akpm@linux-foundation.org Cc: shy828301@gmail.com, linux-mm@kvack.org, linux-kernel@vger.kernel.org Subject: [v3 PATCH 4/7] mm: khugepaged: use transhuge_vma_suitable replace open-code Date: Mon, 6 Jun 2022 14:44:11 -0700 Message-Id: <20220606214414.736109-5-shy828301@gmail.com> X-Mailer: git-send-email 2.26.3 In-Reply-To: <20220606214414.736109-1-shy828301@gmail.com> References: <20220606214414.736109-1-shy828301@gmail.com> MIME-Version: 1.0 X-Stat-Signature: aehat14npackcqd6qwaxnd1ekhngyfws Authentication-Results: imf30.hostedemail.com; dkim=pass header.d=gmail.com header.s=20210112 header.b="mXJ/hahx"; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (imf30.hostedemail.com: domain of shy828301@gmail.com designates 209.85.210.178 as permitted sender) smtp.mailfrom=shy828301@gmail.com X-Rspam-User: X-Rspamd-Server: rspam11 X-Rspamd-Queue-Id: 336958002B X-HE-Tag: 1654551827-787565 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: The hugepage_vma_revalidate() needs to check if the address is still in the aligned HPAGE_PMD_SIZE area of the vma when reacquiring mmap_lock, but it was open-coded, use transhuge_vma_suitable() to do the job. And add proper comments for transhuge_vma_suitable(). Signed-off-by: Yang Shi --- include/linux/huge_mm.h | 6 ++++++ mm/khugepaged.c | 5 +---- 2 files changed, 7 insertions(+), 4 deletions(-) diff --git a/include/linux/huge_mm.h b/include/linux/huge_mm.h index a8f61db47f2a..79d5919beb83 100644 --- a/include/linux/huge_mm.h +++ b/include/linux/huge_mm.h @@ -128,6 +128,12 @@ static inline bool transhuge_vma_size_ok(struct vm_area_struct *vma) return false; } +/* + * Do the below checks: + * - For non-anon vma, check if the vm_pgoff is HPAGE_PMD_NR aligned. + * - For all vmas, check if the haddr is in an aligned HPAGE_PMD_SIZE + * area. + */ static inline bool transhuge_vma_suitable(struct vm_area_struct *vma, unsigned long addr) { diff --git a/mm/khugepaged.c b/mm/khugepaged.c index 7a5d1c1a1833..ca1754d3a827 100644 --- a/mm/khugepaged.c +++ b/mm/khugepaged.c @@ -951,7 +951,6 @@ static int hugepage_vma_revalidate(struct mm_struct *mm, unsigned long address, struct vm_area_struct **vmap) { struct vm_area_struct *vma; - unsigned long hstart, hend; if (unlikely(khugepaged_test_exit(mm))) return SCAN_ANY_PROCESS; @@ -960,9 +959,7 @@ static int hugepage_vma_revalidate(struct mm_struct *mm, unsigned long address, if (!vma) return SCAN_VMA_NULL; - hstart = (vma->vm_start + ~HPAGE_PMD_MASK) & HPAGE_PMD_MASK; - hend = vma->vm_end & HPAGE_PMD_MASK; - if (address < hstart || address + HPAGE_PMD_SIZE > hend) + if (!transhuge_vma_suitable(vma, address)) return SCAN_ADDRESS_RANGE; if (!hugepage_vma_check(vma, vma->vm_flags)) return SCAN_VMA_CHECK; From patchwork Mon Jun 6 21:44:12 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yang Shi X-Patchwork-Id: 12871107 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id E8ECEC433EF for ; Mon, 6 Jun 2022 21:44:30 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 6FBD56B0078; Mon, 6 Jun 2022 17:44:30 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 6B05B6B007B; Mon, 6 Jun 2022 17:44:30 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 525E16B007D; Mon, 6 Jun 2022 17:44:30 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0015.hostedemail.com [216.40.44.15]) by kanga.kvack.org (Postfix) with ESMTP id 3F1CF6B0078 for ; Mon, 6 Jun 2022 17:44:30 -0400 (EDT) Received: from smtpin17.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay01.hostedemail.com (Postfix) with ESMTP id 186D160C64 for ; Mon, 6 Jun 2022 21:44:30 +0000 (UTC) X-FDA: 79549140300.17.F37D50D Received: from mail-pj1-f52.google.com (mail-pj1-f52.google.com [209.85.216.52]) by imf07.hostedemail.com (Postfix) with ESMTP id 8CF9640012 for ; Mon, 6 Jun 2022 21:44:13 +0000 (UTC) Received: by mail-pj1-f52.google.com with SMTP id q12-20020a17090a304c00b001e2d4fb0eb4so18991206pjl.4 for ; Mon, 06 Jun 2022 14:44:27 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=hc3IsUtoDZHlbZz0pKu30HageYNlh4gUZJ0bt5eOoac=; b=ovR/IKgd3FrhNSybEgHRSha6sg3pcMQaSjwDNFQAK9l4/69jfwfcwuFOhND3KawDGB F9I5DSIpK/DYLjEZuokouhEM4T4Amtt3pIH0CZw0Q2a2WhUlbSlv+PAJlR4dmKxKnOSG q53HRkahqgfn267VBDNMdXolYjMgrX9UHO6tUqU4lBCoWACO9n50B3209hpS0n6CcarA nrtdfNFeIlF+1FuQbeU1mcDKx1suwgs212M6Zdl4yhgciSxs2HODGHD5msdQRIbdgBOq J2GumkWaEf52nfTEcQNI9D8h5BfHETV86Ol0dL2u1IX300bbvqB/J76X1Gr09WvSbTeA p12A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=hc3IsUtoDZHlbZz0pKu30HageYNlh4gUZJ0bt5eOoac=; b=Uh7G4x5B6D0VIBDRWKRAtmdmerXNveHdSixqSCG9atGoy7HHyOMxQet1t3zKi5vhPO 5s8RJbu5rS8H3YrpvAOEXxoAXOEb6YoN8aGsre99jRCyerzd+7wwK7v/9yz5bU/8h3i4 rqZ9K6mqWdOcj0MfT2KHgf53VxAnRuMYKoIXcQudOvtS1UhiscT9yfyjc+4QJJaq7PDj jIkqC1yg/yxMrYDxj6an6WXsssQpFXdkp3oqeLxfHPJ25ggZuwH68Qx2TLQOLdMMXBTo nHaKLTcDFy8rDTQIlC2c5F7SHe2exNePX4w+StruyLf0D/obDsQxA36xVi6/fD0JTzqc 45EA== X-Gm-Message-State: AOAM533yvI3fBGM9HIuRYJEKRs6sLtIQkQnDZtyA9Fg0g/MC7419YMIU BBnKrtz8uYRER7V7uI2plJw= X-Google-Smtp-Source: ABdhPJw1vCIJ1tEESyDDFocIeFBC1uMymoEmS7+PYZF4AcUzt+PLNG4KF514jqx5+ae8P8M3ji4Ulw== X-Received: by 2002:a17:90a:ce84:b0:1e4:d803:fad with SMTP id g4-20020a17090ace8400b001e4d8030fadmr34335734pju.99.1654551866768; Mon, 06 Jun 2022 14:44:26 -0700 (PDT) Received: from localhost.biz ([8.25.197.27]) by smtp.gmail.com with ESMTPSA id a4-20020a170903100400b0016397da033csm10881675plb.62.2022.06.06.14.44.25 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 06 Jun 2022 14:44:26 -0700 (PDT) From: Yang Shi To: vbabka@suse.cz, kirill.shutemov@linux.intel.com, willy@infradead.org, akpm@linux-foundation.org Cc: shy828301@gmail.com, linux-mm@kvack.org, linux-kernel@vger.kernel.org Subject: [v3 PATCH 5/7] mm: thp: kill transparent_hugepage_active() Date: Mon, 6 Jun 2022 14:44:12 -0700 Message-Id: <20220606214414.736109-6-shy828301@gmail.com> X-Mailer: git-send-email 2.26.3 In-Reply-To: <20220606214414.736109-1-shy828301@gmail.com> References: <20220606214414.736109-1-shy828301@gmail.com> MIME-Version: 1.0 X-Rspamd-Server: rspam07 X-Rspamd-Queue-Id: 8CF9640012 X-Stat-Signature: tjsgfhmrdnuzejokt6ggzs4upb8n89zo X-Rspam-User: Authentication-Results: imf07.hostedemail.com; dkim=pass header.d=gmail.com header.s=20210112 header.b="ovR/IKgd"; spf=pass (imf07.hostedemail.com: domain of shy828301@gmail.com designates 209.85.216.52 as permitted sender) smtp.mailfrom=shy828301@gmail.com; dmarc=pass (policy=none) header.from=gmail.com X-HE-Tag: 1654551853-169741 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: The transparent_hugepage_active() was introduced to show THP eligibility bit in smaps in proc, smaps is the only user. But it actually does the similar check as hugepage_vma_check() which is used by khugepaged. We definitely don't have to maintain two similar checks, so kill transparent_hugepage_active(). Also move hugepage_vma_check() to huge_memory.c and huge_mm.h since it is not only for khugepaged anymore. Signed-off-by: Yang Shi --- fs/proc/task_mmu.c | 2 +- include/linux/huge_mm.h | 16 +++++++----- include/linux/khugepaged.h | 4 +-- mm/huge_memory.c | 50 ++++++++++++++++++++++++++++++++----- mm/khugepaged.c | 51 +++----------------------------------- 5 files changed, 60 insertions(+), 63 deletions(-) diff --git a/fs/proc/task_mmu.c b/fs/proc/task_mmu.c index 2dd8c8a66924..fd79566e204c 100644 --- a/fs/proc/task_mmu.c +++ b/fs/proc/task_mmu.c @@ -860,7 +860,7 @@ static int show_smap(struct seq_file *m, void *v) __show_smap(m, &mss, false); seq_printf(m, "THPeligible: %d\n", - transparent_hugepage_active(vma)); + hugepage_vma_check(vma, vma->vm_flags, true)); if (arch_pkeys_enabled()) seq_printf(m, "ProtectionKey: %8u\n", vma_pkey(vma)); diff --git a/include/linux/huge_mm.h b/include/linux/huge_mm.h index 79d5919beb83..f561c3e16def 100644 --- a/include/linux/huge_mm.h +++ b/include/linux/huge_mm.h @@ -209,7 +209,9 @@ static inline bool file_thp_enabled(struct vm_area_struct *vma) !inode_is_open_for_write(inode) && S_ISREG(inode->i_mode); } -bool transparent_hugepage_active(struct vm_area_struct *vma); +bool hugepage_vma_check(struct vm_area_struct *vma, + unsigned long vm_flags, + bool smaps); #define transparent_hugepage_use_zero_page() \ (transparent_hugepage_flags & \ @@ -358,11 +360,6 @@ static inline bool __transparent_hugepage_enabled(struct vm_area_struct *vma) return false; } -static inline bool transparent_hugepage_active(struct vm_area_struct *vma) -{ - return false; -} - static inline bool transhuge_vma_size_ok(struct vm_area_struct *vma) { return false; @@ -380,6 +377,13 @@ static inline bool transhuge_vma_enabled(struct vm_area_struct *vma, return false; } +static inline bool hugepage_vma_check(struct vm_area_struct *vma, + unsigned long vm_flags, + bool smaps) +{ + return false; +} + static inline void prep_transhuge_page(struct page *page) {} #define transparent_hugepage_flags 0UL diff --git a/include/linux/khugepaged.h b/include/linux/khugepaged.h index 392d34c3c59a..8a6452e089ca 100644 --- a/include/linux/khugepaged.h +++ b/include/linux/khugepaged.h @@ -10,8 +10,6 @@ extern struct attribute_group khugepaged_attr_group; extern int khugepaged_init(void); extern void khugepaged_destroy(void); extern int start_stop_khugepaged(void); -extern bool hugepage_vma_check(struct vm_area_struct *vma, - unsigned long vm_flags); extern void __khugepaged_enter(struct mm_struct *mm); extern void __khugepaged_exit(struct mm_struct *mm); extern void khugepaged_enter_vma(struct vm_area_struct *vma, @@ -57,7 +55,7 @@ static inline void khugepaged_enter(struct vm_area_struct *vma, { if (!test_bit(MMF_VM_HUGEPAGE, &vma->vm_mm->flags) && khugepaged_enabled()) { - if (hugepage_vma_check(vma, vm_flags)) + if (hugepage_vma_check(vma, vm_flags, false)) __khugepaged_enter(vma->vm_mm); } } diff --git a/mm/huge_memory.c b/mm/huge_memory.c index 36ada544e494..bc8370856e85 100644 --- a/mm/huge_memory.c +++ b/mm/huge_memory.c @@ -69,18 +69,56 @@ static atomic_t huge_zero_refcount; struct page *huge_zero_page __read_mostly; unsigned long huge_zero_pfn __read_mostly = ~0UL; -bool transparent_hugepage_active(struct vm_area_struct *vma) +bool hugepage_vma_check(struct vm_area_struct *vma, + unsigned long vm_flags, + bool smaps) { + if (!transhuge_vma_enabled(vma, vm_flags)) + return false; + + if (vm_flags & VM_NO_KHUGEPAGED) + return false; + + /* Don't run khugepaged against DAX vma */ + if (vma_is_dax(vma)) + return false; + + if (vma->vm_file && !IS_ALIGNED((vma->vm_start >> PAGE_SHIFT) - + vma->vm_pgoff, HPAGE_PMD_NR)) + return false; + if (!transhuge_vma_size_ok(vma)) return false; - if (vma_is_anonymous(vma)) - return __transparent_hugepage_enabled(vma); - if (vma_is_shmem(vma)) + + /* Enabled via shmem mount options or sysfs settings. */ + if (shmem_file(vma->vm_file)) return shmem_huge_enabled(vma); - if (transhuge_vma_enabled(vma, vma->vm_flags) && file_thp_enabled(vma)) + + if (!khugepaged_enabled()) + return false; + + /* THP settings require madvise. */ + if (!(vm_flags & VM_HUGEPAGE) && !khugepaged_always()) + return false; + + /* Only regular file is valid */ + if (file_thp_enabled(vma)) return true; - return false; + if (!vma_is_anonymous(vma)) + return false; + + if (vma_is_temporary_stack(vma)) + return false; + + /* + * THPeligible bit of smaps should show 1 for proper VMAs even + * though anon_vma is not initialized yet. + */ + if (!vma->anon_vma) + return smaps; + + return true; } static bool get_huge_zero_page(void) diff --git a/mm/khugepaged.c b/mm/khugepaged.c index ca1754d3a827..aa0769e3b0d9 100644 --- a/mm/khugepaged.c +++ b/mm/khugepaged.c @@ -437,49 +437,6 @@ static inline int khugepaged_test_exit(struct mm_struct *mm) return atomic_read(&mm->mm_users) == 0; } -bool hugepage_vma_check(struct vm_area_struct *vma, - unsigned long vm_flags) -{ - if (!transhuge_vma_enabled(vma, vm_flags)) - return false; - - if (vm_flags & VM_NO_KHUGEPAGED) - return false; - - /* Don't run khugepaged against DAX vma */ - if (vma_is_dax(vma)) - return false; - - if (vma->vm_file && !IS_ALIGNED((vma->vm_start >> PAGE_SHIFT) - - vma->vm_pgoff, HPAGE_PMD_NR)) - return false; - - if (!transhuge_vma_size_ok(vma)) - return false; - - /* Enabled via shmem mount options or sysfs settings. */ - if (shmem_file(vma->vm_file)) - return shmem_huge_enabled(vma); - - if (!khugepaged_enabled()) - return false; - - /* THP settings require madvise. */ - if (!(vm_flags & VM_HUGEPAGE) && !khugepaged_always()) - return false; - - /* Only regular file is valid */ - if (file_thp_enabled(vma)) - return true; - - if (!vma->anon_vma || !vma_is_anonymous(vma)) - return false; - if (vma_is_temporary_stack(vma)) - return false; - - return true; -} - void __khugepaged_enter(struct mm_struct *mm) { struct mm_slot *mm_slot; @@ -516,7 +473,7 @@ void khugepaged_enter_vma(struct vm_area_struct *vma, { if (!test_bit(MMF_VM_HUGEPAGE, &vma->vm_mm->flags) && khugepaged_enabled()) { - if (hugepage_vma_check(vma, vm_flags)) + if (hugepage_vma_check(vma, vm_flags, false)) __khugepaged_enter(vma->vm_mm); } } @@ -961,7 +918,7 @@ static int hugepage_vma_revalidate(struct mm_struct *mm, unsigned long address, if (!transhuge_vma_suitable(vma, address)) return SCAN_ADDRESS_RANGE; - if (!hugepage_vma_check(vma, vma->vm_flags)) + if (!hugepage_vma_check(vma, vma->vm_flags, false)) return SCAN_VMA_CHECK; return 0; } @@ -1442,7 +1399,7 @@ void collapse_pte_mapped_thp(struct mm_struct *mm, unsigned long addr) * the valid THP. Add extra VM_HUGEPAGE so hugepage_vma_check() * will not fail the vma for missing VM_HUGEPAGE */ - if (!hugepage_vma_check(vma, vma->vm_flags | VM_HUGEPAGE)) + if (!hugepage_vma_check(vma, vma->vm_flags | VM_HUGEPAGE, false)) return; /* Keep pmd pgtable for uffd-wp; see comment in retract_page_tables() */ @@ -2132,7 +2089,7 @@ static unsigned int khugepaged_scan_mm_slot(unsigned int pages, progress++; break; } - if (!hugepage_vma_check(vma, vma->vm_flags)) { + if (!hugepage_vma_check(vma, vma->vm_flags, false)) { skip: progress++; continue; From patchwork Mon Jun 6 21:44:13 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yang Shi X-Patchwork-Id: 12871110 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 62286C43334 for ; Mon, 6 Jun 2022 21:44:35 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 3EF1C6B007E; Mon, 6 Jun 2022 17:44:32 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 377896B0080; Mon, 6 Jun 2022 17:44:32 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 217C58D0001; Mon, 6 Jun 2022 17:44:32 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0016.hostedemail.com [216.40.44.16]) by kanga.kvack.org (Postfix) with ESMTP id 118026B007E for ; Mon, 6 Jun 2022 17:44:32 -0400 (EDT) Received: from smtpin10.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay01.hostedemail.com (Postfix) with ESMTP id D8BE460CB1 for ; Mon, 6 Jun 2022 21:44:31 +0000 (UTC) X-FDA: 79549140342.10.7F88483 Received: from mail-pf1-f177.google.com (mail-pf1-f177.google.com [209.85.210.177]) by imf12.hostedemail.com (Postfix) with ESMTP id 480414007B for ; Mon, 6 Jun 2022 21:43:43 +0000 (UTC) Received: by mail-pf1-f177.google.com with SMTP id g205so13749706pfb.11 for ; Mon, 06 Jun 2022 14:44:29 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=g/7Ab+lUqbHBKquM7X74tLL7I91M9MvXubqbYkGPOVk=; b=Zl67+u/eyQ1S32sx6Hq1gU3wy6Jj/gEwFrHwfDLfj5kXKbGTga32otHuWaVSHP/BTG 8FV8IFVMzeLwVIZPfohvtCHeQCORqyrMg6C5n5DKyzUkGHywc1SKe9OhZwKnjugA25Ut oQmqgR79kf6dZDTuSd/cGXe1DdZvA87Cy0Z4FGorlcY51sotwt6s2SdxKwzuhX7Wab7h xJLyf6bAEAOAic6jUhaoLF1K6RopUJ03duy3BiMOQL+oMllO4wJU88Rau/eyArAdDaG9 08JtmscsPZLvfVuxDVuX46pUVLpQILO8k86SvuapQ2t6NPh5WDVDdchZrZOa1dSVfyym q/Ew== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=g/7Ab+lUqbHBKquM7X74tLL7I91M9MvXubqbYkGPOVk=; b=RxpgxUcWXUnUO6b7nEy2Gau5cm3JqxCmLU3TipN5w29n5JDwm2iHlHd1viOcHAkuBh GhVqN24P4IwXJwaoy4vTsGXCM2qGlai7r7ruzNEsbs5QzTN/ILn1NoRQuyoFACr+dzUP qr2C0YBrh2U9zzBbWAGYM44hz2mKKNJtH4Ur2yHltjHPwsptws0jO1xicx9/tJRbMTRV CEMbKRZG63sqHq2DQbxO8i+c2RGCCGfv39h+Dw2GT2Ss3EQtGZr3w0Wlk/oH9HdaLIBa xXvb6qVOsEZyqKd/HY+oAumU7DwnVFQbQxdpND4WfpqU6wRTCvik6bELtwGlU1xHlOr0 YdpQ== X-Gm-Message-State: AOAM530xNJw67zt2811JvFK4WcEomvEA7JkbGaZ9aDzhCEvVLbkpi0TB gI5bAluCO97P24et5aoOyTM= X-Google-Smtp-Source: ABdhPJx6iX/VMjSijaMzPoK9DuQWOtvamU7nFGhMGyi89Cl/lcGM4rONgFuIrgrd2gOAWSyeHWMSnw== X-Received: by 2002:a65:6d0f:0:b0:3fd:8437:c35b with SMTP id bf15-20020a656d0f000000b003fd8437c35bmr9804463pgb.24.1654551868662; Mon, 06 Jun 2022 14:44:28 -0700 (PDT) Received: from localhost.biz ([8.25.197.27]) by smtp.gmail.com with ESMTPSA id a4-20020a170903100400b0016397da033csm10881675plb.62.2022.06.06.14.44.26 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 06 Jun 2022 14:44:27 -0700 (PDT) From: Yang Shi To: vbabka@suse.cz, kirill.shutemov@linux.intel.com, willy@infradead.org, akpm@linux-foundation.org Cc: shy828301@gmail.com, linux-mm@kvack.org, linux-kernel@vger.kernel.org Subject: [v3 PATCH 6/7] mm: thp: kill __transhuge_page_enabled() Date: Mon, 6 Jun 2022 14:44:13 -0700 Message-Id: <20220606214414.736109-7-shy828301@gmail.com> X-Mailer: git-send-email 2.26.3 In-Reply-To: <20220606214414.736109-1-shy828301@gmail.com> References: <20220606214414.736109-1-shy828301@gmail.com> MIME-Version: 1.0 X-Rspamd-Server: rspam06 X-Rspamd-Queue-Id: 480414007B X-Stat-Signature: c14ryommzsm4pmm45ny6rj7z65m66d4w X-Rspam-User: Authentication-Results: imf12.hostedemail.com; dkim=pass header.d=gmail.com header.s=20210112 header.b="Zl67+u/e"; spf=pass (imf12.hostedemail.com: domain of shy828301@gmail.com designates 209.85.210.177 as permitted sender) smtp.mailfrom=shy828301@gmail.com; dmarc=pass (policy=none) header.from=gmail.com X-HE-Tag: 1654551823-141738 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: The page fault path checks THP eligibility with __transhuge_page_enabled() which does the similar thing as hugepage_vma_check(), so use hugepage_vma_check() instead. However page fault allows DAX and !anon_vma cases, so added a new flag, in_pf, to hugepage_vma_check() to make page fault work correctly. The in_pf flag is also used to skip shmem and file THP for page fault since shmem handles THP in its own shmem_fault() and file THP allocation on fault is not supported yet. Also remove hugepage_vma_enabled() since hugepage_vma_check() is the only caller now, it is not necessary to have a helper function. Signed-off-by: Yang Shi --- fs/proc/task_mmu.c | 2 +- include/linux/huge_mm.h | 57 ++------------------------------------ include/linux/khugepaged.h | 2 +- mm/huge_memory.c | 25 ++++++++++++----- mm/khugepaged.c | 8 +++--- mm/memory.c | 7 +++-- 6 files changed, 31 insertions(+), 70 deletions(-) diff --git a/fs/proc/task_mmu.c b/fs/proc/task_mmu.c index fd79566e204c..a0850303baec 100644 --- a/fs/proc/task_mmu.c +++ b/fs/proc/task_mmu.c @@ -860,7 +860,7 @@ static int show_smap(struct seq_file *m, void *v) __show_smap(m, &mss, false); seq_printf(m, "THPeligible: %d\n", - hugepage_vma_check(vma, vma->vm_flags, true)); + hugepage_vma_check(vma, vma->vm_flags, true, false)); if (arch_pkeys_enabled()) seq_printf(m, "ProtectionKey: %8u\n", vma_pkey(vma)); diff --git a/include/linux/huge_mm.h b/include/linux/huge_mm.h index f561c3e16def..d478e8875023 100644 --- a/include/linux/huge_mm.h +++ b/include/linux/huge_mm.h @@ -153,48 +153,6 @@ static inline bool transhuge_vma_suitable(struct vm_area_struct *vma, return true; } -static inline bool transhuge_vma_enabled(struct vm_area_struct *vma, - unsigned long vm_flags) -{ - /* Explicitly disabled through madvise. */ - if ((vm_flags & VM_NOHUGEPAGE) || - test_bit(MMF_DISABLE_THP, &vma->vm_mm->flags)) - return false; - return true; -} - -/* - * to be used on vmas which are known to support THP. - * Use transparent_hugepage_active otherwise - */ -static inline bool __transparent_hugepage_enabled(struct vm_area_struct *vma) -{ - - /* - * If the hardware/firmware marked hugepage support disabled. - */ - if (transparent_hugepage_flags & (1 << TRANSPARENT_HUGEPAGE_NEVER_DAX)) - return false; - - if (!transhuge_vma_enabled(vma, vma->vm_flags)) - return false; - - if (vma_is_temporary_stack(vma)) - return false; - - if (transparent_hugepage_flags & (1 << TRANSPARENT_HUGEPAGE_FLAG)) - return true; - - if (vma_is_dax(vma)) - return true; - - if (transparent_hugepage_flags & - (1 << TRANSPARENT_HUGEPAGE_REQ_MADV_FLAG)) - return !!(vma->vm_flags & VM_HUGEPAGE); - - return false; -} - static inline bool file_thp_enabled(struct vm_area_struct *vma) { struct inode *inode; @@ -211,7 +169,7 @@ static inline bool file_thp_enabled(struct vm_area_struct *vma) bool hugepage_vma_check(struct vm_area_struct *vma, unsigned long vm_flags, - bool smaps); + bool smaps, bool in_pf); #define transparent_hugepage_use_zero_page() \ (transparent_hugepage_flags & \ @@ -355,11 +313,6 @@ static inline bool folio_test_pmd_mappable(struct folio *folio) return false; } -static inline bool __transparent_hugepage_enabled(struct vm_area_struct *vma) -{ - return false; -} - static inline bool transhuge_vma_size_ok(struct vm_area_struct *vma) { return false; @@ -371,15 +324,9 @@ static inline bool transhuge_vma_suitable(struct vm_area_struct *vma, return false; } -static inline bool transhuge_vma_enabled(struct vm_area_struct *vma, - unsigned long vm_flags) -{ - return false; -} - static inline bool hugepage_vma_check(struct vm_area_struct *vma, unsigned long vm_flags, - bool smaps) + bool smaps, bool in_pf) { return false; } diff --git a/include/linux/khugepaged.h b/include/linux/khugepaged.h index 8a6452e089ca..e047be601268 100644 --- a/include/linux/khugepaged.h +++ b/include/linux/khugepaged.h @@ -55,7 +55,7 @@ static inline void khugepaged_enter(struct vm_area_struct *vma, { if (!test_bit(MMF_VM_HUGEPAGE, &vma->vm_mm->flags) && khugepaged_enabled()) { - if (hugepage_vma_check(vma, vm_flags, false)) + if (hugepage_vma_check(vma, vm_flags, false, false)) __khugepaged_enter(vma->vm_mm); } } diff --git a/mm/huge_memory.c b/mm/huge_memory.c index bc8370856e85..b95786ada466 100644 --- a/mm/huge_memory.c +++ b/mm/huge_memory.c @@ -71,17 +71,25 @@ unsigned long huge_zero_pfn __read_mostly = ~0UL; bool hugepage_vma_check(struct vm_area_struct *vma, unsigned long vm_flags, - bool smaps) + bool smaps, bool in_pf) { - if (!transhuge_vma_enabled(vma, vm_flags)) + /* Explicitly disabled through madvise or prctl. */ + if ((vm_flags & VM_NOHUGEPAGE) || + test_bit(MMF_DISABLE_THP, &vma->vm_mm->flags)) + return false; + /* + * If the hardware/firmware marked hugepage support disabled. + */ + if (transparent_hugepage_flags & (1 << TRANSPARENT_HUGEPAGE_NEVER_DAX)) return false; + /* Special VMA and hugetlb VMA */ if (vm_flags & VM_NO_KHUGEPAGED) return false; - /* Don't run khugepaged against DAX vma */ + /* khugepaged doesn't collapse DAX vma, but page fault is fine. */ if (vma_is_dax(vma)) - return false; + return in_pf; if (vma->vm_file && !IS_ALIGNED((vma->vm_start >> PAGE_SHIFT) - vma->vm_pgoff, HPAGE_PMD_NR)) @@ -91,7 +99,7 @@ bool hugepage_vma_check(struct vm_area_struct *vma, return false; /* Enabled via shmem mount options or sysfs settings. */ - if (shmem_file(vma->vm_file)) + if (!in_pf && shmem_file(vma->vm_file)) return shmem_huge_enabled(vma); if (!khugepaged_enabled()) @@ -102,7 +110,7 @@ bool hugepage_vma_check(struct vm_area_struct *vma, return false; /* Only regular file is valid */ - if (file_thp_enabled(vma)) + if (!in_pf && file_thp_enabled(vma)) return true; if (!vma_is_anonymous(vma)) @@ -114,9 +122,12 @@ bool hugepage_vma_check(struct vm_area_struct *vma, /* * THPeligible bit of smaps should show 1 for proper VMAs even * though anon_vma is not initialized yet. + * + * Allow page fault since anon_vma may be not initialized until + * the first page fault. */ if (!vma->anon_vma) - return smaps; + return (smaps || in_pf); return true; } diff --git a/mm/khugepaged.c b/mm/khugepaged.c index aa0769e3b0d9..ab6183c5489f 100644 --- a/mm/khugepaged.c +++ b/mm/khugepaged.c @@ -473,7 +473,7 @@ void khugepaged_enter_vma(struct vm_area_struct *vma, { if (!test_bit(MMF_VM_HUGEPAGE, &vma->vm_mm->flags) && khugepaged_enabled()) { - if (hugepage_vma_check(vma, vm_flags, false)) + if (hugepage_vma_check(vma, vm_flags, false, false)) __khugepaged_enter(vma->vm_mm); } } @@ -918,7 +918,7 @@ static int hugepage_vma_revalidate(struct mm_struct *mm, unsigned long address, if (!transhuge_vma_suitable(vma, address)) return SCAN_ADDRESS_RANGE; - if (!hugepage_vma_check(vma, vma->vm_flags, false)) + if (!hugepage_vma_check(vma, vma->vm_flags, false, false)) return SCAN_VMA_CHECK; return 0; } @@ -1399,7 +1399,7 @@ void collapse_pte_mapped_thp(struct mm_struct *mm, unsigned long addr) * the valid THP. Add extra VM_HUGEPAGE so hugepage_vma_check() * will not fail the vma for missing VM_HUGEPAGE */ - if (!hugepage_vma_check(vma, vma->vm_flags | VM_HUGEPAGE, false)) + if (!hugepage_vma_check(vma, vma->vm_flags | VM_HUGEPAGE, false, false)) return; /* Keep pmd pgtable for uffd-wp; see comment in retract_page_tables() */ @@ -2089,7 +2089,7 @@ static unsigned int khugepaged_scan_mm_slot(unsigned int pages, progress++; break; } - if (!hugepage_vma_check(vma, vma->vm_flags, false)) { + if (!hugepage_vma_check(vma, vma->vm_flags, false, false)) { skip: progress++; continue; diff --git a/mm/memory.c b/mm/memory.c index bc5d40eec5d5..673f7561a30a 100644 --- a/mm/memory.c +++ b/mm/memory.c @@ -4962,6 +4962,7 @@ static vm_fault_t __handle_mm_fault(struct vm_area_struct *vma, .gfp_mask = __get_fault_gfp_mask(vma), }; struct mm_struct *mm = vma->vm_mm; + unsigned long vm_flags = vma->vm_flags; pgd_t *pgd; p4d_t *p4d; vm_fault_t ret; @@ -4975,7 +4976,8 @@ static vm_fault_t __handle_mm_fault(struct vm_area_struct *vma, if (!vmf.pud) return VM_FAULT_OOM; retry_pud: - if (pud_none(*vmf.pud) && __transparent_hugepage_enabled(vma)) { + if (pud_none(*vmf.pud) && + hugepage_vma_check(vma, vm_flags, false, true)) { ret = create_huge_pud(&vmf); if (!(ret & VM_FAULT_FALLBACK)) return ret; @@ -5008,7 +5010,8 @@ static vm_fault_t __handle_mm_fault(struct vm_area_struct *vma, if (pud_trans_unstable(vmf.pud)) goto retry_pud; - if (pmd_none(*vmf.pmd) && __transparent_hugepage_enabled(vma)) { + if (pmd_none(*vmf.pmd) && + hugepage_vma_check(vma, vm_flags, false, true)) { ret = create_huge_pmd(&vmf); if (!(ret & VM_FAULT_FALLBACK)) return ret; From patchwork Mon Jun 6 21:44:14 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yang Shi X-Patchwork-Id: 12871109 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id B4780CCA473 for ; Mon, 6 Jun 2022 21:44:33 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id CF19A6B007D; Mon, 6 Jun 2022 17:44:31 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id C9FD26B007E; Mon, 6 Jun 2022 17:44:31 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id AF18C8D0001; Mon, 6 Jun 2022 17:44:31 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0015.hostedemail.com [216.40.44.15]) by kanga.kvack.org (Postfix) with ESMTP id A04A86B007D for ; Mon, 6 Jun 2022 17:44:31 -0400 (EDT) Received: from smtpin31.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay06.hostedemail.com (Postfix) with ESMTP id 7875835033 for ; Mon, 6 Jun 2022 21:44:31 +0000 (UTC) X-FDA: 79549140342.31.CC61630 Received: from mail-pj1-f50.google.com (mail-pj1-f50.google.com [209.85.216.50]) by imf02.hostedemail.com (Postfix) with ESMTP id 0E40F80083 for ; Mon, 6 Jun 2022 21:44:23 +0000 (UTC) Received: by mail-pj1-f50.google.com with SMTP id l20-20020a17090a409400b001dd2a9d555bso13722549pjg.0 for ; Mon, 06 Jun 2022 14:44:30 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=yPXZCZEMUQaqYwKbEzwy4HUgqvcP6p/dfhKRtiPq7Hw=; b=JhXbAthDaDChPe0CIE4lyCWV2sCMXUskpdnFfr0MSsPjrEj6BkRL0mh0lH3+QrxtSK EenH1GV5UxsPW8wRBeB676I26nUwYNyy94grGsy5diUQYVvPnd44oNlg7fZdm7x4uOG6 GIb46aOwj5teb0/tSJ0XYsOqw+loQfIs6+8RldDC9kP/ogs6bRtwqDrtRVN3G2f8jzIL oJbi95NWG0ZA1uTgB5GJxIstWMhDTTzkDHR1Mi7qJ9MVQGLULQyLixJbiyhWH630r56+ zV+o2dt6Pr/AV5Sh+k3eIMVFA4SjiiZros8bZtNrYeNBlcZ/VwZEx/OIXCs/JU61QD7p FzKw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=yPXZCZEMUQaqYwKbEzwy4HUgqvcP6p/dfhKRtiPq7Hw=; b=s//sRQ8oprJC3vYhCi6+Jx2V0ohmtbZq3/v8jP+20IXm9b4F+46hFuCrV8Z2Jtf5nF ZbYARGoJ68GSDlOTe3jV40u72LLZuMJLSzTRjAnl9IJ+/wpx2+N3ggeh8q9yHcX/5SFK g0Kc5qMOhUISuQShoQsqk++cU0pAZlm8m7H8WNqFL33h6nIJZJZAJ/GUBwsEfvC5X7ds MogwYw/uciBQH/hwmHy0/eDlhze0N3QD6+PbP7INDylz/OYcMfVRcobMGI7IjUJJ9TKi mjqtdK9eehqVA6J2WDX1pRzHWAFyUoANhmJq/+YtPjsIkpsEGdPwG00DOexsUaADQ4Jt Y8fA== X-Gm-Message-State: AOAM533q/qcc/sZv2kdpVhJajzWtWPdvA+eOqFFDg+dJvN/TkTLyHkKi AwURq3udCu49/i24taTdpLEY3MdfnRUMMA== X-Google-Smtp-Source: ABdhPJyDZQtvOLvDSpVY2YEfud++1wRn/OaNDsUtVpZmzEHtUgNOTteY+F+x1j6onYQEtMIS01ebLQ== X-Received: by 2002:a17:902:c2c1:b0:161:c6af:278e with SMTP id c1-20020a170902c2c100b00161c6af278emr26163480pla.151.1654551870055; Mon, 06 Jun 2022 14:44:30 -0700 (PDT) Received: from localhost.biz ([8.25.197.27]) by smtp.gmail.com with ESMTPSA id a4-20020a170903100400b0016397da033csm10881675plb.62.2022.06.06.14.44.28 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 06 Jun 2022 14:44:29 -0700 (PDT) From: Yang Shi To: vbabka@suse.cz, kirill.shutemov@linux.intel.com, willy@infradead.org, akpm@linux-foundation.org Cc: shy828301@gmail.com, linux-mm@kvack.org, linux-kernel@vger.kernel.org Subject: [v3 PATCH 7/7] mm: khugepaged: reorg some khugepaged helpers Date: Mon, 6 Jun 2022 14:44:14 -0700 Message-Id: <20220606214414.736109-8-shy828301@gmail.com> X-Mailer: git-send-email 2.26.3 In-Reply-To: <20220606214414.736109-1-shy828301@gmail.com> References: <20220606214414.736109-1-shy828301@gmail.com> MIME-Version: 1.0 Authentication-Results: imf02.hostedemail.com; dkim=pass header.d=gmail.com header.s=20210112 header.b=JhXbAthD; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (imf02.hostedemail.com: domain of shy828301@gmail.com designates 209.85.216.50 as permitted sender) smtp.mailfrom=shy828301@gmail.com X-Rspamd-Server: rspam03 X-Rspamd-Queue-Id: 0E40F80083 X-Rspam-User: X-Stat-Signature: dbbskkh1n4p83xbid9hronfazpunr8oe X-HE-Tag: 1654551863-527486 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: The khugepaged_{enabled|always|req_madv} are not khugepaged only anymore, move them to huge_mm.h and rename to hugepage_flags_xxx, and remove khugepaged_req_madv due to no users. Also move khugepaged_defrag to khugepaged.c since its only caller is in that file, it doesn't have to be in a header file. Signed-off-by: Yang Shi --- include/linux/huge_mm.h | 8 ++++++++ include/linux/khugepaged.h | 17 +---------------- mm/huge_memory.c | 4 ++-- mm/khugepaged.c | 18 +++++++++++------- 4 files changed, 22 insertions(+), 25 deletions(-) diff --git a/include/linux/huge_mm.h b/include/linux/huge_mm.h index d478e8875023..ce2d05ee4816 100644 --- a/include/linux/huge_mm.h +++ b/include/linux/huge_mm.h @@ -116,6 +116,14 @@ extern struct kobj_attribute shmem_enabled_attr; extern unsigned long transparent_hugepage_flags; +#define hugepage_flags_enabled() \ + (transparent_hugepage_flags & \ + ((1<flags)) @@ -53,8 +39,7 @@ static inline void khugepaged_exit(struct mm_struct *mm) static inline void khugepaged_enter(struct vm_area_struct *vma, unsigned long vm_flags) { - if (!test_bit(MMF_VM_HUGEPAGE, &vma->vm_mm->flags) && - khugepaged_enabled()) { + if (!test_bit(MMF_VM_HUGEPAGE, &vma->vm_mm->flags)) { if (hugepage_vma_check(vma, vm_flags, false, false)) __khugepaged_enter(vma->vm_mm); } diff --git a/mm/huge_memory.c b/mm/huge_memory.c index b95786ada466..866b98a39496 100644 --- a/mm/huge_memory.c +++ b/mm/huge_memory.c @@ -102,11 +102,11 @@ bool hugepage_vma_check(struct vm_area_struct *vma, if (!in_pf && shmem_file(vma->vm_file)) return shmem_huge_enabled(vma); - if (!khugepaged_enabled()) + if (!hugepage_flags_enabled()) return false; /* THP settings require madvise. */ - if (!(vm_flags & VM_HUGEPAGE) && !khugepaged_always()) + if (!(vm_flags & VM_HUGEPAGE) && !hugepage_flags_always()) return false; /* Only regular file is valid */ diff --git a/mm/khugepaged.c b/mm/khugepaged.c index ab6183c5489f..2523c085625a 100644 --- a/mm/khugepaged.c +++ b/mm/khugepaged.c @@ -472,7 +472,7 @@ void khugepaged_enter_vma(struct vm_area_struct *vma, unsigned long vm_flags) { if (!test_bit(MMF_VM_HUGEPAGE, &vma->vm_mm->flags) && - khugepaged_enabled()) { + hugepage_flags_enabled()) { if (hugepage_vma_check(vma, vm_flags, false, false)) __khugepaged_enter(vma->vm_mm); } @@ -763,6 +763,10 @@ static bool khugepaged_scan_abort(int nid) return false; } +#define khugepaged_defrag() \ + (transparent_hugepage_flags & \ + (1<