From patchwork Wed Jun 15 17:29:21 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yang Shi X-Patchwork-Id: 12882801 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 0F37CCCA473 for ; Wed, 15 Jun 2022 17:29:41 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 68B966B0075; Wed, 15 Jun 2022 13:29:40 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 613406B0078; Wed, 15 Jun 2022 13:29:40 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 3F1A96B007B; Wed, 15 Jun 2022 13:29:40 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0013.hostedemail.com [216.40.44.13]) by kanga.kvack.org (Postfix) with ESMTP id 304346B0075 for ; Wed, 15 Jun 2022 13:29:40 -0400 (EDT) Received: from smtpin24.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay06.hostedemail.com (Postfix) with ESMTP id EA269353B9 for ; Wed, 15 Jun 2022 17:29:39 +0000 (UTC) X-FDA: 79581157278.24.4BFB853 Received: from mail-pj1-f43.google.com (mail-pj1-f43.google.com [209.85.216.43]) by imf09.hostedemail.com (Postfix) with ESMTP id 8743B14008D for ; Wed, 15 Jun 2022 17:29:36 +0000 (UTC) Received: by mail-pj1-f43.google.com with SMTP id g16-20020a17090a7d1000b001ea9f820449so2754824pjl.5 for ; Wed, 15 Jun 2022 10:29:36 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=51sNwNc9iGRZC8aX2+MPWXbJAssH24vr8KC9AQWwCZ4=; b=DA1ymx6Eusxll6nKGGOtuCA1e5yLxoL17ga00PUZEbMSGUwZ9QG6+NBJuPfd7ilrmc vxzciGDA9U1JKH4VjkUhxyoesCyPzaLeNmcLKGq2yivT4DaqFKlc2ONucsFlXYa0GFhn nlwiSSWq4Fu6wWNsgWNPShUy524Ut88IgYHLjMWkb9gyH3pmJkLL99fTKnmqxnJZh/cu q6A8rMdMbGH6pv03ku6Mp75PoUjbWnjkDtLFaQBSegDQOFObnqm0jYznkWl4N7JJycvK U7IW31KkKAzmgrgRgQw3xFuSzOZqlDBEt+MnTO135pasLPl+TY1lV5wINAfU/h9zAtby eaiQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=51sNwNc9iGRZC8aX2+MPWXbJAssH24vr8KC9AQWwCZ4=; b=OX8ML0s9tgF3wkEIRJDtO/IGXiQdhjqa4XYZDVm8yJcoi7jBYcd8WhJb9gxIJB23VN 1aDz/jHNpWggxg2nCulUVY3wQzNsmAo5wbjo/iEyU7oy1PYmgJ7S5MOo2SUUzhaI2zim S8AnmNv52BxDuoKInHSwWJt1s1yyxAFz6EVHBVJs7JgHoh1NeWQdx63IBQNjeOrxMQGH oyhEScsNkxpK4Y27D6FNwR9VeW6U4eYzJ8bL5jdxKzU351BwRCy7UgAnJO97nttK13CH sjlH6QxkCnEWeBcdL213ZrGJIyGwTmd4MFTXeeWRQ5H/y+E89/AK1gf25xP4VJd2oh/H fX6w== X-Gm-Message-State: AJIora+Ud1F9zocAkWCOr/bsFYnHhPJ3N5mf1QEXOHZqdB2PfjoFRrjS MvBH6aNGBgxuczjay212q3YWJQdhxmk= X-Google-Smtp-Source: AGRyM1ubIIBH2uc5bj3/MaNUe6wnfsdVL/fOlKAmVLhqdVvcHkAScJirEVp37jaO3ZqktGgU+ZYlcg== X-Received: by 2002:a17:90a:31cf:b0:1c9:f9b8:68c7 with SMTP id j15-20020a17090a31cf00b001c9f9b868c7mr11499592pjf.34.1655314176105; Wed, 15 Jun 2022 10:29:36 -0700 (PDT) Received: from localhost.localdomain (c-67-174-241-145.hsd1.ca.comcast.net. [67.174.241.145]) by smtp.gmail.com with ESMTPSA id w14-20020a1709029a8e00b001676f87473fsm9552244plp.302.2022.06.15.10.29.34 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 15 Jun 2022 10:29:35 -0700 (PDT) From: Yang Shi To: vbabka@suse.cz, kirill.shutemov@linux.intel.com, willy@infradead.org, zokeefe@google.com, linmiaohe@huawei.com, akpm@linux-foundation.org Cc: shy828301@gmail.com, linux-mm@kvack.org, linux-kernel@vger.kernel.org Subject: [v4 PATCH 2/7] mm: thp: consolidate vma size check to transhuge_vma_suitable Date: Wed, 15 Jun 2022 10:29:21 -0700 Message-Id: <20220615172926.546974-3-shy828301@gmail.com> X-Mailer: git-send-email 2.26.3 In-Reply-To: <20220615172926.546974-1-shy828301@gmail.com> References: <20220615172926.546974-1-shy828301@gmail.com> MIME-Version: 1.0 ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1655314178; a=rsa-sha256; cv=none; b=ge8Yq2IHPok+PldcK50KRk7gAbNdjVLNrREXO1w+p+feqoRyWqCfZ2FdfunJDkSgcEo1tW K5I+9grlw5iLZKQPfEhczJ9Ws0mMCRPbBEGIE9noWCMJuvfNRmQztDlwQuXQszGXty8JLq Y6wf/YyqywWE7vtpnzl+C2mnyW8EstM= ARC-Authentication-Results: i=1; imf09.hostedemail.com; dkim=pass header.d=gmail.com header.s=20210112 header.b=DA1ymx6E; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (imf09.hostedemail.com: domain of shy828301@gmail.com designates 209.85.216.43 as permitted sender) smtp.mailfrom=shy828301@gmail.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1655314178; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=51sNwNc9iGRZC8aX2+MPWXbJAssH24vr8KC9AQWwCZ4=; b=1yNUYD/Sqxt48Fde9S5ZzKkyL7qZOPElAHc3W6ps2xaJUmtNpHKBcdOwqE+JkQRjcQmKHH ykVyevz2jYAwDjOq3AZr20GBSQZ+f5y3O8OR0QUNjF3YClDjXmTo2k2V26IWplpbU4wBfe /7dYFHgAbZs6Vwj/yFIR6wsfbdxy9/g= X-Rspamd-Queue-Id: 8743B14008D X-Rspam-User: Authentication-Results: imf09.hostedemail.com; dkim=pass header.d=gmail.com header.s=20210112 header.b=DA1ymx6E; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (imf09.hostedemail.com: domain of shy828301@gmail.com designates 209.85.216.43 as permitted sender) smtp.mailfrom=shy828301@gmail.com X-Rspamd-Server: rspam06 X-Stat-Signature: idw3d7awe1xbbr3pw9r4m175fyfmudzu X-HE-Tag: 1655314176-688146 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: There are couple of places that check whether the vma size is ok for THP or whether address fits, they are open coded and duplicate, use transhuge_vma_suitable() to do the job by passing in (vma->end - HPAGE_PMD_SIZE). Move vma size check into hugepage_vma_check(). This will make khugepaged_enter() is as same as khugepaged_enter_vma(). There is just one caller for khugepaged_enter(), replace it to khugepaged_enter_vma() and remove khugepaged_enter(). Signed-off-by: Yang Shi Reviewed-by: Zach O'Keefe --- include/linux/huge_mm.h | 11 +++++++++++ include/linux/khugepaged.h | 14 -------------- mm/huge_memory.c | 2 +- mm/khugepaged.c | 19 ++++++------------- 4 files changed, 18 insertions(+), 28 deletions(-) diff --git a/include/linux/huge_mm.h b/include/linux/huge_mm.h index 648cb3ce7099..8a5a8bfce0f5 100644 --- a/include/linux/huge_mm.h +++ b/include/linux/huge_mm.h @@ -116,6 +116,17 @@ extern struct kobj_attribute shmem_enabled_attr; extern unsigned long transparent_hugepage_flags; +/* + * Do the below checks: + * - For file vma, check if the linear page offset of vma is + * HPAGE_PMD_NR aligned within the file. The hugepage is + * guaranteed to be hugepage-aligned within the file, but we must + * check that the PMD-aligned addresses in the VMA map to + * PMD-aligned offsets within the file, else the hugepage will + * not be PMD-mappable. + * - For all vmas, check if the haddr is in an aligned HPAGE_PMD_SIZE + * area. + */ static inline bool transhuge_vma_suitable(struct vm_area_struct *vma, unsigned long addr) { diff --git a/include/linux/khugepaged.h b/include/linux/khugepaged.h index 392d34c3c59a..31ca8a7f78f4 100644 --- a/include/linux/khugepaged.h +++ b/include/linux/khugepaged.h @@ -51,16 +51,6 @@ static inline void khugepaged_exit(struct mm_struct *mm) if (test_bit(MMF_VM_HUGEPAGE, &mm->flags)) __khugepaged_exit(mm); } - -static inline void khugepaged_enter(struct vm_area_struct *vma, - unsigned long vm_flags) -{ - if (!test_bit(MMF_VM_HUGEPAGE, &vma->vm_mm->flags) && - khugepaged_enabled()) { - if (hugepage_vma_check(vma, vm_flags)) - __khugepaged_enter(vma->vm_mm); - } -} #else /* CONFIG_TRANSPARENT_HUGEPAGE */ static inline void khugepaged_fork(struct mm_struct *mm, struct mm_struct *oldmm) { @@ -68,10 +58,6 @@ static inline void khugepaged_fork(struct mm_struct *mm, struct mm_struct *oldmm static inline void khugepaged_exit(struct mm_struct *mm) { } -static inline void khugepaged_enter(struct vm_area_struct *vma, - unsigned long vm_flags) -{ -} static inline void khugepaged_enter_vma(struct vm_area_struct *vma, unsigned long vm_flags) { diff --git a/mm/huge_memory.c b/mm/huge_memory.c index 4f9bbb4eab23..b530462c4493 100644 --- a/mm/huge_memory.c +++ b/mm/huge_memory.c @@ -726,7 +726,7 @@ vm_fault_t do_huge_pmd_anonymous_page(struct vm_fault *vmf) return VM_FAULT_FALLBACK; if (unlikely(anon_vma_prepare(vma))) return VM_FAULT_OOM; - khugepaged_enter(vma, vma->vm_flags); + khugepaged_enter_vma(vma, vma->vm_flags); if (!(vmf->flags & FAULT_FLAG_WRITE) && !mm_forbids_zeropage(vma->vm_mm) && diff --git a/mm/khugepaged.c b/mm/khugepaged.c index b1dab94c0f1e..db0b334a7d1f 100644 --- a/mm/khugepaged.c +++ b/mm/khugepaged.c @@ -450,8 +450,8 @@ bool hugepage_vma_check(struct vm_area_struct *vma, if (vma_is_dax(vma)) return false; - if (vma->vm_file && !IS_ALIGNED((vma->vm_start >> PAGE_SHIFT) - - vma->vm_pgoff, HPAGE_PMD_NR)) + /* Check alignment for file vma and size for both file and anon vma */ + if (!transhuge_vma_suitable(vma, (vma->vm_end - HPAGE_PMD_SIZE))) return false; /* Enabled via shmem mount options or sysfs settings. */ @@ -512,9 +512,7 @@ void khugepaged_enter_vma(struct vm_area_struct *vma, unsigned long vm_flags) { if (!test_bit(MMF_VM_HUGEPAGE, &vma->vm_mm->flags) && - khugepaged_enabled() && - (((vma->vm_start + ~HPAGE_PMD_MASK) & HPAGE_PMD_MASK) < - (vma->vm_end & HPAGE_PMD_MASK))) { + khugepaged_enabled()) { if (hugepage_vma_check(vma, vm_flags)) __khugepaged_enter(vma->vm_mm); } @@ -950,7 +948,6 @@ static int hugepage_vma_revalidate(struct mm_struct *mm, unsigned long address, struct vm_area_struct **vmap) { struct vm_area_struct *vma; - unsigned long hstart, hend; if (unlikely(khugepaged_test_exit(mm))) return SCAN_ANY_PROCESS; @@ -959,9 +956,7 @@ static int hugepage_vma_revalidate(struct mm_struct *mm, unsigned long address, if (!vma) return SCAN_VMA_NULL; - hstart = (vma->vm_start + ~HPAGE_PMD_MASK) & HPAGE_PMD_MASK; - hend = vma->vm_end & HPAGE_PMD_MASK; - if (address < hstart || address + HPAGE_PMD_SIZE > hend) + if (!transhuge_vma_suitable(vma, address)) return SCAN_ADDRESS_RANGE; if (!hugepage_vma_check(vma, vma->vm_flags)) return SCAN_VMA_CHECK; @@ -2147,10 +2142,8 @@ static unsigned int khugepaged_scan_mm_slot(unsigned int pages, progress++; continue; } - hstart = (vma->vm_start + ~HPAGE_PMD_MASK) & HPAGE_PMD_MASK; - hend = vma->vm_end & HPAGE_PMD_MASK; - if (hstart >= hend) - goto skip; + hstart = round_up(vma->vm_start, HPAGE_PMD_SIZE); + hend = round_down(vma->vm_end, HPAGE_PMD_SIZE); if (khugepaged_scan.address > hend) goto skip; if (khugepaged_scan.address < hstart)