From patchwork Fri Aug 12 01:28:35 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Zach O'Keefe X-Patchwork-Id: 12941849 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 00687C25B0F for ; Fri, 12 Aug 2022 01:29:16 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 5B9AE6B0073; Thu, 11 Aug 2022 21:29:16 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 53FFE8E0002; Thu, 11 Aug 2022 21:29:16 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 3935A6B0078; Thu, 11 Aug 2022 21:29:16 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0013.hostedemail.com [216.40.44.13]) by kanga.kvack.org (Postfix) with ESMTP id 27E9B6B0073 for ; Thu, 11 Aug 2022 21:29:16 -0400 (EDT) Received: from smtpin25.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay01.hostedemail.com (Postfix) with ESMTP id 13EBF1C734E for ; Fri, 12 Aug 2022 01:29:14 +0000 (UTC) X-FDA: 79789207428.25.E90CE58 Received: from mail-pj1-f74.google.com (mail-pj1-f74.google.com [209.85.216.74]) by imf26.hostedemail.com (Postfix) with ESMTP id A9E0C140060 for ; Fri, 12 Aug 2022 01:29:13 +0000 (UTC) Received: by mail-pj1-f74.google.com with SMTP id on14-20020a17090b1d0e00b001f842dd5e90so3047379pjb.5 for ; Thu, 11 Aug 2022 18:29:13 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20210112; h=cc:to:from:subject:references:mime-version:message-id:in-reply-to :date:from:to:cc; bh=3iNP6zzDyFKPjsrNUzPJ8WhOfhtMXJft8o7718TkpWE=; b=pdm58Oa3Sy2Q4gJG2uJa0h3HSlrvH4VUtS4MBRAotBmpCrP4VfML/N/WzeRAHZ9Xy3 M6Q4b974ahYiDBcSTpFgwWt1/hmjELsDMPJBgRW12s+ZrdVVfoVA88RDyxw9KBXPdtDS GwZN1rTRuiTZTxNniPCals66YxkQBeDcCJDd/5xSSRjzTzyMUv3nrHTBE7y/f9HNYeBs zWgQhK+NYQepy9gxKZPkxyUKAANjkA7W2McNEs9kX3assJUll4aYvxSKmWD1g+jo///D S4aMK8yna5QPJ1WZnsUsMHN8fGn6NrOPWKKs4p0fxDByrnZ6Uib0VO5B9ejPRwqz8Y8W d+rg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=cc:to:from:subject:references:mime-version:message-id:in-reply-to :date:x-gm-message-state:from:to:cc; bh=3iNP6zzDyFKPjsrNUzPJ8WhOfhtMXJft8o7718TkpWE=; b=zVW65CPkoTlLLT0MqybZV42tP5J62tMlPEcqHDpT7GPUibVHaxGzXN5ygBCDJ4DjJQ f3f7Yg9TvCdK2aYks4nU4/7DuXNe8OqTo7IQtYBgho5lUPjnsJyqn9feI1jCpViYsWXL gPcLIGym1YhirV++F7yHElYo1uFUZmdbxdNKpLDcLRbYc65znkqhZ0hTgdww9gzFsaai pO2W6xMyFTWQgwvNVBDNil4XD7fbonTtqOT3dPP6QmXkBD8pxZyYfjaj+HKdmeBS8Pg/ RHAHlvaq/gN1TN3zuWNA0t4xCaENMdk++w6iPI++DL76ColmpIXC0yJjffLbPF+QnLIk N/3g== X-Gm-Message-State: ACgBeo3nbl/AVFJKSVxw57csAfpBI2NF04fxWpU/oXxZWhnOho4c7GJO UqInv3LLVaVMUkwi3j0W+4cgq2zXSG9Odq+ygWxRzKp54GUUmXd2kRZJmCgpHuoHVkdiVqpnRo1 QJgS7gYtgk4YJdeFOHHkSjc32yIOZEBccbmW9uOlu6u48MSJu6U9ZD4pe3Hg= X-Google-Smtp-Source: AA6agR5T6qes+UZZRpwNV667eYHAPei2wa9gIaW+alTbufHg04rqIT/4gFGD4y9PawJFUtufj1YABMoeXd20 X-Received: from zokeefe3.c.googlers.com ([fda3:e722:ac3:cc00:7f:e700:c0a8:1b6]) (user=zokeefe job=sendgmr) by 2002:a05:6a00:428d:b0:52e:6305:14c with SMTP id bx13-20020a056a00428d00b0052e6305014cmr1486532pfb.10.1660267752438; Thu, 11 Aug 2022 18:29:12 -0700 (PDT) Date: Thu, 11 Aug 2022 18:28:35 -0700 In-Reply-To: <20220812012843.3948330-1-zokeefe@google.com> Message-Id: <20220812012843.3948330-2-zokeefe@google.com> Mime-Version: 1.0 References: <20220812012843.3948330-1-zokeefe@google.com> X-Mailer: git-send-email 2.37.1.559.g78731f0fdb-goog Subject: [PATCH mm-unstable 1/9] mm/shmem: add flag to enforce shmem THP in hugepage_vma_check() From: "Zach O'Keefe" To: linux-mm@kvack.org Cc: Andrew Morton , linux-api@vger.kernel.org, Axel Rasmussen , James Houghton , Hugh Dickins , Yang Shi , Miaohe Lin , David Hildenbrand , David Rientjes , Matthew Wilcox , Pasha Tatashin , Peter Xu , Rongwei Wang , SeongJae Park , Song Liu , Vlastimil Babka , Chris Kennelly , "Kirill A. Shutemov" , Minchan Kim , Patrick Xia , "Zach O'Keefe" ARC-Authentication-Results: i=1; imf26.hostedemail.com; dkim=pass header.d=google.com header.s=20210112 header.b=pdm58Oa3; dmarc=pass (policy=reject) header.from=google.com; spf=pass (imf26.hostedemail.com: domain of 36Kz1YgcKCDcshdXXYXZhhZeX.Vhfebgnq-ffdoTVd.hkZ@flex--zokeefe.bounces.google.com designates 209.85.216.74 as permitted sender) smtp.mailfrom=36Kz1YgcKCDcshdXXYXZhhZeX.Vhfebgnq-ffdoTVd.hkZ@flex--zokeefe.bounces.google.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1660267753; a=rsa-sha256; cv=none; b=Y8hU5PSuOBd5e5gNlbv/P58nHBVW/at13zkJnduBD26GSN+6b8sN5Or9ajcTvGfRFwEi8j 8ufUsuGybbJm0QGIAwoslJa/efm/Yo+wdqwYgz0eHhI1ry2DYQt3QYmXYyWI+sTxUWC5D+ cQjYHpj8ocezTu0Q8Mx80SJzV5+HtwY= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1660267753; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=3iNP6zzDyFKPjsrNUzPJ8WhOfhtMXJft8o7718TkpWE=; b=5gcnB8qcQsuTjsYAAlUgCYQ/c94EqSdq8auLmiKhLATxgxQKwyHvzQNRpxFdgtc9ZCpcd6 YIvIi9w0JBrCc3ZgIGop7FsEPCoAIH5rf3iVgKg9e9em6+COKvOjqNDNQSaNLIHIrA1/Yj ZWgl+nDkEtd3TWQ/plP3zglQ4q9uVfs= X-Rspam-User: Authentication-Results: imf26.hostedemail.com; dkim=pass header.d=google.com header.s=20210112 header.b=pdm58Oa3; dmarc=pass (policy=reject) header.from=google.com; spf=pass (imf26.hostedemail.com: domain of 36Kz1YgcKCDcshdXXYXZhhZeX.Vhfebgnq-ffdoTVd.hkZ@flex--zokeefe.bounces.google.com designates 209.85.216.74 as permitted sender) smtp.mailfrom=36Kz1YgcKCDcshdXXYXZhhZeX.Vhfebgnq-ffdoTVd.hkZ@flex--zokeefe.bounces.google.com X-Stat-Signature: 1eoq1pmbgdq5gp9jfna6s9a3hecg79nu X-Rspamd-Server: rspam11 X-Rspamd-Queue-Id: A9E0C140060 X-HE-Tag: 1660267753-948301 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Extend 'mm/thp: add flag to enforce sysfs THP in hugepage_vma_check()' to shmem, allowing callers to ignore /sys/kernel/transparent_hugepage/shmem_enabled and tmpfs huge= mount. This is intended to be used by MADV_COLLAPSE, and the rationale is analogous to the anon/file case: MADV_COLLAPSE is not coupled to directives that advise the kernel's decisions on when THPs should be considered eligible. shmem/tmpfs always claims large folio support, regardless of sysfs or mount options. Signed-off-by: Zach O'Keefe --- include/linux/shmem_fs.h | 10 ++++++---- mm/huge_memory.c | 2 +- mm/shmem.c | 18 +++++++++--------- 3 files changed, 16 insertions(+), 14 deletions(-) diff --git a/include/linux/shmem_fs.h b/include/linux/shmem_fs.h index ff0b990de83d..f5e9b01dbf4c 100644 --- a/include/linux/shmem_fs.h +++ b/include/linux/shmem_fs.h @@ -92,11 +92,13 @@ extern struct page *shmem_read_mapping_page_gfp(struct address_space *mapping, extern void shmem_truncate_range(struct inode *inode, loff_t start, loff_t end); int shmem_unuse(unsigned int type); -extern bool shmem_is_huge(struct vm_area_struct *vma, - struct inode *inode, pgoff_t index); -static inline bool shmem_huge_enabled(struct vm_area_struct *vma) +extern bool shmem_is_huge(struct vm_area_struct *vma, struct inode *inode, + pgoff_t index, bool shmem_huge_force); +static inline bool shmem_huge_enabled(struct vm_area_struct *vma, + bool shmem_huge_force) { - return shmem_is_huge(vma, file_inode(vma->vm_file), vma->vm_pgoff); + return shmem_is_huge(vma, file_inode(vma->vm_file), vma->vm_pgoff, + shmem_huge_force); } extern unsigned long shmem_swap_usage(struct vm_area_struct *vma); extern unsigned long shmem_partial_swap_usage(struct address_space *mapping, diff --git a/mm/huge_memory.c b/mm/huge_memory.c index ac858fd9c1f1..af154d31ffe1 100644 --- a/mm/huge_memory.c +++ b/mm/huge_memory.c @@ -118,7 +118,7 @@ bool hugepage_vma_check(struct vm_area_struct *vma, unsigned long vm_flags, * own flags. */ if (!in_pf && shmem_file(vma->vm_file)) - return shmem_huge_enabled(vma); + return shmem_huge_enabled(vma, !enforce_sysfs); /* Enforce sysfs THP requirements as necessary */ if (enforce_sysfs && diff --git a/mm/shmem.c b/mm/shmem.c index 42e5888bf84d..b9bab1abf142 100644 --- a/mm/shmem.c +++ b/mm/shmem.c @@ -472,20 +472,20 @@ static bool shmem_confirm_swap(struct address_space *mapping, static int shmem_huge __read_mostly = SHMEM_HUGE_NEVER; -bool shmem_is_huge(struct vm_area_struct *vma, - struct inode *inode, pgoff_t index) +bool shmem_is_huge(struct vm_area_struct *vma, struct inode *inode, + pgoff_t index, bool shmem_huge_force) { loff_t i_size; if (!S_ISREG(inode->i_mode)) return false; - if (shmem_huge == SHMEM_HUGE_DENY) - return false; if (vma && ((vma->vm_flags & VM_NOHUGEPAGE) || test_bit(MMF_DISABLE_THP, &vma->vm_mm->flags))) return false; - if (shmem_huge == SHMEM_HUGE_FORCE) + if (shmem_huge == SHMEM_HUGE_FORCE || shmem_huge_force) return true; + if (shmem_huge == SHMEM_HUGE_DENY) + return false; switch (SHMEM_SB(inode->i_sb)->huge) { case SHMEM_HUGE_ALWAYS: @@ -680,8 +680,8 @@ static long shmem_unused_huge_count(struct super_block *sb, #define shmem_huge SHMEM_HUGE_DENY -bool shmem_is_huge(struct vm_area_struct *vma, - struct inode *inode, pgoff_t index) +bool shmem_is_huge(struct vm_area_struct *vma, struct inode *inode, + pgoff_t index, bool shmem_huge_force) { return false; } @@ -1069,7 +1069,7 @@ static int shmem_getattr(struct user_namespace *mnt_userns, STATX_ATTR_NODUMP); generic_fillattr(&init_user_ns, inode, stat); - if (shmem_is_huge(NULL, inode, 0)) + if (shmem_is_huge(NULL, inode, 0, false)) stat->blksize = HPAGE_PMD_SIZE; if (request_mask & STATX_BTIME) { @@ -1910,7 +1910,7 @@ static int shmem_getpage_gfp(struct inode *inode, pgoff_t index, return 0; } - if (!shmem_is_huge(vma, inode, index)) + if (!shmem_is_huge(vma, inode, index, false)) goto alloc_nohuge; huge_gfp = vma_thp_gfp_mask(vma);