From patchwork Tue Jul 16 13:59:04 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ryan Roberts X-Patchwork-Id: 13734539 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id D8FDFC3DA49 for ; Tue, 16 Jul 2024 13:59:20 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 6EDB16B00A5; Tue, 16 Jul 2024 09:59:20 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 64E8D6B00A8; Tue, 16 Jul 2024 09:59:20 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 47C296B00A9; Tue, 16 Jul 2024 09:59:20 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0014.hostedemail.com [216.40.44.14]) by kanga.kvack.org (Postfix) with ESMTP id 239466B00A5 for ; Tue, 16 Jul 2024 09:59:20 -0400 (EDT) Received: from smtpin05.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay10.hostedemail.com (Postfix) with ESMTP id CC972C05CB for ; Tue, 16 Jul 2024 13:59:19 +0000 (UTC) X-FDA: 82345772838.05.62FEC15 Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by imf14.hostedemail.com (Postfix) with ESMTP id 2F96D100011 for ; Tue, 16 Jul 2024 13:59:17 +0000 (UTC) Authentication-Results: imf14.hostedemail.com; dkim=none; dmarc=pass (policy=none) header.from=arm.com; spf=pass (imf14.hostedemail.com: domain of ryan.roberts@arm.com designates 217.140.110.172 as permitted sender) smtp.mailfrom=ryan.roberts@arm.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1721138329; a=rsa-sha256; cv=none; b=yaMwjh0kWZCLduVU9uRg1rvVgJXdf8j3XmWiXRwgTIZox7Z+BuQ57hx4CcNerFBDzsUQO3 IGUFxV1vxDnto2giGFSWe/NtGXwD9ACYTQqabSO7xgn684rW3xDNHcERZzA7SsYLvPqclm 1Tf4GCjtCwecd0Lat5x+l4kxRgZ+1bY= ARC-Authentication-Results: i=1; imf14.hostedemail.com; dkim=none; dmarc=pass (policy=none) header.from=arm.com; spf=pass (imf14.hostedemail.com: domain of ryan.roberts@arm.com designates 217.140.110.172 as permitted sender) smtp.mailfrom=ryan.roberts@arm.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1721138329; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=jmrStxLjhMaj29NsB3Tqe1xB1c2JHYb2g5NuK9PJop8=; b=vJE8xXj6PyjItxXjBZQPckyROBmxLYVzxliSW6CBGjiZUHRbibvnXyPAZFkRDpJOZ6cjmU UjpOvr2HexOVIxC3XsbLCEuznmKiWFeeKtODsPcGYEkR2g6uV0UyQTZRadICcsRK7W5SxB 2inO7B2xZqRhilRdPTBvo+nj+JV+RYU= Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id BE2AF106F; Tue, 16 Jul 2024 06:59:42 -0700 (PDT) Received: from e125769.cambridge.arm.com (e125769.cambridge.arm.com [10.1.196.27]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id 00A7F3F762; Tue, 16 Jul 2024 06:59:15 -0700 (PDT) From: Ryan Roberts To: Andrew Morton , Hugh Dickins , Jonathan Corbet , "Matthew Wilcox (Oracle)" , David Hildenbrand , Barry Song , Lance Yang , Baolin Wang , Gavin Shan Cc: Ryan Roberts , linux-kernel@vger.kernel.org, linux-mm@kvack.org Subject: [PATCH v2 1/3] mm: Cleanup count_mthp_stat() definition Date: Tue, 16 Jul 2024 14:59:04 +0100 Message-ID: <20240716135907.4047689-2-ryan.roberts@arm.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20240716135907.4047689-1-ryan.roberts@arm.com> References: <20240716135907.4047689-1-ryan.roberts@arm.com> MIME-Version: 1.0 X-Rspamd-Queue-Id: 2F96D100011 X-Rspam-User: X-Rspamd-Server: rspam05 X-Stat-Signature: djgsg78bubs7ufif5ndpuoqgweyy8b5u X-HE-Tag: 1721138357-788899 X-HE-Meta: U2FsdGVkX19pvB9irzrulwPjq7+Am60ihw3iNYxKtI3+aS58hIe565h9Ge9pRWNIIMwzqYOyLPl1/ELEqcNbiHKLg2YiKmfCRCwGCfDyHTrLjBQ+11oq1yh1McBiMfA4dPfoV5opfDbFySZ8zh/868Cq2fawKOpSvbbrPbQhG+32VIq1t7TTWSNIqBT310YrH+kbH2Cj7wawCMiT/4ZsPuO326I6Y7jIq1hCRGkUGRhQVdkMPuk22JrCbLSnFN9PNdsv2kELUKYMRTYN9kO4w7ihL5idj9kViRDFHVL2sWzMe/aKj+7Ee7m0xj25OnDhCZtgcwCJB3aXTC8IYkVUnELdRoOTyH5ow5Sv7DPukze8S7sR15lE84VsA2TEvvnjgWZPvHfFLy2EstjAKIysRRAvyjDPzkf/jRkFtdaS+drM4PxSCIjUtrhP8d7kCrZM7uQMqqjgSHj5hUlVanHttH5KeWiI4iyYdp0G6iDdpaNLqLm3NZrtNT+uG9uIG3lmlD5MGyInJXPTVCWMdr4tlVjXm1ekJvaiiJsYKWlkGX6MDiJDO1nsnLDi6YBUvqJjMPHMNljl6aCYMGEJGz1WtIltU9J+ED4X5ag5Q7M9Ed2Uzbh88IYqwB12qP68q4yIJtIXdKqgspTDgTbtOiG99gaKPyr/j8tgTKnGkZuYfKqlIaahWZejlK8+fp5La0uiB2zSm/W8f9tCnqKPwEY61o9qia6qszV0L4fjGTggTP4ZiYlub30ql43FT/pOdOvzz4E/9uP2HebmyFQJ2UtgLOonjJO9eB4wR9zeSM4TU+agquapoi4ViDlGiD38Xj5k4ZS/SKzqYaCgw7Ub3TglIHu7RPxD7IXI6Uw7FDwYETscQ9A0tZ/EydmbiqNDb2i7+btoCvDMQ3u9bHPi7DRNnLFdHpzl8pQEVIUoYfVCXZYT14UV6JBdXJw6wrBw2YUWJKwl7BTMTqazaHFpFvJ 3JJnRljq WhX+Rw/MOHFb3ZpZPjOoWQ3CGTa32ySkgzUZAmHnbunQjPES/UGiNAz78cqEYeq8frfz2QzZ5oUNCDyl1KfyQxxBzwatrzL4VZAaY3o8XouZtgc4CtzidZ1hYfbNeQAc+fgAr5G5qXWf/56xUO2j1ZVg1c7YZJK6OOhwXMHSezRVtgEC2m1XYHzurBIMOkH2M9NN4rcg5D5CFv0rTo5JhEPSrukHDVLI+I1v146VCJOd2W+K6Lt8PkKyvdagw20TU0G9fisfOMsdCbyoDf66R7ch5TNCJk4Ni5uG8XgCdd+mjVzo9CAZJhxKSmlAL0HTSNjDmPe0zAMQjhfM= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Let's move count_mthp_stat() so that it's always defined, even when THP is disabled. Previously uses of the function in files such as shmem.c, which are compiled even when THP is disabled, required ugly THP ifdeferry. With this cleanup, we can remove those ifdefs and the function resolves to a nop when THP is disabled. I shortly plan to call count_mthp_stat() from more THP-invariant source files. Signed-off-by: Ryan Roberts Acked-by: Barry Song Reviewed-by: Baolin Wang Reviewed-by: Lance Yang Acked-by: David Hildenbrand --- include/linux/huge_mm.h | 70 ++++++++++++++++++++--------------------- mm/memory.c | 2 -- mm/shmem.c | 6 ---- 3 files changed, 35 insertions(+), 43 deletions(-) diff --git a/include/linux/huge_mm.h b/include/linux/huge_mm.h index e25d9ebfdf89..b8c63c3e967f 100644 --- a/include/linux/huge_mm.h +++ b/include/linux/huge_mm.h @@ -114,6 +114,41 @@ extern struct kobj_attribute thpsize_shmem_enabled_attr; #define HPAGE_PUD_MASK (~(HPAGE_PUD_SIZE - 1)) #define HPAGE_PUD_SIZE ((1UL) << HPAGE_PUD_SHIFT) +enum mthp_stat_item { + MTHP_STAT_ANON_FAULT_ALLOC, + MTHP_STAT_ANON_FAULT_FALLBACK, + MTHP_STAT_ANON_FAULT_FALLBACK_CHARGE, + MTHP_STAT_SWPOUT, + MTHP_STAT_SWPOUT_FALLBACK, + MTHP_STAT_SHMEM_ALLOC, + MTHP_STAT_SHMEM_FALLBACK, + MTHP_STAT_SHMEM_FALLBACK_CHARGE, + MTHP_STAT_SPLIT, + MTHP_STAT_SPLIT_FAILED, + MTHP_STAT_SPLIT_DEFERRED, + __MTHP_STAT_COUNT +}; + +#if defined(CONFIG_TRANSPARENT_HUGEPAGE) && defined(CONFIG_SYSFS) +struct mthp_stat { + unsigned long stats[ilog2(MAX_PTRS_PER_PTE) + 1][__MTHP_STAT_COUNT]; +}; + +DECLARE_PER_CPU(struct mthp_stat, mthp_stats); + +static inline void count_mthp_stat(int order, enum mthp_stat_item item) +{ + if (order <= 0 || order > PMD_ORDER) + return; + + this_cpu_inc(mthp_stats.stats[order][item]); +} +#else +static inline void count_mthp_stat(int order, enum mthp_stat_item item) +{ +} +#endif + #ifdef CONFIG_TRANSPARENT_HUGEPAGE extern unsigned long transparent_hugepage_flags; @@ -269,41 +304,6 @@ struct thpsize { #define to_thpsize(kobj) container_of(kobj, struct thpsize, kobj) -enum mthp_stat_item { - MTHP_STAT_ANON_FAULT_ALLOC, - MTHP_STAT_ANON_FAULT_FALLBACK, - MTHP_STAT_ANON_FAULT_FALLBACK_CHARGE, - MTHP_STAT_SWPOUT, - MTHP_STAT_SWPOUT_FALLBACK, - MTHP_STAT_SHMEM_ALLOC, - MTHP_STAT_SHMEM_FALLBACK, - MTHP_STAT_SHMEM_FALLBACK_CHARGE, - MTHP_STAT_SPLIT, - MTHP_STAT_SPLIT_FAILED, - MTHP_STAT_SPLIT_DEFERRED, - __MTHP_STAT_COUNT -}; - -struct mthp_stat { - unsigned long stats[ilog2(MAX_PTRS_PER_PTE) + 1][__MTHP_STAT_COUNT]; -}; - -#ifdef CONFIG_SYSFS -DECLARE_PER_CPU(struct mthp_stat, mthp_stats); - -static inline void count_mthp_stat(int order, enum mthp_stat_item item) -{ - if (order <= 0 || order > PMD_ORDER) - return; - - this_cpu_inc(mthp_stats.stats[order][item]); -} -#else -static inline void count_mthp_stat(int order, enum mthp_stat_item item) -{ -} -#endif - #define transparent_hugepage_use_zero_page() \ (transparent_hugepage_flags & \ (1<vm_mm, MM_ANONPAGES, nr_pages); -#ifdef CONFIG_TRANSPARENT_HUGEPAGE count_mthp_stat(folio_order(folio), MTHP_STAT_ANON_FAULT_ALLOC); -#endif folio_add_new_anon_rmap(folio, vma, addr, RMAP_EXCLUSIVE); folio_add_lru_vma(folio, vma); setpte: diff --git a/mm/shmem.c b/mm/shmem.c index f24dfbd387ba..fce1343f44e6 100644 --- a/mm/shmem.c +++ b/mm/shmem.c @@ -1776,9 +1776,7 @@ static struct folio *shmem_alloc_and_add_folio(struct vm_fault *vmf, if (pages == HPAGE_PMD_NR) count_vm_event(THP_FILE_FALLBACK); -#ifdef CONFIG_TRANSPARENT_HUGEPAGE count_mthp_stat(order, MTHP_STAT_SHMEM_FALLBACK); -#endif order = next_order(&suitable_orders, order); } } else { @@ -1803,10 +1801,8 @@ static struct folio *shmem_alloc_and_add_folio(struct vm_fault *vmf, count_vm_event(THP_FILE_FALLBACK); count_vm_event(THP_FILE_FALLBACK_CHARGE); } -#ifdef CONFIG_TRANSPARENT_HUGEPAGE count_mthp_stat(folio_order(folio), MTHP_STAT_SHMEM_FALLBACK); count_mthp_stat(folio_order(folio), MTHP_STAT_SHMEM_FALLBACK_CHARGE); -#endif } goto unlock; } @@ -2180,9 +2176,7 @@ static int shmem_get_folio_gfp(struct inode *inode, pgoff_t index, if (!IS_ERR(folio)) { if (folio_test_pmd_mappable(folio)) count_vm_event(THP_FILE_ALLOC); -#ifdef CONFIG_TRANSPARENT_HUGEPAGE count_mthp_stat(folio_order(folio), MTHP_STAT_SHMEM_ALLOC); -#endif goto alloced; } if (PTR_ERR(folio) == -EEXIST)