From patchwork Fri Apr 5 10:27:03 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Barry Song <21cnbao@gmail.com> X-Patchwork-Id: 13618835 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 0493EC67861 for ; Fri, 5 Apr 2024 10:27:34 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 8E3A16B014A; Fri, 5 Apr 2024 06:27:33 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 8942A6B014B; Fri, 5 Apr 2024 06:27:33 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 75B896B014C; Fri, 5 Apr 2024 06:27:33 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0013.hostedemail.com [216.40.44.13]) by kanga.kvack.org (Postfix) with ESMTP id 558206B014A for ; Fri, 5 Apr 2024 06:27:33 -0400 (EDT) Received: from smtpin27.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay01.hostedemail.com (Postfix) with ESMTP id 2159B1C146D for ; Fri, 5 Apr 2024 10:27:33 +0000 (UTC) X-FDA: 81975101586.27.7071502 Received: from mail-pf1-f179.google.com (mail-pf1-f179.google.com [209.85.210.179]) by imf26.hostedemail.com (Postfix) with ESMTP id 42B7C140015 for ; Fri, 5 Apr 2024 10:27:31 +0000 (UTC) Authentication-Results: imf26.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b=C0hwgWCP; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (imf26.hostedemail.com: domain of 21cnbao@gmail.com designates 209.85.210.179 as permitted sender) smtp.mailfrom=21cnbao@gmail.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1712312851; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=4ZWljBCC9KrF5CNn3q7EM6c1BT8pMAntpGQOmXRtUQU=; b=Xzy1GhElZJ1DMQt490xX8CDuoCSHL/fy8vObT7E80POW0S3jEoJPiJrAqtqxjzbAAWX5qi 0J5LEehhbMHKQLIUvTQlbZVJiCrJrYBTZWMy3iCOwL+19ehCwuwvgc1Lt61oA6wqb/6KOd h3EXg6QpHm01xYzZadbU/1aPaKl/qDE= ARC-Authentication-Results: i=1; imf26.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b=C0hwgWCP; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (imf26.hostedemail.com: domain of 21cnbao@gmail.com designates 209.85.210.179 as permitted sender) smtp.mailfrom=21cnbao@gmail.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1712312851; a=rsa-sha256; cv=none; b=RkPu6Mb6pLmQ+U5N7MdaUJQcYFWfPiTxF3jbpbULBV6sxJ/CamL6Ml3MP+3U7wTssikTLS ZWk6R6U0F9KaOWVKH9dsxvRsPgT9kd/ASxz8/zVD3wt3gl7n5Pqm94ZleZSqkZbEdMsuEC 7BjXArdbadskR+fQIe3THsI/Obejn9Q= Received: by mail-pf1-f179.google.com with SMTP id d2e1a72fcca58-6ecfeefe94cso358134b3a.0 for ; Fri, 05 Apr 2024 03:27:31 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1712312850; x=1712917650; darn=kvack.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=4ZWljBCC9KrF5CNn3q7EM6c1BT8pMAntpGQOmXRtUQU=; b=C0hwgWCP85xS4sSIE8DFEk/8gmpsfCTrcTNAobefoRtPLgNyRxZgRLiZpS3r7i0gMx UEAsVBApNNMkKk8sbaci38mOB6ytqfCkK9s0Lmt518muiNN7JikBlm1YU8t2h0Vk9bjV pXrnwd7xNqFNqFHVTrxJtg05eL+YCQOltvgv2TlhY+pMQMZKLl9lVcBn6/z4y4oAepFS rq3Fx7HjU9pjLgPSneOk/suO2UxJdVKey67UyIsEZWwzkIDHpNHck4ThlIS4F853VB3p pu/ZB/6WeS3I51ObByBV14FktXVcjlBbgBLyVvnLcuzMWC8uf6KGrKCvuBVZ/WUVwucq 5erw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1712312850; x=1712917650; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=4ZWljBCC9KrF5CNn3q7EM6c1BT8pMAntpGQOmXRtUQU=; b=LZFnfkIAkEYlf7YpWUVSitKVXAqbkQ3eZXeBeevNAYG4//cG5go+QSwGm68Im6uD3O k6VyJ6Dmm2sYPoWc6aZno/aqf3Jn/MVOAAfRga3G8L1wJd8qqj0YvqMqQGQJMfKiOxUd eyERD5gr4BHvtTr6AYkut6pkMFoLWBFf12sr9ocRDHr1a304olR980v2e4+ciamW/MCS dxr4QUGp48mO8hl8OEnRrbW7DTbCxrSnInXsamrRxJnrfTC3Q7VhyTn+hRGbnv1eA4Bt UEz8pO/qksgd7QB1dQKlZyHYlcsjdVJJUbF/KfCfa8g3/N5sSJjANjV2Hgmk7CP9Uwvl tOQA== X-Forwarded-Encrypted: i=1; AJvYcCXs2mptotpuqe3Q3eNi9ue7TOBaDvUaWir2Pu60iHwcriMsp4N5DjRuKshYzTXgIuJS+KJAEIEL8OfaH1m5Et61RfQ= X-Gm-Message-State: AOJu0YzwjQ/ys5AHZfqkqjshiY8Pkw8miiR74HI5b+m+HKK49dxHvCBG B5WcdyP4XhQskUCoq5i/6pwbxHiO55eOwNRAredp11FNavacOXMf X-Google-Smtp-Source: AGHT+IGtMKiEbIdv0FWwPdoZQEr/By0dCwSDG7UHQlGksBrCqvl3FEssVeROn/+hGtNQ6NcBdk65wA== X-Received: by 2002:a05:6a21:a5a9:b0:1a7:2451:aeac with SMTP id gd41-20020a056a21a5a900b001a72451aeacmr1784226pzc.16.1712312850194; Fri, 05 Apr 2024 03:27:30 -0700 (PDT) Received: from localhost.localdomain ([2407:7000:8942:5500:aaa1:59ff:fe57:eb97]) by smtp.gmail.com with ESMTPSA id f20-20020a056a00229400b006ed06aa3604sm113334pfe.82.2024.04.05.03.27.24 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 05 Apr 2024 03:27:29 -0700 (PDT) From: Barry Song <21cnbao@gmail.com> To: david@redhat.com, akpm@linux-foundation.org, linux-mm@kvack.org, ryan.roberts@arm.com Cc: cerasuolodomenico@gmail.com, chrisl@kernel.org, kasong@tencent.com, peterx@redhat.com, surenb@google.com, v-songbaohua@oppo.com, willy@infradead.org, yosryahmed@google.com, yuzhao@google.com, linux-kernel@vger.kernel.org Subject: [PATCH v4 1/2] mm: add per-order mTHP anon_alloc and anon_alloc_fallback counters Date: Fri, 5 Apr 2024 23:27:03 +1300 Message-Id: <20240405102704.77559-2-21cnbao@gmail.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20240405102704.77559-1-21cnbao@gmail.com> References: <20240405102704.77559-1-21cnbao@gmail.com> MIME-Version: 1.0 X-Rspamd-Queue-Id: 42B7C140015 X-Rspam-User: X-Rspamd-Server: rspam02 X-Stat-Signature: zwgaiiinpcpqk67fa4h8oo85etj85dmx X-HE-Tag: 1712312851-129602 X-HE-Meta: U2FsdGVkX1/0ePHxkyJ/FfTB/SrhZccj0xr41LvQt5D2YvvkavH1giHFGuP3lU2TcLRIFneAdivzTi5uKHj1VXY++cBqmhGvIfjWcPozk9Ir+ynEHcTAZIsmxTq4pVMflMg0PEjq+8ZtroC+ronZrHMuON0Tz191vPFDNfnSYB4sc/iKlxJdqsN0p4hcjP4SlXZYwAaIQ/lYrrREdXDbglAq6XzKzOTUXlMeLjSKfqr8F3Hmub2RTq86zMR/w6bYdRDDvdk/KuARievu2CW8jDGp4UKSbQ733Wtv7vsJCBsYEG6MH39tNjESycBGNp80Y/wvJBQhea4uV78Hx4JsyH9K9quS0xJhOhVmQ+41vnLsJ4h4+fI2uuAs+rqdKSSrL7jeGwE+y7qTv+aJKKgW2QkuVCerWrZvb7YvM2SPg9XTjGscizG24LrEKN7xMkE94TAzkDh1ESJGwPdKDNzwfvz661XkOeQSy3Bs1nY1AXklX9Mzcj5ZOMFwTe0PNiNytPfVwVRPt535nF7H1GizeiCo+mphAsH0ONiEWfzeRFI03C8U4jYRTmgYohNOd0h2TKa8CS2YyFIojG9JQ/fB6vLfqseRSFqOEfW8oWgByUf9tF7xnZQjed1uisOFQVPCqOWbWbivCd7d0JaA+j+uKFMgVCInTwPzn0T1sfQbtrCg6gXRZyCcA/YkFsGtDKsyoQhvxY75JKDG2BisHBHJtqtrqHsSDQhl0bvcba2zKM3nMbGfB4JhTJGZD0N4sxPaOp5S8mZprNR7xOTC6oJ8xLEckDpKFl3/XKtU8Ktzwoie8Imco+rYCJkI7odCQe0hY5AWpvhjOkOyOxgEvrlY0n/QHDzjv7VrGEfyvTZ7DmVWcrtIjLEVJnB6MhzGTEJ66T0MirJZW5m+FQ4JYcaUNZJWh6DmSRHm8DeRurAepTCUcudac9jTtD6nGaDePonAndl0NesnP1gs7RooRFd 863bnF4H 5yAdWt/DNbwYgMOBr58ENAhoCoLy7d7yJdkpmwXpxd9PmzfLW2vbcjtTUAl7zakoraSy4fi3VfmP/NRtIBDrqs6PiipCDIvWZTR7OhPATjTOKouv5s/vl6TpPPWFbFc5VfwQXkTNB8asE4dddG009vfYjraP9L6/MTte8J+jbPhp03gHf65ADnaBRbo0kGxZtiiKm2qELieZnobQ+8RhGzIKwYj8nqxpiweerL+v6RN96Vju0ZWRWE3pEMAJmXYpvw7E3RMGR5N1C2Y4ReOwDuuqTpgnmkA6LJXLFo3kcizEEPwJ705c1ZyRnr9RUQ+82HsLm8CfGIvDoMD5ca3Ed7f1xd2R6Yxs2c3+LKqyHhXYv5r6H3vEI2rDdO9EQydJr5DLrvRLwC23iGAebLu0ADpaYkwzV5c4pjwqX6Mz/kyyRMpG317HGRf4jfv5mwQjnFo6g0z3TKwxRxJHMDndZ0/P3yc+bMQjxWtFePs2d9dK5REbDBmaRzJU91feXalhDSiBOUz7lgToFovcKoMXXq7AcDqrDAi4O6OPZWNVtKRKsFmdYJ8bn5rWbAgjrZrHLhqaz X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: From: Barry Song Profiling a system blindly with mTHP has become challenging due to the lack of visibility into its operations. Presenting the success rate of mTHP allocations appears to be pressing need. Recently, I've been experiencing significant difficulty debugging performance improvements and regressions without these figures. It's crucial for us to understand the true effectiveness of mTHP in real-world scenarios, especially in systems with fragmented memory. This patch sets up the framework for per-order mTHP counters, starting with the introduction of anon_alloc and anon_alloc_fallback counters. Incorporating additional counters should now be straightforward as well. Signed-off-by: Barry Song --- include/linux/huge_mm.h | 19 ++++++++++++++++ mm/huge_memory.c | 48 +++++++++++++++++++++++++++++++++++++++++ mm/memory.c | 2 ++ 3 files changed, 69 insertions(+) diff --git a/include/linux/huge_mm.h b/include/linux/huge_mm.h index e896ca4760f6..c5d33017a4dd 100644 --- a/include/linux/huge_mm.h +++ b/include/linux/huge_mm.h @@ -264,6 +264,25 @@ unsigned long thp_vma_allowable_orders(struct vm_area_struct *vma, enforce_sysfs, orders); } +enum mthp_stat_item { + MTHP_STAT_ANON_ALLOC, + MTHP_STAT_ANON_ALLOC_FALLBACK, + __MTHP_STAT_COUNT +}; + +struct mthp_stat { + unsigned long stats[PMD_ORDER + 1][__MTHP_STAT_COUNT]; +}; + +DECLARE_PER_CPU(struct mthp_stat, mthp_stats); + +static inline void count_mthp_stat(int order, enum mthp_stat_item item) +{ + if (unlikely(order > PMD_ORDER)) + return; + this_cpu_inc(mthp_stats.stats[order][item]); +} + #define transparent_hugepage_use_zero_page() \ (transparent_hugepage_flags & \ (1<stats[order][item]; + } + + return sum; +} + +#define DEFINE_MTHP_STAT_ATTR(_name, _index) \ +static ssize_t _name##_show(struct kobject *kobj, \ + struct kobj_attribute *attr, char *buf) \ +{ \ + int order = to_thpsize(kobj)->order; \ + \ + return sysfs_emit(buf, "%lu\n", sum_mthp_stat(order, _index)); \ +} \ +static struct kobj_attribute _name##_attr = __ATTR_RO(_name) + +DEFINE_MTHP_STAT_ATTR(anon_alloc, MTHP_STAT_ANON_ALLOC); +DEFINE_MTHP_STAT_ATTR(anon_alloc_fallback, MTHP_STAT_ANON_ALLOC_FALLBACK); + +static struct attribute *stats_attrs[] = { + &anon_alloc_attr.attr, + &anon_alloc_fallback_attr.attr, + NULL, +}; + +static struct attribute_group stats_attr_group = { + .name = "stats", + .attrs = stats_attrs, +}; + static struct thpsize *thpsize_create(int order, struct kobject *parent) { unsigned long size = (PAGE_SIZE << order) / SZ_1K; @@ -549,6 +589,12 @@ static struct thpsize *thpsize_create(int order, struct kobject *parent) return ERR_PTR(ret); } + ret = sysfs_create_group(&thpsize->kobj, &stats_attr_group); + if (ret) { + kobject_put(&thpsize->kobj); + return ERR_PTR(ret); + } + thpsize->order = order; return thpsize; } @@ -1050,8 +1096,10 @@ vm_fault_t do_huge_pmd_anonymous_page(struct vm_fault *vmf) folio = vma_alloc_folio(gfp, HPAGE_PMD_ORDER, vma, haddr, true); if (unlikely(!folio)) { count_vm_event(THP_FAULT_FALLBACK); + count_mthp_stat(HPAGE_PMD_ORDER, MTHP_STAT_ANON_ALLOC_FALLBACK); return VM_FAULT_FALLBACK; } + count_mthp_stat(HPAGE_PMD_ORDER, MTHP_STAT_ANON_ALLOC); return __do_huge_pmd_anonymous_page(vmf, &folio->page, gfp); } diff --git a/mm/memory.c b/mm/memory.c index 649e3ed94487..1723c8ddf9cb 100644 --- a/mm/memory.c +++ b/mm/memory.c @@ -4374,8 +4374,10 @@ static struct folio *alloc_anon_folio(struct vm_fault *vmf) } folio_throttle_swaprate(folio, gfp); clear_huge_page(&folio->page, vmf->address, 1 << order); + count_mthp_stat(order, MTHP_STAT_ANON_ALLOC); return folio; } + count_mthp_stat(order, MTHP_STAT_ANON_ALLOC_FALLBACK); next: order = next_order(&orders, order); } From patchwork Fri Apr 5 10:27:04 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Barry Song <21cnbao@gmail.com> X-Patchwork-Id: 13618836 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id DB348CD1296 for ; Fri, 5 Apr 2024 10:27:39 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 6E7E36B014C; Fri, 5 Apr 2024 06:27:39 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 698E26B014D; Fri, 5 Apr 2024 06:27:39 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 55FB26B014E; Fri, 5 Apr 2024 06:27:39 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0012.hostedemail.com [216.40.44.12]) by kanga.kvack.org (Postfix) with ESMTP id 36AF86B014C for ; Fri, 5 Apr 2024 06:27:39 -0400 (EDT) Received: from smtpin09.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay03.hostedemail.com (Postfix) with ESMTP id ED377A10A7 for ; Fri, 5 Apr 2024 10:27:38 +0000 (UTC) X-FDA: 81975101796.09.69A26EB Received: from mail-pf1-f172.google.com (mail-pf1-f172.google.com [209.85.210.172]) by imf24.hostedemail.com (Postfix) with ESMTP id 0D58218001B for ; Fri, 5 Apr 2024 10:27:36 +0000 (UTC) Authentication-Results: imf24.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b=BH3UqqhY; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (imf24.hostedemail.com: domain of 21cnbao@gmail.com designates 209.85.210.172 as permitted sender) smtp.mailfrom=21cnbao@gmail.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1712312857; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=ykuY6i2Fu38nsDbXE/VtCFZLlgGKZpXgb6cmPoEsG8o=; b=LqMFoGj/p4cKlYj7TXf0VHvIqGXNjwGeDEMKmmSnv3AzKr0D92Cl0151IOX4d9bqEWBK7E EkcaaGNA3S+28R82Serbn85tQ2y3Zg4ZqWVLtj16YqaZRx6jPwFMpmlXXuqj+KQskveAHo wfdScp6Uv87RiXlHKq2anPF+PBPKppc= ARC-Authentication-Results: i=1; imf24.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b=BH3UqqhY; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (imf24.hostedemail.com: domain of 21cnbao@gmail.com designates 209.85.210.172 as permitted sender) smtp.mailfrom=21cnbao@gmail.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1712312857; a=rsa-sha256; cv=none; b=Mv/VNi4zYqkSGjOL78hoL4JY8WGpUXMBbrJslNp37VutKkhaXOSVyYVpFX+iG4ESFnHaqp ye5C8TXHZzY/TV4EJD3ROd9M2A58CaYsZkH3bdF2cQnOxwhpcYj5vldwnw23EAEm+GFSzS sQONqnNtVNeVGbN9v8nk5v9eR2DqGM0= Received: by mail-pf1-f172.google.com with SMTP id d2e1a72fcca58-6ecfeefe94cso358195b3a.0 for ; Fri, 05 Apr 2024 03:27:36 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1712312856; x=1712917656; darn=kvack.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=ykuY6i2Fu38nsDbXE/VtCFZLlgGKZpXgb6cmPoEsG8o=; b=BH3UqqhYdtoYAgVBtjCfPvqob04cNmKsxYAlAxyyusdFrVF+3yIY5raZpLJILda6Cs I9zD3dtHtxgsIa9Ooa0TJP4x0KzbdR2Z46eVixzLguLKgZJaKA8LNG1ZvQR5RjY+Ahb2 G1incT0wkQlaZvsDqs/zBwxPUFeVjORGSNv3owreoOp39uL0YI6Q4d9nQdCj2YY7nZFA MnCYXD69mlAiH+wSxscj1wp2uxhC2/LRz+BxhhDo2ycSq6W0cem1fDDCAaHUU/hdFHU+ zEOttCMpWciKl8z/iLlVYUtDFgYmy5xdMI0B0dBCCC9JWwCKeVwtm42+WfmAptNSsN/v 4HBw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1712312856; x=1712917656; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=ykuY6i2Fu38nsDbXE/VtCFZLlgGKZpXgb6cmPoEsG8o=; b=gURoUcBdsPvNllBka60gt850Ekr1hzEWBsFVajIS+j80FJEj1fI2EghV2f11ZCyt1E D66ERlsG6Ui3Ebn6a6raLZv59x+FKvAF1DjSy61QvyycD5Px8IhrTUqwjIAW9o8fqtNe xnWCbs25gnSR47G4CuaaQ18oXSaS+ZqP0vCE0/vwYDPH+bxnofGEvrD7RYcMeMff0a7A NIAvOS863HBD2D1xmoQPCy4fzAk0//nTy1jVAGMFZFl5HbzZ/QeeEeu6SnWv6HeJiyVd NXJRExW05WXaFHX6ZtC6mnjhk+bFqyEHXpU0rW+XTlZ+DBlPB8isWic1g+lPU7TRSxmK KkIw== X-Forwarded-Encrypted: i=1; AJvYcCUWxKHM2xWo5Ii2B/VMXYvUV+owNnDqjGjT765cFRtKgQx9zZ/jWuHhBFretj9My6mEltdlwEMFedfRiH+lxJXYGls= X-Gm-Message-State: AOJu0YwjLns9m6V5rbfvYyKG9lTTEq0z9w3KyMAxMBzBKJ0pzjCeRkNF IE8aawIuuTrhuWckIx5g4OLkpLpeFayjdisDCMbHPX8x7oCuL+5K X-Google-Smtp-Source: AGHT+IH7lyqbZgiy6gFtYdrA72hCZMiuUcdajSkaC+qbDc4j+Oz8Sw3oeAQrQ+KFY+9839aPVaFHvA== X-Received: by 2002:a05:6a00:2341:b0:6ea:9b37:c288 with SMTP id j1-20020a056a00234100b006ea9b37c288mr1454381pfj.15.1712312855698; Fri, 05 Apr 2024 03:27:35 -0700 (PDT) Received: from localhost.localdomain ([2407:7000:8942:5500:aaa1:59ff:fe57:eb97]) by smtp.gmail.com with ESMTPSA id f20-20020a056a00229400b006ed06aa3604sm113334pfe.82.2024.04.05.03.27.30 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 05 Apr 2024 03:27:35 -0700 (PDT) From: Barry Song <21cnbao@gmail.com> To: david@redhat.com, akpm@linux-foundation.org, linux-mm@kvack.org, ryan.roberts@arm.com Cc: cerasuolodomenico@gmail.com, chrisl@kernel.org, kasong@tencent.com, peterx@redhat.com, surenb@google.com, v-songbaohua@oppo.com, willy@infradead.org, yosryahmed@google.com, yuzhao@google.com, linux-kernel@vger.kernel.org Subject: [PATCH v4 2/2] mm: add per-order mTHP anon_swpout and anon_swpout_fallback counters Date: Fri, 5 Apr 2024 23:27:04 +1300 Message-Id: <20240405102704.77559-3-21cnbao@gmail.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20240405102704.77559-1-21cnbao@gmail.com> References: <20240405102704.77559-1-21cnbao@gmail.com> MIME-Version: 1.0 X-Rspam-User: X-Rspamd-Server: rspam12 X-Rspamd-Queue-Id: 0D58218001B X-Stat-Signature: f3wpn89wqh1yu9i4jekbcfxocy5jkm9h X-HE-Tag: 1712312856-385349 X-HE-Meta: U2FsdGVkX18w0UW4OyV16+alsBCs/bLvfIMG8Buu7yWSZFjk9O8FAAsnLlRvkPJJhcP1ETvaQutFpiEmxg2KttleD8J7M2TN5VTWfBfeXnegrbNWgnWDXVejbeMmyZ+u8I2NJF8Fg4YAl8zfEU//UzwvcS6HOHjl6X96aClzlW6jHVDGz2N+kAUCSIc31LF0dKqonpF2pzYclpCCZXki0l0u0PYNY6qPgTXpDylAquuYZa2FnzC8NUzcquEn1gZk4Qt6IPGsI1svG1eF6CIhONw3/cBTmlFwGbbRY6c3mHzwy8FjESFAihZA9njB7qjT3oj/ypgcLGluW11B81JHbTtqlo5KszgrvDZVX8A+DghamBIgnjd96cSbizGpCHHIy3Sia0LXKlFhC+uq8ta71X2ZGVY80mFbyfEP8GejuIrX2jKkzhs6X+lyre0UJc/qthnXbNWLLUEHRgGfj21+lCXQTUFtTgdju4MFIBHecHMg+yCYioQoiE2QtwjrQ7AopTDdKPR80ryEiYkq4TUdoLbwM5zVjVVnnE1jK8cWlbAowtptKXq6zjaH//2G5WfmbKioFS+De0vEGM4LUuwyiUUNM6ht9tURlDlQHe+rtjyAZCAptAtMGLYlmc2lLDS6c5nmhTTarOWB/YuT1dQO9LKyCmt6dPogrv3d4R7i3EhkZ55KiadbcCCXo98mwYGkQzNRZqOgySmX3MKglBSkO53pPaIv00jg+w/Zl8ueHUuczg7PgMuYdvhvuL2pSt3xxI0PGL5GN8Tw/gzIP/kzD0sHjnZHQomAtM9nfLs2gb2PGHeqtjC4kVcF4CTOA3bJe+Uwwop+ZSuyfw5M+4hNW/U8zYnnXaO9dgrUEfdwIwKd09DlYnecnpURNocpGHg4PJ1iIkqhSUvOKrCMfUNEQpar6/dmOZQv94YDkGz1pBoylAuKiQF3oc0H9TTAWeDymubjsI4FPi7VcC+T13c ZeYW9uZQ grCnbVpBh3p7UrNBQmZTUzhqdJnXu1Myu5jHF0+IZrz0FeSm3rH90WpacDr2tVzSHhtd7en1JtCgH5BFs6MPKHlfFDPaHqKV9pXQtpvlqn4yUZBJFn35KuRFEGK+DBE/xS8lXxS9PDvwZMSfp8saGiqug7NCParbs5GWP0icAxbI0pIoaj0jPj0F5/o1PmpfyDnn9LWUdkT1qJmirg6CWmp+YuZcU091uktcD2eIKcuB8E4rxoR99GsEU6c6e8y7hc0w3dAOyadKmw6DxdrQDMVcUr1yM7w/YP1wXv/W8Ag1kLQkaobo36S3VwhvCWfGdYl9Pq1DZ1z69nIm5CcgH3gVgFP88ttO8yf43vDqZLUbrfpijLXTiYHmWKdXvCN1Y2cM0zjKfuuQX6fPSy0W+NIVw061CnzKdfjM9vxOKQqDcEWFmr4lf6zUrUJKHpXggWuaMPTkj3cQKEuWujQZW3aROFccZGX9s9cs4/CgFaxP6nRkacaHcuE2GMESJHwfn2Xb43cBePKS16zxFvEgZe37i6wAwqWctSNIyR/LY/Fr2xu6ArdxifXRANjo9ZtpxpKbE X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: From: Barry Song This helps to display the fragmentation situation of the swapfile, knowing the proportion of how much we haven't split large folios. So far, we only support non-split swapout for anon memory, with the possibility of expanding to shmem in the future. So, we add the "anon" prefix to the counter names. Signed-off-by: Barry Song --- include/linux/huge_mm.h | 2 ++ mm/huge_memory.c | 4 ++++ mm/page_io.c | 6 +++++- mm/vmscan.c | 3 +++ 4 files changed, 14 insertions(+), 1 deletion(-) diff --git a/include/linux/huge_mm.h b/include/linux/huge_mm.h index c5d33017a4dd..1d893a358df6 100644 --- a/include/linux/huge_mm.h +++ b/include/linux/huge_mm.h @@ -267,6 +267,8 @@ unsigned long thp_vma_allowable_orders(struct vm_area_struct *vma, enum mthp_stat_item { MTHP_STAT_ANON_ALLOC, MTHP_STAT_ANON_ALLOC_FALLBACK, + MTHP_STAT_ANON_SWPOUT, + MTHP_STAT_ANON_SWPOUT_FALLBACK, __MTHP_STAT_COUNT }; diff --git a/mm/huge_memory.c b/mm/huge_memory.c index 5b875f0fc923..28113f8fdf18 100644 --- a/mm/huge_memory.c +++ b/mm/huge_memory.c @@ -554,10 +554,14 @@ static struct kobj_attribute _name##_attr = __ATTR_RO(_name) DEFINE_MTHP_STAT_ATTR(anon_alloc, MTHP_STAT_ANON_ALLOC); DEFINE_MTHP_STAT_ATTR(anon_alloc_fallback, MTHP_STAT_ANON_ALLOC_FALLBACK); +DEFINE_MTHP_STAT_ATTR(anon_swpout, MTHP_STAT_ANON_SWPOUT); +DEFINE_MTHP_STAT_ATTR(anon_swpout_fallback, MTHP_STAT_ANON_SWPOUT_FALLBACK); static struct attribute *stats_attrs[] = { &anon_alloc_attr.attr, &anon_alloc_fallback_attr.attr, + &anon_swpout_attr.attr, + &anon_swpout_fallback_attr.attr, NULL, }; diff --git a/mm/page_io.c b/mm/page_io.c index a9a7c236aecc..7669452e8b4d 100644 --- a/mm/page_io.c +++ b/mm/page_io.c @@ -212,13 +212,17 @@ int swap_writepage(struct page *page, struct writeback_control *wbc) static inline void count_swpout_vm_event(struct folio *folio) { + long nr_pages = folio_nr_pages(folio); + #ifdef CONFIG_TRANSPARENT_HUGEPAGE if (unlikely(folio_test_pmd_mappable(folio))) { count_memcg_folio_events(folio, THP_SWPOUT, 1); count_vm_event(THP_SWPOUT); } + if (nr_pages > 0) + count_mthp_stat(folio_order(folio), MTHP_STAT_ANON_SWPOUT); #endif - count_vm_events(PSWPOUT, folio_nr_pages(folio)); + count_vm_events(PSWPOUT, nr_pages); } #if defined(CONFIG_MEMCG) && defined(CONFIG_BLK_CGROUP) diff --git a/mm/vmscan.c b/mm/vmscan.c index ffc4553c8615..b30e6294f82a 100644 --- a/mm/vmscan.c +++ b/mm/vmscan.c @@ -1247,6 +1247,9 @@ static unsigned int shrink_folio_list(struct list_head *folio_list, count_vm_event( THP_SWPOUT_FALLBACK); } + if (nr_pages > 0) + count_mthp_stat(get_order(nr_pages * PAGE_SIZE), + MTHP_STAT_ANON_SWPOUT_FALLBACK); #endif if (!add_to_swap(folio)) goto activate_locked_split;