From patchwork Wed Oct 23 17:50:37 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dongjoo Seo X-Patchwork-Id: 13847761 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 53A1CCFA45C for ; Wed, 23 Oct 2024 17:50:45 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id D98706B0083; Wed, 23 Oct 2024 13:50:44 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id D6ED66B0085; Wed, 23 Oct 2024 13:50:44 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id C0F2B6B0088; Wed, 23 Oct 2024 13:50:44 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0014.hostedemail.com [216.40.44.14]) by kanga.kvack.org (Postfix) with ESMTP id A07F86B0083 for ; Wed, 23 Oct 2024 13:50:44 -0400 (EDT) Received: from smtpin30.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay08.hostedemail.com (Postfix) with ESMTP id D9B6B140C8E for ; Wed, 23 Oct 2024 17:50:25 +0000 (UTC) X-FDA: 82705606704.30.AEC31AC Received: from mail-pg1-f178.google.com (mail-pg1-f178.google.com [209.85.215.178]) by imf25.hostedemail.com (Postfix) with ESMTP id 2E9F2A000A for ; Wed, 23 Oct 2024 17:50:29 +0000 (UTC) Authentication-Results: imf25.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b=J7qp+tXM; spf=pass (imf25.hostedemail.com: domain of dongjoo.linux.dev@gmail.com designates 209.85.215.178 as permitted sender) smtp.mailfrom=dongjoo.linux.dev@gmail.com; dmarc=pass (policy=none) header.from=gmail.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1729705765; a=rsa-sha256; cv=none; b=SaTXq+KTVmnQoREhSWLB9tNUtis7d3Wapr09yKV3QT5zBcbIvreVN0T/XO2nDuqZOC4Lhw +M1IkLhEgxg487UXHzJjFcCWji4uFbAkSB+VjC2y8IHFy7KH1P8LrruJCiYVHYdfWc8bDl p5nRYhJhq10F1JJh3Tt+XvNTAK6zayE= ARC-Authentication-Results: i=1; imf25.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b=J7qp+tXM; spf=pass (imf25.hostedemail.com: domain of dongjoo.linux.dev@gmail.com designates 209.85.215.178 as permitted sender) smtp.mailfrom=dongjoo.linux.dev@gmail.com; dmarc=pass (policy=none) header.from=gmail.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1729705765; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:references:dkim-signature; bh=Tckut9jQa7rI5qyZ9d3+QecKLLm/EO/R4ZdBFp+YfyY=; b=jD6XPWU/OC3AFutYuWGG3Ybyrx7l8Y4FcfhXag0DwAFJkVKdX9qYiOPo2ufVKMpCny7jA9 AexLe5A8Z4kbdFMoJ6PSuZ07FDoFaFXyySeyc/dDDr9OQ9iUUJBt8vlPOj1A2i8SZs/7sH K2liw2JzpMPU4y/PhlyOL7kIcaj5RrQ= Received: by mail-pg1-f178.google.com with SMTP id 41be03b00d2f7-7ea12e0dc7aso26100a12.3 for ; Wed, 23 Oct 2024 10:50:42 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1729705841; x=1730310641; darn=kvack.org; h=content-transfer-encoding:mime-version:message-id:date:subject:cc :to:from:from:to:cc:subject:date:message-id:reply-to; bh=Tckut9jQa7rI5qyZ9d3+QecKLLm/EO/R4ZdBFp+YfyY=; b=J7qp+tXMcou0P/tnl6fS2ZYOGONtYvjtB7+v25pJ9X36jrvx9cuKYmSz4+/cfQzygp 9n5lz6KLvKaaK63bPHBHfrXZ3mJFke9/PYy+Jg8U/JX6f+okxa15POcwDeT63wwSffYG lYI8b61uSnNQQC0H4ubV3KT0Uubn8if0/mwaA+/3X2m+Ni3gJl9Z/46d59RybtwkeGin Epx8f7qlz7VvMN44uK/ThY8BKyfWmqc+M+yT78/vvsGH5x1UR/qiXkf+/N2DI7k1hneB kFxHkFXcgr16y+UHnqAT2e+zvj2zIis4GjZdIoL4PgTZ1R9xLwTnadcjJeChklIuoB3L r9Lg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1729705841; x=1730310641; h=content-transfer-encoding:mime-version:message-id:date:subject:cc :to:from:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=Tckut9jQa7rI5qyZ9d3+QecKLLm/EO/R4ZdBFp+YfyY=; b=vC6Kv6YWT2E81zeVo3Nal+XfZCHmly23gw/92MvUxPw57YdtOI3qPv9sS4GnfI6Aqh a/SoxljjWO/FK9Eley57bo9w07g01+DYGropPoNzPfgKAVOlZZC9DxKbklkQ6NSwK1Cs FiWvsUM7WfWB5RqDqRf2/G/gEtsmbItkou7lQLUY3t0cozTRvxU330cF2KMBvwzEJnpY iXuXtIPHmSRvAcqHodverAl929WQCA9eTahaagXYR6zy4KZY+w890WYOiOPa8ZG65Fvx ny5OcKmWcQXi7t2cu2ZvU7PooL6of3oixGRWKg91kPHeS0plaInyyAt21EOiggRQEyON tzeQ== X-Forwarded-Encrypted: i=1; AJvYcCUTCpas+dtrj67KOLR3Y5FRD+OkLvDrLdkWIvXA7Zm3m8MbWn8fuQmK6PUJ3MfmPJM2x/0XusPG2Q==@kvack.org X-Gm-Message-State: AOJu0YxLjOLINPMAxdJqp/NM9arEpAqB5HObiiEUouzGZ7Srv37tczGk H/nPm8Km1qEDXoSEjkgPCqZ54lvHe0fRhIzqmqDvNgi82EXC+dYE X-Google-Smtp-Source: AGHT+IHbC9BxbdNRtNpmuEEUlHVpNB2/puTTvWOYiXtmO2xMU+xZAuBvui1C9Ac0Lt++cOuZApV5Mg== X-Received: by 2002:a05:6a20:7b1e:b0:1d9:83cc:fd48 with SMTP id adf61e73a8af0-1d983ccfe02mr542463637.17.1729705841218; Wed, 23 Oct 2024 10:50:41 -0700 (PDT) Received: from eqbm-smc020.dtc.local ([149.97.161.244]) by smtp.gmail.com with ESMTPSA id d2e1a72fcca58-71ec13eb02bsm6817795b3a.178.2024.10.23.10.50.40 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 23 Oct 2024 10:50:40 -0700 (PDT) From: Dongjoo Seo To: akpm@linux-foundation.org, linux-mm@kvack.org, linux-kernel@vger.kernel.org Cc: dave@stgolabs.net, dan.j.williams@intel.com, nifan@outlook.com, a.manzanares@samsung.com, Dongjoo Seo Subject: [PATCH] mm/page_alloc: fix NUMA stats update for cpu-less nodes Date: Wed, 23 Oct 2024 10:50:37 -0700 Message-Id: <20241023175037.9125-1-dongjoo.linux.dev@gmail.com> X-Mailer: git-send-email 2.39.2 MIME-Version: 1.0 X-Stat-Signature: 8anu5itjwns3wce7nzddhb5i3mcty8xt X-Rspamd-Queue-Id: 2E9F2A000A X-Rspam-User: X-Rspamd-Server: rspam10 X-HE-Tag: 1729705829-107472 X-HE-Meta: U2FsdGVkX185eSz7ndGe2V6Jf3D4AE7LKj0ora6UK5yT/l57YFxoswpArnKV+BcS2tdKiOzCfLCEaMRVZcCzawR7cWM8CnL+1XpEtkT8ZwNKODhiYjMWI4ndWYFTqpYIl8mdIrl1yq8+/fkec/hDAugbcp7ygwi3vCaXdCHHV5K5uT1yh5MXd2yJBXs6icHnjP1zqMwdV65mbo5MjCj2/mqdhnAIdtlbiP6DOLrZeSfW9JnqIFqvtOqMHC0G01uaB47cnvEPFPCtTp/JullhDDPNXWuA6JCITKZZTpCrhE0OxXoP0TgKJ4Hn/bdGdHZVKXm0RotZ0IP+rNZIAw+AxHCFOIBwohxoTsAO+UXqaYcGUPnaKxrffOh5FVbhpjjU7bJFaIoHgHY9ebatmXWfT/1GB/Lp17h3PTsJHoxGO6GbgJ1ldbrCPVppimkYcHGYM9sjsaTe+Wb9+WVRxTnB7m7a/q2XIgvAuusRFqF1pjpL7Cy8KxkYutnO5E4QLKRitq2IPJ63D74Q5K3Fi0UCGaGvd+L6MmMQf4suHS+Aftl8N1SlOtLhoVyP8LuSEp9vCMUrmVpf8pw8BBysLeSV9bSieF3lafh2hxh2E04KCnX7BIXXEl1anyL549ncml2NRJKhrh2PgdboFTUDauJdJudQ/sRBzN9lDo8KwOFy2g5yHLDDDBlFZKh+XVPCazLsO6AJbq/vvFobOQegp6FQYf8vOBOMGnsYhHaVo+iYgBARK74PhaO8iJwGlLvvuKw24yr7EnDgh+QxYLnF80vXxjWA3ncJy38o4pdCZ1/H5BB667vHtTif9HtTw6uk6R0BUUmA/7B0B4dtqj9O5ck0es4Jom7jGmhVsO9XrZNUstSmVh7xgU034VqUWxSRGB4lYr9BpYiYje//Wj/4edY5lhLD8doVDO8S5bTmmCNElSKGKz9k2+/K8o4eiZ+NcUGwewfqFaTW+2/e7jZxjaO 9VY5eQWY IYWPVWHip0eGQVg3vKsy+hLY0H1p8w0Q8rv9RrSmXWfk9IjLbUzZ8Ew2PChRjIz+xYl+kD7l2uEvkivoDz5Pq2GMMpKQYRKu8ICYLdZqSA2GLye3ZlcEuwNvIZCYN0biqf4c1zq8mxXfwLJWhRIW94iRWt+/k2Bf4r020HyCJ8rqs6t8gC+Axa+BPsbD/Rl/tICSWqu8Xjb2PRVbPsyY3Hn6HD4nselZJvctkO3wY8dBqpk5E3fng0GqHtO22fbqbdzBryNDNmM8vWRJxMPlr/g/p+m2y8x9mKFwSOT8Yl0yWxeZPstA61Qj6sO4qxvOaapworiXvQsrL3cAqw0xqFx8jvwJkx/aAW6n6BtrgumdEROXJXHelZyKNDB/9F3CJxlNC69lfmimHb/+0Ysc1m+c5E9573jmYRm47uyy49YUmQYCGt/WJSfekxBqMwEcaNvCqOEz8F3Ovc4L8s06sppM06MvVwtRLt6bQrzdHhgBYtZLckGxQ3tCSKYtg3EShIXhwsSJni7IZ3Jna3DVqp9aFk7A0LPilKuSk6l0/ATMu9RjfdBVPxe7tNm7o5fxgqces7wtkuo07ZziwGgGEp858zJM+BPLfmpuNVrXAldTFHMXEwbRuSkl14sGnXliRACtVaBLG2g9tXZu3x/2rjknNBw== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: This patch corrects this issue by: 1. Checking if the zone or preferred zone is CPU-less before updating the NUMA stats. 2. Ensuring NUMA_HIT is only updated if the zone is not CPU-less. 3. Ensuring NUMA_FOREIGN is only updated if the preferred zone is not CPU-less. Example Before and After Patch: - Before Patch: node0 node1 node2 numa_hit 86333181 114338269 5108 numa_miss 5199455 0 56844591 numa_foreign 32281033 29763013 0 interleave_hit 91 91 0 local_node 86326417 114288458 0 other_node 5206219 49768 56849702 - After Patch: node0 node1 node2 numa_hit 2523058 9225528 0 numa_miss 150213 10226 21495942 numa_foreign 17144215 4501270 0 interleave_hit 91 94 0 local_node 2493918 9208226 0 other_node 179351 27528 21495942 In the case of memoryless node, when a process prefers a node with no memory(e.g., because it is running on a CPU local to that node), the kernel treats a nearby node with memory as the preffered node. As a result, such allocation do not increment the numa_foreign counter on the memoryless node, leading to skewed NUMA_HIT, NUMA_MISS, and NUMA_FOREIGN stat for the nearest node. Similarly, in the context of cpuless nodes, this patch ensures that NUMA statistics are accurately updated by adding checks to prevent the miscounting of memory allocations when the involved nodes have no CPUs. This ensures more precise tracking of memory access patterns accross all nodes, regardless of whether they have CPUs or not, improving the overall reliability of NUMA stat. The reason is that page allocation from dev_dax, cpuset, memcg .. comes with preferred allocating zone in cpuless node and its hard to track the zone info for miss information. Signed-off-by: Dongjoo Seo Cc: Davidlohr Bueso Cc: Fan Ni Cc: Dan Williams Cc: Adam Manzanares --- mm/page_alloc.c | 10 ++++++---- 1 file changed, 6 insertions(+), 4 deletions(-) diff --git a/mm/page_alloc.c b/mm/page_alloc.c index 0f33dab6d344..2981466e8e1a 100644 --- a/mm/page_alloc.c +++ b/mm/page_alloc.c @@ -2894,19 +2894,21 @@ static inline void zone_statistics(struct zone *preferred_zone, struct zone *z, { #ifdef CONFIG_NUMA enum numa_stat_item local_stat = NUMA_LOCAL; + bool z_is_cpuless = !node_state(zone_to_nid(z), N_CPU); + bool pref_is_cpuless = !node_state(zone_to_nid(preferred_zone), N_CPU); - /* skip numa counters update if numa stats is disabled */ if (!static_branch_likely(&vm_numa_stat_key)) return; - if (zone_to_nid(z) != numa_node_id()) + if (zone_to_nid(z) != numa_node_id() || z_is_cpuless) local_stat = NUMA_OTHER; - if (zone_to_nid(z) == zone_to_nid(preferred_zone)) + if (zone_to_nid(z) == zone_to_nid(preferred_zone) && !z_is_cpuless) __count_numa_events(z, NUMA_HIT, nr_account); else { __count_numa_events(z, NUMA_MISS, nr_account); - __count_numa_events(preferred_zone, NUMA_FOREIGN, nr_account); + if (!pref_is_cpuless) + __count_numa_events(preferred_zone, NUMA_FOREIGN, nr_account); } __count_numa_events(z, local_stat, nr_account); #endif