From patchwork Fri Jun 28 06:09:23 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Ho-Ren (Jack) Chuang" X-Patchwork-Id: 13715498 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 77888C2BBCA for ; Fri, 28 Jun 2024 06:09:49 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 08E7E6B0099; Fri, 28 Jun 2024 02:09:49 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id F09B86B009A; Fri, 28 Jun 2024 02:09:48 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id CE70D6B009B; Fri, 28 Jun 2024 02:09:48 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0017.hostedemail.com [216.40.44.17]) by kanga.kvack.org (Postfix) with ESMTP id AB10A6B0099 for ; Fri, 28 Jun 2024 02:09:48 -0400 (EDT) Received: from smtpin16.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay04.hostedemail.com (Postfix) with ESMTP id 5C7691A16B6 for ; Fri, 28 Jun 2024 06:09:48 +0000 (UTC) X-FDA: 82279271256.16.683CBD3 Received: from out-176.mta1.migadu.com (out-176.mta1.migadu.com [95.215.58.176]) by imf18.hostedemail.com (Postfix) with ESMTP id 67B5E1C000D for ; Fri, 28 Jun 2024 06:09:44 +0000 (UTC) Authentication-Results: imf18.hostedemail.com; dkim=pass header.d=linux.dev header.s=key1 header.b=ozNmNKLx; spf=pass (imf18.hostedemail.com: domain of horen.chuang@linux.dev designates 95.215.58.176 as permitted sender) smtp.mailfrom=horen.chuang@linux.dev; dmarc=pass (policy=none) header.from=linux.dev ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1719554966; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=o/wXQWGav/NessrnFMRHKMQL87wUrpOC9eqQBIItt68=; b=h68MqS6KoyAD36wEA6WbU+dZgJKSXMu0SIWo8mDfbVVCAPZ0MGBDrnHllt/Xepa6pHaxiE 0yGrOKVUBa1yXYHhAnQ5hhOMSRauXvwoEFEtFtJ+a619QdKd9ktzI/x7RPMpd3VhqcZV/r YSvpYvCWmmRhQmD+Rrt+FjqC8UgYKjI= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1719554966; a=rsa-sha256; cv=none; b=ctGuVr/dCa8yE6U9olmCONuUkdGXOZZCaO5sktVJYv+tDTDxWaC0mltHL+vfmW24nhql6R Mod8wHsM/LDZD/jZzspkN9Ze852/yhysloPQUETjf/WXMwWvT+sfMa32gN46lNCYnqnfOS 5Rn0WmneeZfYpgRB4NiIdSjZCDE+jDA= ARC-Authentication-Results: i=1; imf18.hostedemail.com; dkim=pass header.d=linux.dev header.s=key1 header.b=ozNmNKLx; spf=pass (imf18.hostedemail.com: domain of horen.chuang@linux.dev designates 95.215.58.176 as permitted sender) smtp.mailfrom=horen.chuang@linux.dev; dmarc=pass (policy=none) header.from=linux.dev X-Envelope-To: jonathan.cameron@huawei.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.dev; s=key1; t=1719554982; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=o/wXQWGav/NessrnFMRHKMQL87wUrpOC9eqQBIItt68=; b=ozNmNKLx7zHsfjGoKtTomX6lQR3xt4PbhMXFd8+/mQLV7/CTmKoDozleczUMPTB7/2URCA bJoZ4hvC8inLmZOIMrtIoltwu3L3JUXgkNLZaOV5fWa5q8EAkQizAhl00ojXGtM0CK/tnT KnuB2zFDjyF2eMZLo9Y6cbO0VB3/Eo0= X-Envelope-To: ying.huang@intel.com X-Envelope-To: gourry.memverge@gmail.com X-Envelope-To: aneesh.kumar@linux.ibm.com X-Envelope-To: mhocko@suse.com X-Envelope-To: tj@kernel.org X-Envelope-To: john@jagalactic.com X-Envelope-To: emirakhur@micron.com X-Envelope-To: vtavarespetr@micron.com X-Envelope-To: ravis.opensrc@micron.com X-Envelope-To: apopple@nvidia.com X-Envelope-To: sthanneeru@micron.com X-Envelope-To: sj@kernel.org X-Envelope-To: rafael@kernel.org X-Envelope-To: lenb@kernel.org X-Envelope-To: akpm@linux-foundation.org X-Envelope-To: dave.jiang@intel.com X-Envelope-To: dan.j.williams@intel.com X-Envelope-To: jonathan.cameron@huawei.com X-Envelope-To: horen.chuang@linux.dev X-Envelope-To: linux-acpi@vger.kernel.org X-Envelope-To: linux-kernel@vger.kernel.org X-Envelope-To: linux-mm@kvack.org X-Envelope-To: horenc@vt.edu X-Envelope-To: horenchuang@bytedance.com X-Envelope-To: horenchuang@gmail.com X-Envelope-To: linux-cxl@vger.kernel.org X-Envelope-To: qemu-devel@nongnu.org X-Report-Abuse: Please report any abuse attempt to abuse@migadu.com and include these headers. From: "Ho-Ren (Jack) Chuang" To: "Jonathan Cameron" , "Huang, Ying" , "Gregory Price" , aneesh.kumar@linux.ibm.com, mhocko@suse.com, tj@kernel.org, john@jagalactic.com, "Eishan Mirakhur" , "Vinicius Tavares Petrucci" , "Ravis OpenSrc" , "Alistair Popple" , "Srinivasulu Thanneeru" , "SeongJae Park" , "Rafael J. Wysocki" , Len Brown , Andrew Morton , Dave Jiang , Dan Williams , Jonathan Cameron , "Ho-Ren (Jack) Chuang" , linux-acpi@vger.kernel.org, linux-kernel@vger.kernel.org, linux-mm@kvack.org Cc: "Ho-Ren (Jack) Chuang" , "Ho-Ren (Jack) Chuang" , "Ho-Ren (Jack) Chuang" , linux-cxl@vger.kernel.org, qemu-devel@nongnu.org Subject: [PATCH v2 1/1] memory tier: consolidate the initialization of memory tiers Date: Fri, 28 Jun 2024 06:09:23 +0000 Message-Id: <20240628060925.303309-2-horen.chuang@linux.dev> In-Reply-To: <20240628060925.303309-1-horen.chuang@linux.dev> References: <20240628060925.303309-1-horen.chuang@linux.dev> MIME-Version: 1.0 X-Migadu-Flow: FLOW_OUT X-Stat-Signature: rtiz95p6a61tiyc3gjga9jsn371qdgch X-Rspamd-Queue-Id: 67B5E1C000D X-Rspam-User: X-Rspamd-Server: rspam08 X-HE-Tag: 1719554984-748006 X-HE-Meta: U2FsdGVkX1+Mgu8iOdLOi3xt52ww3l4r83Ni7bHbGiajtmGIYfzt5zljKVzRzNME2fwMTHvzrryB8gbyjF3X6om0C3hSNicV1JFimdI0W4hL0z18i3UmjalnWFLCH6qAyup5G6Eui9lRsyInfHigKnkouKJlKn9Fx/rC3TAicNEX12cbToMq8a2JYAvtlz/7c3gBXTD+e62I3fGEV3YZ6wVzO57DPxy4tBPDZvpSNBtI4GoE45B3hQkJYvn+zoWJgk/mr+n6EUE8PVZ1dYLRB4L7V3mYBMHz+czkR0r3S0RkYrzKkBvOxLzkr3tGtBWmEomQ1ZEoTBuN+ZXMjayzZ0gcVNeLxIdjsZqBdpQ0MRrlgcaiml1SbpGV2hVSUUuth/Dj+Y05ULWGDH3KamU3SwF4gR+iAglKe+i/Un53d6dHKdfq/cqJRjtdkeevJUE1N8d8cjoKoQzCE4lxlfcxRg/4EF8DDKLFfcvLIN74RxYb9SabzUbrfYADS9jnRA6si/vb3YKJEqEd9ebUKv7+oxVQxnZr0vaJtKK6Z0y8YJM7pPXLWYgupwPOMvJ14Dm5/oEKNuqCzAq3ROK203iPjTrx0ZawD8KapZxQpLk0S1WYnc9D/JDbTMGErq5hY/6FBs4pYbCQ0HZCVAa8MQk75mkkw/9nViUyJ15EGiJQ8ALWpWGsjtcA/socOua7/PMRxQkRlAU62j29idVXRspOqVgYN9SFMwUPYFd/J7lEEmgqDCXALGKNh5hIkieckI9rlEAwJLUda3FLLF73ckKBy82wvfbsPaC86khUmsrPKjDW+aChniTZCuI1JmZEdX1lO1K4CI/ZfOlwLTJaaoS6mWmfmbimRUqPBcXg8ne7qddVDIi7LyVXA1svvuH+ZtENP8UR2JvaP+1c3akg9JHNc7h/LdjOrUJWb4Ap/DGiBbUWSyXxUgSO1vre7WM8u9Uebf8v0trzpxMPjQYo/fg 5ymuCMha sWtSttuPPvKJP8LwqnPg895sb6tCRNQAnUG+hQotYoOkg0niuOhi39EqUxK/hC1upiDLeKzh0YILxr/vripQKpr25Eij4BUdYAhe5lJJJdGzFEklJ+kPbGPtQQZ1Sxmnfi08ekmh9MB+Qq0uHnz9hjZHefhSNX48KC47dLNCh1VFBoKZoXaaCgXrMRk2PkAlFU+qse5QMeot35KiE2hS/noTRkRjRm34WEdO49CBWGnRMOrAFrxNpyaeJxb4x6doiD9lQ4zZbA1F9bkw= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: If we simply move the set_node_memory_tier() from memory_tier_init() to late_initcall(), it will result in HMAT not registering the mt_adistance_algorithm callback function, because set_node_memory_tier() is not performed during the memory tiering initialization phase, leading to a lack of correct default_dram information. Therefore, we introduced a nodemask to pass the information of the default DRAM nodes. The reason for not choosing to reuse default_dram_type->nodes is that it is not clean enough. So in the end, we use a __initdata variable, which is a variable that is released once initialization is complete, including both CPU and memory nodes for HMAT to iterate through. Besides, since default_dram_type may be checked/used during the initialization process of HMAT and drivers, it is better to keep the allocation of default_dram_type in memory_tier_init(). Signed-off-by: Ho-Ren (Jack) Chuang Suggested-by: Jonathan Cameron --- drivers/acpi/numa/hmat.c | 5 +-- include/linux/memory-tiers.h | 2 ++ mm/memory-tiers.c | 59 +++++++++++++++--------------------- 3 files changed, 28 insertions(+), 38 deletions(-) diff --git a/drivers/acpi/numa/hmat.c b/drivers/acpi/numa/hmat.c index 2c8ccc91ebe6..a2f9e7a4b479 100644 --- a/drivers/acpi/numa/hmat.c +++ b/drivers/acpi/numa/hmat.c @@ -940,10 +940,7 @@ static int hmat_set_default_dram_perf(void) struct memory_target *target; struct access_coordinate *attrs; - if (!default_dram_type) - return -EIO; - - for_each_node_mask(nid, default_dram_type->nodes) { + for_each_node_mask(nid, default_dram_nodes) { pxm = node_to_pxm(nid); target = find_mem_target(pxm); if (!target) diff --git a/include/linux/memory-tiers.h b/include/linux/memory-tiers.h index 0d70788558f4..fa61ad9c4d75 100644 --- a/include/linux/memory-tiers.h +++ b/include/linux/memory-tiers.h @@ -38,6 +38,7 @@ struct access_coordinate; #ifdef CONFIG_NUMA extern bool numa_demotion_enabled; extern struct memory_dev_type *default_dram_type; +extern nodemask_t default_dram_nodes __initdata; struct memory_dev_type *alloc_memory_type(int adistance); void put_memory_type(struct memory_dev_type *memtype); void init_node_memory_type(int node, struct memory_dev_type *default_type); @@ -76,6 +77,7 @@ static inline bool node_is_toptier(int node) #define numa_demotion_enabled false #define default_dram_type NULL +#define default_dram_nodes NODE_MASK_NONE /* * CONFIG_NUMA implementation returns non NULL error. */ diff --git a/mm/memory-tiers.c b/mm/memory-tiers.c index 6632102bd5c9..a19a90c3ad36 100644 --- a/mm/memory-tiers.c +++ b/mm/memory-tiers.c @@ -43,6 +43,7 @@ static LIST_HEAD(memory_tiers); static LIST_HEAD(default_memory_types); static struct node_memory_type_map node_memory_types[MAX_NUMNODES]; struct memory_dev_type *default_dram_type; +nodemask_t default_dram_nodes __initdata = NODE_MASK_NONE; static const struct bus_type memory_tier_subsys = { .name = "memory_tiering", @@ -671,28 +672,38 @@ EXPORT_SYMBOL_GPL(mt_put_memory_types); /* * This is invoked via `late_initcall()` to initialize memory tiers for - * CPU-less memory nodes after driver initialization, which is - * expected to provide `adistance` algorithms. + * memory nodes, both with and without CPUs. After the initialization of + * firmware and devices, adistance algorithms are expected to be provided. */ static int __init memory_tier_late_init(void) { int nid; + struct memory_tier *memtier; + get_online_mems(); guard(mutex)(&memory_tier_lock); + /* + * Look at all the existing and uninitialized N_MEMORY nodes and + * add them to default memory tier or to a tier if we already have + * memory types assigned. + */ for_each_node_state(nid, N_MEMORY) { /* - * Some device drivers may have initialized memory tiers - * between `memory_tier_init()` and `memory_tier_late_init()`, - * potentially bringing online memory nodes and - * configuring memory tiers. Exclude them here. + * Some device drivers may have initialized + * memory tiers, potentially bringing memory nodes + * online and configuring memory tiers. + * Exclude them here. */ if (node_memory_types[nid].memtype) continue; - set_node_memory_tier(nid); + memtier = set_node_memory_tier(nid); + if (IS_ERR(memtier)) + /* Continue with memtiers we are able to setup. */ + break; } - establish_demotion_targets(); + put_online_mems(); return 0; } @@ -875,8 +886,7 @@ static int __meminit memtier_hotplug_callback(struct notifier_block *self, static int __init memory_tier_init(void) { - int ret, node; - struct memory_tier *memtier; + int ret; ret = subsys_virtual_register(&memory_tier_subsys, NULL); if (ret) @@ -887,7 +897,8 @@ static int __init memory_tier_init(void) GFP_KERNEL); WARN_ON(!node_demotion); #endif - mutex_lock(&memory_tier_lock); + + guard(mutex)(&memory_tier_lock); /* * For now we can have 4 faster memory tiers with smaller adistance * than default DRAM tier. @@ -897,29 +908,9 @@ static int __init memory_tier_init(void) if (IS_ERR(default_dram_type)) panic("%s() failed to allocate default DRAM tier\n", __func__); - /* - * Look at all the existing N_MEMORY nodes and add them to - * default memory tier or to a tier if we already have memory - * types assigned. - */ - for_each_node_state(node, N_MEMORY) { - if (!node_state(node, N_CPU)) - /* - * Defer memory tier initialization on - * CPUless numa nodes. These will be initialized - * after firmware and devices are initialized. - */ - continue; - - memtier = set_node_memory_tier(node); - if (IS_ERR(memtier)) - /* - * Continue with memtiers we are able to setup - */ - break; - } - establish_demotion_targets(); - mutex_unlock(&memory_tier_lock); + /* Record nodes with memory and CPU to set default DRAM performance. */ + nodes_and(default_dram_nodes, node_states[N_MEMORY], + node_states[N_CPU]); hotplug_memory_notifier(memtier_hotplug_callback, MEMTIER_HOTPLUG_PRI); return 0;