| Message ID | 20231009204259.875232-1-gregory.price@memverge.com |
|---|---|
| Headers | show |
| Series | mm: mempolicy: Multi-tier weighted interleaving | expand |
On Wed, Oct 11, 2023 at 10:15:02PM +0100, Matthew Wilcox wrote: > On Mon, Oct 09, 2023 at 04:42:56PM -0400, Gregory Price wrote: > > == Mutex to Semaphore change: > > > > The memory tiering subsystem is extended in this patch set to have > > externally available information (weights), and therefore additional > > controls need to be added to ensure values are not changed (or tiers > > changed/added/removed) during various calculations. > > > > Since it is expected that many threads will be accessing this data > > during allocations, a mutex is not appropriate. > > > > Since write-updates (weight changes, hotplug events) are rare events, > > a simple rw semaphore is sufficient. > > Given how you're using it, wouldn't the existing RCU mechanism be > better than converting this to an rwsem? > ... yes, and a smarter person would have just done that first :P derp derp, thanks, I'll update. ~Gregory
On Mon, Oct 09, 2023 at 04:42:56PM -0400, Gregory Price wrote: > == Mutex to Semaphore change: > > The memory tiering subsystem is extended in this patch set to have > externally available information (weights), and therefore additional > controls need to be added to ensure values are not changed (or tiers > changed/added/removed) during various calculations. > > Since it is expected that many threads will be accessing this data > during allocations, a mutex is not appropriate. > > Since write-updates (weight changes, hotplug events) are rare events, > a simple rw semaphore is sufficient. Given how you're using it, wouldn't the existing RCU mechanism be better than converting this to an rwsem?
Gregory Price <gourry.memverge@gmail.com> writes: > v2: change memtier mutex to semaphore > add source-node relative weighting > add remaining mempolicy integration code > > = v2 Notes > > Developed in colaboration with original authors to deconflict > similar efforts to extend mempolicy to take weights directly. > > == Mutex to Semaphore change: > > The memory tiering subsystem is extended in this patch set to have > externally available information (weights), and therefore additional > controls need to be added to ensure values are not changed (or tiers > changed/added/removed) during various calculations. > > Since it is expected that many threads will be accessing this data > during allocations, a mutex is not appropriate. IIUC, this is a change for performance. If so, please show some performance data. > Since write-updates (weight changes, hotplug events) are rare events, > a simple rw semaphore is sufficient. > > == Source-node relative weighting: > > Tiers can now be weighted differently based on the node requesting > the weight. For example CPU-Nodes 0 and 1 may have different weights > for the same CXL memory tier, because topologically the number of > NUMA hops is greater (or any other physical topological difference > resulting in different effective latency or bandwidth values) > > 1. Set weights for DDR (tier4) and CXL(teir22) tiers. > echo source_node:weight > /path/to/interleave_weight If source_node is considered, why not consider target_node too? On a system with only 1 tier (DRAM), do you want weighted interleaving among NUMA nodes? If so, why tie weighted interleaving with memory tiers? Why not just introduce weighted interleaving for NUMA nodes? > # Set tier4 weight from node 0 to 85 > echo 0:85 > /sys/devices/virtual/memory_tiering/memory_tier4/interleave_weight > # Set tier4 weight from node 1 to 65 > echo 1:65 > /sys/devices/virtual/memory_tiering/memory_tier4/interleave_weight > # Set tier22 weight from node 0 to 15 > echo 0:15 > /sys/devices/virtual/memory_tiering/memory_tier22/interleave_weight > # Set tier22 weight from node 1 to 10 > echo 1:10 > /sys/devices/virtual/memory_tiering/memory_tier22/interleave_weight -- Best Regards, Huang, Ying
On Mon, Oct 16, 2023 at 03:57:52PM +0800, Huang, Ying wrote: > Gregory Price <gourry.memverge@gmail.com> writes: > > > == Mutex to Semaphore change: > > > > Since it is expected that many threads will be accessing this data > > during allocations, a mutex is not appropriate. > > IIUC, this is a change for performance. If so, please show some > performance data. > This change will be dropped in v3 in favor of the existing RCU mechanism in memory-tiers.c as pointed out by Matthew. > > == Source-node relative weighting: > > > > 1. Set weights for DDR (tier4) and CXL(teir22) tiers. > > echo source_node:weight > /path/to/interleave_weight > > If source_node is considered, why not consider target_node too? On a > system with only 1 tier (DRAM), do you want weighted interleaving among > NUMA nodes? If so, why tie weighted interleaving with memory tiers? > Why not just introduce weighted interleaving for NUMA nodes? > The short answer: Practicality and ease-of-use. The long answer: We have been discussing how to make this more flexible.. Personally, I agree with you. If Task A is on Socket 0, the weight on Socket 0 DRAM should not be the same as the weight on Socket 1 DRAM. However, right now, DRAM nodes are lumped into the same tier together, resulting in them having the same weight. If you scrollback through the list, you'll find an RFC I posted for set_mempolicy2 which implements weighted interleave in mm/mempolicy. However, mm/mempolicy is extremely `current-centric` at the moment, so that makes changing weights at runtime (in response to a hotplug event, for example) very difficult. I still think there is room to extend set_mempolicy to allow task-defined weights to take preference over tier defined weights. We have discussed adding the following features to memory-tiers: 1) breaking up tiers to allow 1 tier per node, as opposed to defaulting to lumping all nodes of a simlar quality into the same tier 2) enabling movemnet of nodes between tiers (for the purpose of reconfiguring due to hotplug and other situations) For users that require fine-grained control over each individual node, this would allow for weights to be applied per-node, because a node=tier. For the majority of use cases, it would allow clumping of nodes into tiers based on physical topology and performance class, and then allow for the general weighting to apply. This seems like the most obvious use-case that a majority of users would use, and also the easiest to set-up in the short term. That said, there are probably 3 or 4 different ways/places to implement this feature. The question is what is the clear and obvious way? I don't have a definitive answer for that, hence the RFC. There are at least 5 proposals that i know of at the moment 1) mempolicy 2) memory-tiers 3) memory-block interleaving? (weighting among blocks inside a node) Maybe relevant if Dynamic Capacity devices arrive, but it seems like the wrong place to do this. 4) multi-device nodes (e.g. cxl create-region ... mem0 mem1...) 5) "just do it in hardware" > > # Set tier4 weight from node 0 to 85 > > echo 0:85 > /sys/devices/virtual/memory_tiering/memory_tier4/interleave_weight > > # Set tier4 weight from node 1 to 65 > > echo 1:65 > /sys/devices/virtual/memory_tiering/memory_tier4/interleave_weight > > # Set tier22 weight from node 0 to 15 > > echo 0:15 > /sys/devices/virtual/memory_tiering/memory_tier22/interleave_weight > > # Set tier22 weight from node 1 to 10 > > echo 1:10 > /sys/devices/virtual/memory_tiering/memory_tier22/interleave_weight > > -- > Best Regards, > Huang, Ying
On Wed, Oct 18, 2023 at 04:29:02PM +0800, Huang, Ying wrote: > Gregory Price <gregory.price@memverge.com> writes: > > > There are at least 5 proposals that i know of at the moment > > > > 1) mempolicy > > 2) memory-tiers > > 3) memory-block interleaving? (weighting among blocks inside a node) > > Maybe relevant if Dynamic Capacity devices arrive, but it seems > > like the wrong place to do this. > > 4) multi-device nodes (e.g. cxl create-region ... mem0 mem1...) > > 5) "just do it in hardware" > > It may be easier to start with the use case. What is the practical use > cases in your mind that can not be satisfied with simple per-memory-tier > weight? Can you compare the memory layout with different proposals? > Before I delve in, one clarifying question: When you asked whether weights should be part of node or memory-tiers, i took that to mean whether it should be part of mempolicy or memory-tiers. Were you suggesting that weights should actually be part of drivers/base/node.c? Because I had not considered that, and this seems reasonable, easy to implement, and would not require tying mempolicy.c to memory-tiers.c Beyond this, i think there's been 3 imagined use cases (now, including this). a) numactl --weighted-interleave=Node:weight,0:16,1:4,... b) echo weight > /sys/.../memory-tiers/memtier/access0/interleave_weight numactl --interleave=0,1 c) echo weight > /sys/bus/node/node0/access0/interleave_weight numactl --interleave=0,1 d) options b or c, but with --weighted-interleave=0,1 instead this requires libnuma changes to pick up, but it retains --interleave as-is to avoid user confusion. The downside of an approach like A (which was my original approach), was that the weights cannot really change should a node be hotplugged. Tasks would need to detect this and change the policy themselves. That's not a good solution. However in both B and C's design, weights can be rebalanced in response to any number of events. Ultimately B and C are equivalent, but the placement in nodes is cleaner and more intuitive. If memory-tiers wants to use/change this information, there's nothing that prevents it. Assuming this is your meaning, I agree and I will pivot to this. ~Gregory
On Thu, Oct 19, 2023 at 02:28:42PM +0800, Huang, Ying wrote: > Gregory Price <gregory.price@memverge.com> writes: > > Were you suggesting that weights should actually be part of > > drivers/base/node.c? > > Yes. drivers/base/node.c vs. memory tiers. > Then yes I agree this can and probably should be placed there, especially since I see accessor details are now being exposed at that level, which can be used to auto-generate weights (assuming HMAT/CDAT data exposed by devices is actually accurate). > > Assuming this is your meaning, I agree and I will pivot to this. > > Can you give a not-so-abstract example? For example, on a system with > node 0, 1, 2, 3, memory tiers 4 (0, 1), 22 (2, 3), .... A workload runs > on CPU of node 0, ...., interleaves memory on node 0, 1, ... Then > compare the different behavior (including memory bandwidth) with node > and memory-tier based solution. ah, I see. Example 1: A single-socket system with multiple CXL memory devices === CPU Node: node0 CXL Nodes: node1, node2 Bandwidth attributes (in theory): node0 - 8 channels - ~307GB/s node1 - x16 link - 64GB/s node2 - x8 link - 32GB/s In a system like this, the optimal distribution of memory on an interleave for maximizing bandwidth is about 76%/16%/8%. for the sake of simplicity: --weighted-interleave=0:76,1:16,0:8 but realistically we could make the weights sysfs values in the node Regardless of the mechanism to engage this, the most effective way to capture this in the system is by applying weights to nodes, not tiers. If done in tiers, each node would be assigned to its own tier, making the mechanism equivalent. So you might as well simplify the whole thing and chop the memtier component out. Is this configuration realistic? *shrug* - technically possible. And in fact most hardware or driver based interleaving mechanisms would not really be able to manage an interleave region across these nodes, at least not without placing the x16 driver in x8 mode, or just having the wrong distribution %'s. Example 2: A dual-socket system with 1 CXL device per socket === CPU Nodes: node0, node1 CXL Nodes: node2, node3 (on sockets 0 and 1 respective) Bandwidth Attributes (in theory): nodes 0 & 1 - 8 channels - ~307GB/s ea. nodes 2 & 3 - x16 link - 64GB/s ea. This is similar to example #1, but with one difference: A task running on node 0 should not treat nodes 0 and 1 the same, nor nodes 2 and 3. This is because on access to nodes 1 and 3, the cross-socket link (UPI, or whatever AMD calls it) becomes a bandwidth chokepoint. So from the perspective of node 0, the "real total" available bandwidth is about 307GB+64GB+(41.6GB * UPI Links) in the case of intel. so the best result you could get is around 307+64+164=535GB/s if you have the full 4 links. You'd want to distribute the cross-socket traffic proportional to UPI, not the total. This leaves us with weights of: node0 - 57% node1 - 26% node2 - 12% node3 - 5% Again, naturally nodes are the place to carry the weights here. In this scenario, placing it in memory-tiers would require that 1 tier per node existed. Example 3: A single-socket system with 2 CXL devices === Different than example 1: Both devices are the same. CPU Node: node0 CXL Nodes: node1, node2 Bandwidth attributes (in theory): node0 - 8 channels - ~307GB/s node1/2 - x16 link - 64GB/s In this case you want the weights to be: 70/15/15% respectively Caveat: A user may, in fact, use the CXL driver to implement a single node which naturally interleaves the 2 devices. In this case it's the same as a 1-socket, 1-device setup which is trivially 1-node-per-tier, and therefore weights should live with nodes. In the case of a single memory tier, you could simply make this 70/30. However, and this the the argument against placing it in the memory-tier: the user is free to hack-off any of the chosen numa nodes via mempolicy, which makes the aggregated weight meaningless. Example: --weighted-interleave=0,1 Under my current code, if I set the weights to 70/30 in the memory-tiers code, the result is that node1 inherits the full 30% defined in the tier, which leads to a suboptimal distribution. What you actually want in this case is about 83/17% split. However, this also presents an issue for placing the weights in the nodes: A node weight is meaningless outside the context of the active context. If I have 2 nodes and i set their weights to 70/30, and I hack off node1, i can't have 70% of all memory go to node0, I have to send 100% of the memory to node0 - making the weight functionally meaningless. So this would imply a single global weight set on the nodes is ALSO a bad solution, and instead it would be better to extend set_mempolicy to have a --weighted-interleave option that reads HMAT/CDAT provided bandwidth data and generates the weights for the selected nodes as part of the policy. The downside of this is that if the HMAT/CDAT data is garbage, the policy becomes garbage. To mitigate this, we should consider allowing userland to override those values explicitly for the purpose of weighted interleave should the HMAT/CDAT information be garbage/lies. Another downside to this is that nodemask changes require recalculation of weights, which may introduce some racey conditions, but that can probably be managed. Should we carry weights in node, memtier, or mempolicy? === The current analysis suggest carrying it in memory-tier would simply require memory-tier to make 1 tier per node - which may or may not be consistent with other goals of the memtier subsystem. The pros of placing a weight in node is that the "effective" weight in the context of the entire system can be calculated at the time nodes are created. If, at interleave time, the interface required a node+nodemask then it's probably preferable to forego manual weighting, and simply calculate based on HMAT/CDAT data. The downside of placing it in nodes is that mempolicy is free to set the interleave set to some combination of nodes, and this would prevent any nodes created after process launch from being used in the interleave set unless the software detected the hotplug event. I don't know how much of a real concern this is, but it is a limitation. The other option is to add --weighted-interleave, but have mempolicy generate the weights based on node-provided CDAT/HMAT data (or overrides), which keep almost everything inside of mempolicy except for a couple of interfaces to drivers/base/node.c that allow querying of that data. Summarize: === The weights are actually a function of bandwidth, and can probably be calculated on the fly - rather than being manually set. However, we may want to consider allowing the bandwidth attributes being exposed by CDAT/HMAT to be overridden should the user discover they are functionally incorrect. (for reference: I have seen this myself, where a device published 5GB/s but actually achieves 22GB/s). for reference, they are presently RO: static DEVICE_ATTR_RO(property) ACCESS_ATTR(read_bandwidth); ACCESS_ATTR(read_latency); ACCESS_ATTR(write_bandwidth); ACCESS_ATTR(write_latency); ~Gregory
Gregory Price <gregory.price@memverge.com> writes: > On Mon, Oct 16, 2023 at 03:57:52PM +0800, Huang, Ying wrote: >> Gregory Price <gourry.memverge@gmail.com> writes: >> >> > == Mutex to Semaphore change: >> > >> > Since it is expected that many threads will be accessing this data >> > during allocations, a mutex is not appropriate. >> >> IIUC, this is a change for performance. If so, please show some >> performance data. >> > > This change will be dropped in v3 in favor of the existing > RCU mechanism in memory-tiers.c as pointed out by Matthew. > >> > == Source-node relative weighting: >> > >> > 1. Set weights for DDR (tier4) and CXL(teir22) tiers. >> > echo source_node:weight > /path/to/interleave_weight >> >> If source_node is considered, why not consider target_node too? On a >> system with only 1 tier (DRAM), do you want weighted interleaving among >> NUMA nodes? If so, why tie weighted interleaving with memory tiers? >> Why not just introduce weighted interleaving for NUMA nodes? >> > > The short answer: Practicality and ease-of-use. > > The long answer: We have been discussing how to make this more flexible.. > > Personally, I agree with you. If Task A is on Socket 0, the weight on > Socket 0 DRAM should not be the same as the weight on Socket 1 DRAM. > However, right now, DRAM nodes are lumped into the same tier together, > resulting in them having the same weight. > > If you scrollback through the list, you'll find an RFC I posted for > set_mempolicy2 which implements weighted interleave in mm/mempolicy. > However, mm/mempolicy is extremely `current-centric` at the moment, > so that makes changing weights at runtime (in response to a hotplug > event, for example) very difficult. > > I still think there is room to extend set_mempolicy to allow > task-defined weights to take preference over tier defined weights. > > We have discussed adding the following features to memory-tiers: > > 1) breaking up tiers to allow 1 tier per node, as opposed to defaulting > to lumping all nodes of a simlar quality into the same tier > > 2) enabling movemnet of nodes between tiers (for the purpose of > reconfiguring due to hotplug and other situations) > > For users that require fine-grained control over each individual node, > this would allow for weights to be applied per-node, because a > node=tier. For the majority of use cases, it would allow clumping of > nodes into tiers based on physical topology and performance class, and > then allow for the general weighting to apply. This seems like the most > obvious use-case that a majority of users would use, and also the > easiest to set-up in the short term. > > That said, there are probably 3 or 4 different ways/places to implement > this feature. The question is what is the clear and obvious way? > I don't have a definitive answer for that, hence the RFC. > > There are at least 5 proposals that i know of at the moment > > 1) mempolicy > 2) memory-tiers > 3) memory-block interleaving? (weighting among blocks inside a node) > Maybe relevant if Dynamic Capacity devices arrive, but it seems > like the wrong place to do this. > 4) multi-device nodes (e.g. cxl create-region ... mem0 mem1...) > 5) "just do it in hardware" It may be easier to start with the use case. What is the practical use cases in your mind that can not be satisfied with simple per-memory-tier weight? Can you compare the memory layout with different proposals? >> > # Set tier4 weight from node 0 to 85 >> > echo 0:85 > /sys/devices/virtual/memory_tiering/memory_tier4/interleave_weight >> > # Set tier4 weight from node 1 to 65 >> > echo 1:65 > /sys/devices/virtual/memory_tiering/memory_tier4/interleave_weight >> > # Set tier22 weight from node 0 to 15 >> > echo 0:15 > /sys/devices/virtual/memory_tiering/memory_tier22/interleave_weight >> > # Set tier22 weight from node 1 to 10 >> > echo 1:10 > /sys/devices/virtual/memory_tiering/memory_tier22/interleave_weight -- Best Regards, Huang, Ying
Forget to Cc more people. "Huang, Ying" <ying.huang@intel.com> writes: > Gregory Price <gregory.price@memverge.com> writes: > >> On Mon, Oct 16, 2023 at 03:57:52PM +0800, Huang, Ying wrote: >>> Gregory Price <gourry.memverge@gmail.com> writes: >>> >>> > == Mutex to Semaphore change: >>> > >>> > Since it is expected that many threads will be accessing this data >>> > during allocations, a mutex is not appropriate. >>> >>> IIUC, this is a change for performance. If so, please show some >>> performance data. >>> >> >> This change will be dropped in v3 in favor of the existing >> RCU mechanism in memory-tiers.c as pointed out by Matthew. >> >>> > == Source-node relative weighting: >>> > >>> > 1. Set weights for DDR (tier4) and CXL(teir22) tiers. >>> > echo source_node:weight > /path/to/interleave_weight >>> >>> If source_node is considered, why not consider target_node too? On a >>> system with only 1 tier (DRAM), do you want weighted interleaving among >>> NUMA nodes? If so, why tie weighted interleaving with memory tiers? >>> Why not just introduce weighted interleaving for NUMA nodes? >>> >> >> The short answer: Practicality and ease-of-use. >> >> The long answer: We have been discussing how to make this more flexible.. >> >> Personally, I agree with you. If Task A is on Socket 0, the weight on >> Socket 0 DRAM should not be the same as the weight on Socket 1 DRAM. >> However, right now, DRAM nodes are lumped into the same tier together, >> resulting in them having the same weight. >> >> If you scrollback through the list, you'll find an RFC I posted for >> set_mempolicy2 which implements weighted interleave in mm/mempolicy. >> However, mm/mempolicy is extremely `current-centric` at the moment, >> so that makes changing weights at runtime (in response to a hotplug >> event, for example) very difficult. >> >> I still think there is room to extend set_mempolicy to allow >> task-defined weights to take preference over tier defined weights. >> >> We have discussed adding the following features to memory-tiers: >> >> 1) breaking up tiers to allow 1 tier per node, as opposed to defaulting >> to lumping all nodes of a simlar quality into the same tier >> >> 2) enabling movemnet of nodes between tiers (for the purpose of >> reconfiguring due to hotplug and other situations) >> >> For users that require fine-grained control over each individual node, >> this would allow for weights to be applied per-node, because a >> node=tier. For the majority of use cases, it would allow clumping of >> nodes into tiers based on physical topology and performance class, and >> then allow for the general weighting to apply. This seems like the most >> obvious use-case that a majority of users would use, and also the >> easiest to set-up in the short term. >> >> That said, there are probably 3 or 4 different ways/places to implement >> this feature. The question is what is the clear and obvious way? >> I don't have a definitive answer for that, hence the RFC. >> >> There are at least 5 proposals that i know of at the moment >> >> 1) mempolicy >> 2) memory-tiers >> 3) memory-block interleaving? (weighting among blocks inside a node) >> Maybe relevant if Dynamic Capacity devices arrive, but it seems >> like the wrong place to do this. >> 4) multi-device nodes (e.g. cxl create-region ... mem0 mem1...) >> 5) "just do it in hardware" > > It may be easier to start with the use case. What is the practical use > cases in your mind that can not be satisfied with simple per-memory-tier > weight? Can you compare the memory layout with different proposals? > >>> > # Set tier4 weight from node 0 to 85 >>> > echo 0:85 > /sys/devices/virtual/memory_tiering/memory_tier4/interleave_weight >>> > # Set tier4 weight from node 1 to 65 >>> > echo 1:65 > /sys/devices/virtual/memory_tiering/memory_tier4/interleave_weight >>> > # Set tier22 weight from node 0 to 15 >>> > echo 0:15 > /sys/devices/virtual/memory_tiering/memory_tier22/interleave_weight >>> > # Set tier22 weight from node 1 to 10 >>> > echo 1:10 > /sys/devices/virtual/memory_tiering/memory_tier22/interleave_weight > > -- > Best Regards, > Huang, Ying
Gregory Price <gregory.price@memverge.com> writes: > On Wed, Oct 18, 2023 at 04:29:02PM +0800, Huang, Ying wrote: >> Gregory Price <gregory.price@memverge.com> writes: >> >> > There are at least 5 proposals that i know of at the moment >> > >> > 1) mempolicy >> > 2) memory-tiers >> > 3) memory-block interleaving? (weighting among blocks inside a node) >> > Maybe relevant if Dynamic Capacity devices arrive, but it seems >> > like the wrong place to do this. >> > 4) multi-device nodes (e.g. cxl create-region ... mem0 mem1...) >> > 5) "just do it in hardware" >> >> It may be easier to start with the use case. What is the practical use >> cases in your mind that can not be satisfied with simple per-memory-tier >> weight? Can you compare the memory layout with different proposals? >> > > Before I delve in, one clarifying question: When you asked whether > weights should be part of node or memory-tiers, i took that to mean > whether it should be part of mempolicy or memory-tiers. > > Were you suggesting that weights should actually be part of > drivers/base/node.c? Yes. drivers/base/node.c vs. memory tiers. > Because I had not considered that, and this seems reasonable, easy to > implement, and would not require tying mempolicy.c to memory-tiers.c > > > > Beyond this, i think there's been 3 imagined use cases (now, including > this). > > a) > numactl --weighted-interleave=Node:weight,0:16,1:4,... > > b) > echo weight > /sys/.../memory-tiers/memtier/access0/interleave_weight > numactl --interleave=0,1 > > c) > echo weight > /sys/bus/node/node0/access0/interleave_weight > numactl --interleave=0,1 > > d) > options b or c, but with --weighted-interleave=0,1 instead > this requires libnuma changes to pick up, but it retains --interleave > as-is to avoid user confusion. > > The downside of an approach like A (which was my original approach), was > that the weights cannot really change should a node be hotplugged. Tasks > would need to detect this and change the policy themselves. That's not > a good solution. > > However in both B and C's design, weights can be rebalanced in response > to any number of events. Ultimately B and C are equivalent, but > the placement in nodes is cleaner and more intuitive. If memory-tiers > wants to use/change this information, there's nothing that prevents it. > > Assuming this is your meaning, I agree and I will pivot to this. Can you give a not-so-abstract example? For example, on a system with node 0, 1, 2, 3, memory tiers 4 (0, 1), 22 (2, 3), .... A workload runs on CPU of node 0, ...., interleaves memory on node 0, 1, ... Then compare the different behavior (including memory bandwidth) with node and memory-tier based solution. -- Best Regards, Huang, Ying
On Fri, Oct 20, 2023 at 02:11:40PM +0800, Huang, Ying wrote: > Gregory Price <gregory.price@memverge.com> writes: > > > [...snip...] > > Example 2: A dual-socket system with 1 CXL device per socket > > === > > CPU Nodes: node0, node1 > > CXL Nodes: node2, node3 (on sockets 0 and 1 respective) > > [...snip...] > > This is similar to example #1, but with one difference: A task running > > on node 0 should not treat nodes 0 and 1 the same, nor nodes 2 and 3. [...snip...] > > This leaves us with weights of: > > > > node0 - 57% > > node1 - 26% > > node2 - 12% > > node3 - 5% > > > > Does the workload run on CPU of node 0 only? This appears unreasonable. Depends. if a user explicitly launches with `numactl --cpunodebind=0` then yes, you can force a task (and all its children) to run on node0. If a workload multi-threaded enough to run on both sockets, then you are right that you'd want to basically limit cross-socket traffic by binding individual threads to nodes that don't cross sockets - if at all feasible this may not be feasible). But at that point, we're getting into the area of numa-aware software. That's a bit beyond the scope of this - which is to enable a coarse grained interleaving solution that can easily be accessed with something like `numactl --interleave` or `numactl --weighted-interleave`. > If the memory bandwidth requirement of the workload is so large that CXL > is used to expand bandwidth, why not run workload on CPU of node 1 and > use the full memory bandwidth of node 1? Settings are NOT one size fits all. You can certainly come up with another scenario in which these weights are not optimal. If we're running enough threads that we need multiple sockets to run them concurrently, then the memory distribution weights become much more complex. Without more precise control over task placement and preventing task migration, you can't really get an "optimal" placement. What I'm really saying is "Task placement is a more powerful function for predicting performance than memory placement". However, user software would need to implement a pseudo-scheduler and explicit data placement to be the most optimized. Beyond this, there is only so much we can do from a `numactl` perspective. tl;dr: We can't get a perfect system here, because getting a best-case for all possible scenarios is an probably undecidable problem. You will always be able to generate an example wherein the system is not optimal. > > If the workload run on CPU of node 0 and node 1, then the cross-socket > traffic should be minimized if possible. That is, threads/processes on > node 0 should interleave memory of node 0 and node 2, while that on node > 1 should interleave memory of node 1 and node 3. This can be done with set_mempolicy() with MPOL_INTERLEAVE and set the nodemask to the what you describe. Those tasks need to also prevent themselves from being migrated as well. But this can absolutely be done. In this scenario, the weights need to be re-calculated to be based on the bandwidth of the nodes in the mempolicy nodemask, which is what i described in the last email. ~Gregory
Gregory Price <gregory.price@memverge.com> writes: [snip] > Example 1: A single-socket system with multiple CXL memory devices > === > CPU Node: node0 > CXL Nodes: node1, node2 > > Bandwidth attributes (in theory): > node0 - 8 channels - ~307GB/s > node1 - x16 link - 64GB/s > node2 - x8 link - 32GB/s > > In a system like this, the optimal distribution of memory on an > interleave for maximizing bandwidth is about 76%/16%/8%. > > for the sake of simplicity: --weighted-interleave=0:76,1:16,0:8 > but realistically we could make the weights sysfs values in the node > > Regardless of the mechanism to engage this, the most effective way to > capture this in the system is by applying weights to nodes, not tiers. > If done in tiers, each node would be assigned to its own tier, making > the mechanism equivalent. So you might as well simplify the whole thing > and chop the memtier component out. > > Is this configuration realistic? *shrug* - technically possible. And in > fact most hardware or driver based interleaving mechanisms would not > really be able to manage an interleave region across these nodes, at > least not without placing the x16 driver in x8 mode, or just having the > wrong distribution %'s. > > > > Example 2: A dual-socket system with 1 CXL device per socket > === > CPU Nodes: node0, node1 > CXL Nodes: node2, node3 (on sockets 0 and 1 respective) > > Bandwidth Attributes (in theory): > nodes 0 & 1 - 8 channels - ~307GB/s ea. > nodes 2 & 3 - x16 link - 64GB/s ea. > > This is similar to example #1, but with one difference: A task running > on node 0 should not treat nodes 0 and 1 the same, nor nodes 2 and 3. > This is because on access to nodes 1 and 3, the cross-socket link (UPI, > or whatever AMD calls it) becomes a bandwidth chokepoint. > > So from the perspective of node 0, the "real total" available bandwidth > is about 307GB+64GB+(41.6GB * UPI Links) in the case of intel. so the > best result you could get is around 307+64+164=535GB/s if you have the > full 4 links. > > You'd want to distribute the cross-socket traffic proportional to UPI, > not the total. > > This leaves us with weights of: > > node0 - 57% > node1 - 26% > node2 - 12% > node3 - 5% > > Again, naturally nodes are the place to carry the weights here. In this > scenario, placing it in memory-tiers would require that 1 tier per node > existed. Does the workload run on CPU of node 0 only? This appears unreasonable. If the memory bandwidth requirement of the workload is so large that CXL is used to expand bandwidth, why not run workload on CPU of node 1 and use the full memory bandwidth of node 1? If the workload run on CPU of node 0 and node 1, then the cross-socket traffic should be minimized if possible. That is, threads/processes on node 0 should interleave memory of node 0 and node 2, while that on node 1 should interleave memory of node 1 and node 3. But TBH, I lacks knowledge about the real life workloads. So, my understanding may be wrong. Please correct me for any mistake. -- Best Regards, Huang, Ying [snip]
Gregory Price <gregory.price@memverge.com> writes: > On Fri, Oct 20, 2023 at 02:11:40PM +0800, Huang, Ying wrote: >> Gregory Price <gregory.price@memverge.com> writes: >> >> > > [...snip...] >> > Example 2: A dual-socket system with 1 CXL device per socket >> > === >> > CPU Nodes: node0, node1 >> > CXL Nodes: node2, node3 (on sockets 0 and 1 respective) >> > > [...snip...] >> > This is similar to example #1, but with one difference: A task running >> > on node 0 should not treat nodes 0 and 1 the same, nor nodes 2 and 3. > [...snip...] >> > This leaves us with weights of: >> > >> > node0 - 57% >> > node1 - 26% >> > node2 - 12% >> > node3 - 5% >> > >> >> Does the workload run on CPU of node 0 only? This appears unreasonable. > > Depends. if a user explicitly launches with `numactl --cpunodebind=0` > then yes, you can force a task (and all its children) to run on node0. IIUC, in your example, the `numactl` command line will be numactl --cpunodebind=0 --weighted-interleave=0,1,2,3 That is, the CPU is restricted to node 0, while memory is distributed to all nodes. This doesn't sound like reasonable for me. > If a workload multi-threaded enough to run on both sockets, then you are > right that you'd want to basically limit cross-socket traffic by binding > individual threads to nodes that don't cross sockets - if at all > feasible this may not be feasible). > > But at that point, we're getting into the area of numa-aware software. > That's a bit beyond the scope of this - which is to enable a coarse > grained interleaving solution that can easily be accessed with something > like `numactl --interleave` or `numactl --weighted-interleave`. > >> If the memory bandwidth requirement of the workload is so large that CXL >> is used to expand bandwidth, why not run workload on CPU of node 1 and >> use the full memory bandwidth of node 1? > > Settings are NOT one size fits all. You can certainly come up with another > scenario in which these weights are not optimal. > > If we're running enough threads that we need multiple sockets to run > them concurrently, then the memory distribution weights become much more > complex. Without more precise control over task placement and > preventing task migration, you can't really get an "optimal" placement. > > What I'm really saying is "Task placement is a more powerful function > for predicting performance than memory placement". However, user > software would need to implement a pseudo-scheduler and explicit data > placement to be the most optimized. Beyond this, there is only so much > we can do from a `numactl` perspective. > > tl;dr: We can't get a perfect system here, because getting a best-case > for all possible scenarios is an probably undecidable problem. You will > always be able to generate an example wherein the system is not optimal. > >> >> If the workload run on CPU of node 0 and node 1, then the cross-socket >> traffic should be minimized if possible. That is, threads/processes on >> node 0 should interleave memory of node 0 and node 2, while that on node >> 1 should interleave memory of node 1 and node 3. > > This can be done with set_mempolicy() with MPOL_INTERLEAVE and set the > nodemask to the what you describe. Those tasks need to also prevent > themselves from being migrated as well. But this can absolutely be > done. > > In this scenario, the weights need to be re-calculated to be based on > the bandwidth of the nodes in the mempolicy nodemask, which is what i > described in the last email. IMHO, we should keep thing as simple as possible, only add complexity if necessary. -- Best Regards, Huang, Ying
On Mon, Oct 23, 2023 at 10:09:56AM +0800, Huang, Ying wrote: > Gregory Price <gregory.price@memverge.com> writes: > > > Depends. if a user explicitly launches with `numactl --cpunodebind=0` > > then yes, you can force a task (and all its children) to run on node0. > > IIUC, in your example, the `numactl` command line will be > > numactl --cpunodebind=0 --weighted-interleave=0,1,2,3 > > That is, the CPU is restricted to node 0, while memory is distributed to > all nodes. This doesn't sound like reasonable for me. > It being reasonable isn't really relevant. You can do this today with normal interleave: numactl --cpunodebind=0 --interleave=0,1,2,3 The only difference between this method and that is the application of weights. Doesn't seem reasonable to lock users out of doing it. > > IMHO, we should keep thing as simple as possible, only add complexity if > necessary. > Not allowing it is more complicated than allowing it.
Gregory Price <gregory.price@memverge.com> writes: > On Mon, Oct 23, 2023 at 10:09:56AM +0800, Huang, Ying wrote: >> Gregory Price <gregory.price@memverge.com> writes: >> >> > Depends. if a user explicitly launches with `numactl --cpunodebind=0` >> > then yes, you can force a task (and all its children) to run on node0. >> >> IIUC, in your example, the `numactl` command line will be >> >> numactl --cpunodebind=0 --weighted-interleave=0,1,2,3 >> >> That is, the CPU is restricted to node 0, while memory is distributed to >> all nodes. This doesn't sound like reasonable for me. >> > > It being reasonable isn't really relevant. You can do this today with > normal interleave: > > numactl --cpunodebind=0 --interleave=0,1,2,3 > > The only difference between this method and that is the application of > weights. Doesn't seem reasonable to lock users out of doing it. Do you have some real use case? -- Best Regards, Huang, Ying
On Wed, Oct 25, 2023 at 09:13:01AM +0800, Huang, Ying wrote: > Gregory Price <gregory.price@memverge.com> writes: > > > On Mon, Oct 23, 2023 at 10:09:56AM +0800, Huang, Ying wrote: > >> Gregory Price <gregory.price@memverge.com> writes: > >> > >> > Depends. if a user explicitly launches with `numactl --cpunodebind=0` > >> > then yes, you can force a task (and all its children) to run on node0. > >> > >> IIUC, in your example, the `numactl` command line will be > >> > >> numactl --cpunodebind=0 --weighted-interleave=0,1,2,3 > >> > >> That is, the CPU is restricted to node 0, while memory is distributed to > >> all nodes. This doesn't sound like reasonable for me. > >> > > > > It being reasonable isn't really relevant. You can do this today with > > normal interleave: > > > > numactl --cpunodebind=0 --interleave=0,1,2,3 > > > > The only difference between this method and that is the application of > > weights. Doesn't seem reasonable to lock users out of doing it. > > Do you have some real use case? > I don't, but this is how mempolicy and numactl presently work. You can do this today with the current kernel. I'm simply extending it to include weights. ~Gregory
Gregory Price <gregory.price@memverge.com> writes: > On Wed, Oct 25, 2023 at 09:13:01AM +0800, Huang, Ying wrote: >> Gregory Price <gregory.price@memverge.com> writes: >> >> > On Mon, Oct 23, 2023 at 10:09:56AM +0800, Huang, Ying wrote: >> >> Gregory Price <gregory.price@memverge.com> writes: >> >> >> >> > Depends. if a user explicitly launches with `numactl --cpunodebind=0` >> >> > then yes, you can force a task (and all its children) to run on node0. >> >> >> >> IIUC, in your example, the `numactl` command line will be >> >> >> >> numactl --cpunodebind=0 --weighted-interleave=0,1,2,3 >> >> >> >> That is, the CPU is restricted to node 0, while memory is distributed to >> >> all nodes. This doesn't sound like reasonable for me. >> >> >> > >> > It being reasonable isn't really relevant. You can do this today with >> > normal interleave: >> > >> > numactl --cpunodebind=0 --interleave=0,1,2,3 >> > >> > The only difference between this method and that is the application of >> > weights. Doesn't seem reasonable to lock users out of doing it. >> >> Do you have some real use case? >> > > I don't, but this is how mempolicy and numactl presently work. You can > do this today with the current kernel. I'm simply extending it to > include weights. The extending adds complexity to the kernel code and changes the kernel ABI. So, IMHO, we need some real life use case to prove the added complexity is necessary. For example, in [1], Johannes showed the use case to support to add per-memory-tier interleave weight. [1] https://lore.kernel.org/all/20220607171949.85796-1-hannes@cmpxchg.org/ -- Best Regards, Huang, Ying
On Mon, Oct 30, 2023 at 10:20:14AM +0800, Huang, Ying wrote: > Gregory Price <gregory.price@memverge.com> writes: > > The extending adds complexity to the kernel code and changes the kernel > ABI. So, IMHO, we need some real life use case to prove the added > complexity is necessary. > > For example, in [1], Johannes showed the use case to support to add > per-memory-tier interleave weight. > > [1] https://lore.kernel.org/all/20220607171949.85796-1-hannes@cmpxchg.org/ > > -- > Best Regards, > Huang, Ying Sorry, I misunderstood your question. The use case is the same as the N:M interleave strategy between tiers, and in fact the proposal for weights was directly inspired by the patch you posted. We're searching for the best way to implement weights. We've discussed placing these weights in: 1) mempolicy : https://lore.kernel.org/linux-cxl/20230914235457.482710-1-gregory.price@memverge.com/ 2) tiers https://lore.kernel.org/linux-cxl/20231009204259.875232-1-gregory.price@memverge.com/ and now 3) the nodes themselves RFC not posted yet The use case is the exact same as the patch you posted, which is to enable optimal distribution of memory to maximize memory bandwidth usage. The use case is straight forward - Consider a machine with the following numa nodes: 1) Socket 0 - DRAM - ~400GB/s bandwidth local, less cross-socket 2) Socket 1 - DRAM - ~400GB/s bandwidth local, less cross socket 3) CXL Memory Attached to Socket 0 with ~64GB/s per link. 4) CXL Memory Attached to Socket 1 with ~64GB/s per link. The goal is to enable mempolicy to implement weighted interleave such that a thread running on socket 0 can effectively spread its memory across each numa node (or some subset there-of) such that it maximizes its bandwidth usage across the various devices. For example, lets consider a system with only 1 & 2 (2 sockets w/ DRAM). On an Intel System with UPI, the "effective" bandwidth available for a task on Socket 0 is not 800GB/s, it's about 450-500GB/s split about 300/200 between the sockets (you never get the full amount, and UPI limits cross-socket bandwidth). Today `numactl --interleave` will split your memory 50:50 between sockets, which is just blatantly suboptimal. In this case you would prefer a 3:2 distribution (literally weights of 3 and 2 respectively). The extension to CXL becomes obvious then, as each individual node, respective to its CPU placement, has a different optimal weight. Of course the question becomes "what if a task uses more threads than a single socket has to offer", and the answer there is essentially the same as the answer today: Then that process must become "numa-aware" to make the best use of the available resources. However, for software capable of exhausting bandwidth with from a single socket (which on intel takes about 16-20 threads with certain access patterns), then a weighted-interleave system provided via some interface like `numactl --weighted-interleave` with weights either set in numa nodes or mempolicy is sufficient. ~Gregory
Gregory Price <gregory.price@memverge.com> writes: > On Mon, Oct 30, 2023 at 10:20:14AM +0800, Huang, Ying wrote: >> Gregory Price <gregory.price@memverge.com> writes: >> >> The extending adds complexity to the kernel code and changes the kernel >> ABI. So, IMHO, we need some real life use case to prove the added >> complexity is necessary. >> >> For example, in [1], Johannes showed the use case to support to add >> per-memory-tier interleave weight. >> >> [1] https://lore.kernel.org/all/20220607171949.85796-1-hannes@cmpxchg.org/ >> >> -- >> Best Regards, >> Huang, Ying > > Sorry, I misunderstood your question. > > The use case is the same as the N:M interleave strategy between tiers, > and in fact the proposal for weights was directly inspired by the patch > you posted. We're searching for the best way to implement weights. > > We've discussed placing these weights in: > > 1) mempolicy : > https://lore.kernel.org/linux-cxl/20230914235457.482710-1-gregory.price@memverge.com/ > > 2) tiers > https://lore.kernel.org/linux-cxl/20231009204259.875232-1-gregory.price@memverge.com/ > > and now > 3) the nodes themselves > RFC not posted yet > > The use case is the exact same as the patch you posted, which is to enable > optimal distribution of memory to maximize memory bandwidth usage. > > The use case is straight forward - Consider a machine with the following > numa nodes: > > 1) Socket 0 - DRAM - ~400GB/s bandwidth local, less cross-socket > 2) Socket 1 - DRAM - ~400GB/s bandwidth local, less cross socket > 3) CXL Memory Attached to Socket 0 with ~64GB/s per link. > 4) CXL Memory Attached to Socket 1 with ~64GB/s per link. > > The goal is to enable mempolicy to implement weighted interleave such > that a thread running on socket 0 can effectively spread its memory > across each numa node (or some subset there-of) such that it maximizes > its bandwidth usage across the various devices. > > For example, lets consider a system with only 1 & 2 (2 sockets w/ DRAM). > > On an Intel System with UPI, the "effective" bandwidth available for a > task on Socket 0 is not 800GB/s, it's about 450-500GB/s split about > 300/200 between the sockets (you never get the full amount, and UPI limits > cross-socket bandwidth). > > Today `numactl --interleave` will split your memory 50:50 between > sockets, which is just blatantly suboptimal. In this case you would > prefer a 3:2 distribution (literally weights of 3 and 2 respectively). > > The extension to CXL becomes obvious then, as each individual node, > respective to its CPU placement, has a different optimal weight. > > > Of course the question becomes "what if a task uses more threads than a > single socket has to offer", and the answer there is essentially the > same as the answer today: Then that process must become "numa-aware" to > make the best use of the available resources. > > However, for software capable of exhausting bandwidth with from a single > socket (which on intel takes about 16-20 threads with certain access > patterns), then a weighted-interleave system provided via some interface > like `numactl --weighted-interleave` with weights either set in numa > nodes or mempolicy is sufficient. I think that these are all possible in theory. Thanks for detailed explanation! Now the question is whether these issues are relevant in practice. Whether are all workloads with the extreme high memory bandwidth requirement NUMA-aware? Or multi-process instead of multi-thread? Whether is the cross-socket traffic avoided as much as possible in practice? I have no answer to these questions. Do you have? Or someone else can answer them? -- Best Regards, Huang, Ying
v2: change memtier mutex to semaphore add source-node relative weighting add remaining mempolicy integration code = v2 Notes Developed in colaboration with original authors to deconflict similar efforts to extend mempolicy to take weights directly. == Mutex to Semaphore change: The memory tiering subsystem is extended in this patch set to have externally available information (weights), and therefore additional controls need to be added to ensure values are not changed (or tiers changed/added/removed) during various calculations. Since it is expected that many threads will be accessing this data during allocations, a mutex is not appropriate. Since write-updates (weight changes, hotplug events) are rare events, a simple rw semaphore is sufficient. == Source-node relative weighting: Tiers can now be weighted differently based on the node requesting the weight. For example CPU-Nodes 0 and 1 may have different weights for the same CXL memory tier, because topologically the number of NUMA hops is greater (or any other physical topological difference resulting in different effective latency or bandwidth values) 1. Set weights for DDR (tier4) and CXL(teir22) tiers. echo source_node:weight > /path/to/interleave_weight # Set tier4 weight from node 0 to 85 echo 0:85 > /sys/devices/virtual/memory_tiering/memory_tier4/interleave_weight # Set tier4 weight from node 1 to 65 echo 1:65 > /sys/devices/virtual/memory_tiering/memory_tier4/interleave_weight # Set tier22 weight from node 0 to 15 echo 0:15 > /sys/devices/virtual/memory_tiering/memory_tier22/interleave_weight # Set tier22 weight from node 1 to 10 echo 1:10 > /sys/devices/virtual/memory_tiering/memory_tier22/interleave_weight == Mempolicy integration Two new functions have been added to memory-tiers.c * memtier_get_node_weight - Get the effective weight for a given node * memtier_get_total_weight - Get the "total effective weight" for a given nodemask. These functions are used by the following functions in mempolicy: * interleave_nodes * offset_il_nodes * alloc_pages_bulk_array_interleave The weight values are used to determine how many pages should be allocated per-node as interleave rounds occur. To avoid holding the memtier semaphore for long periods of time (e.g. during the calls that actually allocate pages), there is a small race condition during bulk allocation between calculating the total weight of a node mask and fetching each individual node weight - but this is managed by simply detecting the over/under allocation conditions and handling them accordingly. ~Gregory === original RFC ==== From: Ravi Shankar <ravis.opensrc@micron.com> Hello, The current interleave policy operates by interleaving page requests among nodes defined in the memory policy. To accommodate the introduction of memory tiers for various memory types (e.g., DDR, CXL, HBM, PMEM, etc.), a mechanism is needed for interleaving page requests across these memory types or tiers. This can be achieved by implementing an interleaving method that considers the tier weights. The tier weight will determine the proportion of nodes to select from those specified in the memory policy. A tier weight can be assigned to each memory type within the system. Hasan Al Maruf had put forth a proposal for interleaving between two tiers, namely the top tier and the low tier. However, this patch was not adopted due to constraints on the number of available tiers. https://lore.kernel.org/linux-mm/YqD0%2FtzFwXvJ1gK6@cmpxchg.org/T/ New proposed changes: 1. Introducea sysfs entry to allow setting the interleave weight for each memory tier. 2. Each tier with a default weight of 1, indicating a standard 1:1 proportion. 3. Distribute the weight of that tier in a uniform manner across all nodes. 4. Modifications to the existing interleaving algorithm to support the implementation of multi-tier interleaving based on tier-weights. This is inline with Huang, Ying's presentation in lpc22, 16th slide in https://lpc.events/event/16/contributions/1209/attachments/1042/1995/\ Live%20In%20a%20World%20With%20Multiple%20Memory%20Types.pdf Observed a significant increase (165%) in bandwidth utilization with the newly proposed multi-tier interleaving compared to the traditional 1:1 interleaving approach between DDR and CXL tier nodes, where 85% of the bandwidth is allocated to DDR tier and 15% to CXL tier with MLC -w2 option. Usage Example: 1. Set weights for DDR (tier4) and CXL(teir22) tiers. echo 85 > /sys/devices/virtual/memory_tiering/memory_tier4/interleave_weight echo 15 > /sys/devices/virtual/memory_tiering/memory_tier22/interleave_weight 2. Interleave between DRR(tier4, node-0) and CXL (tier22, node-1) using numactl numactl -i0,1 mlc --loaded_latency W2 Gregory Price (3): mm/memory-tiers: change mutex to rw semaphore mm/memory-tiers: Introduce sysfs for tier interleave weights mm/mempolicy: modify interleave mempolicy to use memtier weights include/linux/memory-tiers.h | 16 ++++ include/linux/mempolicy.h | 3 + mm/memory-tiers.c | 179 +++++++++++++++++++++++++++++++---- mm/mempolicy.c | 148 +++++++++++++++++++++++------ 4 files changed, 297 insertions(+), 49 deletions(-)