Message ID | 20250207211104.30009-3-arighi@nvidia.com (mailing list archive) |
---|---|
State | Not Applicable |
Headers | show |
Series | [1/6] mm/numa: Introduce numa_nearest_nodemask() | expand |
Context | Check | Description |
---|---|---|
netdev/tree_selection | success | Not a local patch |
On Fri, Feb 07, 2025 at 09:40:49PM +0100, Andrea Righi wrote: > +/** > + * for_each_numa_node - iterate over nodes at increasing distances from a > + * given starting node. > + * @node: the iteration variable and the starting node. > + * @unvisited: a nodemask to keep track of the unvisited nodes. > + * @state: state of NUMA nodes to iterate. > + * > + * This macro iterates over NUMA node IDs in increasing distance from the > + * starting @node and yields MAX_NUMNODES when all the nodes have been > + * visited. > + * > + * The difference between for_each_node() and for_each_numa_node() is that > + * the former allows to iterate over nodes in numerical order, whereas the > + * latter iterates over nodes in increasing order of distance. > + * > + * This complexity of this iterator is O(N^2), where N represents the > + * number of nodes, as each iteration involves scanning all nodes to > + * find the one with the shortest distance. > + * > + * Requires rcu_lock to be held. > + */ > +#define for_each_numa_node(node, unvisited, state) \ > + for (int start = (node), \ > + node = numa_nearest_nodemask((start), (state), &(unvisited)); \ > + node < MAX_NUMNODES; \ > + node_clear(node, (unvisited)), \ > + node = numa_nearest_nodemask((start), (state), &(unvisited))) > + > /** > * for_each_numa_hop_mask - iterate over cpumasks of increasing NUMA distance > * from a given node. Bikeshedding: Maybe this has already been argued back and forth but I find the distinction between for_each_node() and for_each_numa_node() way too subtle. I wouldn't suspect that they are doing different things when glancing through their usages in isolation. Can we add *something* to the name that indicates that this is iteration by distance? The next one uses "hop" which is fine, "_by_dist" can be fine too, or even "_from_nearest". I don't really care which but let's make the name clearly signal what it's doing. Thanks.
On Fri, Feb 07, 2025 at 11:46:51AM -1000, Tejun Heo wrote: > On Fri, Feb 07, 2025 at 09:40:49PM +0100, Andrea Righi wrote: > > +/** > > + * for_each_numa_node - iterate over nodes at increasing distances from a > > + * given starting node. > > + * @node: the iteration variable and the starting node. > > + * @unvisited: a nodemask to keep track of the unvisited nodes. > > + * @state: state of NUMA nodes to iterate. > > + * > > + * This macro iterates over NUMA node IDs in increasing distance from the > > + * starting @node and yields MAX_NUMNODES when all the nodes have been > > + * visited. > > + * > > + * The difference between for_each_node() and for_each_numa_node() is that > > + * the former allows to iterate over nodes in numerical order, whereas the > > + * latter iterates over nodes in increasing order of distance. > > + * > > + * This complexity of this iterator is O(N^2), where N represents the > > + * number of nodes, as each iteration involves scanning all nodes to > > + * find the one with the shortest distance. > > + * > > + * Requires rcu_lock to be held. > > + */ > > +#define for_each_numa_node(node, unvisited, state) \ > > + for (int start = (node), \ > > + node = numa_nearest_nodemask((start), (state), &(unvisited)); \ > > + node < MAX_NUMNODES; \ > > + node_clear(node, (unvisited)), \ > > + node = numa_nearest_nodemask((start), (state), &(unvisited))) > > + > > /** > > * for_each_numa_hop_mask - iterate over cpumasks of increasing NUMA distance > > * from a given node. > > Bikeshedding: Maybe this has already been argued back and forth but I find > the distinction between for_each_node() and for_each_numa_node() way too > subtle. I wouldn't suspect that they are doing different things when > glancing through their usages in isolation. Can we add *something* to the > name that indicates that this is iteration by distance? The next one uses > "hop" which is fine, "_by_dist" can be fine too, or even "_from_nearest". I > don't really care which but let's make the name clearly signal what it's > doing. > > Thanks. How about for_each_node_state_by_dist()? It's essentialy a variant of for_each_node_state(), as it also accepts a state, with the only difference that node IDs are returned in increasing distance order. -Andrea
On Fri, Feb 07, 2025 at 10:55:18PM +0100, Andrea Righi wrote: > How about for_each_node_state_by_dist()? It's essentialy a variant of > for_each_node_state(), as it also accepts a state, with the only difference > that node IDs are returned in increasing distance order. Sounds fine by me. Yury? Thanks.
On Fri, Feb 07, 2025 at 09:40:49PM +0100, Andrea Righi wrote: > Introduce the new helper for_each_numa_node() to iterate over node IDs > in order of increasing NUMA distance from a given starting node. > > This iterator is similar to for_each_numa_hop_mask(), but instead of > providing a cpumask at each iteration, it provides a node ID. > > Example usage: > > nodemask_t unvisited = NODE_MASK_ALL; > int node, start = cpu_to_node(smp_processor_id()); > > node = start; > for_each_numa_node(node, unvisited, N_ONLINE) > pr_info("node (%d, %d) -> %d\n", > start, node, node_distance(start, node)); > > On a system with equidistant nodes: > > $ numactl -H > ... > node distances: > node 0 1 2 3 > 0: 10 20 20 20 > 1: 20 10 20 20 > 2: 20 20 10 20 > 3: 20 20 20 10 > > Output of the example above (on node 0): > > [ 7.367022] node (0, 0) -> 10 > [ 7.367151] node (0, 1) -> 20 > [ 7.367186] node (0, 2) -> 20 > [ 7.367247] node (0, 3) -> 20 > > On a system with non-equidistant nodes (simulated using virtme-ng): Great to see virtme-ng maturing! > $ numactl -H > ... > node distances: > node 0 1 2 3 > 0: 10 51 31 41 > 1: 51 10 21 61 > 2: 31 21 10 11 > 3: 41 61 11 10 > > Output of the example above (on node 0): > > [ 8.953644] node (0, 0) -> 10 > [ 8.953712] node (0, 2) -> 31 > [ 8.953764] node (0, 3) -> 41 > [ 8.953817] node (0, 1) -> 51 > > Cc: Yury Norov <yury.norov@gmail.com> > Signed-off-by: Andrea Righi <arighi@nvidia.com> > --- > include/linux/topology.h | 28 ++++++++++++++++++++++++++++ > 1 file changed, 28 insertions(+) > > diff --git a/include/linux/topology.h b/include/linux/topology.h > index 52f5850730b3e..09c18ee8be0eb 100644 > --- a/include/linux/topology.h > +++ b/include/linux/topology.h > @@ -261,6 +261,34 @@ sched_numa_hop_mask(unsigned int node, unsigned int hops) > } > #endif /* CONFIG_NUMA */ > > +/** > + * for_each_numa_node - iterate over nodes at increasing distances from a > + * given starting node. Nit: in increasing distance order, starting from a given node > + * @node: the iteration variable and the starting node. > + * @unvisited: a nodemask to keep track of the unvisited nodes. > + * @state: state of NUMA nodes to iterate. > + * > + * This macro iterates over NUMA node IDs in increasing distance from the > + * starting @node and yields MAX_NUMNODES when all the nodes have been > + * visited. Please also mention that the unvisited nodemask will be empty when it finish. > + * > + * The difference between for_each_node() and for_each_numa_node() is that > + * the former allows to iterate over nodes in numerical order, whereas the > + * latter iterates over nodes in increasing order of distance. > + * > + * This complexity of this iterator is O(N^2), where N represents the > + * number of nodes, as each iteration involves scanning all nodes to > + * find the one with the shortest distance. > + * > + * Requires rcu_lock to be held. > + */ > +#define for_each_numa_node(node, unvisited, state) \ > + for (int start = (node), \ > + node = numa_nearest_nodemask((start), (state), &(unvisited)); \ > + node < MAX_NUMNODES; \ > + node_clear(node, (unvisited)), \ > + node = numa_nearest_nodemask((start), (state), &(unvisited))) > + > /** > * for_each_numa_hop_mask - iterate over cpumasks of increasing NUMA distance > * from a given node. > -- > 2.48.1
On Fri, Feb 07, 2025 at 11:56:30AM -1000, Tejun Heo wrote: > On Fri, Feb 07, 2025 at 10:55:18PM +0100, Andrea Righi wrote: > > How about for_each_node_state_by_dist()? It's essentialy a variant of > > for_each_node_state(), as it also accepts a state, with the only difference > > that node IDs are returned in increasing distance order. > > Sounds fine by me. Yury? for_each_node_numadist() maybe? Whichever you choose is good to me.
diff --git a/include/linux/topology.h b/include/linux/topology.h index 52f5850730b3e..09c18ee8be0eb 100644 --- a/include/linux/topology.h +++ b/include/linux/topology.h @@ -261,6 +261,34 @@ sched_numa_hop_mask(unsigned int node, unsigned int hops) } #endif /* CONFIG_NUMA */ +/** + * for_each_numa_node - iterate over nodes at increasing distances from a + * given starting node. + * @node: the iteration variable and the starting node. + * @unvisited: a nodemask to keep track of the unvisited nodes. + * @state: state of NUMA nodes to iterate. + * + * This macro iterates over NUMA node IDs in increasing distance from the + * starting @node and yields MAX_NUMNODES when all the nodes have been + * visited. + * + * The difference between for_each_node() and for_each_numa_node() is that + * the former allows to iterate over nodes in numerical order, whereas the + * latter iterates over nodes in increasing order of distance. + * + * This complexity of this iterator is O(N^2), where N represents the + * number of nodes, as each iteration involves scanning all nodes to + * find the one with the shortest distance. + * + * Requires rcu_lock to be held. + */ +#define for_each_numa_node(node, unvisited, state) \ + for (int start = (node), \ + node = numa_nearest_nodemask((start), (state), &(unvisited)); \ + node < MAX_NUMNODES; \ + node_clear(node, (unvisited)), \ + node = numa_nearest_nodemask((start), (state), &(unvisited))) + /** * for_each_numa_hop_mask - iterate over cpumasks of increasing NUMA distance * from a given node.
Introduce the new helper for_each_numa_node() to iterate over node IDs in order of increasing NUMA distance from a given starting node. This iterator is similar to for_each_numa_hop_mask(), but instead of providing a cpumask at each iteration, it provides a node ID. Example usage: nodemask_t unvisited = NODE_MASK_ALL; int node, start = cpu_to_node(smp_processor_id()); node = start; for_each_numa_node(node, unvisited, N_ONLINE) pr_info("node (%d, %d) -> %d\n", start, node, node_distance(start, node)); On a system with equidistant nodes: $ numactl -H ... node distances: node 0 1 2 3 0: 10 20 20 20 1: 20 10 20 20 2: 20 20 10 20 3: 20 20 20 10 Output of the example above (on node 0): [ 7.367022] node (0, 0) -> 10 [ 7.367151] node (0, 1) -> 20 [ 7.367186] node (0, 2) -> 20 [ 7.367247] node (0, 3) -> 20 On a system with non-equidistant nodes (simulated using virtme-ng): $ numactl -H ... node distances: node 0 1 2 3 0: 10 51 31 41 1: 51 10 21 61 2: 31 21 10 11 3: 41 61 11 10 Output of the example above (on node 0): [ 8.953644] node (0, 0) -> 10 [ 8.953712] node (0, 2) -> 31 [ 8.953764] node (0, 3) -> 41 [ 8.953817] node (0, 1) -> 51 Cc: Yury Norov <yury.norov@gmail.com> Signed-off-by: Andrea Righi <arighi@nvidia.com> --- include/linux/topology.h | 28 ++++++++++++++++++++++++++++ 1 file changed, 28 insertions(+)