| Message ID | 20250214194134.658939-5-arighi@nvidia.com (mailing list archive) |
|---|---|
| State | Not Applicable |
| Headers | show |
| Series | [1/8] nodemask: add nodes_copy() | expand |
| Context | Check | Description |
|---|---|---|
| netdev/tree_selection | success | Not a local patch |
On Fri, Feb 14, 2025 at 08:40:03PM +0100, Andrea Righi wrote: > Introduce the new helper for_each_node_numadist() to iterate over node > IDs in order of increasing NUMA distance from a given starting node. > > This iterator is somehow similar to for_each_numa_hop_mask(), but > instead of providing a cpumask at each iteration, it provides a node ID. > > Example usage: > > nodemask_t unvisited = NODE_MASK_ALL; > int node, start = cpu_to_node(smp_processor_id()); > > node = start; > for_each_node_numadist(node, unvisited) > pr_info("node (%d, %d) -> %d\n", > start, node, node_distance(start, node)); > > On a system with equidistant nodes: > > $ numactl -H > ... > node distances: > node 0 1 2 3 > 0: 10 20 20 20 > 1: 20 10 20 20 > 2: 20 20 10 20 > 3: 20 20 20 10 > > Output of the example above (on node 0): > > [ 7.367022] node (0, 0) -> 10 > [ 7.367151] node (0, 1) -> 20 > [ 7.367186] node (0, 2) -> 20 > [ 7.367247] node (0, 3) -> 20 > > On a system with non-equidistant nodes (simulated using virtme-ng): > > $ numactl -H > ... > node distances: > node 0 1 2 3 > 0: 10 51 31 41 > 1: 51 10 21 61 > 2: 31 21 10 11 > 3: 41 61 11 10 > > Output of the example above (on node 0): > > [ 8.953644] node (0, 0) -> 10 > [ 8.953712] node (0, 2) -> 31 > [ 8.953764] node (0, 3) -> 41 > [ 8.953817] node (0, 1) -> 51 > > Suggested-by: Yury Norov [NVIDIA] <yury.norov@gmail.com> > Signed-off-by: Andrea Righi <arighi@nvidia.com> Acked-by: Yury Norov [NVIDIA] <yury.norov@gmail.com> > --- > include/linux/topology.h | 30 ++++++++++++++++++++++++++++++ > 1 file changed, 30 insertions(+) > > diff --git a/include/linux/topology.h b/include/linux/topology.h > index 52f5850730b3e..a1815f4395ab6 100644 > --- a/include/linux/topology.h > +++ b/include/linux/topology.h > @@ -261,6 +261,36 @@ sched_numa_hop_mask(unsigned int node, unsigned int hops) > } > #endif /* CONFIG_NUMA */ > > +/** > + * for_each_node_numadist() - iterate over nodes in increasing distance > + * order, starting from a given node > + * @node: the iteration variable and the starting node. > + * @unvisited: a nodemask to keep track of the unvisited nodes. > + * > + * This macro iterates over NUMA node IDs in increasing distance from the > + * starting @node and yields MAX_NUMNODES when all the nodes have been > + * visited. > + * > + * Note that by the time the loop completes, the @unvisited nodemask will > + * be fully cleared, unless the loop exits early. > + * > + * The difference between for_each_node() and for_each_node_numadist() is > + * that the former allows to iterate over nodes in numerical order, whereas > + * the latter iterates over nodes in increasing order of distance. > + * > + * This complexity of this iterator is O(N^2), where N represents the > + * number of nodes, as each iteration involves scanning all nodes to > + * find the one with the shortest distance. > + * > + * Requires rcu_lock to be held. > + */ > +#define for_each_node_numadist(node, unvisited) \ > + for (int __start = (node), \ > + (node) = nearest_node_nodemask((__start), &(unvisited)); \ > + (node) < MAX_NUMNODES; \ > + node_clear((node), (unvisited)), \ > + (node) = nearest_node_nodemask((__start), &(unvisited))) > + > /** > * for_each_numa_hop_mask - iterate over cpumasks of increasing NUMA distance > * from a given node. > -- > 2.48.1
Hello, On Fri, Feb 14, 2025 at 04:16:53PM -0500, Yury Norov wrote: > > Suggested-by: Yury Norov [NVIDIA] <yury.norov@gmail.com> > > Signed-off-by: Andrea Righi <arighi@nvidia.com> > > Acked-by: Yury Norov [NVIDIA] <yury.norov@gmail.com> Yury, how do you want to route the topology updates? Do they usually go through tip? Thanks.
On Fri, Feb 14, 2025 at 11:29:03AM -1000, Tejun Heo wrote: > Hello, > > On Fri, Feb 14, 2025 at 04:16:53PM -0500, Yury Norov wrote: > > > Suggested-by: Yury Norov [NVIDIA] <yury.norov@gmail.com> > > > Signed-off-by: Andrea Righi <arighi@nvidia.com> > > > > Acked-by: Yury Norov [NVIDIA] <yury.norov@gmail.com> > > Yury, how do you want to route the topology updates? Do they usually go > through tip? Can you take it with the rest of the series? I can move it with my branch if you prefer.
On Fri, Feb 14, 2025 at 04:30:50PM -0500, Yury Norov wrote: > On Fri, Feb 14, 2025 at 11:29:03AM -1000, Tejun Heo wrote: > > Hello, > > > > On Fri, Feb 14, 2025 at 04:16:53PM -0500, Yury Norov wrote: > > > > Suggested-by: Yury Norov [NVIDIA] <yury.norov@gmail.com> > > > > Signed-off-by: Andrea Righi <arighi@nvidia.com> > > > > > > Acked-by: Yury Norov [NVIDIA] <yury.norov@gmail.com> > > > > Yury, how do you want to route the topology updates? Do they usually go > > through tip? > > Can you take it with the rest of the series? I can move it with my > branch if you prefer. Will route it through sched_ext. Thanks.
diff --git a/include/linux/topology.h b/include/linux/topology.h index 52f5850730b3e..a1815f4395ab6 100644 --- a/include/linux/topology.h +++ b/include/linux/topology.h @@ -261,6 +261,36 @@ sched_numa_hop_mask(unsigned int node, unsigned int hops) } #endif /* CONFIG_NUMA */ +/** + * for_each_node_numadist() - iterate over nodes in increasing distance + * order, starting from a given node + * @node: the iteration variable and the starting node. + * @unvisited: a nodemask to keep track of the unvisited nodes. + * + * This macro iterates over NUMA node IDs in increasing distance from the + * starting @node and yields MAX_NUMNODES when all the nodes have been + * visited. + * + * Note that by the time the loop completes, the @unvisited nodemask will + * be fully cleared, unless the loop exits early. + * + * The difference between for_each_node() and for_each_node_numadist() is + * that the former allows to iterate over nodes in numerical order, whereas + * the latter iterates over nodes in increasing order of distance. + * + * This complexity of this iterator is O(N^2), where N represents the + * number of nodes, as each iteration involves scanning all nodes to + * find the one with the shortest distance. + * + * Requires rcu_lock to be held. + */ +#define for_each_node_numadist(node, unvisited) \ + for (int __start = (node), \ + (node) = nearest_node_nodemask((__start), &(unvisited)); \ + (node) < MAX_NUMNODES; \ + node_clear((node), (unvisited)), \ + (node) = nearest_node_nodemask((__start), &(unvisited))) + /** * for_each_numa_hop_mask - iterate over cpumasks of increasing NUMA distance * from a given node.
Introduce the new helper for_each_node_numadist() to iterate over node IDs in order of increasing NUMA distance from a given starting node. This iterator is somehow similar to for_each_numa_hop_mask(), but instead of providing a cpumask at each iteration, it provides a node ID. Example usage: nodemask_t unvisited = NODE_MASK_ALL; int node, start = cpu_to_node(smp_processor_id()); node = start; for_each_node_numadist(node, unvisited) pr_info("node (%d, %d) -> %d\n", start, node, node_distance(start, node)); On a system with equidistant nodes: $ numactl -H ... node distances: node 0 1 2 3 0: 10 20 20 20 1: 20 10 20 20 2: 20 20 10 20 3: 20 20 20 10 Output of the example above (on node 0): [ 7.367022] node (0, 0) -> 10 [ 7.367151] node (0, 1) -> 20 [ 7.367186] node (0, 2) -> 20 [ 7.367247] node (0, 3) -> 20 On a system with non-equidistant nodes (simulated using virtme-ng): $ numactl -H ... node distances: node 0 1 2 3 0: 10 51 31 41 1: 51 10 21 61 2: 31 21 10 11 3: 41 61 11 10 Output of the example above (on node 0): [ 8.953644] node (0, 0) -> 10 [ 8.953712] node (0, 2) -> 31 [ 8.953764] node (0, 3) -> 41 [ 8.953817] node (0, 1) -> 51 Suggested-by: Yury Norov [NVIDIA] <yury.norov@gmail.com> Signed-off-by: Andrea Righi <arighi@nvidia.com> --- include/linux/topology.h | 30 ++++++++++++++++++++++++++++++ 1 file changed, 30 insertions(+)