| Message ID | 20190104230546.GA18977@p100.box (mailing list archive) |
|---|---|
| State | Accepted, archived |
| Headers | show |
| Series | parisc: Improve initial IRQ to CPU assignment | expand |
On 1/4/19 6:05 PM, Helge Deller wrote: > On parisc, each IRQ can only be handled by one CPU, and currently CPU0 > is choosen as default for handling all IRQs by default. > With this patch we now assign each requested IRQ to one of the online > CPUs (and thus distribute the IRQs across all CPUs), even without an > instance of irqbalance running. Will this take into consideration the features in default_smp_affinity and allow processor cores to be considered 'non interrupt' service state? Dennis
On 05.01.19 00:21, Dennis Clarke wrote: > On 1/4/19 6:05 PM, Helge Deller wrote: >> On parisc, each IRQ can only be handled by one CPU, and currently CPU0 >> is choosen as default for handling all IRQs by default. >> With this patch we now assign each requested IRQ to one of the online >> CPUs (and thus distribute the IRQs across all CPUs), even without an >> instance of irqbalance running. > > Will this take into consideration the features in default_smp_affinity > and allow processor cores to be considered 'non interrupt' service > state? It seems the feature of default_smp_affinity to disable specific cores from servicing interrupts doesn't work on the parisc platform. On a 2-CPU machine I booted with the "irqaffinity=0" kernel commandline option (to enable CPU#0 and disable CPU#1), which then led to a hanging kernel when CPU#1 ist started. On parisc, each CPU needs to handle at least it's timer and IPMI interrupt. Any idea what I should try? Helge
On 1/5/19 4:21 PM, Helge Deller wrote: > On 05.01.19 00:21, Dennis Clarke wrote: >> On 1/4/19 6:05 PM, Helge Deller wrote: >>> On parisc, each IRQ can only be handled by one CPU, and currently CPU0 >>> is choosen as default for handling all IRQs by default. >>> With this patch we now assign each requested IRQ to one of the online >>> CPUs (and thus distribute the IRQs across all CPUs), even without an >>> instance of irqbalance running. >> >> Will this take into consideration the features in default_smp_affinity >> and allow processor cores to be considered 'non interrupt' service >> state? > > It seems the feature of default_smp_affinity to disable specific cores from > servicing interrupts doesn't work on the parisc platform. > On a 2-CPU machine I booted with the "irqaffinity=0" kernel commandline option > (to enable CPU#0 and disable CPU#1), which then led to a hanging kernel when > CPU#1 ist started. > On parisc, each CPU needs to handle at least it's timer and IPMI interrupt. > Any idea what I should try? Sorry, the first question was "does this work there" and we now know it is most likely "no". I tried similar on ppc64 and also arrived at nothing but problems. I am somewhat focused on digging into RISC-V at the moment and your post raised the question in my mind. One of the features I once enjoyed about Solaris was that it could easily bring cpu cores online and offline as well as disable irq service disruptions on any core with a trivial one line command. I always wanted to port something similar over to linux but would start with something interesting like RISC-V. I'll get back to you on this one. Dennis Clarke ps: Also I still have those black superdomes standing around doing nothing and still want to get serial attached, powered up, etc. Yet another news years resolution post-it note idea.
On 05.01.19 22:45, Dennis Clarke wrote: > On 1/5/19 4:21 PM, Helge Deller wrote: >> On 05.01.19 00:21, Dennis Clarke wrote: >>> On 1/4/19 6:05 PM, Helge Deller wrote: >>>> On parisc, each IRQ can only be handled by one CPU, and currently CPU0 >>>> is choosen as default for handling all IRQs by default. >>>> With this patch we now assign each requested IRQ to one of the online >>>> CPUs (and thus distribute the IRQs across all CPUs), even without an >>>> instance of irqbalance running. >>> >>> Will this take into consideration the features in default_smp_affinity >>> and allow processor cores to be considered 'non interrupt' service >>> state? >> >> It seems the feature of default_smp_affinity to disable specific cores from >> servicing interrupts doesn't work on the parisc platform. >> On a 2-CPU machine I booted with the "irqaffinity=0" kernel commandline option >> (to enable CPU#0 and disable CPU#1), which then led to a hanging kernel when >> CPU#1 ist started. >> On parisc, each CPU needs to handle at least it's timer and IPMI interrupt. >> Any idea what I should try? > > Sorry, the first question was "does this work there" and we now know it > is most likely "no". I tried similar on ppc64 and also arrived at > nothing but problems. > > I am somewhat focused on digging into RISC-V at the moment and your post > raised the question in my mind. > > One of the features I once enjoyed about Solaris was that it could > easily bring cpu cores online and offline as well as disable irq service > disruptions on any core with a trivial one line command. That's already possible on Linux (e.g. on x86,power,...). > I always wanted to port something similar over to linux but would > start with something interesting like RISC-V. There shouldn't be much to port. Even on parisc, enabling and disabling irqs on specific CPUs is possible today with simple echo-commands into the files in /proc/irq/#/... (with the exception of timer and IPI irqs). Onlining and offlining cpu cores is on my todo list for parisc. So, I think it shouldn't take much to do that on RISC-V too. > ps: Also I still have those black superdomes standing around doing > nothing and still want to get serial attached, powered up, etc. > Yet another news years resolution post-it note idea. Yes, would be interesting. Helge
diff --git a/arch/parisc/kernel/irq.c b/arch/parisc/kernel/irq.c index fd6d873..04e8755 100644 --- a/arch/parisc/kernel/irq.c +++ b/arch/parisc/kernel/irq.c @@ -117,7 +117,10 @@ int cpu_check_affinity(struct irq_data *d, const struct cpumask *dest) return -EINVAL; /* whatever mask they set, we just allow one CPU */ - cpu_dest = cpumask_first_and(dest, cpu_online_mask); + cpu_dest = cpumask_next_and(d->irq & (num_online_cpus()-1), + dest, cpu_online_mask); + if (cpu_dest >= nr_cpu_ids) + cpu_dest = cpumask_first_and(dest, cpu_online_mask); return cpu_dest; }
On parisc, each IRQ can only be handled by one CPU, and currently CPU0 is choosen as default for handling all IRQs by default. With this patch we now assign each requested IRQ to one of the online CPUs (and thus distribute the IRQs across all CPUs), even without an instance of irqbalance running. Signed-off-by: Helge Deller <deller@gmx.de>