| Message ID | 20240912093051.452172-1-arikalo@gmail.com (mailing list archive) |
|---|---|
| Headers | show |
| Series | MIPS: Support I6500 multi-cluster configuration | expand |
Hello Aleksandar, > Taken from Paul Burton MIPS repo with minor changes from Chao-ying Fu. > Tested with 64r6el_defconfig on Boston board in 2 cluster/2 VPU and > 1 cluster/4 VPU configurations. With all three patches applied, I was able to get my second CPU cluster working. If you re-send the series with these three patches included, I'd be happy to add my Tested-by tag. Here are some relevant extracts from the boot log: Linux version 6.12.0-rc2-00015-g44e960e85e4b (gclement@BLaptop) (mips-img-linux-gnu-gcc (Codescape GNU Tools 2021.09-01 for MIPS IMG Linux) 11.2.0, GNU ld (Codescape GNU Tools 2021.09-01 for MIPS IMG Linux) 2.31.1) #368 SMP Fri Oct 18 14:53:35 CEST 2024 [...] VP topology {4,4,4,4},{4,4,4,4} total 32 [...] HCI (Hardware Cache Init for the L2 cache) in GCR_L2_RAM_CONFIG from the CM3 is broken MMID allocator initialised with 65536 entries rcu: Hierarchical SRCU implementation. rcu: Max phase no-delay instances is 1000. Timer migration: 2 hierarchy levels; 8 children per group; 2 crossnode level smp: Bringing up secondary CPUs ... Primary instruction cache 64kB, VIPT, 4-way, linesize 64 bytes. Primary data cache 32kB, 4-way, PIPT, no aliases, linesize 64 bytes MIPS secondary cache 1024kB, 16-way, linesize 64 bytes. CPU1 revision is: 0001b031 (MIPS I6500) FPU revision is: 20f30320 MSA revision is: 00000320 Counter synchronization [CPU#0 -> CPU#1]: Measured 24 cycles counter warp between CPUs Primary instruction cache 64kB, VIPT, 4-way, linesize 64 bytes. Primary data cache 32kB, 4-way, PIPT, no aliases, linesize 64 bytes MIPS secondary cache 1024kB, 16-way, linesize 64 bytes. CPU2 revision is: 0001b031 (MIPS I6500) FPU revision is: 20f30320 MSA revision is: 00000320 Counter synchronization [CPU#0 -> CPU#2]: passed Primary instruction cache 64kB, VIPT, 4-way, linesize 64 bytes. Primary data cache 32kB, 4-way, PIPT, no aliases, linesize 64 bytes MIPS secondary cache 1024kB, 16-way, linesize 64 bytes. CPU3 revision is: 0001b031 (MIPS I6500) FPU revision is: 20f30320 MSA revision is: 00000320 Counter synchronization [CPU#0 -> CPU#3]: Measured 2 cycles counter warp between CPUs Primary instruction cache 64kB, VIPT, 4-way, linesize 64 bytes. Primary data cache 32kB, 4-way, PIPT, no aliases, linesize 64 bytes MIPS secondary cache 1024kB, 16-way, linesize 64 bytes. CPU4 revision is: 0001b031 (MIPS I6500) FPU revision is: 20f30320 MSA revision is: 00000320 Counter synchronization [CPU#0 -> CPU#4]: passed Primary instruction cache 64kB, VIPT, 4-way, linesize 64 bytes. Primary data cache 32kB, 4-way, PIPT, no aliases, linesize 64 bytes MIPS secondary cache 1024kB, 16-way, linesize 64 bytes. CPU5 revision is: 0001b031 (MIPS I6500) FPU revision is: 20f30320 MSA revision is: 00000320 Counter synchronization [CPU#0 -> CPU#5]: passed Primary instruction cache 64kB, VIPT, 4-way, linesize 64 bytes. Primary data cache 32kB, 4-way, PIPT, no aliases, linesize 64 bytes MIPS secondary cache 1024kB, 16-way, linesize 64 bytes. CPU6 revision is: 0001b031 (MIPS I6500) FPU revision is: 20f30320 MSA revision is: 00000320 Counter synchronization [CPU#0 -> CPU#6]: Measured 2 cycles counter warp between CPUs Primary instruction cache 64kB, VIPT, 4-way, linesize 64 bytes. Primary data cache 32kB, 4-way, PIPT, no aliases, linesize 64 bytes MIPS secondary cache 1024kB, 16-way, linesize 64 bytes. CPU7 revision is: 0001b031 (MIPS I6500) FPU revision is: 20f30320 MSA revision is: 00000320 Counter synchronization [CPU#0 -> CPU#7]: Measured 16 cycles counter warp between CPUs Primary instruction cache 64kB, VIPT, 4-way, linesize 64 bytes. Primary data cache 32kB, 4-way, PIPT, no aliases, linesize 64 bytes MIPS secondary cache 1024kB, 16-way, linesize 64 bytes. CPU8 revision is: 0001b031 (MIPS I6500) FPU revision is: 20f30320 MSA revision is: 00000320 Counter synchronization [CPU#0 -> CPU#8]: Measured 4 cycles counter warp between CPUs Primary instruction cache 64kB, VIPT, 4-way, linesize 64 bytes. Primary data cache 32kB, 4-way, PIPT, no aliases, linesize 64 bytes MIPS secondary cache 1024kB, 16-way, linesize 64 bytes. CPU9 revision is: 0001b031 (MIPS I6500) FPU revision is: 20f30320 MSA revision is: 00000320 Counter synchronization [CPU#0 -> CPU#9]: passed Primary instruction cache 64kB, VIPT, 4-way, linesize 64 bytes. Primary data cache 32kB, 4-way, PIPT, no aliases, linesize 64 bytes MIPS secondary cache 1024kB, 16-way, linesize 64 bytes. CPU10 revision is: 0001b031 (MIPS I6500) FPU revision is: 20f30320 MSA revision is: 00000320 Counter synchronization [CPU#0 -> CPU#10]: passed Primary instruction cache 64kB, VIPT, 4-way, linesize 64 bytes. Primary data cache 32kB, 4-way, PIPT, no aliases, linesize 64 bytes MIPS secondary cache 1024kB, 16-way, linesize 64 bytes. CPU11 revision is: 0001b031 (MIPS I6500) FPU revision is: 20f30320 MSA revision is: 00000320 Counter synchronization [CPU#0 -> CPU#11]: passed Primary instruction cache 64kB, VIPT, 4-way, linesize 64 bytes. Primary data cache 32kB, 4-way, PIPT, no aliases, linesize 64 bytes MIPS secondary cache 1024kB, 16-way, linesize 64 bytes. CPU12 revision is: 0001b031 (MIPS I6500) FPU revision is: 20f30320 MSA revision is: 00000320 Counter synchronization [CPU#0 -> CPU#12]: passed Primary instruction cache 64kB, VIPT, 4-way, linesize 64 bytes. Primary data cache 32kB, 4-way, PIPT, no aliases, linesize 64 bytes MIPS secondary cache 1024kB, 16-way, linesize 64 bytes. CPU13 revision is: 0001b031 (MIPS I6500) FPU revision is: 20f30320 MSA revision is: 00000320 Counter synchronization [CPU#0 -> CPU#13]: passed Primary instruction cache 64kB, VIPT, 4-way, linesize 64 bytes. Primary data cache 32kB, 4-way, PIPT, no aliases, linesize 64 bytes MIPS secondary cache 1024kB, 16-way, linesize 64 bytes. CPU14 revision is: 0001b031 (MIPS I6500) FPU revision is: 20f30320 MSA revision is: 00000320 Counter synchronization [CPU#0 -> CPU#14]: passed Primary instruction cache 64kB, VIPT, 4-way, linesize 64 bytes. Primary data cache 32kB, 4-way, PIPT, no aliases, linesize 64 bytes MIPS secondary cache 1024kB, 16-way, linesize 64 bytes. CPU15 revision is: 0001b031 (MIPS I6500) FPU revision is: 20f30320 MSA revision is: 00000320 Counter synchronization [CPU#0 -> CPU#15]: Measured 2 cycles counter warp between CPUs Primary instruction cache 64kB, VIPT, 4-way, linesize 64 bytes. Primary data cache 32kB, 4-way, PIPT, no aliases, linesize 64 bytes MIPS secondary cache 1024kB, 16-way, linesize 64 bytes. CPU16 revision is: 0001b031 (MIPS I6500) FPU revision is: 20f30320 MSA revision is: 00000320 Counter synchronization [CPU#0 -> CPU#16]: passed Primary instruction cache 64kB, VIPT, 4-way, linesize 64 bytes. Primary data cache 32kB, 4-way, PIPT, no aliases, linesize 64 bytes MIPS secondary cache 1024kB, 16-way, linesize 64 bytes. CPU17 revision is: 0001b031 (MIPS I6500) FPU revision is: 20f30320 MSA revision is: 00000320 Counter synchronization [CPU#0 -> CPU#17]: passed Primary instruction cache 64kB, VIPT, 4-way, linesize 64 bytes. Primary data cache 32kB, 4-way, PIPT, no aliases, linesize 64 bytes MIPS secondary cache 1024kB, 16-way, linesize 64 bytes. CPU18 revision is: 0001b031 (MIPS I6500) FPU revision is: 20f30320 MSA revision is: 00000320 Counter synchronization [CPU#0 -> CPU#18]: passed Primary instruction cache 64kB, VIPT, 4-way, linesize 64 bytes. Primary data cache 32kB, 4-way, PIPT, no aliases, linesize 64 bytes MIPS secondary cache 1024kB, 16-way, linesize 64 bytes. CPU19 revision is: 0001b031 (MIPS I6500) FPU revision is: 20f30320 MSA revision is: 00000320 Counter synchronization [CPU#0 -> CPU#19]: passed Primary instruction cache 64kB, VIPT, 4-way, linesize 64 bytes. Primary data cache 32kB, 4-way, PIPT, no aliases, linesize 64 bytes MIPS secondary cache 1024kB, 16-way, linesize 64 bytes. CPU20 revision is: 0001b031 (MIPS I6500) FPU revision is: 20f30320 MSA revision is: 00000320 Counter synchronization [CPU#0 -> CPU#20]: passed Primary instruction cache 64kB, VIPT, 4-way, linesize 64 bytes. Primary data cache 32kB, 4-way, PIPT, no aliases, linesize 64 bytes MIPS secondary cache 1024kB, 16-way, linesize 64 bytes. CPU21 revision is: 0001b031 (MIPS I6500) FPU revision is: 20f30320 MSA revision is: 00000320 Counter synchronization [CPU#0 -> CPU#21]: passed Primary instruction cache 64kB, VIPT, 4-way, linesize 64 bytes. Primary data cache 32kB, 4-way, PIPT, no aliases, linesize 64 bytes MIPS secondary cache 1024kB, 16-way, linesize 64 bytes. CPU22 revision is: 0001b031 (MIPS I6500) FPU revision is: 20f30320 MSA revision is: 00000320 Counter synchronization [CPU#0 -> CPU#22]: passed Primary instruction cache 64kB, VIPT, 4-way, linesize 64 bytes. Primary data cache 32kB, 4-way, PIPT, no aliases, linesize 64 bytes MIPS secondary cache 1024kB, 16-way, linesize 64 bytes. CPU23 revision is: 0001b031 (MIPS I6500) FPU revision is: 20f30320 MSA revision is: 00000320 Counter synchronization [CPU#0 -> CPU#23]: passed Primary instruction cache 64kB, VIPT, 4-way, linesize 64 bytes. Primary data cache 32kB, 4-way, PIPT, no aliases, linesize 64 bytes MIPS secondary cache 1024kB, 16-way, linesize 64 bytes. CPU24 revision is: 0001b031 (MIPS I6500) FPU revision is: 20f30320 MSA revision is: 00000320 Counter synchronization [CPU#0 -> CPU#24]: passed Primary instruction cache 64kB, VIPT, 4-way, linesize 64 bytes. Primary data cache 32kB, 4-way, PIPT, no aliases, linesize 64 bytes MIPS secondary cache 1024kB, 16-way, linesize 64 bytes. CPU25 revision is: 0001b031 (MIPS I6500) FPU revision is: 20f30320 MSA revision is: 00000320 Counter synchronization [CPU#0 -> CPU#25]: passed Primary instruction cache 64kB, VIPT, 4-way, linesize 64 bytes. Primary data cache 32kB, 4-way, PIPT, no aliases, linesize 64 bytes MIPS secondary cache 1024kB, 16-way, linesize 64 bytes. CPU26 revision is: 0001b031 (MIPS I6500) FPU revision is: 20f30320 MSA revision is: 00000320 Counter synchronization [CPU#0 -> CPU#26]: passed Primary instruction cache 64kB, VIPT, 4-way, linesize 64 bytes. Primary data cache 32kB, 4-way, PIPT, no aliases, linesize 64 bytes MIPS secondary cache 1024kB, 16-way, linesize 64 bytes. CPU27 revision is: 0001b031 (MIPS I6500) FPU revision is: 20f30320 MSA revision is: 00000320 Counter synchronization [CPU#0 -> CPU#27]: passed Primary instruction cache 64kB, VIPT, 4-way, linesize 64 bytes. Primary data cache 32kB, 4-way, PIPT, no aliases, linesize 64 bytes MIPS secondary cache 1024kB, 16-way, linesize 64 bytes. CPU28 revision is: 0001b031 (MIPS I6500) FPU revision is: 20f30320 MSA revision is: 00000320 Counter synchronization [CPU#0 -> CPU#28]: passed Primary instruction cache 64kB, VIPT, 4-way, linesize 64 bytes. Primary data cache 32kB, 4-way, PIPT, no aliases, linesize 64 bytes MIPS secondary cache 1024kB, 16-way, linesize 64 bytes. CPU29 revision is: 0001b031 (MIPS I6500) FPU revision is: 20f30320 MSA revision is: 00000320 Counter synchronization [CPU#0 -> CPU#29]: passed Primary instruction cache 64kB, VIPT, 4-way, linesize 64 bytes. Primary data cache 32kB, 4-way, PIPT, no aliases, linesize 64 bytes MIPS secondary cache 1024kB, 16-way, linesize 64 bytes. CPU30 revision is: 0001b031 (MIPS I6500) FPU revision is: 20f30320 MSA revision is: 00000320 Counter synchronization [CPU#0 -> CPU#30]: passed Primary instruction cache 64kB, VIPT, 4-way, linesize 64 bytes. Primary data cache 32kB, 4-way, PIPT, no aliases, linesize 64 bytes MIPS secondary cache 1024kB, 16-way, linesize 64 bytes. CPU31 revision is: 0001b031 (MIPS I6500) FPU revision is: 20f30320 MSA revision is: 00000320 Counter synchronization [CPU#0 -> CPU#31]: passed smp: Brought up 1 node, 32 CPUs Regards, Gregory > > v6: > - Re-base onto the master branch, with no functionality impact. > - Correct the issue reported by the kernel test robot. > > v5: > - Drop FDC related changes (patches 12, 13, and 14). > - Apply changes suggested by Thomas Gleixner (patches 3 and 4). > - Add #include <linux/cpumask.h> to patch 1, suggested by Thomas Bogendoerfer. > - Add Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org> for the patch 08/11. > - Add Tested-by: Serge Semin <fancer.lancer@gmail.com> for the entire series. > - Correct some commit messages. > > v4: > - Re-base onto the master branch, with no functionality impact. > - Refactor MIPS FDC driver in the context of multicluster support. > > v3: > - Add Reviewed-by: Jiaxun Yang <jiaxun.yang@flygoat.com> for the patch 02/12. > - Add the changes requested by Marc Zyngier for the 3/12 patch. > - Remove the patch 11/12 (a consequence of a discussion between Jiaxun Yang > and Marc Zyngier. > - Re-base onto the master branch, with no functionality impact. > > v2: > - Apply correct Signed-off-by to avoid confusion. > > Chao-ying Fu (1): > irqchip/mips-gic: Setup defaults in each cluster > > Paul Burton (8): > irqchip/mips-gic: Introduce for_each_online_cpu_gic() > irqchip/mips-gic: Support multi-cluster in for_each_online_cpu_gic() > irqchip/mips-gic: Multi-cluster support > clocksource: mips-gic-timer: Always use cluster 0 counter as > clocksource > clocksource: mips-gic-timer: Enable counter when CPUs start > MIPS: pm-cps: Use per-CPU variables as per-CPU, not per-core > MIPS: CPS: Introduce struct cluster_boot_config > MIPS: CPS: Boot CPUs in secondary clusters > > arch/mips/include/asm/mips-cm.h | 18 ++ > arch/mips/include/asm/smp-cps.h | 7 +- > arch/mips/kernel/asm-offsets.c | 3 + > arch/mips/kernel/cps-vec.S | 19 +- > arch/mips/kernel/mips-cm.c | 4 +- > arch/mips/kernel/pm-cps.c | 35 ++-- > arch/mips/kernel/smp-cps.c | 285 ++++++++++++++++++++++----- > drivers/clocksource/mips-gic-timer.c | 45 ++++- > drivers/irqchip/Kconfig | 1 + > drivers/irqchip/irq-mips-gic.c | 257 ++++++++++++++++++++---- > 10 files changed, 560 insertions(+), 114 deletions(-) > > -- > 2.25.1
Taken from Paul Burton MIPS repo with minor changes from Chao-ying Fu. Tested with 64r6el_defconfig on Boston board in 2 cluster/2 VPU and 1 cluster/4 VPU configurations. v6: - Re-base onto the master branch, with no functionality impact. - Correct the issue reported by the kernel test robot. v5: - Drop FDC related changes (patches 12, 13, and 14). - Apply changes suggested by Thomas Gleixner (patches 3 and 4). - Add #include <linux/cpumask.h> to patch 1, suggested by Thomas Bogendoerfer. - Add Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org> for the patch 08/11. - Add Tested-by: Serge Semin <fancer.lancer@gmail.com> for the entire series. - Correct some commit messages. v4: - Re-base onto the master branch, with no functionality impact. - Refactor MIPS FDC driver in the context of multicluster support. v3: - Add Reviewed-by: Jiaxun Yang <jiaxun.yang@flygoat.com> for the patch 02/12. - Add the changes requested by Marc Zyngier for the 3/12 patch. - Remove the patch 11/12 (a consequence of a discussion between Jiaxun Yang and Marc Zyngier. - Re-base onto the master branch, with no functionality impact. v2: - Apply correct Signed-off-by to avoid confusion. Chao-ying Fu (1): irqchip/mips-gic: Setup defaults in each cluster Paul Burton (8): irqchip/mips-gic: Introduce for_each_online_cpu_gic() irqchip/mips-gic: Support multi-cluster in for_each_online_cpu_gic() irqchip/mips-gic: Multi-cluster support clocksource: mips-gic-timer: Always use cluster 0 counter as clocksource clocksource: mips-gic-timer: Enable counter when CPUs start MIPS: pm-cps: Use per-CPU variables as per-CPU, not per-core MIPS: CPS: Introduce struct cluster_boot_config MIPS: CPS: Boot CPUs in secondary clusters arch/mips/include/asm/mips-cm.h | 18 ++ arch/mips/include/asm/smp-cps.h | 7 +- arch/mips/kernel/asm-offsets.c | 3 + arch/mips/kernel/cps-vec.S | 19 +- arch/mips/kernel/mips-cm.c | 4 +- arch/mips/kernel/pm-cps.c | 35 ++-- arch/mips/kernel/smp-cps.c | 285 ++++++++++++++++++++++----- drivers/clocksource/mips-gic-timer.c | 45 ++++- drivers/irqchip/Kconfig | 1 + drivers/irqchip/irq-mips-gic.c | 257 ++++++++++++++++++++---- 10 files changed, 560 insertions(+), 114 deletions(-)