| Message ID | 20210618123851.1344518-1-anup.patel@wdc.com (mailing list archive) |
|---|---|
| Headers | show |
| Series | Linux RISC-V ACLINT Support | expand |
Hi Marc, I have taken the approach of IPI domains (like you suggested) in this series. What do you think ? Regards, Anup On Fri, Jun 18, 2021 at 6:09 PM Anup Patel <anup.patel@wdc.com> wrote: > > Most of the existing RISC-V platforms use SiFive CLINT to provide M-level > timer and IPI support whereas S-level uses SBI calls for timer and IPI > support. Also, the SiFive CLINT device is a single device providing both > timer and IPI functionality so RISC-V platforms can't partially implement > SiFive CLINT device and provide alternate mechanism for timer and IPI. > > The RISC-V Advacned Core Local Interruptor (ACLINT) tries to address the > limitations of SiFive CLINT by: > 1) Taking modular approach and defining timer and IPI functionality as > separate devices so that RISC-V platforms can include only required > devices > 2) Providing dedicated MMIO device for S-level IPIs so that SBI calls > can be avoided for IPIs in Linux RISC-V > 3) Allowing multiple instances of timer and IPI devices for a > multi-socket (or multi-die) NUMA systems > 4) Being backward compatible to SiFive CLINT so that existing RISC-V > platforms stay compliant with RISC-V ACLINT specification > > Latest RISC-V ACLINT specification (will be frozen in a month) can be > found at: > https://github.com/riscv/riscv-aclint/blob/main/riscv-aclint.adoc > > This series adds RISC-V ACLINT support and can be found in riscv_aclint_v2 > branch at: > https://github.com/avpatel/linux > > To test this series, the RISC-V ACLINT support for QEMU and OpenSBI > can be found in the riscv_aclint_v1 branch at: > https://github.com/avpatel/qemu > https://github.com/avpatel/opensbi > > Changes since v1: > - Added a new PATCH3 to treat IPIs as normal Linux IRQs for RISC-V kernel > - New SBI IPI call based irqchip driver in PATCH3 which is only initialized > by riscv_ipi_setup() when no Linux IRQ numbers are available for IPIs > - Moved DT bindings patches before corresponding driver patches > - Implemented ACLINT SWI driver as a irqchip driver in PATCH7 > - Minor nit fixes pointed by Bin Meng > > Anup Patel (11): > RISC-V: Clear SIP bit only when using SBI IPI operations > RISC-V: Use common print prefix in smp.c > RISC-V: Treat IPIs as normal Linux IRQs > RISC-V: Allow marking IPIs as suitable for remote FENCEs > RISC-V: Use IPIs for remote TLB flush when possible > dt-bindings: interrupt-controller: Add ACLINT MSWI and SSWI bindings > irqchip: Add ACLINT software interrupt driver > RISC-V: Select ACLINT SWI driver for virt machine > dt-bindings: timer: Add ACLINT MTIMER bindings > clocksource: clint: Add support for ACLINT MTIMER device > MAINTAINERS: Add entry for RISC-V ACLINT drivers > > .../riscv,aclint-swi.yaml | 82 ++++++ > .../bindings/timer/riscv,aclint-mtimer.yaml | 55 ++++ > MAINTAINERS | 9 + > arch/riscv/Kconfig | 1 + > arch/riscv/Kconfig.socs | 1 + > arch/riscv/include/asm/sbi.h | 2 + > arch/riscv/include/asm/smp.h | 48 +++- > arch/riscv/kernel/Makefile | 1 + > arch/riscv/kernel/cpu-hotplug.c | 2 + > arch/riscv/kernel/irq.c | 1 + > arch/riscv/kernel/sbi-ipi.c | 223 ++++++++++++++ > arch/riscv/kernel/sbi.c | 15 - > arch/riscv/kernel/smp.c | 171 +++++------ > arch/riscv/kernel/smpboot.c | 4 +- > arch/riscv/mm/tlbflush.c | 62 +++- > drivers/clocksource/timer-clint.c | 58 ++-- > drivers/irqchip/Kconfig | 11 + > drivers/irqchip/Makefile | 1 + > drivers/irqchip/irq-aclint-swi.c | 271 ++++++++++++++++++ > drivers/irqchip/irq-riscv-intc.c | 55 ++-- > 20 files changed, 879 insertions(+), 194 deletions(-) > create mode 100644 Documentation/devicetree/bindings/interrupt-controller/riscv,aclint-swi.yaml > create mode 100644 Documentation/devicetree/bindings/timer/riscv,aclint-mtimer.yaml > create mode 100644 arch/riscv/kernel/sbi-ipi.c > create mode 100644 drivers/irqchip/irq-aclint-swi.c > > -- > 2.25.1 >
Hi Marc, On Mon, Jul 26, 2021 at 8:02 PM Marc Zyngier <maz@kernel.org> wrote: > > On Mon, 26 Jul 2021 13:45:20 +0100, > Anup Patel <anup@brainfault.org> wrote: > > > > Hi Marc, > > > > I have taken the approach of IPI domains (like you suggested) in this series. > > > > What do you think ? > > I have commented on the irqchip driver. > > As for the RISC-V specific code, I'll let the architecture maintainers > look into it. I guess the elephant in the room is that this spec seems > to be evolving, and that there is no HW implementation (how this > driver maps on SF's CLINT is anybody's guess). The SiFive CLINT is a more convoluted device and provides M-level timer functionality and M-level IPI functionality in one MMIO device. The RISC-V ACLINT specification is more modular and backward compatible with the SiFive CLINT. In fact, a SiFive CLINT device can be viewed as a ACLINT MSWI device + ACLINT MTIMER device. This means existing RISC-V boards having SiFive CLINT will be automatically compliant to the RISC-V ACLINT specification. Here's the RISC-V ACLINT spec: https://github.com/riscv/riscv-aclint/blob/main/riscv-aclint.adoc The RISC-V ACLINT spec is quite stable and we are not seeing any further changes hence I sent out RFC PATCHes to get feedback. The RISC-V ACLINT spec will be frozen before 2021 end (i.e. before next RISC-V summit). The Linux NoMMU kernel (M-level) will use an ACLINT MSWI device for IPI support whereas the regular Linux MMU kernel (S-level) will use an ACLINT SSWI device for IPI support. The ACLINT SWI driver is a common IPI driver for both ACLINT MSWI (Linux NoMMU) and ACLINT SSWI (Linux MMU). In fact, the ACLINT SWI also works for IPI part (i.e. MSWI) of SiFive CLINT. Regards, Anup > > M. > > -- > Without deviation from the norm, progress is not possible.
On Mon, 26 Jul 2021 13:45:20 +0100, Anup Patel <anup@brainfault.org> wrote: > > Hi Marc, > > I have taken the approach of IPI domains (like you suggested) in this series. > > What do you think ? I have commented on the irqchip driver. As for the RISC-V specific code, I'll let the architecture maintainers look into it. I guess the elephant in the room is that this spec seems to be evolving, and that there is no HW implementation (how this driver maps on SF's CLINT is anybody's guess). M.
On Mon, 26 Jul 2021 06:01:01 PDT (-0700), anup@brainfault.org wrote: > Hi Marc, > > On Mon, Jul 26, 2021 at 8:02 PM Marc Zyngier <maz@kernel.org> wrote: >> >> On Mon, 26 Jul 2021 13:45:20 +0100, >> Anup Patel <anup@brainfault.org> wrote: >> > >> > Hi Marc, >> > >> > I have taken the approach of IPI domains (like you suggested) in this series. >> > >> > What do you think ? >> >> I have commented on the irqchip driver. >> >> As for the RISC-V specific code, I'll let the architecture maintainers >> look into it. I guess the elephant in the room is that this spec seems >> to be evolving, and that there is no HW implementation (how this >> driver maps on SF's CLINT is anybody's guess). There's a long history of interrupt controller efforts from the RISC-V foundation, and we've yet to have any of them result in hardware. > The SiFive CLINT is a more convoluted device and provides M-level > timer functionality and M-level IPI functionality in one MMIO device. > > The RISC-V ACLINT specification is more modular and backward > compatible with the SiFive CLINT. In fact, a SiFive CLINT device > can be viewed as a ACLINT MSWI device + ACLINT MTIMER device. > This means existing RISC-V boards having SiFive CLINT will be > automatically compliant to the RISC-V ACLINT specification. So is there any hardware that this new specification enables? It seems to be a more convoluted way to describe the same mess we're already in. I'm not really inclined to take a bunch of code that just does the same thing via a more complicated specification. > Here's the RISC-V ACLINT spec: > https://github.com/riscv/riscv-aclint/blob/main/riscv-aclint.adoc > > The RISC-V ACLINT spec is quite stable and we are not seeing any > further changes hence I sent out RFC PATCHes to get feedback. The > RISC-V ACLINT spec will be frozen before 2021 end (i.e. before next > RISC-V summit). Have you talked to the other ISA folks about that? As far as I can tell this new spec allows for multiple MTIME registers, which seems to be in direct contradiction to the single -MTIME register as defined in the ISA manual. It also seems to be vaguely incompatible WRT the definition of SSIP, but I'm not sure that one really matters all that much as it's not like old software can write the new registers. I just talked to Krste and Andrew, they say they haven't heard of any of this. I don't know what's going on over there, but it's very hard to review anything when I can't even tell where the ISA is defined. > The Linux NoMMU kernel (M-level) will use an ACLINT MSWI device > for IPI support whereas the regular Linux MMU kernel (S-level) will > use an ACLINT SSWI device for IPI support. > > The ACLINT SWI driver is a common IPI driver for both ACLINT > MSWI (Linux NoMMU) and ACLINT SSWI (Linux MMU). In fact, > the ACLINT SWI also works for IPI part (i.e. MSWI) of SiFive CLINT. > > Regards, > Anup > >> >> M. >> >> -- >> Without deviation from the norm, progress is not possible.
On Thu, Jul 29, 2021 at 10:00 AM Palmer Dabbelt <palmerdabbelt@google.com> wrote: > > On Mon, 26 Jul 2021 06:01:01 PDT (-0700), anup@brainfault.org wrote: > > Hi Marc, > > > > On Mon, Jul 26, 2021 at 8:02 PM Marc Zyngier <maz@kernel.org> wrote: > >> > >> On Mon, 26 Jul 2021 13:45:20 +0100, > >> Anup Patel <anup@brainfault.org> wrote: > >> > > >> > Hi Marc, > >> > > >> > I have taken the approach of IPI domains (like you suggested) in this series. > >> > > >> > What do you think ? > >> > >> I have commented on the irqchip driver. > >> > >> As for the RISC-V specific code, I'll let the architecture maintainers > >> look into it. I guess the elephant in the room is that this spec seems > >> to be evolving, and that there is no HW implementation (how this > >> driver maps on SF's CLINT is anybody's guess). > > There's a long history of interrupt controller efforts from the RISC-V > foundation, and we've yet to have any of them result in hardware. The RISC-V AIA group was formed last year. Can you point me to which interrupt controller efforts you are referring to. > > > The SiFive CLINT is a more convoluted device and provides M-level > > timer functionality and M-level IPI functionality in one MMIO device. > > > > The RISC-V ACLINT specification is more modular and backward > > compatible with the SiFive CLINT. In fact, a SiFive CLINT device > > can be viewed as a ACLINT MSWI device + ACLINT MTIMER device. > > This means existing RISC-V boards having SiFive CLINT will be > > automatically compliant to the RISC-V ACLINT specification. > > So is there any hardware that this new specification enables? It seems > to be a more convoluted way to describe the same mess we're already in. > I'm not really inclined to take a bunch of code that just does the same > thing via a more complicated specification. Nope, it is much cleaner and modular compared to SiFive CLINT and it is also backward compatible to SiFive CLINT. Can you elaborate what part of the code you are not okay with ? > > > Here's the RISC-V ACLINT spec: > > https://github.com/riscv/riscv-aclint/blob/main/riscv-aclint.adoc > > > > The RISC-V ACLINT spec is quite stable and we are not seeing any > > further changes hence I sent out RFC PATCHes to get feedback. The > > RISC-V ACLINT spec will be frozen before 2021 end (i.e. before next > > RISC-V summit). > > Have you talked to the other ISA folks about that? This spec is being developed by the RISC-V AIA group based on the feedback from ISA folks and HW architects. > > As far as I can tell this new spec allows for multiple MTIME registers, > which seems to be in direct contradiction to the single -MTIME register > as defined in the ISA manual. It also seems to be vaguely incompatible > WRT the definition of SSIP, but I'm not sure that one really matters all > that much as it's not like old software can write the new registers. The ACLINT spec clearly defines that if we have multiple MTIME registers then these registers must be synchronized to meet the architecture requirements. The spec also defines a software mechanism for MTIME synchronization. It is also possible for multiple MTIMER devices to share same MTIME register. Please refer to the latest ACLINT spec. > > I just talked to Krste and Andrew, they say they haven't heard of any of > this. I don't know what's going on over there, but it's very hard to > review anything when I can't even tell where the ISA is defined. I am surprised by the respone you got because the ACLINT spec is being developed by a working group of RISC-V International. In fact, it is hosted on the RISC-V International GitHub. How can we host it on RISC-V GitHub if it is not an official spec being developed by RVI. Regards, Anup > > > The Linux NoMMU kernel (M-level) will use an ACLINT MSWI device > > for IPI support whereas the regular Linux MMU kernel (S-level) will > > use an ACLINT SSWI device for IPI support. > > > > The ACLINT SWI driver is a common IPI driver for both ACLINT > > MSWI (Linux NoMMU) and ACLINT SSWI (Linux MMU). In fact, > > the ACLINT SWI also works for IPI part (i.e. MSWI) of SiFive CLINT. > > > > Regards, > > Anup > > > >> > >> M. > >> > >> -- > >> Without deviation from the norm, progress is not possible.
On Thu, Jul 29, 2021 at 10:00 AM Palmer Dabbelt <palmerdabbelt@google.com> wrote: > > On Mon, 26 Jul 2021 06:01:01 PDT (-0700), anup@brainfault.org wrote: > > Hi Marc, > > > > On Mon, Jul 26, 2021 at 8:02 PM Marc Zyngier <maz@kernel.org> wrote: > >> > >> On Mon, 26 Jul 2021 13:45:20 +0100, > >> Anup Patel <anup@brainfault.org> wrote: > >> > > >> > Hi Marc, > >> > > >> > I have taken the approach of IPI domains (like you suggested) in this series. > >> > > >> > What do you think ? > >> > >> I have commented on the irqchip driver. > >> > >> As for the RISC-V specific code, I'll let the architecture maintainers > >> look into it. I guess the elephant in the room is that this spec seems > >> to be evolving, and that there is no HW implementation (how this > >> driver maps on SF's CLINT is anybody's guess). > > There's a long history of interrupt controller efforts from the RISC-V > foundation, and we've yet to have any of them result in hardware. > > > The SiFive CLINT is a more convoluted device and provides M-level > > timer functionality and M-level IPI functionality in one MMIO device. > > > > The RISC-V ACLINT specification is more modular and backward > > compatible with the SiFive CLINT. In fact, a SiFive CLINT device > > can be viewed as a ACLINT MSWI device + ACLINT MTIMER device. > > This means existing RISC-V boards having SiFive CLINT will be > > automatically compliant to the RISC-V ACLINT specification. > > So is there any hardware that this new specification enables? It seems > to be a more convoluted way to describe the same mess we're already in. > I'm not really inclined to take a bunch of code that just does the same > thing via a more complicated specification. > > > Here's the RISC-V ACLINT spec: > > https://github.com/riscv/riscv-aclint/blob/main/riscv-aclint.adoc > > > > The RISC-V ACLINT spec is quite stable and we are not seeing any > > further changes hence I sent out RFC PATCHes to get feedback. The > > RISC-V ACLINT spec will be frozen before 2021 end (i.e. before next > > RISC-V summit). > > Have you talked to the other ISA folks about that? > > As far as I can tell this new spec allows for multiple MTIME registers, > which seems to be in direct contradiction to the single -MTIME register The RISC-V ISA spec only mandates a single view of MTIME registers for all the HARTs The RISC-V ISA spec does not mandate a single physical MTIME register. In fact, multi-socket and multi-die systems will end-up having multiple physical MTIME registers so on such systems these physical MTIME registers need to be synchronized with hardware assistance. Please refer to the MTIME synchronization section of ACLINT specification. > as defined in the ISA manual. It also seems to be vaguely incompatible > WRT the definition of SSIP, but I'm not sure that one really matters all > that much as it's not like old software can write the new registers. Please loot at the ACLINT SSWI spec again. The SSIP bit definition is to be modified where mip.SSIP bit is logical OR of software writable bit and external SSIP signal. This way older software which uses SBI call based IPI injection will work fine. In fact, this aspect of SSIP bit is tested on QEMU RISC-V as well. > > I just talked to Krste and Andrew, they say they haven't heard of any of > this. I don't know what's going on over there, but it's very hard to > review anything when I can't even tell where the ISA is defined. Like mentioned in other thread, please first to talk to the actual spec owners first. There are lot of working groups and activities going on in RVI. Regards, Anup > > > The Linux NoMMU kernel (M-level) will use an ACLINT MSWI device > > for IPI support whereas the regular Linux MMU kernel (S-level) will > > use an ACLINT SSWI device for IPI support. > > > > The ACLINT SWI driver is a common IPI driver for both ACLINT > > MSWI (Linux NoMMU) and ACLINT SSWI (Linux MMU). In fact, > > the ACLINT SWI also works for IPI part (i.e. MSWI) of SiFive CLINT. > > > > Regards, > > Anup > > > >> > >> M. > >> > >> -- > >> Without deviation from the norm, progress is not possible.