[V7] irqchip/sifive-plic: Fixup EOI failed when masked

Commit Message

Guo Ren Nov. 5, 2021, 9:47 a.m. UTC

From: Guo Ren <[e-mail]>

When using "devm_request_threaded_irq(,,,,IRQF_ONESHOT,,)" in the driver,
only the first interrupt could be handled, and continue irq is blocked by
hw. Because the riscv plic couldn't complete masked irq source which has
been disabled in enable register. The bug was firstly reported in [1].

Here is the description of Interrupt Completion in PLIC spec [2]:

The PLIC signals it has completed executing an interrupt handler by
writing the interrupt ID it received from the claim to the claim/complete
register. The PLIC does not check whether the completion ID is the same
as the last claim ID for that target. If the completion ID does not match
an interrupt source that is currently enabled for the target, the
                         ^^ ^^^^^^^^^ ^^^^^^^
completion is silently ignored.

[1] http://lists.infradead.org/pipermail/linux-riscv/2021-July/007441.html
[2] https://github.com/riscv/riscv-plic-spec/blob/8bc15a35d07c9edf7b5d23fec9728302595ffc4d/riscv-plic.adoc

Fixes: bb0fed1c60cc ("irqchip/sifive-plic: Switch to fasteoi flow")
Reported-by: Vincent Pelletier <[e-mail]>
Tested-by: Nikita Shubin <[e-mail]>
Signed-off-by: Guo Ren <[e-mail]>
Cc: [e-mail]
Cc: Anup Patel <[e-mail]>
Cc: Thomas Gleixner <[e-mail]>
Cc: Marc Zyngier <[e-mail]>
Cc: Palmer Dabbelt <[e-mail]>
Cc: Atish Patra <[e-mail]>
Cc: Nikita Shubin <[e-mail]>
Cc: incent Pelletier <[e-mail]>

---

Changes since V7:
 - Add Fixes tag
 - Add Tested-by
 - Add Cc stable

Changes since V6:
 - Propagate to plic_irq_eoi for all riscv,plic by Nikita Shubin
 - Remove thead related codes

Changes since V5:
 - Move back to mask/unmask
 - Fixup the problem in eoi callback
 - Remove allwinner,sun20i-d1 IRQCHIP_DECLARE
 - Rewrite comment log

Changes since V4:
 - Update comment by Anup

Changes since V3:
 - Rename "c9xx" to "c900"
 - Add sifive_plic_chip and thead_plic_chip for difference

Changes since V2:
 - Add a separate compatible string "thead,c9xx-plic"
 - set irq_mask/unmask of "plic_chip" to NULL and point
   irq_enable/disable of "plic_chip" to plic_irq_mask/unmask
 - Add a detailed comment block in plic_init() about the
   differences in Claim/Completion process of RISC-V PLIC and C9xx
   PLIC.
---
 drivers/irqchip/irq-sifive-plic.c | 8 +++++++-
 1 file changed, 7 insertions(+), 1 deletion(-)

Comments

Aurelien Jarno Nov. 6, 2021, 12:11 p.m. UTC | #1

On 2021-11-05 17:47, [e-mail] wrote:
> From: Guo Ren <[e-mail]>
> 
> When using "devm_request_threaded_irq(,,,,IRQF_ONESHOT,,)" in the driver,
> only the first interrupt could be handled, and continue irq is blocked by
> hw. Because the riscv plic couldn't complete masked irq source which has
> been disabled in enable register. The bug was firstly reported in [1].
> 
> Here is the description of Interrupt Completion in PLIC spec [2]:
> 
> The PLIC signals it has completed executing an interrupt handler by
> writing the interrupt ID it received from the claim to the claim/complete
> register. The PLIC does not check whether the completion ID is the same
> as the last claim ID for that target. If the completion ID does not match
> an interrupt source that is currently enabled for the target, the
>                          ^^ ^^^^^^^^^ ^^^^^^^
> completion is silently ignored.
> 
> [1] http://lists.infradead.org/pipermail/linux-riscv/2021-July/007441.html
> [2] https://github.com/riscv/riscv-plic-spec/blob/8bc15a35d07c9edf7b5d23fec9728302595ffc4d/riscv-plic.adoc
> 
> Fixes: bb0fed1c60cc ("irqchip/sifive-plic: Switch to fasteoi flow")
> Reported-by: Vincent Pelletier <[e-mail]>
> Tested-by: Nikita Shubin <[e-mail]>
> Signed-off-by: Guo Ren <[e-mail]>
> Cc: [e-mail]
> Cc: Anup Patel <[e-mail]>
> Cc: Thomas Gleixner <[e-mail]>
> Cc: Marc Zyngier <[e-mail]>
> Cc: Palmer Dabbelt <[e-mail]>
> Cc: Atish Patra <[e-mail]>
> Cc: Nikita Shubin <[e-mail]>
> Cc: incent Pelletier <[e-mail]>

Thanks for this patch. From what I understand, it fixes among other
things the possibility to read the DA9063 RTC more than once.

Does it means that we could now enable it in the device tree? I mean
something like the following patch that unfortunately I can't test now:

diff --git a/arch/riscv/boot/dts/sifive/hifive-unmatched-a00.dts b/arch/riscv/boot/dts/sifive/hifive-unmatched-a00.dts
index 2e4ea84f27e7..c357b48582f7 100644
--- a/arch/riscv/boot/dts/sifive/hifive-unmatched-a00.dts
+++ b/arch/riscv/boot/dts/sifive/hifive-unmatched-a00.dts
@@ -70,6 +70,10 @@ pmic@58 {
 		interrupts = <1 IRQ_TYPE_LEVEL_LOW>;
 		interrupt-controller;
 
+		onkey {
+			compatible = "dlg,da9063-onkey";
+		};
+
 		regulators {
 			vdd_bcore1: bcore1 {
 				regulator-min-microvolt = <900000>;
@@ -205,6 +209,14 @@ vdd_ldo11: ldo11 {
 				regulator-always-on;
 			};
 		};
+
+		rtc {
+			compatible = "dlg,da9063-rtc";
+		};
+
+		wdt {
+			compatible = "dlg,da9063-watchdog";
+		};
 	};
 };

Nikita Shubin Nov. 6, 2021, 1:28 p.m. UTC | #2

Hello, Aurelien!

On Sat, 6 Nov 2021 13:11:51 +0100
Aurelien Jarno <[e-mail]> wrote:

> On 2021-11-05 17:47, [e-mail] wrote:
> > From: Guo Ren <[e-mail]>
> > 
> > When using "devm_request_threaded_irq(,,,,IRQF_ONESHOT,,)" in the
> > driver, only the first interrupt could be handled, and continue irq
> > is blocked by hw. Because the riscv plic couldn't complete masked
> > irq source which has been disabled in enable register. The bug was
> > firstly reported in [1].
> > 
> > Here is the description of Interrupt Completion in PLIC spec [2]:
> > 
> > The PLIC signals it has completed executing an interrupt handler by
> > writing the interrupt ID it received from the claim to the
> > claim/complete register. The PLIC does not check whether the
> > completion ID is the same as the last claim ID for that target. If
> > the completion ID does not match an interrupt source that is
> > currently enabled for the target, the ^^ ^^^^^^^^^ ^^^^^^^
> > completion is silently ignored.
> > 
> > [1]
> > http://lists.infradead.org/pipermail/linux-riscv/2021-July/007441.html
> > [2]
> > https://github.com/riscv/riscv-plic-spec/blob/8bc15a35d07c9edf7b5d23fec9728302595ffc4d/riscv-plic.adoc
> > 
> > Fixes: bb0fed1c60cc ("irqchip/sifive-plic: Switch to fasteoi flow")
> > Reported-by: Vincent Pelletier <[e-mail]>
> > Tested-by: Nikita Shubin <[e-mail]>
> > Signed-off-by: Guo Ren <[e-mail]>
> > Cc: [e-mail]
> > Cc: Anup Patel <[e-mail]>
> > Cc: Thomas Gleixner <[e-mail]>
> > Cc: Marc Zyngier <[e-mail]>
> > Cc: Palmer Dabbelt <[e-mail]>
> > Cc: Atish Patra <[e-mail]>
> > Cc: Nikita Shubin <[e-mail]>
> > Cc: incent Pelletier <[e-mail]>  
> 
> Thanks for this patch. From what I understand, it fixes among other
> things the possibility to read the DA9063 RTC more than once.

Well RTC works definitely, through you need some patch to use it as a
wakeup source:

https://lkml.org/lkml/2021/11/1/800

and, AFAIK currently SiFive Unmatched lack's PM.

I still haven't tested the Watchdog and Onkey.

> 
> Does it means that we could now enable it in the device tree? I mean
> something like the following patch that unfortunately I can't test
> now:

Adding RTC is really useful and adding Watchdog along with Onkey
shouldn't make any harm.

> 
> diff --git a/arch/riscv/boot/dts/sifive/hifive-unmatched-a00.dts
> b/arch/riscv/boot/dts/sifive/hifive-unmatched-a00.dts index
> 2e4ea84f27e7..c357b48582f7 100644 ---
> a/arch/riscv/boot/dts/sifive/hifive-unmatched-a00.dts +++
> b/arch/riscv/boot/dts/sifive/hifive-unmatched-a00.dts @@ -70,6 +70,10
> @@ pmic@58 { interrupts = <1 IRQ_TYPE_LEVEL_LOW>;
>  		interrupt-controller;
>  
> +		onkey {
> +			compatible = "dlg,da9063-onkey";
> +		};
> +
>  		regulators {
>  			vdd_bcore1: bcore1 {
>  				regulator-min-microvolt = <900000>;
> @@ -205,6 +209,14 @@ vdd_ldo11: ldo11 {
>  				regulator-always-on;
>  			};
>  		};
> +
> +		rtc {
> +			compatible = "dlg,da9063-rtc";
> +		};
> +
> +		wdt {
> +			compatible = "dlg,da9063-watchdog";
> +		};
>  	};
>  };
>

Anup Patel Nov. 6, 2021, 1:45 p.m. UTC | #3

On Fri, Nov 5, 2021 at 3:18 PM <[e-mail]> wrote:
>
> From: Guo Ren <[e-mail]>
>
> When using "devm_request_threaded_irq(,,,,IRQF_ONESHOT,,)" in the driver,
> only the first interrupt could be handled, and continue irq is blocked by
> hw. Because the riscv plic couldn't complete masked irq source which has
> been disabled in enable register. The bug was firstly reported in [1].
>
> Here is the description of Interrupt Completion in PLIC spec [2]:
>
> The PLIC signals it has completed executing an interrupt handler by
> writing the interrupt ID it received from the claim to the claim/complete
> register. The PLIC does not check whether the completion ID is the same
> as the last claim ID for that target. If the completion ID does not match
> an interrupt source that is currently enabled for the target, the
>                          ^^ ^^^^^^^^^ ^^^^^^^
> completion is silently ignored.
>
> [1] http://lists.infradead.org/pipermail/linux-riscv/2021-July/007441.html
> [2] https://github.com/riscv/riscv-plic-spec/blob/8bc15a35d07c9edf7b5d23fec9728302595ffc4d/riscv-plic.adoc
>
> Fixes: bb0fed1c60cc ("irqchip/sifive-plic: Switch to fasteoi flow")
> Reported-by: Vincent Pelletier <[e-mail]>
> Tested-by: Nikita Shubin <[e-mail]>
> Signed-off-by: Guo Ren <[e-mail]>

Looks good to me.

Reviewed-by: Anup Patel <[e-mail]>

Regards,
Anup

> Cc: [e-mail]
> Cc: Anup Patel <[e-mail]>
> Cc: Thomas Gleixner <[e-mail]>
> Cc: Marc Zyngier <[e-mail]>
> Cc: Palmer Dabbelt <[e-mail]>
> Cc: Atish Patra <[e-mail]>
> Cc: Nikita Shubin <[e-mail]>
> Cc: incent Pelletier <[e-mail]>
>
> ---
>
> Changes since V7:
>  - Add Fixes tag
>  - Add Tested-by
>  - Add Cc stable
>
> Changes since V6:
>  - Propagate to plic_irq_eoi for all riscv,plic by Nikita Shubin
>  - Remove thead related codes
>
> Changes since V5:
>  - Move back to mask/unmask
>  - Fixup the problem in eoi callback
>  - Remove allwinner,sun20i-d1 IRQCHIP_DECLARE
>  - Rewrite comment log
>
> Changes since V4:
>  - Update comment by Anup
>
> Changes since V3:
>  - Rename "c9xx" to "c900"
>  - Add sifive_plic_chip and thead_plic_chip for difference
>
> Changes since V2:
>  - Add a separate compatible string "thead,c9xx-plic"
>  - set irq_mask/unmask of "plic_chip" to NULL and point
>    irq_enable/disable of "plic_chip" to plic_irq_mask/unmask
>  - Add a detailed comment block in plic_init() about the
>    differences in Claim/Completion process of RISC-V PLIC and C9xx
>    PLIC.
> ---
>  drivers/irqchip/irq-sifive-plic.c | 8 +++++++-
>  1 file changed, 7 insertions(+), 1 deletion(-)
>
> diff --git a/drivers/irqchip/irq-sifive-plic.c b/drivers/irqchip/irq-sifive-plic.c
> index cf74cfa82045..259065d271ef 100644
> --- a/drivers/irqchip/irq-sifive-plic.c
> +++ b/drivers/irqchip/irq-sifive-plic.c
> @@ -163,7 +163,13 @@ static void plic_irq_eoi(struct irq_data *d)
>  {
>         struct plic_handler *handler = this_cpu_ptr(&plic_handlers);
>
> -       writel(d->hwirq, handler->hart_base + CONTEXT_CLAIM);
> +       if (irqd_irq_masked(d)) {
> +               plic_irq_unmask(d);
> +               writel(d->hwirq, handler->hart_base + CONTEXT_CLAIM);
> +               plic_irq_mask(d);
> +       } else {
> +               writel(d->hwirq, handler->hart_base + CONTEXT_CLAIM);
> +       }
>  }
>
>  static struct irq_chip plic_chip = {
> --
> 2.25.1
>

Guo Ren Nov. 7, 2021, 1:09 p.m. UTC | #4

On Sat, Nov 6, 2021 at 9:45 PM Anup Patel <[e-mail]> wrote:
>
> On Fri, Nov 5, 2021 at 3:18 PM <[e-mail]> wrote:
> >
> > From: Guo Ren <[e-mail]>
> >
> > When using "devm_request_threaded_irq(,,,,IRQF_ONESHOT,,)" in the driver,
> > only the first interrupt could be handled, and continue irq is blocked by
> > hw. Because the riscv plic couldn't complete masked irq source which has
> > been disabled in enable register. The bug was firstly reported in [1].
> >
> > Here is the description of Interrupt Completion in PLIC spec [2]:
> >
> > The PLIC signals it has completed executing an interrupt handler by
> > writing the interrupt ID it received from the claim to the claim/complete
> > register. The PLIC does not check whether the completion ID is the same
> > as the last claim ID for that target. If the completion ID does not match
> > an interrupt source that is currently enabled for the target, the
> >                          ^^ ^^^^^^^^^ ^^^^^^^
> > completion is silently ignored.
> >
> > [1] http://lists.infradead.org/pipermail/linux-riscv/2021-July/007441.html
> > [2] https://github.com/riscv/riscv-plic-spec/blob/8bc15a35d07c9edf7b5d23fec9728302595ffc4d/riscv-plic.adoc
> >
> > Fixes: bb0fed1c60cc ("irqchip/sifive-plic: Switch to fasteoi flow")
> > Reported-by: Vincent Pelletier <[e-mail]>
> > Tested-by: Nikita Shubin <[e-mail]>
> > Signed-off-by: Guo Ren <[e-mail]>
>
> Looks good to me.
>
> Reviewed-by: Anup Patel <[e-mail]>
Thx

@Marc Zyngier
Could you help to take the patch into your tree include "Anup's
Reviewed-by"? Or Let me update a new version of the patch.

>
> Regards,
> Anup
>
> > Cc: [e-mail]
> > Cc: Anup Patel <[e-mail]>
> > Cc: Thomas Gleixner <[e-mail]>
> > Cc: Marc Zyngier <[e-mail]>
> > Cc: Palmer Dabbelt <[e-mail]>
> > Cc: Atish Patra <[e-mail]>
> > Cc: Nikita Shubin <[e-mail]>
> > Cc: incent Pelletier <[e-mail]>
> >
> > ---
> >
> > Changes since V7:
> >  - Add Fixes tag
> >  - Add Tested-by
> >  - Add Cc stable
> >
> > Changes since V6:
> >  - Propagate to plic_irq_eoi for all riscv,plic by Nikita Shubin
> >  - Remove thead related codes
> >
> > Changes since V5:
> >  - Move back to mask/unmask
> >  - Fixup the problem in eoi callback
> >  - Remove allwinner,sun20i-d1 IRQCHIP_DECLARE
> >  - Rewrite comment log
> >
> > Changes since V4:
> >  - Update comment by Anup
> >
> > Changes since V3:
> >  - Rename "c9xx" to "c900"
> >  - Add sifive_plic_chip and thead_plic_chip for difference
> >
> > Changes since V2:
> >  - Add a separate compatible string "thead,c9xx-plic"
> >  - set irq_mask/unmask of "plic_chip" to NULL and point
> >    irq_enable/disable of "plic_chip" to plic_irq_mask/unmask
> >  - Add a detailed comment block in plic_init() about the
> >    differences in Claim/Completion process of RISC-V PLIC and C9xx
> >    PLIC.
> > ---
> >  drivers/irqchip/irq-sifive-plic.c | 8 +++++++-
> >  1 file changed, 7 insertions(+), 1 deletion(-)
> >
> > diff --git a/drivers/irqchip/irq-sifive-plic.c b/drivers/irqchip/irq-sifive-plic.c
> > index cf74cfa82045..259065d271ef 100644
> > --- a/drivers/irqchip/irq-sifive-plic.c
> > +++ b/drivers/irqchip/irq-sifive-plic.c
> > @@ -163,7 +163,13 @@ static void plic_irq_eoi(struct irq_data *d)
> >  {
> >         struct plic_handler *handler = this_cpu_ptr(&plic_handlers);
> >
> > -       writel(d->hwirq, handler->hart_base + CONTEXT_CLAIM);
> > +       if (irqd_irq_masked(d)) {
> > +               plic_irq_unmask(d);
> > +               writel(d->hwirq, handler->hart_base + CONTEXT_CLAIM);
> > +               plic_irq_mask(d);
> > +       } else {
> > +               writel(d->hwirq, handler->hart_base + CONTEXT_CLAIM);
> > +       }
> >  }
> >
> >  static struct irq_chip plic_chip = {
> > --
> > 2.25.1
> >

Marc Zyngier Nov. 7, 2021, 1:13 p.m. UTC | #5

On 2021-11-07 13:09, Guo Ren wrote:
> On Sat, Nov 6, 2021 at 9:45 PM Anup Patel <[e-mail]> wrote:
>> 
>> On Fri, Nov 5, 2021 at 3:18 PM <[e-mail]> wrote:
>> >
>> > From: Guo Ren <[e-mail]>
>> >
>> > When using "devm_request_threaded_irq(,,,,IRQF_ONESHOT,,)" in the driver,
>> > only the first interrupt could be handled, and continue irq is blocked by
>> > hw. Because the riscv plic couldn't complete masked irq source which has
>> > been disabled in enable register. The bug was firstly reported in [1].
>> >
>> > Here is the description of Interrupt Completion in PLIC spec [2]:
>> >
>> > The PLIC signals it has completed executing an interrupt handler by
>> > writing the interrupt ID it received from the claim to the claim/complete
>> > register. The PLIC does not check whether the completion ID is the same
>> > as the last claim ID for that target. If the completion ID does not match
>> > an interrupt source that is currently enabled for the target, the
>> >                          ^^ ^^^^^^^^^ ^^^^^^^
>> > completion is silently ignored.
>> >
>> > [1] http://lists.infradead.org/pipermail/linux-riscv/2021-July/007441.html
>> > [2] https://github.com/riscv/riscv-plic-spec/blob/8bc15a35d07c9edf7b5d23fec9728302595ffc4d/riscv-plic.adoc
>> >
>> > Fixes: bb0fed1c60cc ("irqchip/sifive-plic: Switch to fasteoi flow")
>> > Reported-by: Vincent Pelletier <[e-mail]>
>> > Tested-by: Nikita Shubin <[e-mail]>
>> > Signed-off-by: Guo Ren <[e-mail]>
>> 
>> Looks good to me.
>> 
>> Reviewed-by: Anup Patel <[e-mail]>
> Thx
> 
> @Marc Zyngier
> Could you help to take the patch into your tree include "Anup's
> Reviewed-by"? Or Let me update a new version of the patch.

You should have received [1], which shows you what I have done.

         M.

[1] 
https://lore.kernel.org/all/163620881803.626.5045336370262044443.tip-bot2@tip-bot2/

Patch

diff --git a/drivers/irqchip/irq-sifive-plic.c b/drivers/irqchip/irq-sifive-plic.c
index cf74cfa82045..259065d271ef 100644
--- a/drivers/irqchip/irq-sifive-plic.c
+++ b/drivers/irqchip/irq-sifive-plic.c
@@ -163,7 +163,13 @@  static void plic_irq_eoi(struct irq_data *d)
 {
 	struct plic_handler *handler = this_cpu_ptr(&plic_handlers);
 
-	writel(d->hwirq, handler->hart_base + CONTEXT_CLAIM);
+	if (irqd_irq_masked(d)) {
+		plic_irq_unmask(d);
+		writel(d->hwirq, handler->hart_base + CONTEXT_CLAIM);
+		plic_irq_mask(d);
+	} else {
+		writel(d->hwirq, handler->hart_base + CONTEXT_CLAIM);
+	}
 }
 
 static struct irq_chip plic_chip = {