[RFC,2/4] irqchip/riscv-intc: Support large non-standard hwirq number

Commit Message

Yu-Chien Peter Lin Sept. 7, 2023, 2:16 a.m. UTC

Currently, the implementation of the RISC-V INTC driver uses the
interrupt cause as hwirq and has a limitation of supporting a
maximum of 64 hwirqs. However, according to the privileged spec,
interrupt cause >= 16 are defined for platform use.

This limitation prevents us from fully utilizing the available
local interrupt sources. Additionally, the hwirqs used on RISC-V
are sparse, with only interrupt numbers 1, 5 and 9 (plus Sscofpmf
or T-Head's PMU irq) being currently used for supervisor mode.

The patch switches to using irq_domain_create_tree() which
creates the radix tree map, allowing us to handle a larger
number of hwirqs.

Signed-off-by: Yu Chien Peter Lin <[e-mail]>
Reviewed-by: Charles Ci-Jyun Wu <[e-mail]>
Reviewed-by: Leo Yu-Chi Liang <[e-mail]>

---
There are 3 hwirqs of local interrupt source exceed 64 defined in
AX45MP datasheet [1] Table 56: AX45MP-1C scause Value After Trap:
- 256+16 Slave port ECC error interrupt (S-mode)
- 256+17 Bus write transaction error interrupt (S-mode)
- 256+18 Performance monitor overflow interrupt(S-mode)

[1] http://www.andestech.com/wp-content/uploads/AX45MP-1C-Rev.-5.0.0-Datasheet.pdf
---
 drivers/irqchip/irq-riscv-intc.c | 10 ++++------
 1 file changed, 4 insertions(+), 6 deletions(-)

Comments

Clément Léger Sept. 7, 2023, 10:22 a.m. UTC | #1

On 07/09/2023 04:16, Yu Chien Peter Lin wrote:
> Currently, the implementation of the RISC-V INTC driver uses the
> interrupt cause as hwirq and has a limitation of supporting a
> maximum of 64 hwirqs. However, according to the privileged spec,
> interrupt cause >= 16 are defined for platform use.
> 
> This limitation prevents us from fully utilizing the available
> local interrupt sources. Additionally, the hwirqs used on RISC-V
> are sparse, with only interrupt numbers 1, 5 and 9 (plus Sscofpmf
> or T-Head's PMU irq) being currently used for supervisor mode.
> 
> The patch switches to using irq_domain_create_tree() which
> creates the radix tree map, allowing us to handle a larger
> number of hwirqs.
> 
> Signed-off-by: Yu Chien Peter Lin <[e-mail]>
> Reviewed-by: Charles Ci-Jyun Wu <[e-mail]>
> Reviewed-by: Leo Yu-Chi Liang <[e-mail]>
> 
> ---
> There are 3 hwirqs of local interrupt source exceed 64 defined in
> AX45MP datasheet [1] Table 56: AX45MP-1C scause Value After Trap:
> - 256+16 Slave port ECC error interrupt (S-mode)
> - 256+17 Bus write transaction error interrupt (S-mode)
> - 256+18 Performance monitor overflow interrupt(S-mode)
> 
> [1] http://www.andestech.com/wp-content/uploads/AX45MP-1C-Rev.-5.0.0-Datasheet.pdf
> ---
>  drivers/irqchip/irq-riscv-intc.c | 10 ++++------
>  1 file changed, 4 insertions(+), 6 deletions(-)
> 
> diff --git a/drivers/irqchip/irq-riscv-intc.c b/drivers/irqchip/irq-riscv-intc.c
> index 4adeee1bc391..76e1229c45de 100644
> --- a/drivers/irqchip/irq-riscv-intc.c
> +++ b/drivers/irqchip/irq-riscv-intc.c
> @@ -24,8 +24,8 @@ static asmlinkage void riscv_intc_irq(struct pt_regs *regs)
>  {
>  	unsigned long cause = regs->cause & ~CAUSE_IRQ_FLAG;
>  
> -	if (unlikely(cause >= BITS_PER_LONG))
> -		panic("unexpected interrupt cause");
> +	if (!irq_find_mapping(intc_domain, cause))
> +		panic("unexpected interrupt cause: %ld", cause);

Hi Yu,

It seems like generic_handle_domain_irq() returns -EINVAL if provided
with NULL (which will happen if the interrupt does not have a mapping
due to __irq_resolve_mapping returning NULL) so maybe it is possible to
remove this check (since __irq_resolve_mapping() is also called by
generic_handle_domain_irq()) and panic if generic_handle_domain_irq()
returns -EINVAL ? That would avoid calling __irq_resolve_mapping() twice
for really rare cases.

Clément

>  
>  	generic_handle_domain_irq(intc_domain, cause);
>  }
> @@ -117,8 +117,8 @@ static int __init riscv_intc_init_common(struct fwnode_handle *fn)
>  {
>  	int rc;
>  
> -	intc_domain = irq_domain_create_linear(fn, BITS_PER_LONG,
> -					       &riscv_intc_domain_ops, NULL);
> +	intc_domain = irq_domain_create_tree(fn, &riscv_intc_domain_ops,
> +					     NULL);
>  	if (!intc_domain) {
>  		pr_err("unable to add IRQ domain\n");
>  		return -ENXIO;
> @@ -132,8 +132,6 @@ static int __init riscv_intc_init_common(struct fwnode_handle *fn)
>  
>  	riscv_set_intc_hwnode_fn(riscv_intc_hwnode);
>  
> -	pr_info("%d local interrupts mapped\n", BITS_PER_LONG);
> -
>  	return 0;
>  }
>

Anup Patel Sept. 7, 2023, 1:06 p.m. UTC | #2

On Thu, Sep 7, 2023 at 7:48 AM Yu Chien Peter Lin
<[e-mail]> wrote:
>
> Currently, the implementation of the RISC-V INTC driver uses the
> interrupt cause as hwirq and has a limitation of supporting a
> maximum of 64 hwirqs. However, according to the privileged spec,
> interrupt cause >= 16 are defined for platform use.
>
> This limitation prevents us from fully utilizing the available
> local interrupt sources. Additionally, the hwirqs used on RISC-V
> are sparse, with only interrupt numbers 1, 5 and 9 (plus Sscofpmf
> or T-Head's PMU irq) being currently used for supervisor mode.
>
> The patch switches to using irq_domain_create_tree() which
> creates the radix tree map, allowing us to handle a larger
> number of hwirqs.
>
> Signed-off-by: Yu Chien Peter Lin <[e-mail]>
> Reviewed-by: Charles Ci-Jyun Wu <[e-mail]>
> Reviewed-by: Leo Yu-Chi Liang <[e-mail]>
>
> ---
> There are 3 hwirqs of local interrupt source exceed 64 defined in
> AX45MP datasheet [1] Table 56: AX45MP-1C scause Value After Trap:
> - 256+16 Slave port ECC error interrupt (S-mode)
> - 256+17 Bus write transaction error interrupt (S-mode)
> - 256+18 Performance monitor overflow interrupt(S-mode)
>
> [1] http://www.andestech.com/wp-content/uploads/AX45MP-1C-Rev.-5.0.0-Datasheet.pdf
> ---
>  drivers/irqchip/irq-riscv-intc.c | 10 ++++------
>  1 file changed, 4 insertions(+), 6 deletions(-)
>
> diff --git a/drivers/irqchip/irq-riscv-intc.c b/drivers/irqchip/irq-riscv-intc.c
> index 4adeee1bc391..76e1229c45de 100644
> --- a/drivers/irqchip/irq-riscv-intc.c
> +++ b/drivers/irqchip/irq-riscv-intc.c
> @@ -24,8 +24,8 @@ static asmlinkage void riscv_intc_irq(struct pt_regs *regs)
>  {
>         unsigned long cause = regs->cause & ~CAUSE_IRQ_FLAG;
>
> -       if (unlikely(cause >= BITS_PER_LONG))
> -               panic("unexpected interrupt cause");
> +       if (!irq_find_mapping(intc_domain, cause))
> +               panic("unexpected interrupt cause: %ld", cause);

Checking irq_find_mapping() is redundant here instead check the return
value of generic_handle_domain_irq() and print warning on error.

>
>         generic_handle_domain_irq(intc_domain, cause);
>  }
> @@ -117,8 +117,8 @@ static int __init riscv_intc_init_common(struct fwnode_handle *fn)
>  {
>         int rc;
>
> -       intc_domain = irq_domain_create_linear(fn, BITS_PER_LONG,
> -                                              &riscv_intc_domain_ops, NULL);
> +       intc_domain = irq_domain_create_tree(fn, &riscv_intc_domain_ops,
> +                                            NULL);

This is incomplete because you have additional customization on-top-of
vanilla RISC-V INTC.

I suggest to do the following:
1) Define an enum of types of INTC (such as generic, andestech, etc)
2) Define new compatible string "andestec,cpu-intc" for you custom INTC
    and pass that information to riscv_intc_init_common()
3) Extend riscv_intc_domain_map() to use custom andestech_intc_chip
    for the custom local irqs. The andestech_intc_chip will provide andes
    specific mask/unmask mechanism.

>         if (!intc_domain) {
>                 pr_err("unable to add IRQ domain\n");
>                 return -ENXIO;
> @@ -132,8 +132,6 @@ static int __init riscv_intc_init_common(struct fwnode_handle *fn)
>
>         riscv_set_intc_hwnode_fn(riscv_intc_hwnode);
>
> -       pr_info("%d local interrupts mapped\n", BITS_PER_LONG);
> -
>         return 0;
>  }
>
> --
> 2.34.1
>

Regards,
Anup

Yu-Chien Peter Lin Sept. 11, 2023, 8:04 a.m. UTC | #3

On Thu, Sep 07, 2023 at 12:22:55PM +0200, Clément Léger wrote:
> 
> 
> On 07/09/2023 04:16, Yu Chien Peter Lin wrote:
> > Currently, the implementation of the RISC-V INTC driver uses the
> > interrupt cause as hwirq and has a limitation of supporting a
> > maximum of 64 hwirqs. However, according to the privileged spec,
> > interrupt cause >= 16 are defined for platform use.
> > 
> > This limitation prevents us from fully utilizing the available
> > local interrupt sources. Additionally, the hwirqs used on RISC-V
> > are sparse, with only interrupt numbers 1, 5 and 9 (plus Sscofpmf
> > or T-Head's PMU irq) being currently used for supervisor mode.
> > 
> > The patch switches to using irq_domain_create_tree() which
> > creates the radix tree map, allowing us to handle a larger
> > number of hwirqs.
> > 
> > Signed-off-by: Yu Chien Peter Lin <[e-mail]>
> > Reviewed-by: Charles Ci-Jyun Wu <[e-mail]>
> > Reviewed-by: Leo Yu-Chi Liang <[e-mail]>
> > 
> > ---
> > There are 3 hwirqs of local interrupt source exceed 64 defined in
> > AX45MP datasheet [1] Table 56: AX45MP-1C scause Value After Trap:
> > - 256+16 Slave port ECC error interrupt (S-mode)
> > - 256+17 Bus write transaction error interrupt (S-mode)
> > - 256+18 Performance monitor overflow interrupt(S-mode)
> > 
> > [1] http://www.andestech.com/wp-content/uploads/AX45MP-1C-Rev.-5.0.0-Datasheet.pdf
> > ---
> >  drivers/irqchip/irq-riscv-intc.c | 10 ++++------
> >  1 file changed, 4 insertions(+), 6 deletions(-)
> > 
> > diff --git a/drivers/irqchip/irq-riscv-intc.c b/drivers/irqchip/irq-riscv-intc.c
> > index 4adeee1bc391..76e1229c45de 100644
> > --- a/drivers/irqchip/irq-riscv-intc.c
> > +++ b/drivers/irqchip/irq-riscv-intc.c
> > @@ -24,8 +24,8 @@ static asmlinkage void riscv_intc_irq(struct pt_regs *regs)
> >  {
> >  	unsigned long cause = regs->cause & ~CAUSE_IRQ_FLAG;
> >  
> > -	if (unlikely(cause >= BITS_PER_LONG))
> > -		panic("unexpected interrupt cause");
> > +	if (!irq_find_mapping(intc_domain, cause))
> > +		panic("unexpected interrupt cause: %ld", cause);
> 
> Hi Yu,
> 
> It seems like generic_handle_domain_irq() returns -EINVAL if provided
> with NULL (which will happen if the interrupt does not have a mapping
> due to __irq_resolve_mapping returning NULL) so maybe it is possible to
> remove this check (since __irq_resolve_mapping() is also called by
> generic_handle_domain_irq()) and panic if generic_handle_domain_irq()
> returns -EINVAL ? That would avoid calling __irq_resolve_mapping() twice
> for really rare cases.
> 
> Clément

Hi Clément,

Thanks for catching this, will fix it in the patch v2.

Best regards,
Peter Lin

Yu-Chien Peter Lin Sept. 11, 2023, 8:12 a.m. UTC | #4

On Thu, Sep 07, 2023 at 06:36:52PM +0530, Anup Patel wrote:
> On Thu, Sep 7, 2023 at 7:48 AM Yu Chien Peter Lin
> <[e-mail]> wrote:
> >
> > Currently, the implementation of the RISC-V INTC driver uses the
> > interrupt cause as hwirq and has a limitation of supporting a
> > maximum of 64 hwirqs. However, according to the privileged spec,
> > interrupt cause >= 16 are defined for platform use.
> >
> > This limitation prevents us from fully utilizing the available
> > local interrupt sources. Additionally, the hwirqs used on RISC-V
> > are sparse, with only interrupt numbers 1, 5 and 9 (plus Sscofpmf
> > or T-Head's PMU irq) being currently used for supervisor mode.
> >
> > The patch switches to using irq_domain_create_tree() which
> > creates the radix tree map, allowing us to handle a larger
> > number of hwirqs.
> >
> > Signed-off-by: Yu Chien Peter Lin <[e-mail]>
> > Reviewed-by: Charles Ci-Jyun Wu <[e-mail]>
> > Reviewed-by: Leo Yu-Chi Liang <[e-mail]>
> >
> > ---
> > There are 3 hwirqs of local interrupt source exceed 64 defined in
> > AX45MP datasheet [1] Table 56: AX45MP-1C scause Value After Trap:
> > - 256+16 Slave port ECC error interrupt (S-mode)
> > - 256+17 Bus write transaction error interrupt (S-mode)
> > - 256+18 Performance monitor overflow interrupt(S-mode)
> >
> > [1] http://www.andestech.com/wp-content/uploads/AX45MP-1C-Rev.-5.0.0-Datasheet.pdf
> > ---
> >  drivers/irqchip/irq-riscv-intc.c | 10 ++++------
> >  1 file changed, 4 insertions(+), 6 deletions(-)
> >
> > diff --git a/drivers/irqchip/irq-riscv-intc.c b/drivers/irqchip/irq-riscv-intc.c
> > index 4adeee1bc391..76e1229c45de 100644
> > --- a/drivers/irqchip/irq-riscv-intc.c
> > +++ b/drivers/irqchip/irq-riscv-intc.c
> > @@ -24,8 +24,8 @@ static asmlinkage void riscv_intc_irq(struct pt_regs *regs)
> >  {
> >         unsigned long cause = regs->cause & ~CAUSE_IRQ_FLAG;
> >
> > -       if (unlikely(cause >= BITS_PER_LONG))
> > -               panic("unexpected interrupt cause");
> > +       if (!irq_find_mapping(intc_domain, cause))
> > +               panic("unexpected interrupt cause: %ld", cause);
> 
> Checking irq_find_mapping() is redundant here instead check the return
> value of generic_handle_domain_irq() and print warning on error.
> 
> >
> >         generic_handle_domain_irq(intc_domain, cause);
> >  }
> > @@ -117,8 +117,8 @@ static int __init riscv_intc_init_common(struct fwnode_handle *fn)
> >  {
> >         int rc;
> >
> > -       intc_domain = irq_domain_create_linear(fn, BITS_PER_LONG,
> > -                                              &riscv_intc_domain_ops, NULL);
> > +       intc_domain = irq_domain_create_tree(fn, &riscv_intc_domain_ops,
> > +                                            NULL);
> 
> This is incomplete because you have additional customization on-top-of
> vanilla RISC-V INTC.
> 
> I suggest to do the following:
> 1) Define an enum of types of INTC (such as generic, andestech, etc)
> 2) Define new compatible string "andestec,cpu-intc" for you custom INTC
>     and pass that information to riscv_intc_init_common()
> 3) Extend riscv_intc_domain_map() to use custom andestech_intc_chip
>     for the custom local irqs. The andestech_intc_chip will provide andes
>     specific mask/unmask mechanism.

Hi Anup,

Sure, we will introduce the Andes INTC for a custom IRQ chip.

Thanks,
Peter Lin

> >         if (!intc_domain) {
> >                 pr_err("unable to add IRQ domain\n");
> >                 return -ENXIO;
> > @@ -132,8 +132,6 @@ static int __init riscv_intc_init_common(struct fwnode_handle *fn)
> >
> >         riscv_set_intc_hwnode_fn(riscv_intc_hwnode);
> >
> > -       pr_info("%d local interrupts mapped\n", BITS_PER_LONG);
> > -
> >         return 0;
> >  }
> >
> > --
> > 2.34.1
> >
> 
> Regards,
> Anup

Patch

diff --git a/drivers/irqchip/irq-riscv-intc.c b/drivers/irqchip/irq-riscv-intc.c
index 4adeee1bc391..76e1229c45de 100644
--- a/drivers/irqchip/irq-riscv-intc.c
+++ b/drivers/irqchip/irq-riscv-intc.c
@@ -24,8 +24,8 @@  static asmlinkage void riscv_intc_irq(struct pt_regs *regs)
 {
 	unsigned long cause = regs->cause & ~CAUSE_IRQ_FLAG;
 
-	if (unlikely(cause >= BITS_PER_LONG))
-		panic("unexpected interrupt cause");
+	if (!irq_find_mapping(intc_domain, cause))
+		panic("unexpected interrupt cause: %ld", cause);
 
 	generic_handle_domain_irq(intc_domain, cause);
 }
@@ -117,8 +117,8 @@  static int __init riscv_intc_init_common(struct fwnode_handle *fn)
 {
 	int rc;
 
-	intc_domain = irq_domain_create_linear(fn, BITS_PER_LONG,
-					       &riscv_intc_domain_ops, NULL);
+	intc_domain = irq_domain_create_tree(fn, &riscv_intc_domain_ops,
+					     NULL);
 	if (!intc_domain) {
 		pr_err("unable to add IRQ domain\n");
 		return -ENXIO;
@@ -132,8 +132,6 @@  static int __init riscv_intc_init_common(struct fwnode_handle *fn)
 
 	riscv_set_intc_hwnode_fn(riscv_intc_hwnode);
 
-	pr_info("%d local interrupts mapped\n", BITS_PER_LONG);
-
 	return 0;
 }