Message ID | 1414151970-6626-5-git-send-email-thomas.petazzoni@free-electrons.com (mailing list archive) |
---|---|
State | New, archived |
Headers | show |
Hi Thomas, On 24/10/2014 13:59, Thomas Petazzoni wrote: > This commit adds suspend/resume support to the irqchip driver used on > Armada XP platforms (amongst others). It does so by adding a set of > suspend/resume syscore_ops, that will respectively save and restore > the necessary registers to ensure interrupts continue to work after > resume. > > It is worth mentioning that the affinity is lost during a > suspend/resume cycle, because when a secondary CPU is brought > off-line, all interrupts that are assigned to this CPU in terms of > affinity gets re-assigned to a still running CPU. Therefore, right > before entering suspend, all interrupts are assigned to the boot CPU. So what about /proc/irq/*/smp_affinity ? Do this files still represent accurate information? Thanks, Gregory
Dear Gregory CLEMENT, On Mon, 03 Nov 2014 18:38:10 +0100, Gregory CLEMENT wrote: > > It is worth mentioning that the affinity is lost during a > > suspend/resume cycle, because when a secondary CPU is brought > > off-line, all interrupts that are assigned to this CPU in terms of > > affinity gets re-assigned to a still running CPU. Therefore, right > > before entering suspend, all interrupts are assigned to the boot CPU. > > So what about /proc/irq/*/smp_affinity ? > > Do this files still represent accurate information? See migrate_irqs() in arch/arm/kernel/irq.c. Basically, when you do a suspend, the kernel hot unplugs the non-boot CPUs: in our case CPU1, CPU2 and CPU3. The migrate_irqs() function above makes sure that the affinity used for each interrupt continue to contain at least one online CPU. Therefore, it modifies the affinity information across suspend/resume. For example, let's say you initially have an affinity of 0xf (i.e all CPUs can receive the interrupt) and you modify it to 0x2 (i.e only CPU1 can take the interrupt). Then, when migrate_irqs() is called when CPU1 is brought offline, it will realize that the interrupt will no longer be affine to any CPU, and it will re-assign the affinity to the current cpu_online_mask (which will be 0xd, i.e all CPUs except CPU1). Therefore, when you get out of suspend, the affinity for this interrupt will be 0xd. Note that in this case, a message is displayed during the suspend procedure: IRQ19 no longer affine to CPU1 On the other hand, if you leave the affinity to its default of 0xf, it remains to 0xf, because there's always at least one online CPU in the affinity (the boot CPU). Same thing if you change the affinity to 0x3 for example, since 0x3 is an affinity that contains the boot CPU (of course, assuming the boot CPU is CPU0). That's not a logic that is controlled in the irqchip driver, it's entirely handled by the kernel core. I hope this explanation clarifies the situation. Thanks, Thomas
diff --git a/drivers/irqchip/irq-armada-370-xp.c b/drivers/irqchip/irq-armada-370-xp.c index 9e630f2..3c2b89d 100644 --- a/drivers/irqchip/irq-armada-370-xp.c +++ b/drivers/irqchip/irq-armada-370-xp.c @@ -26,6 +26,7 @@ #include <linux/of_pci.h> #include <linux/irqdomain.h> #include <linux/slab.h> +#include <linux/syscore_ops.h> #include <linux/msi.h> #include <asm/mach/arch.h> #include <asm/exception.h> @@ -66,6 +67,7 @@ static void __iomem *per_cpu_int_base; static void __iomem *main_int_base; static struct irq_domain *armada_370_xp_mpic_domain; +static u32 doorbell_mask_reg; #ifdef CONFIG_PCI_MSI static struct irq_domain *armada_370_xp_msi_domain; static DECLARE_BITMAP(msi_used, PCI_MSI_DOORBELL_NR); @@ -474,6 +476,54 @@ armada_370_xp_handle_irq(struct pt_regs *regs) } while (1); } +static int armada_370_xp_mpic_suspend(void) +{ + doorbell_mask_reg = readl(per_cpu_int_base + + ARMADA_370_XP_IN_DRBEL_MSK_OFFS); + return 0; +} + +static void armada_370_xp_mpic_resume(void) +{ + int nirqs; + irq_hw_number_t irq; + + /* Re-enable interrupts */ + nirqs = (readl(main_int_base + ARMADA_370_XP_INT_CONTROL) >> 2) & 0x3ff; + for (irq = 0; irq < nirqs; irq++) { + struct irq_data *data; + int virq; + + virq = irq_linear_revmap(armada_370_xp_mpic_domain, irq); + if (virq == 0) + continue; + + if (irq != ARMADA_370_XP_TIMER0_PER_CPU_IRQ) + writel(irq, per_cpu_int_base + + ARMADA_370_XP_INT_CLEAR_MASK_OFFS); + else + writel(irq, main_int_base + + ARMADA_370_XP_INT_SET_ENABLE_OFFS); + + data = irq_get_irq_data(virq); + if (!irqd_irq_disabled(data)) + armada_370_xp_irq_unmask(data); + } + + /* Reconfigure doorbells for IPIs and MSIs */ + writel(doorbell_mask_reg, + per_cpu_int_base + ARMADA_370_XP_IN_DRBEL_MSK_OFFS); + if (doorbell_mask_reg & IPI_DOORBELL_MASK) + writel(0, per_cpu_int_base + ARMADA_370_XP_INT_CLEAR_MASK_OFFS); + if (doorbell_mask_reg & PCI_MSI_DOORBELL_MASK) + writel(1, per_cpu_int_base + ARMADA_370_XP_INT_CLEAR_MASK_OFFS); +} + +struct syscore_ops armada_370_xp_mpic_syscore_ops = { + .suspend = armada_370_xp_mpic_suspend, + .resume = armada_370_xp_mpic_resume, +}; + static int __init armada_370_xp_mpic_of_init(struct device_node *node, struct device_node *parent) { @@ -530,6 +580,8 @@ static int __init armada_370_xp_mpic_of_init(struct device_node *node, armada_370_xp_mpic_handle_cascade_irq); } + register_syscore_ops(&armada_370_xp_mpic_syscore_ops); + return 0; }
This commit adds suspend/resume support to the irqchip driver used on Armada XP platforms (amongst others). It does so by adding a set of suspend/resume syscore_ops, that will respectively save and restore the necessary registers to ensure interrupts continue to work after resume. It is worth mentioning that the affinity is lost during a suspend/resume cycle, because when a secondary CPU is brought off-line, all interrupts that are assigned to this CPU in terms of affinity gets re-assigned to a still running CPU. Therefore, right before entering suspend, all interrupts are assigned to the boot CPU. Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com> Cc: Thomas Gleixner <tglx@linutronix.de> Cc: Jason Cooper <jason@lakedaemon.net> Cc: linux-kernel@vger.kernel.org --- drivers/irqchip/irq-armada-370-xp.c | 52 +++++++++++++++++++++++++++++++++++++ 1 file changed, 52 insertions(+)