| Message ID | 20201124200308.1110744-3-lvivier@redhat.com (mailing list archive) |
|---|---|
| State | Superseded, archived |
| Headers | show |
| Series | powerpc/pseries: fix MSI/X IRQ affinity on pseries | expand |
On Tue, Nov 24 2020 at 21:03, Laurent Vivier wrote: > With virtio multiqueue, normally each queue IRQ is mapped to a CPU. > > This problem cannot be shown on x86_64 for two reasons: There is only _ONE_ reason why this is not a problem on x86. x86 uses the generic PCI/MSI domain which supports this out of the box. > - the call path traverses arch_setup_msi_irqs() that is arch specific: > > virtscsi_probe() > virtscsi_init() > vp_modern_find_vqs() > vp_find_vqs() > vp_find_vqs_msix() > pci_alloc_irq_vectors_affinity() > __pci_enable_msix_range() > pci_msi_setup_msi_irqs() > arch_setup_msi_irqs() > rtas_setup_msi_irqs() This is a problem on _all_ variants of PPC MSI providers, not only for pseries. It's not restricted to virtscsi devices either, that's just the device which made you discover this. Thanks, tglx
diff --git a/arch/powerpc/platforms/pseries/msi.c b/arch/powerpc/platforms/pseries/msi.c index 42ba08eaea91..58197f92c6a2 100644 --- a/arch/powerpc/platforms/pseries/msi.c +++ b/arch/powerpc/platforms/pseries/msi.c @@ -458,7 +458,7 @@ static int rtas_setup_msi_irqs(struct pci_dev *pdev, int nvec_in, int type) return hwirq; } - virq = irq_create_mapping(NULL, hwirq, NULL); + virq = irq_create_mapping(NULL, hwirq, entry->affinity); if (!virq) { pr_debug("rtas_msi: Failed mapping hwirq %d\n", hwirq);
With virtio multiqueue, normally each queue IRQ is mapped to a CPU. But since commit 0d9f0a52c8b9f ("virtio_scsi: use virtio IRQ affinity") this is broken on pseries. The affinity is correctly computed in msi_desc but this is not applied to the system IRQs. It appears the affinity is correctly passed to rtas_setup_msi_irqs() but lost at this point and never passed to irq_domain_alloc_descs() (see commit 06ee6d571f0e ("genirq: Add affinity hint to irq allocation")) because irq_create_mapping() doesn't take an affinity parameter. As the previous patch has added the affinity parameter to irq_create_mapping() we can forward the affinity from rtas_setup_msi_irqs() to irq_domain_alloc_descs(). With this change, the virtqueues are correctly dispatched between the CPUs on pseries. This problem cannot be shown on x86_64 for two reasons: - the call path traverses arch_setup_msi_irqs() that is arch specific: virtscsi_probe() virtscsi_init() vp_modern_find_vqs() vp_find_vqs() vp_find_vqs_msix() pci_alloc_irq_vectors_affinity() __pci_enable_msix_range() pci_msi_setup_msi_irqs() arch_setup_msi_irqs() rtas_setup_msi_irqs() irq_create_mapping() irq_domain_alloc_descs() __irq_alloc_descs() - and x86_64 has CONFIG_PCI_MSI_IRQ_DOMAIN that uses another path: virtscsi_probe() virtscsi_init() vp_modern_find_vqs() vp_find_vqs() vp_find_vqs_msix() pci_alloc_irq_vectors_affinity() __pci_enable_msix_range() __msi_domain_alloc_irqs() __irq_domain_alloc_irqs() __irq_alloc_descs() Signed-off-by: Laurent Vivier <lvivier@redhat.com> --- arch/powerpc/platforms/pseries/msi.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)