| Message ID | 20201125150932.1150619-3-lvivier@redhat.com (mailing list archive) |
|---|---|
| State | New, archived |
| Headers | show |
| Series | powerpc/pseries: fix MSI/X IRQ affinity on pseries | expand |
On 11/25/20, Laurent Vivier <lvivier@redhat.com> wrote: > With virtio multiqueue, normally each queue IRQ is mapped to a CPU. > > But since commit 0d9f0a52c8b9f ("virtio_scsi: use virtio IRQ affinity") > this is broken on pseries. Please add "Fixes" tag. Thanks! > > The affinity is correctly computed in msi_desc but this is not applied > to the system IRQs. > > It appears the affinity is correctly passed to rtas_setup_msi_irqs() but > lost at this point and never passed to irq_domain_alloc_descs() > (see commit 06ee6d571f0e ("genirq: Add affinity hint to irq allocation")) > because irq_create_mapping() doesn't take an affinity parameter. > > As the previous patch has added the affinity parameter to > irq_create_mapping() we can forward the affinity from rtas_setup_msi_irqs() > to irq_domain_alloc_descs(). > > With this change, the virtqueues are correctly dispatched between the CPUs > on pseries. > > BugId: https://bugzilla.redhat.com/show_bug.cgi?id=1702939 > Signed-off-by: Laurent Vivier <lvivier@redhat.com> > Reviewed-by: Greg Kurz <groug@kaod.org> > --- > arch/powerpc/platforms/pseries/msi.c | 3 ++- > 1 file changed, 2 insertions(+), 1 deletion(-) > > diff --git a/arch/powerpc/platforms/pseries/msi.c > b/arch/powerpc/platforms/pseries/msi.c > index 133f6adcb39c..b3ac2455faad 100644 > --- a/arch/powerpc/platforms/pseries/msi.c > +++ b/arch/powerpc/platforms/pseries/msi.c > @@ -458,7 +458,8 @@ static int rtas_setup_msi_irqs(struct pci_dev *pdev, int > nvec_in, int type) > return hwirq; > } > > - virq = irq_create_mapping(NULL, hwirq); > + virq = irq_create_mapping_affinity(NULL, hwirq, > + entry->affinity); > > if (!virq) { > pr_debug("rtas_msi: Failed mapping hwirq %d\n", hwirq); > -- > 2.28.0 > >
On 25/11/2020 17:05, Denis Kirjanov wrote: > On 11/25/20, Laurent Vivier <lvivier@redhat.com> wrote: >> With virtio multiqueue, normally each queue IRQ is mapped to a CPU. >> >> But since commit 0d9f0a52c8b9f ("virtio_scsi: use virtio IRQ affinity") >> this is broken on pseries. > > Please add "Fixes" tag. In fact, the code in commit 0d9f0a52c8b9f is correct. The problem is with MSI/X irq affinity and pseries. So this patch fixes more than virtio_scsi. I put this information because this commit allows to clearly show the problem. Perhaps I should remove this line in fact? Thanks, Laurent > > Thanks! > >> >> The affinity is correctly computed in msi_desc but this is not applied >> to the system IRQs. >> >> It appears the affinity is correctly passed to rtas_setup_msi_irqs() but >> lost at this point and never passed to irq_domain_alloc_descs() >> (see commit 06ee6d571f0e ("genirq: Add affinity hint to irq allocation")) >> because irq_create_mapping() doesn't take an affinity parameter. >> >> As the previous patch has added the affinity parameter to >> irq_create_mapping() we can forward the affinity from rtas_setup_msi_irqs() >> to irq_domain_alloc_descs(). >> >> With this change, the virtqueues are correctly dispatched between the CPUs >> on pseries. >> >> BugId: https://bugzilla.redhat.com/show_bug.cgi?id=1702939 >> Signed-off-by: Laurent Vivier <lvivier@redhat.com> >> Reviewed-by: Greg Kurz <groug@kaod.org> >> --- >> arch/powerpc/platforms/pseries/msi.c | 3 ++- >> 1 file changed, 2 insertions(+), 1 deletion(-) >> >> diff --git a/arch/powerpc/platforms/pseries/msi.c >> b/arch/powerpc/platforms/pseries/msi.c >> index 133f6adcb39c..b3ac2455faad 100644 >> --- a/arch/powerpc/platforms/pseries/msi.c >> +++ b/arch/powerpc/platforms/pseries/msi.c >> @@ -458,7 +458,8 @@ static int rtas_setup_msi_irqs(struct pci_dev *pdev, int >> nvec_in, int type) >> return hwirq; >> } >> >> - virq = irq_create_mapping(NULL, hwirq); >> + virq = irq_create_mapping_affinity(NULL, hwirq, >> + entry->affinity); >> >> if (!virq) { >> pr_debug("rtas_msi: Failed mapping hwirq %d\n", hwirq); >> -- >> 2.28.0 >> >> >
On 2020-11-25 16:24, Laurent Vivier wrote: > On 25/11/2020 17:05, Denis Kirjanov wrote: >> On 11/25/20, Laurent Vivier <lvivier@redhat.com> wrote: >>> With virtio multiqueue, normally each queue IRQ is mapped to a CPU. >>> >>> But since commit 0d9f0a52c8b9f ("virtio_scsi: use virtio IRQ >>> affinity") >>> this is broken on pseries. >> >> Please add "Fixes" tag. > > In fact, the code in commit 0d9f0a52c8b9f is correct. > > The problem is with MSI/X irq affinity and pseries. So this patch > fixes more than virtio_scsi. I put this information because this > commit allows to clearly show the problem. Perhaps I should remove > this line in fact? This patch does not fix virtio_scsi at all, which as you noticed, is correct. It really fixes the PPC MSI setup, which is starting to show its age. So getting rid of the reference seems like the right thing to do. I'm also not keen on the BugId thing. It should really be a lore link. I also cannot find any such tag in the kernel, nor is it a documented practice. The last reference to a Bugzilla entry seems to have happened with 786b5219081ff16 (five years ago). Thanks, M.
On Wed, 25 Nov 2020 16:42:30 +0000 Marc Zyngier <maz@kernel.org> wrote: > On 2020-11-25 16:24, Laurent Vivier wrote: > > On 25/11/2020 17:05, Denis Kirjanov wrote: > >> On 11/25/20, Laurent Vivier <lvivier@redhat.com> wrote: > >>> With virtio multiqueue, normally each queue IRQ is mapped to a CPU. > >>> > >>> But since commit 0d9f0a52c8b9f ("virtio_scsi: use virtio IRQ > >>> affinity") > >>> this is broken on pseries. > >> > >> Please add "Fixes" tag. > > > > In fact, the code in commit 0d9f0a52c8b9f is correct. > > > > The problem is with MSI/X irq affinity and pseries. So this patch > > fixes more than virtio_scsi. I put this information because this > > commit allows to clearly show the problem. Perhaps I should remove > > this line in fact? > > This patch does not fix virtio_scsi at all, which as you noticed, is > correct. It really fixes the PPC MSI setup, which is starting to show > its age. So getting rid of the reference seems like the right thing to > do. > > I'm also not keen on the BugId thing. It should really be a lore link. > I also cannot find any such tag in the kernel, nor is it a documented > practice. The last reference to a Bugzilla entry seems to have happened > with 786b5219081ff16 (five years ago). > My bad, I suggested BugId to Laurent but the intent was actually BugLink, which seems to be commonly used in the kernel. Cheers, -- Greg > Thanks, > > M.
Marc Zyngier <maz@kernel.org> writes: > On 2020-11-25 16:24, Laurent Vivier wrote: >> On 25/11/2020 17:05, Denis Kirjanov wrote: >>> On 11/25/20, Laurent Vivier <lvivier@redhat.com> wrote: >>>> With virtio multiqueue, normally each queue IRQ is mapped to a CPU. >>>> >>>> But since commit 0d9f0a52c8b9f ("virtio_scsi: use virtio IRQ >>>> affinity") >>>> this is broken on pseries. >>> >>> Please add "Fixes" tag. >> >> In fact, the code in commit 0d9f0a52c8b9f is correct. >> >> The problem is with MSI/X irq affinity and pseries. So this patch >> fixes more than virtio_scsi. I put this information because this >> commit allows to clearly show the problem. Perhaps I should remove >> this line in fact? > > This patch does not fix virtio_scsi at all, which as you noticed, is > correct. It really fixes the PPC MSI setup, which is starting to show > its age. So getting rid of the reference seems like the right thing to > do. It's still useful to refer to that commit if the code worked prior to that commit. But you should make it clearer that 0d9f0a52c8b9f wasn't in error, it just exposed an existing shortcoming of the arch code. cheers
Laurent Vivier <lvivier@redhat.com> writes: > With virtio multiqueue, normally each queue IRQ is mapped to a CPU. > > But since commit 0d9f0a52c8b9f ("virtio_scsi: use virtio IRQ affinity") > this is broken on pseries. > > The affinity is correctly computed in msi_desc but this is not applied > to the system IRQs. > > It appears the affinity is correctly passed to rtas_setup_msi_irqs() but > lost at this point and never passed to irq_domain_alloc_descs() > (see commit 06ee6d571f0e ("genirq: Add affinity hint to irq allocation")) > because irq_create_mapping() doesn't take an affinity parameter. > > As the previous patch has added the affinity parameter to > irq_create_mapping() we can forward the affinity from rtas_setup_msi_irqs() > to irq_domain_alloc_descs(). > > With this change, the virtqueues are correctly dispatched between the CPUs > on pseries. > > BugId: https://bugzilla.redhat.com/show_bug.cgi?id=1702939 > Signed-off-by: Laurent Vivier <lvivier@redhat.com> > Reviewed-by: Greg Kurz <groug@kaod.org> > --- > arch/powerpc/platforms/pseries/msi.c | 3 ++- > 1 file changed, 2 insertions(+), 1 deletion(-) Acked-by: Michael Ellerman <mpe@ellerman.id.au> cheers > diff --git a/arch/powerpc/platforms/pseries/msi.c b/arch/powerpc/platforms/pseries/msi.c > index 133f6adcb39c..b3ac2455faad 100644 > --- a/arch/powerpc/platforms/pseries/msi.c > +++ b/arch/powerpc/platforms/pseries/msi.c > @@ -458,7 +458,8 @@ static int rtas_setup_msi_irqs(struct pci_dev *pdev, int nvec_in, int type) > return hwirq; > } > > - virq = irq_create_mapping(NULL, hwirq); > + virq = irq_create_mapping_affinity(NULL, hwirq, > + entry->affinity); > > if (!virq) { > pr_debug("rtas_msi: Failed mapping hwirq %d\n", hwirq); > -- > 2.28.0
diff --git a/arch/powerpc/platforms/pseries/msi.c b/arch/powerpc/platforms/pseries/msi.c index 133f6adcb39c..b3ac2455faad 100644 --- a/arch/powerpc/platforms/pseries/msi.c +++ b/arch/powerpc/platforms/pseries/msi.c @@ -458,7 +458,8 @@ static int rtas_setup_msi_irqs(struct pci_dev *pdev, int nvec_in, int type) return hwirq; } - virq = irq_create_mapping(NULL, hwirq); + virq = irq_create_mapping_affinity(NULL, hwirq, + entry->affinity); if (!virq) { pr_debug("rtas_msi: Failed mapping hwirq %d\n", hwirq);
With virtio multiqueue, normally each queue IRQ is mapped to a CPU. But since commit 0d9f0a52c8b9f ("virtio_scsi: use virtio IRQ affinity") this is broken on pseries. The affinity is correctly computed in msi_desc but this is not applied to the system IRQs. It appears the affinity is correctly passed to rtas_setup_msi_irqs() but lost at this point and never passed to irq_domain_alloc_descs() (see commit 06ee6d571f0e ("genirq: Add affinity hint to irq allocation")) because irq_create_mapping() doesn't take an affinity parameter. As the previous patch has added the affinity parameter to irq_create_mapping() we can forward the affinity from rtas_setup_msi_irqs() to irq_domain_alloc_descs(). With this change, the virtqueues are correctly dispatched between the CPUs on pseries. BugId: https://bugzilla.redhat.com/show_bug.cgi?id=1702939 Signed-off-by: Laurent Vivier <lvivier@redhat.com> Reviewed-by: Greg Kurz <groug@kaod.org> --- arch/powerpc/platforms/pseries/msi.c | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-)