| Message ID | 20190827085344.30799-5-ming.lei@redhat.com (mailing list archive) |
|---|---|
| State | Not Applicable |
| Headers | show |
| Series | genirq/nvme: add IRQF_RESCUE_THREAD for avoiding IRQ flood | expand |
On Tue, Aug 27, 2019 at 04:53:44PM +0800, Ming Lei wrote: > In case of IRQF_RESCUE_THREAD, the threaded handler is only used to > handle interrupt when IRQ flood comes, use irq's affinity for this thread > so that scheduler may select other not too busy CPUs for handling the > interrupt. > > Cc: Long Li <longli@microsoft.com> > Cc: Ingo Molnar <mingo@redhat.com>, > Cc: Peter Zijlstra <peterz@infradead.org> > Cc: Keith Busch <keith.busch@intel.com> > Cc: Jens Axboe <axboe@fb.com> > Cc: Christoph Hellwig <hch@lst.de> > Cc: Sagi Grimberg <sagi@grimberg.me> > Cc: John Garry <john.garry@huawei.com> > Cc: Thomas Gleixner <tglx@linutronix.de> > Cc: Hannes Reinecke <hare@suse.com> > Cc: linux-nvme@lists.infradead.org > Cc: linux-scsi@vger.kernel.org > Signed-off-by: Ming Lei <ming.lei@redhat.com> > --- > kernel/irq/manage.c | 13 ++++++++++++- > 1 file changed, 12 insertions(+), 1 deletion(-) > > diff --git a/kernel/irq/manage.c b/kernel/irq/manage.c > index 1566abbf50e8..03bc041348b7 100644 > --- a/kernel/irq/manage.c > +++ b/kernel/irq/manage.c > @@ -968,7 +968,18 @@ irq_thread_check_affinity(struct irq_desc *desc, struct irqaction *action) > if (cpumask_available(desc->irq_common_data.affinity)) { > const struct cpumask *m; > > - m = irq_data_get_effective_affinity_mask(&desc->irq_data); > + /* > + * Managed IRQ's affinity is setup gracefull on MUNA locality, s/MUNA/NUMA > + * also if IRQF_RESCUE_THREAD is set, interrupt flood has been > + * triggered, so ask scheduler to run the thread on CPUs > + * specified by this interrupt's affinity. > + */ > + if ((action->flags & IRQF_RESCUE_THREAD) && > + irqd_affinity_is_managed(&desc->irq_data)) > + m = desc->irq_common_data.affinity; > + else > + m = irq_data_get_effective_affinity_mask( > + &desc->irq_data); > cpumask_copy(mask, m); > } else { > valid = false; > --
On 27/08/2019 09:53, Ming Lei wrote: > In case of IRQF_RESCUE_THREAD, the threaded handler is only used to > handle interrupt when IRQ flood comes, use irq's affinity for this thread > so that scheduler may select other not too busy CPUs for handling the > interrupt. > > Cc: Long Li <longli@microsoft.com> > Cc: Ingo Molnar <mingo@redhat.com>, > Cc: Peter Zijlstra <peterz@infradead.org> > Cc: Keith Busch <keith.busch@intel.com> > Cc: Jens Axboe <axboe@fb.com> > Cc: Christoph Hellwig <hch@lst.de> > Cc: Sagi Grimberg <sagi@grimberg.me> > Cc: John Garry <john.garry@huawei.com> > Cc: Thomas Gleixner <tglx@linutronix.de> > Cc: Hannes Reinecke <hare@suse.com> > Cc: linux-nvme@lists.infradead.org > Cc: linux-scsi@vger.kernel.org > Signed-off-by: Ming Lei <ming.lei@redhat.com> > --- > kernel/irq/manage.c | 13 ++++++++++++- > 1 file changed, 12 insertions(+), 1 deletion(-) > > diff --git a/kernel/irq/manage.c b/kernel/irq/manage.c > index 1566abbf50e8..03bc041348b7 100644 > --- a/kernel/irq/manage.c > +++ b/kernel/irq/manage.c > @@ -968,7 +968,18 @@ irq_thread_check_affinity(struct irq_desc *desc, struct irqaction *action) > if (cpumask_available(desc->irq_common_data.affinity)) { > const struct cpumask *m; > > - m = irq_data_get_effective_affinity_mask(&desc->irq_data); > + /* > + * Managed IRQ's affinity is setup gracefull on MUNA locality, gracefully > + * also if IRQF_RESCUE_THREAD is set, interrupt flood has been > + * triggered, so ask scheduler to run the thread on CPUs > + * specified by this interrupt's affinity. > + */ Hi Ming, > + if ((action->flags & IRQF_RESCUE_THREAD) && > + irqd_affinity_is_managed(&desc->irq_data)) This doesn't look to solve the other issue I reported - that being that we handle the interrupt in a threaded handler natively, and the hard irq+threaded handler fully occupies the cpu, limiting throughput. So can we expand the scope to cover that scenario also? I don't think that it’s right to solve that separately. So if we're continuing this approach, can we add separate judgment for spreading the cpumask for the threaded part? Thanks, John > + m = desc->irq_common_data.affinity; > + else > + m = irq_data_get_effective_affinity_mask( > + &desc->irq_data); > cpumask_copy(mask, m); > } else { > valid = false; >
diff --git a/kernel/irq/manage.c b/kernel/irq/manage.c index 1566abbf50e8..03bc041348b7 100644 --- a/kernel/irq/manage.c +++ b/kernel/irq/manage.c @@ -968,7 +968,18 @@ irq_thread_check_affinity(struct irq_desc *desc, struct irqaction *action) if (cpumask_available(desc->irq_common_data.affinity)) { const struct cpumask *m; - m = irq_data_get_effective_affinity_mask(&desc->irq_data); + /* + * Managed IRQ's affinity is setup gracefull on MUNA locality, + * also if IRQF_RESCUE_THREAD is set, interrupt flood has been + * triggered, so ask scheduler to run the thread on CPUs + * specified by this interrupt's affinity. + */ + if ((action->flags & IRQF_RESCUE_THREAD) && + irqd_affinity_is_managed(&desc->irq_data)) + m = desc->irq_common_data.affinity; + else + m = irq_data_get_effective_affinity_mask( + &desc->irq_data); cpumask_copy(mask, m); } else { valid = false;
In case of IRQF_RESCUE_THREAD, the threaded handler is only used to handle interrupt when IRQ flood comes, use irq's affinity for this thread so that scheduler may select other not too busy CPUs for handling the interrupt. Cc: Long Li <longli@microsoft.com> Cc: Ingo Molnar <mingo@redhat.com>, Cc: Peter Zijlstra <peterz@infradead.org> Cc: Keith Busch <keith.busch@intel.com> Cc: Jens Axboe <axboe@fb.com> Cc: Christoph Hellwig <hch@lst.de> Cc: Sagi Grimberg <sagi@grimberg.me> Cc: John Garry <john.garry@huawei.com> Cc: Thomas Gleixner <tglx@linutronix.de> Cc: Hannes Reinecke <hare@suse.com> Cc: linux-nvme@lists.infradead.org Cc: linux-scsi@vger.kernel.org Signed-off-by: Ming Lei <ming.lei@redhat.com> --- kernel/irq/manage.c | 13 ++++++++++++- 1 file changed, 12 insertions(+), 1 deletion(-)