| Message ID | 20171003154920.31566-3-acme@kernel.org (mailing list archive) |
|---|---|
| State | Not Applicable |
| Headers | show |
On 2017-10-03 12:49:20 [-0300], Arnaldo Carvalho de Melo wrote: > From: Arnaldo Carvalho de Melo <acme@redhat.com> > > The hfi1 driver calls request_threaded_irq with two parameters: > > handler = receive_context_interrupt; > thread = receive_context_thread; > request_threaded_irq(me->msix.vector, handler, thread, 0, me->name, arg); > > And tries to process packets on the hard irq one, receive_context_interrupt(), > only waking up the thread (returning IRQ_WAKE_THREAD) when some threshold is > crossed in the number of packets available in the NIC, trying to balance > latency and bandwidth. > > But in a CONFIG_PREEMPT_RT_FULL kernel it ends up calling spin locks from the > hard irq handler (receive_context_interrupt) which causes BUGs like this: If I am not mistaken current devel-tree of RT (and a few releases before that) handle that case correctly and force-thread both threads. Sebastian -- To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
On 10/3/2017 11:49 AM, Arnaldo Carvalho de Melo wrote: > Cc: Clark Williams <williams@redhat.com> > Cc: Dean Luick <dean.luick@intel.com> > Cc: Dennis Dalessandro <dennis.dalessandro@intel.com> > Cc: Doug Ledford <dledford@redhat.com> > Cc: Julia Cartwright <julia@ni.com> > Cc: Kaike Wan <kaike.wan@intel.com> > Cc: Leon Romanovsky <leonro@mellanox.com> > Cc: linux-rdma@vger.kernel.org > Cc: Peter Zijlstra <peterz@infradead.org> > Cc: Sebastian Andrzej Siewior <sebastian.siewior@linutronix.de> > Cc: Sebastian Sanchez <sebastian.sanchez@intel.com> > Cc: Steven Rostedt <rostedt@goodmis.org> > Cc: Thomas Gleixner <tglx@linutronix.de> > Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com> Mike has been working with these developers off list and has provided a slightly different approach. I'm going to let them drive that before I add an RB/Ack here. -Dennis -- To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Em Tue, Oct 10, 2017 at 03:06:00PM -0400, Dennis Dalessandro escreveu: > On 10/3/2017 11:49 AM, Arnaldo Carvalho de Melo wrote: > > Cc: Clark Williams <williams@redhat.com> > > Cc: Dean Luick <dean.luick@intel.com> > > Cc: Dennis Dalessandro <dennis.dalessandro@intel.com> > > Cc: Doug Ledford <dledford@redhat.com> > > Cc: Julia Cartwright <julia@ni.com> > > Cc: Kaike Wan <kaike.wan@intel.com> > > Cc: Leon Romanovsky <leonro@mellanox.com> > > Cc: linux-rdma@vger.kernel.org > > Cc: Peter Zijlstra <peterz@infradead.org> > > Cc: Sebastian Andrzej Siewior <sebastian.siewior@linutronix.de> > > Cc: Sebastian Sanchez <sebastian.sanchez@intel.com> > > Cc: Steven Rostedt <rostedt@goodmis.org> > > Cc: Thomas Gleixner <tglx@linutronix.de> > > Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com> > > Mike has been working with these developers off list and has provided a > slightly different approach. I'm going to let them drive that before I add > an RB/Ack here. This may not even be needed, i.e. Sebastian Siewior said that recent rt patches already identify the way this driver uses threaded interrupts and automagically does the right thing, Sebastian? He mentioned this in this message: https://marc.info/?l=linux-rt-users&m=150722087514307&w=2 But I replied to Mike and don't recall getting an answer to my last question. Meanwhile a kernel with these two patches survived the original bug reported at Red Hat's bugzilla. - Arnaldo -- To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
On 2017-10-10 16:15:29 [-0300], Arnaldo Carvalho de Melo wrote: > This may not even be needed, i.e. Sebastian Siewior said that recent > rt patches already identify the way this driver uses threaded interrupts > and automagically does the right thing, Sebastian? > > He mentioned this in this message: > > https://marc.info/?l=linux-rt-users&m=150722087514307&w=2 > > But I replied to Mike and don't recall getting an answer to my last > question. correct, since 4.1.7-rt8 https://git.kernel.org/pub/scm/linux/kernel/git/rt/linux-rt-devel.git/commit/?h=v4.1.7-rt8-patches&id=7461758b9e982e4ea6280ce9308492e7cceda2ed > - Arnaldo Sebastian -- To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Em Wed, Oct 11, 2017 at 12:44:56PM +0200, Sebastian Andrzej Siewior escreveu: > On 2017-10-10 16:15:29 [-0300], Arnaldo Carvalho de Melo wrote: > > This may not even be needed, i.e. Sebastian Siewior said that recent > > rt patches already identify the way this driver uses threaded interrupts > > and automagically does the right thing, Sebastian? > > > > He mentioned this in this message: > > > > https://marc.info/?l=linux-rt-users&m=150722087514307&w=2 > > > > But I replied to Mike and don't recall getting an answer to my last > > question. > > correct, since 4.1.7-rt8 > https://git.kernel.org/pub/scm/linux/kernel/git/rt/linux-rt-devel.git/commit/?h=v4.1.7-rt8-patches&id=7461758b9e982e4ea6280ce9308492e7cceda2ed Ok, I'll try backporting that change and will try without 2/2. - Arnaldo -- To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Em Wed, Oct 11, 2017 at 12:44:56PM +0200, Sebastian Andrzej Siewior escreveu: > On 2017-10-10 16:15:29 [-0300], Arnaldo Carvalho de Melo wrote: > > This may not even be needed, i.e. Sebastian Siewior said that recent > > rt patches already identify the way this driver uses threaded interrupts > > and automagically does the right thing, Sebastian? > > > > He mentioned this in this message: > > > > https://marc.info/?l=linux-rt-users&m=150722087514307&w=2 > > > > But I replied to Mike and don't recall getting an answer to my last > > question. > > correct, since 4.1.7-rt8 > https://git.kernel.org/pub/scm/linux/kernel/git/rt/linux-rt-devel.git/commit/?h=v4.1.7-rt8-patches&id=7461758b9e982e4ea6280ce9308492e7cceda2ed Ouch, it seems I looked at the wrong place, as patches-4.11.12-rt14 doesn't have this one, right? Off I go trying to make sense of this... - Arnaldo -- To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Em Wed, Oct 11, 2017 at 04:07:45PM -0300, Arnaldo Carvalho de Melo escreveu: > Em Wed, Oct 11, 2017 at 12:44:56PM +0200, Sebastian Andrzej Siewior escreveu: > > On 2017-10-10 16:15:29 [-0300], Arnaldo Carvalho de Melo wrote: > > > This may not even be needed, i.e. Sebastian Siewior said that recent > > > rt patches already identify the way this driver uses threaded interrupts > > > and automagically does the right thing, Sebastian? > > > > > > He mentioned this in this message: > > > > > > https://marc.info/?l=linux-rt-users&m=150722087514307&w=2 > > > > > > But I replied to Mike and don't recall getting an answer to my last > > > question. > > > > correct, since 4.1.7-rt8 > > https://git.kernel.org/pub/scm/linux/kernel/git/rt/linux-rt-devel.git/commit/?h=v4.1.7-rt8-patches&id=7461758b9e982e4ea6280ce9308492e7cceda2ed > > Ouch, it seems I looked at the wrong place, as patches-4.11.12-rt14 > doesn't have this one, right? > > Off I go trying to make sense of this... Ok, this was merged upstream on v4.4: [acme@jouet linux]$ git log -1 --oneline 2a1d3ab8986d1 2a1d3ab8986d genirq: Handle force threading of irqs with primary and thread handler nevermind :-) - Arnaldo -- To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
diff --git a/drivers/infiniband/hw/hfi1/chip.c b/drivers/infiniband/hw/hfi1/chip.c index 121a4c920f1b..733a00d8ea4c 100644 --- a/drivers/infiniband/hw/hfi1/chip.c +++ b/drivers/infiniband/hw/hfi1/chip.c @@ -8226,15 +8226,17 @@ static irqreturn_t receive_context_interrupt(int irq, void *data) { struct hfi1_ctxtdata *rcd = data; struct hfi1_devdata *dd = rcd->dd; - int disposition; - int present; trace_hfi1_receive_interrupt(dd, rcd->ctxt); this_cpu_inc(*dd->int_counter); aspm_ctx_disable(rcd); +#ifdef CONFIG_PREEMPT_RT_FULL + return IRQ_WAKE_THREAD; +#else +{ /* receive interrupt remains blocked while processing packets */ - disposition = rcd->do_interrupt(rcd, 0); + int disposition = rcd->do_interrupt(rcd, 0), present; /* * Too many packets were seen while processing packets in this @@ -8257,6 +8259,8 @@ static irqreturn_t receive_context_interrupt(int irq, void *data) return IRQ_HANDLED; } +#endif +} /* * Receive packet thread handler. This expects to be invoked with the