diff mbox series

[1/3] blk-mq: Don't complete on a remote CPU in force threaded mode

Message ID 20201204191356.2516405-2-bigeasy@linutronix.de (mailing list archive)
State New, archived
Headers show
Series blk-mq: Don't complete in IRQ, use llist_head | expand

Commit Message

Sebastian Andrzej Siewior Dec. 4, 2020, 7:13 p.m. UTC
With force threaded interrupts enabled, raising softirq from an SMP
function call will always result in waking the ksoftirqd thread. This is
not optimal given that the thread runs at SCHED_OTHER priority.

Completing the request in hard IRQ-context on PREEMPT_RT (which enforces
the force threaded mode) is bad because the completion handler may
acquire sleeping locks which violate the locking context.

Disable request completing on a remote CPU in force threaded mode.

Signed-off-by: Sebastian Andrzej Siewior <bigeasy@linutronix.de>
---
 block/blk-mq.c | 8 ++++++++
 1 file changed, 8 insertions(+)

Comments

Christoph Hellwig Dec. 8, 2020, 1:10 p.m. UTC | #1
On Fri, Dec 04, 2020 at 08:13:54PM +0100, Sebastian Andrzej Siewior wrote:
> With force threaded interrupts enabled, raising softirq from an SMP
> function call will always result in waking the ksoftirqd thread. This is
> not optimal given that the thread runs at SCHED_OTHER priority.
> 
> Completing the request in hard IRQ-context on PREEMPT_RT (which enforces
> the force threaded mode) is bad because the completion handler may
> acquire sleeping locks which violate the locking context.
> 
> Disable request completing on a remote CPU in force threaded mode.
> 
> Signed-off-by: Sebastian Andrzej Siewior <bigeasy@linutronix.de>

Looks good,

Reviewed-by: Christoph Hellwig <hch@lst.de>
diff mbox series

Patch

diff --git a/block/blk-mq.c b/block/blk-mq.c
index b9c2efab5db78..7091bb097c63f 100644
--- a/block/blk-mq.c
+++ b/block/blk-mq.c
@@ -650,6 +650,14 @@  static inline bool blk_mq_complete_need_ipi(struct request *rq)
 	if (!IS_ENABLED(CONFIG_SMP) ||
 	    !test_bit(QUEUE_FLAG_SAME_COMP, &rq->q->queue_flags))
 		return false;
+	/*
+	 * With force threaded interrupts enabled, raising softirq from an SMP
+	 * function call will always result in waking the ksoftirqd thread.
+	 * This is probably worse than completing the request on a different
+	 * cache domain.
+	 */
+	if (force_irqthreads)
+		return false;
 
 	/* same CPU or cache domain?  Complete locally */
 	if (cpu == rq->mq_ctx->cpu ||