[2/4] locking/ww_mutex: Remove redundant wakeups in ww_mutex_set_context_slowpath

Commit Message

Nicolai Hähnle Nov. 23, 2016, 11:25 a.m. UTC

From: Nicolai Hähnle <[e-mail]>

When ww_mutex_set_context_slowpath runs, we are in one of two situations:

1. The current task was woken up by ww_mutex_unlock.

2. The current task is racing with ww_mutex_unlock: We entered the slow
   path while lock->base.count <= 0, but skipped the wait in
   __mutex_lock_common.

In both cases, all tasks that are currently contending for the lock are
either racing as in point 2 and blocked on lock->wait_lock, or they have
been or will be woken up by ww_mutex_unlock.

Either way, they will check their stamp against ours without having to be
woken up again.

This improvement is possible only after the changed behavior of
ww_mutex_unlock from the previous patch ("locking/ww_mutex: Fix a deadlock
affecting ww_mutexes").

Cc: Peter Zijlstra <[e-mail]>
Cc: Ingo Molnar <[e-mail]>
Cc: Chris Wilson <[e-mail]>
Cc: Maarten Lankhorst <[e-mail]>
Cc: [e-mail]
Signed-off-by: Nicolai Hähnle <[e-mail]>
---
 kernel/locking/mutex.c | 17 ++++-------------
 1 file changed, 4 insertions(+), 13 deletions(-)

Patch

diff --git a/kernel/locking/mutex.c b/kernel/locking/mutex.c
index 7fbf9b4..7c09376 100644
--- a/kernel/locking/mutex.c
+++ b/kernel/locking/mutex.c
@@ -192,8 +192,10 @@  ww_mutex_set_context_fastpath(struct ww_mutex *lock,
 }
 
 /*
- * After acquiring lock in the slowpath set ctx and wake up any
- * waiters so they can recheck.
+ * After acquiring lock in the slowpath set ctx.
+ *
+ * Unlike the fast path, other waiters are already woken up by ww_mutex_unlock,
+ * so we don't have to do it again here.
  *
  * Callers must hold the mutex wait_lock.
  */
@@ -201,19 +203,8 @@  static __always_inline void
 ww_mutex_set_context_slowpath(struct ww_mutex *lock,
 			      struct ww_acquire_ctx *ctx)
 {
-	struct mutex_waiter *cur;
-
 	ww_mutex_lock_acquired(lock, ctx);
 	lock->ctx = ctx;
-
-	/*
-	 * Give any possible sleeping processes the chance to wake up,
-	 * so they can recheck if they have to back off.
-	 */
-	list_for_each_entry(cur, &lock->base.wait_list, list) {
-		debug_mutex_wake_waiter(&lock->base, cur);
-		wake_up_process(cur->task);
-	}
 }
 
 #ifdef CONFIG_MUTEX_SPIN_ON_OWNER