diff mbox series

[1/2] epoll: check ep_events_available() upon timeout

Message ID 20201028180202.952079-1-soheil.kdev@gmail.com (mailing list archive)
State New, archived
Headers show
Series [1/2] epoll: check ep_events_available() upon timeout | expand

Commit Message

Soheil Hassas Yeganeh Oct. 28, 2020, 6:02 p.m. UTC
From: Soheil Hassas Yeganeh <soheil@google.com>

After abc610e01c66, we break out of the ep_poll loop upon timeout,
without checking whether there is any new events available.  Prior to
that patch-series we always called ep_events_available() after
exiting the loop.

This can cause races and missed wakeups. For example, consider
the following scenario reported by Guantao Liu:

Suppose we have an eventfd added using EPOLLET to an epollfd.

Thread 1: Sleeps for just below 5ms and then writes to an eventfd.
Thread 2: Calls epoll_wait with a timeout of 5 ms. If it sees an
          event of the eventfd, it will write back on that fd.
Thread 3: Calls epoll_wait with a negative timeout.

Prior to abc610e01c66, it is guaranteed that Thread 3 will wake up
either by Thread 1 or Thread 2.  After abc610e01c66, Thread 3 can
be blocked indefinitely if Thread 2 sees a timeout right before
the write to the eventfd by Thread 1. Thread 2 will be woken up from
schedule_hrtimeout_range and, with evail 0, it will not call

To fix this issue, while holding the lock, try to remove the thread that
timed out the wait queue and check whether it was woken up or not.

Fixes: abc610e01c66 ("fs/epoll: avoid barrier after an epoll_wait(2) timeout")
Reported-by: Guantao Liu <guantaol@google.com>
Tested-by: Guantao Liu <guantaol@google.com>
Signed-off-by: Soheil Hassas Yeganeh <soheil@google.com>
Reviewed-by: Eric Dumazet <edumazet@google.com>
Acked-by: Willem de Bruijn <willemb@google.com>
Reviewed-by: Khazhismel Kumykov <khazhy@google.com>
Cc: Davidlohr Bueso <dave@stgolabs.net>
 fs/eventpoll.c | 16 +++++++++++++++-
 1 file changed, 15 insertions(+), 1 deletion(-)
diff mbox series


diff --git a/fs/eventpoll.c b/fs/eventpoll.c
index 4df61129566d..11388436b85a 100644
--- a/fs/eventpoll.c
+++ b/fs/eventpoll.c
@@ -1907,7 +1907,21 @@  static int ep_poll(struct eventpoll *ep, struct epoll_event __user *events,
 		if (!schedule_hrtimeout_range(to, slack, HRTIMER_MODE_ABS)) {
 			timed_out = 1;
-			break;
+			__set_current_state(TASK_RUNNING);
+			/*
+			 * Acquire the lock and try to remove this thread from
+			 * the wait queue. If this thread is not on the wait
+			 * queue, it has woken up after its timeout ended
+			 * before it could re-acquire the lock. In that case,
+			 * try to harvest some events.
+			 */
+			write_lock_irq(&ep->lock);
+			if (!list_empty(&wait.entry))
+				__remove_wait_queue(&ep->wq, &wait);
+			else
+				eavail = 1;
+			write_unlock_irq(&ep->lock);
+			goto send_events;
 		/* We were woken up, thus go and try to harvest some events */