[REPOST,for-next,1/5] IB/iser: Drain the tx cq once before looping on the rx cq

Commit Message

Or Gerlitz April 1, 2014, 1:28 p.m. UTC

From: Roi Dayan <[e-mail]>

The iser disconnection flow isn't done before all the inflight recv/send buffers
posted to the QP are either flushed or normally completed to the CQ that
serves this connection. The condition check is done in iser_handle_comp_error().

Currently, its possible for the send buffer completion which makes the posted
send buffers counter to reach zero to be polled in the drain tx call which is
after the rx cq was fully drained. Since this completion might be not an error
one (e.g. of the logout request iSCSI PDU) we will skip iser_handle_comp_error()
so the connection will never terminate from the iscsi stack point of view,
and we hang.

To resolve this race, do the draining of the tx cq before the loop on the rx cq.

Issue: 375623

Change-Id: I38952f738ac0e3373f6651987136bcc0fed18b7c
Signed-off-by: Roi Dayan <[e-mail]>
Signed-off-by: Or Gerlitz <[e-mail]>
---
 drivers/infiniband/ulp/iser/iser_verbs.c |    9 ++++++---
 1 files changed, 6 insertions(+), 3 deletions(-)

Comments

Or Gerlitz April 1, 2014, 1:33 p.m. UTC | #1

On 01/04/2014 16:28, Or Gerlitz wrote:
> Issue: 375623
>
> Change-Id: I38952f738ac0e3373f6651987136bcc0fed18b7c
Roland, I see that in patches 1 and 2 I screwed it up and left notes 
from our internal system, care to clean this up before applying or you 
want me to re-send?
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to [e-mail]
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Patch

diff --git a/drivers/infiniband/ulp/iser/iser_verbs.c b/drivers/infiniband/ulp/iser/iser_verbs.c
index abbb6ec..a29f37e 100644
--- a/drivers/infiniband/ulp/iser/iser_verbs.c
+++ b/drivers/infiniband/ulp/iser/iser_verbs.c
@@ -1112,8 +1112,12 @@  static void iser_cq_tasklet_fn(unsigned long data)
 	 struct iser_rx_desc *desc;
 	 unsigned long	     xfer_len;
 	struct iser_conn *ib_conn;
-	int completed_tx, completed_rx;
-	completed_tx = completed_rx = 0;
+	int completed_tx, completed_rx = 0;
+
+	/* First do tx drain, so in a case where we have rx flushes and a successful
+	 * tx completion we will still go through completion error handling.
+	 */
+	completed_tx = iser_drain_tx_cq(device, cq_index);
 
 	while (ib_poll_cq(cq, 1, &wc) == 1) {
 		desc	 = (struct iser_rx_desc *) (unsigned long) wc.wr_id;
@@ -1141,7 +1145,6 @@  static void iser_cq_tasklet_fn(unsigned long data)
 	 * " would not cause interrupts to be missed"                       */
 	ib_req_notify_cq(cq, IB_CQ_NEXT_COMP);
 
-	completed_tx += iser_drain_tx_cq(device, cq_index);
 	iser_dbg("got %d rx %d tx completions\n", completed_rx, completed_tx);
 }