diff mbox

Use ib_drain_qp instead of ib_drain_rq in ib_srp

Message ID 6ed6633b-d4ff-4293-c270-42a58e89f076@sandisk.com (mailing list archive)
State Deferred
Headers show

Commit Message

Bart Van Assche Dec. 8, 2016, 12:54 a.m. UTC 
On 12/07/2016 03:34 PM, Max Gurtovoy wrote:
> they will work in case of srp (the only user for DIRECT approach in the
> ULP's), but don't you think it's a hard requirement for the caller to
> ensure that no ib_poll_cq can be called during ib_set_cq_poll_context ?
> There was some thought in the past to share cq's among ULP's that is
> currently on hold (in this approach we can't be sure other context don't
> call ib_poll_cq). In other hand we can avoid sharing DIRECT cq's.
> Anyway, I thought of passing "struct ib_drain_cqe" as an arg to drain
> function from ULP and wait for completion (with tmo) in the ULP level,
> after direct ib_process_cq_direct.
> I don't have an implementation yet, but I hope I explained my idea.

Hello Max,

How about the attached two patches? The approach of these patches should 
be compatible with CQ sharing.

Bart.
diff mbox

Patch

From 593d9c8eb7676486dfc7839f6adc5949ce8b4a3c Mon Sep 17 00:00:00 2001
From: Bart Van Assche <bart.vanassche@sandisk.com>
Date: Tue, 6 Dec 2016 08:49:11 -0800
Subject: [PATCH 2/2] IB/srp: Drain the send queue before destroying a QP

A quote from the IB spec:

However, if the Consumer does not wait for the Affiliated Asynchronous
Last WQE Reached Event, then WQE and Data Segment leakage may occur.
Therefore, it is good programming practice to tear down a QP that is
associated with an SRQ by using the following process:
* Put the QP in the Error State;
* wait for the Affiliated Asynchronous Last WQE Reached Event;
* either:
  * drain the CQ by invoking the Poll CQ verb and either wait for CQ
    to be empty or the number of Poll CQ operations has exceeded CQ
    capacity size; or
  * post another WR that completes on the same CQ and wait for this WR to return as a WC;
* and then invoke a Destroy QP or Reset QP.

Signed-off-by: Bart Van Assche <bart.vanassche@sandisk.com>
Cc: Christoph Hellwig <hch@lst.de>
Cc: Max Gurtovoy <maxg@mellanox.com>
---
 drivers/infiniband/ulp/srp/ib_srp.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/drivers/infiniband/ulp/srp/ib_srp.c b/drivers/infiniband/ulp/srp/ib_srp.c
index 6c8d6847f920..5cc19b2b70c1 100644
--- a/drivers/infiniband/ulp/srp/ib_srp.c
+++ b/drivers/infiniband/ulp/srp/ib_srp.c
@@ -472,7 +472,7 @@  static struct srp_fr_pool *srp_alloc_fr_pool(struct srp_target_port *target)
  */
 static void srp_destroy_qp(struct ib_qp *qp)
 {
-	ib_drain_rq(qp);
+	ib_drain_qp(qp);
 	ib_destroy_qp(qp);
 }
 
-- 
2.11.0