diff mbox

[2/7] IB/rxe: Advance the consumer pointer before posting the CQE

Message ID 1479479809-10798-2-git-send-email-andrew.boyer@dell.com (mailing list archive)
State Superseded
Headers show

Commit Message

Andrew Boyer Nov. 18, 2016, 2:36 p.m. UTC
A simple userspace application might poll the CQ, find a completion,
and then attempt to post a new WQE to the SQ. A spurious error can
occur if the userspace application detects a full SQ in the instant
before the kernel is able to advance the SQ consumer pointer.

This is noticeable when using single-entry SQs with ibv_rc_pingpong()
if lots of kernel and userspace library debugging is enabled.

Signed-off-by: Andrew Boyer <andrew.boyer@dell.com>
---
 drivers/infiniband/sw/rxe/rxe_comp.c | 5 +++--
 1 file changed, 3 insertions(+), 2 deletions(-)

Comments

yonatan cohen Nov. 22, 2016, 3:24 p.m. UTC | #1
On 11/18/2016 4:36 PM, Andrew Boyer wrote:
> A simple userspace application might poll the CQ, find a completion,
> and then attempt to post a new WQE to the SQ. A spurious error can
> occur if the userspace application detects a full SQ in the instant
> before the kernel is able to advance the SQ consumer pointer.
>
> This is noticeable when using single-entry SQs with ibv_rc_pingpong()
> if lots of kernel and userspace library debugging is enabled.
>
> Signed-off-by: Andrew Boyer <andrew.boyer@dell.com>
> ---
>  drivers/infiniband/sw/rxe/rxe_comp.c | 5 +++--
>  1 file changed, 3 insertions(+), 2 deletions(-)
>
> diff --git a/drivers/infiniband/sw/rxe/rxe_comp.c b/drivers/infiniband/sw/rxe/rxe_comp.c
> index 6c5e29d..d46c49b 100644
> --- a/drivers/infiniband/sw/rxe/rxe_comp.c
> +++ b/drivers/infiniband/sw/rxe/rxe_comp.c
> @@ -420,11 +420,12 @@ static void do_complete(struct rxe_qp *qp, struct rxe_send_wqe *wqe)
>  	    (wqe->wr.send_flags & IB_SEND_SIGNALED) ||
>  	    (qp->req.state == QP_STATE_ERROR)) {
>  		make_send_cqe(qp, wqe, &cqe);
> +		advance_consumer(qp->sq.queue);
>  		rxe_cq_post(qp->scq, &cqe, 0);
> +	} else {
> +		advance_consumer(qp->sq.queue);
>  	}
>
> -	advance_consumer(qp->sq.queue);
> -
>  	/*
>  	 * we completed something so let req run again
>  	 * if it is trying to fence
>
Reviewed-by: Yonatan Cohen <yonatanc@mellanox.com>
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
diff mbox

Patch

diff --git a/drivers/infiniband/sw/rxe/rxe_comp.c b/drivers/infiniband/sw/rxe/rxe_comp.c
index 6c5e29d..d46c49b 100644
--- a/drivers/infiniband/sw/rxe/rxe_comp.c
+++ b/drivers/infiniband/sw/rxe/rxe_comp.c
@@ -420,11 +420,12 @@  static void do_complete(struct rxe_qp *qp, struct rxe_send_wqe *wqe)
 	    (wqe->wr.send_flags & IB_SEND_SIGNALED) ||
 	    (qp->req.state == QP_STATE_ERROR)) {
 		make_send_cqe(qp, wqe, &cqe);
+		advance_consumer(qp->sq.queue);
 		rxe_cq_post(qp->scq, &cqe, 0);
+	} else {
+		advance_consumer(qp->sq.queue);
 	}
 
-	advance_consumer(qp->sq.queue);
-
 	/*
 	 * we completed something so let req run again
 	 * if it is trying to fence