From patchwork Mon Aug 8 18:04:15 2016 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jarod Wilson X-Patchwork-Id: 9269101 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork.web.codeaurora.org (Postfix) with ESMTP id 6F8C460839 for ; Mon, 8 Aug 2016 18:04:31 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 63C0D2621F for ; Mon, 8 Aug 2016 18:04:31 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 525DB27FBC; Mon, 8 Aug 2016 18:04:31 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-6.9 required=2.0 tests=BAYES_00,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id A79D12621F for ; Mon, 8 Aug 2016 18:04:30 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1752176AbcHHSE3 (ORCPT ); Mon, 8 Aug 2016 14:04:29 -0400 Received: from mx1.redhat.com ([209.132.183.28]:45822 "EHLO mx1.redhat.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1752168AbcHHSE3 (ORCPT ); Mon, 8 Aug 2016 14:04:29 -0400 Received: from int-mx14.intmail.prod.int.phx2.redhat.com (int-mx14.intmail.prod.int.phx2.redhat.com [10.5.11.27]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id BE6A420279; Mon, 8 Aug 2016 18:04:28 +0000 (UTC) Received: from hp-dl360pgen8-07.khw.lab.eng.bos.redhat.com (hp-dl360pgen8-07.khw.lab.eng.bos.redhat.com [10.16.184.47]) by int-mx14.intmail.prod.int.phx2.redhat.com (8.14.4/8.14.4) with ESMTP id u78I4Rbv015065; Mon, 8 Aug 2016 14:04:28 -0400 From: Jarod Wilson To: linux-rdma@vger.kernel.org Cc: Jarod Wilson , Jason Gunthorpe , Yishai Hadas Subject: [PATCH v2 libmlx5] fix err return values to match ibv_post_send expectations Date: Mon, 8 Aug 2016 14:04:15 -0400 Message-Id: <1470679455-28911-1-git-send-email-jarod@redhat.com> In-Reply-To: <20160808173333.GA26622@obsidianresearch.com> References: <20160808173333.GA26622@obsidianresearch.com> X-Scanned-By: MIMEDefang 2.68 on 10.5.11.27 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.29]); Mon, 08 Aug 2016 18:04:28 +0000 (UTC) Sender: linux-rdma-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-rdma@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP The man page for ibv_post_send says: RETURN VALUE ibv_post_send() returns 0 on success, or the value of errno on failure (which indicates the failure reason). QEMU looks for the return value, and in the ENOMEM case, waits and retries, but with mlx5, it ends up dropping requests and hanging, because of the unexpected -1 return instead of ENOMEM. The fix is simple: set err = E instead of -1, and eliminate use of errno = in _mlx5_post_send, just have mlx5_post_send return the err from _mlx5_post_send instead. This fix has been confirmed to resolves the issues seen with QEMU. While we're at it, fix the MW_DEBUG code paths to no muck with errno either. v2: per discussion with Jason Gunthorpe, don't set errno in mlx5_post_send Reported-by: Dr. David Alan Gilbert Tested-by: Dr. David Alan Gilbert CC: Jason Gunthorpe CC: Yishai Hadas Signed-off-by: Jarod Wilson --- src/qp.c | 27 +++++++++------------------ 1 file changed, 9 insertions(+), 18 deletions(-) diff --git a/src/qp.c b/src/qp.c index 51e1176..38352e9 100644 --- a/src/qp.c +++ b/src/qp.c @@ -590,8 +590,7 @@ static inline int _mlx5_post_send(struct ibv_qp *ibqp, struct ibv_send_wr *wr, if (unlikely(wr->opcode < 0 || wr->opcode >= sizeof mlx5_ib_opcode / sizeof mlx5_ib_opcode[0])) { mlx5_dbg(fp, MLX5_DBG_QP_SEND, "bad opcode %d\n", wr->opcode); - errno = EINVAL; - err = -1; + err = EINVAL; *bad_wr = wr; goto out; } @@ -599,8 +598,7 @@ static inline int _mlx5_post_send(struct ibv_qp *ibqp, struct ibv_send_wr *wr, if (unlikely(mlx5_wq_overflow(&qp->sq, nreq, to_mcq(qp->ibv_qp->send_cq)))) { mlx5_dbg(fp, MLX5_DBG_QP_SEND, "work queue overflow\n"); - errno = ENOMEM; - err = -1; + err = ENOMEM; *bad_wr = wr; goto out; } @@ -608,8 +606,7 @@ static inline int _mlx5_post_send(struct ibv_qp *ibqp, struct ibv_send_wr *wr, if (unlikely(wr->num_sge > qp->sq.max_gs)) { mlx5_dbg(fp, MLX5_DBG_QP_SEND, "max gs exceeded %d (max = %d)\n", wr->num_sge, qp->sq.max_gs); - errno = ENOMEM; - err = -1; + err = ENOMEM; *bad_wr = wr; goto out; } @@ -899,22 +896,16 @@ int mlx5_post_send(struct ibv_qp *ibqp, struct ibv_send_wr *wr, { #ifdef MW_DEBUG if (wr->opcode == IBV_WR_BIND_MW) { - if (wr->bind_mw.mw->type == IBV_MW_TYPE_1) { - errno = EINVAL; - return errno; - } + if (wr->bind_mw.mw->type == IBV_MW_TYPE_1) + return EINVAL; if (!wr->bind_mw.bind_info.mr || !wr->bind_mw.bind_info.addr || - !wr->bind_mw.bind_info.length) { - errno = EINVAL; - return errno; - } + !wr->bind_mw.bind_info.length) + return EINVAL; - if (wr->bind_mw.bind_info.mr->pd != wr->bind_mw.mw->pd) { - errno = EINVAL; - return errno; - } + if (wr->bind_mw.bind_info.mr->pd != wr->bind_mw.mw->pd) + return EINVAL; } #endif