From patchwork Thu Feb 16 19:23:08 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jason Gunthorpe X-Patchwork-Id: 9578275 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork.web.codeaurora.org (Postfix) with ESMTP id A7D3E60244 for ; Thu, 16 Feb 2017 19:53:23 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 9C72A28667 for ; Thu, 16 Feb 2017 19:53:23 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 915392866B; Thu, 16 Feb 2017 19:53:23 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-6.8 required=2.0 tests=BAYES_00,DKIM_SIGNED, RCVD_IN_DNSWL_HI,T_DKIM_INVALID autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 140E928667 for ; Thu, 16 Feb 2017 19:53:23 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S933035AbdBPTxW (ORCPT ); Thu, 16 Feb 2017 14:53:22 -0500 Received: from quartz.orcorp.ca ([184.70.90.242]:38936 "EHLO quartz.orcorp.ca" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S932933AbdBPTxV (ORCPT ); Thu, 16 Feb 2017 14:53:21 -0500 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=obsidianresearch.com; s=rsa1; h=References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From; bh=gxZYUwxfAv3tgLy6Ngg5vE+4xC+6zWOhOGmhsbo/8aM=; b=Rxn+1TeQku7Yw6DHFku7xQJawXz+G+bh1ljQaer5ClaKzBTZCTebEEw8rD6wKX3ZnPasxMaqSSrKP64NDoy13ire6Wa1Skbzn/5vsdhfzMiMdOCs15G37qL4hjjFwrTxC4B0cHfLcEO8HgiQvcku7qZbBsKIJ8dRsJRRZ4vapAY=; Received: from [10.0.0.156] (helo=jggl.edm.orcorp.ca) by quartz.orcorp.ca with esmtps (TLS1.2:ECDHE_RSA_AES_128_CBC_SHA256:128) (Exim 4.84_2) (envelope-from ) id 1ceReR-00010K-1M; Thu, 16 Feb 2017 12:23:35 -0700 From: Jason Gunthorpe To: linux-rdma@vger.kernel.org Cc: Adit Ranadive Subject: [PATCH rdma-core 13/14] vmw_pvrdma: Update to use new udma write barriers Date: Thu, 16 Feb 2017 12:23:08 -0700 Message-Id: <1487272989-8215-14-git-send-email-jgunthorpe@obsidianresearch.com> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1487272989-8215-1-git-send-email-jgunthorpe@obsidianresearch.com> References: <1487272989-8215-1-git-send-email-jgunthorpe@obsidianresearch.com> X-Broken-Reverse-DNS: no host name found for IP address 10.0.0.156 Sender: linux-rdma-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-rdma@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP For some reason write barriers were placed after the writes, move them before. Signed-off-by: Jason Gunthorpe Acked-by: Adit Ranadive --- providers/vmw_pvrdma/cq.c | 6 +++--- providers/vmw_pvrdma/qp.c | 8 ++++---- 2 files changed, 7 insertions(+), 7 deletions(-) diff --git a/providers/vmw_pvrdma/cq.c b/providers/vmw_pvrdma/cq.c index f24d80742678bd..701f0522f7b0dd 100644 --- a/providers/vmw_pvrdma/cq.c +++ b/providers/vmw_pvrdma/cq.c @@ -109,7 +109,7 @@ retry: if (!cqe) return CQ_EMPTY; - rmb(); + udma_from_device_barrier(); if (ctx->qp_tbl[cqe->qp & 0xFFFF]) *cur_qp = (struct pvrdma_qp *)ctx->qp_tbl[cqe->qp & 0xFFFF]; @@ -184,11 +184,11 @@ void pvrdma_cq_clean_int(struct pvrdma_cq *cq, uint32_t qpn) if (tail < 0) tail = cq->cqe_cnt - 1; curr_cqe = get_cqe(cq, curr); - rmb(); + udma_from_device_barrier(); if ((curr_cqe->qp & 0xFFFF) != qpn) { if (curr != tail) { cqe = get_cqe(cq, tail); - rmb(); + udma_from_device_barrier(); *cqe = *curr_cqe; } tail--; diff --git a/providers/vmw_pvrdma/qp.c b/providers/vmw_pvrdma/qp.c index d2e2189fda6de4..116063ee07c83b 100644 --- a/providers/vmw_pvrdma/qp.c +++ b/providers/vmw_pvrdma/qp.c @@ -404,11 +404,10 @@ int pvrdma_post_send(struct ibv_qp *ibqp, struct ibv_send_wr *wr, sge++; } + udma_to_device_barrier(); pvrdma_idx_ring_inc(&(qp->sq.ring_state->prod_tail), qp->sq.wqe_cnt); - wmb(); - qp->sq.wrid[ind] = wr->wr_id; ++ind; if (ind >= qp->sq.wqe_cnt) @@ -416,11 +415,12 @@ int pvrdma_post_send(struct ibv_qp *ibqp, struct ibv_send_wr *wr, } out: - if (nreq) + if (nreq) { + udma_to_device_barrier(); pvrdma_write_uar_qp(ctx->uar, PVRDMA_UAR_QP_SEND | ibqp->qp_num); + } - wmb(); pthread_spin_unlock(&qp->sq.lock); return ret;