From patchwork Mon Dec 17 13:16:28 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Potnuri Bharat Teja X-Patchwork-Id: 10733461 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 5362A1399 for ; Mon, 17 Dec 2018 13:17:27 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 4510929217 for ; Mon, 17 Dec 2018 13:17:27 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 39939292AF; Mon, 17 Dec 2018 13:17:27 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.9 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 2F7A629217 for ; Mon, 17 Dec 2018 13:17:26 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727580AbeLQNR0 (ORCPT ); Mon, 17 Dec 2018 08:17:26 -0500 Received: from stargate.chelsio.com ([12.32.117.8]:13714 "EHLO stargate.chelsio.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726415AbeLQNRZ (ORCPT ); Mon, 17 Dec 2018 08:17:25 -0500 Received: from localhost (mehrangarh.blr.asicdesigners.com [10.193.185.169]) by stargate.chelsio.com (8.13.8/8.13.8) with ESMTP id wBHDHFZ3007813; Mon, 17 Dec 2018 05:17:16 -0800 From: Potnuri Bharat Teja To: dledford@redhat.com, jgg@mellanox.com Cc: linux-rdma@vger.kernel.org, bharat@chelsio.com Subject: [PATCH rdma-core 2/2] cxgb4: Post write with completion WR for write and send with invalidate WR Date: Mon, 17 Dec 2018 18:46:28 +0530 Message-Id: <20181217131628.32697-3-bharat@chelsio.com> X-Mailer: git-send-email 2.18.0.232.gb7bd9486b055 In-Reply-To: <20181217131628.32697-1-bharat@chelsio.com> References: <20181217131628.32697-1-bharat@chelsio.com> Sender: linux-rdma-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-rdma@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP Post a Write with completion for write WR followed by a send with invalidate WR as well. Signed-off-by: Potnuri Bharat Teja --- providers/cxgb4/qp.c | 19 +++++++++++++------ 1 file changed, 13 insertions(+), 6 deletions(-) diff --git a/providers/cxgb4/qp.c b/providers/cxgb4/qp.c index 6f341a787d93..11a411393821 100644 --- a/providers/cxgb4/qp.c +++ b/providers/cxgb4/qp.c @@ -311,7 +311,10 @@ static void build_rdma_write_cmpl(struct t4_sq *sq, wcwr->stag_sink = htobe32(wr->wr.rdma.rkey); wcwr->to_sink = htobe64(wr->wr.rdma.remote_addr); - wcwr->stag_inv = 0; + if (wr->next->opcode == IBV_WR_SEND) + wcwr->stag_inv = 0; + else + wcwr->stag_inv = htobe32(wr->next->invalidate_rkey); wcwr->r2 = 0; wcwr->r3 = 0; @@ -446,7 +449,10 @@ static void post_write_cmpl(struct c4iw_qp *qhp, struct ibv_send_wr *wr) /* SEND swsqe */ swsqe = &qhp->wq.sq.sw_sq[qhp->wq.sq.pidx]; - swsqe->opcode = FW_RI_SEND; + if (wr->next->opcode == IBV_WR_SEND) + swsqe->opcode = FW_RI_SEND; + else + swsqe->opcode = FW_RI_SEND_WITH_INV; swsqe->idx = qhp->wq.sq.pidx; swsqe->complete = 0; swsqe->signaled = send_signaled; @@ -495,9 +501,9 @@ int c4iw_post_send(struct ibv_qp *ibqp, struct ibv_send_wr *wr, /* * Fastpath for NVMe-oF target WRITE + SEND_WITH_INV wr chain which is * the response for small NVMEe-oF READ requests. If the chain is - * exactly a WRITE->SEND_WITH_INV and the sgl depths and lengths - * meet the requirements of the fw_ri_write_cmpl_wr work request, - * then build and post the write_cmpl WR. If any of the tests + * exactly a WRITE->SEND_WITH_INV or a WRITE->SEND and the sgl depths + * and lengths meet the requirements of the fw_ri_write_cmpl_wr work + * request, then build and post the write_cmpl WR. If any of the tests * below are not true, then we continue on with the tradtional WRITE * and SEND WRs. */ @@ -506,7 +512,8 @@ int c4iw_post_send(struct ibv_qp *ibqp, struct ibv_send_wr *wr, wr && wr->next && !wr->next->next && wr->opcode == IBV_WR_RDMA_WRITE && wr->sg_list[0].length && wr->num_sge <= T4_WRITE_CMPL_MAX_SGL && - wr->next->opcode == IBV_WR_SEND && + (wr->next->opcode == IBV_WR_SEND_WITH_INV || + wr->next->opcode == IBV_WR_SEND) && wr->next->sg_list[0].length == T4_WRITE_CMPL_MAX_CQE && wr->next->num_sge == 1 && num_wrs >= 2) { post_write_cmpl(qhp, wr);