From patchwork Sun Sep 23 07:51:14 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Lijun Ou X-Patchwork-Id: 10611609 X-Patchwork-Delegate: jgg@ziepe.ca Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 08349112B for ; Sun, 23 Sep 2018 07:53:26 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id EE9502874F for ; Sun, 23 Sep 2018 07:53:25 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id E1D19287D4; Sun, 23 Sep 2018 07:53:25 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.9 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 58EBD2874F for ; Sun, 23 Sep 2018 07:53:25 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726050AbeIWNt5 (ORCPT ); Sun, 23 Sep 2018 09:49:57 -0400 Received: from szxga07-in.huawei.com ([45.249.212.35]:48323 "EHLO huawei.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S1726029AbeIWNt5 (ORCPT ); Sun, 23 Sep 2018 09:49:57 -0400 Received: from DGGEMS413-HUB.china.huawei.com (unknown [172.30.72.58]) by Forcepoint Email with ESMTP id 079BD75AD569E; Sun, 23 Sep 2018 15:53:21 +0800 (CST) Received: from localhost.localdomain (10.67.212.75) by DGGEMS413-HUB.china.huawei.com (10.3.19.213) with Microsoft SMTP Server id 14.3.399.0; Sun, 23 Sep 2018 15:53:19 +0800 From: Lijun Ou To: , CC: , Subject: [PATCH V2 rdma-core] libhns: Add atomic support for hip08 user mode Date: Sun, 23 Sep 2018 15:51:14 +0800 Message-ID: <1537689074-61049-1-git-send-email-oulijun@huawei.com> X-Mailer: git-send-email 1.9.1 MIME-Version: 1.0 X-Originating-IP: [10.67.212.75] X-CFilter-Loop: Reflected Sender: linux-rdma-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-rdma@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP It needs to post the work request of atomic type when the run atomic function by the hardware supported. it incldues fetchadd and cmpswap operation. Signed-off-by: Lijun Ou --- V1->V2: - Check by travis CI --- providers/hns/hns_roce_u_hw_v2.c | 28 +++++++++++++++++++++++++++- providers/hns/hns_roce_u_hw_v2.h | 5 +++++ 2 files changed, 32 insertions(+), 1 deletion(-) diff --git a/providers/hns/hns_roce_u_hw_v2.c b/providers/hns/hns_roce_u_hw_v2.c index bc2ef47..60d8294 100644 --- a/providers/hns/hns_roce_u_hw_v2.c +++ b/providers/hns/hns_roce_u_hw_v2.c @@ -45,6 +45,18 @@ static void set_data_seg_v2(struct hns_roce_v2_wqe_data_seg *dseg, dseg->len = htole32(sg->length); } +static void set_atomic_seg(struct hns_roce_wqe_atomic_seg *aseg, + struct ibv_send_wr *wr) +{ + if (wr->opcode == IBV_WR_ATOMIC_CMP_AND_SWP) { + aseg->fetchadd_swap_data = htole64(wr->wr.atomic.swap); + aseg->cmp_data = htole64(wr->wr.atomic.compare_add); + } else { + aseg->fetchadd_swap_data = htole64(wr->wr.atomic.compare_add); + aseg->cmp_data = 0; + } +} + static void hns_roce_v2_handle_error_cqe(struct hns_roce_v2_cqe *cqe, struct ibv_wc *wc) { @@ -680,6 +692,11 @@ static int hns_roce_u_v2_post_send(struct ibv_qp *ibvqp, struct ibv_send_wr *wr, RC_SQ_WQE_BYTE_4_OPCODE_M, RC_SQ_WQE_BYTE_4_OPCODE_S, HNS_ROCE_WQE_OP_ATOMIC_COM_AND_SWAP); + rc_sq_wqe->rkey = htole32(wr->wr.atomic.rkey); + rc_sq_wqe->va = + htole32(wr->wr.atomic.remote_addr); + wqe += sizeof(struct hns_roce_v2_wqe_data_seg); + set_atomic_seg(wqe, wr); break; case IBV_WR_ATOMIC_FETCH_AND_ADD: @@ -687,6 +704,11 @@ static int hns_roce_u_v2_post_send(struct ibv_qp *ibvqp, struct ibv_send_wr *wr, RC_SQ_WQE_BYTE_4_OPCODE_M, RC_SQ_WQE_BYTE_4_OPCODE_S, HNS_ROCE_WQE_OP_ATOMIC_FETCH_AND_ADD); + rc_sq_wqe->rkey = htole32(wr->wr.atomic.rkey); + rc_sq_wqe->va = + htole32(wr->wr.atomic.remote_addr); + wqe += sizeof(struct hns_roce_v2_wqe_data_seg); + set_atomic_seg(wqe, wr); break; default: roce_set_field(rc_sq_wqe->byte_4, @@ -705,7 +727,11 @@ static int hns_roce_u_v2_post_send(struct ibv_qp *ibvqp, struct ibv_send_wr *wr, break; } - dseg = wqe; + if (wr->opcode == IBV_WR_ATOMIC_FETCH_AND_ADD || + wr->opcode == IBV_WR_ATOMIC_CMP_AND_SWP) + dseg = wqe - sizeof(struct hns_roce_v2_wqe_data_seg); + else + dseg = wqe; /* Inline */ if (wr->send_flags & IBV_SEND_INLINE && wr->num_sge) { diff --git a/providers/hns/hns_roce_u_hw_v2.h b/providers/hns/hns_roce_u_hw_v2.h index 0cb264f..99c7b99 100644 --- a/providers/hns/hns_roce_u_hw_v2.h +++ b/providers/hns/hns_roce_u_hw_v2.h @@ -275,4 +275,9 @@ struct hns_roce_v2_wqe_raddr_seg { __le64 raddr; }; +struct hns_roce_wqe_atomic_seg { + __le64 fetchadd_swap_data; + __le64 cmp_data; +}; + #endif /* _HNS_ROCE_U_HW_V2_H */