From patchwork Wed Mar 14 15:15:10 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yishai Hadas X-Patchwork-Id: 10282425 X-Patchwork-Delegate: leon@leon.nu Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork.web.codeaurora.org (Postfix) with ESMTP id B38B6602BD for ; Wed, 14 Mar 2018 15:15:53 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id A0ED828A45 for ; Wed, 14 Mar 2018 15:15:53 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 9403B28ABA; Wed, 14 Mar 2018 15:15:53 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-6.9 required=2.0 tests=BAYES_00, RCVD_IN_DNSWL_HI, UNPARSEABLE_RELAY autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 61C4528AA5 for ; Wed, 14 Mar 2018 15:15:52 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1751362AbeCNPPu (ORCPT ); Wed, 14 Mar 2018 11:15:50 -0400 Received: from mail-il-dmz.mellanox.com ([193.47.165.129]:40557 "EHLO mellanox.co.il" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S1751348AbeCNPPt (ORCPT ); Wed, 14 Mar 2018 11:15:49 -0400 Received: from Internal Mail-Server by MTLPINE1 (envelope-from yishaih@mellanox.com) with ESMTPS (AES256-SHA encrypted); 14 Mar 2018 17:16:21 +0200 Received: from vnc17.mtl.labs.mlnx (vnc17.mtl.labs.mlnx [10.7.2.17]) by labmailer.mlnx (8.13.8/8.13.8) with ESMTP id w2EFFhWk002375; Wed, 14 Mar 2018 17:15:43 +0200 Received: from vnc17.mtl.labs.mlnx (vnc17.mtl.labs.mlnx [127.0.0.1]) by vnc17.mtl.labs.mlnx (8.13.8/8.13.8) with ESMTP id w2EFFhl6000625; Wed, 14 Mar 2018 17:15:43 +0200 Received: (from yishaih@localhost) by vnc17.mtl.labs.mlnx (8.13.8/8.13.8/Submit) id w2EFFhsq000624; Wed, 14 Mar 2018 17:15:43 +0200 From: Yishai Hadas To: linux-rdma@vger.kernel.org Cc: yishaih@mellanox.com, bodong@mellanox.com, jgg@mellanox.com, majd@mellanox.com, Alex Rosenbaum Subject: [PATCH rdma-core 2/2] mlx5: Support ibv_modify_qp_rate_limit Date: Wed, 14 Mar 2018 17:15:10 +0200 Message-Id: <1521040510-447-3-git-send-email-yishaih@mellanox.com> X-Mailer: git-send-email 1.8.2.3 In-Reply-To: <1521040510-447-1-git-send-email-yishaih@mellanox.com> References: <1521040510-447-1-git-send-email-yishaih@mellanox.com> Sender: linux-rdma-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-rdma@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP From: Bodong Wang Implement the verb ibv_modify_qp_rate_limit to support configuration of burst control information. The command is done through the user verbs with some driver specific data as part of the UHW. In case the device doesn't support burst info or rate_limit is 0 an error will be returned. Signed-off-by: Bodong Wang Signed-off-by: Alex Rosenbaum Reviewed-by: Yishai Hadas --- providers/mlx5/mlx5-abi.h | 19 ++++++++++++++++++- providers/mlx5/mlx5.c | 1 + providers/mlx5/mlx5.h | 3 +++ providers/mlx5/verbs.c | 33 +++++++++++++++++++++++++++++++++ 4 files changed, 55 insertions(+), 1 deletion(-) diff --git a/providers/mlx5/mlx5-abi.h b/providers/mlx5/mlx5-abi.h index fe302d1..77b0868 100644 --- a/providers/mlx5/mlx5-abi.h +++ b/providers/mlx5/mlx5-abi.h @@ -314,9 +314,14 @@ struct mlx5_rss_caps { __u8 reserved[7]; }; +enum mlx5_ib_packet_pacing_cap_flags { + MLX5_IB_PP_SUPPORT_BURST = 1 << 0, +}; + struct mlx5_packet_pacing_caps { struct ibv_packet_pacing_caps caps; - __u32 reserved; + __u8 cap_flags; /* enum mlx5_ib_packet_pacing_cap_flags */ + __u8 reserved[3]; }; enum mlx5_mpw_caps { @@ -357,4 +362,16 @@ struct mlx5_modify_qp_resp_ex { __u32 dctn; }; +struct mlx5_ib_burst_info { + __u32 max_burst_sz; + __u16 typical_pkt_sz; +}; + +struct mlx5_ib_modify_qp { + struct ibv_modify_qp_ex ibv_cmd; + __u32 comp_mask; + struct mlx5_ib_burst_info burst_info; + __u8 reserved[6]; +}; + #endif /* MLX5_ABI_H */ diff --git a/providers/mlx5/mlx5.c b/providers/mlx5/mlx5.c index 55b466c..8ded23c 100644 --- a/providers/mlx5/mlx5.c +++ b/providers/mlx5/mlx5.c @@ -129,6 +129,7 @@ static const struct verbs_context_ops mlx5_ctx_common_ops = { .destroy_wq = mlx5_destroy_wq, .get_srq_num = mlx5_get_srq_num, .modify_cq = mlx5_modify_cq, + .modify_qp_rate_limit = mlx5_modify_qp_rate_limit, .modify_wq = mlx5_modify_wq, .open_xrcd = mlx5_open_xrcd, .post_srq_ops = mlx5_post_srq_ops, diff --git a/providers/mlx5/mlx5.h b/providers/mlx5/mlx5.h index 96d0ee3..9f640f7 100644 --- a/providers/mlx5/mlx5.h +++ b/providers/mlx5/mlx5.h @@ -298,6 +298,7 @@ struct mlx5_context { struct mlx5dv_sw_parsing_caps sw_parsing_caps; struct mlx5dv_striding_rq_caps striding_rq_caps; uint32_t tunnel_offloads_caps; + struct mlx5_packet_pacing_caps packet_pacing_caps; pthread_mutex_t dyn_bfregs_mutex; /* protects the dynamic bfregs allocation */ uint32_t num_dyn_bfregs; uint32_t *count_dyn_bfregs; @@ -751,6 +752,8 @@ int mlx5_query_qp(struct ibv_qp *qp, struct ibv_qp_attr *attr, struct ibv_qp_init_attr *init_attr); int mlx5_modify_qp(struct ibv_qp *qp, struct ibv_qp_attr *attr, int attr_mask); +int mlx5_modify_qp_rate_limit(struct ibv_qp *qp, + struct ibv_qp_rate_limit_attr *attr); int mlx5_destroy_qp(struct ibv_qp *qp); void mlx5_init_qp_indices(struct mlx5_qp *qp); void mlx5_init_rwq_indices(struct mlx5_rwq *rwq); diff --git a/providers/mlx5/verbs.c b/providers/mlx5/verbs.c index cab6547..a6899c5 100644 --- a/providers/mlx5/verbs.c +++ b/providers/mlx5/verbs.c @@ -2131,6 +2131,38 @@ int mlx5_modify_qp(struct ibv_qp *qp, struct ibv_qp_attr *attr, return ret; } +int mlx5_modify_qp_rate_limit(struct ibv_qp *qp, + struct ibv_qp_rate_limit_attr *attr) +{ + struct ibv_qp_attr qp_attr = {}; + struct ib_uverbs_ex_modify_qp_resp resp = {}; + struct mlx5_ib_modify_qp cmd = {}; + struct mlx5_context *mctx = to_mctx(qp->context); + int ret; + + if (attr->comp_mask) + return EINVAL; + + if ((attr->max_burst_sz || + attr->typical_pkt_sz) && + (!attr->rate_limit || + !(mctx->packet_pacing_caps.cap_flags & + MLX5_IB_PP_SUPPORT_BURST))) + return EINVAL; + + cmd.burst_info.max_burst_sz = attr->max_burst_sz; + cmd.burst_info.typical_pkt_sz = attr->typical_pkt_sz; + qp_attr.rate_limit = attr->rate_limit; + + ret = ibv_cmd_modify_qp_ex(qp, &qp_attr, IBV_QP_RATE_LIMIT, + &cmd.ibv_cmd, + sizeof(cmd.ibv_cmd), sizeof(cmd), + &resp, + sizeof(resp), sizeof(resp)); + + return ret; +} + #define RROCE_UDP_SPORT_MIN 0xC000 #define RROCE_UDP_SPORT_MAX 0xFFFF struct ibv_ah *mlx5_create_ah(struct ibv_pd *pd, struct ibv_ah_attr *attr) @@ -2603,6 +2635,7 @@ int mlx5_query_device_ex(struct ibv_context *context, mctx->sw_parsing_caps = resp.sw_parsing_caps; mctx->striding_rq_caps = resp.striding_rq_caps.caps; mctx->tunnel_offloads_caps = resp.tunnel_offloads_caps; + mctx->packet_pacing_caps = resp.packet_pacing_caps; if (resp.flags & MLX5_QUERY_DEV_RESP_FLAGS_CQE_128B_COMP) mctx->vendor_cap_flags |= MLX5_VENDOR_CAP_FLAGS_CQE_128B_COMP;