From patchwork Thu May 17 10:39:30 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yishai Hadas X-Patchwork-Id: 10406321 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork.web.codeaurora.org (Postfix) with ESMTP id C6001602C2 for ; Thu, 17 May 2018 10:40:15 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id A204128A5D for ; Thu, 17 May 2018 10:40:15 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id A068028A75; Thu, 17 May 2018 10:40:15 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.9 required=2.0 tests=BAYES_00, MAILING_LIST_MULTI, RCVD_IN_DNSWL_HI,UNPARSEABLE_RELAY autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 0713328A5D for ; Thu, 17 May 2018 10:40:11 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1751626AbeEQKkJ (ORCPT ); Thu, 17 May 2018 06:40:09 -0400 Received: from mail-il-dmz.mellanox.com ([193.47.165.129]:44999 "EHLO mellanox.co.il" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S1751671AbeEQKkD (ORCPT ); Thu, 17 May 2018 06:40:03 -0400 Received: from Internal Mail-Server by MTLPINE1 (envelope-from yishaih@mellanox.com) with ESMTPS (AES256-SHA encrypted); 17 May 2018 13:41:47 +0300 Received: from vnc17.mtl.labs.mlnx (vnc17.mtl.labs.mlnx [10.7.2.17]) by labmailer.mlnx (8.13.8/8.13.8) with ESMTP id w4HAduCE002330; Thu, 17 May 2018 13:39:56 +0300 Received: from vnc17.mtl.labs.mlnx (vnc17.mtl.labs.mlnx [127.0.0.1]) by vnc17.mtl.labs.mlnx (8.13.8/8.13.8) with ESMTP id w4HAdu42032395; Thu, 17 May 2018 13:39:56 +0300 Received: (from yishaih@localhost) by vnc17.mtl.labs.mlnx (8.13.8/8.13.8/Submit) id w4HAduPE032394; Thu, 17 May 2018 13:39:56 +0300 From: Yishai Hadas To: linux-rdma@vger.kernel.org Cc: yishaih@mellanox.com, raeds@mellanox.com Subject: [PATCH rdma-core 04/11] verbs: Introduce counters attach point flow Date: Thu, 17 May 2018 13:39:30 +0300 Message-Id: <1526553577-32273-5-git-send-email-yishaih@mellanox.com> X-Mailer: git-send-email 1.8.2.3 In-Reply-To: <1526553577-32273-1-git-send-email-yishaih@mellanox.com> References: <1526553577-32273-1-git-send-email-yishaih@mellanox.com> Sender: linux-rdma-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-rdma@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP From: Raed Salem Attach individual counter description definition to a flow verb object at a specific index location. This counters object will start collecting values after it is bound to the flow verbs resource. A static binding of counters to a flow can be done when referencing the ibv_counters object when creating a new flow with ibv_create_flow(), whereas dynamic binding is done post flow object creation during its lifetime. In a static binding no additional counter_description can be attached to this ibv_counters object once it was bound to a verb object, attempting to do so will result in an EBUSY error. Supported capabilities of specific counter_description values per verbs object can be tested by checking the return value for success or ENOTSUP errno. Attaching a counters handle to multiple objects of the same type will accumulate the values into a single index, e.g.: creating several ibv_flow(s) with the same ibv_counters handle will collect the values from all relevant flows into the relevant index location when reading the values from ibv_read_counters(). Downstream patch will introduce the read counter verb where the runtime values of counters can be read from the hardware. Signed-off-by: Raed Salem Signed-off-by: Yishai Hadas --- libibverbs/driver.h | 3 + libibverbs/dummy_ops.c | 9 ++ libibverbs/man/CMakeLists.txt | 1 + libibverbs/man/ibv_attach_counters_point_flow.3.md | 134 +++++++++++++++++++++ libibverbs/verbs.h | 27 +++++ 5 files changed, 174 insertions(+) create mode 100644 libibverbs/man/ibv_attach_counters_point_flow.3.md diff --git a/libibverbs/driver.h b/libibverbs/driver.h index 11a6d08..1030b0c 100644 --- a/libibverbs/driver.h +++ b/libibverbs/driver.h @@ -211,6 +211,9 @@ struct verbs_context_ops { struct ibv_td *(*alloc_td)(struct ibv_context *context, struct ibv_td_init_attr *init_attr); void (*async_event)(struct ibv_async_event *event); + int (*attach_counters_point_flow)(struct ibv_counters *counters, + struct ibv_counter_attach_attr *attr, + struct ibv_flow *flow); int (*attach_mcast)(struct ibv_qp *qp, const union ibv_gid *gid, uint16_t lid); int (*bind_mw)(struct ibv_qp *qp, struct ibv_mw *mw, diff --git a/libibverbs/dummy_ops.c b/libibverbs/dummy_ops.c index 3f4d641..1ccb5b3 100644 --- a/libibverbs/dummy_ops.c +++ b/libibverbs/dummy_ops.c @@ -71,6 +71,13 @@ static void async_event(struct ibv_async_event *event) { } +static int attach_counters_point_flow(struct ibv_counters *counters, + struct ibv_counter_attach_attr *attr, + struct ibv_flow *flow) +{ + return ENOSYS; +} + static int attach_mcast(struct ibv_qp *qp, const union ibv_gid *gid, uint16_t lid) { @@ -421,6 +428,7 @@ const struct verbs_context_ops verbs_dummy_ops = { alloc_pd, alloc_td, async_event, + attach_counters_point_flow, attach_mcast, bind_mw, close_xrcd, @@ -507,6 +515,7 @@ void verbs_set_ops(struct verbs_context *vctx, SET_OP(vctx, alloc_parent_domain); SET_OP(vctx, alloc_td); SET_OP(ctx, async_event); + SET_OP(vctx, attach_counters_point_flow); SET_OP(ctx, attach_mcast); SET_OP(ctx, bind_mw); SET_OP(vctx, close_xrcd); diff --git a/libibverbs/man/CMakeLists.txt b/libibverbs/man/CMakeLists.txt index aa5a07a..c4fec80 100644 --- a/libibverbs/man/CMakeLists.txt +++ b/libibverbs/man/CMakeLists.txt @@ -5,6 +5,7 @@ rdma_man_pages( ibv_alloc_pd.3 ibv_alloc_td.3 ibv_asyncwatch.1 + ibv_attach_counters_point_flow.3.md ibv_attach_mcast.3.md ibv_bind_mw.3 ibv_create_ah.3 diff --git a/libibverbs/man/ibv_attach_counters_point_flow.3.md b/libibverbs/man/ibv_attach_counters_point_flow.3.md new file mode 100644 index 0000000..8aee43e --- /dev/null +++ b/libibverbs/man/ibv_attach_counters_point_flow.3.md @@ -0,0 +1,134 @@ +--- +date: 2018-04-02 +footer: libibverbs +header: "Libibverbs Programmer's Manual" +layout: page +license: 'Licensed under the OpenIB.org BSD license (FreeBSD Variant) - See COPYING.md' +section: 3 +title: IBV_ATTACH_COUNTERS_POINT +--- +# NAME + +**ibv_attach_counters_point_flow**(3) - attach individual counter definition to +a flow object + +# SYNOPSIS + +```c +#include + +int ibv_attach_counters_point_flow(struct ibv_counters *counters, + struct ibv_counter_attach_attr *counter_attach_attr, + struct ibv_flow *flow); +``` + +# DESCRIPTION + +Attach counters point are a family of APIs to attach individual counter +description definition to a verb object at a specific index location. + +Counters object will start collecting values after it is bound to the verb object +resource. + +A static attach can be created when NULL is provided instead of the reference +to the verbs object (e.g.: in case of flow providing NULL instead of *flow*). +In this case, this counters object will only start collecting values after it is +bound to the verbs resource, for flow this is when referencing the counters handle +when creating a flow with **ibv_create_flow**(). + +Once an ibv_counters is bound statically to a verbs resource, no additional attach +is allowed till the counter object is not bound to any verb object. + +The argument counter_desc specifies which counter value should be collected. It +is defined in verbs.h as one of the enum ibv_counter_description options. + +Supported capabilities of specific counter_desc values per verbs object can be +tested by checking the return value for success or ENOTSUP errno. + +Attaching a counters handle to multiple objects of the same type will accumulate +the values into a single index. e.g.: creating several ibv_flows with the same +ibv_counters handle will collect the values from all relevant flows into the +relevant index location when reading the values from **ibv_read_counters**(), +setting the index more than once with different or same counter_desc will +aggregate the values from all relevant counters into the relevant index +location. + +The runtime values of counters can be read from the hardware by calling +**ibv_read_counters**(). + +# ARGUMENTS + +*counters* +: Existing counters to attach new counter point on. + +*counter_attach_attr* +: An ibv_counter_attach_attr struct, as defined in verbs.h. + +*flow* +: Existing flow to attach a new counters point on (in static mode +it must be NULL). + +## *counter_attach_attr* Argument + +```c +struct ibv_counter_attach_attr { + enum ibv_counter_description counter_desc; + uint32_t index; + uint32_t comp_mask; +}; +``` + +## *counter_desc* Argument + +```c +enum ibv_counter_description { + IBV_COUNTER_PACKETS, + IBV_COUNTER_BYTES, +}; +``` + +*index* +: Desired location of the specific counter at the counters object. + +*comp_mask* +: Bitmask specifying what fields in the structure are valid. + +# RETURN VALUE + +**ibv_attach_counters_point_flow**() returns 0 on success, or the value of errno +on failure (which indicates the failure reason) + +# ERRORS + +EINVAL +: invalid argument(s) passed + +ENOTSUP +: *counter_desc* is not supported on the requested object + +EBUSY +: the counter object is already bound to a flow, additional attach calls is not allowed (valid for static attach only) + +ENOMEM +: not enough memory + +# NOTES +Counter values in each index location are cleared upon creation when calling +**ibv_create_counters**(). +Attaching counters points will only increase these values accordingly. + +# EXAMPLE + +An example of use of **ibv_attach_counters_point_flow**() is shown in +**ibv_read_counters**(3) + +# SEE ALSO + +**ibv_create_counters**(3), **ibv_destroy_counters**(3), +**ibv_read_counters**(3), **ibv_create_flow**(3) + +# AUTHORS + +Raed Salem + +Alex Rosenbaum diff --git a/libibverbs/verbs.h b/libibverbs/verbs.h index 6eceadc..066396f 100644 --- a/libibverbs/verbs.h +++ b/libibverbs/verbs.h @@ -1726,6 +1726,17 @@ struct ibv_counters { struct ibv_context *context; }; +enum ibv_counter_description { + IBV_COUNTER_PACKETS, + IBV_COUNTER_BYTES, +}; + +struct ibv_counter_attach_attr { + enum ibv_counter_description counter_desc; + uint32_t index; /* Desired location index of the counter at the counters object */ + uint32_t comp_mask; +}; + enum ibv_values_mask { IBV_VALUES_MASK_RAW_CLOCK = 1 << 0, IBV_VALUES_MASK_RESERVED = 1 << 1 @@ -1738,6 +1749,9 @@ struct ibv_values_ex { struct verbs_context { /* "grows up" - new fields go here */ + int (*attach_counters_point_flow)(struct ibv_counters *counters, + struct ibv_counter_attach_attr *attr, + struct ibv_flow *flow); struct ibv_counters *(*create_counters)(struct ibv_context *context, struct ibv_counters_init_attr *init_attr); int (*destroy_counters)(struct ibv_counters *counters); @@ -2833,6 +2847,19 @@ static inline int ibv_destroy_counters(struct ibv_counters *counters) return vctx->destroy_counters(counters); } +static inline int ibv_attach_counters_point_flow(struct ibv_counters *counters, + struct ibv_counter_attach_attr *attr, + struct ibv_flow *flow) +{ + struct verbs_context *vctx; + + vctx = verbs_get_ctx_op(counters->context, attach_counters_point_flow); + if (!vctx) + return ENOSYS; + + return vctx->attach_counters_point_flow(counters, attr, flow); +} + #ifdef __cplusplus } #endif