From patchwork Fri Mar 31 17:04:52 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ira Weiny X-Patchwork-Id: 9656873 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork.web.codeaurora.org (Postfix) with ESMTP id 25DFD601D2 for ; Fri, 31 Mar 2017 17:04:59 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 1393628653 for ; Fri, 31 Mar 2017 17:04:59 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 0706B286DF; Fri, 31 Mar 2017 17:04:59 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-6.9 required=2.0 tests=BAYES_00,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 0EA5F28653 for ; Fri, 31 Mar 2017 17:04:58 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S933204AbdCaRE5 (ORCPT ); Fri, 31 Mar 2017 13:04:57 -0400 Received: from mga06.intel.com ([134.134.136.31]:56901 "EHLO mga06.intel.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S933246AbdCaRE4 (ORCPT ); Fri, 31 Mar 2017 13:04:56 -0400 Received: from orsmga004.jf.intel.com ([10.7.209.38]) by orsmga104.jf.intel.com with ESMTP; 31 Mar 2017 10:04:54 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.36,252,1486454400"; d="scan'208";a="72299183" Received: from phlsvsds.ph.intel.com ([10.228.195.38]) by orsmga004.jf.intel.com with ESMTP; 31 Mar 2017 10:04:53 -0700 Received: from phlsvsds.ph.intel.com (localhost.localdomain [127.0.0.1]) by phlsvsds.ph.intel.com (8.13.8/8.13.8) with ESMTP id v2VH4rX8019082; Fri, 31 Mar 2017 13:04:53 -0400 Received: (from iweiny@localhost) by phlsvsds.ph.intel.com (8.13.8/8.13.8/Submit) id v2VH4rUi019079; Fri, 31 Mar 2017 13:04:53 -0400 X-Authentication-Warning: phlsvsds.ph.intel.com: iweiny set sender to ira.weiny@intel.com using -f From: ira.weiny@intel.com To: Doug Ledford , linux-rdma@vger.kernel.org Cc: Sean Hefty , Hal Rosenstock , Mike Marciniszyn , Dennis Dalessandro , Ira Weiny Subject: [PATCH for-next 1/6] IB/MAD: Add send path trace points Date: Fri, 31 Mar 2017 13:04:52 -0400 Message-Id: <1490979892-19031-2-git-send-email-ira.weiny@intel.com> X-Mailer: git-send-email 1.8.2.3 In-Reply-To: <1490979892-19031-1-git-send-email-ira.weiny@intel.com> References: <1490979892-19031-1-git-send-email-ira.weiny@intel.com> Sender: linux-rdma-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-rdma@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP From: Ira Weiny Use the standard Linux trace mechanism to trace MADs being sent. 3 trace points are added, when the MAD is posted to the qp, when the MAD is completed, and when the MAD completes in error. Signed-off-by: Ira Weiny --- Changes since v1: move be64_to_cpu to print rather than fast assign Clean up trace statements Update copyright dates. --- drivers/infiniband/core/mad.c | 53 +++++++++++- include/trace/events/ib_mad.h | 191 ++++++++++++++++++++++++++++++++++++++++++ 2 files changed, 243 insertions(+), 1 deletion(-) create mode 100644 include/trace/events/ib_mad.h diff --git a/drivers/infiniband/core/mad.c b/drivers/infiniband/core/mad.c index a009f7132c73..64f55bd85277 100644 --- a/drivers/infiniband/core/mad.c +++ b/drivers/infiniband/core/mad.c @@ -3,7 +3,7 @@ * Copyright (c) 2005 Intel Corporation. All rights reserved. * Copyright (c) 2005 Mellanox Technologies Ltd. All rights reserved. * Copyright (c) 2009 HNR Consulting. All rights reserved. - * Copyright (c) 2014 Intel Corporation. All rights reserved. + * Copyright (c) 2014,2017 Intel Corporation. All rights reserved. * * This software is available to you under a choice of one of two * licenses. You may choose to be licensed under the terms of the GNU @@ -49,6 +49,31 @@ #include "agent.h" #include "core_priv.h" +struct ib_mad_addr { + u16 dlid; + u8 sl; + u16 pkey; + u32 rqpn; + u32 rqkey; +}; +static void trace_create_mad_addr(struct ib_device *dev, u8 pnum, + struct ib_ud_wr *wr, + struct ib_mad_addr *addr) +{ + struct ib_ah_attr attr; + + memset(&attr, 0, sizeof(attr)); + ib_query_ah(wr->ah, &attr); + + addr->dlid = attr.dlid; + addr->sl = attr.sl; + ib_query_pkey(dev, pnum, wr->pkey_index, &addr->pkey); + addr->rqpn = wr->remote_qpn; + addr->rqkey = wr->remote_qkey; +} +#define CREATE_TRACE_POINTS +#include + static int mad_sendq_size = IB_MAD_QP_SEND_SIZE; static int mad_recvq_size = IB_MAD_QP_RECV_SIZE; @@ -1175,6 +1200,14 @@ int ib_send_mad(struct ib_mad_send_wr_private *mad_send_wr) spin_lock_irqsave(&qp_info->send_queue.lock, flags); if (qp_info->send_queue.count < qp_info->send_queue.max_active) { + if (trace_ib_mad_ib_send_mad_enabled()) { + struct ib_mad_addr addr; + + trace_create_mad_addr(qp_info->port_priv->device, + qp_info->port_priv->port_num, + &mad_send_wr->send_wr, &addr); + trace_ib_mad_ib_send_mad(mad_send_wr, &addr); + } ret = ib_post_send(mad_agent->qp, &mad_send_wr->send_wr.wr, &bad_send_wr); list = &qp_info->send_queue.list; @@ -2431,6 +2464,8 @@ static void ib_mad_send_done(struct ib_cq *cq, struct ib_wc *wc) send_queue = mad_list->mad_queue; qp_info = send_queue->qp_info; + trace_ib_mad_send_done_handler(mad_send_wr, wc); + retry: ib_dma_unmap_single(mad_send_wr->send_buf.mad_agent->device, mad_send_wr->header_mapping, @@ -2462,6 +2497,14 @@ static void ib_mad_send_done(struct ib_cq *cq, struct ib_wc *wc) ib_mad_complete_send_wr(mad_send_wr, &mad_send_wc); if (queued_send_wr) { + if (trace_ib_mad_send_done_resend_enabled()) { + struct ib_mad_addr addr; + + trace_create_mad_addr(qp_info->port_priv->device, + qp_info->port_priv->port_num, + &mad_send_wr->send_wr, &addr); + trace_ib_mad_send_done_resend(queued_send_wr, &addr); + } ret = ib_post_send(qp_info->qp, &queued_send_wr->send_wr.wr, &bad_send_wr); if (ret) { @@ -2511,6 +2554,14 @@ static bool ib_mad_send_error(struct ib_mad_port_private *port_priv, struct ib_send_wr *bad_send_wr; mad_send_wr->retry = 0; + if (trace_ib_mad_mad_error_handler_enabled()) { + struct ib_mad_addr addr; + + trace_create_mad_addr(qp_info->port_priv->device, + qp_info->port_priv->port_num, + &mad_send_wr->send_wr, &addr); + trace_ib_mad_mad_error_handler(mad_send_wr, &addr); + } ret = ib_post_send(qp_info->qp, &mad_send_wr->send_wr.wr, &bad_send_wr); if (!ret) diff --git a/include/trace/events/ib_mad.h b/include/trace/events/ib_mad.h new file mode 100644 index 000000000000..0b7ed004a025 --- /dev/null +++ b/include/trace/events/ib_mad.h @@ -0,0 +1,191 @@ +/* + * Copyright (c) 2017 Intel Corporation. All rights reserved. + * + * This software is available to you under a choice of one of two + * licenses. You may choose to be licensed under the terms of the GNU + * General Public License (GPL) Version 2, available from the file + * COPYING in the main directory of this source tree, or the + * OpenIB.org BSD license below: + * + * Redistribution and use in source and binary forms, with or + * without modification, are permitted provided that the following + * conditions are met: + * + * - Redistributions of source code must retain the above + * copyright notice, this list of conditions and the following + * disclaimer. + * + * - Redistributions in binary form must reproduce the above + * copyright notice, this list of conditions and the following + * disclaimer in the documentation and/or other materials + * provided with the distribution. + * + * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, + * EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF + * MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND + * NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS + * BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN + * ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN + * CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE + * SOFTWARE. + * + */ + +#undef TRACE_SYSTEM +#define TRACE_SYSTEM ib_mad + +#if !defined(_TRACE_IB_MAD_H) || defined(TRACE_HEADER_MULTI_READ) +#define _TRACE_IB_MAD_H + +#include +#include + +DECLARE_EVENT_CLASS(ib_mad_send_template, + TP_PROTO(struct ib_mad_send_wr_private *wr, struct ib_mad_addr *addr), + TP_ARGS(wr, addr), + + TP_STRUCT__entry( + __array(char, dev_name, IB_DEVICE_NAME_MAX ) + __field(u8, port_num ) + __field(u32, qp_num ) + __field(void *, agent_priv ) + __field(u64, wrtid ) + __field(int, retries_left ) + __field(int, max_retries ) + __field(int, retry ) + __field(unsigned long, timeout ) + __field(u32, length ) + __field(u8, base_version ) + __field(u8, mgmt_class ) + __field(u8, class_version ) + __field(u8, method ) + __field(u16, status ) + __field(u16, class_specific ) + __field(u64, tid ) + __field(u16, attr_id ) + __field(u32, attr_mod ) + __field(u16, dlid ) + __field(u8, sl ) + __field(u16, pkey ) + __field(u32, rqpn ) + __field(u32, rqkey ) + ), + + TP_fast_assign( + memcpy(__entry->dev_name, wr->mad_agent_priv->agent.device->name, IB_DEVICE_NAME_MAX); + __entry->port_num = wr->mad_agent_priv->agent.port_num; + __entry->qp_num = wr->mad_agent_priv->qp_info->qp->qp_num; + __entry->agent_priv = wr->mad_agent_priv; + __entry->wrtid = wr->tid; + __entry->max_retries = wr->max_retries; + __entry->retries_left = wr->retries_left; + __entry->retry = wr->retry; + __entry->timeout = wr->timeout; + __entry->length = wr->send_buf.hdr_len + + wr->send_buf.data_len; + __entry->base_version = ((struct ib_mad_hdr *)wr->send_buf.mad)->base_version; + __entry->mgmt_class = ((struct ib_mad_hdr *)wr->send_buf.mad)->mgmt_class; + __entry->class_version = ((struct ib_mad_hdr *)wr->send_buf.mad)->class_version; + __entry->method = ((struct ib_mad_hdr *)wr->send_buf.mad)->method; + __entry->status = ((struct ib_mad_hdr *)wr->send_buf.mad)->status; + __entry->class_specific = ((struct ib_mad_hdr *)wr->send_buf.mad)->class_specific; + __entry->tid = ((struct ib_mad_hdr *)wr->send_buf.mad)->tid; + __entry->attr_id = ((struct ib_mad_hdr *)wr->send_buf.mad)->attr_id; + __entry->attr_mod = ((struct ib_mad_hdr *)wr->send_buf.mad)->attr_mod; + __entry->dlid = addr->dlid; + __entry->sl = addr->sl; + __entry->pkey = addr->pkey; + __entry->rqpn = addr->rqpn; + __entry->rqkey = addr->rqkey; + ), + + TP_printk("%s:%d QP%d agent %p: " \ + "wrtid 0x%llx; %d/%d retries(%d); timeout %lu length %d : hdr : " \ + "base_ver 0x%x class 0x%x class_ver 0x%x method 0x%x " \ + "status 0x%x class_specific 0x%x tid 0x%llx attr_id 0x%x attr_mod 0x%x " \ + " => dlid 0x%x sl %d pkey 0x%x rpqn 0x%x rqpkey 0x%x", + __entry->dev_name, __entry->port_num, __entry->qp_num, + __entry->agent_priv, be64_to_cpu(__entry->wrtid), + __entry->retries_left, __entry->max_retries, + __entry->retry, __entry->timeout, __entry->length, + __entry->base_version, __entry->mgmt_class, __entry->class_version, + __entry->method, be16_to_cpu(__entry->status), + be16_to_cpu(__entry->class_specific), + be64_to_cpu(__entry->tid), be16_to_cpu(__entry->attr_id), + be32_to_cpu(__entry->attr_mod), + be16_to_cpu(__entry->dlid), __entry->sl, __entry->pkey, __entry->rqpn, + __entry->rqkey + ) +); + +DEFINE_EVENT(ib_mad_send_template, ib_mad_mad_error_handler, + TP_PROTO(struct ib_mad_send_wr_private *wr, struct ib_mad_addr *addr), + TP_ARGS(wr, addr)); +DEFINE_EVENT(ib_mad_send_template, ib_mad_ib_send_mad, + TP_PROTO(struct ib_mad_send_wr_private *wr, struct ib_mad_addr *addr), + TP_ARGS(wr, addr)); +DEFINE_EVENT(ib_mad_send_template, ib_mad_send_done_resend, + TP_PROTO(struct ib_mad_send_wr_private *wr, struct ib_mad_addr *addr), + TP_ARGS(wr, addr)); + +TRACE_EVENT(ib_mad_send_done_handler, + TP_PROTO(struct ib_mad_send_wr_private *wr, struct ib_wc *wc), + TP_ARGS(wr, wc), + + TP_STRUCT__entry( + __array(char, dev_name, IB_DEVICE_NAME_MAX ) + __field(u8, port_num ) + __field(u32, qp_num ) + __field(void *, agent_priv ) + __field(u64, wrtid ) + __field(int, retries_left ) + __field(int, max_retries ) + __field(int, retry ) + __field(unsigned long, timeout ) + __field(u8, base_version ) + __field(u8, mgmt_class ) + __field(u8, class_version ) + __field(u8, method ) + __field(u16, status ) + __field(u16, wc_status ) + __field(u32, length ) + ), + + TP_fast_assign( + memcpy(__entry->dev_name, wr->mad_agent_priv->agent.device->name, IB_DEVICE_NAME_MAX); + __entry->port_num = wr->mad_agent_priv->agent.port_num; + __entry->qp_num = wr->mad_agent_priv->qp_info->qp->qp_num; + __entry->agent_priv = wr->mad_agent_priv; + __entry->wrtid = wr->tid; + __entry->max_retries = wr->max_retries; + __entry->retries_left = wr->retries_left; + __entry->retry = wr->retry; + __entry->timeout = wr->timeout; + __entry->base_version = ((struct ib_mad_hdr *)wr->send_buf.mad)->base_version; + __entry->mgmt_class = ((struct ib_mad_hdr *)wr->send_buf.mad)->mgmt_class; + __entry->class_version = ((struct ib_mad_hdr *)wr->send_buf.mad)->class_version; + __entry->method = ((struct ib_mad_hdr *)wr->send_buf.mad)->method; + __entry->status = ((struct ib_mad_hdr *)wr->send_buf.mad)->status; + __entry->wc_status = wc->status; + __entry->length = wc->byte_len; + ), + + TP_printk("%s:%d QP%d : SEND WC Status %d : agent %p: " \ + "wrtid 0x%llx %d/%d retries(%d) timeout %lu length %d: hdr : " \ + "base_ver 0x%x class 0x%x class_ver 0x%x method 0x%x " \ + "status 0x%x", + __entry->dev_name, __entry->port_num, __entry->qp_num, + __entry->wc_status, + __entry->agent_priv, be64_to_cpu(__entry->wrtid), + __entry->retries_left, __entry->max_retries, + __entry->retry, __entry->timeout, + __entry->length, + __entry->base_version, __entry->mgmt_class, __entry->class_version, + __entry->method, be16_to_cpu(__entry->status) + ) +); + + +#endif /* _TRACE_IB_MAD_H */ + +#include