From patchwork Mon Mar 12 17:43:00 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Raju Rangoju X-Patchwork-Id: 10276875 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork.web.codeaurora.org (Postfix) with ESMTP id 373AD6038F for ; Mon, 12 Mar 2018 17:43:48 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 3D6A728A31 for ; Mon, 12 Mar 2018 17:43:48 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 31CA628DB2; Mon, 12 Mar 2018 17:43:48 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-6.9 required=2.0 tests=BAYES_00,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 1EFCE28A31 for ; Mon, 12 Mar 2018 17:43:47 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S932576AbeCLRnq (ORCPT ); Mon, 12 Mar 2018 13:43:46 -0400 Received: from stargate.chelsio.com ([12.32.117.8]:46550 "EHLO stargate.chelsio.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S932231AbeCLRnp (ORCPT ); Mon, 12 Mar 2018 13:43:45 -0400 Received: from localhost ([10.193.185.238]) by stargate.chelsio.com (8.13.8/8.13.8) with ESMTP id w2CHhZrl023303; Mon, 12 Mar 2018 10:43:40 -0700 From: Raju Rangoju To: dledford@redhat.com Cc: linux-rdma@vger.kernel.org, swise@opengridcomputing.com, rajur@chelsio.com Subject: [PATCH 2/5] cxgb4: Add debugfs support for SRQ Date: Mon, 12 Mar 2018 23:13:00 +0530 Message-Id: <20180312174303.4333-3-rajur@chelsio.com> X-Mailer: git-send-email 2.12.0 In-Reply-To: <20180312174303.4333-1-rajur@chelsio.com> References: <20180312174303.4333-1-rajur@chelsio.com> Sender: linux-rdma-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-rdma@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP - This patch adds srq debugfs files - If hw supports an srq table, then export a debugfs file to dump the table Signed-off-by: Raju Rangoju Reviewed-by: Steve Wise --- drivers/net/ethernet/chelsio/cxgb4/Makefile | 2 +- drivers/net/ethernet/chelsio/cxgb4/cxgb4.h | 1 + drivers/net/ethernet/chelsio/cxgb4/cxgb4_debugfs.c | 2 + drivers/net/ethernet/chelsio/cxgb4/srq.c | 230 +++++++++++++++++++++ drivers/net/ethernet/chelsio/cxgb4/srq.h | 71 +++++++ 5 files changed, 305 insertions(+), 1 deletion(-) create mode 100644 drivers/net/ethernet/chelsio/cxgb4/srq.c create mode 100644 drivers/net/ethernet/chelsio/cxgb4/srq.h diff --git a/drivers/net/ethernet/chelsio/cxgb4/Makefile b/drivers/net/ethernet/chelsio/cxgb4/Makefile index 53b6a02c778e..bea6a059a8f1 100644 --- a/drivers/net/ethernet/chelsio/cxgb4/Makefile +++ b/drivers/net/ethernet/chelsio/cxgb4/Makefile @@ -6,7 +6,7 @@ obj-$(CONFIG_CHELSIO_T4) += cxgb4.o cxgb4-objs := cxgb4_main.o l2t.o smt.o t4_hw.o sge.o clip_tbl.o cxgb4_ethtool.o \ - cxgb4_uld.o sched.o cxgb4_filter.o cxgb4_tc_u32.o \ + cxgb4_uld.o srq.o sched.o cxgb4_filter.o cxgb4_tc_u32.o \ cxgb4_ptp.o cxgb4_tc_flower.o cxgb4_cudbg.o \ cudbg_common.o cudbg_lib.o cudbg_zlib.o cxgb4-$(CONFIG_CHELSIO_T4_DCB) += cxgb4_dcb.o diff --git a/drivers/net/ethernet/chelsio/cxgb4/cxgb4.h b/drivers/net/ethernet/chelsio/cxgb4/cxgb4.h index 9040e13ce4b7..1bb30b5a92a0 100644 --- a/drivers/net/ethernet/chelsio/cxgb4/cxgb4.h +++ b/drivers/net/ethernet/chelsio/cxgb4/cxgb4.h @@ -946,6 +946,7 @@ struct adapter { /* Ethtool Dump */ struct ethtool_dump eth_dump; + struct srq_data *srq; }; /* Support for "sched-class" command to allow a TX Scheduling Class to be diff --git a/drivers/net/ethernet/chelsio/cxgb4/cxgb4_debugfs.c b/drivers/net/ethernet/chelsio/cxgb4/cxgb4_debugfs.c index 2822bbff73e8..e5d47aeb748b 100644 --- a/drivers/net/ethernet/chelsio/cxgb4/cxgb4_debugfs.c +++ b/drivers/net/ethernet/chelsio/cxgb4/cxgb4_debugfs.c @@ -44,6 +44,7 @@ #include "t4fw_api.h" #include "cxgb4_debugfs.h" #include "clip_tbl.h" +#include "srq.h" #include "l2t.h" #include "cudbg_if.h" #include "cudbg_lib_common.h" @@ -2999,6 +3000,7 @@ int t4_setup_debugfs(struct adapter *adap) { "blocked_fl", &blocked_fl_fops, S_IRUSR | S_IWUSR, 0 }, { "meminfo", &meminfo_fops, S_IRUSR, 0 }, { "crypto", &chcr_stats_debugfs_fops, S_IRUSR, 0 }, + { "srq", &t4_srq_debugfs_fops, S_IRUSR, 0 }, }; /* Debug FS nodes common to all T5 and later adapters. diff --git a/drivers/net/ethernet/chelsio/cxgb4/srq.c b/drivers/net/ethernet/chelsio/cxgb4/srq.c new file mode 100644 index 000000000000..b60158b11fbf --- /dev/null +++ b/drivers/net/ethernet/chelsio/cxgb4/srq.c @@ -0,0 +1,230 @@ +/* + * This file is part of the Chelsio T4 Ethernet driver for Linux. + * + * Copyright (c) 2017-2018 Chelsio Communications, Inc. All rights reserved. + * + * This software is available to you under a choice of one of two + * licenses. You may choose to be licensed under the terms of the GNU + * General Public License (GPL) Version 2, available from the file + * COPYING in the main directory of this source tree, or the + * OpenIB.org BSD license below: + * + * Redistribution and use in source and binary forms, with or + * without modification, are permitted provided that the following + * conditions are met: + * + * - Redistributions of source code must retain the above + * copyright notice, this list of conditions and the following + * disclaimer. + * + * - Redistributions in binary form must reproduce the above + * copyright notice, this list of conditions and the following + * disclaimer in the documentation and/or other materials + * provided with the distribution. + * + * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, + * EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF + * MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND + * NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS + * BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN + * ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN + * CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE + * SOFTWARE. + */ + +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include "cxgb4.h" +#include "t4_msg.h" +#include "t4_regs.h" +#include "srq.h" + +struct srq_data *t4_init_srq(int srq_size) +{ + int i; + struct srq_data *s; + int size = sizeof(*s) + srq_size*sizeof(struct srq_entry); + + s = kzalloc(size, GFP_KERNEL | __GFP_NOWARN); + if (!s) + s = vzalloc(size); + if (!s) + return NULL; + + s->srq_size = srq_size; + s->rpl_count = 0; + init_completion(&s->comp); + mutex_init(&s->srq_mutex); + + for (i = 0; i < s->srq_size; ++i) { + memset(&s->srqtab[i], 0, sizeof(struct srq_data)); + s->srqtab[i].idx = i; + } + return s; +} + +/* + * do_srq_read_entries: read the SRQ table + * @adap: Pointer to the adapter + * + * Send CPL_SRQ_TABLE_REQ messages for each entry. + * Contents will be returned in CPL_SRQ_TABLE_RPL messages. + * + */ +static int do_srq_read_entries(struct adapter *adap) +{ + struct cpl_srq_table_req *req; + struct sk_buff *skb; + int i; + + /* Read the srq */ + for (i = 0; i < adap->srq->srq_size; i++) { + skb = alloc_skb(sizeof(*req), GFP_KERNEL); + if (!skb) + return -ENOMEM; + req = (struct cpl_srq_table_req *) + __skb_put(skb, sizeof(*req)); + memset(req, 0, sizeof(*req)); + INIT_TP_WR(req, 0); + OPCODE_TID(req) = htonl(MK_OPCODE_TID(CPL_SRQ_TABLE_REQ, + TID_TID_V(i) | + TID_QID_V(adap->sge.fw_evtq.abs_id))); + req->idx = i; + + /* do_srq_table_rpl() will make it valid */ + adap->srq->srqtab[i].valid = 0; + t4_mgmt_tx(adap, skb); + } + return 0; +} + +void do_srq_table_rpl(struct adapter *adap, const struct cpl_srq_table_rpl *rpl) +{ + struct srq_data *s = adap->srq; + unsigned int idx = TID_TID_G(GET_TID(rpl)); + struct srq_entry *e; + + if (unlikely(rpl->status != CPL_CONTAINS_READ_RPL)) { + dev_err(adap->pdev_dev, + "Unexpected SRQ_TABLE_RPL status %u for entry %u\n", + rpl->status, idx); + goto out; + } + + /* Store the read entry */ + e = &s->srqtab[idx]; + e->valid = 1; + WARN_ON_ONCE(e->idx != idx); + e->pdid = SRQT_PDID_G(be64_to_cpu(rpl->rsvd_pdid)); + e->qlen = SRQT_QLEN_G(be32_to_cpu(rpl->qlen_qbase)); + e->qbase = SRQT_QBASE_G(be32_to_cpu(rpl->qlen_qbase)); + e->cur_msn = be16_to_cpu(rpl->cur_msn); + e->max_msn = be16_to_cpu(rpl->max_msn); +out: + if (++s->rpl_count == s->srq_size) + complete(&s->comp); +} + +static void *srq_get_idx(struct seq_file *seq, loff_t pos) +{ + struct adapter *adap = seq->private; + struct srq_data *s = adap->srq; + + return pos >= s->srq_size ? NULL : &s->srqtab[pos]; +} + +static void *srq_seq_start(struct seq_file *seq, loff_t *pos) +{ + return *pos ? srq_get_idx(seq, *pos - 1) : SEQ_START_TOKEN; +} + +static void *srq_seq_next(struct seq_file *seq, void *v, loff_t *pos) +{ + v = srq_get_idx(seq, *pos); + if (v) + ++*pos; + return v; +} + +static void srq_seq_stop(struct seq_file *seq, void *v) +{ +} + +static int srq_seq_show(struct seq_file *seq, void *v) +{ + if (v == SEQ_START_TOKEN) + seq_puts(seq, " Idx PDID QBASE QLEN Cur MSN Max MSN\n"); + else { + struct srq_entry *e = v; + + if (e->valid) + seq_printf(seq, "%4u %4u %8u %4u %4u %4u\n", + e->idx, e->pdid, + e->qbase, e->qlen, + e->cur_msn, e->max_msn); + else + seq_printf(seq, "-\n"); + } + return 0; +} + +static const struct seq_operations srq_seq_ops = { + .start = srq_seq_start, + .next = srq_seq_next, + .stop = srq_seq_stop, + .show = srq_seq_show +}; + +static int srq_seq_open(struct inode *inode, struct file *file) +{ + struct adapter *adap = inode->i_private; + struct srq_data *s = adap->srq; + struct seq_file *seq; + int rc = -ENODEV; + + if (!(adap->flags & FULL_INIT_DONE) || !s) + goto out; + + rc = seq_open(file, &srq_seq_ops); + if (rc) + goto out; + + seq = file->private_data; + + mutex_lock(&s->srq_mutex); + init_completion(&s->comp); + s->rpl_count = 0; + rc = do_srq_read_entries(adap); + if (rc) { + mutex_unlock(&s->srq_mutex); + goto out; + } + rc = wait_for_completion_timeout(&s->comp, SRQ_WAIT_TO); + + /* !rc means we timed out */ + if (rc) { + seq->private = adap; + rc = 0; + } else + rc = -ETIMEDOUT; + mutex_unlock(&s->srq_mutex); +out: + return rc; +} + +const struct file_operations t4_srq_debugfs_fops = { + .owner = THIS_MODULE, + .open = srq_seq_open, + .read = seq_read, + .llseek = seq_lseek, + .release = seq_release, +}; + diff --git a/drivers/net/ethernet/chelsio/cxgb4/srq.h b/drivers/net/ethernet/chelsio/cxgb4/srq.h new file mode 100644 index 000000000000..1b2bbd142ba7 --- /dev/null +++ b/drivers/net/ethernet/chelsio/cxgb4/srq.h @@ -0,0 +1,71 @@ +/* + * This file is part of the Chelsio T4 Ethernet driver for Linux. + * + * Copyright (c) 2017-2018 Chelsio Communications, Inc. All rights reserved. + * + * This software is available to you under a choice of one of two + * licenses. You may choose to be licensed under the terms of the GNU + * General Public License (GPL) Version 2, available from the file + * COPYING in the main directory of this source tree, or the + * OpenIB.org BSD license below: + * + * Redistribution and use in source and binary forms, with or + * without modification, are permitted provided that the following + * conditions are met: + * + * - Redistributions of source code must retain the above + * copyright notice, this list of conditions and the following + * disclaimer. + * + * - Redistributions in binary form must reproduce the above + * copyright notice, this list of conditions and the following + * disclaimer in the documentation and/or other materials + * provided with the distribution. + * + * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, + * EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF + * MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND + * NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS + * BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN + * ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN + * CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE + * SOFTWARE. + */ + +#ifndef __CXGB4_SRQ_H +#define __CXGB4_SRQ_H + +#include +#include +#include + +struct adapter; +struct file_operations; +struct cpl_srq_table_rpl; + +enum { + SRQ_WAIT_TO = (HZ * 5), +}; + +struct srq_entry { + u8 valid; + u8 idx; + u8 qlen; + u16 pdid; + u16 cur_msn; + u16 max_msn; + u32 qbase; +}; + +struct srq_data { + unsigned int srq_size; + struct completion comp; + struct mutex srq_mutex; + int rpl_count; + struct srq_entry srqtab[0]; +}; + +struct srq_data *t4_init_srq(int); +void do_srq_table_rpl(struct adapter *adap, const struct cpl_srq_table_rpl *rpl); +extern const struct file_operations t4_srq_debugfs_fops; +#endif /* __CXGB4_SRQ_H */