From patchwork Tue Oct 5 17:15:34 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Subbaraya Sundeep X-Patchwork-Id: 12537341 X-Patchwork-Delegate: kuba@kernel.org Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id DD70AC433EF for ; Tue, 5 Oct 2021 17:15:49 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id C68826121F for ; Tue, 5 Oct 2021 17:15:49 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S236669AbhJERRj (ORCPT ); Tue, 5 Oct 2021 13:17:39 -0400 Received: from mx0a-0016f401.pphosted.com ([67.231.148.174]:9830 "EHLO mx0b-0016f401.pphosted.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S234459AbhJERRi (ORCPT ); Tue, 5 Oct 2021 13:17:38 -0400 Received: from pps.filterd (m0045849.ppops.net [127.0.0.1]) by mx0a-0016f401.pphosted.com (8.16.1.2/8.16.1.2) with SMTP id 195FuOGD026606; Tue, 5 Oct 2021 10:15:47 -0700 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=marvell.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-type; s=pfpt0220; bh=ddFJlLlBQrf2ipQ2AhaSMEDFBbYbxXQoMcZ6boNIc5c=; b=Y+UHaPH/3/LDLhrc/pLyU2v2GR+ONS5jMxfemGs/ZMrbBSvQXBWPpS1pTV2SsEIGum98 yA0L0nw/jKhoUVAoIObc0gihYu8WD0ZUG1M3vy7YgCuI7AUe0Nuy9bBpaccK/bTx08tU 4YUYddpUZs/CE08GJNcCfnLR027OfAYiPbHZD1iXjmtTS9iep9DHIdr2XUSiRZBi+tcR SkbZNF4bCb5G0uN5mvaLLo0pIZb3kLzv60yzB6s+9LTxdl2ZU6gv+1kI6YopsqxrVTcq D429P1039VOhjIyEK7s8/P8q2i8x9/MTFeU1zsp1ai92yHJuzpY9Slt1+LlAY05K+BNI 1g== Received: from dc5-exch02.marvell.com ([199.233.59.182]) by mx0a-0016f401.pphosted.com with ESMTP id 3bgmv5t48r-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-SHA384 bits=256 verify=NOT); Tue, 05 Oct 2021 10:15:46 -0700 Received: from DC5-EXCH01.marvell.com (10.69.176.38) by DC5-EXCH02.marvell.com (10.69.176.39) with Microsoft SMTP Server (TLS) id 15.0.1497.18; Tue, 5 Oct 2021 10:15:45 -0700 Received: from maili.marvell.com (10.69.176.80) by DC5-EXCH01.marvell.com (10.69.176.38) with Microsoft SMTP Server id 15.0.1497.18 via Frontend Transport; Tue, 5 Oct 2021 10:15:45 -0700 Received: from hyd1358.marvell.com (unknown [10.29.37.11]) by maili.marvell.com (Postfix) with ESMTP id 19D9E3F7081; Tue, 5 Oct 2021 10:15:42 -0700 (PDT) From: Subbaraya Sundeep To: , , CC: , , , Subbaraya Sundeep Subject: [net-next PATCH 1/3] octeontx2-pf: Simplify the receive buffer size calculation Date: Tue, 5 Oct 2021 22:45:34 +0530 Message-ID: <1633454136-14679-2-git-send-email-sbhatta@marvell.com> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1633454136-14679-1-git-send-email-sbhatta@marvell.com> References: <1633454136-14679-1-git-send-email-sbhatta@marvell.com> MIME-Version: 1.0 X-Proofpoint-GUID: ZT8Xsb-mSBpUg-kxUGL2mMPT6HGdanku X-Proofpoint-ORIG-GUID: ZT8Xsb-mSBpUg-kxUGL2mMPT6HGdanku X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.182.1,Aquarius:18.0.790,Hydra:6.0.391,FMLib:17.0.607.475 definitions=2021-10-05_03,2021-10-04_01,2020-04-07_01 Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org X-Patchwork-Delegate: kuba@kernel.org This patch separates the logic of configuring hardware maximum transmit frame size and receive frame size. This simplifies the logic to calculate receive buffer size and using cqe descriptor of different size. Also additional size of skb_shared_info structure is allocated for each receive buffer pointer given to hardware which is not necessary. Hence change the size calculation to remove the size of skb_shared_info. Add a check for array out of bounds while adding fragments to the network stack. Signed-off-by: Subbaraya Sundeep Signed-off-by: Hariprasad Kelam Signed-off-by: Sunil Goutham --- .../ethernet/marvell/octeontx2/nic/otx2_common.c | 10 +++----- .../ethernet/marvell/octeontx2/nic/otx2_common.h | 2 +- .../net/ethernet/marvell/octeontx2/nic/otx2_pf.c | 15 +++++------ .../net/ethernet/marvell/octeontx2/nic/otx2_txrx.c | 30 +++++++++++++++------- .../net/ethernet/marvell/octeontx2/nic/otx2_txrx.h | 4 +-- 5 files changed, 34 insertions(+), 27 deletions(-) diff --git a/drivers/net/ethernet/marvell/octeontx2/nic/otx2_common.c b/drivers/net/ethernet/marvell/octeontx2/nic/otx2_common.c index 0c89eb8..66da31f 100644 --- a/drivers/net/ethernet/marvell/octeontx2/nic/otx2_common.c +++ b/drivers/net/ethernet/marvell/octeontx2/nic/otx2_common.c @@ -231,7 +231,7 @@ int otx2_hw_set_mtu(struct otx2_nic *pfvf, int mtu) return -ENOMEM; } - req->maxlen = pfvf->max_frs; + req->maxlen = pfvf->netdev->mtu + OTX2_ETH_HLEN + OTX2_HW_TIMESTAMP_LEN; err = otx2_sync_mbox_msg(&pfvf->mbox); mutex_unlock(&pfvf->mbox.lock); @@ -590,7 +590,7 @@ int otx2_txschq_config(struct otx2_nic *pfvf, int lvl) u64 schq, parent; u64 dwrr_val; - dwrr_val = mtu_to_dwrr_weight(pfvf, pfvf->max_frs); + dwrr_val = mtu_to_dwrr_weight(pfvf, pfvf->tx_max_pktlen); req = otx2_mbox_alloc_msg_nix_txschq_cfg(&pfvf->mbox); if (!req) @@ -603,9 +603,7 @@ int otx2_txschq_config(struct otx2_nic *pfvf, int lvl) /* Set topology e.t.c configuration */ if (lvl == NIX_TXSCH_LVL_SMQ) { req->reg[0] = NIX_AF_SMQX_CFG(schq); - req->regval[0] = ((pfvf->netdev->max_mtu + OTX2_ETH_HLEN) << 8) - | OTX2_MIN_MTU; - + req->regval[0] = ((u64)pfvf->tx_max_pktlen << 8) | OTX2_MIN_MTU; req->regval[0] |= (0x20ULL << 51) | (0x80ULL << 39) | (0x2ULL << 36); req->num_regs++; @@ -800,7 +798,7 @@ int otx2_sq_aq_init(void *dev, u16 qidx, u16 sqb_aura) aq->sq.ena = 1; /* Only one SMQ is allocated, map all SQ's to that SMQ */ aq->sq.smq = pfvf->hw.txschq_list[NIX_TXSCH_LVL_SMQ][0]; - aq->sq.smq_rr_quantum = mtu_to_dwrr_weight(pfvf, pfvf->max_frs); + aq->sq.smq_rr_quantum = mtu_to_dwrr_weight(pfvf, pfvf->tx_max_pktlen); aq->sq.default_chan = pfvf->hw.tx_chan_base; aq->sq.sqe_stype = NIX_STYPE_STF; /* Cache SQB */ aq->sq.sqb_aura = sqb_aura; diff --git a/drivers/net/ethernet/marvell/octeontx2/nic/otx2_common.h b/drivers/net/ethernet/marvell/octeontx2/nic/otx2_common.h index a05f6bd..61e5281 100644 --- a/drivers/net/ethernet/marvell/octeontx2/nic/otx2_common.h +++ b/drivers/net/ethernet/marvell/octeontx2/nic/otx2_common.h @@ -326,7 +326,7 @@ struct otx2_nic { struct net_device *netdev; struct dev_hw_ops *hw_ops; void *iommu_domain; - u16 max_frs; + u16 tx_max_pktlen; u16 rbsize; /* Receive buffer size */ #define OTX2_FLAG_RX_TSTAMP_ENABLED BIT_ULL(0) diff --git a/drivers/net/ethernet/marvell/octeontx2/nic/otx2_pf.c b/drivers/net/ethernet/marvell/octeontx2/nic/otx2_pf.c index f24e920..1e0d0c9c 100644 --- a/drivers/net/ethernet/marvell/octeontx2/nic/otx2_pf.c +++ b/drivers/net/ethernet/marvell/octeontx2/nic/otx2_pf.c @@ -1312,16 +1312,14 @@ static int otx2_get_rbuf_size(struct otx2_nic *pf, int mtu) * NIX transfers entire data using 6 segments/buffers and writes * a CQE_RX descriptor with those segment addresses. First segment * has additional data prepended to packet. Also software omits a - * headroom of 128 bytes and sizeof(struct skb_shared_info) in - * each segment. Hence the total size of memory needed - * to receive a packet with 'mtu' is: + * headroom of 128 bytes in each segment. Hence the total size of + * memory needed to receive a packet with 'mtu' is: * frame size = mtu + additional data; - * memory = frame_size + (headroom + struct skb_shared_info size) * 6; + * memory = frame_size + headroom * 6; * each receive buffer size = memory / 6; */ frame_size = mtu + OTX2_ETH_HLEN + OTX2_HW_TIMESTAMP_LEN; - total_size = frame_size + (OTX2_HEAD_ROOM + - OTX2_DATA_ALIGN(sizeof(struct skb_shared_info))) * 6; + total_size = frame_size + OTX2_HEAD_ROOM * 6; rbuf_size = total_size / 6; return ALIGN(rbuf_size, 2048); @@ -1343,7 +1341,8 @@ static int otx2_init_hw_resources(struct otx2_nic *pf) hw->sqpool_cnt = hw->tot_tx_queues; hw->pool_cnt = hw->rqpool_cnt + hw->sqpool_cnt; - pf->max_frs = pf->netdev->mtu + OTX2_ETH_HLEN + OTX2_HW_TIMESTAMP_LEN; + /* Maximum hardware supported transmit length */ + pf->tx_max_pktlen = pf->netdev->max_mtu + OTX2_ETH_HLEN; pf->rbsize = otx2_get_rbuf_size(pf, pf->netdev->mtu); @@ -1807,7 +1806,7 @@ static netdev_tx_t otx2_xmit(struct sk_buff *skb, struct net_device *netdev) /* Check for minimum and maximum packet length */ if (skb->len <= ETH_HLEN || - (!skb_shinfo(skb)->gso_size && skb->len > pf->max_frs)) { + (!skb_shinfo(skb)->gso_size && skb->len > pf->tx_max_pktlen)) { dev_kfree_skb(skb); return NETDEV_TX_OK; } diff --git a/drivers/net/ethernet/marvell/octeontx2/nic/otx2_txrx.c b/drivers/net/ethernet/marvell/octeontx2/nic/otx2_txrx.c index 2d2b673..0cc6353 100644 --- a/drivers/net/ethernet/marvell/octeontx2/nic/otx2_txrx.c +++ b/drivers/net/ethernet/marvell/octeontx2/nic/otx2_txrx.c @@ -181,8 +181,9 @@ static void otx2_set_rxtstamp(struct otx2_nic *pfvf, skb_hwtstamps(skb)->hwtstamp = ns_to_ktime(tsns); } -static void otx2_skb_add_frag(struct otx2_nic *pfvf, struct sk_buff *skb, - u64 iova, int len, struct nix_rx_parse_s *parse) +static bool otx2_skb_add_frag(struct otx2_nic *pfvf, struct sk_buff *skb, + u64 iova, int len, struct nix_rx_parse_s *parse, + int qidx) { struct page *page; int off = 0; @@ -203,11 +204,22 @@ static void otx2_skb_add_frag(struct otx2_nic *pfvf, struct sk_buff *skb, } page = virt_to_page(va); - skb_add_rx_frag(skb, skb_shinfo(skb)->nr_frags, page, - va - page_address(page) + off, len - off, pfvf->rbsize); + if (likely(skb_shinfo(skb)->nr_frags < MAX_SKB_FRAGS)) { + skb_add_rx_frag(skb, skb_shinfo(skb)->nr_frags, page, + va - page_address(page) + off, + len - off, pfvf->rbsize); + + otx2_dma_unmap_page(pfvf, iova - OTX2_HEAD_ROOM, + pfvf->rbsize, DMA_FROM_DEVICE); + return true; + } - otx2_dma_unmap_page(pfvf, iova - OTX2_HEAD_ROOM, - pfvf->rbsize, DMA_FROM_DEVICE); + /* If more than MAX_SKB_FRAGS fragments are received then + * give back those buffer pointers to hardware for reuse. + */ + pfvf->hw_ops->aura_freeptr(pfvf, qidx, iova & ~0x07ULL); + + return false; } static void otx2_set_rxhash(struct otx2_nic *pfvf, @@ -349,9 +361,9 @@ static void otx2_rcv_pkt_handler(struct otx2_nic *pfvf, seg_addr = &sg->seg_addr; seg_size = (void *)sg; for (seg = 0; seg < sg->segs; seg++, seg_addr++) { - otx2_skb_add_frag(pfvf, skb, *seg_addr, seg_size[seg], - parse); - cq->pool_ptrs++; + if (otx2_skb_add_frag(pfvf, skb, *seg_addr, + seg_size[seg], parse, cq->cq_idx)) + cq->pool_ptrs++; } start += sizeof(*sg); } diff --git a/drivers/net/ethernet/marvell/octeontx2/nic/otx2_txrx.h b/drivers/net/ethernet/marvell/octeontx2/nic/otx2_txrx.h index 5c05774..f1a04cf 100644 --- a/drivers/net/ethernet/marvell/octeontx2/nic/otx2_txrx.h +++ b/drivers/net/ethernet/marvell/octeontx2/nic/otx2_txrx.h @@ -39,9 +39,7 @@ #define RCV_FRAG_LEN(x) \ ((RCV_FRAG_LEN1(x) < 2048) ? 2048 : RCV_FRAG_LEN1(x)) -#define DMA_BUFFER_LEN(x) \ - ((x) - OTX2_HEAD_ROOM - \ - OTX2_DATA_ALIGN(sizeof(struct skb_shared_info))) +#define DMA_BUFFER_LEN(x) ((x) - OTX2_HEAD_ROOM) /* IRQ triggered when NIX_LF_CINTX_CNT[ECOUNT] * is equal to this value. From patchwork Tue Oct 5 17:15:35 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Subbaraya Sundeep X-Patchwork-Id: 12537345 X-Patchwork-Delegate: kuba@kernel.org Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 45D3DC433F5 for ; Tue, 5 Oct 2021 17:15:58 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 2D6FE6117A for ; Tue, 5 Oct 2021 17:15:58 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S236738AbhJERRr (ORCPT ); Tue, 5 Oct 2021 13:17:47 -0400 Received: from mx0b-0016f401.pphosted.com ([67.231.156.173]:21896 "EHLO mx0b-0016f401.pphosted.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S236700AbhJERRo (ORCPT ); Tue, 5 Oct 2021 13:17:44 -0400 Received: from pps.filterd (m0045851.ppops.net [127.0.0.1]) by mx0b-0016f401.pphosted.com (8.16.1.2/8.16.1.2) with SMTP id 195FuFe4028844; Tue, 5 Oct 2021 10:15:51 -0700 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=marvell.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-type; s=pfpt0220; bh=lsTPQJbq1BoWC63f6Z/OJhplQKO6S+lgfgJHE06Md1E=; b=QOcCWlVO+ZgljbcPyajv0cSD+K9TJ8m0Lo73e0aEc6EcCmpMSLMMQ78FlpNfuksrNbXr E/Tmy0Ik6kfmmHfgK4EMbXZJb7xX6Fm7Qu0qFm50uemRuIwhrzb9RakwaZ5rV2yfKh38 dITp515Spy3+0k9ir5uwJDIAPLpcya7/MULauyarOjHSlo1v+8rFq5Ku+EtClxg/zuNS pyNWhOZ6FiFfCzlaPcc1R7VqoQkJyTa5Pmwr2orqR9cFKdsvGHnCbTfshey1CyR6rUBI Hq6ACAdxSxxME23kx3o3/rgFeVrp1FVYMwsnYyrY2hj+C6ITPPdpxF6/Q8ZjQcyir8+c JQ== Received: from dc5-exch02.marvell.com ([199.233.59.182]) by mx0b-0016f401.pphosted.com with ESMTP id 3bgr1qgu05-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-SHA384 bits=256 verify=NOT); Tue, 05 Oct 2021 10:15:50 -0700 Received: from DC5-EXCH01.marvell.com (10.69.176.38) by DC5-EXCH02.marvell.com (10.69.176.39) with Microsoft SMTP Server (TLS) id 15.0.1497.18; Tue, 5 Oct 2021 10:15:48 -0700 Received: from maili.marvell.com (10.69.176.80) by DC5-EXCH01.marvell.com (10.69.176.38) with Microsoft SMTP Server id 15.0.1497.18 via Frontend Transport; Tue, 5 Oct 2021 10:15:48 -0700 Received: from hyd1358.marvell.com (unknown [10.29.37.11]) by maili.marvell.com (Postfix) with ESMTP id C3A263F7080; Tue, 5 Oct 2021 10:15:45 -0700 (PDT) From: Subbaraya Sundeep To: , , CC: , , , Subbaraya Sundeep Subject: [net-next PATCH 2/3] octeontx2-pf: Add devlink param to vary cqe size Date: Tue, 5 Oct 2021 22:45:35 +0530 Message-ID: <1633454136-14679-3-git-send-email-sbhatta@marvell.com> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1633454136-14679-1-git-send-email-sbhatta@marvell.com> References: <1633454136-14679-1-git-send-email-sbhatta@marvell.com> MIME-Version: 1.0 X-Proofpoint-ORIG-GUID: WuvCG2Tgvh8E8s9LuwV40OymMwBTwJf8 X-Proofpoint-GUID: WuvCG2Tgvh8E8s9LuwV40OymMwBTwJf8 X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.182.1,Aquarius:18.0.790,Hydra:6.0.391,FMLib:17.0.607.475 definitions=2021-10-05_03,2021-10-04_01,2020-04-07_01 Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org X-Patchwork-Delegate: kuba@kernel.org Completion Queue Entry(CQE) is a descriptor written by hardware to notify software about the send and receive completion status. The CQE can be of size 128 or 512 bytes. A 512 bytes CQE can hold more receive fragments pointers compared to 128 bytes CQE. This patch adds devlink param to change CQE descriptor size. Signed-off-by: Subbaraya Sundeep Signed-off-by: Sunil Goutham --- .../ethernet/marvell/octeontx2/nic/otx2_common.c | 10 +++- .../ethernet/marvell/octeontx2/nic/otx2_common.h | 1 + .../ethernet/marvell/octeontx2/nic/otx2_devlink.c | 56 ++++++++++++++++++++++ .../net/ethernet/marvell/octeontx2/nic/otx2_pf.c | 2 + .../net/ethernet/marvell/octeontx2/nic/otx2_vf.c | 2 + 5 files changed, 69 insertions(+), 2 deletions(-) diff --git a/drivers/net/ethernet/marvell/octeontx2/nic/otx2_common.c b/drivers/net/ethernet/marvell/octeontx2/nic/otx2_common.c index 66da31f..3777f41 100644 --- a/drivers/net/ethernet/marvell/octeontx2/nic/otx2_common.c +++ b/drivers/net/ethernet/marvell/octeontx2/nic/otx2_common.c @@ -222,8 +222,11 @@ EXPORT_SYMBOL(otx2_set_mac_address); int otx2_hw_set_mtu(struct otx2_nic *pfvf, int mtu) { struct nix_frs_cfg *req; + u16 maxlen; int err; + maxlen = otx2_get_max_mtu(pfvf) + OTX2_ETH_HLEN + OTX2_HW_TIMESTAMP_LEN; + mutex_lock(&pfvf->mbox.lock); req = otx2_mbox_alloc_msg_nix_set_hw_frs(&pfvf->mbox); if (!req) { @@ -233,6 +236,9 @@ int otx2_hw_set_mtu(struct otx2_nic *pfvf, int mtu) req->maxlen = pfvf->netdev->mtu + OTX2_ETH_HLEN + OTX2_HW_TIMESTAMP_LEN; + if (is_otx2_lbkvf(pfvf->pdev)) + req->maxlen = maxlen; + err = otx2_sync_mbox_msg(&pfvf->mbox); mutex_unlock(&pfvf->mbox.lock); return err; @@ -1036,7 +1042,7 @@ int otx2_config_nix(struct otx2_nic *pfvf) struct nix_lf_alloc_rsp *rsp; int err; - pfvf->qset.xqe_size = NIX_XQESZ_W16 ? 128 : 512; + pfvf->qset.xqe_size = pfvf->hw.xqe_size; /* Get memory to put this msg */ nixlf = otx2_mbox_alloc_msg_nix_lf_alloc(&pfvf->mbox); @@ -1049,7 +1055,7 @@ int otx2_config_nix(struct otx2_nic *pfvf) nixlf->cq_cnt = pfvf->qset.cq_cnt; nixlf->rss_sz = MAX_RSS_INDIR_TBL_SIZE; nixlf->rss_grps = MAX_RSS_GROUPS; - nixlf->xqe_sz = NIX_XQESZ_W16; + nixlf->xqe_sz = pfvf->hw.xqe_size == 128 ? NIX_XQESZ_W16 : NIX_XQESZ_W64; /* We don't know absolute NPA LF idx attached. * AF will replace 'RVU_DEFAULT_PF_FUNC' with * NPA LF attached to this RVU PF/VF. diff --git a/drivers/net/ethernet/marvell/octeontx2/nic/otx2_common.h b/drivers/net/ethernet/marvell/octeontx2/nic/otx2_common.h index 61e5281..6e0d1ac 100644 --- a/drivers/net/ethernet/marvell/octeontx2/nic/otx2_common.h +++ b/drivers/net/ethernet/marvell/octeontx2/nic/otx2_common.h @@ -177,6 +177,7 @@ struct otx2_hw { u16 pool_cnt; u16 rqpool_cnt; u16 sqpool_cnt; + u16 xqe_size; /* NPA */ u32 stack_pg_ptrs; /* No of ptrs per stack page */ diff --git a/drivers/net/ethernet/marvell/octeontx2/nic/otx2_devlink.c b/drivers/net/ethernet/marvell/octeontx2/nic/otx2_devlink.c index 777a270..98450e1 100644 --- a/drivers/net/ethernet/marvell/octeontx2/nic/otx2_devlink.c +++ b/drivers/net/ethernet/marvell/octeontx2/nic/otx2_devlink.c @@ -64,9 +64,60 @@ static int otx2_dl_mcam_count_get(struct devlink *devlink, u32 id, return 0; } +static int otx2_dl_cqe_size_validate(struct devlink *devlink, u32 id, + union devlink_param_value val, + struct netlink_ext_ack *extack) +{ + if (val.vu16 != 128 && val.vu16 != 512) { + NL_SET_ERR_MSG_MOD(extack, + "Only 128 or 512 byte descriptor allowed"); + return -EINVAL; + } + + return 0; +} + +static int otx2_dl_cqe_size_set(struct devlink *devlink, u32 id, + struct devlink_param_gset_ctx *ctx) +{ + struct otx2_devlink *otx2_dl = devlink_priv(devlink); + struct otx2_nic *pfvf = otx2_dl->pfvf; + struct net_device *netdev; + int err = 0; + bool if_up; + + rtnl_lock(); + + netdev = pfvf->netdev; + if_up = netif_running(netdev); + if (if_up) + netdev->netdev_ops->ndo_stop(netdev); + + pfvf->hw.xqe_size = ctx->val.vu16; + + if (if_up) + err = netdev->netdev_ops->ndo_open(netdev); + + rtnl_unlock(); + + return err; +} + +static int otx2_dl_cqe_size_get(struct devlink *devlink, u32 id, + struct devlink_param_gset_ctx *ctx) +{ + struct otx2_devlink *otx2_dl = devlink_priv(devlink); + struct otx2_nic *pfvf = otx2_dl->pfvf; + + ctx->val.vu16 = pfvf->hw.xqe_size; + + return 0; +} + enum otx2_dl_param_id { OTX2_DEVLINK_PARAM_ID_BASE = DEVLINK_PARAM_GENERIC_ID_MAX, OTX2_DEVLINK_PARAM_ID_MCAM_COUNT, + OTX2_DEVLINK_PARAM_ID_CQE_SIZE, }; static const struct devlink_param otx2_dl_params[] = { @@ -75,6 +126,11 @@ static const struct devlink_param otx2_dl_params[] = { BIT(DEVLINK_PARAM_CMODE_RUNTIME), otx2_dl_mcam_count_get, otx2_dl_mcam_count_set, otx2_dl_mcam_count_validate), + DEVLINK_PARAM_DRIVER(OTX2_DEVLINK_PARAM_ID_CQE_SIZE, + "completion_descriptor_size", DEVLINK_PARAM_TYPE_U16, + BIT(DEVLINK_PARAM_CMODE_RUNTIME), + otx2_dl_cqe_size_get, otx2_dl_cqe_size_set, + otx2_dl_cqe_size_validate), }; /* Devlink OPs */ diff --git a/drivers/net/ethernet/marvell/octeontx2/nic/otx2_pf.c b/drivers/net/ethernet/marvell/octeontx2/nic/otx2_pf.c index 1e0d0c9c..8618cf7 100644 --- a/drivers/net/ethernet/marvell/octeontx2/nic/otx2_pf.c +++ b/drivers/net/ethernet/marvell/octeontx2/nic/otx2_pf.c @@ -2624,6 +2624,8 @@ static int otx2_probe(struct pci_dev *pdev, const struct pci_device_id *id) hw->tx_queues = qcount; hw->tot_tx_queues = qcount; hw->max_queues = qcount; + /* Use CQE of 128 byte descriptor size by default */ + hw->xqe_size = 128; num_vec = pci_msix_vec_count(pdev); hw->irq_name = devm_kmalloc_array(&hw->pdev->dev, num_vec, NAME_SIZE, diff --git a/drivers/net/ethernet/marvell/octeontx2/nic/otx2_vf.c b/drivers/net/ethernet/marvell/octeontx2/nic/otx2_vf.c index 980219a..672be05 100644 --- a/drivers/net/ethernet/marvell/octeontx2/nic/otx2_vf.c +++ b/drivers/net/ethernet/marvell/octeontx2/nic/otx2_vf.c @@ -587,6 +587,8 @@ static int otx2vf_probe(struct pci_dev *pdev, const struct pci_device_id *id) hw->tx_queues = qcount; hw->max_queues = qcount; hw->tot_tx_queues = qcount; + /* Use CQE of 128 byte descriptor size by default */ + hw->xqe_size = 128; hw->irq_name = devm_kmalloc_array(&hw->pdev->dev, num_vec, NAME_SIZE, GFP_KERNEL); From patchwork Tue Oct 5 17:15:36 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Subbaraya Sundeep X-Patchwork-Id: 12537343 X-Patchwork-Delegate: kuba@kernel.org Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 71C01C433FE for ; Tue, 5 Oct 2021 17:15:56 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 5B06F6117A for ; Tue, 5 Oct 2021 17:15:56 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S236729AbhJERRp (ORCPT ); Tue, 5 Oct 2021 13:17:45 -0400 Received: from mx0a-0016f401.pphosted.com ([67.231.148.174]:10942 "EHLO mx0b-0016f401.pphosted.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S236711AbhJERRo (ORCPT ); Tue, 5 Oct 2021 13:17:44 -0400 Received: from pps.filterd (m0045849.ppops.net [127.0.0.1]) by mx0a-0016f401.pphosted.com (8.16.1.2/8.16.1.2) with SMTP id 195Fub5D026710; Tue, 5 Oct 2021 10:15:53 -0700 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=marvell.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-type; s=pfpt0220; bh=Um9s9/TLHNrNMcB2c9SFF6AaxMvdu82WG4m2JK9f+8k=; b=OQp4WDQxAoFHeA/ybo1cMav78PdqH/mjDzJYr44wXKckjqXA+2jcPXPRVgtMEBcNof// Pmw5y0f9eK+rUuppmeiANDDEFpkwsaSIujStsny591pqaa92ODWV6OvD+r/pjSWvuTUV V6x4ls0H0Zvyua+OuYRGOSwlFpnWUYcKyqxonPGITU7WEpxkcoO9Ut4fgUEhFCVtGAFF av9l11TbaIa9Q5dgm3G7UMe49wMr1AxcLS4L/BlswgSdq2VbwDsmSf0zYNj6fghqUYxp FAWS8DGSqfwyBteFPQps4kEWRB9j6LrZ/3IykIjAB1RGNFNcDRYWxIT6jY5S0HvNme1C wQ== Received: from dc5-exch01.marvell.com ([199.233.59.181]) by mx0a-0016f401.pphosted.com with ESMTP id 3bgmv5t497-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-SHA384 bits=256 verify=NOT); Tue, 05 Oct 2021 10:15:52 -0700 Received: from DC5-EXCH02.marvell.com (10.69.176.39) by DC5-EXCH01.marvell.com (10.69.176.38) with Microsoft SMTP Server (TLS) id 15.0.1497.18; Tue, 5 Oct 2021 10:15:51 -0700 Received: from maili.marvell.com (10.69.176.80) by DC5-EXCH02.marvell.com (10.69.176.39) with Microsoft SMTP Server id 15.0.1497.18 via Frontend Transport; Tue, 5 Oct 2021 10:15:51 -0700 Received: from hyd1358.marvell.com (unknown [10.29.37.11]) by maili.marvell.com (Postfix) with ESMTP id 2D13D3F707C; Tue, 5 Oct 2021 10:15:48 -0700 (PDT) From: Subbaraya Sundeep To: , , CC: , , , Subbaraya Sundeep Subject: [net-next PATCH 3/3] octeontx2-pf: Add devlink param to vary rbuf size Date: Tue, 5 Oct 2021 22:45:36 +0530 Message-ID: <1633454136-14679-4-git-send-email-sbhatta@marvell.com> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1633454136-14679-1-git-send-email-sbhatta@marvell.com> References: <1633454136-14679-1-git-send-email-sbhatta@marvell.com> MIME-Version: 1.0 X-Proofpoint-GUID: vwQlzVhR040hVQKN4MCDYFcHbshzwvpI X-Proofpoint-ORIG-GUID: vwQlzVhR040hVQKN4MCDYFcHbshzwvpI X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.182.1,Aquarius:18.0.790,Hydra:6.0.391,FMLib:17.0.607.475 definitions=2021-10-05_03,2021-10-04_01,2020-04-07_01 Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org X-Patchwork-Delegate: kuba@kernel.org The size of receive buffers for receiving packets is calculated based on the mtu of interface. This patch adds devlink parameter to utilize user given size for receive buffers instead. By changing CQE descriptor size and receive buffer sizes the number of buffer pointers used by hardware for a large packet can be configured. Signed-off-by: Subbaraya Sundeep Signed-off-by: Sunil Goutham --- .../ethernet/marvell/octeontx2/nic/otx2_common.h | 1 + .../ethernet/marvell/octeontx2/nic/otx2_devlink.c | 60 ++++++++++++++++++++++ .../net/ethernet/marvell/octeontx2/nic/otx2_pf.c | 7 +++ .../net/ethernet/marvell/octeontx2/nic/otx2_vf.c | 5 ++ 4 files changed, 73 insertions(+) diff --git a/drivers/net/ethernet/marvell/octeontx2/nic/otx2_common.h b/drivers/net/ethernet/marvell/octeontx2/nic/otx2_common.h index 6e0d1ac..f885abe 100644 --- a/drivers/net/ethernet/marvell/octeontx2/nic/otx2_common.h +++ b/drivers/net/ethernet/marvell/octeontx2/nic/otx2_common.h @@ -178,6 +178,7 @@ struct otx2_hw { u16 rqpool_cnt; u16 sqpool_cnt; u16 xqe_size; + u16 rbuf_fixed_size; /* NPA */ u32 stack_pg_ptrs; /* No of ptrs per stack page */ diff --git a/drivers/net/ethernet/marvell/octeontx2/nic/otx2_devlink.c b/drivers/net/ethernet/marvell/octeontx2/nic/otx2_devlink.c index 98450e1..0fc7e32 100644 --- a/drivers/net/ethernet/marvell/octeontx2/nic/otx2_devlink.c +++ b/drivers/net/ethernet/marvell/octeontx2/nic/otx2_devlink.c @@ -64,6 +64,60 @@ static int otx2_dl_mcam_count_get(struct devlink *devlink, u32 id, return 0; } +static int otx2_dl_rbuf_size_validate(struct devlink *devlink, u32 id, + union devlink_param_value val, + struct netlink_ext_ack *extack) +{ + /* Hardware supports max size of 32k for a receive buffer + * and 1536 is typical ethernet frame size. + */ + if (val.vu16 < 1536 || val.vu16 > 32768) { + NL_SET_ERR_MSG_MOD(extack, + "Receive buffer range is 1536 - 32768"); + return -EINVAL; + } + + return 0; +} + +static int otx2_dl_rbuf_size_set(struct devlink *devlink, u32 id, + struct devlink_param_gset_ctx *ctx) +{ + struct otx2_devlink *otx2_dl = devlink_priv(devlink); + struct otx2_nic *pfvf = otx2_dl->pfvf; + struct net_device *netdev; + int err = 0; + bool if_up; + + rtnl_lock(); + + netdev = pfvf->netdev; + if_up = netif_running(netdev); + if (if_up) + netdev->netdev_ops->ndo_stop(netdev); + + pfvf->hw.rbuf_fixed_size = ALIGN(ctx->val.vu16, OTX2_ALIGN) + + OTX2_HEAD_ROOM; + + if (if_up) + err = netdev->netdev_ops->ndo_open(netdev); + + rtnl_unlock(); + + return err; +} + +static int otx2_dl_rbuf_size_get(struct devlink *devlink, u32 id, + struct devlink_param_gset_ctx *ctx) +{ + struct otx2_devlink *otx2_dl = devlink_priv(devlink); + struct otx2_nic *pfvf = otx2_dl->pfvf; + + ctx->val.vu16 = pfvf->hw.rbuf_fixed_size; + + return 0; +} + static int otx2_dl_cqe_size_validate(struct devlink *devlink, u32 id, union devlink_param_value val, struct netlink_ext_ack *extack) @@ -118,6 +172,7 @@ enum otx2_dl_param_id { OTX2_DEVLINK_PARAM_ID_BASE = DEVLINK_PARAM_GENERIC_ID_MAX, OTX2_DEVLINK_PARAM_ID_MCAM_COUNT, OTX2_DEVLINK_PARAM_ID_CQE_SIZE, + OTX2_DEVLINK_PARAM_ID_RBUF_SIZE, }; static const struct devlink_param otx2_dl_params[] = { @@ -131,6 +186,11 @@ static const struct devlink_param otx2_dl_params[] = { BIT(DEVLINK_PARAM_CMODE_RUNTIME), otx2_dl_cqe_size_get, otx2_dl_cqe_size_set, otx2_dl_cqe_size_validate), + DEVLINK_PARAM_DRIVER(OTX2_DEVLINK_PARAM_ID_RBUF_SIZE, + "receive_buffer_size", DEVLINK_PARAM_TYPE_U16, + BIT(DEVLINK_PARAM_CMODE_RUNTIME), + otx2_dl_rbuf_size_get, otx2_dl_rbuf_size_set, + otx2_dl_rbuf_size_validate), }; /* Devlink OPs */ diff --git a/drivers/net/ethernet/marvell/octeontx2/nic/otx2_pf.c b/drivers/net/ethernet/marvell/octeontx2/nic/otx2_pf.c index 8618cf7..1be524d3 100644 --- a/drivers/net/ethernet/marvell/octeontx2/nic/otx2_pf.c +++ b/drivers/net/ethernet/marvell/octeontx2/nic/otx2_pf.c @@ -65,6 +65,10 @@ static int otx2_change_mtu(struct net_device *netdev, int new_mtu) netdev_info(netdev, "Changing MTU from %d to %d\n", netdev->mtu, new_mtu); netdev->mtu = new_mtu; + /* Modify receive buffer size based on MTU and do not + * use the fixed size set. + */ + pf->hw.rbuf_fixed_size = 0; if (if_up) err = otx2_open(netdev); @@ -1306,6 +1310,9 @@ static int otx2_get_rbuf_size(struct otx2_nic *pf, int mtu) int total_size; int rbuf_size; + if (pf->hw.rbuf_fixed_size) + return pf->hw.rbuf_fixed_size; + /* The data transferred by NIX to memory consists of actual packet * plus additional data which has timestamp and/or EDSA/HIGIG2 * headers if interface is configured in corresponding modes. diff --git a/drivers/net/ethernet/marvell/octeontx2/nic/otx2_vf.c b/drivers/net/ethernet/marvell/octeontx2/nic/otx2_vf.c index 672be05..229d051 100644 --- a/drivers/net/ethernet/marvell/octeontx2/nic/otx2_vf.c +++ b/drivers/net/ethernet/marvell/octeontx2/nic/otx2_vf.c @@ -439,6 +439,7 @@ static void otx2vf_do_set_rx_mode(struct work_struct *work) static int otx2vf_change_mtu(struct net_device *netdev, int new_mtu) { + struct otx2_nic *vf = netdev_priv(netdev); bool if_up = netif_running(netdev); int err = 0; @@ -448,6 +449,10 @@ static int otx2vf_change_mtu(struct net_device *netdev, int new_mtu) netdev_info(netdev, "Changing MTU from %d to %d\n", netdev->mtu, new_mtu); netdev->mtu = new_mtu; + /* Modify receive buffer size based on MTU and do not + * use the fixed size set. + */ + vf->hw.rbuf_fixed_size = 0; if (if_up) err = otx2vf_open(netdev);