From patchwork Wed Sep 15 03:52:51 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sukadev Bhattiprolu X-Patchwork-Id: 12495435 X-Patchwork-Delegate: kuba@kernel.org Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 9F1F5C433F5 for ; Wed, 15 Sep 2021 03:53:28 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 8644260F46 for ; Wed, 15 Sep 2021 03:53:28 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S236238AbhIODyp (ORCPT ); Tue, 14 Sep 2021 23:54:45 -0400 Received: from mx0b-001b2d01.pphosted.com ([148.163.158.5]:16258 "EHLO mx0a-001b2d01.pphosted.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S236243AbhIODyX (ORCPT ); Tue, 14 Sep 2021 23:54:23 -0400 Received: from pps.filterd (m0098419.ppops.net [127.0.0.1]) by mx0b-001b2d01.pphosted.com (8.16.1.2/8.16.0.43) with SMTP id 18F1xoHu021384 for ; Tue, 14 Sep 2021 23:53:05 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ibm.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding; s=pp1; bh=gNwRShj1fCPOjbvcZzcbmqfKAtdoDisxZvVIpOU9ySc=; b=YwkIP9lyzSPCM+OP6GVQEDODeuZ/3DttAuFQb1TkxqsfmP03p1ChpsxpCJSHfMfL3FZF b7/7Mt5lrBEI8I9OeQJfSKQ0LZimHQY5Y7DcyvfhiOlBh0s6vwlsMgMwjbBlo2LQt1Kt Zb78d9GkVy0N8aeCtFQhmmba86KneR4HAkRaZbNQoD5tpMzTb/JsBxZqpE4PrxMuV3J5 +tw31Qd/26+UUxjiU6ZbOMuJfHgJH1TpcrPKVh1bshiNpMypIJIFicJB951CqggIodEG cwO2iv9BAoIXDVlgMU+bned6KC6oe5cnof9PxEofyZ5/2a4b/eKSTAHhc5VKxG+sZwbO vg== Received: from ppma02wdc.us.ibm.com (aa.5b.37a9.ip4.static.sl-reverse.com [169.55.91.170]) by mx0b-001b2d01.pphosted.com with ESMTP id 3b37nn1kg5-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT) for ; Tue, 14 Sep 2021 23:53:04 -0400 Received: from pps.filterd (ppma02wdc.us.ibm.com [127.0.0.1]) by ppma02wdc.us.ibm.com (8.16.1.2/8.16.1.2) with SMTP id 18F3lMj6023145 for ; Wed, 15 Sep 2021 03:53:04 GMT Received: from b01cxnp23033.gho.pok.ibm.com (b01cxnp23033.gho.pok.ibm.com [9.57.198.28]) by ppma02wdc.us.ibm.com with ESMTP id 3b0m3bg560-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT) for ; Wed, 15 Sep 2021 03:53:04 +0000 Received: from b01ledav004.gho.pok.ibm.com (b01ledav004.gho.pok.ibm.com [9.57.199.109]) by b01cxnp23033.gho.pok.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id 18F3r3g214287226 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Wed, 15 Sep 2021 03:53:03 GMT Received: from b01ledav004.gho.pok.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 537D7112063; Wed, 15 Sep 2021 03:53:03 +0000 (GMT) Received: from b01ledav004.gho.pok.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id E4473112061; Wed, 15 Sep 2021 03:53:01 +0000 (GMT) Received: from suka-w540.ibmuc.com (unknown [9.77.142.77]) by b01ledav004.gho.pok.ibm.com (Postfix) with ESMTP; Wed, 15 Sep 2021 03:53:01 +0000 (GMT) From: Sukadev Bhattiprolu To: netdev@vger.kernel.org Cc: Brian King , cforno12@linux.ibm.com, Dany Madden , Rick Lindsley , Rick Lindsley Subject: [PATCH net-next RESEND v2 1/9] ibmvnic: Consolidate code in replenish_rx_pool() Date: Tue, 14 Sep 2021 20:52:51 -0700 Message-Id: <20210915035259.355092-2-sukadev@linux.ibm.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20210915035259.355092-1-sukadev@linux.ibm.com> References: <20210915035259.355092-1-sukadev@linux.ibm.com> MIME-Version: 1.0 X-TM-AS-GCONF: 00 X-Proofpoint-ORIG-GUID: rm7TCnnyoNP_txH4Wc7wcusE9ALEHnCe X-Proofpoint-GUID: rm7TCnnyoNP_txH4Wc7wcusE9ALEHnCe X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.182.1,Aquarius:18.0.687,Hydra:6.0.235,FMLib:17.0.607.475 definitions=2020-10-13_15,2020-10-13_02,2020-04-07_01 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 priorityscore=1501 adultscore=0 bulkscore=0 impostorscore=0 mlxscore=0 malwarescore=0 mlxlogscore=999 lowpriorityscore=0 suspectscore=0 spamscore=0 phishscore=0 clxscore=1015 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2109030001 definitions=main-2109150000 Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org X-Patchwork-Delegate: kuba@kernel.org For better readability, consolidate related code in replenish_rx_pool() and add some comments. Reviewed-by: Rick Lindsley Reviewed-by: Dany Madden Signed-off-by: Sukadev Bhattiprolu --- drivers/net/ethernet/ibm/ibmvnic.c | 10 +++++++--- 1 file changed, 7 insertions(+), 3 deletions(-) diff --git a/drivers/net/ethernet/ibm/ibmvnic.c b/drivers/net/ethernet/ibm/ibmvnic.c index a775c69e4fd7..e8b1231be485 100644 --- a/drivers/net/ethernet/ibm/ibmvnic.c +++ b/drivers/net/ethernet/ibm/ibmvnic.c @@ -371,6 +371,8 @@ static void replenish_rx_pool(struct ibmvnic_adapter *adapter, } index = pool->free_map[pool->next_free]; + pool->free_map[pool->next_free] = IBMVNIC_INVALID_MAP; + pool->next_free = (pool->next_free + 1) % pool->size; if (pool->rx_buff[index].skb) dev_err(dev, "Inconsistent free_map!\n"); @@ -380,14 +382,15 @@ static void replenish_rx_pool(struct ibmvnic_adapter *adapter, dst = pool->long_term_buff.buff + offset; memset(dst, 0, pool->buff_size); dma_addr = pool->long_term_buff.addr + offset; - pool->rx_buff[index].data = dst; - pool->free_map[pool->next_free] = IBMVNIC_INVALID_MAP; + /* add the skb to an rx_buff in the pool */ + pool->rx_buff[index].data = dst; pool->rx_buff[index].dma = dma_addr; pool->rx_buff[index].skb = skb; pool->rx_buff[index].pool_index = pool->index; pool->rx_buff[index].size = pool->buff_size; + /* queue the rx_buff for the next send_subcrq_indirect */ sub_crq = &ind_bufp->indir_arr[ind_bufp->index++]; memset(sub_crq, 0, sizeof(*sub_crq)); sub_crq->rx_add.first = IBMVNIC_CRQ_CMD; @@ -405,7 +408,8 @@ static void replenish_rx_pool(struct ibmvnic_adapter *adapter, shift = 8; #endif sub_crq->rx_add.len = cpu_to_be32(pool->buff_size << shift); - pool->next_free = (pool->next_free + 1) % pool->size; + + /* if send_subcrq_indirect queue is full, flush to VIOS */ if (ind_bufp->index == IBMVNIC_MAX_IND_DESCS || i == count - 1) { lpar_rc = From patchwork Wed Sep 15 03:52:52 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sukadev Bhattiprolu X-Patchwork-Id: 12495437 X-Patchwork-Delegate: kuba@kernel.org Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id C2D61C433F5 for ; Wed, 15 Sep 2021 03:53:31 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id A193460F3A for ; Wed, 15 Sep 2021 03:53:31 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S236481AbhIODyr (ORCPT ); Tue, 14 Sep 2021 23:54:47 -0400 Received: from mx0a-001b2d01.pphosted.com ([148.163.156.1]:35790 "EHLO mx0a-001b2d01.pphosted.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S236402AbhIODyZ (ORCPT ); Tue, 14 Sep 2021 23:54:25 -0400 Received: from pps.filterd (m0187473.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.16.1.2/8.16.0.43) with SMTP id 18F3piZi039423 for ; Tue, 14 Sep 2021 23:53:07 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ibm.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding; s=pp1; bh=yzvZRREa1kH9KKs3VqfWD5NvSJDKi7LX77rQ21o/+/4=; b=WPzaCuWOw/b1U81mvoQf0Oc4B4Lm8XMgie5Uv1LovRkfumfajbaduxhd+kbdqMx5sB2b P+N0Nfp3sPrLU8Fe4T+i3zHHVrt2aOQuYT+T3VZs8q553f43jJBTDn/iJ9OrMvICqwsB BuOt+vA/LK5GP1Jh55wDDa5LkAuqsyqBB5+MG2dALc3CDnX/+tI2/Tx6JOiXEfnpW+FN HP/0ZQyFCYOpCjXkyb+OPOvUvXWVtnif8nQJNVJnbenGVs5Qjr0Ibo84knXJefjvGRSv 37ft9v2oEPYQD+qgrt5N57E21YDhwMwRQ7G/YlPMb6lYnKWWzOOd8KGT3t874usDouhh fQ== Received: from ppma03dal.us.ibm.com (b.bd.3ea9.ip4.static.sl-reverse.com [169.62.189.11]) by mx0a-001b2d01.pphosted.com with ESMTP id 3b32u2fc6r-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT) for ; Tue, 14 Sep 2021 23:53:07 -0400 Received: from pps.filterd (ppma03dal.us.ibm.com [127.0.0.1]) by ppma03dal.us.ibm.com (8.16.1.2/8.16.1.2) with SMTP id 18F3lMKN016682 for ; Wed, 15 Sep 2021 03:53:06 GMT Received: from b01cxnp22034.gho.pok.ibm.com (b01cxnp22034.gho.pok.ibm.com [9.57.198.24]) by ppma03dal.us.ibm.com with ESMTP id 3b0m3bhkjr-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT) for ; Wed, 15 Sep 2021 03:53:06 +0000 Received: from b01ledav004.gho.pok.ibm.com (b01ledav004.gho.pok.ibm.com [9.57.199.109]) by b01cxnp22034.gho.pok.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id 18F3r5oq38535496 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Wed, 15 Sep 2021 03:53:05 GMT Received: from b01ledav004.gho.pok.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 48F0D11206B; Wed, 15 Sep 2021 03:53:05 +0000 (GMT) Received: from b01ledav004.gho.pok.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id BA0F2112066; Wed, 15 Sep 2021 03:53:03 +0000 (GMT) Received: from suka-w540.ibmuc.com (unknown [9.77.142.77]) by b01ledav004.gho.pok.ibm.com (Postfix) with ESMTP; Wed, 15 Sep 2021 03:53:03 +0000 (GMT) From: Sukadev Bhattiprolu To: netdev@vger.kernel.org Cc: Brian King , cforno12@linux.ibm.com, Dany Madden , Rick Lindsley , Rick Lindsley Subject: [PATCH net-next RESEND v2 2/9] ibmvnic: Fix up some comments and messages Date: Tue, 14 Sep 2021 20:52:52 -0700 Message-Id: <20210915035259.355092-3-sukadev@linux.ibm.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20210915035259.355092-1-sukadev@linux.ibm.com> References: <20210915035259.355092-1-sukadev@linux.ibm.com> MIME-Version: 1.0 X-TM-AS-GCONF: 00 X-Proofpoint-GUID: oqKL2CIeZtSOnjZI_e4mTEIy12OsR4On X-Proofpoint-ORIG-GUID: oqKL2CIeZtSOnjZI_e4mTEIy12OsR4On X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.182.1,Aquarius:18.0.790,Hydra:6.0.391,FMLib:17.0.607.475 definitions=2021-09-11_02,2021-09-09_01,2020-04-07_01 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 mlxscore=0 spamscore=0 bulkscore=0 impostorscore=0 suspectscore=0 mlxlogscore=999 lowpriorityscore=0 clxscore=1015 adultscore=0 phishscore=0 priorityscore=1501 malwarescore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2109030001 definitions=main-2109140109 Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org X-Patchwork-Delegate: kuba@kernel.org Add/update some comments/function headers and fix up some messages. Reviewed-by: Rick Lindsley Reviewed-by: Dany Madden Signed-off-by: Sukadev Bhattiprolu --- Changelog [v2]: [Jakub Kicinski] Fix kdoc issues --- drivers/net/ethernet/ibm/ibmvnic.c | 44 ++++++++++++++++++++++++++---- 1 file changed, 38 insertions(+), 6 deletions(-) diff --git a/drivers/net/ethernet/ibm/ibmvnic.c b/drivers/net/ethernet/ibm/ibmvnic.c index e8b1231be485..cd04c3eb6c41 100644 --- a/drivers/net/ethernet/ibm/ibmvnic.c +++ b/drivers/net/ethernet/ibm/ibmvnic.c @@ -243,14 +243,13 @@ static int alloc_long_term_buff(struct ibmvnic_adapter *adapter, rc = ibmvnic_wait_for_completion(adapter, &adapter->fw_done, 10000); if (rc) { - dev_err(dev, - "Long term map request aborted or timed out,rc = %d\n", + dev_err(dev, "LTB map request aborted or timed out, rc = %d\n", rc); goto out; } if (adapter->fw_done_rc) { - dev_err(dev, "Couldn't map long term buffer,rc = %d\n", + dev_err(dev, "Couldn't map LTB, rc = %d\n", adapter->fw_done_rc); rc = -1; goto out; @@ -281,7 +280,9 @@ static void free_long_term_buff(struct ibmvnic_adapter *adapter, adapter->reset_reason != VNIC_RESET_MOBILITY && adapter->reset_reason != VNIC_RESET_TIMEOUT) send_request_unmap(adapter, ltb->map_id); + dma_free_coherent(dev, ltb->size, ltb->buff, ltb->addr); + ltb->buff = NULL; ltb->map_id = 0; } @@ -574,6 +575,12 @@ static int reset_rx_pools(struct ibmvnic_adapter *adapter) return 0; } +/** + * release_rx_pools() - Release any rx pools attached to @adapter. + * @adapter: ibmvnic adapter + * + * Safe to call this multiple times - even if no pools are attached. + */ static void release_rx_pools(struct ibmvnic_adapter *adapter) { struct ibmvnic_rx_pool *rx_pool; @@ -628,6 +635,9 @@ static int init_rx_pools(struct net_device *netdev) return -1; } + /* Set num_active_rx_pools early. If we fail below after partial + * allocation, release_rx_pools() will know how many to look for. + */ adapter->num_active_rx_pools = rxadd_subcrqs; for (i = 0; i < rxadd_subcrqs; i++) { @@ -646,6 +656,7 @@ static int init_rx_pools(struct net_device *netdev) rx_pool->free_map = kcalloc(rx_pool->size, sizeof(int), GFP_KERNEL); if (!rx_pool->free_map) { + dev_err(dev, "Couldn't alloc free_map %d\n", i); release_rx_pools(adapter); return -1; } @@ -739,10 +750,19 @@ static void release_one_tx_pool(struct ibmvnic_adapter *adapter, free_long_term_buff(adapter, &tx_pool->long_term_buff); } +/** + * release_tx_pools() - Release any tx pools attached to @adapter. + * @adapter: ibmvnic adapter + * + * Safe to call this multiple times - even if no pools are attached. + */ static void release_tx_pools(struct ibmvnic_adapter *adapter) { int i; + /* init_tx_pools() ensures that ->tx_pool and ->tso_pool are + * both NULL or both non-NULL. So we only need to check one. + */ if (!adapter->tx_pool) return; @@ -793,6 +813,7 @@ static int init_one_tx_pool(struct net_device *netdev, static int init_tx_pools(struct net_device *netdev) { struct ibmvnic_adapter *adapter = netdev_priv(netdev); + struct device *dev = &adapter->vdev->dev; int tx_subcrqs; u64 buff_size; int i, rc; @@ -805,17 +826,27 @@ static int init_tx_pools(struct net_device *netdev) adapter->tso_pool = kcalloc(tx_subcrqs, sizeof(struct ibmvnic_tx_pool), GFP_KERNEL); + /* To simplify release_tx_pools() ensure that ->tx_pool and + * ->tso_pool are either both NULL or both non-NULL. + */ if (!adapter->tso_pool) { kfree(adapter->tx_pool); adapter->tx_pool = NULL; return -1; } + /* Set num_active_tx_pools early. If we fail below after partial + * allocation, release_tx_pools() will know how many to look for. + */ adapter->num_active_tx_pools = tx_subcrqs; for (i = 0; i < tx_subcrqs; i++) { buff_size = adapter->req_mtu + VLAN_HLEN; buff_size = ALIGN(buff_size, L1_CACHE_BYTES); + + dev_dbg(dev, "Init tx pool %d [%llu, %llu]\n", + i, adapter->req_tx_entries_per_subcrq, buff_size); + rc = init_one_tx_pool(netdev, &adapter->tx_pool[i], adapter->req_tx_entries_per_subcrq, buff_size); @@ -4774,9 +4805,10 @@ static void handle_query_map_rsp(union ibmvnic_crq *crq, dev_err(dev, "Error %ld in QUERY_MAP_RSP\n", rc); return; } - netdev_dbg(netdev, "page_size = %d\ntot_pages = %d\nfree_pages = %d\n", - crq->query_map_rsp.page_size, crq->query_map_rsp.tot_pages, - crq->query_map_rsp.free_pages); + netdev_dbg(netdev, "page_size = %d\ntot_pages = %u\nfree_pages = %u\n", + crq->query_map_rsp.page_size, + __be32_to_cpu(crq->query_map_rsp.tot_pages), + __be32_to_cpu(crq->query_map_rsp.free_pages)); } static void handle_query_cap_rsp(union ibmvnic_crq *crq, From patchwork Wed Sep 15 03:52:53 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sukadev Bhattiprolu X-Patchwork-Id: 12495439 X-Patchwork-Delegate: kuba@kernel.org Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 866A8C433EF for ; Wed, 15 Sep 2021 03:53:33 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 6E22D60724 for ; Wed, 15 Sep 2021 03:53:33 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S236503AbhIODyu (ORCPT ); Tue, 14 Sep 2021 23:54:50 -0400 Received: from mx0a-001b2d01.pphosted.com ([148.163.156.1]:49624 "EHLO mx0a-001b2d01.pphosted.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S236495AbhIODy1 (ORCPT ); Tue, 14 Sep 2021 23:54:27 -0400 Received: from pps.filterd (m0098409.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.16.1.2/8.16.0.43) with SMTP id 18ENxhhw026824 for ; Tue, 14 Sep 2021 23:53:09 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ibm.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding; s=pp1; bh=M6wOHtzBzQR8PzwJII4hlzFN1NNx5ohs41Ao5oYuLo4=; b=MrPwLJk74tnqkfmL7i+nJiMnSciOOT9AINxl00Rtij3IKWEcHbme7u+Uedeq5/72QtvC Wbl51cQX2CHZkNxjajQRK0zlArh1Nmz8pPce71PTK21mjxBZP5oy8CbbUXAKPqBREYyT kcjDgmlPb4nB62czr9Jk/b8fU+mT0HSbO2nZs6HbeoQC7YLsayldnvjjkX2cobz22nYK zXonrsE6GyHLWcNQniU9iPJf/hSNhOvBfUsyh8Eo2ETIY3hl59KtboR2dCTiXt+GqL8V gUw3ncJKn/W3h38wz5k3KM4ceevZBODVA6KV2TQTVbwVA+JXIAA1MA6eWbItEeOXx96e Vg== Received: from ppma03wdc.us.ibm.com (ba.79.3fa9.ip4.static.sl-reverse.com [169.63.121.186]) by mx0a-001b2d01.pphosted.com with ESMTP id 3b35wabnue-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT) for ; Tue, 14 Sep 2021 23:53:09 -0400 Received: from pps.filterd (ppma03wdc.us.ibm.com [127.0.0.1]) by ppma03wdc.us.ibm.com (8.16.1.2/8.16.1.2) with SMTP id 18F3lPU2006021 for ; Wed, 15 Sep 2021 03:53:07 GMT Received: from b01cxnp22035.gho.pok.ibm.com (b01cxnp22035.gho.pok.ibm.com [9.57.198.25]) by ppma03wdc.us.ibm.com with ESMTP id 3b0m3b84xk-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT) for ; Wed, 15 Sep 2021 03:53:07 +0000 Received: from b01ledav004.gho.pok.ibm.com (b01ledav004.gho.pok.ibm.com [9.57.199.109]) by b01cxnp22035.gho.pok.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id 18F3r7m937159316 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Wed, 15 Sep 2021 03:53:07 GMT Received: from b01ledav004.gho.pok.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 3957E112063; Wed, 15 Sep 2021 03:53:07 +0000 (GMT) Received: from b01ledav004.gho.pok.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 976FB112061; Wed, 15 Sep 2021 03:53:05 +0000 (GMT) Received: from suka-w540.ibmuc.com (unknown [9.77.142.77]) by b01ledav004.gho.pok.ibm.com (Postfix) with ESMTP; Wed, 15 Sep 2021 03:53:05 +0000 (GMT) From: Sukadev Bhattiprolu To: netdev@vger.kernel.org Cc: Brian King , cforno12@linux.ibm.com, Dany Madden , Rick Lindsley , Rick Lindsley Subject: [PATCH net-next RESEND v2 3/9] ibmvnic: Use/rename local vars in init_rx_pools Date: Tue, 14 Sep 2021 20:52:53 -0700 Message-Id: <20210915035259.355092-4-sukadev@linux.ibm.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20210915035259.355092-1-sukadev@linux.ibm.com> References: <20210915035259.355092-1-sukadev@linux.ibm.com> MIME-Version: 1.0 X-TM-AS-GCONF: 00 X-Proofpoint-ORIG-GUID: 0aQpEKIb5s4AscG0YtFfc3L_7Ao5J_MG X-Proofpoint-GUID: 0aQpEKIb5s4AscG0YtFfc3L_7Ao5J_MG X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.182.1,Aquarius:18.0.687,Hydra:6.0.235,FMLib:17.0.607.475 definitions=2020-10-13_15,2020-10-13_02,2020-04-07_01 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 mlxscore=0 bulkscore=0 lowpriorityscore=0 spamscore=0 phishscore=0 impostorscore=0 malwarescore=0 adultscore=0 mlxlogscore=999 priorityscore=1501 clxscore=1015 suspectscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2109030001 definitions=main-2109140133 Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org X-Patchwork-Delegate: kuba@kernel.org To make the code more readable, use/rename some local variables. Basically we have a set of pools, num_pools. Each pool has a set of buffers, pool_size and each buffer is of size buff_size. pool_size is a bit ambiguous (whether size in bytes or buffers). Add a comment in the header file to make it explicit. Reviewed-by: Rick Lindsley Reviewed-by: Dany Madden Signed-off-by: Sukadev Bhattiprolu --- drivers/net/ethernet/ibm/ibmvnic.c | 17 +++++++++-------- drivers/net/ethernet/ibm/ibmvnic.h | 2 +- 2 files changed, 10 insertions(+), 9 deletions(-) diff --git a/drivers/net/ethernet/ibm/ibmvnic.c b/drivers/net/ethernet/ibm/ibmvnic.c index cd04c3eb6c41..688489475791 100644 --- a/drivers/net/ethernet/ibm/ibmvnic.c +++ b/drivers/net/ethernet/ibm/ibmvnic.c @@ -620,14 +620,16 @@ static int init_rx_pools(struct net_device *netdev) struct ibmvnic_adapter *adapter = netdev_priv(netdev); struct device *dev = &adapter->vdev->dev; struct ibmvnic_rx_pool *rx_pool; - int rxadd_subcrqs; + u64 num_pools; + u64 pool_size; /* # of buffers in one pool */ u64 buff_size; int i, j; - rxadd_subcrqs = adapter->num_active_rx_scrqs; + num_pools = adapter->num_active_rx_scrqs; + pool_size = adapter->req_rx_add_entries_per_subcrq; buff_size = adapter->cur_rx_buf_sz; - adapter->rx_pool = kcalloc(rxadd_subcrqs, + adapter->rx_pool = kcalloc(num_pools, sizeof(struct ibmvnic_rx_pool), GFP_KERNEL); if (!adapter->rx_pool) { @@ -638,17 +640,16 @@ static int init_rx_pools(struct net_device *netdev) /* Set num_active_rx_pools early. If we fail below after partial * allocation, release_rx_pools() will know how many to look for. */ - adapter->num_active_rx_pools = rxadd_subcrqs; + adapter->num_active_rx_pools = num_pools; - for (i = 0; i < rxadd_subcrqs; i++) { + for (i = 0; i < num_pools; i++) { rx_pool = &adapter->rx_pool[i]; netdev_dbg(adapter->netdev, "Initializing rx_pool[%d], %lld buffs, %lld bytes each\n", - i, adapter->req_rx_add_entries_per_subcrq, - buff_size); + i, pool_size, buff_size); - rx_pool->size = adapter->req_rx_add_entries_per_subcrq; + rx_pool->size = pool_size; rx_pool->index = i; rx_pool->buff_size = ALIGN(buff_size, L1_CACHE_BYTES); rx_pool->active = 1; diff --git a/drivers/net/ethernet/ibm/ibmvnic.h b/drivers/net/ethernet/ibm/ibmvnic.h index 22df602323bc..5652566818fb 100644 --- a/drivers/net/ethernet/ibm/ibmvnic.h +++ b/drivers/net/ethernet/ibm/ibmvnic.h @@ -827,7 +827,7 @@ struct ibmvnic_rx_buff { struct ibmvnic_rx_pool { struct ibmvnic_rx_buff *rx_buff; - int size; + int size; /* # of buffers in the pool */ int index; int buff_size; atomic_t available; From patchwork Wed Sep 15 03:52:54 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sukadev Bhattiprolu X-Patchwork-Id: 12495441 X-Patchwork-Delegate: kuba@kernel.org Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id C9EB6C433FE for ; Wed, 15 Sep 2021 03:53:34 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id B8BFB61164 for ; Wed, 15 Sep 2021 03:53:34 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S236506AbhIODyw (ORCPT ); Tue, 14 Sep 2021 23:54:52 -0400 Received: from mx0b-001b2d01.pphosted.com ([148.163.158.5]:46498 "EHLO mx0a-001b2d01.pphosted.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S236507AbhIODyb (ORCPT ); Tue, 14 Sep 2021 23:54:31 -0400 Received: from pps.filterd (m0098419.ppops.net [127.0.0.1]) by mx0b-001b2d01.pphosted.com (8.16.1.2/8.16.0.43) with SMTP id 18F1xtox021443 for ; Tue, 14 Sep 2021 23:53:11 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ibm.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding; s=pp1; bh=qK539ZJP6twyLxjqj4LwRUZgKMsNonqGDmsnEYIeWKs=; b=iaJ0HbAQMDs/YHG37LbBfvH6AHWXTWaaD/xcqcYbJEduVR7hGFV4KG2q0adV7K77QbHa 7RWukRgg8qPPzfa8M0hwaHEo+gOwAx6RIkv+KTQLNcwE0QFf1zN7uDmQi4h8m55cdH5u oNVEsd+rJsc9lrsbJPeH17ShSpvlbRh745kwG+4qKXs0jxMiL7bs2FbyO9177fiCX1T5 CbsRaU4ZQA5SP6c1d3Rm2Wr0SdECxDsWvw6sm4sWWKpm3F/iXLQOFkqCiqt+mXFBlx9X aDzEm2MEZCqLfANoWfI8HKt7BWydtmYFvq9fNA9m0E5pUHlmIYvD6lrT7Q/1MhCHway3 nw== Received: from ppma02dal.us.ibm.com (a.bd.3ea9.ip4.static.sl-reverse.com [169.62.189.10]) by mx0b-001b2d01.pphosted.com with ESMTP id 3b37nn1kht-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT) for ; Tue, 14 Sep 2021 23:53:11 -0400 Received: from pps.filterd (ppma02dal.us.ibm.com [127.0.0.1]) by ppma02dal.us.ibm.com (8.16.1.2/8.16.1.2) with SMTP id 18F3lpjN024768 for ; Wed, 15 Sep 2021 03:53:11 GMT Received: from b01cxnp23032.gho.pok.ibm.com (b01cxnp23032.gho.pok.ibm.com [9.57.198.27]) by ppma02dal.us.ibm.com with ESMTP id 3b0m3bhjq7-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT) for ; Wed, 15 Sep 2021 03:53:11 +0000 Received: from b01ledav004.gho.pok.ibm.com (b01ledav004.gho.pok.ibm.com [9.57.199.109]) by b01cxnp23032.gho.pok.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id 18F3rATZ51642630 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Wed, 15 Sep 2021 03:53:10 GMT Received: from b01ledav004.gho.pok.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id DC3E3112061; Wed, 15 Sep 2021 03:53:09 +0000 (GMT) Received: from b01ledav004.gho.pok.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 9CEA0112067; Wed, 15 Sep 2021 03:53:07 +0000 (GMT) Received: from suka-w540.ibmuc.com (unknown [9.77.142.77]) by b01ledav004.gho.pok.ibm.com (Postfix) with ESMTP; Wed, 15 Sep 2021 03:53:07 +0000 (GMT) From: Sukadev Bhattiprolu To: netdev@vger.kernel.org Cc: Brian King , cforno12@linux.ibm.com, Dany Madden , Rick Lindsley , Rick Lindsley Subject: [PATCH net-next RESEND v2 4/9] ibmvnic: Use/rename local vars in init_tx_pools Date: Tue, 14 Sep 2021 20:52:54 -0700 Message-Id: <20210915035259.355092-5-sukadev@linux.ibm.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20210915035259.355092-1-sukadev@linux.ibm.com> References: <20210915035259.355092-1-sukadev@linux.ibm.com> MIME-Version: 1.0 X-TM-AS-GCONF: 00 X-Proofpoint-ORIG-GUID: UTi3XRGhVFbCfuw87pWeTbhQO7QAR0WL X-Proofpoint-GUID: UTi3XRGhVFbCfuw87pWeTbhQO7QAR0WL X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.182.1,Aquarius:18.0.687,Hydra:6.0.235,FMLib:17.0.607.475 definitions=2020-10-13_15,2020-10-13_02,2020-04-07_01 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 priorityscore=1501 adultscore=0 bulkscore=0 impostorscore=0 mlxscore=0 malwarescore=0 mlxlogscore=999 lowpriorityscore=0 suspectscore=0 spamscore=0 phishscore=0 clxscore=1015 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2109030001 definitions=main-2109150000 Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org X-Patchwork-Delegate: kuba@kernel.org Use/rename local variables in init_tx_pools() for consistency with init_rx_pools() and for readability. Also add some comments Reviewed-by: Rick Lindsley Reviewed-by: Dany Madden Signed-off-by: Sukadev Bhattiprolu --- drivers/net/ethernet/ibm/ibmvnic.c | 30 ++++++++++++++++-------------- 1 file changed, 16 insertions(+), 14 deletions(-) diff --git a/drivers/net/ethernet/ibm/ibmvnic.c b/drivers/net/ethernet/ibm/ibmvnic.c index 688489475791..97041b319beb 100644 --- a/drivers/net/ethernet/ibm/ibmvnic.c +++ b/drivers/net/ethernet/ibm/ibmvnic.c @@ -781,31 +781,31 @@ static void release_tx_pools(struct ibmvnic_adapter *adapter) static int init_one_tx_pool(struct net_device *netdev, struct ibmvnic_tx_pool *tx_pool, - int num_entries, int buf_size) + int pool_size, int buf_size) { struct ibmvnic_adapter *adapter = netdev_priv(netdev); int i; - tx_pool->tx_buff = kcalloc(num_entries, + tx_pool->tx_buff = kcalloc(pool_size, sizeof(struct ibmvnic_tx_buff), GFP_KERNEL); if (!tx_pool->tx_buff) return -1; if (alloc_long_term_buff(adapter, &tx_pool->long_term_buff, - num_entries * buf_size)) + pool_size * buf_size)) return -1; - tx_pool->free_map = kcalloc(num_entries, sizeof(int), GFP_KERNEL); + tx_pool->free_map = kcalloc(pool_size, sizeof(int), GFP_KERNEL); if (!tx_pool->free_map) return -1; - for (i = 0; i < num_entries; i++) + for (i = 0; i < pool_size; i++) tx_pool->free_map[i] = i; tx_pool->consumer_index = 0; tx_pool->producer_index = 0; - tx_pool->num_buffers = num_entries; + tx_pool->num_buffers = pool_size; tx_pool->buf_size = buf_size; return 0; @@ -815,17 +815,20 @@ static int init_tx_pools(struct net_device *netdev) { struct ibmvnic_adapter *adapter = netdev_priv(netdev); struct device *dev = &adapter->vdev->dev; - int tx_subcrqs; + int num_pools; + u64 pool_size; /* # of buffers in pool */ u64 buff_size; int i, rc; - tx_subcrqs = adapter->num_active_tx_scrqs; - adapter->tx_pool = kcalloc(tx_subcrqs, + pool_size = adapter->req_tx_entries_per_subcrq; + num_pools = adapter->num_active_tx_scrqs; + + adapter->tx_pool = kcalloc(num_pools, sizeof(struct ibmvnic_tx_pool), GFP_KERNEL); if (!adapter->tx_pool) return -1; - adapter->tso_pool = kcalloc(tx_subcrqs, + adapter->tso_pool = kcalloc(num_pools, sizeof(struct ibmvnic_tx_pool), GFP_KERNEL); /* To simplify release_tx_pools() ensure that ->tx_pool and * ->tso_pool are either both NULL or both non-NULL. @@ -839,9 +842,9 @@ static int init_tx_pools(struct net_device *netdev) /* Set num_active_tx_pools early. If we fail below after partial * allocation, release_tx_pools() will know how many to look for. */ - adapter->num_active_tx_pools = tx_subcrqs; + adapter->num_active_tx_pools = num_pools; - for (i = 0; i < tx_subcrqs; i++) { + for (i = 0; i < num_pools; i++) { buff_size = adapter->req_mtu + VLAN_HLEN; buff_size = ALIGN(buff_size, L1_CACHE_BYTES); @@ -849,8 +852,7 @@ static int init_tx_pools(struct net_device *netdev) i, adapter->req_tx_entries_per_subcrq, buff_size); rc = init_one_tx_pool(netdev, &adapter->tx_pool[i], - adapter->req_tx_entries_per_subcrq, - buff_size); + pool_size, buff_size); if (rc) { release_tx_pools(adapter); return rc; From patchwork Wed Sep 15 03:52:55 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sukadev Bhattiprolu X-Patchwork-Id: 12495443 X-Patchwork-Delegate: kuba@kernel.org Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 37C8DC433F5 for ; Wed, 15 Sep 2021 03:53:36 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 1983561164 for ; Wed, 15 Sep 2021 03:53:36 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S236518AbhIODyx (ORCPT ); Tue, 14 Sep 2021 23:54:53 -0400 Received: from mx0a-001b2d01.pphosted.com ([148.163.156.1]:32994 "EHLO mx0a-001b2d01.pphosted.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S236500AbhIODyc (ORCPT ); Tue, 14 Sep 2021 23:54:32 -0400 Received: from pps.filterd (m0098410.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.16.1.2/8.16.0.43) with SMTP id 18F3BtBL029878 for ; Tue, 14 Sep 2021 23:53:14 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ibm.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding; s=pp1; bh=/STvjzj5KsmkCt1yq1JYgvvnD8RXosDpF7js/9V3idA=; b=q7jm6VBdhISUpw8Kj+OGPEHUjR0NQ9PcWMYR+i30/i7WQz0I/UwktGxsWn3KFbq7hNvJ xlTel90D2FyjVxhuOwHQEOj8MppAHe016OM5AsGwB+g1va3ObKVfxVtn1TQVoPfF3iLn tKvo8lENj10CuKZFHm9DPODPdzrw7dAqv3xN/irNruJtX0VfWNfqGNrPIvvYZj3qL8w5 JdSRTKIkdd2hYghMoSe158L61bo3+fPKK9VyASZwUc9KpZHJuDIisr1Wk/Yj855QSxH1 ZVehCmmVTqOFWib/sIF50n5nUmDhM/9Oe+0sBO4pqiYyB8Lei0DYySFp63HJEm5OMqnF 4w== Received: from ppma01dal.us.ibm.com (83.d6.3fa9.ip4.static.sl-reverse.com [169.63.214.131]) by mx0a-001b2d01.pphosted.com with ESMTP id 3b353fvhhs-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT) for ; Tue, 14 Sep 2021 23:53:14 -0400 Received: from pps.filterd (ppma01dal.us.ibm.com [127.0.0.1]) by ppma01dal.us.ibm.com (8.16.1.2/8.16.1.2) with SMTP id 18F3lNw0030301 for ; Wed, 15 Sep 2021 03:53:13 GMT Received: from b01cxnp22034.gho.pok.ibm.com (b01cxnp22034.gho.pok.ibm.com [9.57.198.24]) by ppma01dal.us.ibm.com with ESMTP id 3b0m3d9ja8-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT) for ; Wed, 15 Sep 2021 03:53:13 +0000 Received: from b01ledav004.gho.pok.ibm.com (b01ledav004.gho.pok.ibm.com [9.57.199.109]) by b01cxnp22034.gho.pok.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id 18F3rCq922282534 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Wed, 15 Sep 2021 03:53:12 GMT Received: from b01ledav004.gho.pok.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 0B2FB112061; Wed, 15 Sep 2021 03:53:12 +0000 (GMT) Received: from b01ledav004.gho.pok.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 2BA40112065; Wed, 15 Sep 2021 03:53:10 +0000 (GMT) Received: from suka-w540.ibmuc.com (unknown [9.77.142.77]) by b01ledav004.gho.pok.ibm.com (Postfix) with ESMTP; Wed, 15 Sep 2021 03:53:10 +0000 (GMT) From: Sukadev Bhattiprolu To: netdev@vger.kernel.org Cc: Brian King , cforno12@linux.ibm.com, Dany Madden , Rick Lindsley , Rick Lindsley Subject: [PATCH net-next RESEND v2 5/9] ibmvnic: init_tx_pools move loop-invariant code Date: Tue, 14 Sep 2021 20:52:55 -0700 Message-Id: <20210915035259.355092-6-sukadev@linux.ibm.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20210915035259.355092-1-sukadev@linux.ibm.com> References: <20210915035259.355092-1-sukadev@linux.ibm.com> MIME-Version: 1.0 X-TM-AS-GCONF: 00 X-Proofpoint-ORIG-GUID: FLoW7UlT8wG-P8s-9fOyfJmhIInUeN1y X-Proofpoint-GUID: FLoW7UlT8wG-P8s-9fOyfJmhIInUeN1y X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.182.1,Aquarius:18.0.687,Hydra:6.0.235,FMLib:17.0.607.475 definitions=2020-10-13_15,2020-10-13_02,2020-04-07_01 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 suspectscore=0 clxscore=1015 bulkscore=0 adultscore=0 impostorscore=0 priorityscore=1501 spamscore=0 mlxlogscore=999 mlxscore=0 malwarescore=0 lowpriorityscore=0 phishscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2109030001 definitions=main-2109140123 Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org X-Patchwork-Delegate: kuba@kernel.org In init_tx_pools() move some loop-invariant code out of the loop. Reviewed-by: Rick Lindsley Reviewed-by: Dany Madden Signed-off-by: Sukadev Bhattiprolu --- drivers/net/ethernet/ibm/ibmvnic.c | 5 ++--- 1 file changed, 2 insertions(+), 3 deletions(-) diff --git a/drivers/net/ethernet/ibm/ibmvnic.c b/drivers/net/ethernet/ibm/ibmvnic.c index 97041b319beb..bb9b8aec9c9b 100644 --- a/drivers/net/ethernet/ibm/ibmvnic.c +++ b/drivers/net/ethernet/ibm/ibmvnic.c @@ -843,11 +843,10 @@ static int init_tx_pools(struct net_device *netdev) * allocation, release_tx_pools() will know how many to look for. */ adapter->num_active_tx_pools = num_pools; + buff_size = adapter->req_mtu + VLAN_HLEN; + buff_size = ALIGN(buff_size, L1_CACHE_BYTES); for (i = 0; i < num_pools; i++) { - buff_size = adapter->req_mtu + VLAN_HLEN; - buff_size = ALIGN(buff_size, L1_CACHE_BYTES); - dev_dbg(dev, "Init tx pool %d [%llu, %llu]\n", i, adapter->req_tx_entries_per_subcrq, buff_size); From patchwork Wed Sep 15 03:52:56 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sukadev Bhattiprolu X-Patchwork-Id: 12495445 X-Patchwork-Delegate: kuba@kernel.org Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 37C95C433F5 for ; Wed, 15 Sep 2021 03:53:38 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 1A12A61164 for ; Wed, 15 Sep 2021 03:53:38 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S236371AbhIODyy (ORCPT ); Tue, 14 Sep 2021 23:54:54 -0400 Received: from mx0a-001b2d01.pphosted.com ([148.163.156.1]:16752 "EHLO mx0a-001b2d01.pphosted.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S236159AbhIODye (ORCPT ); Tue, 14 Sep 2021 23:54:34 -0400 Received: from pps.filterd (m0187473.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.16.1.2/8.16.0.43) with SMTP id 18F0CRrC034722 for ; Tue, 14 Sep 2021 23:53:16 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ibm.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding; s=pp1; bh=Jsk2R6ZUYFJG7D3Xc8Da4lDSV4BiePSMPVBBYZ4hBB4=; b=mTPUTXeR6xzYvpXH+ZUOeqOwjLwXfZ9O2e+bKRYeweDHds5y55gVMyLYZXJ3ixiJ1att XfR16tX8ctved0atsFyeB6LrlZE31xLHrsnaOJ22VRV0JKvsH6uFJi3ImYtyPkte/jkt OSybrqQd8ttYS7Pd2Veng8ZjShz48sbD0VnkEWqgrJ3mMe18St1r7W6gxF69eCIBW29L pxubhCzUmDHxg28iLtX+VVXWF3VlYwuU3+4oXw3iPE/PaY80yOkbFMWkjptrf0zBPMgx aPl3KGMolOs0C9nG41ZKMc5R2JJ3cozfO9YV40KaHBxrZ2bTWfHapPf94r9D+A2QNddh PQ== Received: from ppma03wdc.us.ibm.com (ba.79.3fa9.ip4.static.sl-reverse.com [169.63.121.186]) by mx0a-001b2d01.pphosted.com with ESMTP id 3b32u2fc8j-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT) for ; Tue, 14 Sep 2021 23:53:15 -0400 Received: from pps.filterd (ppma03wdc.us.ibm.com [127.0.0.1]) by ppma03wdc.us.ibm.com (8.16.1.2/8.16.1.2) with SMTP id 18F3lPU9006021 for ; Wed, 15 Sep 2021 03:53:14 GMT Received: from b01cxnp22035.gho.pok.ibm.com (b01cxnp22035.gho.pok.ibm.com [9.57.198.25]) by ppma03wdc.us.ibm.com with ESMTP id 3b0m3b8506-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT) for ; Wed, 15 Sep 2021 03:53:14 +0000 Received: from b01ledav004.gho.pok.ibm.com (b01ledav004.gho.pok.ibm.com [9.57.199.109]) by b01cxnp22035.gho.pok.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id 18F3rEqO39715178 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Wed, 15 Sep 2021 03:53:14 GMT Received: from b01ledav004.gho.pok.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id F1A7D112061; Wed, 15 Sep 2021 03:53:13 +0000 (GMT) Received: from b01ledav004.gho.pok.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id B5962112063; Wed, 15 Sep 2021 03:53:12 +0000 (GMT) Received: from suka-w540.ibmuc.com (unknown [9.77.142.77]) by b01ledav004.gho.pok.ibm.com (Postfix) with ESMTP; Wed, 15 Sep 2021 03:53:12 +0000 (GMT) From: Sukadev Bhattiprolu To: netdev@vger.kernel.org Cc: Brian King , cforno12@linux.ibm.com, Dany Madden , Rick Lindsley , Rick Lindsley Subject: [PATCH net-next RESEND v2 6/9] ibmvnic: Use bitmap for LTB map_ids Date: Tue, 14 Sep 2021 20:52:56 -0700 Message-Id: <20210915035259.355092-7-sukadev@linux.ibm.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20210915035259.355092-1-sukadev@linux.ibm.com> References: <20210915035259.355092-1-sukadev@linux.ibm.com> MIME-Version: 1.0 X-TM-AS-GCONF: 00 X-Proofpoint-GUID: ERFmOV_0jKiRIvdWHdDPeyUDrJ4pMtzG X-Proofpoint-ORIG-GUID: ERFmOV_0jKiRIvdWHdDPeyUDrJ4pMtzG X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.182.1,Aquarius:18.0.790,Hydra:6.0.391,FMLib:17.0.607.475 definitions=2021-09-11_02,2021-09-09_01,2020-04-07_01 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 mlxscore=0 spamscore=0 bulkscore=0 impostorscore=0 suspectscore=0 mlxlogscore=999 lowpriorityscore=0 clxscore=1015 adultscore=0 phishscore=0 priorityscore=1501 malwarescore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2109030001 definitions=main-2109140109 Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org X-Patchwork-Delegate: kuba@kernel.org In a follow-on patch, we will reuse long term buffers when possible. When doing so we have to be careful to properly assign map ids. We can no longer assign them sequentially because a lower map id may be available and we could wrap at 255 and collide with an in-use map id. Instead, use a bitmap to track active map ids and to find a free map id. Don't need to take locks here since the map_id only changes during reset and at that time only the reset worker thread should be using the adapter. Noticed this when analyzing an error Dany Madden ran into with the patch set. Reported-by: Dany Madden Reviewed-by: Rick Lindsley Reviewed-by: Dany Madden Signed-off-by: Sukadev Bhattiprolu --- drivers/net/ethernet/ibm/ibmvnic.c | 12 ++++++++---- drivers/net/ethernet/ibm/ibmvnic.h | 3 ++- 2 files changed, 10 insertions(+), 5 deletions(-) diff --git a/drivers/net/ethernet/ibm/ibmvnic.c b/drivers/net/ethernet/ibm/ibmvnic.c index bb9b8aec9c9b..4603597a9c10 100644 --- a/drivers/net/ethernet/ibm/ibmvnic.c +++ b/drivers/net/ethernet/ibm/ibmvnic.c @@ -228,8 +228,9 @@ static int alloc_long_term_buff(struct ibmvnic_adapter *adapter, dev_err(dev, "Couldn't alloc long term buffer\n"); return -ENOMEM; } - ltb->map_id = adapter->map_id; - adapter->map_id++; + ltb->map_id = find_first_zero_bit(adapter->map_ids, + MAX_MAP_ID); + bitmap_set(adapter->map_ids, ltb->map_id, 1); mutex_lock(&adapter->fw_lock); adapter->fw_done_rc = 0; @@ -284,6 +285,8 @@ static void free_long_term_buff(struct ibmvnic_adapter *adapter, dma_free_coherent(dev, ltb->size, ltb->buff, ltb->addr); ltb->buff = NULL; + /* mark this map_id free */ + bitmap_clear(adapter->map_ids, ltb->map_id, 1); ltb->map_id = 0; } @@ -1235,8 +1238,6 @@ static int init_resources(struct ibmvnic_adapter *adapter) return rc; } - adapter->map_id = 1; - rc = init_napi(adapter); if (rc) return rc; @@ -5557,6 +5558,9 @@ static int ibmvnic_probe(struct vio_dev *dev, const struct vio_device_id *id) adapter->vdev = dev; adapter->netdev = netdev; adapter->login_pending = false; + memset(&adapter->map_ids, 0, sizeof(adapter->map_ids)); + /* map_ids start at 1, so ensure map_id 0 is always "in-use" */ + bitmap_set(adapter->map_ids, 0, 1); ether_addr_copy(adapter->mac_addr, mac_addr_p); ether_addr_copy(netdev->dev_addr, adapter->mac_addr); diff --git a/drivers/net/ethernet/ibm/ibmvnic.h b/drivers/net/ethernet/ibm/ibmvnic.h index 5652566818fb..e97f1aa98c05 100644 --- a/drivers/net/ethernet/ibm/ibmvnic.h +++ b/drivers/net/ethernet/ibm/ibmvnic.h @@ -979,7 +979,8 @@ struct ibmvnic_adapter { u64 opt_tx_entries_per_subcrq; u64 opt_rxba_entries_per_subcrq; __be64 tx_rx_desc_req; - u8 map_id; +#define MAX_MAP_ID 255 + DECLARE_BITMAP(map_ids, MAX_MAP_ID); u32 num_active_rx_scrqs; u32 num_active_rx_pools; u32 num_active_rx_napi; From patchwork Wed Sep 15 03:52:57 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sukadev Bhattiprolu X-Patchwork-Id: 12495447 X-Patchwork-Delegate: kuba@kernel.org Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 26F24C433EF for ; Wed, 15 Sep 2021 03:53:39 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 0BB0460F46 for ; Wed, 15 Sep 2021 03:53:39 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S236329AbhIODy4 (ORCPT ); Tue, 14 Sep 2021 23:54:56 -0400 Received: from mx0b-001b2d01.pphosted.com ([148.163.158.5]:8112 "EHLO mx0a-001b2d01.pphosted.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S236190AbhIODyg (ORCPT ); Tue, 14 Sep 2021 23:54:36 -0400 Received: from pps.filterd (m0098414.ppops.net [127.0.0.1]) by mx0b-001b2d01.pphosted.com (8.16.1.2/8.16.0.43) with SMTP id 18F05C4S011645 for ; Tue, 14 Sep 2021 23:53:17 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ibm.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding; s=pp1; bh=Ftenm1l+iGD+nmAxRHfECJIdXtjMa7SfDZvMHSNvO60=; b=eBJKOOpFAEIv26lZqQedQ2qhqEpEpDPkpmpLIRyuLzHeFhzIW2VSmB3mzw71Znc2Jy3G vDOHJSTRaFidKPTIe5FM9PhgEjVVIR1FZlun15fRiQd8iPirwmOwUahZnd9kgO33t9zW 9GTezi8A38vCngilevLkOxpNRQ/R5aPCydSNkkuqO7d6zgCw/rqoa9Gc0qX0O2zMwYBZ uMkwtwRxK2ff5Cf4nrPSass4Zk8sJdPAlPULNrXqquhdY0TLjEBroYxnEOQYj1eblr5C XCzTqbzZpJOn4w3pfjwY247ymv94iwPrdUk5SuaJVuM92Lj9iwXdZ0Bs+kHAMlYXwDEg ww== Received: from ppma02wdc.us.ibm.com (aa.5b.37a9.ip4.static.sl-reverse.com [169.55.91.170]) by mx0b-001b2d01.pphosted.com with ESMTP id 3b35yvkeqv-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT) for ; Tue, 14 Sep 2021 23:53:17 -0400 Received: from pps.filterd (ppma02wdc.us.ibm.com [127.0.0.1]) by ppma02wdc.us.ibm.com (8.16.1.2/8.16.1.2) with SMTP id 18F3lMIU023148 for ; Wed, 15 Sep 2021 03:53:16 GMT Received: from b01cxnp22036.gho.pok.ibm.com (b01cxnp22036.gho.pok.ibm.com [9.57.198.26]) by ppma02wdc.us.ibm.com with ESMTP id 3b0m3bg58a-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT) for ; Wed, 15 Sep 2021 03:53:16 +0000 Received: from b01ledav004.gho.pok.ibm.com (b01ledav004.gho.pok.ibm.com [9.57.199.109]) by b01cxnp22036.gho.pok.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id 18F3rFf012321688 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Wed, 15 Sep 2021 03:53:15 GMT Received: from b01ledav004.gho.pok.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id B28BA112067; Wed, 15 Sep 2021 03:53:15 +0000 (GMT) Received: from b01ledav004.gho.pok.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 4CDC4112061; Wed, 15 Sep 2021 03:53:14 +0000 (GMT) Received: from suka-w540.ibmuc.com (unknown [9.77.142.77]) by b01ledav004.gho.pok.ibm.com (Postfix) with ESMTP; Wed, 15 Sep 2021 03:53:14 +0000 (GMT) From: Sukadev Bhattiprolu To: netdev@vger.kernel.org Cc: Brian King , cforno12@linux.ibm.com, Dany Madden , Rick Lindsley , Rick Lindsley Subject: [PATCH net-next RESEND v2 7/9] ibmvnic: Reuse LTB when possible Date: Tue, 14 Sep 2021 20:52:57 -0700 Message-Id: <20210915035259.355092-8-sukadev@linux.ibm.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20210915035259.355092-1-sukadev@linux.ibm.com> References: <20210915035259.355092-1-sukadev@linux.ibm.com> MIME-Version: 1.0 X-TM-AS-GCONF: 00 X-Proofpoint-ORIG-GUID: FFVKACflJAw-cb9fjqONnyIChnffXZ7A X-Proofpoint-GUID: FFVKACflJAw-cb9fjqONnyIChnffXZ7A X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.182.1,Aquarius:18.0.687,Hydra:6.0.235,FMLib:17.0.607.475 definitions=2020-10-13_15,2020-10-13_02,2020-04-07_01 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 lowpriorityscore=0 suspectscore=0 spamscore=0 bulkscore=0 adultscore=0 priorityscore=1501 phishscore=0 mlxlogscore=999 malwarescore=0 impostorscore=0 mlxscore=0 clxscore=1015 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2109030001 definitions=main-2109140131 Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org X-Patchwork-Delegate: kuba@kernel.org Reuse the long term buffer during a reset as long as its size has not changed. If the size has changed, free it and allocate a new one of the appropriate size. When we do this, alloc_long_term_buff() and reset_long_term_buff() become identical. Drop reset_long_term_buff(). Reviewed-by: Rick Lindsley Reviewed-by: Dany Madden Signed-off-by: Sukadev Bhattiprolu --- Changelog[v2]: [Jakub Kicinski] Fix kdoc issues --- drivers/net/ethernet/ibm/ibmvnic.c | 137 ++++++++++++++++------------- 1 file changed, 74 insertions(+), 63 deletions(-) diff --git a/drivers/net/ethernet/ibm/ibmvnic.c b/drivers/net/ethernet/ibm/ibmvnic.c index 4603597a9c10..dafb36690fdc 100644 --- a/drivers/net/ethernet/ibm/ibmvnic.c +++ b/drivers/net/ethernet/ibm/ibmvnic.c @@ -108,6 +108,8 @@ static int init_crq_queue(struct ibmvnic_adapter *adapter); static int send_query_phys_parms(struct ibmvnic_adapter *adapter); static void ibmvnic_tx_scrq_clean_buffer(struct ibmvnic_adapter *adapter, struct ibmvnic_sub_crq_queue *tx_scrq); +static void free_long_term_buff(struct ibmvnic_adapter *adapter, + struct ibmvnic_long_term_buff *ltb); struct ibmvnic_stat { char name[ETH_GSTRING_LEN]; @@ -214,23 +216,77 @@ static int ibmvnic_wait_for_completion(struct ibmvnic_adapter *adapter, return -ETIMEDOUT; } +/** + * reuse_ltb() - Check if a long term buffer can be reused + * @ltb: The long term buffer to be checked + * @size: The size of the long term buffer. + * + * An LTB can be reused unless its size has changed. + * + * Return: Return true if the LTB can be reused, false otherwise. + */ +static bool reuse_ltb(struct ibmvnic_long_term_buff *ltb, int size) +{ + return (ltb->buff && ltb->size == size); +} + +/** + * alloc_long_term_buff() - Allocate a long term buffer (LTB) + * + * @adapter: ibmvnic adapter associated to the LTB + * @ltb: container object for the LTB + * @size: size of the LTB + * + * Allocate an LTB of the specified size and notify VIOS. + * + * If the given @ltb already has the correct size, reuse it. Otherwise if + * its non-NULL, free it. Then allocate a new one of the correct size. + * Notify the VIOS either way since we may now be working with a new VIOS. + * + * Allocating larger chunks of memory during resets, specially LPM or under + * low memory situations can cause resets to fail/timeout and for LPAR to + * lose connectivity. So hold onto the LTB even if we fail to communicate + * with the VIOS and reuse it on next open. Free LTB when adapter is closed. + * + * Return: 0 if we were able to allocate the LTB and notify the VIOS and + * a negative value otherwise. + */ static int alloc_long_term_buff(struct ibmvnic_adapter *adapter, struct ibmvnic_long_term_buff *ltb, int size) { struct device *dev = &adapter->vdev->dev; int rc; - ltb->size = size; - ltb->buff = dma_alloc_coherent(dev, ltb->size, <b->addr, - GFP_KERNEL); + if (!reuse_ltb(ltb, size)) { + dev_dbg(dev, + "LTB size changed from 0x%llx to 0x%x, reallocating\n", + ltb->size, size); + free_long_term_buff(adapter, ltb); + } - if (!ltb->buff) { - dev_err(dev, "Couldn't alloc long term buffer\n"); - return -ENOMEM; + if (ltb->buff) { + dev_dbg(dev, "Reusing LTB [map %d, size 0x%llx]\n", + ltb->map_id, ltb->size); + } else { + ltb->buff = dma_alloc_coherent(dev, size, <b->addr, + GFP_KERNEL); + if (!ltb->buff) { + dev_err(dev, "Couldn't alloc long term buffer\n"); + return -ENOMEM; + } + ltb->size = size; + + ltb->map_id = find_first_zero_bit(adapter->map_ids, + MAX_MAP_ID); + bitmap_set(adapter->map_ids, ltb->map_id, 1); + + dev_dbg(dev, + "Allocated new LTB [map %d, size 0x%llx]\n", + ltb->map_id, ltb->size); } - ltb->map_id = find_first_zero_bit(adapter->map_ids, - MAX_MAP_ID); - bitmap_set(adapter->map_ids, ltb->map_id, 1); + + /* Ensure ltb is zeroed - specially when reusing it. */ + memset(ltb->buff, 0, ltb->size); mutex_lock(&adapter->fw_lock); adapter->fw_done_rc = 0; @@ -257,10 +313,7 @@ static int alloc_long_term_buff(struct ibmvnic_adapter *adapter, } rc = 0; out: - if (rc) { - dma_free_coherent(dev, ltb->size, ltb->buff, ltb->addr); - ltb->buff = NULL; - } + /* don't free LTB on communication error - see function header */ mutex_unlock(&adapter->fw_lock); return rc; } @@ -290,43 +343,6 @@ static void free_long_term_buff(struct ibmvnic_adapter *adapter, ltb->map_id = 0; } -static int reset_long_term_buff(struct ibmvnic_adapter *adapter, - struct ibmvnic_long_term_buff *ltb) -{ - struct device *dev = &adapter->vdev->dev; - int rc; - - memset(ltb->buff, 0, ltb->size); - - mutex_lock(&adapter->fw_lock); - adapter->fw_done_rc = 0; - - reinit_completion(&adapter->fw_done); - rc = send_request_map(adapter, ltb->addr, ltb->size, ltb->map_id); - if (rc) { - mutex_unlock(&adapter->fw_lock); - return rc; - } - - rc = ibmvnic_wait_for_completion(adapter, &adapter->fw_done, 10000); - if (rc) { - dev_info(dev, - "Reset failed, long term map request timed out or aborted\n"); - mutex_unlock(&adapter->fw_lock); - return rc; - } - - if (adapter->fw_done_rc) { - dev_info(dev, - "Reset failed, attempting to free and reallocate buffer\n"); - free_long_term_buff(adapter, ltb); - mutex_unlock(&adapter->fw_lock); - return alloc_long_term_buff(adapter, ltb, ltb->size); - } - mutex_unlock(&adapter->fw_lock); - return 0; -} - static void deactivate_rx_pools(struct ibmvnic_adapter *adapter) { int i; @@ -548,18 +564,10 @@ static int reset_rx_pools(struct ibmvnic_adapter *adapter) netdev_dbg(adapter->netdev, "Re-setting rx_pool[%d]\n", i); - if (rx_pool->buff_size != buff_size) { - free_long_term_buff(adapter, &rx_pool->long_term_buff); - rx_pool->buff_size = ALIGN(buff_size, L1_CACHE_BYTES); - rc = alloc_long_term_buff(adapter, - &rx_pool->long_term_buff, - rx_pool->size * - rx_pool->buff_size); - } else { - rc = reset_long_term_buff(adapter, - &rx_pool->long_term_buff); - } - + rx_pool->buff_size = ALIGN(buff_size, L1_CACHE_BYTES); + rc = alloc_long_term_buff(adapter, + &rx_pool->long_term_buff, + rx_pool->size * rx_pool->buff_size); if (rc) return rc; @@ -694,9 +702,12 @@ static int init_rx_pools(struct net_device *netdev) static int reset_one_tx_pool(struct ibmvnic_adapter *adapter, struct ibmvnic_tx_pool *tx_pool) { + struct ibmvnic_long_term_buff *ltb; int rc, i; - rc = reset_long_term_buff(adapter, &tx_pool->long_term_buff); + ltb = &tx_pool->long_term_buff; + + rc = alloc_long_term_buff(adapter, ltb, ltb->size); if (rc) return rc; From patchwork Wed Sep 15 03:52:58 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sukadev Bhattiprolu X-Patchwork-Id: 12495449 X-Patchwork-Delegate: kuba@kernel.org Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id BBA93C4332F for ; Wed, 15 Sep 2021 03:53:40 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id A595B6103C for ; Wed, 15 Sep 2021 03:53:40 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S236435AbhIODy5 (ORCPT ); Tue, 14 Sep 2021 23:54:57 -0400 Received: from mx0b-001b2d01.pphosted.com ([148.163.158.5]:17664 "EHLO mx0b-001b2d01.pphosted.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232804AbhIODyi (ORCPT ); Tue, 14 Sep 2021 23:54:38 -0400 Received: from pps.filterd (m0098417.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.16.1.2/8.16.0.43) with SMTP id 18F1Tvua005867 for ; Tue, 14 Sep 2021 23:53:19 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ibm.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding; s=pp1; bh=YkpBmgDxJtLNKDhEDlT+UIY0k1WsRxFY/fIk9yXGiVc=; b=SlNw9ygntVa3a20jQHQoQKJymRNtRvqM9qI/6iEaKJVgUuyoVL4ioQTP1ldk27+v+e/s NrnDHITMKAbl3xaQ/XoGd1nJQpICtqsU90H4S3zSlUPmcN2UWe8brJB/LFj5yI/uJqI7 Oj0hjCWmUeIP52Fr2YjQ131AJk4LGoZOQk7sC2anqlbk03CIs+sWA0D2Zxe7jO0eUO3e GKVRbwzl/7a7Flo4cvNNM7K8MMwMVLx7aW3gTz1Km5tM2vC54zmUaoaB6Yl9n+fX/EDR dv3mw8TZEDBXx+ZN3ZigRsOHJkTl5YKGWoYKwCGAwKrMezPgEsKKwHWOkJIF76JzHRzH 8g== Received: from ppma02wdc.us.ibm.com (aa.5b.37a9.ip4.static.sl-reverse.com [169.55.91.170]) by mx0a-001b2d01.pphosted.com with ESMTP id 3b377kj2cm-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT) for ; Tue, 14 Sep 2021 23:53:19 -0400 Received: from pps.filterd (ppma02wdc.us.ibm.com [127.0.0.1]) by ppma02wdc.us.ibm.com (8.16.1.2/8.16.1.2) with SMTP id 18F3lOwo023322 for ; Wed, 15 Sep 2021 03:53:18 GMT Received: from b01cxnp23033.gho.pok.ibm.com (b01cxnp23033.gho.pok.ibm.com [9.57.198.28]) by ppma02wdc.us.ibm.com with ESMTP id 3b0m3bg58r-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT) for ; Wed, 15 Sep 2021 03:53:18 +0000 Received: from b01ledav004.gho.pok.ibm.com (b01ledav004.gho.pok.ibm.com [9.57.199.109]) by b01cxnp23033.gho.pok.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id 18F3rHed46531000 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Wed, 15 Sep 2021 03:53:17 GMT Received: from b01ledav004.gho.pok.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id D72DD112061; Wed, 15 Sep 2021 03:53:17 +0000 (GMT) Received: from b01ledav004.gho.pok.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 21BC3112065; Wed, 15 Sep 2021 03:53:16 +0000 (GMT) Received: from suka-w540.ibmuc.com (unknown [9.77.142.77]) by b01ledav004.gho.pok.ibm.com (Postfix) with ESMTP; Wed, 15 Sep 2021 03:53:15 +0000 (GMT) From: Sukadev Bhattiprolu To: netdev@vger.kernel.org Cc: Brian King , cforno12@linux.ibm.com, Dany Madden , Rick Lindsley , Rick Lindsley Subject: [PATCH net-next RESEND v2 8/9] ibmvnic: Reuse rx pools when possible Date: Tue, 14 Sep 2021 20:52:58 -0700 Message-Id: <20210915035259.355092-9-sukadev@linux.ibm.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20210915035259.355092-1-sukadev@linux.ibm.com> References: <20210915035259.355092-1-sukadev@linux.ibm.com> MIME-Version: 1.0 X-TM-AS-GCONF: 00 X-Proofpoint-ORIG-GUID: XhsRKa59XFkIv5zMsJ3ZZ2zRDCyQOc1n X-Proofpoint-GUID: XhsRKa59XFkIv5zMsJ3ZZ2zRDCyQOc1n X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.182.1,Aquarius:18.0.687,Hydra:6.0.235,FMLib:17.0.607.475 definitions=2020-10-13_15,2020-10-13_02,2020-04-07_01 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 lowpriorityscore=0 suspectscore=0 bulkscore=0 priorityscore=1501 impostorscore=0 clxscore=1015 malwarescore=0 spamscore=0 phishscore=0 adultscore=0 mlxscore=0 mlxlogscore=999 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2109030001 definitions=main-2109140134 Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org X-Patchwork-Delegate: kuba@kernel.org Rather than releasing the rx pools on and reallocating them on every reset, reuse the rx pools unless the pool parameters (number of pools, size of each pool or size of each buffer in a pool) have changed. If the pool parameters changed, then release the old pools (if any) and allocate new ones. Specifically release rx pools, if: - adapter is removed, - pool parameters change during reset, - we encounter an error when opening the adapter in response to a user request (in ibmvnic_open()). and don't release them: - in __ibmvnic_close() or - on errors in __ibmvnic_open() in the hope that we can reuse them on the next reset. With these, reset_rx_pools() can be dropped because its optimzation is now included in init_rx_pools() itself. cleanup_rx_pools() releases all the skbs associated with the pool and is called from ibmvnic_cleanup(), which is called on every reset. Since we want to reuse skbs across resets, move cleanup_rx_pools() out of ibmvnic_cleanup() and call it only when user closes the adapter. Add two new adapter fields, ->prev_rx_buf_sz, ->prev_rx_pool_size to keep track of the previous values and use them to decide whether to reuse or realloc the pools. Reviewed-by: Rick Lindsley Reviewed-by: Dany Madden Signed-off-by: Sukadev Bhattiprolu --- Changelog[v2]: [Jakub Kicinski] Fix kdoc issues --- drivers/net/ethernet/ibm/ibmvnic.c | 198 +++++++++++++++++++---------- drivers/net/ethernet/ibm/ibmvnic.h | 3 + 2 files changed, 137 insertions(+), 64 deletions(-) diff --git a/drivers/net/ethernet/ibm/ibmvnic.c b/drivers/net/ethernet/ibm/ibmvnic.c index dafb36690fdc..05aabb0effef 100644 --- a/drivers/net/ethernet/ibm/ibmvnic.c +++ b/drivers/net/ethernet/ibm/ibmvnic.c @@ -383,20 +383,27 @@ static void replenish_rx_pool(struct ibmvnic_adapter *adapter, * be 0. */ for (i = ind_bufp->index; i < count; ++i) { - skb = netdev_alloc_skb(adapter->netdev, pool->buff_size); + index = pool->free_map[pool->next_free]; + + /* We maybe reusing the skb from earlier resets. Allocate + * only if necessary. But since the LTB may have changed + * during reset (see init_rx_pools()), update LTB below + * even if reusing skb. + */ + skb = pool->rx_buff[index].skb; if (!skb) { - dev_err(dev, "Couldn't replenish rx buff\n"); - adapter->replenish_no_mem++; - break; + skb = netdev_alloc_skb(adapter->netdev, + pool->buff_size); + if (!skb) { + dev_err(dev, "Couldn't replenish rx buff\n"); + adapter->replenish_no_mem++; + break; + } } - index = pool->free_map[pool->next_free]; pool->free_map[pool->next_free] = IBMVNIC_INVALID_MAP; pool->next_free = (pool->next_free + 1) % pool->size; - if (pool->rx_buff[index].skb) - dev_err(dev, "Inconsistent free_map!\n"); - /* Copy the skb to the long term mapped DMA buffer */ offset = index * pool->buff_size; dst = pool->long_term_buff.buff + offset; @@ -547,45 +554,6 @@ static int init_stats_token(struct ibmvnic_adapter *adapter) return 0; } -static int reset_rx_pools(struct ibmvnic_adapter *adapter) -{ - struct ibmvnic_rx_pool *rx_pool; - u64 buff_size; - int rx_scrqs; - int i, j, rc; - - if (!adapter->rx_pool) - return -1; - - buff_size = adapter->cur_rx_buf_sz; - rx_scrqs = adapter->num_active_rx_pools; - for (i = 0; i < rx_scrqs; i++) { - rx_pool = &adapter->rx_pool[i]; - - netdev_dbg(adapter->netdev, "Re-setting rx_pool[%d]\n", i); - - rx_pool->buff_size = ALIGN(buff_size, L1_CACHE_BYTES); - rc = alloc_long_term_buff(adapter, - &rx_pool->long_term_buff, - rx_pool->size * rx_pool->buff_size); - if (rc) - return rc; - - for (j = 0; j < rx_pool->size; j++) - rx_pool->free_map[j] = j; - - memset(rx_pool->rx_buff, 0, - rx_pool->size * sizeof(struct ibmvnic_rx_buff)); - - atomic_set(&rx_pool->available, 0); - rx_pool->next_alloc = 0; - rx_pool->next_free = 0; - rx_pool->active = 1; - } - - return 0; -} - /** * release_rx_pools() - Release any rx pools attached to @adapter. * @adapter: ibmvnic adapter @@ -606,6 +574,7 @@ static void release_rx_pools(struct ibmvnic_adapter *adapter) netdev_dbg(adapter->netdev, "Releasing rx_pool[%d]\n", i); kfree(rx_pool->free_map); + free_long_term_buff(adapter, &rx_pool->long_term_buff); if (!rx_pool->rx_buff) @@ -624,8 +593,68 @@ static void release_rx_pools(struct ibmvnic_adapter *adapter) kfree(adapter->rx_pool); adapter->rx_pool = NULL; adapter->num_active_rx_pools = 0; + adapter->prev_rx_pool_size = 0; +} + +/** + * reuse_rx_pools() - Check if the existing rx pools can be reused. + * @adapter: ibmvnic adapter + * + * Check if the existing rx pools in the adapter can be reused. The + * pools can be reused if the pool parameters (number of pools, + * number of buffers in the pool and size of each buffer) have not + * changed. + * + * NOTE: This assumes that all pools have the same number of buffers + * which is the case currently. If that changes, we must fix this. + * + * Return: true if the rx pools can be reused, false otherwise. + */ +static bool reuse_rx_pools(struct ibmvnic_adapter *adapter) +{ + u64 old_num_pools, new_num_pools; + u64 old_pool_size, new_pool_size; + u64 old_buff_size, new_buff_size; + + if (!adapter->rx_pool) + return false; + + old_num_pools = adapter->num_active_rx_pools; + new_num_pools = adapter->req_rx_queues; + + old_pool_size = adapter->prev_rx_pool_size; + new_pool_size = adapter->req_rx_add_entries_per_subcrq; + + old_buff_size = adapter->prev_rx_buf_sz; + new_buff_size = adapter->cur_rx_buf_sz; + + /* Require buff size to be exactly same for now */ + if (old_buff_size != new_buff_size) + return false; + + if (old_num_pools == new_num_pools && old_pool_size == new_pool_size) + return true; + + if (old_num_pools < adapter->min_rx_queues || + old_num_pools > adapter->max_rx_queues || + old_pool_size < adapter->min_rx_add_entries_per_subcrq || + old_pool_size > adapter->max_rx_add_entries_per_subcrq) + return false; + + return true; } +/** + * init_rx_pools(): Initialize the set of receiver pools in the adapter. + * @netdev: net device associated with the vnic interface + * + * Initialize the set of receiver pools in the ibmvnic adapter associated + * with the net_device @netdev. If possible, reuse the existing rx pools. + * Otherwise free any existing pools and allocate a new set of pools + * before initializing them. + * + * Return: 0 on success and negative value on error. + */ static int init_rx_pools(struct net_device *netdev) { struct ibmvnic_adapter *adapter = netdev_priv(netdev); @@ -636,10 +665,18 @@ static int init_rx_pools(struct net_device *netdev) u64 buff_size; int i, j; - num_pools = adapter->num_active_rx_scrqs; pool_size = adapter->req_rx_add_entries_per_subcrq; + num_pools = adapter->req_rx_queues; buff_size = adapter->cur_rx_buf_sz; + if (reuse_rx_pools(adapter)) { + dev_dbg(dev, "Reusing rx pools\n"); + goto update_ltb; + } + + /* Allocate/populate the pools. */ + release_rx_pools(adapter); + adapter->rx_pool = kcalloc(num_pools, sizeof(struct ibmvnic_rx_pool), GFP_KERNEL); @@ -663,14 +700,12 @@ static int init_rx_pools(struct net_device *netdev) rx_pool->size = pool_size; rx_pool->index = i; rx_pool->buff_size = ALIGN(buff_size, L1_CACHE_BYTES); - rx_pool->active = 1; rx_pool->free_map = kcalloc(rx_pool->size, sizeof(int), GFP_KERNEL); if (!rx_pool->free_map) { dev_err(dev, "Couldn't alloc free_map %d\n", i); - release_rx_pools(adapter); - return -1; + goto out_release; } rx_pool->rx_buff = kcalloc(rx_pool->size, @@ -678,25 +713,58 @@ static int init_rx_pools(struct net_device *netdev) GFP_KERNEL); if (!rx_pool->rx_buff) { dev_err(dev, "Couldn't alloc rx buffers\n"); - release_rx_pools(adapter); - return -1; + goto out_release; } + } + + adapter->prev_rx_pool_size = pool_size; + adapter->prev_rx_buf_sz = adapter->cur_rx_buf_sz; + +update_ltb: + for (i = 0; i < num_pools; i++) { + rx_pool = &adapter->rx_pool[i]; + dev_dbg(dev, "Updating LTB for rx pool %d [%d, %d]\n", + i, rx_pool->size, rx_pool->buff_size); if (alloc_long_term_buff(adapter, &rx_pool->long_term_buff, - rx_pool->size * rx_pool->buff_size)) { - release_rx_pools(adapter); - return -1; - } + rx_pool->size * rx_pool->buff_size)) + goto out; + + for (j = 0; j < rx_pool->size; ++j) { + struct ibmvnic_rx_buff *rx_buff; - for (j = 0; j < rx_pool->size; ++j) rx_pool->free_map[j] = j; + /* NOTE: Don't clear rx_buff->skb here - will leak + * memory! replenish_rx_pool() will reuse skbs or + * allocate as necessary. + */ + rx_buff = &rx_pool->rx_buff[j]; + rx_buff->dma = 0; + rx_buff->data = 0; + rx_buff->size = 0; + rx_buff->pool_index = 0; + } + + /* Mark pool "empty" so replenish_rx_pools() will + * update the LTB info for each buffer + */ atomic_set(&rx_pool->available, 0); rx_pool->next_alloc = 0; rx_pool->next_free = 0; + /* replenish_rx_pool() may have called deactivate_rx_pools() + * on failover. Ensure pool is active now. + */ + rx_pool->active = 1; } - return 0; +out_release: + release_rx_pools(adapter); +out: + /* We failed to allocate one or more LTBs or map them on the VIOS. + * Hold onto the pools and any LTBs that we did allocate/map. + */ + return -1; } static int reset_one_tx_pool(struct ibmvnic_adapter *adapter, @@ -1072,7 +1140,6 @@ static void release_resources(struct ibmvnic_adapter *adapter) release_vpd_data(adapter); release_tx_pools(adapter); - release_rx_pools(adapter); release_napi(adapter); release_login_buffer(adapter); @@ -1345,6 +1412,7 @@ static int ibmvnic_open(struct net_device *netdev) if (rc) { netdev_err(netdev, "failed to initialize resources\n"); release_resources(adapter); + release_rx_pools(adapter); goto out; } } @@ -1474,7 +1542,6 @@ static void ibmvnic_cleanup(struct net_device *netdev) ibmvnic_napi_disable(adapter); ibmvnic_disable_irqs(adapter); - clean_rx_pools(adapter); clean_tx_pools(adapter); } @@ -1509,6 +1576,7 @@ static int ibmvnic_close(struct net_device *netdev) rc = __ibmvnic_close(netdev); ibmvnic_cleanup(netdev); + clean_rx_pools(adapter); return rc; } @@ -2237,7 +2305,6 @@ static int do_reset(struct ibmvnic_adapter *adapter, !adapter->rx_pool || !adapter->tso_pool || !adapter->tx_pool) { - release_rx_pools(adapter); release_tx_pools(adapter); release_napi(adapter); release_vpd_data(adapter); @@ -2254,9 +2321,10 @@ static int do_reset(struct ibmvnic_adapter *adapter, goto out; } - rc = reset_rx_pools(adapter); + rc = init_rx_pools(netdev); if (rc) { - netdev_dbg(adapter->netdev, "reset rx pools failed (%d)\n", + netdev_dbg(netdev, + "init rx pools failed (%d)\n", rc); goto out; } @@ -5592,6 +5660,7 @@ static int ibmvnic_probe(struct vio_dev *dev, const struct vio_device_id *id) init_completion(&adapter->reset_done); init_completion(&adapter->stats_done); clear_bit(0, &adapter->resetting); + adapter->prev_rx_buf_sz = 0; init_success = false; do { @@ -5692,6 +5761,7 @@ static void ibmvnic_remove(struct vio_dev *dev) unregister_netdevice(netdev); release_resources(adapter); + release_rx_pools(adapter); release_sub_crqs(adapter, 1); release_crq_queue(adapter); diff --git a/drivers/net/ethernet/ibm/ibmvnic.h b/drivers/net/ethernet/ibm/ibmvnic.h index e97f1aa98c05..b73a1b812368 100644 --- a/drivers/net/ethernet/ibm/ibmvnic.h +++ b/drivers/net/ethernet/ibm/ibmvnic.h @@ -986,7 +986,10 @@ struct ibmvnic_adapter { u32 num_active_rx_napi; u32 num_active_tx_scrqs; u32 num_active_tx_pools; + + u32 prev_rx_pool_size; u32 cur_rx_buf_sz; + u32 prev_rx_buf_sz; struct tasklet_struct tasklet; enum vnic_state state; From patchwork Wed Sep 15 03:52:59 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sukadev Bhattiprolu X-Patchwork-Id: 12495451 X-Patchwork-Delegate: kuba@kernel.org Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id DA719C433FE for ; Wed, 15 Sep 2021 03:53:44 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id C1B5560F46 for ; Wed, 15 Sep 2021 03:53:44 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230107AbhIODy7 (ORCPT ); Tue, 14 Sep 2021 23:54:59 -0400 Received: from mx0b-001b2d01.pphosted.com ([148.163.158.5]:30346 "EHLO mx0b-001b2d01.pphosted.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S236279AbhIODyl (ORCPT ); Tue, 14 Sep 2021 23:54:41 -0400 Received: from pps.filterd (m0098417.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.16.1.2/8.16.0.43) with SMTP id 18F1TuLf005836 for ; Tue, 14 Sep 2021 23:53:22 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ibm.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding; s=pp1; bh=9RTzSLJRzQr1R6OZD1Kd9en2LTJQUwi92pdoKS6C3SM=; b=AIb7BbPgQRqseHSonk1CQcThnWLpdX9k45+hxCJ0lqqwnFQIRWbXKfFSFncYvgov+aYo 2x1U4S8weoqaipdjRrGaY3GiDWGrtYrOleoK3D28njiVw7tyfjXF6d9r/Rkjza5hrRHR TplzbbAdXiZDa//TMO4dwPGLOK+8u1YmZNPNQTNECoYYNUGf3Tu8ixpJZ/e0c+bh9qvA yvjWE5sGtjYF78K6IHRkDSE0UmkdM1ng7T8du67VQetSkMSRemMot0sw12JbRVgTkiUP S/gZV5KT+I2OUpjIn9bEaeInsppvT0oP+4oTVf7+rH5V0NunVK5CbNMzEwbnMEyCYlHo 3Q== Received: from ppma01wdc.us.ibm.com (fd.55.37a9.ip4.static.sl-reverse.com [169.55.85.253]) by mx0a-001b2d01.pphosted.com with ESMTP id 3b377kj2d6-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT) for ; Tue, 14 Sep 2021 23:53:22 -0400 Received: from pps.filterd (ppma01wdc.us.ibm.com [127.0.0.1]) by ppma01wdc.us.ibm.com (8.16.1.2/8.16.1.2) with SMTP id 18F3pw2L024179 for ; Wed, 15 Sep 2021 03:53:21 GMT Received: from b01cxnp22034.gho.pok.ibm.com (b01cxnp22034.gho.pok.ibm.com [9.57.198.24]) by ppma01wdc.us.ibm.com with ESMTP id 3b0m3a07r3-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT) for ; Wed, 15 Sep 2021 03:53:21 +0000 Received: from b01ledav004.gho.pok.ibm.com (b01ledav004.gho.pok.ibm.com [9.57.199.109]) by b01cxnp22034.gho.pok.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id 18F3rJcl39584144 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Wed, 15 Sep 2021 03:53:19 GMT Received: from b01ledav004.gho.pok.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 986EB112062; Wed, 15 Sep 2021 03:53:19 +0000 (GMT) Received: from b01ledav004.gho.pok.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 46D7F112066; Wed, 15 Sep 2021 03:53:18 +0000 (GMT) Received: from suka-w540.ibmuc.com (unknown [9.77.142.77]) by b01ledav004.gho.pok.ibm.com (Postfix) with ESMTP; Wed, 15 Sep 2021 03:53:18 +0000 (GMT) From: Sukadev Bhattiprolu To: netdev@vger.kernel.org Cc: Brian King , cforno12@linux.ibm.com, Dany Madden , Rick Lindsley , Rick Lindsley Subject: [PATCH net-next RESEND v2 9/9] ibmvnic: Reuse tx pools when possible Date: Tue, 14 Sep 2021 20:52:59 -0700 Message-Id: <20210915035259.355092-10-sukadev@linux.ibm.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20210915035259.355092-1-sukadev@linux.ibm.com> References: <20210915035259.355092-1-sukadev@linux.ibm.com> MIME-Version: 1.0 X-TM-AS-GCONF: 00 X-Proofpoint-ORIG-GUID: OQubL1AKRx5WTM-l9O0doWmSQhrCxDTt X-Proofpoint-GUID: OQubL1AKRx5WTM-l9O0doWmSQhrCxDTt X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.182.1,Aquarius:18.0.687,Hydra:6.0.235,FMLib:17.0.607.475 definitions=2020-10-13_15,2020-10-13_02,2020-04-07_01 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 lowpriorityscore=0 suspectscore=0 bulkscore=0 priorityscore=1501 impostorscore=0 clxscore=1015 malwarescore=0 spamscore=0 phishscore=0 adultscore=0 mlxscore=0 mlxlogscore=999 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2109030001 definitions=main-2109140134 Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org X-Patchwork-Delegate: kuba@kernel.org Rather than releasing the tx pools on every close and reallocating them on open, reuse the tx pools unless the pool parameters (number of pools, size of each pool or size of each buffer in a pool) have changed. If the pool parameters changed, then release the old pools (if any) and allocate new ones. Specifically release tx pools, if: - adapter is removed, - pool parameters change during reset, - we encounter an error when opening the adapter in response to a user request (in ibmvnic_open()). and don't release them: - in __ibmvnic_close() or - on errors in __ibmvnic_open() in the hope that we can reuse them during this or next reset. With these changes reset_tx_pools() can be dropped because its optimization is now included in init_tx_pools() itself. cleanup_tx_pools() releases all the skbs associated with the pool and is called from ibmvnic_cleanup(), which is called on every reset. Since we want to reuse skbs across resets, move cleanup_tx_pools() out of ibmvnic_cleanup() and call it only when user closes the adapter. Add two new adapter fields, ->prev_mtu, ->prev_tx_pool_size to track the previous values and use them to decide whether to reuse or realloc the pools. Reviewed-by: Rick Lindsley Reviewed-by: Dany Madden Signed-off-by: Sukadev Bhattiprolu --- Changelog[v2]: [Jakub Kicinski] Fix kdoc issues --- drivers/net/ethernet/ibm/ibmvnic.c | 215 +++++++++++++++++++---------- drivers/net/ethernet/ibm/ibmvnic.h | 2 + 2 files changed, 147 insertions(+), 70 deletions(-) diff --git a/drivers/net/ethernet/ibm/ibmvnic.c b/drivers/net/ethernet/ibm/ibmvnic.c index 05aabb0effef..d1be883f933a 100644 --- a/drivers/net/ethernet/ibm/ibmvnic.c +++ b/drivers/net/ethernet/ibm/ibmvnic.c @@ -767,53 +767,6 @@ static int init_rx_pools(struct net_device *netdev) return -1; } -static int reset_one_tx_pool(struct ibmvnic_adapter *adapter, - struct ibmvnic_tx_pool *tx_pool) -{ - struct ibmvnic_long_term_buff *ltb; - int rc, i; - - ltb = &tx_pool->long_term_buff; - - rc = alloc_long_term_buff(adapter, ltb, ltb->size); - if (rc) - return rc; - - memset(tx_pool->tx_buff, 0, - tx_pool->num_buffers * - sizeof(struct ibmvnic_tx_buff)); - - for (i = 0; i < tx_pool->num_buffers; i++) - tx_pool->free_map[i] = i; - - tx_pool->consumer_index = 0; - tx_pool->producer_index = 0; - - return 0; -} - -static int reset_tx_pools(struct ibmvnic_adapter *adapter) -{ - int tx_scrqs; - int i, rc; - - if (!adapter->tx_pool) - return -1; - - tx_scrqs = adapter->num_active_tx_pools; - for (i = 0; i < tx_scrqs; i++) { - ibmvnic_tx_scrq_clean_buffer(adapter, adapter->tx_scrq[i]); - rc = reset_one_tx_pool(adapter, &adapter->tso_pool[i]); - if (rc) - return rc; - rc = reset_one_tx_pool(adapter, &adapter->tx_pool[i]); - if (rc) - return rc; - } - - return 0; -} - static void release_vpd_data(struct ibmvnic_adapter *adapter) { if (!adapter->vpd) @@ -859,13 +812,13 @@ static void release_tx_pools(struct ibmvnic_adapter *adapter) kfree(adapter->tso_pool); adapter->tso_pool = NULL; adapter->num_active_tx_pools = 0; + adapter->prev_tx_pool_size = 0; } static int init_one_tx_pool(struct net_device *netdev, struct ibmvnic_tx_pool *tx_pool, int pool_size, int buf_size) { - struct ibmvnic_adapter *adapter = netdev_priv(netdev); int i; tx_pool->tx_buff = kcalloc(pool_size, @@ -874,13 +827,12 @@ static int init_one_tx_pool(struct net_device *netdev, if (!tx_pool->tx_buff) return -1; - if (alloc_long_term_buff(adapter, &tx_pool->long_term_buff, - pool_size * buf_size)) - return -1; - tx_pool->free_map = kcalloc(pool_size, sizeof(int), GFP_KERNEL); - if (!tx_pool->free_map) + if (!tx_pool->free_map) { + kfree(tx_pool->tx_buff); + tx_pool->tx_buff = NULL; return -1; + } for (i = 0; i < pool_size; i++) tx_pool->free_map[i] = i; @@ -893,6 +845,62 @@ static int init_one_tx_pool(struct net_device *netdev, return 0; } +/** + * reuse_tx_pools() - Check if the existing tx pools can be reused. + * @adapter: ibmvnic adapter + * + * Check if the existing tx pools in the adapter can be reused. The + * pools can be reused if the pool parameters (number of pools, + * number of buffers in the pool and mtu) have not changed. + * + * NOTE: This assumes that all pools have the same number of buffers + * which is the case currently. If that changes, we must fix this. + * + * Return: true if the tx pools can be reused, false otherwise. + */ +static bool reuse_tx_pools(struct ibmvnic_adapter *adapter) +{ + u64 old_num_pools, new_num_pools; + u64 old_pool_size, new_pool_size; + u64 old_mtu, new_mtu; + + if (!adapter->tx_pool) + return false; + + old_num_pools = adapter->num_active_tx_pools; + new_num_pools = adapter->num_active_tx_scrqs; + old_pool_size = adapter->prev_tx_pool_size; + new_pool_size = adapter->req_tx_entries_per_subcrq; + old_mtu = adapter->prev_mtu; + new_mtu = adapter->req_mtu; + + /* Require MTU to be exactly same to reuse pools for now */ + if (old_mtu != new_mtu) + return false; + + if (old_num_pools == new_num_pools && old_pool_size == new_pool_size) + return true; + + if (old_num_pools < adapter->min_tx_queues || + old_num_pools > adapter->max_tx_queues || + old_pool_size < adapter->min_tx_entries_per_subcrq || + old_pool_size > adapter->max_tx_entries_per_subcrq) + return false; + + return true; +} + +/** + * init_tx_pools(): Initialize the set of transmit pools in the adapter. + * @netdev: net device associated with the vnic interface + * + * Initialize the set of transmit pools in the ibmvnic adapter associated + * with the net_device @netdev. If possible, reuse the existing tx pools. + * Otherwise free any existing pools and allocate a new set of pools + * before initializing them. + * + * Return: 0 on success and negative value on error. + */ static int init_tx_pools(struct net_device *netdev) { struct ibmvnic_adapter *adapter = netdev_priv(netdev); @@ -900,7 +908,21 @@ static int init_tx_pools(struct net_device *netdev) int num_pools; u64 pool_size; /* # of buffers in pool */ u64 buff_size; - int i, rc; + int i, j, rc; + + num_pools = adapter->req_tx_queues; + + /* We must notify the VIOS about the LTB on all resets - but we only + * need to alloc/populate pools if either the number of buffers or + * size of each buffer in the pool has changed. + */ + if (reuse_tx_pools(adapter)) { + netdev_dbg(netdev, "Reusing tx pools\n"); + goto update_ltb; + } + + /* Allocate/populate the pools. */ + release_tx_pools(adapter); pool_size = adapter->req_tx_entries_per_subcrq; num_pools = adapter->num_active_tx_scrqs; @@ -925,6 +947,7 @@ static int init_tx_pools(struct net_device *netdev) * allocation, release_tx_pools() will know how many to look for. */ adapter->num_active_tx_pools = num_pools; + buff_size = adapter->req_mtu + VLAN_HLEN; buff_size = ALIGN(buff_size, L1_CACHE_BYTES); @@ -934,21 +957,73 @@ static int init_tx_pools(struct net_device *netdev) rc = init_one_tx_pool(netdev, &adapter->tx_pool[i], pool_size, buff_size); - if (rc) { - release_tx_pools(adapter); - return rc; - } + if (rc) + goto out_release; rc = init_one_tx_pool(netdev, &adapter->tso_pool[i], IBMVNIC_TSO_BUFS, IBMVNIC_TSO_BUF_SZ); - if (rc) { - release_tx_pools(adapter); - return rc; - } + if (rc) + goto out_release; + } + + adapter->prev_tx_pool_size = pool_size; + adapter->prev_mtu = adapter->req_mtu; + +update_ltb: + /* NOTE: All tx_pools have the same number of buffers (which is + * same as pool_size). All tso_pools have IBMVNIC_TSO_BUFS + * buffers (see calls init_one_tx_pool() for these). + * For consistency, we use tx_pool->num_buffers and + * tso_pool->num_buffers below. + */ + rc = -1; + for (i = 0; i < num_pools; i++) { + struct ibmvnic_tx_pool *tso_pool; + struct ibmvnic_tx_pool *tx_pool; + u32 ltb_size; + + tx_pool = &adapter->tx_pool[i]; + ltb_size = tx_pool->num_buffers * tx_pool->buf_size; + if (alloc_long_term_buff(adapter, &tx_pool->long_term_buff, + ltb_size)) + goto out; + + dev_dbg(dev, "Updated LTB for tx pool %d [%p, %d, %d]\n", + i, tx_pool->long_term_buff.buff, + tx_pool->num_buffers, tx_pool->buf_size); + + tx_pool->consumer_index = 0; + tx_pool->producer_index = 0; + + for (j = 0; j < tx_pool->num_buffers; j++) + tx_pool->free_map[j] = j; + + tso_pool = &adapter->tso_pool[i]; + ltb_size = tso_pool->num_buffers * tso_pool->buf_size; + if (alloc_long_term_buff(adapter, &tso_pool->long_term_buff, + ltb_size)) + goto out; + + dev_dbg(dev, "Updated LTB for tso pool %d [%p, %d, %d]\n", + i, tso_pool->long_term_buff.buff, + tso_pool->num_buffers, tso_pool->buf_size); + + tso_pool->consumer_index = 0; + tso_pool->producer_index = 0; + + for (j = 0; j < tso_pool->num_buffers; j++) + tso_pool->free_map[j] = j; } return 0; +out_release: + release_tx_pools(adapter); +out: + /* We failed to allocate one or more LTBs or map them on the VIOS. + * Hold onto the pools and any LTBs that we did allocate/map. + */ + return rc; } static void ibmvnic_napi_enable(struct ibmvnic_adapter *adapter) @@ -1139,8 +1214,6 @@ static void release_resources(struct ibmvnic_adapter *adapter) { release_vpd_data(adapter); - release_tx_pools(adapter); - release_napi(adapter); release_login_buffer(adapter); release_login_rsp_buffer(adapter); @@ -1413,6 +1486,7 @@ static int ibmvnic_open(struct net_device *netdev) netdev_err(netdev, "failed to initialize resources\n"); release_resources(adapter); release_rx_pools(adapter); + release_tx_pools(adapter); goto out; } } @@ -1541,8 +1615,6 @@ static void ibmvnic_cleanup(struct net_device *netdev) ibmvnic_napi_disable(adapter); ibmvnic_disable_irqs(adapter); - - clean_tx_pools(adapter); } static int __ibmvnic_close(struct net_device *netdev) @@ -1577,6 +1649,7 @@ static int ibmvnic_close(struct net_device *netdev) rc = __ibmvnic_close(netdev); ibmvnic_cleanup(netdev); clean_rx_pools(adapter); + clean_tx_pools(adapter); return rc; } @@ -2153,9 +2226,9 @@ static const char *reset_reason_to_string(enum ibmvnic_reset_reason reason) static int do_reset(struct ibmvnic_adapter *adapter, struct ibmvnic_rwi *rwi, u32 reset_state) { + struct net_device *netdev = adapter->netdev; u64 old_num_rx_queues, old_num_tx_queues; u64 old_num_rx_slots, old_num_tx_slots; - struct net_device *netdev = adapter->netdev; int rc; netdev_dbg(adapter->netdev, @@ -2305,7 +2378,6 @@ static int do_reset(struct ibmvnic_adapter *adapter, !adapter->rx_pool || !adapter->tso_pool || !adapter->tx_pool) { - release_tx_pools(adapter); release_napi(adapter); release_vpd_data(adapter); @@ -2314,9 +2386,10 @@ static int do_reset(struct ibmvnic_adapter *adapter, goto out; } else { - rc = reset_tx_pools(adapter); + rc = init_tx_pools(netdev); if (rc) { - netdev_dbg(adapter->netdev, "reset tx pools failed (%d)\n", + netdev_dbg(netdev, + "init tx pools failed (%d)\n", rc); goto out; } @@ -5661,6 +5734,7 @@ static int ibmvnic_probe(struct vio_dev *dev, const struct vio_device_id *id) init_completion(&adapter->stats_done); clear_bit(0, &adapter->resetting); adapter->prev_rx_buf_sz = 0; + adapter->prev_mtu = 0; init_success = false; do { @@ -5762,6 +5836,7 @@ static void ibmvnic_remove(struct vio_dev *dev) release_resources(adapter); release_rx_pools(adapter); + release_tx_pools(adapter); release_sub_crqs(adapter, 1); release_crq_queue(adapter); diff --git a/drivers/net/ethernet/ibm/ibmvnic.h b/drivers/net/ethernet/ibm/ibmvnic.h index b73a1b812368..b8e42f67d897 100644 --- a/drivers/net/ethernet/ibm/ibmvnic.h +++ b/drivers/net/ethernet/ibm/ibmvnic.h @@ -967,6 +967,7 @@ struct ibmvnic_adapter { u64 min_mtu; u64 max_mtu; u64 req_mtu; + u64 prev_mtu; u64 max_multicast_filters; u64 vlan_header_insertion; u64 rx_vlan_header_insertion; @@ -988,6 +989,7 @@ struct ibmvnic_adapter { u32 num_active_tx_pools; u32 prev_rx_pool_size; + u32 prev_tx_pool_size; u32 cur_rx_buf_sz; u32 prev_rx_buf_sz;