From patchwork Mon Sep 30 17:56:34 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Nick Child X-Patchwork-Id: 13816795 X-Patchwork-Delegate: kuba@kernel.org Received: from mx0a-001b2d01.pphosted.com (mx0a-001b2d01.pphosted.com [148.163.156.1]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 952EB18C31 for ; Mon, 30 Sep 2024 17:56:51 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=148.163.156.1 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1727719014; cv=none; b=kpNihVNyqLBK5BvoBXgrLu6FiV2qfRESenpN5VDbaerupDC+vnlWSOPq7E5YRre63ld70GXYx7TQ35UvZTQrVc8T2gpWJa/nHRopjQbLaI2vbIUukf2oi+etVrDlLGp4zh/ODItp6FVu2jiagjToGNmG1SpGHybg0jGdmwzGyFY= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1727719014; c=relaxed/simple; bh=3n3y1czsPL7M9dZqiLXNN3jvvCCXfqcP9LXHcZEhZ3g=; h=From:To:Cc:Subject:Date:Message-ID:MIME-Version; b=XVuvtFcXu6UCRPHwCtscyWSGUB+xIm02JlHCszxEVN4mOoIP3wY/uO2pNDDX1bC6ggzrhldl339ql8jaip8Lgbz4yM3YuzoP1bCchjsUj7uXzFL/FFAtoKbXW9plxDaI1O8iJ/C6cj6OsC2K+d1m7TBZpbzK/Pq4xBtmSHDvc2Y= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.ibm.com; spf=pass smtp.mailfrom=linux.ibm.com; dkim=pass (2048-bit key) header.d=ibm.com header.i=@ibm.com header.b=ktCheKQn; arc=none smtp.client-ip=148.163.156.1 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.ibm.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linux.ibm.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=ibm.com header.i=@ibm.com header.b="ktCheKQn" Received: from pps.filterd (m0353729.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.18.1.2/8.18.1.2) with ESMTP id 48U95uYW013299 for ; Mon, 30 Sep 2024 17:56:50 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ibm.com; h=from :to:cc:subject:date:message-id:mime-version :content-transfer-encoding; s=pp1; bh=05iYjv2H1R05AWg3Ar5Z1KUrIb Qlm2NHYecNifx/gG0=; b=ktCheKQn7F8K5NntMdXNtzQhyI9005YcZzDLUvLPNl BZM/IBR8zko5jdMz8MFberElrN4Wpy5OMSKuPE4z57vypzhimFmodXkzTafpLWr6 s4hgbgEd5AfDPus52g50Wgcy1WAQfnW1VwhvbH1YKmHr+nEsY0XbSCWzN6hiIZMU c56RJGI5mJEfBLrvFWItSUHLV7hXs9uALY/259eU6BNb5X46Ln7fgFoXKU9LKiyr sTAMH/QfFqGYooUeKs6rEy/iXnVCRcztyWV/Mk/rVqZXIzrhIvhs9J4uiyGjHeeR O2OEVKZ+7Gs9tA65TeTMbuLZSRWA9qoyRH2uVJu+GIng== Received: from ppma23.wdc07v.mail.ibm.com (5d.69.3da9.ip4.static.sl-reverse.com [169.61.105.93]) by mx0a-001b2d01.pphosted.com (PPS) with ESMTPS id 41x9fwud87-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT) for ; Mon, 30 Sep 2024 17:56:50 +0000 (GMT) Received: from pps.filterd (ppma23.wdc07v.mail.ibm.com [127.0.0.1]) by ppma23.wdc07v.mail.ibm.com (8.18.1.2/8.18.1.2) with ESMTP id 48UEoZwv014098 for ; Mon, 30 Sep 2024 17:56:49 GMT Received: from smtprelay04.dal12v.mail.ibm.com ([172.16.1.6]) by ppma23.wdc07v.mail.ibm.com (PPS) with ESMTPS id 41xwmjyuw6-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT) for ; Mon, 30 Sep 2024 17:56:49 +0000 Received: from smtpav03.dal12v.mail.ibm.com (smtpav03.dal12v.mail.ibm.com [10.241.53.102]) by smtprelay04.dal12v.mail.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id 48UHuiMn37290294 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Mon, 30 Sep 2024 17:56:44 GMT Received: from smtpav03.dal12v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 7EFF458063; Mon, 30 Sep 2024 17:56:44 +0000 (GMT) Received: from smtpav03.dal12v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 5F14B5803F; Mon, 30 Sep 2024 17:56:44 +0000 (GMT) Received: from tinkpad.ibmuc.com (unknown [9.61.45.36]) by smtpav03.dal12v.mail.ibm.com (Postfix) with ESMTP; Mon, 30 Sep 2024 17:56:44 +0000 (GMT) From: Nick Child To: netdev@vger.kernel.org Cc: haren@linux.ibm.com, ricklind@us.ibm.com, Nick Child Subject: [PATCH net 1/2] ibmvnic: Add stat for tx direct vs tx batched Date: Mon, 30 Sep 2024 12:56:34 -0500 Message-ID: <20240930175635.1670111-1-nnac123@linux.ibm.com> X-Mailer: git-send-email 2.43.5 Precedence: bulk X-Mailing-List: netdev@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-TM-AS-GCONF: 00 X-Proofpoint-ORIG-GUID: RL_41u82WNpjffx3p_OvGUn8BtHSIVLb X-Proofpoint-GUID: RL_41u82WNpjffx3p_OvGUn8BtHSIVLb X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293,Aquarius:18.0.1051,Hydra:6.0.680,FMLib:17.12.62.30 definitions=2024-09-30_17,2024-09-30_01,2024-09-30_01 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 priorityscore=1501 clxscore=1011 malwarescore=0 adultscore=0 spamscore=0 impostorscore=0 mlxscore=0 mlxlogscore=697 bulkscore=0 suspectscore=0 lowpriorityscore=0 phishscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.19.0-2408220000 definitions=main-2409300126 X-Patchwork-Delegate: kuba@kernel.org Allow tracking of packets sent with send_subcrq direct vs indirect. `ethtool -S ` will now provide a counter of the number of uses of each xmit method. This metric will be useful in performance debugging. Signed-off-by: Nick Child --- drivers/net/ethernet/ibm/ibmvnic.c | 23 ++++++++++++++++------- drivers/net/ethernet/ibm/ibmvnic.h | 3 ++- 2 files changed, 18 insertions(+), 8 deletions(-) diff --git a/drivers/net/ethernet/ibm/ibmvnic.c b/drivers/net/ethernet/ibm/ibmvnic.c index 87e693a81433..53b309ddc63b 100644 --- a/drivers/net/ethernet/ibm/ibmvnic.c +++ b/drivers/net/ethernet/ibm/ibmvnic.c @@ -2310,7 +2310,7 @@ static void ibmvnic_tx_scrq_clean_buffer(struct ibmvnic_adapter *adapter, tx_buff = &tx_pool->tx_buff[index]; adapter->netdev->stats.tx_packets--; adapter->netdev->stats.tx_bytes -= tx_buff->skb->len; - adapter->tx_stats_buffers[queue_num].packets--; + adapter->tx_stats_buffers[queue_num].batched_packets--; adapter->tx_stats_buffers[queue_num].bytes -= tx_buff->skb->len; dev_kfree_skb_any(tx_buff->skb); @@ -2402,7 +2402,8 @@ static netdev_tx_t ibmvnic_xmit(struct sk_buff *skb, struct net_device *netdev) unsigned int tx_map_failed = 0; union sub_crq indir_arr[16]; unsigned int tx_dropped = 0; - unsigned int tx_packets = 0; + unsigned int tx_dpackets = 0; + unsigned int tx_bpackets = 0; unsigned int tx_bytes = 0; dma_addr_t data_dma_addr; struct netdev_queue *txq; @@ -2573,6 +2574,7 @@ static netdev_tx_t ibmvnic_xmit(struct sk_buff *skb, struct net_device *netdev) if (lpar_rc != H_SUCCESS) goto tx_err; + tx_dpackets++; goto early_exit; } @@ -2601,6 +2603,8 @@ static netdev_tx_t ibmvnic_xmit(struct sk_buff *skb, struct net_device *netdev) goto tx_err; } + tx_bpackets++; + early_exit: if (atomic_add_return(num_entries, &tx_scrq->used) >= adapter->req_tx_entries_per_subcrq) { @@ -2608,7 +2612,6 @@ static netdev_tx_t ibmvnic_xmit(struct sk_buff *skb, struct net_device *netdev) netif_stop_subqueue(netdev, queue_num); } - tx_packets++; tx_bytes += skb->len; txq_trans_cond_update(txq); ret = NETDEV_TX_OK; @@ -2638,10 +2641,11 @@ static netdev_tx_t ibmvnic_xmit(struct sk_buff *skb, struct net_device *netdev) rcu_read_unlock(); netdev->stats.tx_dropped += tx_dropped; netdev->stats.tx_bytes += tx_bytes; - netdev->stats.tx_packets += tx_packets; + netdev->stats.tx_packets += tx_bpackets + tx_dpackets; adapter->tx_send_failed += tx_send_failed; adapter->tx_map_failed += tx_map_failed; - adapter->tx_stats_buffers[queue_num].packets += tx_packets; + adapter->tx_stats_buffers[queue_num].batched_packets += tx_bpackets; + adapter->tx_stats_buffers[queue_num].direct_packets += tx_dpackets; adapter->tx_stats_buffers[queue_num].bytes += tx_bytes; adapter->tx_stats_buffers[queue_num].dropped_packets += tx_dropped; @@ -3806,7 +3810,10 @@ static void ibmvnic_get_strings(struct net_device *dev, u32 stringset, u8 *data) memcpy(data, ibmvnic_stats[i].name, ETH_GSTRING_LEN); for (i = 0; i < adapter->req_tx_queues; i++) { - snprintf(data, ETH_GSTRING_LEN, "tx%d_packets", i); + snprintf(data, ETH_GSTRING_LEN, "tx%d_batched_packets", i); + data += ETH_GSTRING_LEN; + + snprintf(data, ETH_GSTRING_LEN, "tx%d_direct_packets", i); data += ETH_GSTRING_LEN; snprintf(data, ETH_GSTRING_LEN, "tx%d_bytes", i); @@ -3871,7 +3878,9 @@ static void ibmvnic_get_ethtool_stats(struct net_device *dev, (adapter, ibmvnic_stats[i].offset)); for (j = 0; j < adapter->req_tx_queues; j++) { - data[i] = adapter->tx_stats_buffers[j].packets; + data[i] = adapter->tx_stats_buffers[j].batched_packets; + i++; + data[i] = adapter->tx_stats_buffers[j].direct_packets; i++; data[i] = adapter->tx_stats_buffers[j].bytes; i++; diff --git a/drivers/net/ethernet/ibm/ibmvnic.h b/drivers/net/ethernet/ibm/ibmvnic.h index 94ac36b1408b..a189038d88df 100644 --- a/drivers/net/ethernet/ibm/ibmvnic.h +++ b/drivers/net/ethernet/ibm/ibmvnic.h @@ -213,7 +213,8 @@ struct ibmvnic_statistics { #define NUM_TX_STATS 3 struct ibmvnic_tx_queue_stats { - u64 packets; + u64 batched_packets; + u64 direct_packets; u64 bytes; u64 dropped_packets; }; From patchwork Mon Sep 30 17:56:35 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Nick Child X-Patchwork-Id: 13816796 X-Patchwork-Delegate: kuba@kernel.org Received: from mx0b-001b2d01.pphosted.com (mx0b-001b2d01.pphosted.com [148.163.158.5]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id CB238198E96 for ; Mon, 30 Sep 2024 17:56:55 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=148.163.158.5 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1727719018; cv=none; b=S6UZRwb9WcRPIILPcB5FFfA+9WIM2wHyhLW1QbldWuKgdXK+BlUe0fM8RA1m+VjNLXfMv0ZuTG9HEWqG2pPVCZNNi+0wqEDrDck+P6ATkJ0fHhXmrhr/CAs1IKrxLOTzQnGCXUiS1IRekuZYmTXKbj/NYnW4iU4sOWR8mBr+85A= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1727719018; c=relaxed/simple; bh=eKo/A02EaTF15W6w2BCW6hV0IYdC0sQAiJ0vJbPEeH8=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=KpWgjsLQ4Ly0ZZ7u4d3K9vsijB/jU8CX8FJ3qaEVFHy3yDEKTIInGZB75qQssDbEUwl996vXh9IT3gxVUfcNQVctaBKf6w6TIYT9gOs7oqC53gOhettsEKNUeNLrqe9UhyX/66vmb20QxVCzuJZDgKK3UR/Mp/6jhrfputbpiRA= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.ibm.com; spf=pass smtp.mailfrom=linux.ibm.com; dkim=pass (2048-bit key) header.d=ibm.com header.i=@ibm.com header.b=W0ix6LOl; arc=none smtp.client-ip=148.163.158.5 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.ibm.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linux.ibm.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=ibm.com header.i=@ibm.com header.b="W0ix6LOl" Received: from pps.filterd (m0353725.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.18.1.2/8.18.1.2) with ESMTP id 48UBeouC025312 for ; Mon, 30 Sep 2024 17:56:54 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ibm.com; h=from :to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; s=pp1; bh=D5lY1Z/wx5TWA HnamX6pZiJXQ02Cwt+o3jfInqA3PIQ=; b=W0ix6LOlZIEkwXzmqZp9Owv1vtKe2 zRndMnJi9XrQ4JNiu0QKz4G7WJEuknE7Bc3HqPOd2gDEu+kmOWg3KKy5jPuM5cjq w3oRkqiNS79haJZj7jK/IfMnHCRxQQkrZphk9HB+XGqSKEem0TbVoR3u6UX73ygh VmKt1MxEWUHuq90B3PslDwGfbSPS8Bz8usBhRjjTPsfp65SF51pVq848H8JF0mmb HnYT/mIp25cZmzTf0bxBRncsv0dzrFXL3Hlu2AQfLpX+chjhNvwxi1JTAI0XxxN2 FfZWZRjuHpNX02Xj04raGG/8FjZ94zvCesNWxSivH3thc2JHHxBzbjKiw== Received: from ppma11.dal12v.mail.ibm.com (db.9e.1632.ip4.static.sl-reverse.com [50.22.158.219]) by mx0a-001b2d01.pphosted.com (PPS) with ESMTPS id 41x87kk4j1-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT) for ; Mon, 30 Sep 2024 17:56:54 +0000 (GMT) Received: from pps.filterd (ppma11.dal12v.mail.ibm.com [127.0.0.1]) by ppma11.dal12v.mail.ibm.com (8.18.1.2/8.18.1.2) with ESMTP id 48UFoqsU002386 for ; Mon, 30 Sep 2024 17:56:53 GMT Received: from smtprelay05.dal12v.mail.ibm.com ([172.16.1.7]) by ppma11.dal12v.mail.ibm.com (PPS) with ESMTPS id 41xxu0ykw1-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT) for ; Mon, 30 Sep 2024 17:56:53 +0000 Received: from smtpav03.dal12v.mail.ibm.com (smtpav03.dal12v.mail.ibm.com [10.241.53.102]) by smtprelay05.dal12v.mail.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id 48UHupAY29360488 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Mon, 30 Sep 2024 17:56:51 GMT Received: from smtpav03.dal12v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 2956F58056; Mon, 30 Sep 2024 17:56:51 +0000 (GMT) Received: from smtpav03.dal12v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 0DD655803F; Mon, 30 Sep 2024 17:56:51 +0000 (GMT) Received: from tinkpad.ibmuc.com (unknown [9.61.45.36]) by smtpav03.dal12v.mail.ibm.com (Postfix) with ESMTP; Mon, 30 Sep 2024 17:56:51 +0000 (GMT) From: Nick Child To: netdev@vger.kernel.org Cc: haren@linux.ibm.com, ricklind@us.ibm.com, Nick Child Subject: [PATCH net 2/2] ibmvnic: Inspect header requirements before using scrq direct Date: Mon, 30 Sep 2024 12:56:35 -0500 Message-ID: <20240930175635.1670111-2-nnac123@linux.ibm.com> X-Mailer: git-send-email 2.43.5 In-Reply-To: <20240930175635.1670111-1-nnac123@linux.ibm.com> References: <20240930175635.1670111-1-nnac123@linux.ibm.com> Precedence: bulk X-Mailing-List: netdev@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-TM-AS-GCONF: 00 X-Proofpoint-GUID: Ox5rk0rLpYA8B1VNPQEXLuGasNTMBJTH X-Proofpoint-ORIG-GUID: Ox5rk0rLpYA8B1VNPQEXLuGasNTMBJTH X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293,Aquarius:18.0.1051,Hydra:6.0.680,FMLib:17.12.62.30 definitions=2024-09-30_17,2024-09-30_01,2024-09-30_01 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 malwarescore=0 impostorscore=0 lowpriorityscore=0 clxscore=1015 phishscore=0 spamscore=0 priorityscore=1501 mlxscore=0 mlxlogscore=719 adultscore=0 suspectscore=0 bulkscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.19.0-2408220000 definitions=main-2409300126 X-Patchwork-Delegate: kuba@kernel.org Previously, the TX header requirement for standard frames was ignored. This requirement is a bitstring sent from the VIOS which maps to the type of header information needed during TX. If no header information, is needed then send subcrq direct can be used (which can be more performant). This bitstring was previously ignored for standard packets (AKA non LSO, non CSO) due to the belief that the bitstring was over-cautionary. It turns out that there are some configurations where the backing device does need header information for transmission of standard packets. If the information is not supplied then this causes continuous "Adapter error" transport events. Therefore, this bitstring should be respected and observed before considering the use of send subcrq direct. Fixes: 1c33e29245cc ("ibmvnic: Only record tx completed bytes once per handler") Signed-off-by: Nick Child --- drivers/net/ethernet/ibm/ibmvnic.c | 6 ++++-- 1 file changed, 4 insertions(+), 2 deletions(-) diff --git a/drivers/net/ethernet/ibm/ibmvnic.c b/drivers/net/ethernet/ibm/ibmvnic.c index 53b309ddc63b..cca2ed6ad289 100644 --- a/drivers/net/ethernet/ibm/ibmvnic.c +++ b/drivers/net/ethernet/ibm/ibmvnic.c @@ -2473,9 +2473,11 @@ static netdev_tx_t ibmvnic_xmit(struct sk_buff *skb, struct net_device *netdev) /* if we are going to send_subcrq_direct this then we need to * update the checksum before copying the data into ltb. Essentially * these packets force disable CSO so that we can guarantee that - * FW does not need header info and we can send direct. + * FW does not need header info and we can send direct. Also, vnic + * server must be able to xmit standard packets without header data */ - if (!skb_is_gso(skb) && !ind_bufp->index && !netdev_xmit_more()) { + if (*hdrs == 0 && !skb_is_gso(skb) && + !ind_bufp->index && !netdev_xmit_more()) { use_scrq_send_direct = true; if (skb->ip_summed == CHECKSUM_PARTIAL && skb_checksum_help(skb))