From patchwork Wed Aug 7 23:13:25 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Tony Nguyen X-Patchwork-Id: 13756838 X-Patchwork-Delegate: kuba@kernel.org Received: from mgamail.intel.com (mgamail.intel.com [192.198.163.10]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 06428155399 for ; Wed, 7 Aug 2024 23:13:35 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=192.198.163.10 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1723072417; cv=none; b=QNTbf9+IP/XZKk9QRaLwrw/n3s4Q13Cwd8DwK8Rr6UJWNDEIQOsfxIrcCnv962xoV7fC8dZAQCpk2Jia825DSYKD0TwtmKB+Kh4QG0KDWP1nS9CAtJpYeDQ5kPapRoqFqZamVfCNIfzwZIl+15a357QhyhZ/ygCge2HOV5YsR5k= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1723072417; c=relaxed/simple; bh=5mM5ABSp43dV7RdzUepqRAXJkGdgenmlsijFtwlLMuA=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=eEP5z1X5YD2hL8js00pb9yMZDj/QVYIQEzroJSPjxjaYKzr2F7E1ZkiQlzZW9lgw6vQkWsnB0Qht1MuxfX+v6SNaCPtvjZFhGsyL+IglQ2T0oGQssQNK5hQ4L7Kr3xzf82w+9bhMgPP6HJW7AK9KAKstjBrqfkP+jf8Kjuwq3uw= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com; spf=pass smtp.mailfrom=intel.com; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b=YJCWgZ+D; arc=none smtp.client-ip=192.198.163.10 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=intel.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b="YJCWgZ+D" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1723072416; x=1754608416; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=5mM5ABSp43dV7RdzUepqRAXJkGdgenmlsijFtwlLMuA=; b=YJCWgZ+D5SlH+YOfp8Wo2CW41uVH+pY+WZzOBw771Lks3a5gs+/ptYI4 Nuvf/4uNIVukounhD8F/nxt/1lhDv74SL9xWdf0xnFGUo2CAVYWFO0yi5 o5kgUz7qu3EvI723/1+1ZCotp0Ff2iLjfSlCKFPZqz37UbaQ3TGT/cNuH 2gLE6oMZBqWGob4NG9lQ/TNWhRKth7bXMjw2UT1Q5l14UyCBHhZJf1PUG FKMOYTCRzDT7El/NHP4BuzQoxulBiSscJLIQe0y+kKKHHP6edGHbP1Wb3 ilUKeur0zPR/HdKT3PTky66IUjl9X4Fc96lWrHsnslV0s0DEW4ruIKZsJ w==; X-CSE-ConnectionGUID: OLuKLyfIR+WIZGKb+zJk4Q== X-CSE-MsgGUID: 8vyyRCnNRyCE2Os+M01ZHg== X-IronPort-AV: E=McAfee;i="6700,10204,11157"; a="32577304" X-IronPort-AV: E=Sophos;i="6.09,271,1716274800"; d="scan'208";a="32577304" Received: from orviesa004.jf.intel.com ([10.64.159.144]) by fmvoesa104.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 07 Aug 2024 16:13:34 -0700 X-CSE-ConnectionGUID: DU+/IvOwT9aGgTQpN1fA7w== X-CSE-MsgGUID: IOEk77/VSzmnW2sG3Bgl9w== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.09,271,1716274800"; d="scan'208";a="61956620" Received: from anguy11-upstream.jf.intel.com ([10.166.9.133]) by orviesa004.jf.intel.com with ESMTP; 07 Aug 2024 16:13:34 -0700 From: Tony Nguyen To: davem@davemloft.net, kuba@kernel.org, pabeni@redhat.com, edumazet@google.com, netdev@vger.kernel.org Cc: Faizal Rahim , anthony.l.nguyen@intel.com, vinicius.gomes@intel.com, sasha.neftin@intel.com, richardcochran@gmail.com, horms@kernel.org, rodrigo.cadore@l-acoustics.com, Mor Bar-Gabay Subject: [PATCH net 1/4] igc: Fix packet still tx after gate close by reducing i226 MAC retry buffer Date: Wed, 7 Aug 2024 16:13:25 -0700 Message-ID: <20240807231329.3827092-2-anthony.l.nguyen@intel.com> X-Mailer: git-send-email 2.42.0 In-Reply-To: <20240807231329.3827092-1-anthony.l.nguyen@intel.com> References: <20240807231329.3827092-1-anthony.l.nguyen@intel.com> Precedence: bulk X-Mailing-List: netdev@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Patchwork-Delegate: kuba@kernel.org From: Faizal Rahim Testing uncovered that even when the taprio gate is closed, some packets still transmit. According to i225/6 hardware errata [1], traffic might overflow the planned QBV window. This happens because MAC maintains an internal buffer, primarily for supporting half duplex retries. Therefore, even when the gate closes, residual MAC data in the buffer may still transmit. To mitigate this for i226, reduce the MAC's internal buffer from 192 bytes to the recommended 88 bytes by modifying the RETX_CTL register value. This follows guidelines from: [1] Ethernet Controller I225/I22 Spec Update Rev 2.1 Errata Item 9: TSN: Packet Transmission Might Cross Qbv Window [2] I225/6 SW User Manual Rev 1.2.4: Section 8.11.5 Retry Buffer Control Note that the RETX_CTL register can't be used in TSN mode because half duplex feature cannot coexist with TSN. Test Steps: 1. Send taprio cmd to board A: tc qdisc replace dev enp1s0 parent root handle 100 taprio \ num_tc 4 \ map 3 2 1 0 3 3 3 3 3 3 3 3 3 3 3 3 \ queues 1@0 1@1 1@2 1@3 \ base-time 0 \ sched-entry S 0x07 500000 \ sched-entry S 0x0f 500000 \ flags 0x2 \ txtime-delay 0 Note that for TC3, gate should open for 500us and close for another 500us. 3. Take tcpdump log on Board B. 4. Send udp packets via UDP tai app from Board A to Board B. 5. Analyze tcpdump log via wireshark log on Board B. Ensure that the total time from the first to the last packet received during one cycle for TC3 does not exceed 500us. Fixes: 43546211738e ("igc: Add new device ID's") Signed-off-by: Faizal Rahim Acked-by: Vinicius Costa Gomes Tested-by: Mor Bar-Gabay Signed-off-by: Tony Nguyen --- drivers/net/ethernet/intel/igc/igc_defines.h | 6 ++++ drivers/net/ethernet/intel/igc/igc_tsn.c | 34 ++++++++++++++++++++ 2 files changed, 40 insertions(+) diff --git a/drivers/net/ethernet/intel/igc/igc_defines.h b/drivers/net/ethernet/intel/igc/igc_defines.h index 5f92b3c7c3d4..511384f3ec5c 100644 --- a/drivers/net/ethernet/intel/igc/igc_defines.h +++ b/drivers/net/ethernet/intel/igc/igc_defines.h @@ -404,6 +404,12 @@ #define IGC_DTXMXPKTSZ_TSN 0x19 /* 1600 bytes of max TX DMA packet size */ #define IGC_DTXMXPKTSZ_DEFAULT 0x98 /* 9728-byte Jumbo frames */ +/* Retry Buffer Control */ +#define IGC_RETX_CTL 0x041C +#define IGC_RETX_CTL_WATERMARK_MASK 0xF +#define IGC_RETX_CTL_QBVFULLTH_SHIFT 8 /* QBV Retry Buffer Full Threshold */ +#define IGC_RETX_CTL_QBVFULLEN 0x1000 /* Enable QBV Retry Buffer Full Threshold */ + /* Transmit Scheduling Latency */ /* Latency between transmission scheduling (LaunchTime) and the time * the packet is transmitted to the network in nanosecond. diff --git a/drivers/net/ethernet/intel/igc/igc_tsn.c b/drivers/net/ethernet/intel/igc/igc_tsn.c index 22cefb1eeedf..46d4c3275bbb 100644 --- a/drivers/net/ethernet/intel/igc/igc_tsn.c +++ b/drivers/net/ethernet/intel/igc/igc_tsn.c @@ -78,6 +78,15 @@ void igc_tsn_adjust_txtime_offset(struct igc_adapter *adapter) wr32(IGC_GTXOFFSET, txoffset); } +static void igc_tsn_restore_retx_default(struct igc_adapter *adapter) +{ + struct igc_hw *hw = &adapter->hw; + u32 retxctl; + + retxctl = rd32(IGC_RETX_CTL) & IGC_RETX_CTL_WATERMARK_MASK; + wr32(IGC_RETX_CTL, retxctl); +} + /* Returns the TSN specific registers to their default values after * the adapter is reset. */ @@ -91,6 +100,9 @@ static int igc_tsn_disable_offload(struct igc_adapter *adapter) wr32(IGC_TXPBS, I225_TXPBSIZE_DEFAULT); wr32(IGC_DTXMXPKTSZ, IGC_DTXMXPKTSZ_DEFAULT); + if (igc_is_device_id_i226(hw)) + igc_tsn_restore_retx_default(adapter); + tqavctrl = rd32(IGC_TQAVCTRL); tqavctrl &= ~(IGC_TQAVCTRL_TRANSMIT_MODE_TSN | IGC_TQAVCTRL_ENHANCED_QAV | IGC_TQAVCTRL_FUTSCDDIS); @@ -111,6 +123,25 @@ static int igc_tsn_disable_offload(struct igc_adapter *adapter) return 0; } +/* To partially fix i226 HW errata, reduce MAC internal buffering from 192 Bytes + * to 88 Bytes by setting RETX_CTL register using the recommendation from: + * a) Ethernet Controller I225/I226 Specification Update Rev 2.1 + * Item 9: TSN: Packet Transmission Might Cross the Qbv Window + * b) I225/6 SW User Manual Rev 1.2.4: Section 8.11.5 Retry Buffer Control + */ +static void igc_tsn_set_retx_qbvfullthreshold(struct igc_adapter *adapter) +{ + struct igc_hw *hw = &adapter->hw; + u32 retxctl, watermark; + + retxctl = rd32(IGC_RETX_CTL); + watermark = retxctl & IGC_RETX_CTL_WATERMARK_MASK; + /* Set QBVFULLTH value using watermark and set QBVFULLEN */ + retxctl |= (watermark << IGC_RETX_CTL_QBVFULLTH_SHIFT) | + IGC_RETX_CTL_QBVFULLEN; + wr32(IGC_RETX_CTL, retxctl); +} + static int igc_tsn_enable_offload(struct igc_adapter *adapter) { struct igc_hw *hw = &adapter->hw; @@ -123,6 +154,9 @@ static int igc_tsn_enable_offload(struct igc_adapter *adapter) wr32(IGC_DTXMXPKTSZ, IGC_DTXMXPKTSZ_TSN); wr32(IGC_TXPBS, IGC_TXPBSIZE_TSN); + if (igc_is_device_id_i226(hw)) + igc_tsn_set_retx_qbvfullthreshold(adapter); + for (i = 0; i < adapter->num_tx_queues; i++) { struct igc_ring *ring = adapter->tx_ring[i]; u32 txqctl = 0; From patchwork Wed Aug 7 23:13:26 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Tony Nguyen X-Patchwork-Id: 13756839 X-Patchwork-Delegate: kuba@kernel.org Received: from mgamail.intel.com (mgamail.intel.com [192.198.163.10]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 85A581553BC for ; Wed, 7 Aug 2024 23:13:36 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=192.198.163.10 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1723072418; cv=none; b=PgFtUXbI75aAZx5FV0BHMXrPTtQb3evC/3LdgPoHgz2rFebLE98DCbee6+w0VMrv4pk/dutjqwmj38p3znnGlscaPOol3g+OeBi7sknccevLQAtshVyFyTVVaafvLzTptvWLSGtYH2YYajylfIIPAHxk5RSVCWaBnB2c3zyNVCw= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1723072418; c=relaxed/simple; bh=gYW1Mz/N9tELiFPT0gvgddfr/2nTZcI9aC+jVh1t2Mk=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=PviUCe5S4rE33SKd2OgDg39Udplckh0fEmZVZAVxK6TS7sk5a3X+Y9LSLXNlGIY0dX0eyWkCY5bl7XbozuvWhQhialnMel2sMff0vuEno7DuoHKyEdUsRJTdFCbrwBTRq5fucKofW9Q9Z2twuaXb8bvivjjTPuQ9Id+0Jehpn38= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com; spf=pass smtp.mailfrom=intel.com; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b=oD6psJ8D; arc=none smtp.client-ip=192.198.163.10 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=intel.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b="oD6psJ8D" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1723072416; x=1754608416; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=gYW1Mz/N9tELiFPT0gvgddfr/2nTZcI9aC+jVh1t2Mk=; b=oD6psJ8DX1s5yM4H8+OR9fFbYT1GubVZaLDfFHltMhkCWIlIe9lJBrSu jfXyJaqEYKmfjl9S43qKzeNhsRSwdhCzzL31QuwTPk7zi9zBIOBT69co9 4RnxTH8bVigVBsd8fwyyHh79Zb9N0RGY6t7QG+UAfR88V1DWY3l+AxTjc f3fdG7KNB1l2sxBfrxgxwBnrl2aFri/+xCjD6NzZwn/NvyUWyPG/GuwYD IrbaTEf6fCYK6rApIaleCPoa9+iqBKOP9HWncV13PnePK7rdHhASD7VGV CB1xqkKf0QAt97etjaMcCFKrSmdL356LWcoBJgJCnPSxHwj6VCdvZC0sc w==; X-CSE-ConnectionGUID: Bc7B/8OZTxeV3I46+cOj0Q== X-CSE-MsgGUID: YMNs4VhyRFKp2/h/qS39AA== X-IronPort-AV: E=McAfee;i="6700,10204,11157"; a="32577308" X-IronPort-AV: E=Sophos;i="6.09,271,1716274800"; d="scan'208";a="32577308" Received: from orviesa004.jf.intel.com ([10.64.159.144]) by fmvoesa104.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 07 Aug 2024 16:13:34 -0700 X-CSE-ConnectionGUID: 3bCYnsRVSiWur9MlCXyXvw== X-CSE-MsgGUID: 52YMflCxRUW7mmEgrNHAug== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.09,271,1716274800"; d="scan'208";a="61956622" Received: from anguy11-upstream.jf.intel.com ([10.166.9.133]) by orviesa004.jf.intel.com with ESMTP; 07 Aug 2024 16:13:34 -0700 From: Tony Nguyen To: davem@davemloft.net, kuba@kernel.org, pabeni@redhat.com, edumazet@google.com, netdev@vger.kernel.org Cc: Faizal Rahim , anthony.l.nguyen@intel.com, vinicius.gomes@intel.com, sasha.neftin@intel.com, richardcochran@gmail.com, horms@kernel.org, rodrigo.cadore@l-acoustics.com, Mor Bar-Gabay Subject: [PATCH net 2/4] igc: Fix qbv_config_change_errors logics Date: Wed, 7 Aug 2024 16:13:26 -0700 Message-ID: <20240807231329.3827092-3-anthony.l.nguyen@intel.com> X-Mailer: git-send-email 2.42.0 In-Reply-To: <20240807231329.3827092-1-anthony.l.nguyen@intel.com> References: <20240807231329.3827092-1-anthony.l.nguyen@intel.com> Precedence: bulk X-Mailing-List: netdev@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Patchwork-Delegate: kuba@kernel.org From: Faizal Rahim When user issues these cmds: 1. Either a) or b) a) mqprio with hardware offload disabled b) taprio with txtime-assist feature enabled 2. etf 3. tc qdisc delete 4. taprio with base time in the past At step 4, qbv_config_change_errors wrongly increased by 1. Excerpt from IEEE 802.1Q-2018 8.6.9.3.1: "If AdminBaseTime specifies a time in the past, and the current schedule is running, then: Increment ConfigChangeError counter" qbv_config_change_errors should only increase if base time is in the past and no taprio is active. In user perspective, taprio was not active when first triggered at step 4. However, i225/6 reuses qbv for etf, so qbv is enabled with a dummy schedule at step 2 where it enters igc_tsn_enable_offload() and qbv_count got incremented to 1. At step 4, it enters igc_tsn_enable_offload() again, qbv_count is incremented to 2. Because taprio is running, tc_setup_type is TC_SETUP_QDISC_ETF and qbv_count > 1, qbv_config_change_errors value got incremented. This issue happens due to reliance on qbv_count field where a non-zero value indicates that taprio is running. But qbv_count increases regardless if taprio is triggered by user or by other tsn feature. It does not align with qbv_config_change_errors expectation where it is only concerned with taprio triggered by user. Fixing this by relocating the qbv_config_change_errors logic to igc_save_qbv_schedule(), eliminating reliance on qbv_count and its inaccuracies from i225/6's multiple uses of qbv feature for other TSN features. The new function created: igc_tsn_is_taprio_activated_by_user() uses taprio_offload_enable field to indicate that the current running taprio was triggered by user, instead of triggered by non-qbv feature like etf. Fixes: ae4fe4698300 ("igc: Add qbv_config_change_errors counter") Signed-off-by: Faizal Rahim Reviewed-by: Simon Horman Acked-by: Vinicius Costa Gomes Tested-by: Mor Bar-Gabay Signed-off-by: Tony Nguyen --- drivers/net/ethernet/intel/igc/igc_main.c | 8 ++++++-- drivers/net/ethernet/intel/igc/igc_tsn.c | 16 ++++++++-------- drivers/net/ethernet/intel/igc/igc_tsn.h | 1 + 3 files changed, 15 insertions(+), 10 deletions(-) diff --git a/drivers/net/ethernet/intel/igc/igc_main.c b/drivers/net/ethernet/intel/igc/igc_main.c index 8daf938afc36..dfd6c00b4205 100644 --- a/drivers/net/ethernet/intel/igc/igc_main.c +++ b/drivers/net/ethernet/intel/igc/igc_main.c @@ -6315,12 +6315,16 @@ static int igc_save_qbv_schedule(struct igc_adapter *adapter, if (!validate_schedule(adapter, qopt)) return -EINVAL; + igc_ptp_read(adapter, &now); + + if (igc_tsn_is_taprio_activated_by_user(adapter) && + is_base_time_past(qopt->base_time, &now)) + adapter->qbv_config_change_errors++; + adapter->cycle_time = qopt->cycle_time; adapter->base_time = qopt->base_time; adapter->taprio_offload_enable = true; - igc_ptp_read(adapter, &now); - for (n = 0; n < qopt->num_entries; n++) { struct tc_taprio_sched_entry *e = &qopt->entries[n]; diff --git a/drivers/net/ethernet/intel/igc/igc_tsn.c b/drivers/net/ethernet/intel/igc/igc_tsn.c index 46d4c3275bbb..8ed7b965484d 100644 --- a/drivers/net/ethernet/intel/igc/igc_tsn.c +++ b/drivers/net/ethernet/intel/igc/igc_tsn.c @@ -87,6 +87,14 @@ static void igc_tsn_restore_retx_default(struct igc_adapter *adapter) wr32(IGC_RETX_CTL, retxctl); } +bool igc_tsn_is_taprio_activated_by_user(struct igc_adapter *adapter) +{ + struct igc_hw *hw = &adapter->hw; + + return (rd32(IGC_BASET_H) || rd32(IGC_BASET_L)) && + adapter->taprio_offload_enable; +} + /* Returns the TSN specific registers to their default values after * the adapter is reset. */ @@ -296,14 +304,6 @@ static int igc_tsn_enable_offload(struct igc_adapter *adapter) s64 n = div64_s64(ktime_sub_ns(systim, base_time), cycle); base_time = ktime_add_ns(base_time, (n + 1) * cycle); - - /* Increase the counter if scheduling into the past while - * Gate Control List (GCL) is running. - */ - if ((rd32(IGC_BASET_H) || rd32(IGC_BASET_L)) && - (adapter->tc_setup_type == TC_SETUP_QDISC_TAPRIO) && - (adapter->qbv_count > 1)) - adapter->qbv_config_change_errors++; } else { if (igc_is_device_id_i226(hw)) { ktime_t adjust_time, expires_time; diff --git a/drivers/net/ethernet/intel/igc/igc_tsn.h b/drivers/net/ethernet/intel/igc/igc_tsn.h index b53e6af560b7..98ec845a86bf 100644 --- a/drivers/net/ethernet/intel/igc/igc_tsn.h +++ b/drivers/net/ethernet/intel/igc/igc_tsn.h @@ -7,5 +7,6 @@ int igc_tsn_offload_apply(struct igc_adapter *adapter); int igc_tsn_reset(struct igc_adapter *adapter); void igc_tsn_adjust_txtime_offset(struct igc_adapter *adapter); +bool igc_tsn_is_taprio_activated_by_user(struct igc_adapter *adapter); #endif /* _IGC_BASE_H */ From patchwork Wed Aug 7 23:13:27 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Tony Nguyen X-Patchwork-Id: 13756840 X-Patchwork-Delegate: kuba@kernel.org Received: from mgamail.intel.com (mgamail.intel.com [192.198.163.10]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id B6F35155A21 for ; Wed, 7 Aug 2024 23:13:37 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=192.198.163.10 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1723072419; cv=none; b=At0AUmTiR9WyomAzPho2WFbds8/wlAouIO5O97NOfVKQ49qsFtVbiVdeZ7cCqjfN3ab1Cc4Nh0zt0hpdwMOcOVZwTKhCpPnAq2kDPsIRGBFchkqXU5IP4iPonfxc/V9FsiKXaUQtVdyWONxmYiOVUR0PF9jKSD/Vy9EYFtF2oZg= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1723072419; c=relaxed/simple; bh=+sgRE3ATJn+TS8TiBRcbN9sQCHaISZPkexzHT2Sa89Y=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=TNDEar2MvAWbiRW0pYr5TJIiFshvqUIlMa2rGADMGL4ujT5FrVfQKA1i/6Cr2yhOGVZUPdekBC9rL9A017A075UXJj2E41VjGX7MOqi7azcZqysfJLhji+LSM7Y07Nrc4ARWqzdZ6+HFbCtAx8gm1CItl8RdR6GZPsfNob5rNfs= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com; spf=pass smtp.mailfrom=intel.com; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b=BA91iDZy; arc=none smtp.client-ip=192.198.163.10 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=intel.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b="BA91iDZy" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1723072418; x=1754608418; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=+sgRE3ATJn+TS8TiBRcbN9sQCHaISZPkexzHT2Sa89Y=; b=BA91iDZydrKCf1yMXnxvs2LHUlLLYLAHgKslz8LxLkjpozb3dis572nS 8ej6vs6Ao3rU7kdEED7Cagu60kLZHu0WmUqQqiob+M5TwkTzu/jjK9GFd ouS7R+X+cnIevHL6A0qgnnEq6frY5vyNSzZOYRBc5zt0XrvH+tsESknnW ASAWd40dbQauEHnR0CCenl4PqLefn6EcazoELG0g54PTo81ljiEq6gYb2 Y7T4SAWuja4sBhN/pJdWpPQBbODSAwEhu7cgLK1Juu4PVCobY10njxjbX sjzSx4PiMGF4FwnsRDZslu+8bCNHNSUcjU8OVpYukUluQ4Wpab7s+taW5 g==; X-CSE-ConnectionGUID: 8/uhVFuJRrmYcDw8CA8UuQ== X-CSE-MsgGUID: R/k56QpvQ0CHT3MDh0zxRg== X-IronPort-AV: E=McAfee;i="6700,10204,11157"; a="32577313" X-IronPort-AV: E=Sophos;i="6.09,271,1716274800"; d="scan'208";a="32577313" Received: from orviesa004.jf.intel.com ([10.64.159.144]) by fmvoesa104.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 07 Aug 2024 16:13:34 -0700 X-CSE-ConnectionGUID: FtMIhrW1T0yNHB7F7FGZ5A== X-CSE-MsgGUID: EFq5rzRIRYWTOtvnBJJYZw== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.09,271,1716274800"; d="scan'208";a="61956625" Received: from anguy11-upstream.jf.intel.com ([10.166.9.133]) by orviesa004.jf.intel.com with ESMTP; 07 Aug 2024 16:13:34 -0700 From: Tony Nguyen To: davem@davemloft.net, kuba@kernel.org, pabeni@redhat.com, edumazet@google.com, netdev@vger.kernel.org Cc: Faizal Rahim , anthony.l.nguyen@intel.com, vinicius.gomes@intel.com, sasha.neftin@intel.com, richardcochran@gmail.com, horms@kernel.org, rodrigo.cadore@l-acoustics.com, Mor Bar-Gabay Subject: [PATCH net 3/4] igc: Fix reset adapter logics when tx mode change Date: Wed, 7 Aug 2024 16:13:27 -0700 Message-ID: <20240807231329.3827092-4-anthony.l.nguyen@intel.com> X-Mailer: git-send-email 2.42.0 In-Reply-To: <20240807231329.3827092-1-anthony.l.nguyen@intel.com> References: <20240807231329.3827092-1-anthony.l.nguyen@intel.com> Precedence: bulk X-Mailing-List: netdev@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Patchwork-Delegate: kuba@kernel.org From: Faizal Rahim Following the "igc: Fix TX Hang issue when QBV Gate is close" changes, remaining issues with the reset adapter logic in igc_tsn_offload_apply() have been observed: 1. The reset adapter logics for i225 and i226 differ, although they should be the same according to the guidelines in I225/6 HW Design Section 7.5.2.1 on software initialization during tx mode changes. 2. The i225 resets adapter every time, even though tx mode doesn't change. This occurs solely based on the condition igc_is_device_id_i225() when calling schedule_work(). 3. i226 doesn't reset adapter for tsn->legacy tx mode changes. It only resets adapter for legacy->tsn tx mode transitions. 4. qbv_count introduced in the patch is actually not needed; in this context, a non-zero value of qbv_count is used to indicate if tx mode was unconditionally set to tsn in igc_tsn_enable_offload(). This could be replaced by checking the existing register IGC_TQAVCTRL_TRANSMIT_MODE_TSN bit. This patch resolves all issues and enters schedule_work() to reset the adapter only when changing tx mode. It also removes reliance on qbv_count. qbv_count field will be removed in a future patch. Test ran: 1. Verify reset adapter behaviour in i225/6: a) Enrol a new GCL Reset adapter observed (tx mode change legacy->tsn) b) Enrol a new GCL without deleting qdisc No reset adapter observed (tx mode remain tsn->tsn) c) Delete qdisc Reset adapter observed (tx mode change tsn->legacy) 2. Tested scenario from "igc: Fix TX Hang issue when QBV Gate is closed" to confirm it remains resolved. Fixes: 175c241288c0 ("igc: Fix TX Hang issue when QBV Gate is closed") Signed-off-by: Faizal Rahim Reviewed-by: Simon Horman Acked-by: Vinicius Costa Gomes Tested-by: Mor Bar-Gabay Signed-off-by: Tony Nguyen --- drivers/net/ethernet/intel/igc/igc_tsn.c | 24 +++++++++++++++++++----- 1 file changed, 19 insertions(+), 5 deletions(-) diff --git a/drivers/net/ethernet/intel/igc/igc_tsn.c b/drivers/net/ethernet/intel/igc/igc_tsn.c index 8ed7b965484d..ada751430517 100644 --- a/drivers/net/ethernet/intel/igc/igc_tsn.c +++ b/drivers/net/ethernet/intel/igc/igc_tsn.c @@ -49,6 +49,13 @@ static unsigned int igc_tsn_new_flags(struct igc_adapter *adapter) return new_flags; } +static bool igc_tsn_is_tx_mode_in_tsn(struct igc_adapter *adapter) +{ + struct igc_hw *hw = &adapter->hw; + + return !!(rd32(IGC_TQAVCTRL) & IGC_TQAVCTRL_TRANSMIT_MODE_TSN); +} + void igc_tsn_adjust_txtime_offset(struct igc_adapter *adapter) { struct igc_hw *hw = &adapter->hw; @@ -365,15 +372,22 @@ int igc_tsn_reset(struct igc_adapter *adapter) return err; } -int igc_tsn_offload_apply(struct igc_adapter *adapter) +static bool igc_tsn_will_tx_mode_change(struct igc_adapter *adapter) { - struct igc_hw *hw = &adapter->hw; + bool any_tsn_enabled = !!(igc_tsn_new_flags(adapter) & + IGC_FLAG_TSN_ANY_ENABLED); + + return (any_tsn_enabled && !igc_tsn_is_tx_mode_in_tsn(adapter)) || + (!any_tsn_enabled && igc_tsn_is_tx_mode_in_tsn(adapter)); +} - /* Per I225/6 HW Design Section 7.5.2.1, transmit mode - * cannot be changed dynamically. Require reset the adapter. +int igc_tsn_offload_apply(struct igc_adapter *adapter) +{ + /* Per I225/6 HW Design Section 7.5.2.1 guideline, if tx mode change + * from legacy->tsn or tsn->legacy, then reset adapter is needed. */ if (netif_running(adapter->netdev) && - (igc_is_device_id_i225(hw) || !adapter->qbv_count)) { + igc_tsn_will_tx_mode_change(adapter)) { schedule_work(&adapter->reset_task); return 0; } From patchwork Wed Aug 7 23:13:28 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Tony Nguyen X-Patchwork-Id: 13756841 X-Patchwork-Delegate: kuba@kernel.org Received: from mgamail.intel.com (mgamail.intel.com [192.198.163.10]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 3731D155A4E for ; Wed, 7 Aug 2024 23:13:37 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=192.198.163.10 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1723072419; cv=none; b=HOUemdU/doYYALOQfenervNMmw2LO7BQmQjxMkorQUon4g7rT6EnzItvXeXeKhgZ00E6Mk2mG9ehQSpOTyCsZ4zhc8dNnTkupy2C5DZFP4FAVprw6XpDdKiT77xN7wAo9D0NKriX4lsxJbHRj7q2d3M92FVZt6cPn3cLuT+G3Jc= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1723072419; c=relaxed/simple; bh=s8Gpoq+a1FXQpGV0dffXHi9wn93/tOl3XrZZ5RFQ9xo=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=B8pp4JZkTwjB2j5Ko3NLvLjH4A4py5uEfXp0HLx2obohwvan5NXtzrIRjyzVcg5PTQ4HZi2lJQVwU3nTzklkVAQf+wAElwMZe/0J7GQcViTtneOETDA7JAh/OfsRlRFS6MNTF3y+Xpl4uk9uE9ye8NATvA7seEbxmFVVnktu1c4= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com; spf=pass smtp.mailfrom=intel.com; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b=OzsX+1dW; arc=none smtp.client-ip=192.198.163.10 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=intel.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b="OzsX+1dW" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1723072418; x=1754608418; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=s8Gpoq+a1FXQpGV0dffXHi9wn93/tOl3XrZZ5RFQ9xo=; b=OzsX+1dWTSu5INFdoK7PtXAeFyPZFCOMDpcXPXuwLXwtwxFcmSTbRqPU WlkDmA+vT8Sxf8GAuLb1hp7yCNyxfLjvRJrxhvEu8sVd3RaibtB28hgne C9Lhy1Ahixg9NoZPYpy6W+zwhxhp5eQXpF53k7CuA7ZvLSE59cVbuhfW9 8NHgwAhM/8YZL0TDCXgzsyLuUftweDkHI2Y5Y+SiaCZMmJIeYDk1nDRJ+ nFOKu2ELTZ/E1Y3QxP/kUAttciKBR367rO/eToPEMoTe9xjCPaqJf5t0n FjYl4HI2vRgRzmdiIoU+MNiK7efoFHfP3AJh7Qh4VJCvsI9ZO1zM0kDQc Q==; X-CSE-ConnectionGUID: eAtLRVhwR06/Bpr0628pgg== X-CSE-MsgGUID: z2tACQQdRYmJJxqwlNLHxA== X-IronPort-AV: E=McAfee;i="6700,10204,11157"; a="32577320" X-IronPort-AV: E=Sophos;i="6.09,271,1716274800"; d="scan'208";a="32577320" Received: from orviesa004.jf.intel.com ([10.64.159.144]) by fmvoesa104.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 07 Aug 2024 16:13:34 -0700 X-CSE-ConnectionGUID: NufDwa/1TQauKYTm/Slu0A== X-CSE-MsgGUID: wEfcjE98QH6ue/Jzv2jOTQ== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.09,271,1716274800"; d="scan'208";a="61956629" Received: from anguy11-upstream.jf.intel.com ([10.166.9.133]) by orviesa004.jf.intel.com with ESMTP; 07 Aug 2024 16:13:34 -0700 From: Tony Nguyen To: davem@davemloft.net, kuba@kernel.org, pabeni@redhat.com, edumazet@google.com, netdev@vger.kernel.org Cc: Faizal Rahim , anthony.l.nguyen@intel.com, vinicius.gomes@intel.com, sasha.neftin@intel.com, richardcochran@gmail.com, horms@kernel.org, rodrigo.cadore@l-acoustics.com, Mor Bar-Gabay Subject: [PATCH net 4/4] igc: Fix qbv tx latency by setting gtxoffset Date: Wed, 7 Aug 2024 16:13:28 -0700 Message-ID: <20240807231329.3827092-5-anthony.l.nguyen@intel.com> X-Mailer: git-send-email 2.42.0 In-Reply-To: <20240807231329.3827092-1-anthony.l.nguyen@intel.com> References: <20240807231329.3827092-1-anthony.l.nguyen@intel.com> Precedence: bulk X-Mailing-List: netdev@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Patchwork-Delegate: kuba@kernel.org From: Faizal Rahim A large tx latency issue was discovered during testing when only QBV was enabled. The issue occurs because gtxoffset was not set when QBV is active, it was only set when launch time is active. The patch "igc: Correct the launchtime offset" only sets gtxoffset when the launchtime_enable field is set by the user. Enabling launchtime_enable ultimately sets the register IGC_TXQCTL_QUEUE_MODE_LAUNCHT (referred to as LaunchT in the SW user manual). Section 7.5.2.6 of the IGC i225/6 SW User Manual Rev 1.2.4 states: "The latency between transmission scheduling (launch time) and the time the packet is transmitted to the network is listed in Table 7-61." However, the patch misinterprets the phrase "launch time" in that section by assuming it specifically refers to the LaunchT register, whereas it actually denotes the generic term for when a packet is released from the internal buffer to the MAC transmit logic. This launch time, as per that section, also implicitly refers to the QBV gate open time, where a packet waits in the buffer for the QBV gate to open. Therefore, latency applies whenever QBV is in use. TSN features such as QBU and QAV reuse QBV, making the latency universal to TSN features. Discussed with i226 HW owner (Shalev, Avi) and we were in agreement that the term "launch time" used in Section 7.5.2.6 is not clear and can be easily misinterpreted. Avi will update this section to: "When TQAVCTRL.TRANSMIT_MODE = TSN, the latency between transmission scheduling and the time the packet is transmitted to the network is listed in Table 7-61." Fix this issue by using igc_tsn_is_tx_mode_in_tsn() as a condition to write to gtxoffset, aligning with the newly updated SW User Manual. Tested: 1. Enrol taprio on talker board base-time 0 cycle-time 1000000 flags 0x2 index 0 cmd S gatemask 0x1 interval1 index 0 cmd S gatemask 0x1 interval2 Note: interval1 = interval for a 64 bytes packet to go through interval2 = cycle-time - interval1 2. Take tcpdump on listener board 3. Use udp tai app on talker to send packets to listener 4. Check the timestamp on listener via wireshark Test Result: 100 Mbps: 113 ~193 ns 1000 Mbps: 52 ~ 84 ns 2500 Mbps: 95 ~ 223 ns Note that the test result is similar to the patch "igc: Correct the launchtime offset". Fixes: 790835fcc0cb ("igc: Correct the launchtime offset") Signed-off-by: Faizal Rahim Reviewed-by: Simon Horman Acked-by: Vinicius Costa Gomes Tested-by: Mor Bar-Gabay Signed-off-by: Tony Nguyen --- drivers/net/ethernet/intel/igc/igc_tsn.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/drivers/net/ethernet/intel/igc/igc_tsn.c b/drivers/net/ethernet/intel/igc/igc_tsn.c index ada751430517..d68fa7f3d5f0 100644 --- a/drivers/net/ethernet/intel/igc/igc_tsn.c +++ b/drivers/net/ethernet/intel/igc/igc_tsn.c @@ -61,7 +61,7 @@ void igc_tsn_adjust_txtime_offset(struct igc_adapter *adapter) struct igc_hw *hw = &adapter->hw; u16 txoffset; - if (!is_any_launchtime(adapter)) + if (!igc_tsn_is_tx_mode_in_tsn(adapter)) return; switch (adapter->link_speed) {