diff mbox series

[net-next,6/6] ice: Document tx_scheduling_layers parameter

Message ID 20240422203913.225151-7-anthony.l.nguyen@intel.com (mailing list archive)
State Accepted
Commit 9afff0de30db149a1bf440db26a3ddd6a4f260d8
Delegated to: Netdev Maintainers
Headers show
Series ice: Support 5 layer Tx scheduler topology | expand

Checks

Context Check Description
netdev/series_format success Posting correctly formatted
netdev/tree_selection success Clearly marked for net-next
netdev/ynl success Generated files up to date; no warnings/errors; no diff in generated;
netdev/fixes_present success Fixes tag not required for -next series
netdev/header_inline success No static functions without inline keyword in header files
netdev/build_32bit success Errors and warnings before: 8 this patch: 8
netdev/build_tools success No tools touched, skip
netdev/cc_maintainers success CCed 7 of 7 maintainers
netdev/build_clang success Errors and warnings before: 8 this patch: 8
netdev/verify_signedoff success Signed-off-by tag matches author and committer
netdev/deprecated_api success None detected
netdev/check_selftest success No net selftest shell script
netdev/verify_fixes success No Fixes tag
netdev/build_allmodconfig_warn success Errors and warnings before: 8 this patch: 8
netdev/checkpatch success total: 0 errors, 0 warnings, 0 checks, 53 lines checked
netdev/build_clang_rust success No Rust files in patch. Skipping build
netdev/kdoc success Errors and warnings before: 0 this patch: 0
netdev/source_inline success Was 0 now: 0
netdev/contest success net-next-2024-04-24--15-00 (tests: 995)

Commit Message

Tony Nguyen April 22, 2024, 8:39 p.m. UTC 
From: Michal Wilczynski <michal.wilczynski@intel.com>

New driver specific parameter 'tx_scheduling_layers' was introduced.
Describe parameter in the documentation.

Signed-off-by: Michal Wilczynski <michal.wilczynski@intel.com>
Acked-by: Jakub Kicinski <kuba@kernel.org>
Reviewed-by: Jiri Pirko <jiri@nvidia.com>
Reviewed-by: Przemek Kitszel <przemyslaw.kitszel@intel.com>
Co-developed-by: Mateusz Polchlopek <mateusz.polchlopek@intel.com>
Signed-off-by: Mateusz Polchlopek <mateusz.polchlopek@intel.com>
Signed-off-by: Tony Nguyen <anthony.l.nguyen@intel.com>
---
 Documentation/networking/devlink/ice.rst | 47 ++++++++++++++++++++++++
 1 file changed, 47 insertions(+)

Comments

Bagas Sanjaya April 23, 2024, 12:37 p.m. UTC | #1 
On Mon, Apr 22, 2024 at 01:39:11PM -0700, Tony Nguyen wrote:
> +       The default 9-layer tree topology was deemed best for most workloads,
> +       as it gives an optimal ratio of performance to configurability. However,
> +       for some specific cases, this 9-layer topology might not be desired.
> +       One example would be sending traffic to queues that are not a multiple
> +       of 8. Because the maximum radix is limited to 8 in 9-layer topology,
> +       the 9th queue has a different parent than the rest, and it's given
> +       more bandwidth credits. This causes a problem when the system is
> +       sending traffic to 9 queues:
> +
> +       | tx_queue_0_packets: 24163396
> +       | tx_queue_1_packets: 24164623
> +       | tx_queue_2_packets: 24163188
> +       | tx_queue_3_packets: 24163701
> +       | tx_queue_4_packets: 24163683
> +       | tx_queue_5_packets: 24164668
> +       | tx_queue_6_packets: 23327200
> +       | tx_queue_7_packets: 24163853
> +       | tx_queue_8_packets: 91101417 < Too much traffic is sent from 9th
> +
> <snipped>...
> +       To verify that value has been set:
> +       $ devlink dev param show pci/0000:16:00.0 name tx_scheduling_layers
>  

For consistency with other code blocks, format above as such:

---- >8 ----
diff --git a/Documentation/networking/devlink/ice.rst b/Documentation/networking/devlink/ice.rst
index 830c04354222f8..0039ca45782400 100644
--- a/Documentation/networking/devlink/ice.rst
+++ b/Documentation/networking/devlink/ice.rst
@@ -41,15 +41,17 @@ Parameters
        more bandwidth credits. This causes a problem when the system is
        sending traffic to 9 queues:
 
-       | tx_queue_0_packets: 24163396
-       | tx_queue_1_packets: 24164623
-       | tx_queue_2_packets: 24163188
-       | tx_queue_3_packets: 24163701
-       | tx_queue_4_packets: 24163683
-       | tx_queue_5_packets: 24164668
-       | tx_queue_6_packets: 23327200
-       | tx_queue_7_packets: 24163853
-       | tx_queue_8_packets: 91101417 < Too much traffic is sent from 9th
+       .. code-block:: shell
+
+         tx_queue_0_packets: 24163396
+         tx_queue_1_packets: 24164623
+         tx_queue_2_packets: 24163188
+         tx_queue_3_packets: 24163701
+         tx_queue_4_packets: 24163683
+         tx_queue_5_packets: 24164668
+         tx_queue_6_packets: 23327200
+         tx_queue_7_packets: 24163853
+         tx_queue_8_packets: 91101417 < Too much traffic is sent from 9th
 
        To address this need, you can switch to a 5-layer topology, which
        changes the maximum topology radix to 512. With this enhancement,
@@ -67,7 +69,10 @@ Parameters
        You must do PCI slot powercycle for the selected topology to take effect.
 
        To verify that value has been set:
-       $ devlink dev param show pci/0000:16:00.0 name tx_scheduling_layers
+
+       .. code-block:: shell
+
+         $ devlink dev param show pci/0000:16:00.0 name tx_scheduling_layers
 
 Info versions
 =============

Thanks.
Mateusz Polchlopek April 24, 2024, 9:54 a.m. UTC | #2 
On 4/23/2024 2:37 PM, Bagas Sanjaya wrote:
> On Mon, Apr 22, 2024 at 01:39:11PM -0700, Tony Nguyen wrote:
>> +       The default 9-layer tree topology was deemed best for most workloads,
>> +       as it gives an optimal ratio of performance to configurability. However,
>> +       for some specific cases, this 9-layer topology might not be desired.
>> +       One example would be sending traffic to queues that are not a multiple
>> +       of 8. Because the maximum radix is limited to 8 in 9-layer topology,
>> +       the 9th queue has a different parent than the rest, and it's given
>> +       more bandwidth credits. This causes a problem when the system is
>> +       sending traffic to 9 queues:
>> +
>> +       | tx_queue_0_packets: 24163396
>> +       | tx_queue_1_packets: 24164623
>> +       | tx_queue_2_packets: 24163188
>> +       | tx_queue_3_packets: 24163701
>> +       | tx_queue_4_packets: 24163683
>> +       | tx_queue_5_packets: 24164668
>> +       | tx_queue_6_packets: 23327200
>> +       | tx_queue_7_packets: 24163853
>> +       | tx_queue_8_packets: 91101417 < Too much traffic is sent from 9th
>> +
>> <snipped>...
>> +       To verify that value has been set:
>> +       $ devlink dev param show pci/0000:16:00.0 name tx_scheduling_layers
>>   
> 
> For consistency with other code blocks, format above as such:
> 
> ---- >8 ----
> diff --git a/Documentation/networking/devlink/ice.rst b/Documentation/networking/devlink/ice.rst
> index 830c04354222f8..0039ca45782400 100644
> --- a/Documentation/networking/devlink/ice.rst
> +++ b/Documentation/networking/devlink/ice.rst
> @@ -41,15 +41,17 @@ Parameters
>          more bandwidth credits. This causes a problem when the system is
>          sending traffic to 9 queues:
>   
> -       | tx_queue_0_packets: 24163396
> -       | tx_queue_1_packets: 24164623
> -       | tx_queue_2_packets: 24163188
> -       | tx_queue_3_packets: 24163701
> -       | tx_queue_4_packets: 24163683
> -       | tx_queue_5_packets: 24164668
> -       | tx_queue_6_packets: 23327200
> -       | tx_queue_7_packets: 24163853
> -       | tx_queue_8_packets: 91101417 < Too much traffic is sent from 9th
> +       .. code-block:: shell
> +
> +         tx_queue_0_packets: 24163396
> +         tx_queue_1_packets: 24164623
> +         tx_queue_2_packets: 24163188
> +         tx_queue_3_packets: 24163701
> +         tx_queue_4_packets: 24163683
> +         tx_queue_5_packets: 24164668
> +         tx_queue_6_packets: 23327200
> +         tx_queue_7_packets: 24163853
> +         tx_queue_8_packets: 91101417 < Too much traffic is sent from 9th
>   
>          To address this need, you can switch to a 5-layer topology, which
>          changes the maximum topology radix to 512. With this enhancement,
> @@ -67,7 +69,10 @@ Parameters
>          You must do PCI slot powercycle for the selected topology to take effect.
>   
>          To verify that value has been set:
> -       $ devlink dev param show pci/0000:16:00.0 name tx_scheduling_layers
> +
> +       .. code-block:: shell
> +
> +         $ devlink dev param show pci/0000:16:00.0 name tx_scheduling_layers
>   
>   Info versions
>   =============
> 
> Thanks.
> 

Thank You for reporting that. I will verify this issue soon.
Bagas Sanjaya April 24, 2024, 10:47 a.m. UTC | #3 
On 4/24/24 16:54, Mateusz Polchlopek wrote:
> 
> 
> On 4/23/2024 2:37 PM, Bagas Sanjaya wrote:
>> On Mon, Apr 22, 2024 at 01:39:11PM -0700, Tony Nguyen wrote:
>>> +       The default 9-layer tree topology was deemed best for most workloads,
>>> +       as it gives an optimal ratio of performance to configurability. However,
>>> +       for some specific cases, this 9-layer topology might not be desired.
>>> +       One example would be sending traffic to queues that are not a multiple
>>> +       of 8. Because the maximum radix is limited to 8 in 9-layer topology,
>>> +       the 9th queue has a different parent than the rest, and it's given
>>> +       more bandwidth credits. This causes a problem when the system is
>>> +       sending traffic to 9 queues:
>>> +
>>> +       | tx_queue_0_packets: 24163396
>>> +       | tx_queue_1_packets: 24164623
>>> +       | tx_queue_2_packets: 24163188
>>> +       | tx_queue_3_packets: 24163701
>>> +       | tx_queue_4_packets: 24163683
>>> +       | tx_queue_5_packets: 24164668
>>> +       | tx_queue_6_packets: 23327200
>>> +       | tx_queue_7_packets: 24163853
>>> +       | tx_queue_8_packets: 91101417 < Too much traffic is sent from 9th
>>> +
>>> <snipped>...
>>> +       To verify that value has been set:
>>> +       $ devlink dev param show pci/0000:16:00.0 name tx_scheduling_layers
>>>   
>>
>> For consistency with other code blocks, format above as such:
>>
>> ---- >8 ----
>> diff --git a/Documentation/networking/devlink/ice.rst b/Documentation/networking/devlink/ice.rst
>> index 830c04354222f8..0039ca45782400 100644
>> --- a/Documentation/networking/devlink/ice.rst
>> +++ b/Documentation/networking/devlink/ice.rst
>> @@ -41,15 +41,17 @@ Parameters
>>          more bandwidth credits. This causes a problem when the system is
>>          sending traffic to 9 queues:
>>   -       | tx_queue_0_packets: 24163396
>> -       | tx_queue_1_packets: 24164623
>> -       | tx_queue_2_packets: 24163188
>> -       | tx_queue_3_packets: 24163701
>> -       | tx_queue_4_packets: 24163683
>> -       | tx_queue_5_packets: 24164668
>> -       | tx_queue_6_packets: 23327200
>> -       | tx_queue_7_packets: 24163853
>> -       | tx_queue_8_packets: 91101417 < Too much traffic is sent from 9th
>> +       .. code-block:: shell
>> +
>> +         tx_queue_0_packets: 24163396
>> +         tx_queue_1_packets: 24164623
>> +         tx_queue_2_packets: 24163188
>> +         tx_queue_3_packets: 24163701
>> +         tx_queue_4_packets: 24163683
>> +         tx_queue_5_packets: 24164668
>> +         tx_queue_6_packets: 23327200
>> +         tx_queue_7_packets: 24163853
>> +         tx_queue_8_packets: 91101417 < Too much traffic is sent from 9th
>>            To address this need, you can switch to a 5-layer topology, which
>>          changes the maximum topology radix to 512. With this enhancement,
>> @@ -67,7 +69,10 @@ Parameters
>>          You must do PCI slot powercycle for the selected topology to take effect.
>>            To verify that value has been set:
>> -       $ devlink dev param show pci/0000:16:00.0 name tx_scheduling_layers
>> +
>> +       .. code-block:: shell
>> +
>> +         $ devlink dev param show pci/0000:16:00.0 name tx_scheduling_layers
>>     Info versions
>>   =============
>>
>> Thanks.
>>
> 
> Thank You for reporting that. I will verify this issue soon.

OK, thanks!
diff mbox series

Patch

diff --git a/Documentation/networking/devlink/ice.rst b/Documentation/networking/devlink/ice.rst
index 7f30ebd5debb..830c04354222 100644
--- a/Documentation/networking/devlink/ice.rst
+++ b/Documentation/networking/devlink/ice.rst
@@ -21,6 +21,53 @@  Parameters
    * - ``enable_iwarp``
      - runtime
      - mutually exclusive with ``enable_roce``
+   * - ``tx_scheduling_layers``
+     - permanent
+     - The ice hardware uses hierarchical scheduling for Tx with a fixed
+       number of layers in the scheduling tree. Each of them are decision
+       points. Root node represents a port, while all the leaves represent
+       the queues. This way of configuring the Tx scheduler allows features
+       like DCB or devlink-rate (documented below) to configure how much
+       bandwidth is given to any given queue or group of queues, enabling
+       fine-grained control because scheduling parameters can be configured
+       at any given layer of the tree.
+
+       The default 9-layer tree topology was deemed best for most workloads,
+       as it gives an optimal ratio of performance to configurability. However,
+       for some specific cases, this 9-layer topology might not be desired.
+       One example would be sending traffic to queues that are not a multiple
+       of 8. Because the maximum radix is limited to 8 in 9-layer topology,
+       the 9th queue has a different parent than the rest, and it's given
+       more bandwidth credits. This causes a problem when the system is
+       sending traffic to 9 queues:
+
+       | tx_queue_0_packets: 24163396
+       | tx_queue_1_packets: 24164623
+       | tx_queue_2_packets: 24163188
+       | tx_queue_3_packets: 24163701
+       | tx_queue_4_packets: 24163683
+       | tx_queue_5_packets: 24164668
+       | tx_queue_6_packets: 23327200
+       | tx_queue_7_packets: 24163853
+       | tx_queue_8_packets: 91101417 < Too much traffic is sent from 9th
+
+       To address this need, you can switch to a 5-layer topology, which
+       changes the maximum topology radix to 512. With this enhancement,
+       the performance characteristic is equal as all queues can be assigned
+       to the same parent in the tree. The obvious drawback of this solution
+       is a lower configuration depth of the tree.
+
+       Use the ``tx_scheduling_layer`` parameter with the devlink command
+       to change the transmit scheduler topology. To use 5-layer topology,
+       use a value of 5. For example:
+       $ devlink dev param set pci/0000:16:00.0 name tx_scheduling_layers
+       value 5 cmode permanent
+       Use a value of 9 to set it back to the default value.
+
+       You must do PCI slot powercycle for the selected topology to take effect.
+
+       To verify that value has been set:
+       $ devlink dev param show pci/0000:16:00.0 name tx_scheduling_layers
 
 Info versions
 =============