[net-next,0/2] tcp: even faster connect() under stress

Message

Eric Dumazet March 5, 2025, 3:45 a.m. UTC

This is a followup on the prior series, "tcp: scale connect() under pressure"

Now spinlocks are no longer in the picture, we see a very high cost
of the inet6_ehashfn() function.

In this series (of 2), I change how lport contributes to inet6_ehashfn()
to ensure better cache locality and call inet6_ehashfn()
only once per connect() system call.

This brings an additional 229 % increase of performance
for "neper/tcp_crr -6 -T 200 -F 30000" stress test,
while greatly improving latency metrics.

Before:
  latency_min=0.014131929
  latency_max=17.895073144
  latency_mean=0.505675853
  latency_stddev=2.125164772
  num_samples=307884
  throughput=139866.80

After:
  latency_min=0.003041375
  latency_max=7.056589232
  latency_mean=0.141075048
  latency_stddev=0.526900516
  num_samples=312996
  throughput=320677.21

Eric Dumazet (2):
  inet: change lport contribution to inet_ehashfn() and inet6_ehashfn()
  inet: call inet6_ehashfn() once from inet6_hash_connect()

 include/net/inet_hashtables.h |  4 +++-
 include/net/ip.h              |  2 +-
 net/ipv4/inet_hashtables.c    | 30 ++++++++++++++++++++----------
 net/ipv6/inet6_hashtables.c   | 19 +++++++++++++------
 4 files changed, 37 insertions(+), 18 deletions(-)

Comments

Eric Dumazet March 5, 2025, 4:01 a.m. UTC | #1

On Wed, Mar 5, 2025 at 4:45 AM Eric Dumazet <[e-mail]> wrote:
>
> This is a followup on the prior series, "tcp: scale connect() under pressure"
>
> Now spinlocks are no longer in the picture, we see a very high cost
> of the inet6_ehashfn() function.
>
> In this series (of 2), I change how lport contributes to inet6_ehashfn()
> to ensure better cache locality and call inet6_ehashfn()
> only once per connect() system call.
>
> This brings an additional 229 % increase of performance

This is 129 % additional QPS (going from 139866.80 to 320677.21)

Sorry for the confusion :)

[e-mail] March 6, 2025, 11:40 p.m. UTC | #2

Hello:

This series was applied to netdev/net-next.git (main)
by Jakub Kicinski <[e-mail]>:

On Wed,  5 Mar 2025 03:45:48 +0000 you wrote:
> This is a followup on the prior series, "tcp: scale connect() under pressure"
> 
> Now spinlocks are no longer in the picture, we see a very high cost
> of the inet6_ehashfn() function.
> 
> In this series (of 2), I change how lport contributes to inet6_ehashfn()
> to ensure better cache locality and call inet6_ehashfn()
> only once per connect() system call.
> 
> [...]

Here is the summary with links:
  - [net-next,1/2] inet: change lport contribution to inet_ehashfn() and inet6_ehashfn()
    https://git.kernel.org/netdev/net-next/c/9544d60a2605
  - [net-next,2/2] inet: call inet6_ehashfn() once from inet6_hash_connect()
    https://git.kernel.org/netdev/net-next/c/d4438ce68bf1

You are awesome, thank you!