| Message ID | 20250302124237.3913746-1-edumazet@google.com (mailing list archive) |
|---|---|
| Headers | show |
| Series | tcp: scale connect() under pressure | expand |
Hello: This series was applied to netdev/net-next.git (main) by Jakub Kicinski <kuba@kernel.org>: On Sun, 2 Mar 2025 12:42:33 +0000 you wrote: > Adoption of bhash2 in linux-6.1 made some operations almost twice > more expensive, because of additional locks. > > This series adds RCU in __inet_hash_connect() to help the > case where many attempts need to be made before finding > an available 4-tuple. > > [...] Here is the summary with links: - [net-next,1/4] tcp: use RCU in __inet{6}_check_established() https://git.kernel.org/netdev/net-next/c/ae9d5b19b322 - [net-next,2/4] tcp: optimize inet_use_bhash2_on_bind() https://git.kernel.org/netdev/net-next/c/ca79d80b0b9f - [net-next,3/4] tcp: add RCU management to inet_bind_bucket https://git.kernel.org/netdev/net-next/c/d186f405fdf4 - [net-next,4/4] tcp: use RCU lookup in __inet_hash_connect() https://git.kernel.org/netdev/net-next/c/86c2bc293b81 You are awesome, thank you!
Adoption of bhash2 in linux-6.1 made some operations almost twice more expensive, because of additional locks. This series adds RCU in __inet_hash_connect() to help the case where many attempts need to be made before finding an available 4-tuple. This brings a ~200 % improvement in this experiment: Server: ulimit -n 40000; neper/tcp_crr -T 200 -F 30000 -6 --nolog Client: ulimit -n 40000; neper/tcp_crr -T 200 -F 30000 -6 --nolog -c -H server Before series: utime_start=0.288582 utime_end=1.548707 stime_start=20.637138 stime_end=2002.489845 num_transactions=484453 latency_min=0.156279245 latency_max=20.922042756 latency_mean=1.546521274 latency_stddev=3.936005194 num_samples=312537 throughput=47426.00 perf top on the client: 49.54% [kernel] [k] _raw_spin_lock 25.87% [kernel] [k] _raw_spin_lock_bh 5.97% [kernel] [k] queued_spin_lock_slowpath 5.67% [kernel] [k] __inet_hash_connect 3.53% [kernel] [k] __inet6_check_established 3.48% [kernel] [k] inet6_ehashfn 0.64% [kernel] [k] rcu_all_qs After this series: utime_start=0.271607 utime_end=3.847111 stime_start=18.407684 stime_end=1997.485557 num_transactions=1350742 latency_min=0.014131929 latency_max=17.895073144 latency_mean=0.505675853 # Nice reduction of latency metrics latency_stddev=2.125164772 num_samples=307884 throughput=139866.80 # 194 % increase perf top on client: 56.86% [kernel] [k] __inet6_check_established 17.96% [kernel] [k] __inet_hash_connect 13.88% [kernel] [k] inet6_ehashfn 2.52% [kernel] [k] rcu_all_qs 2.01% [kernel] [k] __cond_resched 0.41% [kernel] [k] _raw_spin_lock Eric Dumazet (4): tcp: use RCU in __inet{6}_check_established() tcp: optimize inet_use_bhash2_on_bind() tcp: add RCU management to inet_bind_bucket tcp: use RCU lookup in __inet_hash_connect() include/net/inet_hashtables.h | 7 ++-- net/ipv4/inet_connection_sock.c | 8 ++-- net/ipv4/inet_hashtables.c | 65 ++++++++++++++++++++++++--------- net/ipv4/inet_timewait_sock.c | 2 +- net/ipv6/inet6_hashtables.c | 23 ++++++++++-- 5 files changed, 75 insertions(+), 30 deletions(-)