[net-next,0/1] net/smc: Optimize rmbs/sndbufs lock

Message

liqiang Nov. 18, 2024, 1:21 p.m. UTC

This patch changes the global lock used by the buf_desc linked list
array in the smc link group to a smaller granularity, which can
reduce competition between links using different bufsizes and make
them perform better.

After applying this patch, the main scenarios that generate benefits
are as follows: multiple threads use different socket buffers
(sk->sk_sndbuf/sk_rcvbuf) to establish connections concurrently.

I constructed the above scenario and compared the performance of
socket buffer distribution in 4 sizes when multi-threaded in
parallel (tested using nginx/wrk):

On server:
smc_run nginx

On client:
smc_run wrk -t <2~128> -c 200 -H "Connection: close" http://127.0.0.1

+-------------------+--------+--------+--------+--------+--------+~
|conns/qps          |  -t 2  | -t 4   | -t 8   | -t 16  | -t 32  |
+-------------------+--------+--------+--------+--------+--------+~
|loopback-ism origin|6824.01 |9011.71 |11571.07|12179.72|11576.88|
+-------------------+--------+--------+--------+--------+--------+~
|loopback-ism after |7280.63 |9508.53 |13173.27|16368.93|14664.51|
+-------------------+--------+--------+--------+--------+--------+~

~--------+--------+
  -t 64  | -t 128 |
~--------+--------+
 11080.98|11909.36|
~--------+--------+
 14664.51|13112.89|
~--------+--------+

Test environment:
QEMU emulator version 1.5.3 @ Intel(R) Xeon(R) CPU E5-2680 v4 @ 2.40GHz

liqiang (1):
  Separate locks for rmbs/sndbufs linked lists of different lengths

 net/smc/smc_core.c | 66 +++++++++++++++++++-------------------
 net/smc/smc_core.h |  9 +++---
 net/smc/smc_llc.c  | 79 +++++++++++++++++++++++++++++++++++++---------
 3 files changed, 103 insertions(+), 51 deletions(-)