| Message ID | 20240408233200.1701282-1-hli@netflix.com (mailing list archive) |
|---|---|
| State | Superseded |
| Delegated to: | Netdev Maintainers |
| Headers | show |
| Series | [net-next,v2] tcp: increase the default TCP scaling ratio | expand |
On Tue, Apr 9, 2024 at 1:32 AM Hechao Li <hli@netflix.com> wrote: > > After commit dfa2f0483360 ("tcp: get rid of sysctl_tcp_adv_win_scale"), > we noticed an application-level timeout due to reduced throughput. > > Before the commit, for a client that sets SO_RCVBUF to 65k, it takes > around 22 seconds to transfer 10M data. After the commit, it takes 40 > seconds. Because our application has a 30-second timeout, this > regression broke the application. > > The reason that it takes longer to transfer data is that > tp->scaling_ratio is initialized to a value that results in ~0.25 of > rcvbuf. In our case, SO_RCVBUF is set to 65536 by the application, which > translates to 2 * 65536 = 131,072 bytes in rcvbuf and hence a ~28k > initial receive window. > > Later, even though the scaling_ratio is updated to a more accurate > skb->len/skb->truesize, which is ~0.66 in our environment, the window > stays at ~0.25 * rcvbuf. This is because tp->window_clamp does not > change together with the tp->scaling_ratio update. < when autotuning is disabled because of SO_RCVBUF > Most modern applications let the kernel do autotuning, and benefit from the increased scaling_ratio. As a result, the > window size is capped at the initial window_clamp, which is also ~0.25 * > rcvbuf, and never grows bigger. > > This patch increases the initial scaling_ratio from ~25% to 50% in order > to be backward compatible with the original default > sysctl_tcp_adv_win_scale. > > Fixes: dfa2f0483360 ("tcp: get rid of sysctl_tcp_adv_win_scale") > Signed-off-by: Hechao Li <hli@netflix.com> > Reviewed-by: Tycho Andersen <tycho@tycho.pizza>, Eric Dumazet <edumazet@google.com> This tag is not standard, please use one line per reviewer. Also please include the link to V1, for further reference. Reviewed-by: Eric Dumazet <edumazet@google.com> Link: https://lore.kernel.org/netdev/20240402215405.432863-1-hli@netflix.com/ Thanks.
diff --git a/include/net/tcp.h b/include/net/tcp.h index 6ae35199d3b3..2bcf30381d75 100644 --- a/include/net/tcp.h +++ b/include/net/tcp.h @@ -1539,11 +1539,10 @@ static inline int tcp_space_from_win(const struct sock *sk, int win) return __tcp_space_from_win(tcp_sk(sk)->scaling_ratio, win); } -/* Assume a conservative default of 1200 bytes of payload per 4K page. +/* Assume a 50% default for skb->len/skb->truesize ratio. * This may be adjusted later in tcp_measure_rcv_mss(). */ -#define TCP_DEFAULT_SCALING_RATIO ((1200 << TCP_RMEM_TO_WIN_SCALE) / \ - SKB_TRUESIZE(4096)) +#define TCP_DEFAULT_SCALING_RATIO (1 << (TCP_RMEM_TO_WIN_SCALE - 1)) static inline void tcp_scaling_ratio_init(struct sock *sk) {
After commit dfa2f0483360 ("tcp: get rid of sysctl_tcp_adv_win_scale"), we noticed an application-level timeout due to reduced throughput. Before the commit, for a client that sets SO_RCVBUF to 65k, it takes around 22 seconds to transfer 10M data. After the commit, it takes 40 seconds. Because our application has a 30-second timeout, this regression broke the application. The reason that it takes longer to transfer data is that tp->scaling_ratio is initialized to a value that results in ~0.25 of rcvbuf. In our case, SO_RCVBUF is set to 65536 by the application, which translates to 2 * 65536 = 131,072 bytes in rcvbuf and hence a ~28k initial receive window. Later, even though the scaling_ratio is updated to a more accurate skb->len/skb->truesize, which is ~0.66 in our environment, the window stays at ~0.25 * rcvbuf. This is because tp->window_clamp does not change together with the tp->scaling_ratio update. As a result, the window size is capped at the initial window_clamp, which is also ~0.25 * rcvbuf, and never grows bigger. This patch increases the initial scaling_ratio from ~25% to 50% in order to be backward compatible with the original default sysctl_tcp_adv_win_scale. Fixes: dfa2f0483360 ("tcp: get rid of sysctl_tcp_adv_win_scale") Signed-off-by: Hechao Li <hli@netflix.com> Reviewed-by: Tycho Andersen <tycho@tycho.pizza>, Eric Dumazet <edumazet@google.com> --- v1->v2: increase the default tcp scaling ratio instead of updating window_clamp and update the commit message --- include/net/tcp.h | 5 ++--- 1 file changed, 2 insertions(+), 3 deletions(-)