| Message ID | 20230807134547.2782227-4-imagedong@tencent.com (mailing list archive) |
|---|---|
| State | Superseded |
| Delegated to: | Netdev Maintainers |
| Headers | show |
| Series | net: tcp: support probing OOM | expand |
On Mon, Aug 7, 2023 at 3:47 PM <menglong8.dong@gmail.com> wrote: > > From: Menglong Dong <imagedong@tencent.com> > > In tcp_retransmit_timer(), a window shrunk connection will be regarded > as timeout if 'tcp_jiffies32 - tp->rcv_tstamp > TCP_RTO_MAX'. This is not > right all the time. > > The retransmits will become zero-window probes in tcp_retransmit_timer() > if the 'snd_wnd==0'. Therefore, the icsk->icsk_rto will come up to > TCP_RTO_MAX sooner or later. > > However, the timer is not precise enough, as it base on timer wheel. > Sorry that I am not good at timer, but I know the concept of time-wheel. Can you remove this line, can we keep focused on the actual patch instead ? Regardless of timer-wheel, a timer can be delayed under load. > The longer of the timer, the rougher it will be. So the timeout is not > triggered after TCP_RTO_MAX, but 122877ms as I tested. > > Therefore, 'tcp_jiffies32 - tp->rcv_tstamp > TCP_RTO_MAX' is always true > once the RTO come up to TCP_RTO_MAX, and the socket will die. > > Fix this by replacing the 'tcp_jiffies32' with '(u32)icsk->icsk_timeout', > which is exact the timestamp of the timeout. Meanwhile, using > "max(tp->retrans_stamp, tp->rcv_tstamp)" as the last updated timestamp in > the receiving path, as "tp->rcv_tstamp" can restart from idle, then > tp->rcv_tstamp could already be a long time (minutes or hours) in the > past even on the first RTO. > > Fixes: 1da177e4c3f4 ("Linux-2.6.12-rc2") > Link: https://lore.kernel.org/netdev/CADxym3YyMiO+zMD4zj03YPM3FBi-1LHi6gSD2XT8pyAMM096pg@mail.gmail.com/ > Signed-off-by: Menglong Dong <imagedong@tencent.com> > --- > v2: > - consider the case of the connection restart from idle, as Neal comment > --- > net/ipv4/tcp_timer.c | 10 +++++++++- > 1 file changed, 9 insertions(+), 1 deletion(-) > > diff --git a/net/ipv4/tcp_timer.c b/net/ipv4/tcp_timer.c > index d45c96c7f5a4..e4b2d8706cae 100644 > --- a/net/ipv4/tcp_timer.c > +++ b/net/ipv4/tcp_timer.c > @@ -454,6 +454,14 @@ static void tcp_fastopen_synack_timer(struct sock *sk, struct request_sock *req) > req->timeout << req->num_timeout, TCP_RTO_MAX); > } > > +static bool tcp_rtx_probe0_timed_out(struct sock *sk) > +{ > + struct tcp_sock *tp = tcp_sk(sk); > + u32 last_ts; > + > + last_ts = max(tp->retrans_stamp, tp->rcv_tstamp); u32 retrans_stamp; /* Timestamp of the last retransmit, u32 rcv_tstamp; /* timestamp of last received ACK (for keepalives) */ Both fields receive tcp_jiffies32 values, which wrap every 2^32 ticks. So max(A, B) won't work as you expect... You need to use before(), after() or something like that. https://en.wikipedia.org/wiki/Serial_number_arithmetic#General_Solution > + return inet_csk(sk)->icsk_timeout - last_ts > TCP_RTO_MAX; > +} > > /** > * tcp_retransmit_timer() - The TCP retransmit timeout handler > @@ -519,7 +527,7 @@ void tcp_retransmit_timer(struct sock *sk) > tp->snd_una, tp->snd_nxt); > } > #endif > - if (tcp_jiffies32 - tp->rcv_tstamp > TCP_RTO_MAX) { > + if (tcp_rtx_probe0_timed_out(sk)) { > tcp_write_err(sk); > goto out; > } > -- > 2.40.1 >
On Mon, Aug 7, 2023 at 10:20 PM Eric Dumazet <edumazet@google.com> wrote: > > On Mon, Aug 7, 2023 at 3:47 PM <menglong8.dong@gmail.com> wrote: > > > > From: Menglong Dong <imagedong@tencent.com> > > > > In tcp_retransmit_timer(), a window shrunk connection will be regarded > > as timeout if 'tcp_jiffies32 - tp->rcv_tstamp > TCP_RTO_MAX'. This is not > > right all the time. > > > > The retransmits will become zero-window probes in tcp_retransmit_timer() > > if the 'snd_wnd==0'. Therefore, the icsk->icsk_rto will come up to > > TCP_RTO_MAX sooner or later. > > > > However, the timer is not precise enough, as it base on timer wheel. > > > Sorry that I am not good at timer, but I know the concept of time-wheel. > > Can you remove this line, can we keep focused on the actual patch instead ? > > Regardless of timer-wheel, a timer can be delayed under load. > Okay! > > The longer of the timer, the rougher it will be. So the timeout is not > > triggered after TCP_RTO_MAX, but 122877ms as I tested. > > > > Therefore, 'tcp_jiffies32 - tp->rcv_tstamp > TCP_RTO_MAX' is always true > > once the RTO come up to TCP_RTO_MAX, and the socket will die. > > > > Fix this by replacing the 'tcp_jiffies32' with '(u32)icsk->icsk_timeout', > > which is exact the timestamp of the timeout. Meanwhile, using > > "max(tp->retrans_stamp, tp->rcv_tstamp)" as the last updated timestamp in > > the receiving path, as "tp->rcv_tstamp" can restart from idle, then > > tp->rcv_tstamp could already be a long time (minutes or hours) in the > > past even on the first RTO. > > > > Fixes: 1da177e4c3f4 ("Linux-2.6.12-rc2") > > Link: https://lore.kernel.org/netdev/CADxym3YyMiO+zMD4zj03YPM3FBi-1LHi6gSD2XT8pyAMM096pg@mail.gmail.com/ > > Signed-off-by: Menglong Dong <imagedong@tencent.com> > > --- > > v2: > > - consider the case of the connection restart from idle, as Neal comment > > --- > > net/ipv4/tcp_timer.c | 10 +++++++++- > > 1 file changed, 9 insertions(+), 1 deletion(-) > > > > diff --git a/net/ipv4/tcp_timer.c b/net/ipv4/tcp_timer.c > > index d45c96c7f5a4..e4b2d8706cae 100644 > > --- a/net/ipv4/tcp_timer.c > > +++ b/net/ipv4/tcp_timer.c > > @@ -454,6 +454,14 @@ static void tcp_fastopen_synack_timer(struct sock *sk, struct request_sock *req) > > req->timeout << req->num_timeout, TCP_RTO_MAX); > > } > > > > +static bool tcp_rtx_probe0_timed_out(struct sock *sk) > > +{ > > + struct tcp_sock *tp = tcp_sk(sk); > > + u32 last_ts; > > + > > + last_ts = max(tp->retrans_stamp, tp->rcv_tstamp); > > u32 retrans_stamp; /* Timestamp of the last retransmit, > u32 rcv_tstamp; /* timestamp of last received ACK (for keepalives) */ > > Both fields receive tcp_jiffies32 values, which wrap every 2^32 ticks. > > So max(A, B) won't work as you expect... > > You need to use before(), after() or something like that. Now I know how the before()/after() works. I thought they simply compared the arguments. I'll use brefore/after here instead. Thanks! Menglong Dong > > https://en.wikipedia.org/wiki/Serial_number_arithmetic#General_Solution > > > > + return inet_csk(sk)->icsk_timeout - last_ts > TCP_RTO_MAX; > > +} > > > > /** > > * tcp_retransmit_timer() - The TCP retransmit timeout handler > > @@ -519,7 +527,7 @@ void tcp_retransmit_timer(struct sock *sk) > > tp->snd_una, tp->snd_nxt); > > } > > #endif > > - if (tcp_jiffies32 - tp->rcv_tstamp > TCP_RTO_MAX) { > > + if (tcp_rtx_probe0_timed_out(sk)) { > > tcp_write_err(sk); > > goto out; > > } > > -- > > 2.40.1 > >
diff --git a/net/ipv4/tcp_timer.c b/net/ipv4/tcp_timer.c index d45c96c7f5a4..e4b2d8706cae 100644 --- a/net/ipv4/tcp_timer.c +++ b/net/ipv4/tcp_timer.c @@ -454,6 +454,14 @@ static void tcp_fastopen_synack_timer(struct sock *sk, struct request_sock *req) req->timeout << req->num_timeout, TCP_RTO_MAX); } +static bool tcp_rtx_probe0_timed_out(struct sock *sk) +{ + struct tcp_sock *tp = tcp_sk(sk); + u32 last_ts; + + last_ts = max(tp->retrans_stamp, tp->rcv_tstamp); + return inet_csk(sk)->icsk_timeout - last_ts > TCP_RTO_MAX; +} /** * tcp_retransmit_timer() - The TCP retransmit timeout handler @@ -519,7 +527,7 @@ void tcp_retransmit_timer(struct sock *sk) tp->snd_una, tp->snd_nxt); } #endif - if (tcp_jiffies32 - tp->rcv_tstamp > TCP_RTO_MAX) { + if (tcp_rtx_probe0_timed_out(sk)) { tcp_write_err(sk); goto out; }