[mptcp-next,0/4] mptcp: performance improvemets

Message

Paolo Abeni Sept. 20, 2023, 12:45 p.m. UTC

Cool subj just to catch more attention;)

This is a follow-up the yday mtg and to the discussion started from
issues/437.

The first 3 patches implement a working support for rcvlowat, with the
intended and relevant side effect of avoiding most TCP-level immediate
acks.

The existing check in:

https://elixir.bootlin.com/linux/latest/source/net/ipv4/tcp_input.c#L5534

newarly always triggers immediate ack, as the any data received
on the subflow is usually immediately moved into the msk. That is, we
nearly always have: tp->rcv_nxt == tp->copied_seq.

That causes an unneeded large amount of (tcp-level) ack. After patch
3, MPTCP behaves much more alike plain TCP, compressing/delaying many
unneeded immediate ack and moving some of them at recvmsg() time.

Such change has quite a relevant effect on the max tput, ranging
from -4% to + 40% depending on the specific setup.
The small regression happens on CPU-bounded bulk transfers, with
not-so-large write buffer and can be elimitated increasing the write
buffer size. Even in such scenario, the overall efficency (B/W divided
by the total CPU cycles consumed) increases.
My personal take is that overall this for the better.

The last patch gives some speed-up to the tx path, just by using the
'correct' (or better) helper to memcpy the data from the user-space
into the kernel buffer.

Paolo Abeni (4):
  mptcp: properly account fastopen data
  mptcp: use plain bool instead of custom binary enum
  mptcp: give rcvlowat some love.
  mptcp: use copy_from_iter helpers on transmit..

 net/mptcp/fastopen.c |  1 +
 net/mptcp/protocol.c | 42 +++++++++++++++++++++++++-----------------
 net/mptcp/protocol.h | 27 +++++++++++++++++++++------
 net/mptcp/sockopt.c  | 31 +++++++++++++++++++++++++++++++
 net/mptcp/subflow.c  | 24 ++++++++++++++++--------
 5 files changed, 94 insertions(+), 31 deletions(-)

Comments

Mat Martineau Sept. 22, 2023, 4:59 p.m. UTC | #1

On Wed, 20 Sep 2023, Paolo Abeni wrote:

> Cool subj just to catch more attention;)
>
> This is a follow-up the yday mtg and to the discussion started from
> issues/437.
>
> The first 3 patches implement a working support for rcvlowat, with the
> intended and relevant side effect of avoiding most TCP-level immediate
> acks.
>
> The existing check in:
>
> https://elixir.bootlin.com/linux/latest/source/net/ipv4/tcp_input.c#L5534
>
> newarly always triggers immediate ack, as the any data received
> on the subflow is usually immediately moved into the msk. That is, we
> nearly always have: tp->rcv_nxt == tp->copied_seq.
>
> That causes an unneeded large amount of (tcp-level) ack. After patch
> 3, MPTCP behaves much more alike plain TCP, compressing/delaying many
> unneeded immediate ack and moving some of them at recvmsg() time.
>
> Such change has quite a relevant effect on the max tput, ranging
> from -4% to + 40% depending on the specific setup.
> The small regression happens on CPU-bounded bulk transfers, with
> not-so-large write buffer and can be elimitated increasing the write
> buffer size. Even in such scenario, the overall efficency (B/W divided
> by the total CPU cycles consumed) increases.
> My personal take is that overall this for the better.
>
> The last patch gives some speed-up to the tx path, just by using the
> 'correct' (or better) helper to memcpy the data from the user-space
> into the kernel buffer.
>

Hi Paolo -

Series looks good to me, thanks for the fixes and enhancements

Reviewed-by: Mat Martineau <[e-mail]>

> Paolo Abeni (4):
>  mptcp: properly account fastopen data
>  mptcp: use plain bool instead of custom binary enum
>  mptcp: give rcvlowat some love.
>  mptcp: use copy_from_iter helpers on transmit..
>
> net/mptcp/fastopen.c |  1 +
> net/mptcp/protocol.c | 42 +++++++++++++++++++++++++-----------------
> net/mptcp/protocol.h | 27 +++++++++++++++++++++------
> net/mptcp/sockopt.c  | 31 +++++++++++++++++++++++++++++++
> net/mptcp/subflow.c  | 24 ++++++++++++++++--------
> 5 files changed, 94 insertions(+), 31 deletions(-)
>
> -- 
> 2.41.0
>
>
>

Matthieu Baerts Sept. 23, 2023, 8:20 a.m. UTC | #2

Hi Paolo, Mat,

On 20/09/2023 14:45, Paolo Abeni wrote:
> Cool subj just to catch more attention;)
> 
> This is a follow-up the yday mtg and to the discussion started from
> issues/437.
> 
> The first 3 patches implement a working support for rcvlowat, with the
> intended and relevant side effect of avoiding most TCP-level immediate
> acks.
> 
> The existing check in:
> 
> https://elixir.bootlin.com/linux/latest/source/net/ipv4/tcp_input.c#L5534
> 
> newarly always triggers immediate ack, as the any data received
> on the subflow is usually immediately moved into the msk. That is, we
> nearly always have: tp->rcv_nxt == tp->copied_seq.
> 
> That causes an unneeded large amount of (tcp-level) ack. After patch
> 3, MPTCP behaves much more alike plain TCP, compressing/delaying many
> unneeded immediate ack and moving some of them at recvmsg() time.
> 
> Such change has quite a relevant effect on the max tput, ranging
> from -4% to + 40% depending on the specific setup.
> The small regression happens on CPU-bounded bulk transfers, with
> not-so-large write buffer and can be elimitated increasing the write
> buffer size. Even in such scenario, the overall efficency (B/W divided
> by the total CPU cycles consumed) increases.
> My personal take is that overall this for the better.
> 
> The last patch gives some speed-up to the tx path, just by using the
> 'correct' (or better) helper to memcpy the data from the user-space
> into the kernel buffer.

Thank you for this nice patch set!

New patches for t/upstream:
- 0188ee6897bb: mptcp: properly account fastopen data
- ea3f76f3c64b: mptcp: use plain bool instead of custom binary enum
- 03cdb460a417: mptcp: give rcvlowat some love
- 11b40c8da614: mptcp: use copy_from_iter helpers on transmit
- Results: 1c2adc4e573c..f2baa63004ad (export)

Tests are now in progress:

https://cirrus-ci.com/github/multipath-tcp/mptcp_net-next/export/20230923T080409

Cheers,
Matt