diff mbox series

[v3] tcp.7: Add description for TCP_FASTOPEN and TCP_FASTOPEN_CONNECT options

Message ID 20210924235456.2413081-1-weiwan@google.com (mailing list archive)
State Not Applicable
Delegated to: Netdev Maintainers
Headers show
Series [v3] tcp.7: Add description for TCP_FASTOPEN and TCP_FASTOPEN_CONNECT options | expand

Checks

Context Check Description
netdev/tree_selection success Not a local patch

Commit Message

Wei Wang Sept. 24, 2021, 11:54 p.m. UTC
TCP_FASTOPEN socket option was added by:
commit 8336886f786fdacbc19b719c1f7ea91eb70706d4
TCP_FASTOPEN_CONNECT socket option was added by the following patch
series:
commit 065263f40f0972d5f1cd294bb0242bd5aa5f06b2
commit 25776aa943401662617437841b3d3ea4693ee98a
commit 19f6d3f3c8422d65b5e3d2162e30ef07c6e21ea2
commit 3979ad7e82dfe3fb94a51c3915e64ec64afa45c3
Add detailed description for these 2 options.
Also add descriptions for /proc entry tcp_fastopen and tcp_fastopen_key.

Signed-off-by: Wei Wang <weiwan@google.com>
---
 man7/tcp.7 | 125 +++++++++++++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 125 insertions(+)

Comments

Wei Wang Oct. 15, 2021, 4:08 p.m. UTC | #1
On Fri, Sep 24, 2021 at 4:54 PM Wei Wang <weiwan@google.com> wrote:
>
> TCP_FASTOPEN socket option was added by:
> commit 8336886f786fdacbc19b719c1f7ea91eb70706d4
> TCP_FASTOPEN_CONNECT socket option was added by the following patch
> series:
> commit 065263f40f0972d5f1cd294bb0242bd5aa5f06b2
> commit 25776aa943401662617437841b3d3ea4693ee98a
> commit 19f6d3f3c8422d65b5e3d2162e30ef07c6e21ea2
> commit 3979ad7e82dfe3fb94a51c3915e64ec64afa45c3
> Add detailed description for these 2 options.
> Also add descriptions for /proc entry tcp_fastopen and tcp_fastopen_key.
>
> Signed-off-by: Wei Wang <weiwan@google.com>
> ---

Hi Alex,

Does this version look OK to you to apply?
Let me know.

Thanks.
Wei

>  man7/tcp.7 | 125 +++++++++++++++++++++++++++++++++++++++++++++++++++++
>  1 file changed, 125 insertions(+)
>
> diff --git a/man7/tcp.7 b/man7/tcp.7
> index 0a7c61a37..bdd4a33ca 100644
> --- a/man7/tcp.7
> +++ b/man7/tcp.7
> @@ -423,6 +423,28 @@ option.
>  .\" Since 2.4.0-test7
>  Enable RFC\ 2883 TCP Duplicate SACK support.
>  .TP
> +.IR tcp_fastopen  " (Bitmask; default: 0x1; since Linux 3.7)"
> +Enables RFC\ 7413 Fast Open support.
> +The flag is used as a bitmap with the following values:
> +.RS
> +.IP 0x1
> +Enables client side Fast Open support
> +.IP 0x2
> +Enables server side Fast Open support
> +.IP 0x4
> +Allows client side to transmit data in SYN without Fast Open option
> +.IP 0x200
> +Allows server side to accept SYN data without Fast Open option
> +.IP 0x400
> +Enables Fast Open on all listeners without
> +.B TCP_FASTOPEN
> +socket option
> +.RE
> +.TP
> +.IR tcp_fastopen_key " (since Linux 3.7)"
> +Set server side RFC\ 7413 Fast Open key to generate Fast Open cookie
> +when server side Fast Open support is enabled.
> +.TP
>  .IR tcp_ecn " (Integer; default: see below; since Linux 2.4)"
>  .\" Since 2.4.0-test7
>  Enable RFC\ 3168 Explicit Congestion Notification.
> @@ -1202,6 +1224,109 @@ Bound the size of the advertised window to this value.
>  The kernel imposes a minimum size of SOCK_MIN_RCVBUF/2.
>  This option should not be used in code intended to be
>  portable.
> +.TP
> +.BR TCP_FASTOPEN " (since Linux 3.6)"
> +This option enables Fast Open (RFC\ 7413) on the listener socket.
> +The value specifies the maximum length of pending SYNs
> +(similar to the backlog argument in
> +.BR listen (2)).
> +Once enabled,
> +the listener socket grants the TCP Fast Open cookie on incoming
> +SYN with TCP Fast Open option.
> +.IP
> +More importantly it accepts the data in SYN with a valid Fast Open cookie
> +and responds SYN-ACK acknowledging both the data and the SYN sequence.
> +.BR accept (2)
> +returns a socket that is available for read and write when the handshake
> +has not completed yet.
> +Thus the data exchange can commence before the handshake completes.
> +This option requires enabling the server-side support on sysctl
> +.IR net.ipv4.tcp_fastopen
> +(see above).
> +For TCP Fast Open client-side support,
> +see
> +.BR send (2)
> +.B MSG_FASTOPEN
> +or
> +.B TCP_FASTOPEN_CONNECT
> +below.
> +.TP
> +.BR TCP_FASTOPEN_CONNECT " (since Linux 4.11)"
> +This option enables an alternative way to perform Fast Open on the active
> +side (client).
> +When this option is enabled,
> +.BR connect (2)
> +would behave differently depending on if a Fast Open cookie is available
> +for the destination.
> +.IP
> +If a cookie is not available (i.e. first contact to the destination),
> +.BR connect (2)
> +behaves as usual by sending a SYN immediately,
> +except the SYN would include an empty Fast Open cookie option to solicit a
> +cookie.
> +.IP
> +If a cookie is available,
> +.BR connect (2)
> +would return 0 immediately but the SYN transmission is defered.
> +A subsequent
> +.BR write (2)
> +or
> +.BR sendmsg (2)
> +would trigger a SYN with data plus cookie in the Fast Open option.
> +In other words,
> +the actual connect operation is deferred until data is supplied.
> +.IP
> +.B Note:
> +While this option is designed for convenience,
> +enabling it does change the behaviors and certain system calls might set
> +different
> +.I errno
> +values.
> +With cookie present,
> +.BR write (2)
> +or
> +.BR sendmsg (2)
> +must be called right after
> +.BR connect (2)
> +in order to send out SYN+data to complete 3WHS and establish connection.
> +Calling
> +.BR read (2)
> +right after
> +.BR connect (2)
> +without
> +.BR write (2)
> +will cause the blocking socket to be blocked forever.
> +.IP
> +The application should  either set
> +.B TCP_FASTOPEN_CONNECT
> +socket option before
> +.BR write (2)
> +or
> +.BR sendmsg (2)
> +,
> +or call
> +.BR write (2)
> +or
> +.BR sendmsg (2)
> +with
> +.B MSG_FASTOPEN
> +flag directly,
> +instead of both on the same connection.
> +.IP
> +Here is the typical call flow with this new option:
> +.IP
> +.in +4n
> +.EX
> +s = socket();
> +setsockopt(s, IPPROTO_TCP, TCP_FASTOPEN_CONNECT, 1, ...);
> +connect(s);
> +write(s); // write() should always follow connect() in order to trigger SYN to go out
> +read(s)/write(s);
> +...
> +close(s);
> +.EE
> +.in
> +.IP
>  .SS Sockets API
>  TCP provides limited support for out-of-band data,
>  in the form of (a single byte of) urgent data.
> --
> 2.33.0.685.g46640cef36-goog
>
Alejandro Colomar Oct. 15, 2021, 10:12 p.m. UTC | #2
Hi Wei,

On 10/15/21 6:08 PM, Wei Wang wrote:
> On Fri, Sep 24, 2021 at 4:54 PM Wei Wang <weiwan@google.com> wrote:
>>
>> TCP_FASTOPEN socket option was added by:
>> commit 8336886f786fdacbc19b719c1f7ea91eb70706d4
>> TCP_FASTOPEN_CONNECT socket option was added by the following patch
>> series:
>> commit 065263f40f0972d5f1cd294bb0242bd5aa5f06b2
>> commit 25776aa943401662617437841b3d3ea4693ee98a
>> commit 19f6d3f3c8422d65b5e3d2162e30ef07c6e21ea2
>> commit 3979ad7e82dfe3fb94a51c3915e64ec64afa45c3
>> Add detailed description for these 2 options.
>> Also add descriptions for /proc entry tcp_fastopen and tcp_fastopen_key.
>>
>> Signed-off-by: Wei Wang <weiwan@google.com>
>> ---
> 
> Hi Alex,
> 
> Does this version look OK to you to apply?
> Let me know.

Sorry, I missed that patch.
Thanks for the ping!  I'll try to have a look at it ASAP.

Thanks,

Alex

> 
> Thanks.
> Wei
Wei Wang Nov. 5, 2021, 5:19 p.m. UTC | #3
On Fri, Oct 15, 2021 at 3:12 PM Alejandro Colomar (man-pages)
<alx.manpages@gmail.com> wrote:
>
> Hi Wei,
>
> On 10/15/21 6:08 PM, Wei Wang wrote:
> > On Fri, Sep 24, 2021 at 4:54 PM Wei Wang <weiwan@google.com> wrote:
> >>
> >> TCP_FASTOPEN socket option was added by:
> >> commit 8336886f786fdacbc19b719c1f7ea91eb70706d4
> >> TCP_FASTOPEN_CONNECT socket option was added by the following patch
> >> series:
> >> commit 065263f40f0972d5f1cd294bb0242bd5aa5f06b2
> >> commit 25776aa943401662617437841b3d3ea4693ee98a
> >> commit 19f6d3f3c8422d65b5e3d2162e30ef07c6e21ea2
> >> commit 3979ad7e82dfe3fb94a51c3915e64ec64afa45c3
> >> Add detailed description for these 2 options.
> >> Also add descriptions for /proc entry tcp_fastopen and tcp_fastopen_key.
> >>
> >> Signed-off-by: Wei Wang <weiwan@google.com>
> >> ---
> >
> > Hi Alex,
> >
> > Does this version look OK to you to apply?
> > Let me know.
>
> Sorry, I missed that patch.
> Thanks for the ping!  I'll try to have a look at it ASAP.
>

Hi Alex,

I am not sure if this patch has been applied yet?
If not, could you help take a look? Let me know if you'd like me to resend it.

Thanks.
Wei

> Thanks,
>
> Alex
>
>
> >
> > Thanks.
> > Wei
>
>
> --
> Alejandro Colomar
> Linux man-pages comaintainer; https://www.kernel.org/doc/man-pages/
> http://www.alejandro-colomar.es/
Alejandro Colomar Nov. 12, 2021, 8:55 p.m. UTC | #4
Hi Wei,

On 11/5/21 18:19, Wei Wang wrote:
> On Fri, Oct 15, 2021 at 3:12 PM Alejandro Colomar (man-pages)
> <alx.manpages@gmail.com> wrote:
>>
>> Hi Wei,
>>
>> On 10/15/21 6:08 PM, Wei Wang wrote:
>>> On Fri, Sep 24, 2021 at 4:54 PM Wei Wang <weiwan@google.com> wrote:
>>>>
>>>> TCP_FASTOPEN socket option was added by:
>>>> commit 8336886f786fdacbc19b719c1f7ea91eb70706d4
>>>> TCP_FASTOPEN_CONNECT socket option was added by the following patch
>>>> series:
>>>> commit 065263f40f0972d5f1cd294bb0242bd5aa5f06b2
>>>> commit 25776aa943401662617437841b3d3ea4693ee98a
>>>> commit 19f6d3f3c8422d65b5e3d2162e30ef07c6e21ea2
>>>> commit 3979ad7e82dfe3fb94a51c3915e64ec64afa45c3
>>>> Add detailed description for these 2 options.
>>>> Also add descriptions for /proc entry tcp_fastopen and tcp_fastopen_key.
>>>>
>>>> Signed-off-by: Wei Wang <weiwan@google.com>

Sorry for the delay.
I have applied it now.
I also applied some minor changes on top of your patch
(see below).

Thanks!
Alex


---

     tcp.7: Minor tweaks to Wei's patch

     - wsfix
     - Boldface literals (see groff_man(7)).
     - Replace '\ ' by \~, per Branden's advise.
     - Use phrasal semantic newlines.
     - Put '...' in a C comment, to avoid interfering with groff.
       It has the side effect that the code example is pure C now.
     - Remove incorrect trailing '.IP'.

     Cc: Wei Wang <weiwan@google.com>
     Signed-off-by: Alejandro Colomar <alx.manpages@gmail.com>

diff --git a/man7/tcp.7 b/man7/tcp.7
index 264e3ccc4..69d85c05a 100644
--- a/man7/tcp.7
+++ b/man7/tcp.7
@@ -423,26 +423,31 @@ option.
  .\" Since 2.4.0-test7
  Enable RFC\ 2883 TCP Duplicate SACK support.
  .TP
-.IR tcp_fastopen  " (Bitmask; default: 0x1; since Linux 3.7)"
-Enables RFC\ 7413 Fast Open support.
+.IR tcp_fastopen " (Bitmask; default: 0x1; since Linux 3.7)"
+Enables RFC\~7413 Fast Open support.
  The flag is used as a bitmap with the following values:
  .RS
-.IP 0x1
+.TP
+.B 0x1
  Enables client side Fast Open support
-.IP 0x2
+.TP
+.B 0x2
  Enables server side Fast Open support
-.IP 0x4
+.TP
+.B 0x4
  Allows client side to transmit data in SYN without Fast Open option
-.IP 0x200
+.TP
+.B 0x200
  Allows server side to accept SYN data without Fast Open option
-.IP 0x400
+.TP
+.B 0x400
  Enables Fast Open on all listeners without
  .B TCP_FASTOPEN
  socket option
  .RE
  .TP
  .IR tcp_fastopen_key " (since Linux 3.7)"
-Set server side RFC\ 7413 Fast Open key to generate Fast Open cookie
+Set server side RFC\~7413 Fast Open key to generate Fast Open cookie
  when server side Fast Open support is enabled.
  .TP
  .IR tcp_ecn " (Integer; default: see below; since Linux 2.4)"
@@ -1226,19 +1231,19 @@ This option should not be used in code intended 
to be
  portable.
  .TP
  .BR TCP_FASTOPEN " (since Linux 3.6)"
-This option enables Fast Open (RFC\ 7413) on the listener socket.
+This option enables Fast Open (RFC\~7413) on the listener socket.
  The value specifies the maximum length of pending SYNs
  (similar to the backlog argument in
  .BR listen (2)).
  Once enabled,
-the listener socket grants the TCP Fast Open cookie on incoming
-SYN with TCP Fast Open option.
+the listener socket grants the TCP Fast Open cookie
+on incoming SYN with TCP Fast Open option.
  .IP
  More importantly it accepts the data in SYN with a valid Fast Open cookie
  and responds SYN-ACK acknowledging both the data and the SYN sequence.
  .BR accept (2)
-returns a socket that is available for read and write when the handshake
-has not completed yet.
+returns a socket that is available for read and write
+when the handshake has not completed yet.
  Thus the data exchange can commence before the handshake completes.
  This option requires enabling the server-side support on sysctl
  .IR net.ipv4.tcp_fastopen
@@ -1252,18 +1257,18 @@ or
  below.
  .TP
  .BR TCP_FASTOPEN_CONNECT " (since Linux 4.11)"
-This option enables an alternative way to perform Fast Open on the active
-side (client).
+This option enables an alternative way to perform Fast Open
+on the active side (client).
  When this option is enabled,
  .BR connect (2)
-would behave differently depending on if a Fast Open cookie is available
-for the destination.
+would behave differently depending on
+if a Fast Open cookie is available for the destination.
  .IP
  If a cookie is not available (i.e. first contact to the destination),
  .BR connect (2)
  behaves as usual by sending a SYN immediately,
-except the SYN would include an empty Fast Open cookie option to solicit a
-cookie.
+except the SYN would include an empty Fast Open cookie option
+to solicit a cookie.
  .IP
  If a cookie is available,
  .BR connect (2)
@@ -1297,13 +1302,12 @@ without
  .BR write (2)
  will cause the blocking socket to be blocked forever.
  .IP
-The application should  either set
+The application should either set
  .B TCP_FASTOPEN_CONNECT
  socket option before
  .BR write (2)
  or
-.BR sendmsg (2)
-,
+.BR sendmsg (2),
  or call
  .BR write (2)
  or
@@ -1322,11 +1326,10 @@ setsockopt(s, IPPROTO_TCP, TCP_FASTOPEN_CONNECT, 
1, ...);
  connect(s);
  write(s); // write() should always follow connect() in order to 
trigger SYN to go out
  read(s)/write(s);
-...
+/* ... */
  close(s);
  .EE
  .in
-.IP
  .SS Sockets API
  TCP provides limited support for out-of-band data,
  in the form of (a single byte of) urgent data.
Wei Wang Nov. 12, 2021, 8:58 p.m. UTC | #5
On Fri, Nov 12, 2021 at 12:56 PM Alejandro Colomar (man-pages)
<alx.manpages@gmail.com> wrote:
>
> Hi Wei,
>
> On 11/5/21 18:19, Wei Wang wrote:
> > On Fri, Oct 15, 2021 at 3:12 PM Alejandro Colomar (man-pages)
> > <alx.manpages@gmail.com> wrote:
> >>
> >> Hi Wei,
> >>
> >> On 10/15/21 6:08 PM, Wei Wang wrote:
> >>> On Fri, Sep 24, 2021 at 4:54 PM Wei Wang <weiwan@google.com> wrote:
> >>>>
> >>>> TCP_FASTOPEN socket option was added by:
> >>>> commit 8336886f786fdacbc19b719c1f7ea91eb70706d4
> >>>> TCP_FASTOPEN_CONNECT socket option was added by the following patch
> >>>> series:
> >>>> commit 065263f40f0972d5f1cd294bb0242bd5aa5f06b2
> >>>> commit 25776aa943401662617437841b3d3ea4693ee98a
> >>>> commit 19f6d3f3c8422d65b5e3d2162e30ef07c6e21ea2
> >>>> commit 3979ad7e82dfe3fb94a51c3915e64ec64afa45c3
> >>>> Add detailed description for these 2 options.
> >>>> Also add descriptions for /proc entry tcp_fastopen and tcp_fastopen_key.
> >>>>
> >>>> Signed-off-by: Wei Wang <weiwan@google.com>
>
> Sorry for the delay.
> I have applied it now.
> I also applied some minor changes on top of your patch
> (see below).
>

Thank you so much for the fixes and thanks for applying, Alex!

> Thanks!
> Alex
>
>
> ---
>
>      tcp.7: Minor tweaks to Wei's patch
>
>      - wsfix
>      - Boldface literals (see groff_man(7)).
>      - Replace '\ ' by \~, per Branden's advise.
>      - Use phrasal semantic newlines.
>      - Put '...' in a C comment, to avoid interfering with groff.
>        It has the side effect that the code example is pure C now.
>      - Remove incorrect trailing '.IP'.
>
>      Cc: Wei Wang <weiwan@google.com>
>      Signed-off-by: Alejandro Colomar <alx.manpages@gmail.com>
>
> diff --git a/man7/tcp.7 b/man7/tcp.7
> index 264e3ccc4..69d85c05a 100644
> --- a/man7/tcp.7
> +++ b/man7/tcp.7
> @@ -423,26 +423,31 @@ option.
>   .\" Since 2.4.0-test7
>   Enable RFC\ 2883 TCP Duplicate SACK support.
>   .TP
> -.IR tcp_fastopen  " (Bitmask; default: 0x1; since Linux 3.7)"
> -Enables RFC\ 7413 Fast Open support.
> +.IR tcp_fastopen " (Bitmask; default: 0x1; since Linux 3.7)"
> +Enables RFC\~7413 Fast Open support.
>   The flag is used as a bitmap with the following values:
>   .RS
> -.IP 0x1
> +.TP
> +.B 0x1
>   Enables client side Fast Open support
> -.IP 0x2
> +.TP
> +.B 0x2
>   Enables server side Fast Open support
> -.IP 0x4
> +.TP
> +.B 0x4
>   Allows client side to transmit data in SYN without Fast Open option
> -.IP 0x200
> +.TP
> +.B 0x200
>   Allows server side to accept SYN data without Fast Open option
> -.IP 0x400
> +.TP
> +.B 0x400
>   Enables Fast Open on all listeners without
>   .B TCP_FASTOPEN
>   socket option
>   .RE
>   .TP
>   .IR tcp_fastopen_key " (since Linux 3.7)"
> -Set server side RFC\ 7413 Fast Open key to generate Fast Open cookie
> +Set server side RFC\~7413 Fast Open key to generate Fast Open cookie
>   when server side Fast Open support is enabled.
>   .TP
>   .IR tcp_ecn " (Integer; default: see below; since Linux 2.4)"
> @@ -1226,19 +1231,19 @@ This option should not be used in code intended
> to be
>   portable.
>   .TP
>   .BR TCP_FASTOPEN " (since Linux 3.6)"
> -This option enables Fast Open (RFC\ 7413) on the listener socket.
> +This option enables Fast Open (RFC\~7413) on the listener socket.
>   The value specifies the maximum length of pending SYNs
>   (similar to the backlog argument in
>   .BR listen (2)).
>   Once enabled,
> -the listener socket grants the TCP Fast Open cookie on incoming
> -SYN with TCP Fast Open option.
> +the listener socket grants the TCP Fast Open cookie
> +on incoming SYN with TCP Fast Open option.
>   .IP
>   More importantly it accepts the data in SYN with a valid Fast Open cookie
>   and responds SYN-ACK acknowledging both the data and the SYN sequence.
>   .BR accept (2)
> -returns a socket that is available for read and write when the handshake
> -has not completed yet.
> +returns a socket that is available for read and write
> +when the handshake has not completed yet.
>   Thus the data exchange can commence before the handshake completes.
>   This option requires enabling the server-side support on sysctl
>   .IR net.ipv4.tcp_fastopen
> @@ -1252,18 +1257,18 @@ or
>   below.
>   .TP
>   .BR TCP_FASTOPEN_CONNECT " (since Linux 4.11)"
> -This option enables an alternative way to perform Fast Open on the active
> -side (client).
> +This option enables an alternative way to perform Fast Open
> +on the active side (client).
>   When this option is enabled,
>   .BR connect (2)
> -would behave differently depending on if a Fast Open cookie is available
> -for the destination.
> +would behave differently depending on
> +if a Fast Open cookie is available for the destination.
>   .IP
>   If a cookie is not available (i.e. first contact to the destination),
>   .BR connect (2)
>   behaves as usual by sending a SYN immediately,
> -except the SYN would include an empty Fast Open cookie option to solicit a
> -cookie.
> +except the SYN would include an empty Fast Open cookie option
> +to solicit a cookie.
>   .IP
>   If a cookie is available,
>   .BR connect (2)
> @@ -1297,13 +1302,12 @@ without
>   .BR write (2)
>   will cause the blocking socket to be blocked forever.
>   .IP
> -The application should  either set
> +The application should either set
>   .B TCP_FASTOPEN_CONNECT
>   socket option before
>   .BR write (2)
>   or
> -.BR sendmsg (2)
> -,
> +.BR sendmsg (2),
>   or call
>   .BR write (2)
>   or
> @@ -1322,11 +1326,10 @@ setsockopt(s, IPPROTO_TCP, TCP_FASTOPEN_CONNECT,
> 1, ...);
>   connect(s);
>   write(s); // write() should always follow connect() in order to
> trigger SYN to go out
>   read(s)/write(s);
> -...
> +/* ... */
>   close(s);
>   .EE
>   .in
> -.IP
>   .SS Sockets API
>   TCP provides limited support for out-of-band data,
>   in the form of (a single byte of) urgent data.
diff mbox series

Patch

diff --git a/man7/tcp.7 b/man7/tcp.7
index 0a7c61a37..bdd4a33ca 100644
--- a/man7/tcp.7
+++ b/man7/tcp.7
@@ -423,6 +423,28 @@  option.
 .\" Since 2.4.0-test7
 Enable RFC\ 2883 TCP Duplicate SACK support.
 .TP
+.IR tcp_fastopen  " (Bitmask; default: 0x1; since Linux 3.7)"
+Enables RFC\ 7413 Fast Open support.
+The flag is used as a bitmap with the following values:
+.RS
+.IP 0x1
+Enables client side Fast Open support
+.IP 0x2
+Enables server side Fast Open support
+.IP 0x4
+Allows client side to transmit data in SYN without Fast Open option
+.IP 0x200
+Allows server side to accept SYN data without Fast Open option
+.IP 0x400
+Enables Fast Open on all listeners without
+.B TCP_FASTOPEN
+socket option
+.RE
+.TP
+.IR tcp_fastopen_key " (since Linux 3.7)"
+Set server side RFC\ 7413 Fast Open key to generate Fast Open cookie
+when server side Fast Open support is enabled.
+.TP
 .IR tcp_ecn " (Integer; default: see below; since Linux 2.4)"
 .\" Since 2.4.0-test7
 Enable RFC\ 3168 Explicit Congestion Notification.
@@ -1202,6 +1224,109 @@  Bound the size of the advertised window to this value.
 The kernel imposes a minimum size of SOCK_MIN_RCVBUF/2.
 This option should not be used in code intended to be
 portable.
+.TP
+.BR TCP_FASTOPEN " (since Linux 3.6)"
+This option enables Fast Open (RFC\ 7413) on the listener socket.
+The value specifies the maximum length of pending SYNs
+(similar to the backlog argument in
+.BR listen (2)).
+Once enabled,
+the listener socket grants the TCP Fast Open cookie on incoming
+SYN with TCP Fast Open option.
+.IP
+More importantly it accepts the data in SYN with a valid Fast Open cookie
+and responds SYN-ACK acknowledging both the data and the SYN sequence.
+.BR accept (2)
+returns a socket that is available for read and write when the handshake
+has not completed yet.
+Thus the data exchange can commence before the handshake completes.
+This option requires enabling the server-side support on sysctl
+.IR net.ipv4.tcp_fastopen
+(see above).
+For TCP Fast Open client-side support,
+see
+.BR send (2)
+.B MSG_FASTOPEN
+or
+.B TCP_FASTOPEN_CONNECT
+below.
+.TP
+.BR TCP_FASTOPEN_CONNECT " (since Linux 4.11)"
+This option enables an alternative way to perform Fast Open on the active
+side (client).
+When this option is enabled,
+.BR connect (2)
+would behave differently depending on if a Fast Open cookie is available
+for the destination.
+.IP
+If a cookie is not available (i.e. first contact to the destination),
+.BR connect (2)
+behaves as usual by sending a SYN immediately,
+except the SYN would include an empty Fast Open cookie option to solicit a
+cookie.
+.IP
+If a cookie is available,
+.BR connect (2)
+would return 0 immediately but the SYN transmission is defered.
+A subsequent
+.BR write (2)
+or
+.BR sendmsg (2)
+would trigger a SYN with data plus cookie in the Fast Open option.
+In other words,
+the actual connect operation is deferred until data is supplied.
+.IP
+.B Note:
+While this option is designed for convenience,
+enabling it does change the behaviors and certain system calls might set
+different
+.I errno
+values.
+With cookie present,
+.BR write (2)
+or
+.BR sendmsg (2)
+must be called right after
+.BR connect (2)
+in order to send out SYN+data to complete 3WHS and establish connection.
+Calling
+.BR read (2)
+right after
+.BR connect (2)
+without
+.BR write (2)
+will cause the blocking socket to be blocked forever.
+.IP
+The application should  either set
+.B TCP_FASTOPEN_CONNECT
+socket option before
+.BR write (2)
+or
+.BR sendmsg (2)
+,
+or call
+.BR write (2)
+or
+.BR sendmsg (2)
+with
+.B MSG_FASTOPEN
+flag directly,
+instead of both on the same connection.
+.IP
+Here is the typical call flow with this new option:
+.IP
+.in +4n
+.EX
+s = socket();
+setsockopt(s, IPPROTO_TCP, TCP_FASTOPEN_CONNECT, 1, ...);
+connect(s);
+write(s); // write() should always follow connect() in order to trigger SYN to go out
+read(s)/write(s);
+...
+close(s);
+.EE
+.in
+.IP
 .SS Sockets API
 TCP provides limited support for out-of-band data,
 in the form of (a single byte of) urgent data.