| Message ID | 20240510030435.120935-2-kuba@kernel.org (mailing list archive) |
|---|---|
| State | RFC |
| Delegated to: | Netdev Maintainers |
| Headers | show |
| Series | add basic PSP encryption for TCP connections | expand |
On 09 May 20:04, Jakub Kicinski wrote: >Add documentation of things which belong in the docs rather >than commit messages. > >Signed-off-by: Jakub Kicinski <kuba@kernel.org> >--- > Documentation/networking/index.rst | 1 + > Documentation/networking/psp.rst | 138 +++++++++++++++++++++++++++++ > 2 files changed, 139 insertions(+) > create mode 100644 Documentation/networking/psp.rst > >diff --git a/Documentation/networking/index.rst b/Documentation/networking/index.rst >index 7664c0bfe461..0376029ecbdf 100644 >--- a/Documentation/networking/index.rst >+++ b/Documentation/networking/index.rst >@@ -94,6 +94,7 @@ Refer to :ref:`netdev-FAQ` for a guide on netdev development process specifics. > ppp_generic > proc_net_tcp > pse-pd/index >+ psp > radiotap-headers > rds > regulatory >diff --git a/Documentation/networking/psp.rst b/Documentation/networking/psp.rst >new file mode 100644 >index 000000000000..a39b464813ab >--- /dev/null >+++ b/Documentation/networking/psp.rst >@@ -0,0 +1,138 @@ >+.. SPDX-License-Identifier: GPL-2.0-only >+ >+===================== >+PSP Security Protocol >+===================== >+ >+Protocol >+======== >+ >+PSP Security Protocol (PSP) was defined at Google and published in: >+ >+https://raw.githubusercontent.com/google/psp/main/doc/PSP_Arch_Spec.pdf >+ >+This section briefly covers protocol aspects crucial for understanding >+the kernel API. Refer to the protocol specification for further details. >+ >+Note that the kernel implementation and documentation uses the term >+"secret state" in place of "master key", it is both less confusing >+to an average developer and is less likely to run afoul any naming >+guidelines. >+ [ ... ] >+User facing API >+=============== >+ >+PSP is designed primarily for hardware offloads. There is currently >+no software fallback for systems which do not have PSP capable NICs. >+There is also no standard (or otherwise defined) way of establishing >+a PSP-secured connection or exchanging the symmetric keys. >+ >+The expectation is that higher layer protocols will take care of >+protocol and key negotiation. For example one may use TLS key exchange, >+announce the PSP capability, and switch to PSP if both endpoints >+are PSP-capable. >+ The documentation doesn't include anything about userspace, other than highlevel remarks on how this is expected to work. What are we planning for userspace? I know we have kperf basic support and some experimental python library, but nothing official or psp centric. I propose to start community driven project with a well established library, with some concrete sample implementation for key negotiation, as a plugin maybe, so anyone can implement their own key-exchange mechanisms on top of the official psp library.
On Fri, 10 May 2024 15:19:23 -0700 Saeed Mahameed wrote: > >+PSP is designed primarily for hardware offloads. There is currently > >+no software fallback for systems which do not have PSP capable NICs. > >+There is also no standard (or otherwise defined) way of establishing > >+a PSP-secured connection or exchanging the symmetric keys. > >+ > >+The expectation is that higher layer protocols will take care of > >+protocol and key negotiation. For example one may use TLS key exchange, > >+announce the PSP capability, and switch to PSP if both endpoints > >+are PSP-capable. > > The documentation doesn't include anything about userspace, other than > highlevel remarks on how this is expected to work. The cover letter does. > What are we planning for userspace? I know we have kperf basic support and > some experimental python library, but nothing official or psp centric. Remind me, how long did it take for kernel TLS support to be merged into OpenSSL? ;) > I propose to start community driven project with a well established > library, with some concrete sample implementation for key negotiation, > as a plugin maybe, so anyone can implement their own key-exchange > mechanisms on top of the official psp library. Yes, I should have CCed Meta's folks who work on TLS [1]. Adding them now. More than happy to facilitate the discussion, maybe Willem can CC the right Google folks, IDK who else... We should start moving with the kernel support, IMO, until we do the user space implementation is stalled. I don't expect that the way we install keys in the kernel would be impacted by the handshake. [1] https://github.com/facebookincubator/fizz
On 11.05.2024 01:11, Jakub Kicinski wrote: > On Fri, 10 May 2024 15:19:23 -0700 Saeed Mahameed wrote: >>> +PSP is designed primarily for hardware offloads. There is currently >>> +no software fallback for systems which do not have PSP capable NICs. >>> +There is also no standard (or otherwise defined) way of establishing >>> +a PSP-secured connection or exchanging the symmetric keys. >>> + >>> +The expectation is that higher layer protocols will take care of >>> +protocol and key negotiation. For example one may use TLS key exchange, >>> +announce the PSP capability, and switch to PSP if both endpoints >>> +are PSP-capable. >> >> The documentation doesn't include anything about userspace, other than >> highlevel remarks on how this is expected to work. > > The cover letter does. > >> What are we planning for userspace? I know we have kperf basic support and >> some experimental python library, but nothing official or psp centric. > > Remind me, how long did it take for kernel TLS support to be merged > into OpenSSL? ;) I believe it was bad timing for OpenSSL. The patches with kTLS support were sitting in the main branch for long time, the problem was postponed release with with jump to new versioning schema. But I agree, there is no easy way to start coding user-space lib without initial support from kernel. >> I propose to start community driven project with a well established >> library, with some concrete sample implementation for key negotiation, >> as a plugin maybe, so anyone can implement their own key-exchange >> mechanisms on top of the official psp library. > > Yes, I should have CCed Meta's folks who work on TLS [1]. Adding them > now. More than happy to facilitate the discussion, maybe Willem can > CC the right Google folks, IDK who else... > > We should start moving with the kernel support, IMO, until we do > the user space implementation is stalled. I don't expect that the > way we install keys in the kernel would be impacted by the handshake. > > [1] https://github.com/facebookincubator/fizz
On 5/11/24 3:41 AM, Vadim Fedorenko wrote: > But I agree, there is no easy way to start coding user-space lib without > initial > support from kernel. The 2 sides should be co-developed; it is the only way to merge a sane uapi.
Jakub Kicinski wrote: > Add documentation of things which belong in the docs rather > than commit messages. > > Signed-off-by: Jakub Kicinski <kuba@kernel.org> > --- > Documentation/networking/index.rst | 1 + > Documentation/networking/psp.rst | 138 +++++++++++++++++++++++++++++ > 2 files changed, 139 insertions(+) > create mode 100644 Documentation/networking/psp.rst > > diff --git a/Documentation/networking/index.rst b/Documentation/networking/index.rst > index 7664c0bfe461..0376029ecbdf 100644 > --- a/Documentation/networking/index.rst > +++ b/Documentation/networking/index.rst > @@ -94,6 +94,7 @@ Refer to :ref:`netdev-FAQ` for a guide on netdev development process specifics. > ppp_generic > proc_net_tcp > pse-pd/index > + psp > radiotap-headers > rds > regulatory > diff --git a/Documentation/networking/psp.rst b/Documentation/networking/psp.rst > new file mode 100644 > index 000000000000..a39b464813ab > --- /dev/null > +++ b/Documentation/networking/psp.rst > @@ -0,0 +1,138 @@ > +.. SPDX-License-Identifier: GPL-2.0-only > + > +===================== > +PSP Security Protocol > +===================== > + > +Protocol > +======== > + > +PSP Security Protocol (PSP) was defined at Google and published in: > + > +https://raw.githubusercontent.com/google/psp/main/doc/PSP_Arch_Spec.pdf > + > +This section briefly covers protocol aspects crucial for understanding > +the kernel API. Refer to the protocol specification for further details. > + > +Note that the kernel implementation and documentation uses the term > +"secret state" in place of "master key", it is both less confusing > +to an average developer and is less likely to run afoul any naming > +guidelines. There is some value in using the same terminology in the code as in the spec. And the session keys are derived from a key. That is more precise than state. Specifically, counter-mode KDF from an AES key. Perhaps device key, instead of master key? > +Derived Rx keys > +--------------- > + > +PSP borrows some terms and mechanisms from IPsec. PSP was designed > +with HW offloads in mind. The key feature of PSP is that Rx keys for every > +connection do not have to be stored by the receiver but can be derived > +from secret state and information present in packet headers. A second less obvious, but neat, feature is that it supports an encryption offset, such that (say) the L4 ports are integrity protected, but not encrypted, to allow for in-network telemetry. > +This makes it possible to implement receivers which require a constant > +amount of memory regardless of the number of connections (``O(1)`` scaling). > + > +Tx keys have to be stored like with any other protocol, Keys can optionally be passed in descriptor. > +The expectation is that higher layer protocols will take care of > +protocol and key negotiation. For example one may use TLS key exchange, > +announce the PSP capability, and switch to PSP if both endpoints > +are PSP-capable. > +Securing a connection > +--------------------- > + > +PSP encryption is currently only supported for TCP connections. > +Rx and Tx keys are allocated separately. First the ``rx-assoc`` > +Netlink command needs to be issued, specifying a target TCP socket. > +Kernel will allocate a new PSP Rx key from the NIC and associate it > +with given socket. At this stage socket will accept both PSP-secured > +and plain text TCP packets. > + > +Tx keys are installed using the ``tx-assoc`` Netlink command. > +Once the Tx keys are installed all data read from the socket will > +be PSP-secured. In other words act of installing Tx keys has the secondary > +effect on the Rx direction, requring all received packets to be encrypted. Consider clarifying the entire state diagram from when one pair initiates upgrade. And some edge cases: - retransmits - TCP fin handshake, if only one peer succeeds - TCP control socket response to encrypted pkt What is the expectation for data already queued for transmission when the tx assocation is made? More generally, what happens for data in flight. One possible simplification is to only allow an upgrade sequence (possibly including in-band exchange of keys) when no other data is in flight. > +Since packet reception is asynchronous, to make it possible for the > +application to trust that any data read from the socket after the ``tx-assoc`` > +call returns success has been encrypted, the kernel will scan the receive > +queue of the socket at ``tx-assoc`` time. If any enqueued packet was received > +in clear text the Tx association will fail, and application should retry > +installing the Tx key after draining the socket (this should not be necessary > +if both endpoints are well behaved). > + > +Rotation notifications > +---------------------- > + > +The rotations of secret state happen asynchornously and are usually typo: asynchronously > +performed by management daemons, not under application control. > +The PSP netlink family will generate a notification whenever keys > +are rotated. The applications are expected to re-establish connections > +before keys are rotated again. Connection key rotation is not supported? I did notice that tx key insertion fails if a key is already present, so this does appear to be the behavior.
diff --git a/Documentation/networking/index.rst b/Documentation/networking/index.rst index 7664c0bfe461..0376029ecbdf 100644 --- a/Documentation/networking/index.rst +++ b/Documentation/networking/index.rst @@ -94,6 +94,7 @@ Refer to :ref:`netdev-FAQ` for a guide on netdev development process specifics. ppp_generic proc_net_tcp pse-pd/index + psp radiotap-headers rds regulatory diff --git a/Documentation/networking/psp.rst b/Documentation/networking/psp.rst new file mode 100644 index 000000000000..a39b464813ab --- /dev/null +++ b/Documentation/networking/psp.rst @@ -0,0 +1,138 @@ +.. SPDX-License-Identifier: GPL-2.0-only + +===================== +PSP Security Protocol +===================== + +Protocol +======== + +PSP Security Protocol (PSP) was defined at Google and published in: + +https://raw.githubusercontent.com/google/psp/main/doc/PSP_Arch_Spec.pdf + +This section briefly covers protocol aspects crucial for understanding +the kernel API. Refer to the protocol specification for further details. + +Note that the kernel implementation and documentation uses the term +"secret state" in place of "master key", it is both less confusing +to an average developer and is less likely to run afoul any naming +guidelines. + +Derived Rx keys +--------------- + +PSP borrows some terms and mechanisms from IPsec. PSP was designed +with HW offloads in mind. The key feature of PSP is that Rx keys for every +connection do not have to be stored by the receiver but can be derived +from secret state and information present in packet headers. +This makes it possible to implement receivers which require a constant +amount of memory regardless of the number of connections (``O(1)`` scaling). + +Tx keys have to be stored like with any other protocol, but Tx is much +less latency sensitive than Rx, and delays in fetching keys from slow +memory is less likely to cause packet drops. + +Key rotation +------------ + +The secret state known only to the receiver is fundamental to the design. +Per specification this state cannot be directly accessible (it must be +impossible to read it out of the hardware of the receiver NIC). +Moreover, it has to be "rotated" periodically (usually daily). Rotation +means that new secret state gets generated (by a random number generator +of the device), and used for all new connections. To avoid disrupting +old connections the old secret state remains in the NIC. A phase bit +carried in the packet headers indicates which generation of secret state +the packet has been encrypted with. + +User facing API +=============== + +PSP is designed primarily for hardware offloads. There is currently +no software fallback for systems which do not have PSP capable NICs. +There is also no standard (or otherwise defined) way of establishing +a PSP-secured connection or exchanging the symmetric keys. + +The expectation is that higher layer protocols will take care of +protocol and key negotiation. For example one may use TLS key exchange, +announce the PSP capability, and switch to PSP if both endpoints +are PSP-capable. + +All configuration of PSP is performed via the PSP netlink family. + +Device discovery +---------------- + +The PSP netlink family defines operations to retrieve information +about the PSP devices available on the system, configure them and +access PSP related statistics. + +Securing a connection +--------------------- + +PSP encryption is currently only supported for TCP connections. +Rx and Tx keys are allocated separately. First the ``rx-assoc`` +Netlink command needs to be issued, specifying a target TCP socket. +Kernel will allocate a new PSP Rx key from the NIC and associate it +with given socket. At this stage socket will accept both PSP-secured +and plain text TCP packets. + +Tx keys are installed using the ``tx-assoc`` Netlink command. +Once the Tx keys are installed all data read from the socket will +be PSP-secured. In other words act of installing Tx keys has the secondary +effect on the Rx direction, requring all received packets to be encrypted. +Since packet reception is asynchronous, to make it possible for the +application to trust that any data read from the socket after the ``tx-assoc`` +call returns success has been encrypted, the kernel will scan the receive +queue of the socket at ``tx-assoc`` time. If any enqueued packet was received +in clear text the Tx association will fail, and application should retry +installing the Tx key after draining the socket (this should not be necessary +if both endpoints are well behaved). + +Rotation notifications +---------------------- + +The rotations of secret state happen asynchornously and are usually +performed by management daemons, not under application control. +The PSP netlink family will generate a notification whenever keys +are rotated. The applications are expected to re-establish connections +before keys are rotated again. + +Kernel implementation +===================== + +Driver notes +------------ + +Drivers are expected to start with no PSP enabled (``psp-versions-ena`` +in ``dev-get`` set to ``0``) whenever possible. The user space should +not depend on this behavior, as future extension may necessitate creation +of devices with PSP already enabled, nonetheless drivers should not enable +PSP by default. Enabling PSP should be the responsibility of the system +component which also takes care of key rotation. + +Note that ``psp-versions-ena`` is expected to be used only for enabling +receive processing. The device is not expected to reject transmit requests +after ``psp-versions-ena`` has been disabled. User may also disable +``psp-versions-ena`` while there are active associations, which will +break all PSP Rx processing. + +Drivers are expected to ensure that secret state is usable upon init +(working keys can be allocated), and that no duplicate keys may be generated +(reuse of SPI without key rotation). Drivers may achieve this by rotating +keys twice before registering the PSP device. + +Drivers must use ``psp_skb_get_assoc_rcu()`` to check if PSP Tx offload +was requested for given skb. On Rx drivers should allocate and populate +the ``SKB_EXT_PSP`` skb extension, and set the skb->decrypted bit to 1. + +Kernel implementation notes +--------------------------- + +PSP implementation follows the TLS offload more closely than the IPsec +offload, with per-socket state, and the use of skb->decrypted to prevent +clear text leaks. + +PSP device is separate from netdev, to make it possible to "delegate" +PSP offload capabilities to software devices (e.g. ``veth``).
Add documentation of things which belong in the docs rather than commit messages. Signed-off-by: Jakub Kicinski <kuba@kernel.org> --- Documentation/networking/index.rst | 1 + Documentation/networking/psp.rst | 138 +++++++++++++++++++++++++++++ 2 files changed, 139 insertions(+) create mode 100644 Documentation/networking/psp.rst