| Message ID | 20240510030435.120935-1-kuba@kernel.org (mailing list archive) |
|---|---|
| Headers | show |
| Series | add basic PSP encryption for TCP connections | expand |
On 10.05.2024 05:04, Jakub Kicinski wrote: > External email: Use caution opening links or attachments > > > Hi! > > Add support for PSP encryption of TCP connections. > > PSP is a protocol out of Google: > https://github.com/google/psp/blob/main/doc/PSP_Arch_Spec.pdf > which shares some similarities with IPsec. I added some more info > in the first patch so I'll keep it short here. > > The protocol can work in multiple modes including tunneling. > But I'm mostly interested in using it as TLS replacement because > of its superior offload characteristics. Hi! Thank you for doing this. I agree that TLS-like socket support is a main use-case. I'd like to hear what you think on a few other use-cases that I think should be considered as well since it may be difficult to add them as an afterthought: - Tunnel mode. What are your plans for tunnel mode? Clearly it is different from the current approach in some aspects, for example, no sockets will be involved. - RDMA. The ultra ethernet group has mentioned RDMA encryption using PSP. Do you think that RDMA verbs will support PSP in a similar manner to sockets? i.e., using netlink to pass parameters to the device and linking QPs to PSP SAs? - Virtualization. How does PSP work from a VM? is the key shared with the hypervisor or is it private per-VM? and what about containers?
On Wed, 29 May 2024 11:16:12 +0200 Boris Pismenny wrote: > Thank you for doing this. I agree that TLS-like socket support > is a main use-case. I'd like to hear what you think on a few > other use-cases that I think should be considered as well > since it may be difficult to add them as an afterthought: > - Tunnel mode. What are your plans for tunnel mode? Clearly it > is different from the current approach in some aspects, for > example, no sockets will be involved. The drivers should only decap for known L4 protos, I think that's the only catch when we add tunnel support. Otherwise it should be fairly straightforward. Open a UDP socket in the kernel. Get a key + SPI using existing ops. Demux within the UDP socket using SPI. > - RDMA. The ultra ethernet group has mentioned RDMA encryption > using PSP. Do you think that RDMA verbs will support PSP in > a similar manner to sockets? i.e., using netlink to pass > parameters to the device and linking QPs to PSP SAs? > - Virtualization. How does PSP work from a VM? is the key > shared with the hypervisor or is it private per-VM? Depends on the deployment and security model, really, but I'd expect the device key is shared, hypervisor is responsible for rotations, and mediates all key ops from the guests. > and what about containers? I tried to apply some of the lessons learned from TLS offload and made the "PSP device" a separate object. This should make it easy to "forward" the offload to software/container netdevs.
On 29.05.2024 20:50, Jakub Kicinski wrote: > On Wed, 29 May 2024 11:16:12 +0200 Boris Pismenny wrote: >> Thank you for doing this. I agree that TLS-like socket support >> is a main use-case. I'd like to hear what you think on a few >> other use-cases that I think should be considered as well >> since it may be difficult to add them as an afterthought: >> - Tunnel mode. What are your plans for tunnel mode? Clearly it >> is different from the current approach in some aspects, for >> example, no sockets will be involved. > The drivers should only decap for known L4 protos, I think that's > the only catch when we add tunnel support. Otherwise it should be > fairly straightforward. Open a UDP socket in the kernel. Get a key > + SPI using existing ops. Demux within the UDP socket using SPI. IIUC, you refer to tunnel mode as if it offloads encryption alone while keeping headers intact. But, what I had in mind is a fully offloaded tunnel. This is called packet offload mode in IPsec, and with encryption such offloads rely on TC. Note that the main use-case for PSP tunnel mode, unlike transport mode, is carrying VM traffic as indicated by the spec: "The tunnel mode packet format is typically used in virtualized environments.". With virtualization, encap/decap offload is an implicit assumption if not a performance necessity. >> - RDMA. The ultra ethernet group has mentioned RDMA encryption >> using PSP. Do you think that RDMA verbs will support PSP in >> a similar manner to sockets? i.e., using netlink to pass >> parameters to the device and linking QPs to PSP SAs? >> - Virtualization. How does PSP work from a VM? is the key >> shared with the hypervisor or is it private per-VM? > Depends on the deployment and security model, really, but I'd > expect the device key is shared, hypervisor is responsible for > rotations, and mediates all key ops from the guests. I can imagine how this will work, but there are a few issues: - Guests may run out of Tx keys, but they can't initiate key rotation without affecting others. This fate sharing between VMs seems undesirable. - Unclear what sort of mediation is the hypervisor expected to provide: on the one hand, block a key rotation request and the requesting guest is denied service, on the other hand, allow key rotation and a guest may spam these requests to the hypervisor, which will also spam other VMs with notifications of key rotation. > >> and what about containers? > I tried to apply some of the lessons learned from TLS offload and made > the "PSP device" a separate object. This should make it easy to > "forward" the offload to software/container netdevs.
On Wed, 29 May 2024 22:01:52 +0200 Boris Pismenny wrote: > > The drivers should only decap for known L4 protos, I think that's > > the only catch when we add tunnel support. Otherwise it should be > > fairly straightforward. Open a UDP socket in the kernel. Get a key > > + SPI using existing ops. Demux within the UDP socket using SPI. > > IIUC, you refer to tunnel mode as if it offloads > encryption alone while keeping headers intact. But, > what I had in mind is a fully offloaded tunnel. > This is called packet offload mode in IPsec, > and with encryption such offloads rely on TC. Do you see any challenge? > > Depends on the deployment and security model, really, but I'd > > expect the device key is shared, hypervisor is responsible for > > rotations, and mediates all key ops from the guests. > > I can imagine how this will work, but there are a few issues: > - Guests may run out of Tx keys, but they can't initiate key > rotation without affecting others. This fate sharing between > VMs seems undesirable. > - Unclear what sort of mediation is the hypervisor expected > to provide: on the one hand, block a key rotation request and > the requesting guest is denied service, on the other hand, > allow key rotation and a guest may spam these requests to > the hypervisor, which will also spam other VMs with > notifications of key rotation. I don't have much experience working with VMs, or a good understanding of what mlx5 does internally. Without access to the details of even a single device which does PSP - any comment I'd make would be too much of a speculation for my taste :( On the NFP I'm pretty sure we could have given every VM a separate device key, no problem.
Hi! Add support for PSP encryption of TCP connections. PSP is a protocol out of Google: https://github.com/google/psp/blob/main/doc/PSP_Arch_Spec.pdf which shares some similarities with IPsec. I added some more info in the first patch so I'll keep it short here. The protocol can work in multiple modes including tunneling. But I'm mostly interested in using it as TLS replacement because of its superior offload characteristics. So this patch does three things: - it adds "core" PSP code PSP is offload-centric, and requires some additional care and feeding, so first chunk of the code exposes device info. This part can be reused by PSP implementations in xfrm, tunneling etc. - TCP integration TLS style Reuse some of the existing concepts from TLS offload, such as attaching crypto state to a socket, marking skbs as "decrypted", egress validation. PSP does not prescribe key exchange protocols. To use PSP as a more efficient TLS offload we intend to perform a TLS handshake ("inline" in the same TCP connection) and negotiate switching to PSP based on capabilities of both endpoints. This is also why I'm not including a software implementation. Nobody would use it in production, software TLS is faster, it has larger crypto records. - mlx5 implementation That's mostly other people's work, not 100% sure those folks consider it ready hence the RFC in the title. But it works :) Not posted, queued a branch [1] are follow up pieces: - standard stats - netdevsim implementation and tests [1] https://github.com/kuba-moo/linux/tree/psp Jakub Kicinski (8): psp: add documentation psp: base PSP device support net: modify core data structures for PSP datapath support tcp: add datapath logic for PSP with inline key exchange psp: add op for rotation of secret state net: psp: add socket security association code net: psp: update the TCP MSS to reflect PSP packet overhead psp: track generations of secret state Raed Salem (7): net/mlx5e: Support PSP offload functionality net/mlx5e: Implement PSP operations .assoc_add and .assoc_del net/mlx5e: Implement PSP Tx data path net/mlx5e: Add PSP steering in local NIC RX net/mlx5e: Configure PSP Rx flow steering rules net/mlx5e: Add Rx data path offload net/mlx5e: Implement PSP key_rotate operation Documentation/netlink/specs/psp.yaml | 186 +++++ Documentation/networking/index.rst | 1 + Documentation/networking/psp.rst | 138 ++++ .../net/ethernet/mellanox/mlx5/core/Kconfig | 11 + .../net/ethernet/mellanox/mlx5/core/Makefile | 5 +- drivers/net/ethernet/mellanox/mlx5/core/en.h | 7 +- .../net/ethernet/mellanox/mlx5/core/en/fs.h | 2 +- .../ethernet/mellanox/mlx5/core/en/params.c | 4 +- .../mellanox/mlx5/core/en_accel/en_accel.h | 50 +- .../mellanox/mlx5/core/en_accel/ipsec_rxtx.h | 2 +- .../mellanox/mlx5/core/en_accel/nisp.c | 209 +++++ .../mellanox/mlx5/core/en_accel/nisp.h | 55 ++ .../mellanox/mlx5/core/en_accel/nisp_fs.c | 737 ++++++++++++++++++ .../mellanox/mlx5/core/en_accel/nisp_fs.h | 30 + .../mlx5/core/en_accel/nisp_offload.c | 52 ++ .../mellanox/mlx5/core/en_accel/nisp_rxtx.c | 304 ++++++++ .../mellanox/mlx5/core/en_accel/nisp_rxtx.h | 124 +++ .../net/ethernet/mellanox/mlx5/core/en_main.c | 9 + .../net/ethernet/mellanox/mlx5/core/en_rx.c | 10 + .../net/ethernet/mellanox/mlx5/core/en_tx.c | 10 +- drivers/net/ethernet/mellanox/mlx5/core/fw.c | 6 + .../ethernet/mellanox/mlx5/core/lib/crypto.h | 1 + .../mellanox/mlx5/core/lib/psp_defs.h | 28 + .../net/ethernet/mellanox/mlx5/core/main.c | 5 + .../net/ethernet/mellanox/mlx5/core/nisp.c | 24 + .../net/ethernet/mellanox/mlx5/core/nisp.h | 15 + include/linux/mlx5/device.h | 4 + include/linux/mlx5/driver.h | 2 + include/linux/mlx5/mlx5_ifc.h | 98 ++- include/linux/netdevice.h | 4 + include/linux/skbuff.h | 3 + include/linux/tcp.h | 3 + include/net/dropreason-core.h | 6 + include/net/psp.h | 12 + include/net/psp/functions.h | 150 ++++ include/net/psp/types.h | 182 +++++ include/net/sock.h | 4 + include/uapi/linux/psp.h | 66 ++ net/Kconfig | 1 + net/Makefile | 1 + net/core/gro.c | 2 + net/core/skbuff.c | 4 + net/core/sock.c | 2 + net/ipv4/inet_connection_sock.c | 2 + net/ipv4/tcp.c | 2 + net/ipv4/tcp_ipv4.c | 13 +- net/ipv4/tcp_minisocks.c | 21 +- net/ipv4/tcp_output.c | 16 +- net/ipv6/ipv6_sockglue.c | 6 +- net/ipv6/tcp_ipv6.c | 22 +- net/mptcp/protocol.c | 2 + net/psp/Kconfig | 15 + net/psp/Makefile | 5 + net/psp/psp-nl-gen.c | 119 +++ net/psp/psp-nl-gen.h | 39 + net/psp/psp.h | 54 ++ net/psp/psp_main.c | 144 ++++ net/psp/psp_nl.c | 517 ++++++++++++ net/psp/psp_sock.c | 276 +++++++ tools/net/ynl/Makefile.deps | 1 + 60 files changed, 3791 insertions(+), 32 deletions(-) create mode 100644 Documentation/netlink/specs/psp.yaml create mode 100644 Documentation/networking/psp.rst create mode 100644 drivers/net/ethernet/mellanox/mlx5/core/en_accel/nisp.c create mode 100644 drivers/net/ethernet/mellanox/mlx5/core/en_accel/nisp.h create mode 100644 drivers/net/ethernet/mellanox/mlx5/core/en_accel/nisp_fs.c create mode 100644 drivers/net/ethernet/mellanox/mlx5/core/en_accel/nisp_fs.h create mode 100644 drivers/net/ethernet/mellanox/mlx5/core/en_accel/nisp_offload.c create mode 100644 drivers/net/ethernet/mellanox/mlx5/core/en_accel/nisp_rxtx.c create mode 100644 drivers/net/ethernet/mellanox/mlx5/core/en_accel/nisp_rxtx.h create mode 100644 drivers/net/ethernet/mellanox/mlx5/core/lib/psp_defs.h create mode 100644 drivers/net/ethernet/mellanox/mlx5/core/nisp.c create mode 100644 drivers/net/ethernet/mellanox/mlx5/core/nisp.h create mode 100644 include/net/psp.h create mode 100644 include/net/psp/functions.h create mode 100644 include/net/psp/types.h create mode 100644 include/uapi/linux/psp.h create mode 100644 net/psp/Kconfig create mode 100644 net/psp/Makefile create mode 100644 net/psp/psp-nl-gen.c create mode 100644 net/psp/psp-nl-gen.h create mode 100644 net/psp/psp.h create mode 100644 net/psp/psp_main.c create mode 100644 net/psp/psp_nl.c create mode 100644 net/psp/psp_sock.c