From patchwork Mon Apr 18 16:49:29 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Chuck Lever III X-Patchwork-Id: 12816862 X-Patchwork-Delegate: kuba@kernel.org Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 0DA98C433FE for ; Mon, 18 Apr 2022 16:49:36 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1346428AbiDRQwN (ORCPT ); Mon, 18 Apr 2022 12:52:13 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:52006 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1346425AbiDRQwM (ORCPT ); Mon, 18 Apr 2022 12:52:12 -0400 Received: from dfw.source.kernel.org (dfw.source.kernel.org [139.178.84.217]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 4C34E32EC7; Mon, 18 Apr 2022 09:49:32 -0700 (PDT) Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by dfw.source.kernel.org (Postfix) with ESMTPS id D5D07612DC; Mon, 18 Apr 2022 16:49:31 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id B5625C385A7; Mon, 18 Apr 2022 16:49:30 +0000 (UTC) Subject: [PATCH RFC 1/5] net: Add distinct sk_psock field From: Chuck Lever To: netdev@vger.kernel.org, linux-nfs@vger.kernel.org, linux-nvme@lists.infradead.org, linux-cifs@vger.kernel.org, linux-fsdevel@vger.kernel.org Cc: ak@tempesta-tech.com, borisp@nvidia.com, simo@redhat.com Date: Mon, 18 Apr 2022 12:49:29 -0400 Message-ID: <165030056960.5073.6664402939918720250.stgit@oracle-102.nfsv4.dev> In-Reply-To: <165030051838.5073.8699008789153780301.stgit@oracle-102.nfsv4.dev> References: <165030051838.5073.8699008789153780301.stgit@oracle-102.nfsv4.dev> User-Agent: StGit/1.5 MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org X-Patchwork-Delegate: kuba@kernel.org X-Patchwork-State: RFC The sk_psock facility populates the sk_user_data field with the address of an extra bit of metadata. User space sockets never populate the sk_user_data field, so this has worked out fine. However, kernel consumers such as the RPC client and server do populate the sk_user_data field. The sk_psock() function cannot tell that the content of sk_user_data does not point to psock metadata, so it will happily return a pointer to something else, cast to a struct sk_psock. Thus kernel consumers and psock currently cannot co-exist. We could educate sk_psock() to return NULL if sk_user_data does not point to a struct sk_psock. However, a more general solution that enables full co-existence psock and other uses of sk_user_data might be more interesting. Move the struct sk_psock address to its own pointer field so that the contents of the sk_user_data field is preserved. Signed-off-by: Chuck Lever Reviewed-by: Hannes Reinecke --- include/linux/skmsg.h | 2 +- include/net/sock.h | 4 +++- net/core/skmsg.c | 6 +++--- 3 files changed, 7 insertions(+), 5 deletions(-) diff --git a/include/linux/skmsg.h b/include/linux/skmsg.h index c5a2d6f50f25..5ef3a07c5b6c 100644 --- a/include/linux/skmsg.h +++ b/include/linux/skmsg.h @@ -277,7 +277,7 @@ static inline void sk_msg_sg_copy_clear(struct sk_msg *msg, u32 start) static inline struct sk_psock *sk_psock(const struct sock *sk) { - return rcu_dereference_sk_user_data(sk); + return rcu_dereference(sk->sk_psock); } static inline void sk_psock_set_state(struct sk_psock *psock, diff --git a/include/net/sock.h b/include/net/sock.h index c4b91fc19b9c..d2a513169527 100644 --- a/include/net/sock.h +++ b/include/net/sock.h @@ -327,7 +327,8 @@ struct sk_filter; * @sk_tskey: counter to disambiguate concurrent tstamp requests * @sk_zckey: counter to order MSG_ZEROCOPY notifications * @sk_socket: Identd and reporting IO signals - * @sk_user_data: RPC layer private data + * @sk_user_data: Upper layer private data + * @sk_psock: socket policy data (bpf) * @sk_frag: cached page frag * @sk_peek_off: current peek_offset value * @sk_send_head: front of stuff to transmit @@ -519,6 +520,7 @@ struct sock { struct socket *sk_socket; void *sk_user_data; + struct sk_psock __rcu *sk_psock; #ifdef CONFIG_SECURITY void *sk_security; #endif diff --git a/net/core/skmsg.c b/net/core/skmsg.c index cc381165ea08..2b3d01d92790 100644 --- a/net/core/skmsg.c +++ b/net/core/skmsg.c @@ -695,7 +695,7 @@ struct sk_psock *sk_psock_init(struct sock *sk, int node) write_lock_bh(&sk->sk_callback_lock); - if (sk->sk_user_data) { + if (sk->sk_psock) { psock = ERR_PTR(-EBUSY); goto out; } @@ -726,7 +726,7 @@ struct sk_psock *sk_psock_init(struct sock *sk, int node) sk_psock_set_state(psock, SK_PSOCK_TX_ENABLED); refcount_set(&psock->refcnt, 1); - rcu_assign_sk_user_data_nocopy(sk, psock); + rcu_assign_pointer(sk->sk_psock, psock); sock_hold(sk); out: @@ -825,7 +825,7 @@ void sk_psock_drop(struct sock *sk, struct sk_psock *psock) { write_lock_bh(&sk->sk_callback_lock); sk_psock_restore_proto(sk, psock); - rcu_assign_sk_user_data(sk, NULL); + rcu_assign_pointer(sk->sk_psock, NULL); if (psock->progs.stream_parser) sk_psock_stop_strp(sk, psock); else if (psock->progs.stream_verdict || psock->progs.skb_verdict) From patchwork Mon Apr 18 16:49:36 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Chuck Lever III X-Patchwork-Id: 12816863 X-Patchwork-Delegate: kuba@kernel.org Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 09107C43219 for ; Mon, 18 Apr 2022 16:49:44 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1346437AbiDRQwV (ORCPT ); Mon, 18 Apr 2022 12:52:21 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:52140 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1346425AbiDRQwT (ORCPT ); Mon, 18 Apr 2022 12:52:19 -0400 Received: from dfw.source.kernel.org (dfw.source.kernel.org [139.178.84.217]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 2A27432ED9; Mon, 18 Apr 2022 09:49:39 -0700 (PDT) Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by dfw.source.kernel.org (Postfix) with ESMTPS id B690E612E4; Mon, 18 Apr 2022 16:49:38 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 9833DC385AA; Mon, 18 Apr 2022 16:49:37 +0000 (UTC) Subject: [PATCH RFC 2/5] tls: build proto after context has been initialized From: Chuck Lever To: netdev@vger.kernel.org, linux-nfs@vger.kernel.org, linux-nvme@lists.infradead.org, linux-cifs@vger.kernel.org, linux-fsdevel@vger.kernel.org Cc: ak@tempesta-tech.com, borisp@nvidia.com, simo@redhat.com Date: Mon, 18 Apr 2022 12:49:36 -0400 Message-ID: <165030057652.5073.10364318727607743572.stgit@oracle-102.nfsv4.dev> In-Reply-To: <165030051838.5073.8699008789153780301.stgit@oracle-102.nfsv4.dev> References: <165030051838.5073.8699008789153780301.stgit@oracle-102.nfsv4.dev> User-Agent: StGit/1.5 MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org X-Patchwork-Delegate: kuba@kernel.org X-Patchwork-State: RFC From: Hannes Reinecke We have to build the proto ops only after the context has been initialized, as otherwise we might crash when I/O is ongoing during initialisation. Signed-off-by: Hannes Reinecke Signed-off-by: Chuck Lever --- net/tls/tls_main.c | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-) diff --git a/net/tls/tls_main.c b/net/tls/tls_main.c index 7b2b0e7ffee4..7eca4d9a83c4 100644 --- a/net/tls/tls_main.c +++ b/net/tls/tls_main.c @@ -836,8 +836,6 @@ static int tls_init(struct sock *sk) struct tls_context *ctx; int rc = 0; - tls_build_proto(sk); - #ifdef CONFIG_TLS_TOE if (tls_toe_bypass(sk)) return 0; @@ -862,6 +860,7 @@ static int tls_init(struct sock *sk) ctx->tx_conf = TLS_BASE; ctx->rx_conf = TLS_BASE; + tls_build_proto(sk); update_sk_prot(sk, ctx); out: write_unlock_bh(&sk->sk_callback_lock); From patchwork Mon Apr 18 16:49:43 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Chuck Lever III X-Patchwork-Id: 12816864 X-Patchwork-Delegate: kuba@kernel.org Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 8DFA6C4321E for ; Mon, 18 Apr 2022 16:50:23 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1346425AbiDRQxA (ORCPT ); Mon, 18 Apr 2022 12:53:00 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:52968 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1346447AbiDRQwa (ORCPT ); Mon, 18 Apr 2022 12:52:30 -0400 Received: from ams.source.kernel.org (ams.source.kernel.org [145.40.68.75]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id DBDF82BC3; Mon, 18 Apr 2022 09:49:47 -0700 (PDT) Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ams.source.kernel.org (Postfix) with ESMTPS id 70666B80FF4; Mon, 18 Apr 2022 16:49:46 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 93B34C385AA; Mon, 18 Apr 2022 16:49:44 +0000 (UTC) Subject: [PATCH RFC 3/5] net/tls: Add an AF_TLSH address family From: Chuck Lever To: netdev@vger.kernel.org, linux-nfs@vger.kernel.org, linux-nvme@lists.infradead.org, linux-cifs@vger.kernel.org, linux-fsdevel@vger.kernel.org Cc: ak@tempesta-tech.com, borisp@nvidia.com, simo@redhat.com Date: Mon, 18 Apr 2022 12:49:43 -0400 Message-ID: <165030058340.5073.5461321687798728373.stgit@oracle-102.nfsv4.dev> In-Reply-To: <165030051838.5073.8699008789153780301.stgit@oracle-102.nfsv4.dev> References: <165030051838.5073.8699008789153780301.stgit@oracle-102.nfsv4.dev> User-Agent: StGit/1.5 MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org X-Patchwork-Delegate: kuba@kernel.org X-Patchwork-State: RFC Add definitions for an AF_TLSH address family. The next patch explains its purpose and operation. Signed-off-by: Chuck Lever Reviewed-by: Hannes Reinecke --- include/linux/socket.h | 4 +++- net/core/sock.c | 2 +- net/socket.c | 1 + security/selinux/hooks.c | 4 +++- security/selinux/include/classmap.h | 4 +++- tools/perf/trace/beauty/include/linux/socket.h | 4 +++- 6 files changed, 14 insertions(+), 5 deletions(-) diff --git a/include/linux/socket.h b/include/linux/socket.h index 6f85f5d957ef..fc28c68e6b5f 100644 --- a/include/linux/socket.h +++ b/include/linux/socket.h @@ -226,8 +226,9 @@ struct ucred { #define AF_MCTP 45 /* Management component * transport protocol */ +#define AF_TLSH 46 /* TLS handshake request */ -#define AF_MAX 46 /* For now.. */ +#define AF_MAX 47 /* For now.. */ /* Protocol families, same as address families. */ #define PF_UNSPEC AF_UNSPEC @@ -278,6 +279,7 @@ struct ucred { #define PF_SMC AF_SMC #define PF_XDP AF_XDP #define PF_MCTP AF_MCTP +#define PF_TLSH AF_TLSH #define PF_MAX AF_MAX /* Maximum queue length specifiable by listen. */ diff --git a/net/core/sock.c b/net/core/sock.c index 1180a0cb0110..81bc14b67468 100644 --- a/net/core/sock.c +++ b/net/core/sock.c @@ -224,7 +224,7 @@ static struct lock_class_key af_family_kern_slock_keys[AF_MAX]; x "AF_IEEE802154", x "AF_CAIF" , x "AF_ALG" , \ x "AF_NFC" , x "AF_VSOCK" , x "AF_KCM" , \ x "AF_QIPCRTR", x "AF_SMC" , x "AF_XDP" , \ - x "AF_MCTP" , \ + x "AF_MCTP" , x "AF_TLSH" , \ x "AF_MAX" static const char *const af_family_key_strings[AF_MAX+1] = { diff --git a/net/socket.c b/net/socket.c index 6887840682bb..fc43ebd5ad66 100644 --- a/net/socket.c +++ b/net/socket.c @@ -214,6 +214,7 @@ static const char * const pf_family_names[] = { [PF_SMC] = "PF_SMC", [PF_XDP] = "PF_XDP", [PF_MCTP] = "PF_MCTP", + [PF_TLSH] = "PF_TLSH", }; /* diff --git a/security/selinux/hooks.c b/security/selinux/hooks.c index e9e959343de9..734e284eb06a 100644 --- a/security/selinux/hooks.c +++ b/security/selinux/hooks.c @@ -1259,7 +1259,9 @@ static inline u16 socket_type_to_security_class(int family, int type, int protoc return SECCLASS_XDP_SOCKET; case PF_MCTP: return SECCLASS_MCTP_SOCKET; -#if PF_MAX > 46 + case PF_TLSH: + return SECCLASS_TLSH_SOCKET; +#if PF_MAX > 47 #error New address family defined, please update this function. #endif } diff --git a/security/selinux/include/classmap.h b/security/selinux/include/classmap.h index 35aac62a662e..673b51f8281d 100644 --- a/security/selinux/include/classmap.h +++ b/security/selinux/include/classmap.h @@ -237,6 +237,8 @@ struct security_class_mapping secclass_map[] = { { COMMON_SOCK_PERMS, NULL } }, { "smc_socket", { COMMON_SOCK_PERMS, NULL } }, + { "tlsh_socket", + { COMMON_SOCK_PERMS, NULL } }, { "infiniband_pkey", { "access", NULL } }, { "infiniband_endport", @@ -257,6 +259,6 @@ struct security_class_mapping secclass_map[] = { { NULL } }; -#if PF_MAX > 46 +#if PF_MAX > 47 #error New address family defined, please update secclass_map. #endif diff --git a/tools/perf/trace/beauty/include/linux/socket.h b/tools/perf/trace/beauty/include/linux/socket.h index 6f85f5d957ef..fc28c68e6b5f 100644 --- a/tools/perf/trace/beauty/include/linux/socket.h +++ b/tools/perf/trace/beauty/include/linux/socket.h @@ -226,8 +226,9 @@ struct ucred { #define AF_MCTP 45 /* Management component * transport protocol */ +#define AF_TLSH 46 /* TLS handshake request */ -#define AF_MAX 46 /* For now.. */ +#define AF_MAX 47 /* For now.. */ /* Protocol families, same as address families. */ #define PF_UNSPEC AF_UNSPEC @@ -278,6 +279,7 @@ struct ucred { #define PF_SMC AF_SMC #define PF_XDP AF_XDP #define PF_MCTP AF_MCTP +#define PF_TLSH AF_TLSH #define PF_MAX AF_MAX /* Maximum queue length specifiable by listen. */ From patchwork Mon Apr 18 16:49:50 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Chuck Lever III X-Patchwork-Id: 12816865 X-Patchwork-Delegate: kuba@kernel.org Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id E305AC4167D for ; Mon, 18 Apr 2022 16:50:23 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1346445AbiDRQxA (ORCPT ); Mon, 18 Apr 2022 12:53:00 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:54350 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1346479AbiDRQws (ORCPT ); Mon, 18 Apr 2022 12:52:48 -0400 Received: from dfw.source.kernel.org (dfw.source.kernel.org [139.178.84.217]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 8B1E126573; Mon, 18 Apr 2022 09:49:53 -0700 (PDT) Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by dfw.source.kernel.org (Postfix) with ESMTPS id 1232A612E6; Mon, 18 Apr 2022 16:49:53 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id B3F89C385A1; Mon, 18 Apr 2022 16:49:51 +0000 (UTC) Subject: [PATCH RFC 4/5] net/tls: Add support for PF_TLSH (a TLS handshake listener) From: Chuck Lever To: netdev@vger.kernel.org, linux-nfs@vger.kernel.org, linux-nvme@lists.infradead.org, linux-cifs@vger.kernel.org, linux-fsdevel@vger.kernel.org Cc: ak@tempesta-tech.com, borisp@nvidia.com, simo@redhat.com Date: Mon, 18 Apr 2022 12:49:50 -0400 Message-ID: <165030059051.5073.16723746870370826608.stgit@oracle-102.nfsv4.dev> In-Reply-To: <165030051838.5073.8699008789153780301.stgit@oracle-102.nfsv4.dev> References: <165030051838.5073.8699008789153780301.stgit@oracle-102.nfsv4.dev> User-Agent: StGit/1.5 MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org X-Patchwork-Delegate: kuba@kernel.org X-Patchwork-State: RFC In-kernel TLS consumers need a way to perform a TLS handshake. In the absence of a handshake implementation in the kernel itself, a mechanism to perform the handshake in user space, using an existing TLS handshake library, is necessary. I've designed a way to pass a connected kernel socket endpoint to user space using the traditional listen/accept mechanism. accept(2) gives us a well-understood way to materialize a socket endpoint as a normal file descriptor in a specific user space process. Like any open socket descriptor, the accepted FD can then be passed to a library such as openSSL to perform a TLS handshake. This prototype currently handles only initiating client-side TLS handshakes. Server-side handshakes and key renegotiation are left to do. Security Considerations ~~~~~~~~ ~~~~~~~~~~~~~~ This prototype is net-namespace aware. The kernel has no mechanism to attest that the listening user space agent is trustworthy. Currently the prototype does not handle multiple listeners that overlap -- multiple listeners in the same net namespace that have overlapping bind addresses. Signed-off-by: Chuck Lever Reviewed-by: Hannes Reinecke --- .../networking/tls-in-kernel-handshake.rst | 103 ++ include/linux/socket.h | 1 include/net/sock.h | 3 include/net/tls.h | 15 include/net/tlsh.h | 22 include/uapi/linux/tls.h | 16 net/core/sock.c | 2 net/tls/Makefile | 2 net/tls/af_tlsh.c | 1040 ++++++++++++++++++++ net/tls/tls_main.c | 10 10 files changed, 1213 insertions(+), 1 deletion(-) create mode 100644 Documentation/networking/tls-in-kernel-handshake.rst create mode 100644 include/net/tlsh.h create mode 100644 net/tls/af_tlsh.c diff --git a/Documentation/networking/tls-in-kernel-handshake.rst b/Documentation/networking/tls-in-kernel-handshake.rst new file mode 100644 index 000000000000..73ed6928f4b2 --- /dev/null +++ b/Documentation/networking/tls-in-kernel-handshake.rst @@ -0,0 +1,103 @@ +.. _kernel_tls: + +======================= +In-Kernel TLS Handshake +======================= + +Overview +======== + +Transport Layer Security (TLS) is a Upper Layer Protocol (ULP) that runs over +TCP. TLS provides end-to-end data integrity and confidentiality. + +kTLS handles the TLS record subprotocol, but does not handle the TLS handshake +subprotocol, used to establish a TLS session. In user space, a TLS library +performs the handshake on a socket which is converted to kTLS operation. In +the kernel it is much the same. The TLS handshake is done in user space by a +library TLS implementation. + + +User agent +========== + +With the current implementation, a user agent is started in each network +namespace where a kernel consumer might require a TLS handshake. This agent +listens on an AF_TLSH socket for requests from the kernel to perform a +handshake on an open and connected TCP socket. + +The open socket is passed to user space via accept(), which creates a file +descriptor. If the handshake completes successfully, the user agent promotes +the socket to use the TLS ULP and sets the session information using the +SOL_TLS socket options. The user agent returns the socket to the kernel by +closing the accepted file descriptor. + + +Kernel Handshake API +==================== + +A kernel consumer initiates a client-side TLS handshake on an open +socket by invoking one of the tls_client_hello() functions. For +example: + +.. code-block:: c + + ret = tls_client_hello_x509(sock, done_func, cookie, priorities, + peerid, cert); + +The function returns zero when the handshake request is under way. A +zero return guarantees the callback function @done_func will be invoked +for this socket. + +The function returns a negative errno if the handshake could not be +started. A negative errno guarantees the callback function @done_func +will not be invoked on this socket. + +The @sock argument is an open and connected IPPROTO_TCP socket. The +caller must hold a reference on the socket to prevent it from being +destroyed while the handshake is in progress. + +@done_func and @cookie are a callback function that is invoked when the +handshake has completed (either successfully or not). The success status +of the handshake is returned via the @status parameter of the callback +function. A good practice is to close and destroy the socket immediately +if the handshake has failed. + +@priorities is a GnuTLS priorities string that controls the handshake. +The special value TLSH_DEFAULT_PRIORITIES causes the handshake to +operate using user space configured default TLS priorities. However, +the caller can use the string to (for example) adjust the handshake to +use a restricted set of ciphers (say, if the kernel is in FIPS mode or +the kernel consumer wants to mandate only a limited set of ciphers). + +@peerid is the serial number of a key on the XXXYYYZZZ keyring that +contains a private key. + +@cert is the serial number of a key on the XXXYYYYZZZ keyring that +contains a {PEM,DER} format x.509 certificate that the user agent +presents to the server as the local peer's identity. + +To initiate a client-side TLS handshake with a pre-shared key, use: + +.. code-block:: c + + ret = tls_client_hello_psk(sock, done_func, cookie, priorities, + peerid); + +@peerid is the serial number of a key on the XXXYYYZZZ keyring that +contains the pre-shared key. + +The other parameters are as above. + + +Other considerations +-------------------- + +While the handshake is under way, the kernel consumer must alter the +socket's sk_data_ready callback function to ignore incoming data. +Once the callback function has been invoked, normal receive operation +can be resumed. + +See tls.rst for details on how a kTLS consumer recognizes incoming +(decrypted) application data, alerts, and handshake packets once the +socket has been promoted to use the TLS ULP. + diff --git a/include/linux/socket.h b/include/linux/socket.h index fc28c68e6b5f..69acb5668d34 100644 --- a/include/linux/socket.h +++ b/include/linux/socket.h @@ -369,6 +369,7 @@ struct ucred { #define SOL_MPTCP 284 #define SOL_MCTP 285 #define SOL_SMC 286 +#define SOL_TLSH 287 /* IPX options */ #define IPX_TYPE 1 diff --git a/include/net/sock.h b/include/net/sock.h index d2a513169527..d5a5d5fd6682 100644 --- a/include/net/sock.h +++ b/include/net/sock.h @@ -353,6 +353,7 @@ struct sk_filter; * @sk_txtime_report_errors: set report errors mode for SO_TXTIME * @sk_txtime_unused: unused txtime flags * @ns_tracker: tracker for netns reference + * @sk_tlsh_priv: private data for TLS handshake upcall */ struct sock { /* @@ -544,6 +545,8 @@ struct sock { #endif struct rcu_head sk_rcu; netns_tracker ns_tracker; + + void *sk_tlsh_priv; }; enum sk_pacing { diff --git a/include/net/tls.h b/include/net/tls.h index b6968a5b5538..6b1bf46daa34 100644 --- a/include/net/tls.h +++ b/include/net/tls.h @@ -51,6 +51,18 @@ #include +struct tlsh_sock { + /* struct sock must remain the first field */ + struct sock th_sk; + + int th_bind_family; +}; + +static inline struct tlsh_sock *tlsh_sk(struct sock *sk) +{ + return (struct tlsh_sock *)sk; +} + /* Maximum data size carried in a TLS record */ #define TLS_MAX_PAYLOAD_SIZE ((size_t)1 << 14) @@ -356,6 +368,9 @@ struct tls_context *tls_ctx_create(struct sock *sk); void tls_ctx_free(struct sock *sk, struct tls_context *ctx); void update_sk_prot(struct sock *sk, struct tls_context *ctx); +int tlsh_pf_create(struct net *net, struct socket *sock, int protocol, + int kern); + int wait_on_pending_writer(struct sock *sk, long *timeo); int tls_sk_query(struct sock *sk, int optname, char __user *optval, int __user *optlen); diff --git a/include/net/tlsh.h b/include/net/tlsh.h new file mode 100644 index 000000000000..8725fd83df60 --- /dev/null +++ b/include/net/tlsh.h @@ -0,0 +1,22 @@ +/* SPDX-License-Identifier: GPL-2.0-only */ +/* + * PF_TLSH protocol family socket handler. + * + * Author: Chuck Lever + * + * Copyright (c) 2021, Oracle and/or its affiliates. + */ + +#ifndef _TLS_HANDSHAKE_H +#define _TLS_HANDSHAKE_H + +extern int tls_client_hello_psk(struct socket *sock, + void (*done)(void *data, int status), + void *data, const char *priorities, + key_serial_t peerid); +extern int tls_client_hello_x509(struct socket *sock, + void (*done)(void *data, int status), + void *data, const char *priorities, + key_serial_t peerid, key_serial_t cert); + +#endif /* _TLS_HANDSHAKE_H */ diff --git a/include/uapi/linux/tls.h b/include/uapi/linux/tls.h index 5f38be0ec0f3..d0ffbb6ea0e4 100644 --- a/include/uapi/linux/tls.h +++ b/include/uapi/linux/tls.h @@ -40,6 +40,22 @@ #define TLS_TX 1 /* Set transmit parameters */ #define TLS_RX 2 /* Set receive parameters */ +/* TLSH socket options */ +#define TLSH_PRIORITIES 1 /* Retrieve TLS priorities string */ +#define TLSH_PEERID 2 /* Retrieve peer identity */ +#define TLSH_HANDSHAKE_TYPE 3 /* Retrieve handshake type */ +#define TLSH_X509_CERTIFICATE 4 /* Retrieve x.509 certificate */ + +#define TLSH_DEFAULT_PRIORITIES (NULL) +#define TLSH_NO_PEERID (0) +#define TLSH_NO_CERT (0) + +/* TLSH handshake types */ +enum tlsh_hs_type { + TLSH_TYPE_CLIENTHELLO_X509, + TLSH_TYPE_CLIENTHELLO_PSK, +}; + /* Supported versions */ #define TLS_VERSION_MINOR(ver) ((ver) & 0xFF) #define TLS_VERSION_MAJOR(ver) (((ver) >> 8) & 0xFF) diff --git a/net/core/sock.c b/net/core/sock.c index 81bc14b67468..d9f700e5ea1a 100644 --- a/net/core/sock.c +++ b/net/core/sock.c @@ -3295,6 +3295,8 @@ void sock_init_data(struct socket *sock, struct sock *sk) sk->sk_incoming_cpu = -1; sk->sk_txrehash = SOCK_TXREHASH_DEFAULT; + sk->sk_tlsh_priv = NULL; + sk_rx_queue_clear(sk); /* * Before updating sk_refcnt, we must commit prior changes to memory diff --git a/net/tls/Makefile b/net/tls/Makefile index f1ffbfe8968d..d159a03b94f3 100644 --- a/net/tls/Makefile +++ b/net/tls/Makefile @@ -7,7 +7,7 @@ CFLAGS_trace.o := -I$(src) obj-$(CONFIG_TLS) += tls.o -tls-y := tls_main.o tls_sw.o tls_proc.o trace.o +tls-y := af_tlsh.o tls_main.o tls_sw.o tls_proc.o trace.o tls-$(CONFIG_TLS_TOE) += tls_toe.o tls-$(CONFIG_TLS_DEVICE) += tls_device.o tls_device_fallback.o diff --git a/net/tls/af_tlsh.c b/net/tls/af_tlsh.c new file mode 100644 index 000000000000..4d1c1de3a474 --- /dev/null +++ b/net/tls/af_tlsh.c @@ -0,0 +1,1040 @@ +// SPDX-License-Identifier: GPL-2.0-only +/* + * PF_TLSH protocol family socket handler. + * + * Author: Chuck Lever + * + * Copyright (c) 2021, Oracle and/or its affiliates. + * + * When a kernel TLS consumer wants to establish a TLS session, it + * makes an AF_TLSH Listener ready. When user space accepts on that + * listener, the kernel fabricates a user space socket endpoint on + * which a user space TLS library can perform the TLS handshake. + * + * Closing the user space descriptor signals to the kernel that the + * library handshake process is complete. If the library has managed + * to initialize the socket's TLS crypto_info, the kernel marks the + * handshake as a success. + */ + +/* + * Socket reference counting + * A: listener socket initial reference + * B: listener socket on the global listener list + * C: listener socket while a ready AF_INET(6) socket is enqueued + * D: listener socket while its accept queue is drained + * + * I: ready AF_INET(6) socket waiting on a listener's accept queue + * J: ready AF_INET(6) socket with a consumer waiting for a completion callback + */ + +#include +#include +#include +#include +#include +#include +#include +#include +#include + +#include +#include +#include +#include +#include +#include +#include +#include +#include + +#include "trace.h" + + +struct tlsh_sock_info { + enum tlsh_hs_type tsi_handshake_type; + + void (*tsi_handshake_done)(void *data, int status); + void *tsi_handshake_data; + char *tsi_tls_priorities; + key_serial_t tsi_peerid; + key_serial_t tsi_certificate; + + struct socket_wq *tsi_saved_wq; + struct socket *tsi_saved_socket; + kuid_t tsi_saved_uid; +}; + +static void tlsh_sock_info_destroy(struct tlsh_sock_info *info) +{ + kfree(info->tsi_tls_priorities); + kfree(info); +} + +static DEFINE_RWLOCK(tlsh_listener_lock); +static HLIST_HEAD(tlsh_listeners); + +static void tlsh_register_listener(struct sock *sk) +{ + write_lock_bh(&tlsh_listener_lock); + sk_add_node(sk, &tlsh_listeners); /* Ref: B */ + write_unlock_bh(&tlsh_listener_lock); +} + +static void tlsh_unregister_listener(struct sock *sk) +{ + write_lock_bh(&tlsh_listener_lock); + sk_del_node_init(sk); /* Ref: B */ + write_unlock_bh(&tlsh_listener_lock); +} + +/** + * tlsh_find_listener - find listener that matches an incoming connection + * @net: net namespace to match + * @family: address family to match + * + * Return values: + * On success, address of a listening AF_TLSH socket + * %NULL: No matching listener found + */ +static struct sock *tlsh_find_listener(struct net *net, unsigned short family) +{ + struct sock *listener; + + read_lock(&tlsh_listener_lock); + + sk_for_each(listener, &tlsh_listeners) { + if (sock_net(listener) != net) + continue; + if (tlsh_sk(listener)->th_bind_family != AF_UNSPEC && + tlsh_sk(listener)->th_bind_family != family) + continue; + + sock_hold(listener); /* Ref: C */ + goto out; + } + listener = NULL; + +out: + read_unlock(&tlsh_listener_lock); + return listener; +} + +/** + * tlsh_accept_enqueue - add a socket to a listener's accept_q + * @listener: listening socket + * @sk: socket to enqueue on @listener + * + * Return values: + * On success, returns 0 + * %-ENOMEM: Memory for skbs has been exhausted + */ +static int tlsh_accept_enqueue(struct sock *listener, struct sock *sk) +{ + struct sk_buff *skb; + + skb = alloc_skb(0, GFP_KERNEL); + if (!skb) + return -ENOMEM; + + sock_hold(sk); /* Ref: I */ + skb->sk = sk; + skb_queue_tail(&listener->sk_receive_queue, skb); + sk_acceptq_added(listener); + listener->sk_data_ready(listener); + return 0; +} + +/** + * tlsh_accept_dequeue - remove a socket from a listener's accept_q + * @listener: listener socket to check + * + * Caller guarantees that @listener won't disappear. + * + * Return values: + * On success, return a TCP socket waiting for TLS service + * %NULL: No sockets on the accept queue + */ +static struct sock *tlsh_accept_dequeue(struct sock *listener) +{ + struct sk_buff *skb; + struct sock *sk; + + skb = skb_dequeue(&listener->sk_receive_queue); + if (!skb) + return NULL; + sk_acceptq_removed(listener); + sock_put(listener); /* Ref: C */ + + sk = skb->sk; + skb->sk = NULL; + kfree_skb(skb); + sock_put(sk); /* Ref: I */ + return sk; +} + +static void tlsh_sock_save(struct sock *sk, + struct tlsh_sock_info *info) +{ + sock_hold(sk); /* Ref: J */ + + write_lock_bh(&sk->sk_callback_lock); + info->tsi_saved_wq = sk->sk_wq_raw; + info->tsi_saved_socket = sk->sk_socket; + info->tsi_saved_uid = sk->sk_uid; + sk->sk_tlsh_priv = info; + write_unlock_bh(&sk->sk_callback_lock); +} + +static void tlsh_sock_clear(struct sock *sk) +{ + struct tlsh_sock_info *info = sk->sk_tlsh_priv; + + write_lock_bh(&sk->sk_callback_lock); + sk->sk_tlsh_priv = NULL; + write_unlock_bh(&sk->sk_callback_lock); + tlsh_sock_info_destroy(info); + sock_put(sk); /* Ref: J (err) */ +} + +static void tlsh_sock_restore_locked(struct sock *sk) +{ + struct tlsh_sock_info *info = sk->sk_tlsh_priv; + + sk->sk_wq_raw = info->tsi_saved_wq; + sk->sk_socket = info->tsi_saved_socket; + sk->sk_uid = info->tsi_saved_uid; + sk->sk_tlsh_priv = NULL; +} + +static bool tlsh_crypto_info_initialized(struct sock *sk) +{ + struct tls_context *ctx = tls_get_ctx(sk); + + return ctx != NULL && + TLS_CRYPTO_INFO_READY(&ctx->crypto_send.info) && + TLS_CRYPTO_INFO_READY(&ctx->crypto_recv.info); +} + +/** + * tlsh_handshake_done - call the registered "done" callback for @sk. + * @sk: socket that was requesting a handshake + * + * Return values: + * %true: Handshake callback was called + * %false: No handshake callback was set, no-op + */ +static bool tlsh_handshake_done(struct sock *sk) +{ + struct tlsh_sock_info *info; + void (*done)(void *data, int status); + void *data; + + write_lock_bh(&sk->sk_callback_lock); + info = sk->sk_tlsh_priv; + if (info) { + done = info->tsi_handshake_done; + data = info->tsi_handshake_data; + + tlsh_sock_restore_locked(sk); + + if (tlsh_crypto_info_initialized(sk)) { + done(data, 0); + } else { + done(data, -EACCES); + } + } + write_unlock_bh(&sk->sk_callback_lock); + + if (info) { + tlsh_sock_info_destroy(info); + sock_put(sk); /* Ref: J */ + return true; + } + return false; +} + +/** + * tlsh_accept_drain - clean up children queued for accept + * @listener: listener socket to drain + * + */ +static void tlsh_accept_drain(struct sock *listener) +{ + struct sock *sk; + + while ((sk = tlsh_accept_dequeue(listener))) + tlsh_handshake_done(sk); +} + +/** + * tlsh_release - free an AF_TLSH socket + * @sock: socket to release + * + * Return values: + * %0: success + */ +static int tlsh_release(struct socket *sock) +{ + struct sock *sk = sock->sk; + struct tlsh_sock *tsk = tlsh_sk(sk); + + if (!sk) + return 0; + + switch (sk->sk_family) { + case AF_INET: + if (!tlsh_handshake_done(sk)) + return inet_release(sock); + return 0; +#if IS_ENABLED(CONFIG_IPV6) + case AF_INET6: + if (!tlsh_handshake_done(sk)) + return inet6_release(sock); + return 0; +#endif + case AF_TLSH: + break; + default: + return 0; + } + + sock_hold(sk); /* Ref: D */ + sock_orphan(sk); + lock_sock(sk); + + tlsh_unregister_listener(sk); + tlsh_accept_drain(sk); + + sk->sk_state = TCP_CLOSE; + sk->sk_shutdown |= SEND_SHUTDOWN; + sk->sk_state_change(sk); + + tsk->th_bind_family = AF_UNSPEC; + sock->sk = NULL; + release_sock(sk); + sock_put(sk); /* Ref: D */ + + sock_put(sk); /* Ref: A */ + return 0; +} + +/** + * tlsh_bind - bind a name to an AF_TLSH socket + * @sock: socket to be bound + * @uaddr: address to bind to + * @addrlen: length in bytes of @uaddr + * + * Binding an AF_TLSH socket defines the family of addresses that + * are able to be accept(2)'d. So, AF_INET for ipv4, AF_INET6 for + * ipv6. + * + * Return values: + * %0: binding was successful. + * %-EPERM: Caller not privileged + * %-EINVAL: Family of @sock or @uaddr not supported + */ +static int tlsh_bind(struct socket *sock, struct sockaddr *uaddr, int addrlen) +{ + struct sock *listener, *sk = sock->sk; + struct tlsh_sock *tsk = tlsh_sk(sk); + + if (!capable(CAP_NET_BIND_SERVICE)) + return -EPERM; + + switch (uaddr->sa_family) { + case AF_INET: + if (addrlen != sizeof(struct sockaddr_in)) + return -EINVAL; + break; +#if IS_ENABLED(CONFIG_IPV6) + case AF_INET6: + if (addrlen != sizeof(struct sockaddr_in6)) + return -EINVAL; + break; +#endif + default: + return -EAFNOSUPPORT; + } + + listener = tlsh_find_listener(sock_net(sk), uaddr->sa_family); + if (listener) { + sock_put(listener); /* Ref: C */ + return -EADDRINUSE; + } + + tsk->th_bind_family = uaddr->sa_family; + return 0; +} + +/** + * tlsh_accept - return a connection waiting for a TLS handshake + * @listener: listener socket which connection requests arrive on + * @newsock: socket to move incoming connection to + * @flags: SOCK_NONBLOCK and/or SOCK_CLOEXEC + * @kern: "boolean": 1 for kernel-internal sockets + * + * Return values: + * %0: @newsock has been initialized. + * %-EPERM: caller is not privileged + */ +static int tlsh_accept(struct socket *listener, struct socket *newsock, int flags, + bool kern) +{ + struct sock *sk = listener->sk, *newsk; + DECLARE_WAITQUEUE(wait, current); + long timeo; + int rc; + + rc = -EPERM; + if (!capable(CAP_NET_BIND_SERVICE)) + goto out; + + lock_sock(sk); + + if (sk->sk_state != TCP_LISTEN) { + rc = -EBADF; + goto out_release; + } + + timeo = sock_rcvtimeo(sk, flags & O_NONBLOCK); + + rc = 0; + add_wait_queue_exclusive(sk_sleep(sk), &wait); + while (!(newsk = tlsh_accept_dequeue(sk))) { + set_current_state(TASK_INTERRUPTIBLE); + if (!timeo) { + rc = -EAGAIN; + break; + } + release_sock(sk); + + timeo = schedule_timeout(timeo); + + lock_sock(sk); + if (sk->sk_state != TCP_LISTEN) { + rc = -EBADF; + break; + } + if (signal_pending(current)) { + rc = sock_intr_errno(timeo); + break; + } + } + set_current_state(TASK_RUNNING); + remove_wait_queue(sk_sleep(sk), &wait); + if (rc) { + tlsh_handshake_done(sk); + goto out_release; + } + + sock_graft(newsk, newsock); + +out_release: + release_sock(sk); +out: + return rc; +} + +/** + * tlsh_getname - retrieve src/dst address information from an AF_TLSH socket + * @sock: socket to query + * @uaddr: buffer to fill in + * @peer: value indicates which address to retrieve + * + * Return values: + * On success, a positive length of the address in @uaddr + * On error, a negative errno + */ +static int tlsh_getname(struct socket *sock, struct sockaddr *uaddr, int peer) +{ + struct sock *sk = sock->sk; + + switch (sk->sk_family) { + case AF_INET: + return inet_getname(sock, uaddr, peer); +#if IS_ENABLED(CONFIG_IPV6) + case AF_INET6: + return inet6_getname(sock, uaddr, peer); +#endif + default: + return -EOPNOTSUPP; + } +} + +/** + * tlsh_poll - check for data ready on an AF_TLSH socket + * @file: file to check for work + * @sock: socket associated with @file + * @wait: poll table + * + * Return values: + * A mask of flags indicating what type of I/O is ready + */ +static __poll_t tlsh_poll(struct file *file, struct socket *sock, + poll_table *wait) +{ + struct sock *sk = sock->sk; + __poll_t mask; + + sock_poll_wait(file, sock, wait); + + mask = 0; + + if (sk->sk_state == TCP_LISTEN) { + if (!skb_queue_empty_lockless(&sk->sk_receive_queue)) + mask |= EPOLLIN | EPOLLRDNORM; + if (sk_is_readable(sk)) + mask |= EPOLLIN | EPOLLRDNORM; + return mask; + } + + if (sk->sk_shutdown == SHUTDOWN_MASK || sk->sk_state == TCP_CLOSE) + mask |= EPOLLHUP; + if (sk->sk_shutdown & RCV_SHUTDOWN) + mask |= EPOLLIN | EPOLLRDNORM | EPOLLRDHUP; + + if (!skb_queue_empty_lockless(&sk->sk_receive_queue)) + mask |= EPOLLIN | EPOLLRDNORM; + if (sk_is_readable(sk)) + mask |= EPOLLIN | EPOLLRDNORM; + + /* This barrier is coupled with smp_wmb() in tcp_reset() */ + smp_rmb(); + if (sk->sk_err || !skb_queue_empty_lockless(&sk->sk_error_queue)) + mask |= EPOLLERR; + + return mask; +} + +/** + * tlsh_listen - move an AF_TLSH socket into a listening state + * @sock: socket to transition to listening state + * @backlog: size of backlog queue + * + * Return values: + * %0: @sock is now in a listening state + * %-EPERM: caller is not privileged + * %-EOPNOTSUPP: @sock is not of a type that supports the listen() operation + */ +static int tlsh_listen(struct socket *sock, int backlog) +{ + struct sock *sk = sock->sk; + unsigned char old_state; + int rc; + + if (!capable(CAP_NET_BIND_SERVICE)) + return -EPERM; + + lock_sock(sk); + + rc = -EOPNOTSUPP; + if (sock->state != SS_UNCONNECTED || sock->type != SOCK_STREAM) + goto out; + old_state = sk->sk_state; + if (!((1 << old_state) & (TCPF_CLOSE | TCPF_LISTEN))) + goto out; + + sk->sk_max_ack_backlog = backlog; + sk->sk_state = TCP_LISTEN; + tlsh_register_listener(sk); + + rc = 0; + +out: + release_sock(sk); + return rc; +} + +/** + * tlsh_shutdown - Shutdown an AF_TLSH socket + * @sock: socket to shut down + * @how: mask + * + * Return values: + * %0: Success + * %-EINVAL: @sock is not of a type that supports a shutdown + */ +static int tlsh_shutdown(struct socket *sock, int how) +{ + struct sock *sk = sock->sk; + + switch (sk->sk_family) { + case AF_INET: + break; +#if IS_ENABLED(CONFIG_IPV6) + case AF_INET6: + break; +#endif + default: + return -EINVAL; + } + + return inet_shutdown(sock, how); +} + +/** + * tlsh_setsockopt - Set a socket option on an AF_TLSH socket + * @sock: socket to act upon + * @level: which network layer to act upon + * @optname: which option to set + * @optval: new value to set + * @optlen: the size of the new value, in bytes + * + * Return values: + * %0: Success + * %-ENOPROTOOPT: The option is unknown at the level indicated. + */ +static int tlsh_setsockopt(struct socket *sock, int level, int optname, + sockptr_t optval, unsigned int optlen) +{ + struct sock *sk = sock->sk; + + switch (sk->sk_family) { + case AF_INET: + break; +#if IS_ENABLED(CONFIG_IPV6) + case AF_INET6: + break; +#endif + default: + return -ENOPROTOOPT; + } + + return sock_common_setsockopt(sock, level, optname, optval, optlen); +} + +static int tlsh_getsockopt_priorities(struct sock *sk, char __user *optval, + int __user *optlen) +{ + struct tlsh_sock_info *info; + int outlen, len, ret; + const char *val; + + if (get_user(len, optlen)) + return -EFAULT; + if (!optval) + return -EINVAL; + + ret = 0; + + sock_hold(sk); + write_lock_bh(&sk->sk_callback_lock); + + info = sk->sk_tlsh_priv; + if (info) { + val = info->tsi_tls_priorities; + } else { + write_unlock_bh(&sk->sk_callback_lock); + ret = -EBUSY; + goto out_put; + } + + write_unlock_bh(&sk->sk_callback_lock); + + if (val) { + outlen = strlen(val); + if (len < outlen) + ret = -EINVAL; + else if (copy_to_user(optval, val, outlen)) + ret = -EFAULT; + } else { + outlen = 0; + } + + + if (put_user(outlen, optlen)) + ret = -EFAULT; + +out_put: + sock_put(sk); + return ret; +} + +static int tlsh_getsockopt_peerid(struct sock *sk, char __user *optval, + int __user *optlen) +{ + struct tlsh_sock_info *info; + int len, val; + + if (get_user(len, optlen)) + return -EFAULT; + if (!optval || (len < sizeof(key_serial_t))) + return -EINVAL; + + write_lock_bh(&sk->sk_callback_lock); + info = sk->sk_tlsh_priv; + if (info) { + val = info->tsi_peerid; + } else { + write_unlock_bh(&sk->sk_callback_lock); + return -EBUSY; + } + write_unlock_bh(&sk->sk_callback_lock); + + if (put_user(len, optlen)) + return -EFAULT; + if (copy_to_user(optval, &val, len)) + return -EFAULT; + return 0; +} + +static int tlsh_getsockopt_type(struct sock *sk, char __user *optval, + int __user *optlen) +{ + struct tlsh_sock_info *info; + int len, val; + + if (get_user(len, optlen)) + return -EFAULT; + if (!optval || (len < sizeof(key_serial_t))) + return -EINVAL; + + write_lock_bh(&sk->sk_callback_lock); + info = sk->sk_tlsh_priv; + if (info) { + val = info->tsi_handshake_type; + } else { + write_unlock_bh(&sk->sk_callback_lock); + return -EBUSY; + } + write_unlock_bh(&sk->sk_callback_lock); + + if (put_user(len, optlen)) + return -EFAULT; + if (copy_to_user(optval, &val, len)) + return -EFAULT; + return 0; +} + +static int tlsh_getsockopt_cert(struct sock *sk, char __user *optval, + int __user *optlen) +{ + struct tlsh_sock_info *info; + int len, val; + + if (get_user(len, optlen)) + return -EFAULT; + if (!optval || (len < sizeof(key_serial_t))) + return -EINVAL; + + write_lock_bh(&sk->sk_callback_lock); + info = sk->sk_tlsh_priv; + if (info) { + val = info->tsi_certificate; + } else { + write_unlock_bh(&sk->sk_callback_lock); + return -EBUSY; + } + write_unlock_bh(&sk->sk_callback_lock); + + if (put_user(len, optlen)) + return -EFAULT; + if (copy_to_user(optval, &val, len)) + return -EFAULT; + return 0; +} + +/** + * tlsh_getsockopt - Retrieve a socket option from an AF_TLSH socket + * @sock: socket to act upon + * @level: which network layer to act upon + * @optname: which option to retrieve + * @optval: a buffer into which to receive the option's value + * @optlen: the size of the receive buffer, in bytes + * + * Return values: + * %0: Success + * %-ENOPROTOOPT: The option is unknown at the level indicated. + * %-EINVAL: Invalid argument + * %-EFAULT: Output memory not write-able + * %-EBUSY: Option value not available + */ +static int tlsh_getsockopt(struct socket *sock, int level, int optname, + char __user *optval, int __user *optlen) +{ + struct sock *sk = sock->sk; + int ret; + + switch (sk->sk_family) { + case AF_INET: + break; +#if IS_ENABLED(CONFIG_IPV6) + case AF_INET6: + break; +#endif + default: + return -ENOPROTOOPT; + } + + if (level != SOL_TLSH) + return sock_common_getsockopt(sock, level, optname, optval, optlen); + + switch (optname) { + case TLSH_PRIORITIES: + ret = tlsh_getsockopt_priorities(sk, optval, optlen); + break; + case TLSH_PEERID: + ret = tlsh_getsockopt_peerid(sk, optval, optlen); + break; + case TLSH_HANDSHAKE_TYPE: + ret = tlsh_getsockopt_type(sk, optval, optlen); + break; + case TLSH_X509_CERTIFICATE: + ret = tlsh_getsockopt_cert(sk, optval, optlen); + break; + default: + ret = -ENOPROTOOPT; + } + + return ret; +} + +/** + * tlsh_sendmsg - Send a message on an AF_TLSH socket + * @sock: socket to send on + * @msg: message to send + * @size: size of message, in bytes + * + * Return values: + * %0: Success + * %-EOPNOTSUPP: Address family does not support this operation + */ +static int tlsh_sendmsg(struct socket *sock, struct msghdr *msg, size_t size) +{ + struct sock *sk = sock->sk; + + switch (sk->sk_family) { + case AF_INET: + break; +#if IS_ENABLED(CONFIG_IPV6) + case AF_INET6: + break; +#endif + default: + return -EOPNOTSUPP; + } + + if (unlikely(inet_send_prepare(sk))) + return -EAGAIN; + return sk->sk_prot->sendmsg(sk, msg, size); +} + +/** + * tlsh_recvmsg - Receive a message from an AF_TLSH socket + * @sock: socket to receive from + * @msg: buffer into which to receive + * @size: size of buffer, in bytes + * @flags: control settings + * + * Return values: + * %0: Success + * %-EOPNOTSUPP: Address family does not support this operation + */ +static int tlsh_recvmsg(struct socket *sock, struct msghdr *msg, size_t size, + int flags) +{ + struct sock *sk = sock->sk; + + switch (sk->sk_family) { + case AF_INET: + break; +#if IS_ENABLED(CONFIG_IPV6) + case AF_INET6: + break; +#endif + default: + return -EOPNOTSUPP; + } + + if (likely(!(flags & MSG_ERRQUEUE))) + sock_rps_record_flow(sk); + return sock_common_recvmsg(sock, msg, size, flags); +} + +static const struct proto_ops tlsh_proto_ops = { + .family = PF_TLSH, + .owner = THIS_MODULE, + + .release = tlsh_release, + .bind = tlsh_bind, + .connect = sock_no_connect, + .socketpair = sock_no_socketpair, + .accept = tlsh_accept, + .getname = tlsh_getname, + .poll = tlsh_poll, + .ioctl = sock_no_ioctl, + .gettstamp = sock_gettstamp, + .listen = tlsh_listen, + .shutdown = tlsh_shutdown, + .setsockopt = tlsh_setsockopt, + .getsockopt = tlsh_getsockopt, + .sendmsg = tlsh_sendmsg, + .recvmsg = tlsh_recvmsg, + .mmap = sock_no_mmap, + .sendpage = sock_no_sendpage, +}; + +static struct proto tlsh_prot = { + .name = "TLSH", + .owner = THIS_MODULE, + .obj_size = sizeof(struct tlsh_sock), +}; + +/** + * tlsh_pf_create - create an AF_TLSH socket + * @net: network namespace to own the new socket + * @sock: socket to initialize + * @protocol: IP protocol number (ignored) + * @kern: "boolean": 1 for kernel-internal sockets + * + * Return values: + * %0: @sock was initialized, and module ref count incremented. + * Negative errno values indicate initialization failed. + */ +int tlsh_pf_create(struct net *net, struct socket *sock, int protocol, int kern) +{ + struct sock *sk; + int rc; + + if (protocol != IPPROTO_TCP) + return -EPROTONOSUPPORT; + + /* only stream sockets are supported */ + if (sock->type != SOCK_STREAM) + return -ESOCKTNOSUPPORT; + + sock->state = SS_UNCONNECTED; + sock->ops = &tlsh_proto_ops; + + /* Ref: A */ + sk = sk_alloc(net, PF_TLSH, GFP_KERNEL, &tlsh_prot, kern); + if (!sk) + return -ENOMEM; + + sock_init_data(sock, sk); + if (sk->sk_prot->init) { + rc = sk->sk_prot->init(sk); + if (rc) + goto err_sk_put; + } + + tlsh_sk(sk)->th_bind_family = AF_UNSPEC; + return 0; + +err_sk_put: + sock_orphan(sk); + sk_free(sk); /* Ref: A (err) */ + return rc; +} + +/** + * tls_client_hello_x509 - request an x.509-based TLS handshake on a socket + * @sock: connected socket on which to perform the handshake + * @done: function to call when the handshake has completed + * @data: token to pass back to @done + * @priorities: GnuTLS TLS priorities string + * @peerid: serial number of key containing private key + * @cert: serial number of key containing client's x.509 certificate + * + * Return values: + * %0: Handshake request enqueue; ->done will be called when complete + * %-ENOENT: No user agent is available + * %-ENOMEM: Memory allocation failed + */ +int tls_client_hello_x509(struct socket *sock, void (*done)(void *data, int status), + void *data, const char *priorities, key_serial_t peerid, + key_serial_t cert) +{ + struct sock *listener, *sk = sock->sk; + struct tlsh_sock_info *info; + int rc; + + listener = tlsh_find_listener(sock_net(sk), sk->sk_family); + if (!listener) + return -ENOENT; + + info = kzalloc(sizeof(*info), GFP_KERNEL); + if (!info) { + sock_put(listener); /* Ref: C (err) */ + return -ENOMEM; + } + + info->tsi_handshake_done = done; + info->tsi_handshake_data = data; + if (priorities && strlen(priorities)) { + info->tsi_tls_priorities = kstrdup(priorities, GFP_KERNEL); + if (!info->tsi_tls_priorities) { + tlsh_sock_info_destroy(info); + sock_put(listener); /* Ref: C (err) */ + return -ENOMEM; + } + } + info->tsi_peerid = peerid; + info->tsi_certificate = cert; + info->tsi_handshake_type = TLSH_TYPE_CLIENTHELLO_X509; + tlsh_sock_save(sk, info); + + rc = tlsh_accept_enqueue(listener, sk); + if (rc) { + tlsh_sock_clear(sk); + sock_put(listener); /* Ref: C (err) */ + } + + return rc; +} +EXPORT_SYMBOL_GPL(tls_client_hello_x509); + +/** + * tls_client_hello_psk - request a PSK-based TLS handshake on a socket + * @sock: connected socket on which to perform the handshake + * @done: function to call when the handshake has completed + * @data: token to pass back to @done + * @priorities: GnuTLS TLS priorities string + * @peerid: serial number of key containing TLS identity + * + * Return values: + * %0: Handshake request enqueue; ->done will be called when complete + * %-ENOENT: No user agent is available + * %-ENOMEM: Memory allocation failed + */ +int tls_client_hello_psk(struct socket *sock, void (*done)(void *data, int status), + void *data, const char *priorities, key_serial_t peerid) +{ + struct sock *listener, *sk = sock->sk; + struct tlsh_sock_info *info; + int rc; + + listener = tlsh_find_listener(sock_net(sk), sk->sk_family); + if (!listener) + return -ENOENT; + + info = kzalloc(sizeof(*info), GFP_KERNEL); + if (!info) { + sock_put(listener); /* Ref: C (err) */ + return -ENOMEM; + } + + info->tsi_handshake_done = done; + info->tsi_handshake_data = data; + if (priorities && strlen(priorities)) { + info->tsi_tls_priorities = kstrdup(priorities, GFP_KERNEL); + if (!info->tsi_tls_priorities) { + tlsh_sock_info_destroy(info); + sock_put(listener); /* Ref: C (err) */ + return -ENOMEM; + } + } + info->tsi_peerid = peerid; + info->tsi_handshake_type = TLSH_TYPE_CLIENTHELLO_PSK; + tlsh_sock_save(sk, info); + + rc = tlsh_accept_enqueue(listener, sk); + if (rc) { + tlsh_sock_clear(sk); + sock_put(listener); /* Ref: C (err) */ + } + + return rc; +} +EXPORT_SYMBOL_GPL(tls_client_hello_psk); diff --git a/net/tls/tls_main.c b/net/tls/tls_main.c index 7eca4d9a83c4..c5e0a7b3aa2e 100644 --- a/net/tls/tls_main.c +++ b/net/tls/tls_main.c @@ -49,6 +49,7 @@ MODULE_AUTHOR("Mellanox Technologies"); MODULE_DESCRIPTION("Transport Layer Security Support"); MODULE_LICENSE("Dual BSD/GPL"); MODULE_ALIAS_TCP_ULP("tls"); +MODULE_ALIAS_NETPROTO(PF_TLSH); enum { TLSV4, @@ -982,6 +983,12 @@ static struct tcp_ulp_ops tcp_tls_ulp_ops __read_mostly = { .get_info_size = tls_get_info_size, }; +static const struct net_proto_family tlsh_pf_ops = { + .family = PF_TLSH, + .create = tlsh_pf_create, + .owner = THIS_MODULE, +}; + static int __init tls_register(void) { int err; @@ -993,11 +1000,14 @@ static int __init tls_register(void) tls_device_init(); tcp_register_ulp(&tcp_tls_ulp_ops); + sock_register(&tlsh_pf_ops); + return 0; } static void __exit tls_unregister(void) { + sock_unregister(PF_TLSH); tcp_unregister_ulp(&tcp_tls_ulp_ops); tls_device_cleanup(); unregister_pernet_subsys(&tls_proc_ops); From patchwork Mon Apr 18 16:49:57 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Chuck Lever III X-Patchwork-Id: 12816869 X-Patchwork-Delegate: kuba@kernel.org Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 5287BC4332F for ; Mon, 18 Apr 2022 16:51:49 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1346486AbiDRQyY (ORCPT ); Mon, 18 Apr 2022 12:54:24 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:52830 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1346481AbiDRQws (ORCPT ); Mon, 18 Apr 2022 12:52:48 -0400 Received: from ams.source.kernel.org (ams.source.kernel.org [145.40.68.75]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 4D7C82B26B; Mon, 18 Apr 2022 09:50:02 -0700 (PDT) Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ams.source.kernel.org (Postfix) with ESMTPS id A76C7B8100E; Mon, 18 Apr 2022 16:50:00 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id B7A2DC385A1; Mon, 18 Apr 2022 16:49:58 +0000 (UTC) Subject: [PATCH RFC 5/5] net/tls: Add observability for AF_TLSH sockets From: Chuck Lever To: netdev@vger.kernel.org, linux-nfs@vger.kernel.org, linux-nvme@lists.infradead.org, linux-cifs@vger.kernel.org, linux-fsdevel@vger.kernel.org Cc: ak@tempesta-tech.com, borisp@nvidia.com, simo@redhat.com Date: Mon, 18 Apr 2022 12:49:57 -0400 Message-ID: <165030059773.5073.6168640435213548957.stgit@oracle-102.nfsv4.dev> In-Reply-To: <165030051838.5073.8699008789153780301.stgit@oracle-102.nfsv4.dev> References: <165030051838.5073.8699008789153780301.stgit@oracle-102.nfsv4.dev> User-Agent: StGit/1.5 MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org X-Patchwork-Delegate: kuba@kernel.org X-Patchwork-State: RFC Signed-off-by: Chuck Lever Reviewed-by: Hannes Reinecke --- net/tls/af_tlsh.c | 50 +++++++ net/tls/trace.c | 3 net/tls/trace.h | 355 +++++++++++++++++++++++++++++++++++++++++++++++++++++ 3 files changed, 402 insertions(+), 6 deletions(-) diff --git a/net/tls/af_tlsh.c b/net/tls/af_tlsh.c index 4d1c1de3a474..31eae3e0d54c 100644 --- a/net/tls/af_tlsh.c +++ b/net/tls/af_tlsh.c @@ -239,8 +239,10 @@ static bool tlsh_handshake_done(struct sock *sk) tlsh_sock_restore_locked(sk); if (tlsh_crypto_info_initialized(sk)) { + trace_tlsh_handshake_ok(sk); done(data, 0); } else { + trace_tlsh_handshake_failed(sk); done(data, -EACCES); } } @@ -282,6 +284,8 @@ static int tlsh_release(struct socket *sock) if (!sk) return 0; + trace_tlsh_release(sock); + switch (sk->sk_family) { case AF_INET: if (!tlsh_handshake_done(sk)) @@ -364,6 +368,7 @@ static int tlsh_bind(struct socket *sock, struct sockaddr *uaddr, int addrlen) } tsk->th_bind_family = uaddr->sa_family; + trace_tlsh_bind(sock); return 0; } @@ -386,6 +391,8 @@ static int tlsh_accept(struct socket *listener, struct socket *newsock, int flag long timeo; int rc; + trace_tlsh_accept(listener); + rc = -EPERM; if (!capable(CAP_NET_BIND_SERVICE)) goto out; @@ -429,6 +436,7 @@ static int tlsh_accept(struct socket *listener, struct socket *newsock, int flag } sock_graft(newsk, newsock); + trace_tlsh_newsock(newsock, newsk); out_release: release_sock(sk); @@ -450,6 +458,8 @@ static int tlsh_getname(struct socket *sock, struct sockaddr *uaddr, int peer) { struct sock *sk = sock->sk; + trace_tlsh_getname(sock); + switch (sk->sk_family) { case AF_INET: return inet_getname(sock, uaddr, peer); @@ -486,6 +496,7 @@ static __poll_t tlsh_poll(struct file *file, struct socket *sock, mask |= EPOLLIN | EPOLLRDNORM; if (sk_is_readable(sk)) mask |= EPOLLIN | EPOLLRDNORM; + trace_tlsh_poll_listener(sock, mask); return mask; } @@ -504,6 +515,7 @@ static __poll_t tlsh_poll(struct file *file, struct socket *sock, if (sk->sk_err || !skb_queue_empty_lockless(&sk->sk_error_queue)) mask |= EPOLLERR; + trace_tlsh_poll(sock, mask); return mask; } @@ -539,6 +551,7 @@ static int tlsh_listen(struct socket *sock, int backlog) sk->sk_state = TCP_LISTEN; tlsh_register_listener(sk); + trace_tlsh_listen(sock); rc = 0; out: @@ -559,6 +572,8 @@ static int tlsh_shutdown(struct socket *sock, int how) { struct sock *sk = sock->sk; + trace_tlsh_shutdown(sock); + switch (sk->sk_family) { case AF_INET: break; @@ -590,6 +605,8 @@ static int tlsh_setsockopt(struct socket *sock, int level, int optname, { struct sock *sk = sock->sk; + trace_tlsh_setsockopt(sock); + switch (sk->sk_family) { case AF_INET: break; @@ -756,6 +773,8 @@ static int tlsh_getsockopt(struct socket *sock, int level, int optname, struct sock *sk = sock->sk; int ret; + trace_tlsh_getsockopt(sock); + switch (sk->sk_family) { case AF_INET: break; @@ -803,6 +822,9 @@ static int tlsh_getsockopt(struct socket *sock, int level, int optname, static int tlsh_sendmsg(struct socket *sock, struct msghdr *msg, size_t size) { struct sock *sk = sock->sk; + int ret; + + trace_tlsh_sendmsg_start(sock, size); switch (sk->sk_family) { case AF_INET: @@ -812,12 +834,19 @@ static int tlsh_sendmsg(struct socket *sock, struct msghdr *msg, size_t size) break; #endif default: - return -EOPNOTSUPP; + ret = -EOPNOTSUPP; + goto out; + } + + if (unlikely(inet_send_prepare(sk))) { + ret = -EAGAIN; + goto out; } + ret = sk->sk_prot->sendmsg(sk, msg, size); - if (unlikely(inet_send_prepare(sk))) - return -EAGAIN; - return sk->sk_prot->sendmsg(sk, msg, size); +out: + trace_tlsh_sendmsg_result(sock, ret); + return ret; } /** @@ -835,6 +864,9 @@ static int tlsh_recvmsg(struct socket *sock, struct msghdr *msg, size_t size, int flags) { struct sock *sk = sock->sk; + int ret; + + trace_tlsh_recvmsg_start(sock, size); switch (sk->sk_family) { case AF_INET: @@ -844,12 +876,17 @@ static int tlsh_recvmsg(struct socket *sock, struct msghdr *msg, size_t size, break; #endif default: - return -EOPNOTSUPP; + ret = -EOPNOTSUPP; + goto out; } if (likely(!(flags & MSG_ERRQUEUE))) sock_rps_record_flow(sk); - return sock_common_recvmsg(sock, msg, size, flags); + ret = sock_common_recvmsg(sock, msg, size, flags); + +out: + trace_tlsh_recvmsg_result(sock, ret); + return ret; } static const struct proto_ops tlsh_proto_ops = { @@ -920,6 +957,7 @@ int tlsh_pf_create(struct net *net, struct socket *sock, int protocol, int kern) } tlsh_sk(sk)->th_bind_family = AF_UNSPEC; + trace_tlsh_pf_create(sock); return 0; err_sk_put: diff --git a/net/tls/trace.c b/net/tls/trace.c index e374913cf9c9..3747ca2ede67 100644 --- a/net/tls/trace.c +++ b/net/tls/trace.c @@ -2,6 +2,9 @@ /* Copyright (C) 2019 Netronome Systems, Inc. */ #include +#include +#include +#include #ifndef __CHECKER__ #define CREATE_TRACE_POINTS diff --git a/net/tls/trace.h b/net/tls/trace.h index 9ba5f600ea43..9302d992edad 100644 --- a/net/tls/trace.h +++ b/net/tls/trace.h @@ -12,6 +12,56 @@ struct sock; +#define show_af_family(family) \ + __print_symbolic(family, \ + { AF_INET, "AF_INET" }, \ + { AF_INET6, "AF_INET6" }, \ + { AF_TLSH, "AF_TLSH" }) + +TRACE_DEFINE_ENUM(TCP_ESTABLISHED); +TRACE_DEFINE_ENUM(TCP_SYN_SENT); +TRACE_DEFINE_ENUM(TCP_SYN_RECV); +TRACE_DEFINE_ENUM(TCP_FIN_WAIT1); +TRACE_DEFINE_ENUM(TCP_FIN_WAIT2); +TRACE_DEFINE_ENUM(TCP_TIME_WAIT); +TRACE_DEFINE_ENUM(TCP_CLOSE); +TRACE_DEFINE_ENUM(TCP_CLOSE_WAIT); +TRACE_DEFINE_ENUM(TCP_LAST_ACK); +TRACE_DEFINE_ENUM(TCP_LISTEN); +TRACE_DEFINE_ENUM(TCP_CLOSING); +TRACE_DEFINE_ENUM(TCP_NEW_SYN_RECV); + +#define show_tcp_state(state) \ + __print_symbolic(state, \ + { TCP_ESTABLISHED, "ESTABLISHED" }, \ + { TCP_SYN_SENT, "SYN_SENT" }, \ + { TCP_SYN_RECV, "SYN_RECV" }, \ + { TCP_FIN_WAIT1, "FIN_WAIT1" }, \ + { TCP_FIN_WAIT2, "FIN_WAIT2" }, \ + { TCP_TIME_WAIT, "TIME_WAIT" }, \ + { TCP_CLOSE, "CLOSE" }, \ + { TCP_CLOSE_WAIT, "CLOSE_WAIT" }, \ + { TCP_LAST_ACK, "LAST_ACK" }, \ + { TCP_LISTEN, "LISTEN" }, \ + { TCP_CLOSING, "CLOSING" }, \ + { TCP_NEW_SYN_RECV, "NEW_SYN_RECV" }) + +#define show_poll_event_mask(mask) \ + __print_flags(mask, "|", \ + { EPOLLIN, "IN" }, \ + { EPOLLPRI, "PRI" }, \ + { EPOLLOUT, "OUT" }, \ + { EPOLLERR, "ERR" }, \ + { EPOLLHUP, "HUP" }, \ + { EPOLLNVAL, "NVAL" }, \ + { EPOLLRDNORM, "RDNORM" }, \ + { EPOLLRDBAND, "RDBAND" }, \ + { EPOLLWRNORM, "WRNORM" }, \ + { EPOLLWRBAND, "WRBAND" }, \ + { EPOLLMSG, "MSG" }, \ + { EPOLLRDHUP, "RDHUP" }) + + TRACE_EVENT(tls_device_offload_set, TP_PROTO(struct sock *sk, int dir, u32 tcp_seq, u8 *rec_no, int ret), @@ -192,6 +242,311 @@ TRACE_EVENT(tls_device_tx_resync_send, ) ); +DECLARE_EVENT_CLASS(tlsh_listener_class, + TP_PROTO( + const struct socket *sock + ), + TP_ARGS(sock), + TP_STRUCT__entry( + __field(const struct socket *, sock) + __field(const struct sock *, sk) + __field(int, refcount) + __field(unsigned long, family) + ), + TP_fast_assign( + const struct sock *sk = sock->sk; + + __entry->sock = sock; + __entry->sk = sk; + __entry->refcount = refcount_read(&sk->sk_refcnt); + __entry->family = tlsh_sk((struct sock *)sk)->th_bind_family; + ), + + TP_printk("listener=%p sk=%p(%d) family=%s", + __entry->sock, __entry->sk, + __entry->refcount, show_af_family(__entry->family) + ) +); + +#define DEFINE_TLSH_LISTENER_EVENT(name) \ + DEFINE_EVENT(tlsh_listener_class, name, \ + TP_PROTO( \ + const struct socket *sock \ + ), \ + TP_ARGS(sock)) + +DEFINE_TLSH_LISTENER_EVENT(tlsh_bind); +DEFINE_TLSH_LISTENER_EVENT(tlsh_accept); +DEFINE_TLSH_LISTENER_EVENT(tlsh_listen); +DEFINE_TLSH_LISTENER_EVENT(tlsh_pf_create); + +TRACE_EVENT(tlsh_newsock, + TP_PROTO( + const struct socket *newsock, + const struct sock *newsk + ), + TP_ARGS(newsock, newsk), + TP_STRUCT__entry( + __field(const struct socket *, newsock) + __field(const struct sock *, newsk) + __field(int, refcount) + __field(unsigned long, family) + ), + TP_fast_assign( + __entry->newsock = newsock; + __entry->newsk = newsk; + __entry->refcount = refcount_read(&newsk->sk_refcnt); + __entry->family = newsk->sk_family; + ), + + TP_printk("newsock=%p newsk=%p(%d) family=%s", + __entry->newsock, __entry->newsk, + __entry->refcount, show_af_family(__entry->family) + ) +); + +DECLARE_EVENT_CLASS(tlsh_proto_op_class, + TP_PROTO( + const struct socket *sock + ), + TP_ARGS(sock), + TP_STRUCT__entry( + __field(const struct socket *, sock) + __field(const struct sock *, sk) + __field(int, refcount) + __field(unsigned long, family) + __field(unsigned long, state) + ), + TP_fast_assign( + const struct sock *sk = sock->sk; + + __entry->sock = sock; + __entry->sk = sk; + __entry->refcount = refcount_read(&sk->sk_refcnt); + __entry->family = sk->sk_family; + __entry->state = sk->sk_state; + ), + + TP_printk("sock=%p sk=%p(%d) family=%s state=%s", + __entry->sock, __entry->sk, __entry->refcount, + show_af_family(__entry->family), + show_tcp_state(__entry->state) + ) +); + +#define DEFINE_TLSH_PROTO_OP_EVENT(name) \ + DEFINE_EVENT(tlsh_proto_op_class, name, \ + TP_PROTO( \ + const struct socket *sock \ + ), \ + TP_ARGS(sock)) + +DEFINE_TLSH_PROTO_OP_EVENT(tlsh_release); +DEFINE_TLSH_PROTO_OP_EVENT(tlsh_getname); +DEFINE_TLSH_PROTO_OP_EVENT(tlsh_shutdown); +DEFINE_TLSH_PROTO_OP_EVENT(tlsh_setsockopt); +DEFINE_TLSH_PROTO_OP_EVENT(tlsh_getsockopt); + +TRACE_EVENT(tlsh_sendmsg_start, + TP_PROTO( + const struct socket *sock, + size_t size + ), + TP_ARGS(sock, size), + TP_STRUCT__entry( + __field(const struct socket *, sock) + __field(const struct sock *, sk) + __field(int, refcount) + __field(unsigned long, family) + __field(unsigned long, state) + __field(const void *, op) + __field(size_t, size) + ), + TP_fast_assign( + const struct sock *sk = sock->sk; + + __entry->sock = sock; + __entry->sk = sk; + __entry->refcount = refcount_read(&sk->sk_refcnt); + __entry->family = sk->sk_family; + __entry->state = sk->sk_state; + __entry->op = sk->sk_prot->sendmsg; + __entry->size = size; + ), + + TP_printk("sock=%p sk=%p(%d) family=%s state=%s size=%zu op=%pS", + __entry->sock, __entry->sk, __entry->refcount, + show_af_family(__entry->family), + show_tcp_state(__entry->state), + __entry->size, __entry->op + ) +); + +TRACE_EVENT(tlsh_recvmsg_start, + TP_PROTO( + const struct socket *sock, + size_t size + ), + TP_ARGS(sock, size), + TP_STRUCT__entry( + __field(const struct socket *, sock) + __field(const struct sock *, sk) + __field(int, refcount) + __field(unsigned long, family) + __field(unsigned long, state) + __field(const void *, op) + __field(size_t, size) + ), + TP_fast_assign( + const struct sock *sk = sock->sk; + + __entry->sock = sock; + __entry->sk = sk; + __entry->refcount = refcount_read(&sk->sk_refcnt); + __entry->family = sk->sk_family; + __entry->state = sk->sk_state; + __entry->op = sk->sk_prot->recvmsg; + __entry->size = size; + ), + + TP_printk("sock=%p sk=%p(%d) family=%s state=%s size=%zu op=%pS", + __entry->sock, __entry->sk, __entry->refcount, + show_af_family(__entry->family), + show_tcp_state(__entry->state), + __entry->size, __entry->op + ) +); + +DECLARE_EVENT_CLASS(tlsh_opmsg_result_class, + TP_PROTO( + const struct socket *sock, + int result + ), + TP_ARGS(sock, result), + TP_STRUCT__entry( + __field(const struct socket *, sock) + __field(const struct sock *, sk) + __field(int, refcount) + __field(unsigned long, family) + __field(unsigned long, state) + __field(int, result) + ), + TP_fast_assign( + const struct sock *sk = sock->sk; + + __entry->sock = sock; + __entry->sk = sk; + __entry->refcount = refcount_read(&sk->sk_refcnt); + __entry->family = sk->sk_family; + __entry->state = sk->sk_state; + __entry->result = result; + ), + + TP_printk("sock=%p sk=%p(%d) family=%s state=%s result=%d", + __entry->sock, __entry->sk, __entry->refcount, + show_af_family(__entry->family), + show_tcp_state(__entry->state), + __entry->result + ) +); + +#define DEFINE_TLSH_OPMSG_RESULT_EVENT(name) \ + DEFINE_EVENT(tlsh_opmsg_result_class, name, \ + TP_PROTO( \ + const struct socket *sock, \ + int result \ + ), \ + TP_ARGS(sock, result)) + +DEFINE_TLSH_OPMSG_RESULT_EVENT(tlsh_sendmsg_result); +DEFINE_TLSH_OPMSG_RESULT_EVENT(tlsh_recvmsg_result); + +TRACE_EVENT(tlsh_poll, + TP_PROTO( + const struct socket *sock, + __poll_t mask + ), + TP_ARGS(sock, mask), + TP_STRUCT__entry( + __field(const struct socket *, sock) + __field(const struct sock *, sk) + __field(int, refcount) + __field(unsigned long, mask) + ), + TP_fast_assign( + const struct sock *sk = sock->sk; + + __entry->sock = sock; + __entry->sk = sk; + __entry->refcount = refcount_read(&sk->sk_refcnt); + __entry->mask = mask; + ), + + TP_printk("sock=%p sk=%p(%d) mask=%s", + __entry->sock, __entry->sk, __entry->refcount, + show_poll_event_mask(__entry->mask) + ) +); + +TRACE_EVENT(tlsh_poll_listener, + TP_PROTO( + const struct socket *sock, + __poll_t mask + ), + TP_ARGS(sock, mask), + TP_STRUCT__entry( + __field(const struct socket *, sock) + __field(const struct sock *, sk) + __field(int, refcount) + __field(unsigned long, mask) + ), + TP_fast_assign( + const struct sock *sk = sock->sk; + + __entry->sock = sock; + __entry->sk = sk; + __entry->refcount = refcount_read(&sk->sk_refcnt); + __entry->mask = mask; + ), + + TP_printk("sock=%p sk=%p(%d) mask=%s", + __entry->sock, __entry->sk, __entry->refcount, + show_poll_event_mask(__entry->mask) + ) +); + +DECLARE_EVENT_CLASS(tlsh_handshake_done_class, + TP_PROTO( + const struct sock *sk + ), + TP_ARGS(sk), + TP_STRUCT__entry( + __field(const struct sock *, sk) + __field(int, refcount) + __field(unsigned long, family) + ), + TP_fast_assign( + __entry->sk = sk; + __entry->refcount = refcount_read(&sk->sk_refcnt); + __entry->family = sk->sk_family; + ), + + TP_printk("sk=%p(%d) family=%s", + __entry->sk, __entry->refcount, + show_af_family(__entry->family) + ) +); + +#define DEFINE_TLSH_HANDSHAKE_DONE_EVENT(name) \ + DEFINE_EVENT(tlsh_handshake_done_class, name, \ + TP_PROTO( \ + const struct sock *sk \ + ), \ + TP_ARGS(sk)) + +DEFINE_TLSH_HANDSHAKE_DONE_EVENT(tlsh_handshake_ok); +DEFINE_TLSH_HANDSHAKE_DONE_EVENT(tlsh_handshake_failed); + #endif /* _TLS_TRACE_H_ */ #undef TRACE_INCLUDE_PATH