diff mbox series

[3/3] btrfs-progs: add TLS arguments to send/receive

Message ID 20201225045037.185537-3-shngmao@gmail.com (mailing list archive)
State New, archived
Headers show
Series [1/3] btrfs-progs: add Kernel TLS to btrfs send/receive | expand

Commit Message

Sheng Mao Dec. 25, 2020, 4:50 a.m. UTC
From: Sheng Mao <shngmao@gmail.com>

If TLS is enabled, btrfs send/receive can accept four more
arguments:

- address to connect to/listen on
- port to connect to/listen on
- keyfile name (optional), PEM format is preferred
- TLS mode: TLS 1.2/1.3 and 128/256 GCM

In TLS mode, btrfs receive assumes -e and btrfs receive
stops after receiving an end cmd marker in the stream.

Issue: #326
Signed-off-by: Sheng Mao <shngmao@gmail.com>
---
 Documentation/btrfs-receive.asciidoc |  13 +++
 Documentation/btrfs-send.asciidoc    |   9 ++
 cmds/receive.c                       | 153 ++++++++++++++++++++++----
 cmds/send.c                          | 155 +++++++++++++++++++++++----
 4 files changed, 292 insertions(+), 38 deletions(-)

Comments

Wang Yugui Dec. 31, 2020, 11:16 a.m. UTC | #1
Hi, Sheng Mao

some feedback.

1, can we use 'listen-addr' for sever side, and 'conn-addr' for client
side?

2, can we support '--tls-mode none' for tcp without TLS, 
and then change 'tls-port' to 'tcp-port'? 

Is there some boost performance for tcp without TLS too?


> +--tls-addr <url>::
> +Address to listen on. It can be an IP address or a domain name.
> +
> +--tls-port <port>::
> +The local port of the TLS connection.
> +
> +--tls-key <file>::
> +Use the key from file; otherwise read key from stdin. Key file is first parsed
> +as PEM format; if parsing fails, file content is treated as binary key.
> +
> +--tls-mode <mode>::
> +Use tls_12_128_gcm, tls_13_128_gcm, tls_12_256_gcm.

Best Regards
Wang Yugui (wangyugui@e16-tech.com)
2020/12/31
Sheng Mao Dec. 31, 2020, 6:33 p.m. UTC | #2
Hi Yugui,

Thank you for the feedback!

1. Yes, we can do that. The reason why I use —tls-addr on both sides is to introduce least vocabulary for users.
2. I don’t have a 10Gpbs NIC to have a thorough benchmark on TLS vs raw sockets. The flame graph shows 
decrypt_skb_update (related to TLS decoding) takes about 3.5% of CPU time for my 1Gbps setup. The transfer 
saturates the bandwidth. Do you have any 10Gbps devices? Would you mind to help me benchmarking after 
introducing —tls-mode none?

Thank you! Happy new year!

Regards,
Sheng

> On Dec 31, 2020, at 04:16, Wang Yugui <wangyugui@e16-tech.com> wrote:
> 
> Hi, Sheng Mao
> 
> some feedback.
> 
> 1, can we use 'listen-addr' for sever side, and 'conn-addr' for client
> side?
> 
> 2, can we support '--tls-mode none' for tcp without TLS, 
> and then change 'tls-port' to 'tcp-port'? 
> 
> Is there some boost performance for tcp without TLS too?
> 
> 
>> +--tls-addr <url>::
>> +Address to listen on. It can be an IP address or a domain name.
>> +
>> +--tls-port <port>::
>> +The local port of the TLS connection.
>> +
>> +--tls-key <file>::
>> +Use the key from file; otherwise read key from stdin. Key file is first parsed
>> +as PEM format; if parsing fails, file content is treated as binary key.
>> +
>> +--tls-mode <mode>::
>> +Use tls_12_128_gcm, tls_13_128_gcm, tls_12_256_gcm.
> 
> Best Regards
> Wang Yugui (wangyugui@e16-tech.com)
> 2020/12/31
> 
>
Wang Yugui Jan. 1, 2021, 5:53 a.m. UTC | #3
Hi, Sheng

> Hi Yugui,
> 
> Thank you for the feedback!
> 
> 1. Yes, we can do that. The reason why I use ―tls-addr on both sides is to introduce least vocabulary for users.
> 2. I don’t have a 10Gpbs NIC to have a thorough benchmark on TLS vs raw sockets. The flame graph shows 
> decrypt_skb_update (related to TLS decoding) takes about 3.5% of CPU time for my 1Gbps setup. The transfer 
> saturates the bandwidth. Do you have any 10Gbps devices? Would you mind to help me benchmarking after 
> introducing ―tls-mode none?

Yes. We can benchmark this for 10G Gbps or 40Gbs.

Best Regards
Wang Yugui (wangyugui@e16-tech.com)
2021/01/01


> Thank you! Happy new year!
> 
> Regards,
> Sheng
> 
> > On Dec 31, 2020, at 04:16, Wang Yugui <wangyugui@e16-tech.com> wrote:
> > 
> > Hi, Sheng Mao
> > 
> > some feedback.
> > 
> > 1, can we use 'listen-addr' for sever side, and 'conn-addr' for client
> > side?
> > 
> > 2, can we support '--tls-mode none' for tcp without TLS, 
> > and then change 'tls-port' to 'tcp-port'? 
> > 
> > Is there some boost performance for tcp without TLS too?
> > 
> > 
> >> +--tls-addr <url>::
> >> +Address to listen on. It can be an IP address or a domain name.
> >> +
> >> +--tls-port <port>::
> >> +The local port of the TLS connection.
> >> +
> >> +--tls-key <file>::
> >> +Use the key from file; otherwise read key from stdin. Key file is first parsed
> >> +as PEM format; if parsing fails, file content is treated as binary key.
> >> +
> >> +--tls-mode <mode>::
> >> +Use tls_12_128_gcm, tls_13_128_gcm, tls_12_256_gcm.
> > 
> > Best Regards
> > Wang Yugui (wangyugui@e16-tech.com)
> > 2020/12/31
> > 
> >
Sheng Mao Jan. 2, 2021, 4:08 a.m. UTC | #4
Hi Yugui,

Happy new year!

Thank you for the help! I have updated the patches according to your request:

- use listen/conn-addr on receive/send respectively
- use tcp-port: later we can support DTLS
- support --tls-mode none: in this mode, send/receive won't check for key file
or prompt for password.

Please let me know if you have any questions! I am very curious about the
performance of kernel TLS too.

Best regards,
Sheng

On Thu, Dec 31, 2020 at 10:53 PM Wang Yugui <wangyugui@e16-tech.com> wrote:
>
> Hi, Sheng
>
> > Hi Yugui,
> >
> > Thank you for the feedback!
> >
> > 1. Yes, we can do that. The reason why I use —tls-addr on both sides is to introduce least vocabulary for users.
> > 2. I don’t have a 10Gpbs NIC to have a thorough benchmark on TLS vs raw sockets. The flame graph shows
> > decrypt_skb_update (related to TLS decoding) takes about 3.5% of CPU time for my 1Gbps setup. The transfer
> > saturates the bandwidth. Do you have any 10Gbps devices? Would you mind to help me benchmarking after
> > introducing —tls-mode none?
>
> Yes. We can benchmark this for 10G Gbps or 40Gbs.
>
> Best Regards
> Wang Yugui (wangyugui@e16-tech.com)
> 2021/01/01
>
>
> > Thank you! Happy new year!
> >
> > Regards,
> > Sheng
> >
> > > On Dec 31, 2020, at 04:16, Wang Yugui <wangyugui@e16-tech.com> wrote:
> > >
> > > Hi, Sheng Mao
> > >
> > > some feedback.
> > >
> > > 1, can we use 'listen-addr' for sever side, and 'conn-addr' for client
> > > side?
> > >
> > > 2, can we support '--tls-mode none' for tcp without TLS,
> > > and then change 'tls-port' to 'tcp-port'?
> > >
> > > Is there some boost performance for tcp without TLS too?
> > >
> > >
> > >> +--tls-addr <url>::
> > >> +Address to listen on. It can be an IP address or a domain name.
> > >> +
> > >> +--tls-port <port>::
> > >> +The local port of the TLS connection.
> > >> +
> > >> +--tls-key <file>::
> > >> +Use the key from file; otherwise read key from stdin. Key file is first parsed
> > >> +as PEM format; if parsing fails, file content is treated as binary key.
> > >> +
> > >> +--tls-mode <mode>::
> > >> +Use tls_12_128_gcm, tls_13_128_gcm, tls_12_256_gcm.
> > >
> > > Best Regards
> > > Wang Yugui (wangyugui@e16-tech.com)
> > > 2020/12/31
> > >
> > >
>
>
diff mbox series

Patch

diff --git a/Documentation/btrfs-receive.asciidoc b/Documentation/btrfs-receive.asciidoc
index e4c4d2c0..0bc70165 100644
--- a/Documentation/btrfs-receive.asciidoc
+++ b/Documentation/btrfs-receive.asciidoc
@@ -38,6 +38,19 @@  A subvolume is made read-only after the receiving process finishes successfully
 -f <FILE>::
 read the stream from <FILE> instead of stdin,
 
+--tls-addr <url>::
+Address to listen on. It can be an IP address or a domain name.
+
+--tls-port <port>::
+The local port of the TLS connection.
+
+--tls-key <file>::
+Use the key from file; otherwise read key from stdin. Key file is first parsed
+as PEM format; if parsing fails, file content is treated as binary key.
+
+--tls-mode <mode>::
+Use tls_12_128_gcm, tls_13_128_gcm, tls_12_256_gcm.
+
 -C|--chroot::
 confine the process to 'path' using `chroot`(1)
 
diff --git a/Documentation/btrfs-send.asciidoc b/Documentation/btrfs-send.asciidoc
index c4a05672..313451d5 100644
--- a/Documentation/btrfs-send.asciidoc
+++ b/Documentation/btrfs-send.asciidoc
@@ -49,6 +49,15 @@  use this snapshot as a clone source for an incremental send (multiple allowed)
 -f <outfile>::
 output is normally written to standard output so it can be, for example, piped
 to btrfs receive. Use this option to write it to a file instead.
+--tls-addr <url>::
+Address of remote receiver. It can be an IP address or a domain name.
+--tls-port <port>::
+The remote port of the TLS connection.
+--tls-key <file>::
+Use the key from file; otherwise read key from stdin. Key file is first parsed
+as PEM format; if parsing fails, file content is treated as binary key.
+--tls-mode <mode>::
+Use tls_12_128_gcm, tls_13_128_gcm, tls_12_256_gcm.
 --no-data::
 send in 'NO_FILE_DATA' mode
 +
diff --git a/cmds/receive.c b/cmds/receive.c
index 2aaba3ff..e1dc5415 100644
--- a/cmds/receive.c
+++ b/cmds/receive.c
@@ -53,8 +53,11 @@ 
 #include "common/help.h"
 #include "common/path-utils.h"
 
-struct btrfs_receive
-{
+#if KTLS_SEND_RECV == 1
+#include "common/ktls.h"
+#endif
+
+struct btrfs_receive {
 	int mnt_fd;
 	int dest_dir_fd;
 
@@ -1216,7 +1219,7 @@  out:
 	return ret;
 }
 
-static const char * const cmd_receive_usage[] = {
+static const char *const cmd_receive_usage[] = {
 	"btrfs receive [options] <mount>\n"
 	"btrfs receive --dump [options]",
 	"Receive subvolumes from a stream",
@@ -1229,22 +1232,28 @@  static const char * const cmd_receive_usage[] = {
 	"After receiving a subvolume, it is immediately set to",
 	"read-only.",
 	"",
-	"-q|--quiet       suppress all messages, except errors",
-	"-f FILE          read the stream from FILE instead of stdin",
-	"-e               terminate after receiving an <end cmd> marker in the stream.",
-	"                 Without this option the receiver side terminates only in case",
-	"                 of an error on end of file.",
-	"-C|--chroot      confine the process to <mount> using chroot",
+	"-q|--quiet        suppress all messages, except errors",
+	"-f FILE           read the stream from FILE instead of stdin",
+#if KTLS_SEND_RECV == 1
+	"--tls-addr <url>      Address to listen on for incoming TLS connection.",
+	"--tls-port <port>     The remote port of the TLS connection",
+	"--tls-key <file>      Use the key from file; otherwise read key from stdin.",
+	"--tls-mode <mode> Use tls_12_128_gcm, tls_13_128_gcm, tls_12_256_gcm."
+#endif
+	"-e                terminate after receiving an <end cmd> marker in the stream.",
+	"                  Without this option the receiver side terminates only in case",
+	"                  of an error on end of file.",
+	"-C|--chroot       confine the process to <mount> using chroot",
 	"-E|--max-errors NERR",
-	"                 terminate as soon as NERR errors occur while",
-	"                 stream processing commands from the stream.",
-	"                 Default value is 1. A value of 0 means no limit.",
-	"-m ROOTMOUNT     the root mount point of the destination filesystem.",
-	"                 If /proc is not accessible, use this to tell us where",
-	"                 this file system is mounted.",
-	"--dump           dump stream metadata, one line per operation,",
-	"                 does not require the MOUNT parameter",
-	"-v               deprecated, alias for global -v option",
+	"                  terminate as soon as NERR errors occur while",
+	"                  stream processing commands from the stream.",
+	"                  Default value is 1. A value of 0 means no limit.",
+	"-m ROOTMOUNT      the root mount point of the destination filesystem.",
+	"                  If /proc is not accessible, use this to tell us where",
+	"                  this file system is mounted.",
+	"--dump            dump stream metadata, one line per operation,",
+	"                  does not require the MOUNT parameter",
+	"-v                deprecated, alias for global -v option",
 	HELPINFO_INSERT_GLOBALS,
 	HELPINFO_INSERT_VERBOSE,
 	HELPINFO_INSERT_QUIET,
@@ -1261,6 +1270,23 @@  static int cmd_receive(const struct cmd_struct *cmd, int argc, char **argv)
 	u64 max_errors = 1;
 	int dump = 0;
 	int ret = 0;
+#if KTLS_SEND_RECV == 1
+	enum {
+		KTLS_IDX_ADDR = 0,
+		KTLS_IDX_PORT = 1,
+		KTLS_IDX_KEY = 2,
+		KTLS_IDX_TLS_MODE = 3,
+		KTLS_IDX_SIZE
+	};
+	char *ktls_args[KTLS_IDX_SIZE];
+	struct ktls_session *ktls_session = NULL;
+	char ktls_username[LOGIN_NAME_MAX] = "btrfs";
+	u32 i = 0;
+	size_t arg_len = 0;
+	int arg_idx = 0;
+
+	explicit_bzero(ktls_args, sizeof(ktls_args));
+#endif
 
 	memset(&rctx, 0, sizeof(rctx));
 	rctx.mnt_fd = -1;
@@ -1285,10 +1311,25 @@  static int cmd_receive(const struct cmd_struct *cmd, int argc, char **argv)
 	optind = 0;
 	while (1) {
 		int c;
-		enum { GETOPT_VAL_DUMP = 257 };
+		enum {
+			GETOPT_VAL_DUMP = 257,
+			GETOPT_VAL_ADDR = 300,
+			GETOPT_VAL_PORT = 301,
+			GETOPT_VAL_KEY = 302,
+			GETOPT_VAL_TLS_MODE = 303,
+		};
 		static const struct option long_opts[] = {
 			{ "max-errors", required_argument, NULL, 'E' },
 			{ "chroot", no_argument, NULL, 'C' },
+#if KTLS_SEND_RECV == 1
+			{ "tls-addr", required_argument, NULL,
+			  GETOPT_VAL_ADDR },
+			{ "tls-port", required_argument, NULL,
+			  GETOPT_VAL_PORT },
+			{ "tls-key", required_argument, NULL, GETOPT_VAL_KEY },
+			{ "tls-mode", required_argument, NULL,
+			  GETOPT_VAL_TLS_MODE },
+#endif
 			{ "dump", no_argument, NULL, GETOPT_VAL_DUMP },
 			{ "quiet", no_argument, NULL, 'q' },
 			{ NULL, 0, NULL, 0 }
@@ -1330,6 +1371,27 @@  static int cmd_receive(const struct cmd_struct *cmd, int argc, char **argv)
 				goto out;
 			}
 			break;
+#if KTLS_SEND_RECV == 1
+		case GETOPT_VAL_ADDR:
+		case GETOPT_VAL_PORT:
+		case GETOPT_VAL_KEY:
+		case GETOPT_VAL_TLS_MODE:
+			arg_len = strlen(optarg);
+			arg_idx = c - GETOPT_VAL_ADDR;
+			ktls_args[arg_idx] = (char *)malloc(arg_len + 1);
+			if (!ktls_args[arg_idx]) {
+				error("fail to allocate buffer (%zu)", arg_len);
+				ret = 1;
+				goto out;
+			}
+			if (arg_copy_path(ktls_args[arg_idx], optarg,
+					  arg_len + 1)) {
+				error("argument too long (%zu)", arg_len);
+				ret = 1;
+				goto out;
+			}
+			break;
+#endif
 		case GETOPT_VAL_DUMP:
 			dump = 1;
 			break;
@@ -1353,6 +1415,50 @@  static int cmd_receive(const struct cmd_struct *cmd, int argc, char **argv)
 		}
 	}
 
+#if KTLS_SEND_RECV == 1
+	if (ktls_args[KTLS_IDX_ADDR]) {
+		if (fromfile[0]) {
+			error("cannot send to both ktls and file");
+			ret = 1;
+			goto out;
+		}
+
+		if (!ktls_args[KTLS_IDX_PORT]) {
+			error("no remote ktls port");
+			ret = 1;
+			goto out;
+		}
+
+		ktls_session = ktls_create_session(false);
+
+		if (ktls_args[KTLS_IDX_TLS_MODE]) {
+			if (ktls_set_tls_mode(ktls_session,
+					      ktls_args[KTLS_IDX_TLS_MODE]))
+				goto out;
+		}
+
+		if (ktls_args[KTLS_IDX_KEY]) {
+			if (ktls_set_psk_session_from_keyfile(
+				    ktls_session, ktls_username,
+				    ktls_args[KTLS_IDX_KEY])) {
+				goto out;
+			};
+		} else {
+			if (ktls_set_psk_session_from_password_prompt(
+				    ktls_session, ktls_username)) {
+				goto out;
+			}
+		}
+
+		receive_fd = ktls_create_sock_oneshot(ktls_session,
+						      ktls_args[KTLS_IDX_ADDR],
+						      ktls_args[KTLS_IDX_PORT]);
+
+		/* socket implies honor end cmd*/
+		rctx.honor_end_cmd = 1;
+	}
+#endif
+
 	if (dump) {
 		struct btrfs_dump_send_args dump_args;
 
@@ -1370,9 +1476,16 @@  static int cmd_receive(const struct cmd_struct *cmd, int argc, char **argv)
 		ret = do_receive(&rctx, tomnt, realmnt, receive_fd, max_errors);
 	}
 
+out:
+#if KTLS_SEND_RECV == 1
+	for (i = KTLS_IDX_ADDR; i < KTLS_IDX_SIZE; i++) {
+		if (ktls_args[i])
+			free(ktls_args[i]);
+	}
+	ktls_destroy_session(ktls_session);
+#endif
 	if (receive_fd != fileno(stdin))
 		close(receive_fd);
-out:
 
 	return !!ret;
 }
diff --git a/cmds/send.c b/cmds/send.c
index b8e3ba12..2b58b0c2 100644
--- a/cmds/send.c
+++ b/cmds/send.c
@@ -46,6 +46,10 @@ 
 #include "common/help.h"
 #include "common/path-utils.h"
 
+#if KTLS_SEND_RECV == 1
+#include "common/ktls.h"
+#endif
+
 #define SEND_BUFFER_SIZE	SZ_64K
 
 
@@ -424,7 +428,7 @@  static void free_send_info(struct btrfs_send *sctx)
 	subvol_uuid_search_finit(&sctx->sus);
 }
 
-static const char * const cmd_send_usage[] = {
+static const char *const cmd_send_usage[] = {
 	"btrfs send [-ve] [-p <parent>] [-c <clone-src>] [-f <outfile>] <subvol> [<subvol>...]",
 	"Send the subvolume(s) to stdout.",
 	"Sends the subvolume(s) specified by <subvol> to stdout.",
@@ -439,21 +443,27 @@  static const char * const cmd_send_usage[] = {
 	"which case 'btrfs send' will determine a suitable parent among the",
 	"clone sources itself.",
 	"",
-	"-e               If sending multiple subvols at once, use the new",
-	"                 format and omit the end-cmd between the subvols.",
-	"-p <parent>      Send an incremental stream from <parent> to",
-	"                 <subvol>.",
-	"-c <clone-src>   Use this snapshot as a clone source for an ",
-	"                 incremental send (multiple allowed)",
-	"-f <outfile>     Output is normally written to stdout. To write to",
-	"                 a file, use this option. An alternative would be to",
-	"                 use pipes.",
-	"--no-data        send in NO_FILE_DATA mode, Note: the output stream",
-	"                 does not contain any file data and thus cannot be used",
-	"                 to transfer changes. This mode is faster and useful to",
-	"                 show the differences in metadata.",
-	"-v|--verbose     deprecated, alias for global -v option",
-	"-q|--quiet       deprecated, alias for global -q option",
+	"-e                If sending multiple subvols at once, use the new",
+	"                  format and omit the end-cmd between the subvols.",
+	"-p <parent>       Send an incremental stream from <parent> to",
+	"                  <subvol>.",
+	"-c <clone-src>    Use this snapshot as a clone source for an ",
+	"                  incremental send (multiple allowed)",
+	"-f <outfile>      Output is normally written to stdout. To write to",
+	"                  a file, use this option. An alternative would be to",
+	"                  use pipes.",
+#if KTLS_SEND_RECV == 1
+	"--tls-addr <url>      Address of remote receiver.",
+	"--tls-port <port>     The remote port of the TLS connection",
+	"--tls-key <file>      Use the key from file; otherwise read key from stdin.",
+	"--tls-mode <mode> Use tls_12_128_gcm, tls_13_128_gcm, tls_12_256_gcm."
+#endif
+	"--no-data         send in NO_FILE_DATA mode, Note: the output stream",
+	"                  does not contain any file data and thus cannot be used",
+	"                  to transfer changes. This mode is faster and useful to",
+	"                  show the differences in metadata.",
+	"-v|--verbose      deprecated, alias for global -v option",
+	"-q|--quiet        deprecated, alias for global -q option",
 	HELPINFO_INSERT_GLOBALS,
 	HELPINFO_INSERT_VERBOSE,
 	HELPINFO_INSERT_QUIET,
@@ -474,6 +484,22 @@  static int cmd_send(const struct cmd_struct *cmd, int argc, char **argv)
 	int full_send = 1;
 	int new_end_cmd_semantic = 0;
 	u64 send_flags = 0;
+#if KTLS_SEND_RECV == 1
+	enum {
+		KTLS_IDX_ADDR = 0,
+		KTLS_IDX_PORT = 1,
+		KTLS_IDX_KEY = 2,
+		KTLS_IDX_TLS_MODE = 3,
+		KTLS_IDX_SIZE
+	};
+	char *ktls_args[KTLS_IDX_SIZE];
+	struct ktls_session *ktls_session = NULL;
+	char ktls_username[LOGIN_NAME_MAX] = "btrfs";
+	size_t arg_len = 0;
+	int arg_idx = 0;
+
+	explicit_bzero(ktls_args, sizeof(ktls_args));
+#endif
 
 	memset(&send, 0, sizeof(send));
 	send.dump_fd = fileno(stdout);
@@ -492,11 +518,26 @@  static int cmd_send(const struct cmd_struct *cmd, int argc, char **argv)
 
 	optind = 0;
 	while (1) {
-		enum { GETOPT_VAL_SEND_NO_DATA = 256 };
+		enum {
+			GETOPT_VAL_SEND_NO_DATA = 256,
+			GETOPT_VAL_ADDR = 300,
+			GETOPT_VAL_PORT = 301,
+			GETOPT_VAL_KEY = 302,
+			GETOPT_VAL_TLS_MODE = 303,
+		};
 		static const struct option long_options[] = {
 			{ "verbose", no_argument, NULL, 'v' },
 			{ "quiet", no_argument, NULL, 'q' },
-			{ "no-data", no_argument, NULL, GETOPT_VAL_SEND_NO_DATA }
+			{ "no-data", no_argument, NULL,
+			  GETOPT_VAL_SEND_NO_DATA },
+			{ "tls-addr", required_argument, NULL,
+			  GETOPT_VAL_ADDR },
+			{ "tls-port", required_argument, NULL,
+			  GETOPT_VAL_PORT },
+			{ "tls-key", required_argument, NULL, GETOPT_VAL_KEY },
+			{ "tls-mode", required_argument, NULL,
+			  GETOPT_VAL_TLS_MODE },
+			{ NULL, 0, NULL, 0  }
 		};
 		int c = getopt_long(argc, argv, "vqec:f:i:p:", long_options, NULL);
 
@@ -581,6 +622,27 @@  static int cmd_send(const struct cmd_struct *cmd, int argc, char **argv)
 			error("option -i was removed, use -c instead");
 			ret = 1;
 			goto out;
+#if KTLS_SEND_RECV == 1
+		case GETOPT_VAL_ADDR:
+		case GETOPT_VAL_PORT:
+		case GETOPT_VAL_KEY:
+		case GETOPT_VAL_TLS_MODE:
+			arg_len = strlen(optarg);
+			arg_idx = c - GETOPT_VAL_ADDR;
+			ktls_args[arg_idx] = (char *)malloc(arg_len + 1);
+			if (!ktls_args[arg_idx]) {
+				error("fail to allocate buffer (%zu)", arg_len);
+				ret = 1;
+				goto out;
+			}
+			if (arg_copy_path(ktls_args[arg_idx], optarg,
+					  arg_len + 1)) {
+				error("argument too long (%zu)", arg_len);
+				ret = 1;
+				goto out;
+			}
+			break;
+#endif
 		case GETOPT_VAL_SEND_NO_DATA:
 			send_flags |= BTRFS_SEND_FLAG_NO_FILE_DATA;
 			break;
@@ -613,6 +675,53 @@  static int cmd_send(const struct cmd_struct *cmd, int argc, char **argv)
 			goto out;
 		}
 	}
+#if KTLS_SEND_RECV == 1
+	if (ktls_args[KTLS_IDX_ADDR]) {
+		if (outname[0]) {
+			error("cannot send to both ktls and file");
+			ret = 1;
+			goto out;
+		}
+
+		if (!ktls_args[KTLS_IDX_PORT]) {
+			error("no remote ktls port");
+			ret = 1;
+			goto out;
+		}
+
+		if (!ktls_args[KTLS_IDX_PORT]) {
+			error("fail to create ktls session");
+			ret = 1;
+			goto out;
+		}
+
+		ktls_session = ktls_create_session(true);
+
+		if (ktls_args[KTLS_IDX_TLS_MODE]) {
+			if (ktls_set_tls_mode(ktls_session,
+					      ktls_args[KTLS_IDX_TLS_MODE]))
+				goto out;
+		}
+
+		if (ktls_args[KTLS_IDX_KEY]) {
+			if (ktls_set_psk_session_from_keyfile(
+				    ktls_session, ktls_username,
+				    ktls_args[KTLS_IDX_KEY])) {
+				goto out;
+			};
+		} else {
+			if (ktls_set_psk_session_from_password_prompt(
+				    ktls_session, ktls_username)) {
+				goto out;
+			}
+		}
+
+		send.dump_fd =
+			ktls_create_sock_oneshot(ktls_session,
+						 ktls_args[KTLS_IDX_ADDR],
+						 ktls_args[KTLS_IDX_PORT]);
+	}
+#endif
 
 	if (isatty(send.dump_fd)) {
 		error(
@@ -755,6 +864,16 @@  out:
 	free(snapshot_parent);
 	free(send.clone_sources);
 	free_send_info(&send);
+#if KTLS_SEND_RECV == 1
+	for (i = KTLS_IDX_ADDR; i < KTLS_IDX_SIZE; i++) {
+		if (ktls_args[i])
+			free(ktls_args[i]);
+	}
+	ktls_destroy_session(ktls_session);
+#endif
+	if (send.dump_fd != fileno(stdin))
+		close(send.dump_fd);
+
 	return !!ret;
 }
 DEFINE_SIMPLE_COMMAND(send, "send");