From patchwork Wed Jan 11 22:13:01 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Matthew John Cheetham X-Patchwork-Id: 13097280 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id E0924C5479D for ; Wed, 11 Jan 2023 22:13:20 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S234866AbjAKWNT (ORCPT ); Wed, 11 Jan 2023 17:13:19 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:33402 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S235894AbjAKWNQ (ORCPT ); Wed, 11 Jan 2023 17:13:16 -0500 Received: from mail-wm1-x330.google.com (mail-wm1-x330.google.com [IPv6:2a00:1450:4864:20::330]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 2B56636337 for ; Wed, 11 Jan 2023 14:13:15 -0800 (PST) Received: by mail-wm1-x330.google.com with SMTP id k22-20020a05600c1c9600b003d1ee3a6289so13740557wms.2 for ; Wed, 11 Jan 2023 14:13:15 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=cc:to:mime-version:content-transfer-encoding:fcc:subject:date:from :references:in-reply-to:message-id:from:to:cc:subject:date :message-id:reply-to; bh=qZfvZ8RTcDBAofAevq0+3MPqq1v9AIkhChP91QQElkQ=; b=bdRspmaPhoB6Ye3Y5Lgabn4ax77jKUz6Jxv0HUBr1GgQFqwSB6m3glf9+oTKXjpsD3 aosy4OyUeYQS8oFSaI5GzdDbSd4V/RCe9fmx7ad3clpJs3m+svj7TfNxijwOOK3zGEI1 ystcJII7KkoHchgHxgzM30Hl8o0P0UzHcRPJeWQVaccLRb2zvlEDVmmNhwbizJnroGD9 G9m8z8vAPiyssA18ebBAS0f7n7tlX6tqDdgegDHngEr/GXuZpMj9SIIgqQezZaRJSNu9 gbANYNNN911KabMaXfr6ifidoX1QvVsaXdKAbLOqHIH4YVEBSZuAny4UJuE+x3AkNkOj ar2Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=cc:to:mime-version:content-transfer-encoding:fcc:subject:date:from :references:in-reply-to:message-id:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=qZfvZ8RTcDBAofAevq0+3MPqq1v9AIkhChP91QQElkQ=; b=t1xh+kdPwHKQ0qODHzHq8cyeQYgvavj8kRwdSoIgzTkxcVP9OIttq8L5r+lnriEJkY 3zS0Vr09eqgoQsqxlktR5dGR27V97W034XJ1dQxUGuFS+epRFRSmMmYLiN1fn8xb+gqr P5SH4v3ys2dcUzfpaann0yCCNAzYFb687WOGnRxJnUU4Qp2oqneHs+mffOqN52zpDEVI CHCZVTPmO64c/uhv3iA4ThzCGmMsedO8a7tY0un+r7XFkjbU3Qmaj4MLFZTpU46WtKfI ganFu4ik664JlZ9b11SxSeUUGZeXtj1emfT83hjtWpymDccEneFvbUy5Ek4WxisBMmde dfOQ== X-Gm-Message-State: AFqh2koAhSRVel8DHLU0hL79NkJYX4QNWbpI5EXct6meEwOqvvwPW0cn 5qwj2nEIL2tMsFJ+Qlwqt3P0gMUJ+9Q= X-Google-Smtp-Source: AMrXdXuQRZgZvWu8JWKXg/RepOsfpvmIAodeN6ms3NuCZP11YLKZRxbWVaoxtCQ3SrCrkzJlF5regQ== X-Received: by 2002:a05:600c:6016:b0:3d3:3c93:af5e with SMTP id az22-20020a05600c601600b003d33c93af5emr54130252wmb.35.1673475193434; Wed, 11 Jan 2023 14:13:13 -0800 (PST) Received: from [127.0.0.1] ([13.74.141.28]) by smtp.gmail.com with ESMTPSA id m18-20020a05600c4f5200b003c6b70a4d69sm22415101wmq.42.2023.01.11.14.13.13 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 11 Jan 2023 14:13:13 -0800 (PST) Message-Id: <74b0de14185120c9d53d7470e59f57fa20a1927f.1673475190.git.gitgitgadget@gmail.com> In-Reply-To: References: Date: Wed, 11 Jan 2023 22:13:01 +0000 Subject: [PATCH v5 01/10] daemon: libify socket setup and option functions Fcc: Sent MIME-Version: 1.0 To: git@vger.kernel.org Cc: Derrick Stolee , Lessley Dennington , Matthew John Cheetham , M Hickford , Jeff Hostetler , Glen Choo , Victoria Dye , Matthew John Cheetham , Matthew John Cheetham Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org From: Matthew John Cheetham From: Matthew John Cheetham Extract functions for setting up listening sockets and keep-alive options from `daemon.c` to new `daemon-utils.{c,h}` files. Remove direct dependencies on global state by inlining the behaviour at the callsites for all libified functions. Signed-off-by: Matthew John Cheetham --- Makefile | 1 + daemon-utils.c | 209 +++++++++++++++++++++++++++++++++++++++++++++++ daemon-utils.h | 23 ++++++ daemon.c | 214 +------------------------------------------------ 4 files changed, 237 insertions(+), 210 deletions(-) create mode 100644 daemon-utils.c create mode 100644 daemon-utils.h diff --git a/Makefile b/Makefile index b258fdbed86..2654094dbb5 100644 --- a/Makefile +++ b/Makefile @@ -1003,6 +1003,7 @@ LIB_OBJS += credential.o LIB_OBJS += csum-file.o LIB_OBJS += ctype.o LIB_OBJS += date.o +LIB_OBJS += daemon-utils.o LIB_OBJS += decorate.o LIB_OBJS += delta-islands.o LIB_OBJS += diagnose.o diff --git a/daemon-utils.c b/daemon-utils.c new file mode 100644 index 00000000000..b96b55962db --- /dev/null +++ b/daemon-utils.c @@ -0,0 +1,209 @@ +#include "cache.h" +#include "daemon-utils.h" + +void set_keep_alive(int sockfd, log_fn logerror) +{ + int ka = 1; + + if (setsockopt(sockfd, SOL_SOCKET, SO_KEEPALIVE, &ka, sizeof(ka)) < 0) { + if (errno != ENOTSOCK) + logerror("unable to set SO_KEEPALIVE on socket: %s", + strerror(errno)); + } +} + +static int set_reuse_addr(int sockfd) +{ + int on = 1; + + return setsockopt(sockfd, SOL_SOCKET, SO_REUSEADDR, + &on, sizeof(on)); +} + +static const char *ip2str(int family, struct sockaddr *sin, socklen_t len) +{ +#ifdef NO_IPV6 + static char ip[INET_ADDRSTRLEN]; +#else + static char ip[INET6_ADDRSTRLEN]; +#endif + + switch (family) { +#ifndef NO_IPV6 + case AF_INET6: + inet_ntop(family, &((struct sockaddr_in6*)sin)->sin6_addr, ip, len); + break; +#endif + case AF_INET: + inet_ntop(family, &((struct sockaddr_in*)sin)->sin_addr, ip, len); + break; + default: + xsnprintf(ip, sizeof(ip), ""); + } + return ip; +} + +#ifndef NO_IPV6 + +static int setup_named_sock(char *listen_addr, int listen_port, + struct socketlist *socklist, int reuseaddr, + log_fn logerror) +{ + int socknum = 0; + char pbuf[NI_MAXSERV]; + struct addrinfo hints, *ai0, *ai; + int gai; + long flags; + + xsnprintf(pbuf, sizeof(pbuf), "%d", listen_port); + memset(&hints, 0, sizeof(hints)); + hints.ai_family = AF_UNSPEC; + hints.ai_socktype = SOCK_STREAM; + hints.ai_protocol = IPPROTO_TCP; + hints.ai_flags = AI_PASSIVE; + + gai = getaddrinfo(listen_addr, pbuf, &hints, &ai0); + if (gai) { + logerror("getaddrinfo() for %s failed: %s", listen_addr, gai_strerror(gai)); + return 0; + } + + for (ai = ai0; ai; ai = ai->ai_next) { + int sockfd; + + sockfd = socket(ai->ai_family, ai->ai_socktype, ai->ai_protocol); + if (sockfd < 0) + continue; + if (sockfd >= FD_SETSIZE) { + logerror("Socket descriptor too large"); + close(sockfd); + continue; + } + +#ifdef IPV6_V6ONLY + if (ai->ai_family == AF_INET6) { + int on = 1; + setsockopt(sockfd, IPPROTO_IPV6, IPV6_V6ONLY, + &on, sizeof(on)); + /* Note: error is not fatal */ + } +#endif + + if (reuseaddr && set_reuse_addr(sockfd)) { + logerror("Could not set SO_REUSEADDR: %s", strerror(errno)); + close(sockfd); + continue; + } + + set_keep_alive(sockfd, logerror); + + if (bind(sockfd, ai->ai_addr, ai->ai_addrlen) < 0) { + logerror("Could not bind to %s: %s", + ip2str(ai->ai_family, ai->ai_addr, ai->ai_addrlen), + strerror(errno)); + close(sockfd); + continue; /* not fatal */ + } + if (listen(sockfd, 5) < 0) { + logerror("Could not listen to %s: %s", + ip2str(ai->ai_family, ai->ai_addr, ai->ai_addrlen), + strerror(errno)); + close(sockfd); + continue; /* not fatal */ + } + + flags = fcntl(sockfd, F_GETFD, 0); + if (flags >= 0) + fcntl(sockfd, F_SETFD, flags | FD_CLOEXEC); + + ALLOC_GROW(socklist->list, socklist->nr + 1, socklist->alloc); + socklist->list[socklist->nr++] = sockfd; + socknum++; + } + + freeaddrinfo(ai0); + + return socknum; +} + +#else /* NO_IPV6 */ + +static int setup_named_sock(char *listen_addr, int listen_port, + struct socketlist *socklist, int reuseaddr, + log_fn logerror) +{ + struct sockaddr_in sin; + int sockfd; + long flags; + + memset(&sin, 0, sizeof sin); + sin.sin_family = AF_INET; + sin.sin_port = htons(listen_port); + + if (listen_addr) { + /* Well, host better be an IP address here. */ + if (inet_pton(AF_INET, listen_addr, &sin.sin_addr.s_addr) <= 0) + return 0; + } else { + sin.sin_addr.s_addr = htonl(INADDR_ANY); + } + + sockfd = socket(AF_INET, SOCK_STREAM, 0); + if (sockfd < 0) + return 0; + + if (reuseaddr && set_reuse_addr(sockfd)) { + logerror("Could not set SO_REUSEADDR: %s", strerror(errno)); + close(sockfd); + return 0; + } + + set_keep_alive(sockfd, logerror); + + if ( bind(sockfd, (struct sockaddr *)&sin, sizeof sin) < 0 ) { + logerror("Could not bind to %s: %s", + ip2str(AF_INET, (struct sockaddr *)&sin, sizeof(sin)), + strerror(errno)); + close(sockfd); + return 0; + } + + if (listen(sockfd, 5) < 0) { + logerror("Could not listen to %s: %s", + ip2str(AF_INET, (struct sockaddr *)&sin, sizeof(sin)), + strerror(errno)); + close(sockfd); + return 0; + } + + flags = fcntl(sockfd, F_GETFD, 0); + if (flags >= 0) + fcntl(sockfd, F_SETFD, flags | FD_CLOEXEC); + + ALLOC_GROW(socklist->list, socklist->nr + 1, socklist->alloc); + socklist->list[socklist->nr++] = sockfd; + return 1; +} + +#endif + +void socksetup(struct string_list *listen_addr, int listen_port, + struct socketlist *socklist, int reuseaddr, + log_fn logerror) +{ + if (!listen_addr->nr) + setup_named_sock(NULL, listen_port, socklist, reuseaddr, + logerror); + else { + int i, socknum; + for (i = 0; i < listen_addr->nr; i++) { + socknum = setup_named_sock(listen_addr->items[i].string, + listen_port, socklist, reuseaddr, + logerror); + + if (socknum == 0) + logerror("unable to allocate any listen sockets for host %s on port %u", + listen_addr->items[i].string, listen_port); + } + } +} diff --git a/daemon-utils.h b/daemon-utils.h new file mode 100644 index 00000000000..6710a2a6dc0 --- /dev/null +++ b/daemon-utils.h @@ -0,0 +1,23 @@ +#ifndef DAEMON_UTILS_H +#define DAEMON_UTILS_H + +#include "git-compat-util.h" +#include "string-list.h" + +typedef void (*log_fn)(const char *msg, ...); + +struct socketlist { + int *list; + size_t nr; + size_t alloc; +}; + +/* Enable sending of keep-alive messages on the socket. */ +void set_keep_alive(int sockfd, log_fn logerror); + +/* Setup a number of sockets to listen on the provided addresses. */ +void socksetup(struct string_list *listen_addr, int listen_port, + struct socketlist *socklist, int reuseaddr, + log_fn logerror); + +#endif diff --git a/daemon.c b/daemon.c index 0ae7d12b5c1..1ed4e705680 100644 --- a/daemon.c +++ b/daemon.c @@ -1,9 +1,9 @@ #include "cache.h" #include "config.h" +#include "daemon-utils.h" #include "pkt-line.h" #include "run-command.h" #include "strbuf.h" -#include "string-list.h" #ifdef NO_INITGROUPS #define initgroups(x, y) (0) /* nothing */ @@ -737,17 +737,6 @@ static void hostinfo_clear(struct hostinfo *hi) strbuf_release(&hi->tcp_port); } -static void set_keep_alive(int sockfd) -{ - int ka = 1; - - if (setsockopt(sockfd, SOL_SOCKET, SO_KEEPALIVE, &ka, sizeof(ka)) < 0) { - if (errno != ENOTSOCK) - logerror("unable to set SO_KEEPALIVE on socket: %s", - strerror(errno)); - } -} - static int execute(void) { char *line = packet_buffer; @@ -759,7 +748,7 @@ static int execute(void) if (addr) loginfo("Connection from %s:%s", addr, port); - set_keep_alive(0); + set_keep_alive(0, logerror); alarm(init_timeout ? init_timeout : timeout); pktlen = packet_read(0, packet_buffer, sizeof(packet_buffer), 0); alarm(0); @@ -938,202 +927,6 @@ static void child_handler(int signo) signal(SIGCHLD, child_handler); } -static int set_reuse_addr(int sockfd) -{ - int on = 1; - - if (!reuseaddr) - return 0; - return setsockopt(sockfd, SOL_SOCKET, SO_REUSEADDR, - &on, sizeof(on)); -} - -struct socketlist { - int *list; - size_t nr; - size_t alloc; -}; - -static const char *ip2str(int family, struct sockaddr *sin, socklen_t len) -{ -#ifdef NO_IPV6 - static char ip[INET_ADDRSTRLEN]; -#else - static char ip[INET6_ADDRSTRLEN]; -#endif - - switch (family) { -#ifndef NO_IPV6 - case AF_INET6: - inet_ntop(family, &((struct sockaddr_in6*)sin)->sin6_addr, ip, len); - break; -#endif - case AF_INET: - inet_ntop(family, &((struct sockaddr_in*)sin)->sin_addr, ip, len); - break; - default: - xsnprintf(ip, sizeof(ip), ""); - } - return ip; -} - -#ifndef NO_IPV6 - -static int setup_named_sock(char *listen_addr, int listen_port, struct socketlist *socklist) -{ - int socknum = 0; - char pbuf[NI_MAXSERV]; - struct addrinfo hints, *ai0, *ai; - int gai; - long flags; - - xsnprintf(pbuf, sizeof(pbuf), "%d", listen_port); - memset(&hints, 0, sizeof(hints)); - hints.ai_family = AF_UNSPEC; - hints.ai_socktype = SOCK_STREAM; - hints.ai_protocol = IPPROTO_TCP; - hints.ai_flags = AI_PASSIVE; - - gai = getaddrinfo(listen_addr, pbuf, &hints, &ai0); - if (gai) { - logerror("getaddrinfo() for %s failed: %s", listen_addr, gai_strerror(gai)); - return 0; - } - - for (ai = ai0; ai; ai = ai->ai_next) { - int sockfd; - - sockfd = socket(ai->ai_family, ai->ai_socktype, ai->ai_protocol); - if (sockfd < 0) - continue; - if (sockfd >= FD_SETSIZE) { - logerror("Socket descriptor too large"); - close(sockfd); - continue; - } - -#ifdef IPV6_V6ONLY - if (ai->ai_family == AF_INET6) { - int on = 1; - setsockopt(sockfd, IPPROTO_IPV6, IPV6_V6ONLY, - &on, sizeof(on)); - /* Note: error is not fatal */ - } -#endif - - if (set_reuse_addr(sockfd)) { - logerror("Could not set SO_REUSEADDR: %s", strerror(errno)); - close(sockfd); - continue; - } - - set_keep_alive(sockfd); - - if (bind(sockfd, ai->ai_addr, ai->ai_addrlen) < 0) { - logerror("Could not bind to %s: %s", - ip2str(ai->ai_family, ai->ai_addr, ai->ai_addrlen), - strerror(errno)); - close(sockfd); - continue; /* not fatal */ - } - if (listen(sockfd, 5) < 0) { - logerror("Could not listen to %s: %s", - ip2str(ai->ai_family, ai->ai_addr, ai->ai_addrlen), - strerror(errno)); - close(sockfd); - continue; /* not fatal */ - } - - flags = fcntl(sockfd, F_GETFD, 0); - if (flags >= 0) - fcntl(sockfd, F_SETFD, flags | FD_CLOEXEC); - - ALLOC_GROW(socklist->list, socklist->nr + 1, socklist->alloc); - socklist->list[socklist->nr++] = sockfd; - socknum++; - } - - freeaddrinfo(ai0); - - return socknum; -} - -#else /* NO_IPV6 */ - -static int setup_named_sock(char *listen_addr, int listen_port, struct socketlist *socklist) -{ - struct sockaddr_in sin; - int sockfd; - long flags; - - memset(&sin, 0, sizeof sin); - sin.sin_family = AF_INET; - sin.sin_port = htons(listen_port); - - if (listen_addr) { - /* Well, host better be an IP address here. */ - if (inet_pton(AF_INET, listen_addr, &sin.sin_addr.s_addr) <= 0) - return 0; - } else { - sin.sin_addr.s_addr = htonl(INADDR_ANY); - } - - sockfd = socket(AF_INET, SOCK_STREAM, 0); - if (sockfd < 0) - return 0; - - if (set_reuse_addr(sockfd)) { - logerror("Could not set SO_REUSEADDR: %s", strerror(errno)); - close(sockfd); - return 0; - } - - set_keep_alive(sockfd); - - if ( bind(sockfd, (struct sockaddr *)&sin, sizeof sin) < 0 ) { - logerror("Could not bind to %s: %s", - ip2str(AF_INET, (struct sockaddr *)&sin, sizeof(sin)), - strerror(errno)); - close(sockfd); - return 0; - } - - if (listen(sockfd, 5) < 0) { - logerror("Could not listen to %s: %s", - ip2str(AF_INET, (struct sockaddr *)&sin, sizeof(sin)), - strerror(errno)); - close(sockfd); - return 0; - } - - flags = fcntl(sockfd, F_GETFD, 0); - if (flags >= 0) - fcntl(sockfd, F_SETFD, flags | FD_CLOEXEC); - - ALLOC_GROW(socklist->list, socklist->nr + 1, socklist->alloc); - socklist->list[socklist->nr++] = sockfd; - return 1; -} - -#endif - -static void socksetup(struct string_list *listen_addr, int listen_port, struct socketlist *socklist) -{ - if (!listen_addr->nr) - setup_named_sock(NULL, listen_port, socklist); - else { - int i, socknum; - for (i = 0; i < listen_addr->nr; i++) { - socknum = setup_named_sock(listen_addr->items[i].string, - listen_port, socklist); - - if (socknum == 0) - logerror("unable to allocate any listen sockets for host %s on port %u", - listen_addr->items[i].string, listen_port); - } - } -} - static int service_loop(struct socketlist *socklist) { struct pollfd *pfd; @@ -1246,7 +1039,8 @@ static int serve(struct string_list *listen_addr, int listen_port, { struct socketlist socklist = { NULL, 0, 0 }; - socksetup(listen_addr, listen_port, &socklist); + socksetup(listen_addr, listen_port, &socklist, reuseaddr, + logerror); if (socklist.nr == 0) die("unable to allocate any listen sockets on port %u", listen_port); From patchwork Wed Jan 11 22:13:02 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Matthew John Cheetham X-Patchwork-Id: 13097281 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 2FC44C46467 for ; Wed, 11 Jan 2023 22:13:24 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S236035AbjAKWNV (ORCPT ); Wed, 11 Jan 2023 17:13:21 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:33404 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S235897AbjAKWNR (ORCPT ); Wed, 11 Jan 2023 17:13:17 -0500 Received: from mail-wm1-x335.google.com (mail-wm1-x335.google.com [IPv6:2a00:1450:4864:20::335]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id EA2443E0E1 for ; Wed, 11 Jan 2023 14:13:15 -0800 (PST) Received: by mail-wm1-x335.google.com with SMTP id j16-20020a05600c1c1000b003d9ef8c274bso9380331wms.0 for ; Wed, 11 Jan 2023 14:13:15 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=cc:to:mime-version:content-transfer-encoding:fcc:subject:date:from :references:in-reply-to:message-id:from:to:cc:subject:date :message-id:reply-to; bh=Rsb9nNGK6aVNwlCaEC3jjexZfnZ8WeFAlLgzTSFgXug=; b=l7g9Zmx1AhK/bfwzU1c4vfF+xyGFFIsMUoYkDtN7CjbZSH2HsQD2/8Bz5S2TdIzts/ My9TnB1JIe/cNfvJMDHv++V++bIO3J96+R0ydCYdhk8zqYXuqD7Yju2fTEBi3k4JWLIL /qFUBeHnGjyGqPhDRcV8pyX3NNWv98yL/iKPPsXySCNz7o3DHbi9OpozrlxTMdxmynTY 4YPbh41IkXYVHgGKhBJTUC3TlheSwzRkuNBLFyNSOlt4+9QD3FVZjPhMFDDS5TmKVUZI VrfV6qf9nyk7YsSOg01YNnoVafT47lQNVQVb0ydMG2Sa+HuVknJDoN8ZKeTEXOmahAMv pmVg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=cc:to:mime-version:content-transfer-encoding:fcc:subject:date:from :references:in-reply-to:message-id:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=Rsb9nNGK6aVNwlCaEC3jjexZfnZ8WeFAlLgzTSFgXug=; b=k0BDqN6uFfrpsnbdM0QNcKz677joYN9b3rM8AEQ4NKRgU6o8+Hrv8Xdq3+tS0LVY4E hzvzrnHLpwsJUkL/bOZiXYVCngvg2PmMCKeG376uUbnRxXz92xy6uiTmqx9eO4dZGrRu D4hVoD9zUtbRoovth9bQvyxairO/1nvhzSId6RJR7yVj4eIAzMze7aw0kc+ABjqk2N5r D2gxMJDNJERxsISNCZws4GSNiezDU7TgBf0FCNlUVjMPGzJvUA75hl+kHoLup0AF1jsY g0UPfd7jqazMcifjUVGRAyFyxqZMafNNlk78LeFkLEsa12+WieE3dSNVfw1WDJXXHyeL iQeQ== X-Gm-Message-State: AFqh2kqcVdQE7eiA+v2Mbas8H2XFOWLvsoUNMrA0sxjtXPynI4vR5Xr9 GcNEkVV4qna/c1JbfqEag6N+nVHnTJE= X-Google-Smtp-Source: AMrXdXsDnJLKgB6zzwIXGoAkVjz8iKCHKMwfBQaTs0z6AJkSsUvQPJLrJQeEYCS8zIgqdX00Xr2yGA== X-Received: by 2002:a05:600c:358f:b0:3d9:7847:96e2 with SMTP id p15-20020a05600c358f00b003d9784796e2mr43905081wmq.2.1673475194258; Wed, 11 Jan 2023 14:13:14 -0800 (PST) Received: from [127.0.0.1] ([13.74.141.28]) by smtp.gmail.com with ESMTPSA id o12-20020a05600c4fcc00b003da0b75de94sm3271396wmq.8.2023.01.11.14.13.13 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 11 Jan 2023 14:13:13 -0800 (PST) Message-Id: In-Reply-To: References: Date: Wed, 11 Jan 2023 22:13:02 +0000 Subject: [PATCH v5 02/10] daemon: libify child process handling functions Fcc: Sent MIME-Version: 1.0 To: git@vger.kernel.org Cc: Derrick Stolee , Lessley Dennington , Matthew John Cheetham , M Hickford , Jeff Hostetler , Glen Choo , Victoria Dye , Matthew John Cheetham , Matthew John Cheetham Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org From: Matthew John Cheetham From: Matthew John Cheetham Extract functions and structures for managing child processes started from the parent daemon-like process from `daemon.c` to the new shared `daemon-utils.{c,h}` files. Signed-off-by: Matthew John Cheetham --- daemon-utils.c | 77 ++++++++++++++++++++++++++++++++++++++++++ daemon-utils.h | 15 ++++++++ daemon.c | 92 +++----------------------------------------------- 3 files changed, 97 insertions(+), 87 deletions(-) diff --git a/daemon-utils.c b/daemon-utils.c index b96b55962db..3804bc60973 100644 --- a/daemon-utils.c +++ b/daemon-utils.c @@ -207,3 +207,80 @@ void socksetup(struct string_list *listen_addr, int listen_port, } } } + +static int addrcmp(const struct sockaddr_storage *s1, + const struct sockaddr_storage *s2) +{ + const struct sockaddr *sa1 = (const struct sockaddr*) s1; + const struct sockaddr *sa2 = (const struct sockaddr*) s2; + + if (sa1->sa_family != sa2->sa_family) + return sa1->sa_family - sa2->sa_family; + if (sa1->sa_family == AF_INET) + return memcmp(&((struct sockaddr_in *)s1)->sin_addr, + &((struct sockaddr_in *)s2)->sin_addr, + sizeof(struct in_addr)); +#ifndef NO_IPV6 + if (sa1->sa_family == AF_INET6) + return memcmp(&((struct sockaddr_in6 *)s1)->sin6_addr, + &((struct sockaddr_in6 *)s2)->sin6_addr, + sizeof(struct in6_addr)); +#endif + return 0; +} + +void add_child(struct child_process *cld, struct sockaddr *addr, socklen_t addrlen, + struct child *firstborn , unsigned int *live_children) +{ + struct child *newborn, **cradle; + + CALLOC_ARRAY(newborn, 1); + live_children++; + memcpy(&newborn->cld, cld, sizeof(*cld)); + memcpy(&newborn->address, addr, addrlen); + for (cradle = &firstborn; *cradle; cradle = &(*cradle)->next) + if (!addrcmp(&(*cradle)->address, &newborn->address)) + break; + newborn->next = *cradle; + *cradle = newborn; +} + +void kill_some_child(struct child *firstborn) +{ + const struct child *blanket, *next; + + if (!(blanket = firstborn)) + return; + + for (; (next = blanket->next); blanket = next) + if (!addrcmp(&blanket->address, &next->address)) { + kill(blanket->cld.pid, SIGTERM); + break; + } +} + +void check_dead_children(struct child *firstborn, unsigned int *live_children, + log_fn loginfo) +{ + int status; + pid_t pid; + + struct child **cradle, *blanket; + for (cradle = &firstborn; (blanket = *cradle);) + if ((pid = waitpid(blanket->cld.pid, &status, WNOHANG)) > 1) { + if (loginfo) { + const char *dead = ""; + if (status) + dead = " (with error)"; + loginfo("[%"PRIuMAX"] Disconnected%s", + (uintmax_t)pid, dead); + } + + /* remove the child */ + *cradle = blanket->next; + live_children--; + child_process_clear(&blanket->cld); + free(blanket); + } else + cradle = &blanket->next; +} diff --git a/daemon-utils.h b/daemon-utils.h index 6710a2a6dc0..fe8d9d05256 100644 --- a/daemon-utils.h +++ b/daemon-utils.h @@ -2,6 +2,7 @@ #define DAEMON_UTILS_H #include "git-compat-util.h" +#include "run-command.h" #include "string-list.h" typedef void (*log_fn)(const char *msg, ...); @@ -20,4 +21,18 @@ void socksetup(struct string_list *listen_addr, int listen_port, struct socketlist *socklist, int reuseaddr, log_fn logerror); +struct child { + struct child *next; + struct child_process cld; + struct sockaddr_storage address; +}; + +void add_child(struct child_process *cld, struct sockaddr *addr, socklen_t addrlen, + struct child *firstborn, unsigned int *live_children); + +void kill_some_child(struct child *firstborn); + +void check_dead_children(struct child *firstborn, unsigned int *live_children, + log_fn loginfo); + #endif diff --git a/daemon.c b/daemon.c index 1ed4e705680..ec3b407ecbc 100644 --- a/daemon.c +++ b/daemon.c @@ -785,93 +785,11 @@ static int execute(void) return -1; } -static int addrcmp(const struct sockaddr_storage *s1, - const struct sockaddr_storage *s2) -{ - const struct sockaddr *sa1 = (const struct sockaddr*) s1; - const struct sockaddr *sa2 = (const struct sockaddr*) s2; - - if (sa1->sa_family != sa2->sa_family) - return sa1->sa_family - sa2->sa_family; - if (sa1->sa_family == AF_INET) - return memcmp(&((struct sockaddr_in *)s1)->sin_addr, - &((struct sockaddr_in *)s2)->sin_addr, - sizeof(struct in_addr)); -#ifndef NO_IPV6 - if (sa1->sa_family == AF_INET6) - return memcmp(&((struct sockaddr_in6 *)s1)->sin6_addr, - &((struct sockaddr_in6 *)s2)->sin6_addr, - sizeof(struct in6_addr)); -#endif - return 0; -} - static int max_connections = 32; static unsigned int live_children; -static struct child { - struct child *next; - struct child_process cld; - struct sockaddr_storage address; -} *firstborn; - -static void add_child(struct child_process *cld, struct sockaddr *addr, socklen_t addrlen) -{ - struct child *newborn, **cradle; - - CALLOC_ARRAY(newborn, 1); - live_children++; - memcpy(&newborn->cld, cld, sizeof(*cld)); - memcpy(&newborn->address, addr, addrlen); - for (cradle = &firstborn; *cradle; cradle = &(*cradle)->next) - if (!addrcmp(&(*cradle)->address, &newborn->address)) - break; - newborn->next = *cradle; - *cradle = newborn; -} - -/* - * This gets called if the number of connections grows - * past "max_connections". - * - * We kill the newest connection from a duplicate IP. - */ -static void kill_some_child(void) -{ - const struct child *blanket, *next; - - if (!(blanket = firstborn)) - return; - - for (; (next = blanket->next); blanket = next) - if (!addrcmp(&blanket->address, &next->address)) { - kill(blanket->cld.pid, SIGTERM); - break; - } -} - -static void check_dead_children(void) -{ - int status; - pid_t pid; - - struct child **cradle, *blanket; - for (cradle = &firstborn; (blanket = *cradle);) - if ((pid = waitpid(blanket->cld.pid, &status, WNOHANG)) > 1) { - const char *dead = ""; - if (status) - dead = " (with error)"; - loginfo("[%"PRIuMAX"] Disconnected%s", (uintmax_t)pid, dead); - - /* remove the child */ - *cradle = blanket->next; - live_children--; - child_process_clear(&blanket->cld); - free(blanket); - } else - cradle = &blanket->next; -} +static struct child *firstborn; static struct strvec cld_argv = STRVEC_INIT; static void handle(int incoming, struct sockaddr *addr, socklen_t addrlen) @@ -879,9 +797,9 @@ static void handle(int incoming, struct sockaddr *addr, socklen_t addrlen) struct child_process cld = CHILD_PROCESS_INIT; if (max_connections && live_children >= max_connections) { - kill_some_child(); + kill_some_child(firstborn); sleep(1); /* give it some time to die */ - check_dead_children(); + check_dead_children(firstborn, &live_children, loginfo); if (live_children >= max_connections) { close(incoming); logerror("Too many children, dropping connection"); @@ -914,7 +832,7 @@ static void handle(int incoming, struct sockaddr *addr, socklen_t addrlen) if (start_command(&cld)) logerror("unable to fork"); else - add_child(&cld, addr, addrlen); + add_child(&cld, addr, addrlen, firstborn, &live_children); } static void child_handler(int signo) @@ -944,7 +862,7 @@ static int service_loop(struct socketlist *socklist) for (;;) { int i; - check_dead_children(); + check_dead_children(firstborn, &live_children, loginfo); if (poll(pfd, socklist->nr, -1) < 0) { if (errno != EINTR) { From patchwork Wed Jan 11 22:13:03 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Matthew John Cheetham X-Patchwork-Id: 13097282 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 47A69C5479D for ; Wed, 11 Jan 2023 22:13:28 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S236094AbjAKWNY (ORCPT ); Wed, 11 Jan 2023 17:13:24 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:33416 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S235978AbjAKWNS (ORCPT ); Wed, 11 Jan 2023 17:13:18 -0500 Received: from mail-wr1-x42e.google.com (mail-wr1-x42e.google.com [IPv6:2a00:1450:4864:20::42e]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 136D83F129 for ; Wed, 11 Jan 2023 14:13:17 -0800 (PST) Received: by mail-wr1-x42e.google.com with SMTP id h16so16405030wrz.12 for ; Wed, 11 Jan 2023 14:13:17 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=cc:to:mime-version:content-transfer-encoding:fcc:subject:date:from :references:in-reply-to:message-id:from:to:cc:subject:date :message-id:reply-to; bh=UCcatjtuaN1IUjSQ2kPLhhk9rJyJf2AjwsCYoHqfbdk=; b=EYDIYZCPm8k4FGHc7OvW199FQofPEFxwe3gP9jaAvXb7eEkJW6I1FQ47DPY6062NQ7 sL84Te3hbPd4Yya5QYsuqe3ZXEJkkSf3WDXA75TNPD9YSk1hfp4Edsb6KVOnclcfTh4r rHQxPTymfClR79RdT0RB0iXFXRGOsk6CC4D9riTY27AP0f/fi9nTiV0bAxasdDpqfJvk W2qjNyN11UVW7/lWbf2ggOjep4k/l9Ou79ejYR0BFoDnsXtD9q8aLVUWPTQbXpJIiIDQ wMPrri4Tbes7bg/S8MWFYJfXBr3IPn75HKOTNYK7Fc4dXoZyZjMPxUaVItP5MoyPkeFc rWcg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=cc:to:mime-version:content-transfer-encoding:fcc:subject:date:from :references:in-reply-to:message-id:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=UCcatjtuaN1IUjSQ2kPLhhk9rJyJf2AjwsCYoHqfbdk=; b=GZjWwT+zlL5L2W7yZ5JxNY0dVYIdOCJBMEeTsNpxhPMSTJYqbq62DkF5jq9Ld6UzS5 9qji45wEGjOU6KD7fIG0pa0MNrbLeq2/dCR0Y6uxu5wbcXYrGTNkhuG0NSLq/3rksrkr 8MgnuKp8k28CUfCBsgCCP9sPXfANKkdngDZd8IijNX5nuItKF194SPTebOPToeIZ4CxQ WSmxxwF2WZ3ZGmc8pNS9PGfChzD4149gzS3yZ8me84U4NPpKmN490g6qgsrwVjq5T1pX WT5NyKF+9rVgnNsoWRxqUm3rN60ygVz0EWiVaqaUmaIIweCIHul/a95chIZ+4fucLvLS nhFA== X-Gm-Message-State: AFqh2kqChUDB3GZe3FHkln6jc5les0tWx6AdV8tGLKytBhUmKCpZ+AVt 0hxKQHdpvRAvnatRKUIqsWKb+axDg08= X-Google-Smtp-Source: AMrXdXv79rXAJZaa/BBbMp2mBdVaLrAd65k23JTJnFvy5ducoRKVZ5BfQFk6i7Lx5ZN1oKPvdCn7Pg== X-Received: by 2002:adf:d22c:0:b0:2bb:eaae:3096 with SMTP id k12-20020adfd22c000000b002bbeaae3096mr8635697wrh.8.1673475195447; Wed, 11 Jan 2023 14:13:15 -0800 (PST) Received: from [127.0.0.1] ([13.74.141.28]) by smtp.gmail.com with ESMTPSA id he12-20020a05600c540c00b003d9ddc82450sm19116532wmb.45.2023.01.11.14.13.14 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 11 Jan 2023 14:13:14 -0800 (PST) Message-Id: <8f176d5955dfc83616a39622972aaa71a71f5599.1673475190.git.gitgitgadget@gmail.com> In-Reply-To: References: Date: Wed, 11 Jan 2023 22:13:03 +0000 Subject: [PATCH v5 03/10] daemon: rename some esoteric/laboured terminology Fcc: Sent MIME-Version: 1.0 To: git@vger.kernel.org Cc: Derrick Stolee , Lessley Dennington , Matthew John Cheetham , M Hickford , Jeff Hostetler , Glen Choo , Victoria Dye , Matthew John Cheetham , Matthew John Cheetham Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org From: Matthew John Cheetham From: Matthew John Cheetham Rename some of the variables and function arguments used to manage child processes. The existing names are esoteric; stretching an analogy too far to the point of being confusing to understand. Rename "firstborn" to simply "first", "newborn" to "new_cld", "blanket" to "current" and "cradle" to "ptr". Signed-off-by: Matthew John Cheetham --- daemon-utils.c | 46 +++++++++++++++++++++++----------------------- daemon-utils.h | 6 +++--- daemon.c | 10 +++++----- 3 files changed, 31 insertions(+), 31 deletions(-) diff --git a/daemon-utils.c b/daemon-utils.c index 3804bc60973..190da01aea9 100644 --- a/daemon-utils.c +++ b/daemon-utils.c @@ -230,44 +230,44 @@ static int addrcmp(const struct sockaddr_storage *s1, } void add_child(struct child_process *cld, struct sockaddr *addr, socklen_t addrlen, - struct child *firstborn , unsigned int *live_children) + struct child *first, unsigned int *live_children) { - struct child *newborn, **cradle; + struct child *new_cld, **current; - CALLOC_ARRAY(newborn, 1); + CALLOC_ARRAY(new_cld, 1); live_children++; - memcpy(&newborn->cld, cld, sizeof(*cld)); - memcpy(&newborn->address, addr, addrlen); - for (cradle = &firstborn; *cradle; cradle = &(*cradle)->next) - if (!addrcmp(&(*cradle)->address, &newborn->address)) + memcpy(&new_cld->cld, cld, sizeof(*cld)); + memcpy(&new_cld->address, addr, addrlen); + for (current = &first; *current; current = &(*current)->next) + if (!addrcmp(&(*current)->address, &new_cld->address)) break; - newborn->next = *cradle; - *cradle = newborn; + new_cld->next = *current; + *current = new_cld; } -void kill_some_child(struct child *firstborn) +void kill_some_child(struct child *first) { - const struct child *blanket, *next; + const struct child *current, *next; - if (!(blanket = firstborn)) + if (!(current = first)) return; - for (; (next = blanket->next); blanket = next) - if (!addrcmp(&blanket->address, &next->address)) { - kill(blanket->cld.pid, SIGTERM); + for (; (next = current->next); current = next) + if (!addrcmp(¤t->address, &next->address)) { + kill(current->cld.pid, SIGTERM); break; } } -void check_dead_children(struct child *firstborn, unsigned int *live_children, +void check_dead_children(struct child *first, unsigned int *live_children, log_fn loginfo) { int status; pid_t pid; - struct child **cradle, *blanket; - for (cradle = &firstborn; (blanket = *cradle);) - if ((pid = waitpid(blanket->cld.pid, &status, WNOHANG)) > 1) { + struct child **ptr, *current; + for (ptr = &first; (current = *ptr);) + if ((pid = waitpid(current->cld.pid, &status, WNOHANG)) > 1) { if (loginfo) { const char *dead = ""; if (status) @@ -277,10 +277,10 @@ void check_dead_children(struct child *firstborn, unsigned int *live_children, } /* remove the child */ - *cradle = blanket->next; + *ptr = current->next; live_children--; - child_process_clear(&blanket->cld); - free(blanket); + child_process_clear(¤t->cld); + free(current); } else - cradle = &blanket->next; + ptr = ¤t->next; } diff --git a/daemon-utils.h b/daemon-utils.h index fe8d9d05256..e87bc7b9567 100644 --- a/daemon-utils.h +++ b/daemon-utils.h @@ -28,11 +28,11 @@ struct child { }; void add_child(struct child_process *cld, struct sockaddr *addr, socklen_t addrlen, - struct child *firstborn, unsigned int *live_children); + struct child *first, unsigned int *live_children); -void kill_some_child(struct child *firstborn); +void kill_some_child(struct child *first); -void check_dead_children(struct child *firstborn, unsigned int *live_children, +void check_dead_children(struct child *first, unsigned int *live_children, log_fn loginfo); #endif diff --git a/daemon.c b/daemon.c index ec3b407ecbc..d3e7d81de18 100644 --- a/daemon.c +++ b/daemon.c @@ -789,7 +789,7 @@ static int max_connections = 32; static unsigned int live_children; -static struct child *firstborn; +static struct child *first_child; static struct strvec cld_argv = STRVEC_INIT; static void handle(int incoming, struct sockaddr *addr, socklen_t addrlen) @@ -797,9 +797,9 @@ static void handle(int incoming, struct sockaddr *addr, socklen_t addrlen) struct child_process cld = CHILD_PROCESS_INIT; if (max_connections && live_children >= max_connections) { - kill_some_child(firstborn); + kill_some_child(first_child); sleep(1); /* give it some time to die */ - check_dead_children(firstborn, &live_children, loginfo); + check_dead_children(first_child, &live_children, loginfo); if (live_children >= max_connections) { close(incoming); logerror("Too many children, dropping connection"); @@ -832,7 +832,7 @@ static void handle(int incoming, struct sockaddr *addr, socklen_t addrlen) if (start_command(&cld)) logerror("unable to fork"); else - add_child(&cld, addr, addrlen, firstborn, &live_children); + add_child(&cld, addr, addrlen, first_child, &live_children); } static void child_handler(int signo) @@ -862,7 +862,7 @@ static int service_loop(struct socketlist *socklist) for (;;) { int i; - check_dead_children(firstborn, &live_children, loginfo); + check_dead_children(first_child, &live_children, loginfo); if (poll(pfd, socklist->nr, -1) < 0) { if (errno != EINTR) { From patchwork Wed Jan 11 22:13:04 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Matthew John Cheetham X-Patchwork-Id: 13097283 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 07AC9C54EBC for ; Wed, 11 Jan 2023 22:13:33 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S236142AbjAKWNb (ORCPT ); Wed, 11 Jan 2023 17:13:31 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:33428 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S236061AbjAKWNU (ORCPT ); Wed, 11 Jan 2023 17:13:20 -0500 Received: from mail-wr1-x434.google.com (mail-wr1-x434.google.com [IPv6:2a00:1450:4864:20::434]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 41E1A36337 for ; Wed, 11 Jan 2023 14:13:18 -0800 (PST) Received: by mail-wr1-x434.google.com with SMTP id az7so16420408wrb.5 for ; Wed, 11 Jan 2023 14:13:18 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=cc:to:mime-version:content-transfer-encoding:fcc:subject:date:from :references:in-reply-to:message-id:from:to:cc:subject:date :message-id:reply-to; bh=cKNlkhSdZ7ACwMk9ugShnF1r2+k5RVjpgvl5Ij8APwY=; b=o0dJJYuiPalsIkQndb78UlPmoX+HyRgm4j6wSkALFKg8is5CmsC82aryHJIMZlo5aR VKgStqNidXzcZP6V8UariwposaKyOyr2WEwvjpPpid3+rwT4qdNJ14ZcL6Micwgl+IUj tEBTHH+qluOqlWjBdwrBfe71vEeXBBgxKph8ZyV9GyniLZhpX3LfLUSrMUlBy0O3q1Kz CZWZQVo/6B/DGMtjVxCrBz9+r7VDivnMp7Ws2GJo+VPNBmSLyRLvSJB6dZvESHcjesZE boMsciJDT78iozpjmTcurI+YqGJoMHDaIHFQ2bC+iJWW3Rq9QmdvC1kgmKQ48OIVWKHY 3DxQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=cc:to:mime-version:content-transfer-encoding:fcc:subject:date:from :references:in-reply-to:message-id:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=cKNlkhSdZ7ACwMk9ugShnF1r2+k5RVjpgvl5Ij8APwY=; b=tLbj90IIEt8fwcMbcy2z2MlgJWe2zVoq/+XehHabzYqLw/IXpMO1xM+6msyo81gG0a Efyd07rO1wpzwYFvuDbk4T/KXBTpfd3mnRNG5iMwRR/VavZnYip5GemGiZ6w5lWiZcgF iYO6DlTrJLN76tJo+XzWvX2Ot3bneeSXvtURXziitjH/6F8jyWWZq4baPgIlKmw1ENDO DPMzjgDqv8Uj9Nh5//6a1QxYpUwLT+Rzp2Al92atHAyk8i8IUUzXJ990qUNJnK48IXV6 k0bHWtssdNFkkale2Z4LiqAvmAAQuw4YAoKf1ohNG17t7dQOxLIv3s8bXJBzJ9n/jTpV X49Q== X-Gm-Message-State: AFqh2kqL/FCLB2IScjTnQQRXKMxuEC3E0FVC28INxcG0BKdcKqDNIlAc veeP/XF2vmm3TaCr00eNQ2XvKLuuNtQ= X-Google-Smtp-Source: AMrXdXutfjvH0W1FSNmrnkZ9P4zZwY8C2w6cMZeVz3c+4EcGs+52Rkm2x7Ib6YNKn9sq7Wo4ypnsOw== X-Received: by 2002:adf:e383:0:b0:267:b8df:932b with SMTP id e3-20020adfe383000000b00267b8df932bmr57048404wrm.23.1673475196344; Wed, 11 Jan 2023 14:13:16 -0800 (PST) Received: from [127.0.0.1] ([13.74.141.28]) by smtp.gmail.com with ESMTPSA id f8-20020a0560001b0800b002423edd7e50sm14688805wrz.32.2023.01.11.14.13.15 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 11 Jan 2023 14:13:15 -0800 (PST) Message-Id: <706fb3781bd383380a7b1fd30495eb2da970b5ec.1673475190.git.gitgitgadget@gmail.com> In-Reply-To: References: Date: Wed, 11 Jan 2023 22:13:04 +0000 Subject: [PATCH v5 04/10] test-http-server: add stub HTTP server test helper Fcc: Sent MIME-Version: 1.0 To: git@vger.kernel.org Cc: Derrick Stolee , Lessley Dennington , Matthew John Cheetham , M Hickford , Jeff Hostetler , Glen Choo , Victoria Dye , Matthew John Cheetham , Matthew John Cheetham Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org From: Matthew John Cheetham From: Matthew John Cheetham Introduce a mini HTTP server helper that in the future will be enhanced to provide a frontend for the git-http-backend, with support for arbitrary authentication schemes. Right now, test-http-server is a pared-down copy of the git-daemon that always returns a 501 Not Implemented response to all callers. Signed-off-by: Matthew John Cheetham --- Makefile | 1 + contrib/buildsystems/CMakeLists.txt | 11 +- t/helper/.gitignore | 1 + t/helper/test-http-server.c | 385 ++++++++++++++++++++++++++++ 4 files changed, 396 insertions(+), 2 deletions(-) create mode 100644 t/helper/test-http-server.c diff --git a/Makefile b/Makefile index 2654094dbb5..3cd61c792ac 100644 --- a/Makefile +++ b/Makefile @@ -865,6 +865,7 @@ TEST_BUILTINS_OBJS += test-xml-encode.o # Do not add more tests here unless they have extra dependencies. Add # them in TEST_BUILTINS_OBJS above. TEST_PROGRAMS_NEED_X += test-fake-ssh +TEST_PROGRAMS_NEED_X += test-http-server TEST_PROGRAMS_NEED_X += test-tool TEST_PROGRAMS = $(patsubst %,t/helper/%$X,$(TEST_PROGRAMS_NEED_X)) diff --git a/contrib/buildsystems/CMakeLists.txt b/contrib/buildsystems/CMakeLists.txt index 2f6e0197ffa..5d949dcb16c 100644 --- a/contrib/buildsystems/CMakeLists.txt +++ b/contrib/buildsystems/CMakeLists.txt @@ -961,6 +961,9 @@ if(BUILD_TESTING) add_executable(test-fake-ssh ${CMAKE_SOURCE_DIR}/t/helper/test-fake-ssh.c) target_link_libraries(test-fake-ssh common-main) +add_executable(test-http-server ${CMAKE_SOURCE_DIR}/t/helper/test-http-server.c) +target_link_libraries(test-http-server common-main) + #reftable-tests parse_makefile_for_sources(test-reftable_SOURCES "REFTABLE_TEST_OBJS") list(TRANSFORM test-reftable_SOURCES PREPEND "${CMAKE_SOURCE_DIR}/") @@ -980,6 +983,11 @@ if(MSVC) PROPERTIES RUNTIME_OUTPUT_DIRECTORY_DEBUG ${CMAKE_BINARY_DIR}/t/helper) set_target_properties(test-fake-ssh test-tool PROPERTIES RUNTIME_OUTPUT_DIRECTORY_RELEASE ${CMAKE_BINARY_DIR}/t/helper) + + set_target_properties(test-http-server + PROPERTIES RUNTIME_OUTPUT_DIRECTORY_DEBUG ${CMAKE_BINARY_DIR}/t/helper) + set_target_properties(test-http-server + PROPERTIES RUNTIME_OUTPUT_DIRECTORY_RELEASE ${CMAKE_BINARY_DIR}/t/helper) endif() #wrapper scripts @@ -987,8 +995,7 @@ set(wrapper_scripts git git-upload-pack git-receive-pack git-upload-archive git-shell git-remote-ext scalar) set(wrapper_test_scripts - test-fake-ssh test-tool) - + test-http-server test-fake-ssh test-tool) foreach(script ${wrapper_scripts}) file(STRINGS ${CMAKE_SOURCE_DIR}/wrap-for-bin.sh content NEWLINE_CONSUME) diff --git a/t/helper/.gitignore b/t/helper/.gitignore index 8c2ddcce95f..9aa9c752997 100644 --- a/t/helper/.gitignore +++ b/t/helper/.gitignore @@ -1,2 +1,3 @@ /test-tool /test-fake-ssh +/test-http-server diff --git a/t/helper/test-http-server.c b/t/helper/test-http-server.c new file mode 100644 index 00000000000..11071b1dd89 --- /dev/null +++ b/t/helper/test-http-server.c @@ -0,0 +1,385 @@ +#include "daemon-utils.h" +#include "config.h" +#include "run-command.h" +#include "strbuf.h" +#include "string-list.h" +#include "trace2.h" +#include "version.h" +#include "dir.h" +#include "date.h" + +#define TR2_CAT "test-http-server" + +static const char *pid_file; +static int verbose; +static int reuseaddr; + +static const char test_http_auth_usage[] = +"http-server [--verbose]\n" +" [--timeout=] [--max-connections=]\n" +" [--reuseaddr] [--pid-file=]\n" +" [--listen=]* [--port=]\n" +; + +static unsigned int timeout; + +static void logreport(const char *label, const char *err, va_list params) +{ + struct strbuf msg = STRBUF_INIT; + + strbuf_addf(&msg, "[%"PRIuMAX"] %s: ", (uintmax_t)getpid(), label); + strbuf_vaddf(&msg, err, params); + strbuf_addch(&msg, '\n'); + + fwrite(msg.buf, sizeof(char), msg.len, stderr); + fflush(stderr); + + strbuf_release(&msg); +} + +__attribute__((format (printf, 1, 2))) +static void logerror(const char *err, ...) +{ + va_list params; + va_start(params, err); + logreport("error", err, params); + va_end(params); +} + +__attribute__((format (printf, 1, 2))) +static void loginfo(const char *err, ...) +{ + va_list params; + if (!verbose) + return; + va_start(params, err); + logreport("info", err, params); + va_end(params); +} + +/* + * The code in this section is used by "worker" instances to service + * a single connection from a client. The worker talks to the client + * on 0 and 1. + */ + +enum worker_result { + /* + * Operation successful. + * Caller *might* keep the socket open and allow keep-alive. + */ + WR_OK = 0, + + /* + * Various errors while processing the request and/or the response. + * Close the socket and clean up. + * Exit child-process with non-zero status. + */ + WR_IO_ERROR = 1<<0, + + /* + * Close the socket and clean up. Does not imply an error. + */ + WR_HANGUP = 1<<1, +}; + +static enum worker_result worker(void) +{ + const char *response = "HTTP/1.1 501 Not Implemented\r\n"; + char *client_addr = getenv("REMOTE_ADDR"); + char *client_port = getenv("REMOTE_PORT"); + enum worker_result wr = WR_OK; + + if (client_addr) + loginfo("Connection from %s:%s", client_addr, client_port); + + set_keep_alive(0, logerror); + + while (1) { + if (write_in_full(STDOUT_FILENO, response, strlen(response)) < 0) { + logerror("unable to write response"); + wr = WR_IO_ERROR; + } + + if (wr != WR_OK) + break; + } + + close(STDIN_FILENO); + close(STDOUT_FILENO); + + return !!(wr & WR_IO_ERROR); +} + +static int max_connections = 32; + +static unsigned int live_children; + +static struct child *first_child; + +static struct strvec cld_argv = STRVEC_INIT; +static void handle(int incoming, struct sockaddr *addr, socklen_t addrlen) +{ + struct child_process cld = CHILD_PROCESS_INIT; + + if (max_connections && live_children >= max_connections) { + kill_some_child(first_child); + sleep(1); /* give it some time to die */ + check_dead_children(first_child, &live_children, loginfo); + if (live_children >= max_connections) { + close(incoming); + logerror("Too many children, dropping connection"); + return; + } + } + + if (addr->sa_family == AF_INET) { + char buf[128] = ""; + struct sockaddr_in *sin_addr = (void *) addr; + inet_ntop(addr->sa_family, &sin_addr->sin_addr, buf, sizeof(buf)); + strvec_pushf(&cld.env, "REMOTE_ADDR=%s", buf); + strvec_pushf(&cld.env, "REMOTE_PORT=%d", + ntohs(sin_addr->sin_port)); +#ifndef NO_IPV6 + } else if (addr->sa_family == AF_INET6) { + char buf[128] = ""; + struct sockaddr_in6 *sin6_addr = (void *) addr; + inet_ntop(AF_INET6, &sin6_addr->sin6_addr, buf, sizeof(buf)); + strvec_pushf(&cld.env, "REMOTE_ADDR=[%s]", buf); + strvec_pushf(&cld.env, "REMOTE_PORT=%d", + ntohs(sin6_addr->sin6_port)); +#endif + } + + strvec_pushv(&cld.args, cld_argv.v); + cld.in = incoming; + cld.out = dup(incoming); + + if (cld.out < 0) + logerror("could not dup() `incoming`"); + else if (start_command(&cld)) + logerror("unable to fork"); + else + add_child(&cld, addr, addrlen, first_child, &live_children); +} + +static void child_handler(int signo) +{ + /* + * Otherwise empty handler because systemcalls will get interrupted + * upon signal receipt + * SysV needs the handler to be rearmed + */ + signal(SIGCHLD, child_handler); +} + +static int service_loop(struct socketlist *socklist) +{ + struct pollfd *pfd; + int i; + + CALLOC_ARRAY(pfd, socklist->nr); + + for (i = 0; i < socklist->nr; i++) { + pfd[i].fd = socklist->list[i]; + pfd[i].events = POLLIN; + } + + signal(SIGCHLD, child_handler); + + for (;;) { + int i; + int nr_ready; + int timeout = (pid_file ? 100 : -1); + + check_dead_children(first_child, &live_children, loginfo); + + nr_ready = poll(pfd, socklist->nr, timeout); + if (nr_ready < 0) { + if (errno != EINTR) { + logerror("Poll failed, resuming: %s", + strerror(errno)); + sleep(1); + } + continue; + } + else if (nr_ready == 0) { + /* + * If we have a pid_file, then we watch it. + * If someone deletes it, we shutdown the service. + * The shell scripts in the test suite will use this. + */ + if (!pid_file || file_exists(pid_file)) + continue; + goto shutdown; + } + + for (i = 0; i < socklist->nr; i++) { + if (pfd[i].revents & POLLIN) { + union { + struct sockaddr sa; + struct sockaddr_in sai; +#ifndef NO_IPV6 + struct sockaddr_in6 sai6; +#endif + } ss; + socklen_t sslen = sizeof(ss); + int incoming = accept(pfd[i].fd, &ss.sa, &sslen); + if (incoming < 0) { + switch (errno) { + case EAGAIN: + case EINTR: + case ECONNABORTED: + continue; + default: + die_errno("accept returned"); + } + } + handle(incoming, &ss.sa, sslen); + } + } + } + +shutdown: + loginfo("Starting graceful shutdown (pid-file gone)"); + for (i = 0; i < socklist->nr; i++) + close(socklist->list[i]); + + return 0; +} + +static int serve(struct string_list *listen_addr, int listen_port) +{ + struct socketlist socklist = { NULL, 0, 0 }; + + socksetup(listen_addr, listen_port, &socklist, reuseaddr, logerror); + if (socklist.nr == 0) + die("unable to allocate any listen sockets on port %u", + listen_port); + + loginfo("Ready to rumble"); + + /* + * Wait to create the pid-file until we've setup the sockets + * and are open for business. + */ + if (pid_file) + write_file(pid_file, "%"PRIuMAX, (uintmax_t) getpid()); + + return service_loop(&socklist); +} + +/* + * This section is executed by both the primary instance and all + * worker instances. So, yes, each child-process re-parses the + * command line argument and re-discovers how it should behave. + */ + +int cmd_main(int argc, const char **argv) +{ + int listen_port = 0; + struct string_list listen_addr = STRING_LIST_INIT_NODUP; + int worker_mode = 0; + int i; + + trace2_cmd_name("test-http-server"); + trace2_cmd_list_config(); + trace2_cmd_list_env_vars(); + setup_git_directory_gently(NULL); + + for (i = 1; i < argc; i++) { + const char *arg = argv[i]; + const char *v; + + if (skip_prefix(arg, "--listen=", &v)) { + string_list_append(&listen_addr, xstrdup_tolower(v)); + continue; + } + if (skip_prefix(arg, "--port=", &v)) { + char *end; + unsigned long n; + n = strtoul(v, &end, 0); + if (*v && !*end) { + listen_port = n; + continue; + } + } + if (!strcmp(arg, "--worker")) { + worker_mode = 1; + trace2_cmd_mode("worker"); + continue; + } + if (!strcmp(arg, "--verbose")) { + verbose = 1; + continue; + } + if (skip_prefix(arg, "--timeout=", &v)) { + timeout = atoi(v); + continue; + } + if (skip_prefix(arg, "--max-connections=", &v)) { + max_connections = atoi(v); + if (max_connections < 0) + max_connections = 0; /* unlimited */ + continue; + } + if (!strcmp(arg, "--reuseaddr")) { + reuseaddr = 1; + continue; + } + if (skip_prefix(arg, "--pid-file=", &v)) { + pid_file = v; + continue; + } + + fprintf(stderr, "error: unknown argument '%s'\n", arg); + usage(test_http_auth_usage); + } + + /* avoid splitting a message in the middle */ + setvbuf(stderr, NULL, _IOFBF, 4096); + + if (listen_port == 0) + listen_port = DEFAULT_GIT_PORT; + + /* + * If no --listen= args are given, the setup_named_sock() + * code will use receive a NULL address and set INADDR_ANY. + * This exposes both internal and external interfaces on the + * port. + * + * Disallow that and default to the internal-use-only loopback + * address. + */ + if (!listen_addr.nr) + string_list_append(&listen_addr, "127.0.0.1"); + + /* + * worker_mode is set in our own child process instances + * (that are bound to a connected socket from a client). + */ + if (worker_mode) + return worker(); + + /* + * `cld_argv` is a bit of a clever hack. The top-level instance + * of test-http-server does the normal bind/listen/accept stuff. + * For each incoming socket, the top-level process spawns + * a child instance of test-http-server *WITH* the additional + * `--worker` argument. This causes the child to set `worker_mode` + * and immediately call `worker()` using the connected socket (and + * without the usual need for fork() or threads). + * + * The magic here is made possible because `cld_argv` is static + * and handle() (called by service_loop()) knows about it. + */ + strvec_push(&cld_argv, argv[0]); + strvec_push(&cld_argv, "--worker"); + for (i = 1; i < argc; ++i) + strvec_push(&cld_argv, argv[i]); + + /* + * Setup primary instance to listen for connections. + */ + return serve(&listen_addr, listen_port); +} From patchwork Wed Jan 11 22:13:05 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Matthew John Cheetham X-Patchwork-Id: 13097285 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 34940C5479D for ; Wed, 11 Jan 2023 22:14:10 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S236198AbjAKWNd (ORCPT ); Wed, 11 Jan 2023 17:13:33 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:33436 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S236062AbjAKWNU (ORCPT ); Wed, 11 Jan 2023 17:13:20 -0500 Received: from mail-wm1-x333.google.com (mail-wm1-x333.google.com [IPv6:2a00:1450:4864:20::333]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 36B6A3E0E1 for ; Wed, 11 Jan 2023 14:13:19 -0800 (PST) Received: by mail-wm1-x333.google.com with SMTP id ja17so12030048wmb.3 for ; Wed, 11 Jan 2023 14:13:19 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=cc:to:mime-version:content-transfer-encoding:fcc:subject:date:from :references:in-reply-to:message-id:from:to:cc:subject:date :message-id:reply-to; bh=qs6h0DoO/ZYvBRFEyqt00y4wrQgWlKREO3Bh+Yt2h4o=; b=W8sn2kweNQwxKuXaWq1FG4y9Hb0vPSqdZ160frLjmZ9BSHqht8uTWOWf5p8pFhkP86 cnfHWOxY+GMe9t+dc1ZZ+zC450Xz0r6K5zZx7rru8iUjZNGcaAg5weUlhOBkCjaCYICZ Y0lAWLNlFJJU/RG65xbpuTeEhi8iVyQrw3et3t8hIOi2Zfw0QlDipNTvX9rwSZ01N3dk BQ4SZXwEQtrJ7EcEFDhrCmAdDwuWghpT6/5O120UoOZzAtB5ZjHixt4iPppHTxABvaY+ dvwT57haMgCH6yu0Sv61badA/IMa1Kui0nnbth36JBJDMLC9Bm/OysET/9m2uU55xHN2 3xlQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=cc:to:mime-version:content-transfer-encoding:fcc:subject:date:from :references:in-reply-to:message-id:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=qs6h0DoO/ZYvBRFEyqt00y4wrQgWlKREO3Bh+Yt2h4o=; b=YKs8OEJkUwYJeKK+LR6TqXgicE+pJfPXE4pA4f+BQt/qSyCJrw6ZGcdoHBoTDwcref h/EbfXBEK+/GSUuiB2YK2zpJCIW1vj9pLobNCWJ5n95PEpVjTsnv7LPksvg+DczSSnNR wGuDLOXpcWsiIjInNYXExsWdvZGmwJGJnJTKOjDlz824XL+sK09mud/f/Ydu0j+aCG/O UGljmucAfGcJYOVREhO2S4gYpL7uil55pdouhZzCdfVHPdlJaKYwoJ8yWVafLdp0CnOn tee3kasO5y/JlIyRuNMbQVvaNq6NytB9QejBjo7PqMC0VkyVNTLFkBqrQp3NoEO1WZE3 6/cg== X-Gm-Message-State: AFqh2ko+xlNdVW+4qHrPsc5uMlQrMa5cF5PpTkOykEbIC8xIo3r7XuJ3 Hw2NV9P3KjAzJMrSZFvidb7b4zPnybQ= X-Google-Smtp-Source: AMrXdXuyGIQJIz8gCojVigkSYk3XlEbXWXrRW1apfW60oAliiALuzmCLqHcNJQd7iUA1QAqHKovyNA== X-Received: by 2002:a05:600c:3844:b0:3d3:4406:8a3a with SMTP id s4-20020a05600c384400b003d344068a3amr64079731wmr.30.1673475197436; Wed, 11 Jan 2023 14:13:17 -0800 (PST) Received: from [127.0.0.1] ([13.74.141.28]) by smtp.gmail.com with ESMTPSA id m25-20020a05600c3b1900b003d9ed49ee2bsm15321910wms.1.2023.01.11.14.13.16 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 11 Jan 2023 14:13:16 -0800 (PST) Message-Id: <6f66bf146b4d6e4044b3c6c2224795918191bc3f.1673475190.git.gitgitgadget@gmail.com> In-Reply-To: References: Date: Wed, 11 Jan 2023 22:13:05 +0000 Subject: [PATCH v5 05/10] test-http-server: add HTTP error response function Fcc: Sent MIME-Version: 1.0 To: git@vger.kernel.org Cc: Derrick Stolee , Lessley Dennington , Matthew John Cheetham , M Hickford , Jeff Hostetler , Glen Choo , Victoria Dye , Matthew John Cheetham , Matthew John Cheetham Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org From: Matthew John Cheetham From: Matthew John Cheetham Introduce a function to the test-http-server test helper to write more full and valid HTTP error responses, including all the standard response headers like `Server` and `Date`. Signed-off-by: Matthew John Cheetham --- t/helper/test-http-server.c | 303 +++++++++++++++++++++++++++++++++++- t/t5556-http-auth.sh | 106 +++++++++++++ 2 files changed, 404 insertions(+), 5 deletions(-) create mode 100755 t/t5556-http-auth.sh diff --git a/t/helper/test-http-server.c b/t/helper/test-http-server.c index 11071b1dd89..67bc16354a1 100644 --- a/t/helper/test-http-server.c +++ b/t/helper/test-http-server.c @@ -83,9 +83,297 @@ enum worker_result { WR_HANGUP = 1<<1, }; +/* + * Fields from a parsed HTTP request. + */ +struct req { + struct strbuf start_line; + + const char *method; + const char *http_version; + + struct strbuf uri_path; + struct strbuf query_args; + + struct string_list header_list; + const char *content_type; + ssize_t content_length; +}; + +#define REQ__INIT { \ + .start_line = STRBUF_INIT, \ + .uri_path = STRBUF_INIT, \ + .query_args = STRBUF_INIT, \ + .header_list = STRING_LIST_INIT_NODUP, \ + .content_type = NULL, \ + .content_length = -1 \ + } + +static void req__release(struct req *req) +{ + strbuf_release(&req->start_line); + + strbuf_release(&req->uri_path); + strbuf_release(&req->query_args); + + string_list_clear(&req->header_list, 0); +} + +static enum worker_result send_http_error( + int fd, + int http_code, const char *http_code_name, + int retry_after_seconds, struct string_list *response_headers, + enum worker_result wr_in) +{ + struct strbuf response_header = STRBUF_INIT; + struct strbuf response_content = STRBUF_INIT; + struct string_list_item *h; + enum worker_result wr; + + strbuf_addf(&response_content, "Error: %d %s\r\n", + http_code, http_code_name); + if (retry_after_seconds > 0) + strbuf_addf(&response_content, "Retry-After: %d\r\n", + retry_after_seconds); + + strbuf_addf (&response_header, "HTTP/1.1 %d %s\r\n", http_code, http_code_name); + strbuf_addstr(&response_header, "Cache-Control: private\r\n"); + strbuf_addstr(&response_header, "Content-Type: text/plain\r\n"); + strbuf_addf (&response_header, "Content-Length: %d\r\n", (int)response_content.len); + if (retry_after_seconds > 0) + strbuf_addf(&response_header, "Retry-After: %d\r\n", retry_after_seconds); + strbuf_addf( &response_header, "Server: test-http-server/%s\r\n", git_version_string); + strbuf_addf( &response_header, "Date: %s\r\n", show_date(time(NULL), 0, DATE_MODE(RFC2822))); + if (response_headers) + for_each_string_list_item(h, response_headers) + strbuf_addf(&response_header, "%s\r\n", h->string); + strbuf_addstr(&response_header, "\r\n"); + + if (write_in_full(fd, response_header.buf, response_header.len) < 0) { + logerror("unable to write response header"); + wr = WR_IO_ERROR; + goto done; + } + + if (write_in_full(fd, response_content.buf, response_content.len) < 0) { + logerror("unable to write response content body"); + wr = WR_IO_ERROR; + goto done; + } + + wr = wr_in; + +done: + strbuf_release(&response_header); + strbuf_release(&response_content); + + return wr; +} + +/* + * Read the HTTP request up to the start of the optional message-body. + * We do this byte-by-byte because we have keep-alive turned on and + * cannot rely on an EOF. + * + * https://tools.ietf.org/html/rfc7230 + * + * We cannot call die() here because our caller needs to properly + * respond to the client and/or close the socket before this + * child exits so that the client doesn't get a connection reset + * by peer error. + */ +static enum worker_result req__read(struct req *req, int fd) +{ + struct strbuf h = STRBUF_INIT; + struct string_list start_line_fields = STRING_LIST_INIT_DUP; + int nr_start_line_fields; + const char *uri_target; + const char *query; + char *hp; + const char *hv; + + enum worker_result result = WR_OK; + + /* + * Read line 0 of the request and split it into component parts: + * + * SP SP CRLF + * + */ + if (strbuf_getwholeline_fd(&req->start_line, fd, '\n') == EOF) { + result = WR_OK | WR_HANGUP; + goto done; + } + + strbuf_trim_trailing_newline(&req->start_line); + + nr_start_line_fields = string_list_split(&start_line_fields, + req->start_line.buf, + ' ', -1); + if (nr_start_line_fields != 3) { + logerror("could not parse request start-line '%s'", + req->start_line.buf); + result = WR_IO_ERROR; + goto done; + } + + req->method = xstrdup(start_line_fields.items[0].string); + req->http_version = xstrdup(start_line_fields.items[2].string); + + uri_target = start_line_fields.items[1].string; + + if (strcmp(req->http_version, "HTTP/1.1")) { + logerror("unsupported version '%s' (expecting HTTP/1.1)", + req->http_version); + result = WR_IO_ERROR; + goto done; + } + + query = strchr(uri_target, '?'); + + if (query) { + strbuf_add(&req->uri_path, uri_target, (query - uri_target)); + strbuf_trim_trailing_dir_sep(&req->uri_path); + strbuf_addstr(&req->query_args, query + 1); + } else { + strbuf_addstr(&req->uri_path, uri_target); + strbuf_trim_trailing_dir_sep(&req->uri_path); + } + + /* + * Read the set of HTTP headers into a string-list. + */ + while (1) { + if (strbuf_getwholeline_fd(&h, fd, '\n') == EOF) + goto done; + strbuf_trim_trailing_newline(&h); + + if (!h.len) + goto done; /* a blank line ends the header */ + + hp = strbuf_detach(&h, NULL); + string_list_append(&req->header_list, hp); + + /* also store common request headers as struct req members */ + if (skip_prefix(hp, "Content-Type: ", &hv)) { + req->content_type = hv; + } else if (skip_prefix(hp, "Content-Length: ", &hv)) { + req->content_length = strtol(hv, &hp, 10); + } + } + + /* + * We do not attempt to read the , if it exists. + * We let our caller read/chunk it in as appropriate. + */ + +done: + string_list_clear(&start_line_fields, 0); + + /* + * This is useful for debugging the request, but very noisy. + */ + if (trace2_is_enabled()) { + struct string_list_item *item; + trace2_printf("%s: %s", TR2_CAT, req->start_line.buf); + trace2_printf("%s: hver: %s", TR2_CAT, req->http_version); + trace2_printf("%s: hmth: %s", TR2_CAT, req->method); + trace2_printf("%s: path: %s", TR2_CAT, req->uri_path.buf); + trace2_printf("%s: qury: %s", TR2_CAT, req->query_args.buf); + if (req->content_length >= 0) + trace2_printf("%s: clen: %d", TR2_CAT, req->content_length); + if (req->content_type) + trace2_printf("%s: ctyp: %s", TR2_CAT, req->content_type); + for_each_string_list_item(item, &req->header_list) + trace2_printf("%s: hdrs: %s", TR2_CAT, item->string); + } + + return result; +} + +static int is_git_request(struct req *req) +{ + static regex_t *smart_http_regex; + static int initialized; + + if (!initialized) { + smart_http_regex = xmalloc(sizeof(*smart_http_regex)); + /* + * This regular expression matches all dumb and smart HTTP + * requests that are currently in use, and defined in + * Documentation/gitprotocol-http.txt. + * + */ + if (regcomp(smart_http_regex, "^/(HEAD|info/refs|" + "objects/info/[^/]+|git-(upload|receive)-pack)$", + REG_EXTENDED)) { + warning("could not compile smart HTTP regex"); + smart_http_regex = NULL; + } + initialized = 1; + } + + return smart_http_regex && + !regexec(smart_http_regex, req->uri_path.buf, 0, NULL, 0); +} + +static enum worker_result do__git(struct req *req) +{ + const char *ok = "HTTP/1.1 200 OK\r\n"; + struct child_process cp = CHILD_PROCESS_INIT; + int res; + + /* + * Note that we always respond with a 200 OK response even if the + * http-backend process exits with an error. This helper is intended + * only to be used to exercise the HTTP auth handling in the Git client, + * and specifically around authentication (not handled by http-backend). + * + * If we wanted to respond with a more 'valid' HTTP response status then + * we'd need to buffer the output of http-backend, wait for and grok the + * exit status of the process, then write the HTTP status line followed + * by the http-backend output. This is outside of the scope of this test + * helper's use at time of writing. + * + * The important auth responses (401) we are handling prior to getting + * to this point. + */ + if (write(STDOUT_FILENO, ok, strlen(ok)) < 0) + return error(_("could not send '%s'"), ok); + + strvec_pushf(&cp.env, "REQUEST_METHOD=%s", req->method); + strvec_pushf(&cp.env, "PATH_TRANSLATED=%s", + req->uri_path.buf); + strvec_push(&cp.env, "SERVER_PROTOCOL=HTTP/1.1"); + if (req->query_args.len) + strvec_pushf(&cp.env, "QUERY_STRING=%s", + req->query_args.buf); + if (req->content_type) + strvec_pushf(&cp.env, "CONTENT_TYPE=%s", + req->content_type); + if (req->content_length >= 0) + strvec_pushf(&cp.env, "CONTENT_LENGTH=%" PRIdMAX, + (intmax_t)req->content_length); + cp.git_cmd = 1; + strvec_push(&cp.args, "http-backend"); + res = run_command(&cp); + close(STDOUT_FILENO); + close(STDIN_FILENO); + return !!res; +} + +static enum worker_result dispatch(struct req *req) +{ + if (is_git_request(req)) + return do__git(req); + + return send_http_error(STDOUT_FILENO, 501, "Not Implemented", -1, NULL, + WR_OK | WR_HANGUP); +} + static enum worker_result worker(void) { - const char *response = "HTTP/1.1 501 Not Implemented\r\n"; + struct req req = REQ__INIT; char *client_addr = getenv("REMOTE_ADDR"); char *client_port = getenv("REMOTE_PORT"); enum worker_result wr = WR_OK; @@ -96,11 +384,16 @@ static enum worker_result worker(void) set_keep_alive(0, logerror); while (1) { - if (write_in_full(STDOUT_FILENO, response, strlen(response)) < 0) { - logerror("unable to write response"); - wr = WR_IO_ERROR; - } + req__release(&req); + + alarm(timeout); + wr = req__read(&req, 0); + alarm(0); + + if (wr != WR_OK) + break; + wr = dispatch(&req); if (wr != WR_OK) break; } diff --git a/t/t5556-http-auth.sh b/t/t5556-http-auth.sh new file mode 100755 index 00000000000..65105a5a6a9 --- /dev/null +++ b/t/t5556-http-auth.sh @@ -0,0 +1,106 @@ +#!/bin/sh + +test_description='test http auth header and credential helper interop' + +TEST_NO_CREATE_REPO=1 +. ./test-lib.sh + +test_set_port GIT_TEST_HTTP_PROTOCOL_PORT + +# Setup a repository +# +REPO_DIR="$TRASH_DIRECTORY"/repo + +# Setup some lookback URLs where test-http-server will be listening. +# We will spawn it directly inside the repo directory, so we avoid +# any need to configure directory mappings etc - we only serve this +# repository from the root '/' of the server. +# +HOST_PORT=127.0.0.1:$GIT_TEST_HTTP_PROTOCOL_PORT +ORIGIN_URL=http://$HOST_PORT/ + +# The pid-file is created by test-http-server when it starts. +# The server will shutdown if/when we delete it (this is easier than +# killing it by PID). +# +PID_FILE="$TRASH_DIRECTORY"/pid-file.pid +SERVER_LOG="$TRASH_DIRECTORY"/OUT.server.log + +PATH="$GIT_BUILD_DIR/t/helper/:$PATH" && export PATH + +test_expect_success 'setup repos' ' + test_create_repo "$REPO_DIR" && + git -C "$REPO_DIR" branch -M main +' + +stop_http_server () { + if ! test -f "$PID_FILE" + then + return 0 + fi + # + # The server will shutdown automatically when we delete the pid-file. + # + rm -f "$PID_FILE" + # + # Give it a few seconds to shutdown (mainly to completely release the + # port before the next test start another instance and it attempts to + # bind to it). + # + for k in 0 1 2 3 4 + do + if grep -q "Starting graceful shutdown" "$SERVER_LOG" + then + return 0 + fi + sleep 1 + done + + echo "stop_http_server: timeout waiting for server shutdown" + return 1 +} + +start_http_server () { + # + # Launch our server into the background in repo_dir. + # + ( + cd "$REPO_DIR" + test-http-server --verbose \ + --listen=127.0.0.1 \ + --port=$GIT_TEST_HTTP_PROTOCOL_PORT \ + --reuseaddr \ + --pid-file="$PID_FILE" \ + "$@" \ + 2>"$SERVER_LOG" & + ) + # + # Give it a few seconds to get started. + # + for k in 0 1 2 3 4 + do + if test -f "$PID_FILE" + then + return 0 + fi + sleep 1 + done + + echo "start_http_server: timeout waiting for server startup" + return 1 +} + +per_test_cleanup () { + stop_http_server && + rm -f OUT.* +} + +test_expect_success 'http auth anonymous no challenge' ' + test_when_finished "per_test_cleanup" && + start_http_server && + + # Attempt to read from a protected repository + git ls-remote $ORIGIN_URL +' + +test_done From patchwork Wed Jan 11 22:13:06 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Matthew John Cheetham X-Patchwork-Id: 13097287 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 57544C46467 for ; Wed, 11 Jan 2023 22:14:27 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S237663AbjAKWOL (ORCPT ); Wed, 11 Jan 2023 17:14:11 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:33454 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S236068AbjAKWNV (ORCPT ); Wed, 11 Jan 2023 17:13:21 -0500 Received: from mail-wm1-x334.google.com (mail-wm1-x334.google.com [IPv6:2a00:1450:4864:20::334]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 1ECE53E85F for ; Wed, 11 Jan 2023 14:13:20 -0800 (PST) Received: by mail-wm1-x334.google.com with SMTP id i17-20020a05600c355100b003d99434b1cfso13754097wmq.1 for ; Wed, 11 Jan 2023 14:13:20 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=cc:to:mime-version:content-transfer-encoding:fcc:subject:date:from :references:in-reply-to:message-id:from:to:cc:subject:date :message-id:reply-to; bh=1vrHUk66dglrraLfjVBbj2TwF/Y7O64XMwZ7YW+qI48=; b=Q+VygEw6ae8AbnFfpCHY8W5k28DOmwr+5C1drbRCzer8UNvlU6kRL/5EPRgTqcne3J i04Q21edtcZeM/NrfiGFXKWTiCY/LiLp1c2GISwPWNFm55Uw6c6PKkK6Stb3OZJ5WBWz iEv5c+bw0i4d3W6VzWchRgr+1hwTQ1Yaevk3+VY1bL7P0G1miVVQB/ZH8Pv+zC+tAlsA re6+mVMNrYTgqUNAYfzHzdSoa+QaSIYPIlCuHp63NDAhlAnkfVaiHXkInsLZDEYuT0As T4e6vJPt+MhYfSGqrjKGIJKtscHibxFUquJHrn9O3Ie53bV9Lfvhd+xeEvcRORtEqZqC J2pw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=cc:to:mime-version:content-transfer-encoding:fcc:subject:date:from :references:in-reply-to:message-id:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=1vrHUk66dglrraLfjVBbj2TwF/Y7O64XMwZ7YW+qI48=; b=PU3emBuumlyT7PXH0BZbloTM05wHfj9AXddIqrf3bqtP4VHz4cBfpNkwo1BrMd/rB9 CByZae3C8+zFIznrsURrdrsD2OCkUxAaS5fjjXjihPN8v5zBf3Jr5BnyEre9m7WOV29t 6dXXeX2Ac6HtQYGBD/VfJ0WxyIdoKj7r6xwkagOqree6b7qYZySUfPjd1W1crnwpvCw9 vQjgIlHqAAsn8TgiMDAtox5SK+xlcM3muK/oOZ91ZYDEzczM4U21Ii7XrjhdjrpF1pgc lHrZSILaXT3ng2aCm6kVDsaQnBCb/PToB/p98n7Xzm8Y1OwyT+/6O0ADc7TMZeHyhs6O nrlg== X-Gm-Message-State: AFqh2kowSXjef5/OTLTWfX2p/o1yd7QsyJnaNTHK/IqMbNR8NOQlbMv/ m/XIoFDNYKEiXQSfkeBxm0meLLjUxRM= X-Google-Smtp-Source: AMrXdXt4UbrO9FaXrJuuLQyiWPzXAyZ1ZoEdv/DEtoEH9PUu+QJz3IibSjIGdcU9HKsUG4LKMfroRg== X-Received: by 2002:a05:600c:3b93:b0:3d2:1bf6:5796 with SMTP id n19-20020a05600c3b9300b003d21bf65796mr54588257wms.35.1673475198477; Wed, 11 Jan 2023 14:13:18 -0800 (PST) Received: from [127.0.0.1] ([13.74.141.28]) by smtp.gmail.com with ESMTPSA id p9-20020a05600c358900b003cffd3c3d6csm21616924wmq.12.2023.01.11.14.13.17 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 11 Jan 2023 14:13:17 -0800 (PST) Message-Id: In-Reply-To: References: Date: Wed, 11 Jan 2023 22:13:06 +0000 Subject: [PATCH v5 06/10] test-http-server: add simple authentication Fcc: Sent MIME-Version: 1.0 To: git@vger.kernel.org Cc: Derrick Stolee , Lessley Dennington , Matthew John Cheetham , M Hickford , Jeff Hostetler , Glen Choo , Victoria Dye , Matthew John Cheetham , Matthew John Cheetham Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org From: Matthew John Cheetham From: Matthew John Cheetham Add simple authentication to the test-http-server test helper. Authentication schemes and sets of valid tokens can be specified via a configuration file (in the normal gitconfig file format). Incoming requests are compared against the set of valid schemes and tokens and only approved if a matching token is found, or if no auth was provided and anonymous auth is enabled. Signed-off-by: Matthew John Cheetham --- t/helper/test-http-server.c | 246 +++++++++++++++++++++++++++++++++++- 1 file changed, 244 insertions(+), 2 deletions(-) diff --git a/t/helper/test-http-server.c b/t/helper/test-http-server.c index 67bc16354a1..dcc326c8652 100644 --- a/t/helper/test-http-server.c +++ b/t/helper/test-http-server.c @@ -7,6 +7,7 @@ #include "version.h" #include "dir.h" #include "date.h" +#include "config.h" #define TR2_CAT "test-http-server" @@ -19,6 +20,7 @@ static const char test_http_auth_usage[] = " [--timeout=] [--max-connections=]\n" " [--reuseaddr] [--pid-file=]\n" " [--listen=]* [--port=]\n" +" [--auth-config=]\n" ; static unsigned int timeout; @@ -317,7 +319,7 @@ static int is_git_request(struct req *req) !regexec(smart_http_regex, req->uri_path.buf, 0, NULL, 0); } -static enum worker_result do__git(struct req *req) +static enum worker_result do__git(struct req *req, const char *user) { const char *ok = "HTTP/1.1 200 OK\r\n"; struct child_process cp = CHILD_PROCESS_INIT; @@ -341,6 +343,9 @@ static enum worker_result do__git(struct req *req) if (write(STDOUT_FILENO, ok, strlen(ok)) < 0) return error(_("could not send '%s'"), ok); + if (user) + strvec_pushf(&cp.env, "REMOTE_USER=%s", user); + strvec_pushf(&cp.env, "REQUEST_METHOD=%s", req->method); strvec_pushf(&cp.env, "PATH_TRANSLATED=%s", req->uri_path.buf); @@ -362,10 +367,234 @@ static enum worker_result do__git(struct req *req) return !!res; } +enum auth_result { + /* No auth module matches the request. */ + AUTH_UNKNOWN = 0, + + /* Auth module denied the request. */ + AUTH_DENY = 1, + + /* Auth module successfully validated the request. */ + AUTH_ALLOW = 2, +}; + +struct auth_module { + char *scheme; + char *challenge_params; + struct string_list *tokens; +}; + +static int allow_anonymous; +static struct auth_module **auth_modules = NULL; +static size_t auth_modules_nr = 0; +static size_t auth_modules_alloc = 0; +static struct strvec extra_headers = STRVEC_INIT; + +static struct auth_module *create_auth_module(const char *scheme, + const char *challenge) +{ + struct auth_module *mod = xmalloc(sizeof(struct auth_module)); + mod->scheme = xstrdup(scheme); + mod->challenge_params = challenge ? xstrdup(challenge) : NULL; + CALLOC_ARRAY(mod->tokens, 1); + string_list_init_dup(mod->tokens); + return mod; +} + +static struct auth_module *get_auth_module(const char *scheme) +{ + int i; + struct auth_module *mod; + for (i = 0; i < auth_modules_nr; i++) { + mod = auth_modules[i]; + if (!strcasecmp(mod->scheme, scheme)) + return mod; + } + + return NULL; +} + +static int add_auth_module(struct auth_module *mod) +{ + if (get_auth_module(mod->scheme)) + return error("duplicate auth scheme '%s'\n", mod->scheme); + + ALLOC_GROW(auth_modules, auth_modules_nr + 1, auth_modules_alloc); + auth_modules[auth_modules_nr++] = mod; + + return 0; +} + +static int is_authed(struct req *req, const char **user, enum worker_result *wr) +{ + enum auth_result result = AUTH_UNKNOWN; + struct string_list hdrs = STRING_LIST_INIT_NODUP; + struct auth_module *mod; + + struct string_list_item *hdr; + struct string_list_item *token; + const char *v; + struct strbuf **split = NULL; + int i; + char *challenge; + + /* + * Check all auth modules and try to validate the request. + * The first Authorization header that matches a known auth module + * scheme will be consulted to either approve or deny the request. + * If no module is found, or if there is no valid token, then 401 error. + * Otherwise, only permit the request if anonymous auth is enabled. + * It's atypical for user agents/clients to send multiple Authorization + * headers, but not explicitly forbidden or defined. + */ + for_each_string_list_item(hdr, &req->header_list) { + if (skip_iprefix(hdr->string, "Authorization: ", &v)) { + split = strbuf_split_str(v, ' ', 2); + if (!split[0] || !split[1]) continue; + + /* trim trailing space ' ' */ + strbuf_setlen(split[0], split[0]->len - 1); + + mod = get_auth_module(split[0]->buf); + if (mod) { + result = AUTH_DENY; + + for_each_string_list_item(token, mod->tokens) { + if (!strcmp(split[1]->buf, token->string)) { + result = AUTH_ALLOW; + break; + } + } + + goto done; + } + } + } + +done: + switch (result) { + case AUTH_ALLOW: + trace2_printf("%s: auth '%s' ALLOW", TR2_CAT, mod->scheme); + *user = "VALID_TEST_USER"; + *wr = WR_OK; + break; + + case AUTH_DENY: + trace2_printf("%s: auth '%s' DENY", TR2_CAT, mod->scheme); + /* fall-through */ + + case AUTH_UNKNOWN: + if (result != AUTH_DENY && allow_anonymous) + break; + + for (i = 0; i < auth_modules_nr; i++) { + mod = auth_modules[i]; + if (mod->challenge_params) + challenge = xstrfmt("WWW-Authenticate: %s %s", + mod->scheme, + mod->challenge_params); + else + challenge = xstrfmt("WWW-Authenticate: %s", + mod->scheme); + string_list_append(&hdrs, challenge); + } + + for (i = 0; i < extra_headers.nr; i++) + string_list_append(&hdrs, extra_headers.v[i]); + + *wr = send_http_error(STDOUT_FILENO, 401, "Unauthorized", -1, + &hdrs, *wr); + } + + strbuf_list_free(split); + string_list_clear(&hdrs, 0); + + return result == AUTH_ALLOW || + (result == AUTH_UNKNOWN && allow_anonymous); +} + +static int split_auth_param(const char *str, char **scheme, char **val, int required_val) +{ + struct strbuf **p = strbuf_split_str(str, ':', 2); + + if (!p[0]) + return -1; + + /* trim trailing ':' */ + if (p[1]) + strbuf_setlen(p[0], p[0]->len - 1); + + if (required_val && !p[1]) + return -1; + + *scheme = strbuf_detach(p[0], NULL); + + if (p[1]) + *val = strbuf_detach(p[1], NULL); + + strbuf_list_free(p); + return 0; +} + +static int read_auth_config(const char *name, const char *val, void *data) +{ + int ret = 0; + char *scheme = NULL; + char *token = NULL; + char *challenge = NULL; + struct auth_module *mod = NULL; + + if (!strcmp(name, "auth.challenge")) { + if (split_auth_param(val, &scheme, &challenge, 0)) { + ret = error("invalid auth challenge '%s'", val); + goto cleanup; + } + + mod = create_auth_module(scheme, challenge); + if (add_auth_module(mod)) { + ret = error("failed to add auth module '%s'", val); + goto cleanup; + } + } + if (!strcmp(name, "auth.token")) { + if (split_auth_param(val, &scheme, &token, 1)) { + ret = error("invalid auth token '%s'", val); + goto cleanup; + } + + mod = get_auth_module(scheme); + if (!mod) { + ret = error("auth scheme not defined '%s'\n", scheme); + goto cleanup; + } + + string_list_append(mod->tokens, token); + } + if (!strcmp(name, "auth.allowanonymous")) { + allow_anonymous = git_config_bool(name, val); + } + if (!strcmp(name, "auth.extraheader")) { + strvec_push(&extra_headers, val); + } + +cleanup: + free(scheme); + free(token); + free(challenge); + + return ret; +} + static enum worker_result dispatch(struct req *req) { + enum worker_result wr = WR_OK; + const char *user = NULL; + + if (!is_authed(req, &user, &wr)) + return wr; + if (is_git_request(req)) - return do__git(req); + return do__git(req, user); return send_http_error(STDOUT_FILENO, 501, "Not Implemented", -1, NULL, WR_OK | WR_HANGUP); @@ -624,6 +853,19 @@ int cmd_main(int argc, const char **argv) pid_file = v; continue; } + if (skip_prefix(arg, "--auth-config=", &v)) { + if (!strlen(v)) { + error("invalid argument - missing file path"); + usage(test_http_auth_usage); + } + + if (git_config_from_file(read_auth_config, v, NULL)) { + error("failed to read auth config file '%s'", v); + usage(test_http_auth_usage); + } + + continue; + } fprintf(stderr, "error: unknown argument '%s'\n", arg); usage(test_http_auth_usage); From patchwork Wed Jan 11 22:13:07 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Matthew John Cheetham X-Patchwork-Id: 13097286 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 67168C5479D for ; Wed, 11 Jan 2023 22:14:27 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S238467AbjAKWON (ORCPT ); Wed, 11 Jan 2023 17:14:13 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:33458 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S236070AbjAKWNV (ORCPT ); Wed, 11 Jan 2023 17:13:21 -0500 Received: from mail-wm1-x333.google.com (mail-wm1-x333.google.com [IPv6:2a00:1450:4864:20::333]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 03BDD3FA29 for ; Wed, 11 Jan 2023 14:13:21 -0800 (PST) Received: by mail-wm1-x333.google.com with SMTP id bg13-20020a05600c3c8d00b003d9712b29d2so15370837wmb.2 for ; Wed, 11 Jan 2023 14:13:20 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=cc:to:mime-version:content-transfer-encoding:fcc:subject:date:from :references:in-reply-to:message-id:from:to:cc:subject:date :message-id:reply-to; bh=qCY5CaeVsGFH4YCKVAhrjSkUdtoSk2eEGMnVuvUbaAA=; b=R5y09UCJHcjk6pz2cshPi/ed6fzcg9J3Rurdxbmg0vQOa48zIWkPPgf0KeSSFLXLdW atoNnHLs4t5RDw0KDyhOb2ycYh0JqIxC0DdFkK4/tqfzmCMI8pAvVLbQXIx6t3Az84jU mkEqkfaT2LeTyg9wwZGDfTDQ69tVMoFMJHVQuoGWtUMfDv5yQaNkaiUCCrCDVoVOZsLw 9vOZ1/FDmfHIurllIbLuOiM+oFwXFO5XcfMdYXmM/0P7TH4zQeJqrfJLH8vwLou49B/x 2RI7ODz14WX2gUhP4QvjwXHDD6KQFLXIeHKtW0ZLUjAY9JSC3A2s5SXFSrLIIQogjBxd 4WOQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=cc:to:mime-version:content-transfer-encoding:fcc:subject:date:from :references:in-reply-to:message-id:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=qCY5CaeVsGFH4YCKVAhrjSkUdtoSk2eEGMnVuvUbaAA=; b=5TEaUm2iHCfJNNPiQWOpwUX74Jm0PtUUbpTzXhIOl90P+HH80eWVUgxaTquOjz0ad+ KC24yceSJ2KSQI2tU/9dwpYlxzjtnWmC60JI1HVYC7avkwWVy4zQ8XtFYV3HqCAimZYO iGowVByHkALYFSSId/BZ4tg+vgY9zrOnv4c0+7FZ/fhTYwXnc4UZkVG9t0YmVW8Z5+dB IEdIzCxvUgRP7zRd4/QqvjL7WO5GfsT8HANwrWeeNAo35Wm/CLMh7uGzPLqY1TfopC6m CdY8Ops6JcVi5/B1G0wmJN7UDvs9GWJZe+6f1tZN0VEd6JWO3l3zLJjwmEYW3HVHkfU5 wIHA== X-Gm-Message-State: AFqh2kqGtI1CYPX1DqBb2PNduvVOCCNlJYehawfsHndTiiyTm2LUIphi +fguatsSfc4/kVyPLcbacM24jqvvYMw= X-Google-Smtp-Source: AMrXdXsZJPsXiD58pMxlIZnUtBY2RP9i2QR793FIYDAtiLZN/Wk4RCR/ciTlNyVowV6vHPKQDnuNqw== X-Received: by 2002:a05:600c:5008:b0:3cf:6f4d:c25d with SMTP id n8-20020a05600c500800b003cf6f4dc25dmr53353727wmr.21.1673475199362; Wed, 11 Jan 2023 14:13:19 -0800 (PST) Received: from [127.0.0.1] ([13.74.141.28]) by smtp.gmail.com with ESMTPSA id m17-20020a05600c3b1100b003cfbbd54178sm7469043wms.2.2023.01.11.14.13.18 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 11 Jan 2023 14:13:18 -0800 (PST) Message-Id: <9c4d25945dda0c5b5a16f0007dfb0575c59facf7.1673475190.git.gitgitgadget@gmail.com> In-Reply-To: References: Date: Wed, 11 Jan 2023 22:13:07 +0000 Subject: [PATCH v5 07/10] http: replace unsafe size_t multiplication with st_mult Fcc: Sent MIME-Version: 1.0 To: git@vger.kernel.org Cc: Derrick Stolee , Lessley Dennington , Matthew John Cheetham , M Hickford , Jeff Hostetler , Glen Choo , Victoria Dye , Matthew John Cheetham , Matthew John Cheetham Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org From: Matthew John Cheetham From: Matthew John Cheetham Replace direct multiplication of two size_t parameters in curl response stream handling callback functions with `st_mult` to guard against overflows. Signed-off-by: Matthew John Cheetham --- http.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/http.c b/http.c index 8a5ba3f4776..a2a80318bb2 100644 --- a/http.c +++ b/http.c @@ -146,7 +146,7 @@ static int http_schannel_use_ssl_cainfo; size_t fread_buffer(char *ptr, size_t eltsize, size_t nmemb, void *buffer_) { - size_t size = eltsize * nmemb; + size_t size = st_mult(eltsize, nmemb); struct buffer *buffer = buffer_; if (size > buffer->buf.len - buffer->posn) @@ -176,7 +176,7 @@ curlioerr ioctl_buffer(CURL *handle, int cmd, void *clientp) size_t fwrite_buffer(char *ptr, size_t eltsize, size_t nmemb, void *buffer_) { - size_t size = eltsize * nmemb; + size_t size = st_mult(eltsize, nmemb); struct strbuf *buffer = buffer_; strbuf_add(buffer, ptr, size); From patchwork Wed Jan 11 22:13:08 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Matthew John Cheetham X-Patchwork-Id: 13097289 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id A179BC5479D for ; Wed, 11 Jan 2023 22:14:38 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S238814AbjAKWOR (ORCPT ); Wed, 11 Jan 2023 17:14:17 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:33466 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S236071AbjAKWNW (ORCPT ); Wed, 11 Jan 2023 17:13:22 -0500 Received: from mail-wr1-x42f.google.com (mail-wr1-x42f.google.com [IPv6:2a00:1450:4864:20::42f]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id D768A3E0E1 for ; Wed, 11 Jan 2023 14:13:21 -0800 (PST) Received: by mail-wr1-x42f.google.com with SMTP id w1so16417895wrt.8 for ; Wed, 11 Jan 2023 14:13:21 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=cc:to:mime-version:content-transfer-encoding:fcc:subject:date:from :references:in-reply-to:message-id:from:to:cc:subject:date :message-id:reply-to; bh=czUORvxUFPS2+/W3hB2lr7qcIFurTGIuT+ES164BjhI=; b=CMPPLJPTmulIcXy8rsmod2NA9oUsnWH1uTigQJqV64TbA8NkiKfDxtJ+3T+UVAOWDf 10pNCy8zWmQui+AhvQk7ofnc9zxMB1O3UnnqjRtmRhWizfIfHFZrrIA5wBfgNYIwnCYA ZyvuwjqiVpv1zU8JbBXXwjxNr0Agc085HZPX2AV3a6x7u+dYqmGeI5FlkkNneCR2/7AG V3bQhWSrOhJSWPBU4/IwKi0g3HPCbGjbjRE6A+TNJnAAtv1lOzqggMr+8+nRSczaoeyv 3LDLmlCS0KP6L9OsCLPyenSD8LmbqY0DFTztV5YZZl69NUl2EQt/2/bcndxoYd/puPLA ra5Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=cc:to:mime-version:content-transfer-encoding:fcc:subject:date:from :references:in-reply-to:message-id:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=czUORvxUFPS2+/W3hB2lr7qcIFurTGIuT+ES164BjhI=; b=35SqvUN79WaWr7npyIEa4s5+g3Em4cGKBbhuDBcHRD40HAyd4v3uNn1lM9oddDrw6k u5oxMSCrQZB7kw19mzcEYPT1mA1vQxORFixE2WRoesoIXH3sKE7nlzy7+vNiEsLgCObd DHzmBMl0AZGR0td1NU6p8p7V/fzrbxuucCPciqWBzHJ1EcnB2i/Xh7d06gFKxXzX1tYl 8qpVR0bqvFYQJSNKa9goa1Js9EciWnrlAEK0Y0e1UfPssVgYOu59ahY9hKNGbW0jCEfm +QiEbwENHDSttRyWQDr02UgN3ZH1qgf6wo0g075PE9hkhqOF9F2FP+OEp2A7OGLKDbHG +SpQ== X-Gm-Message-State: AFqh2kpQaoIVhHddAq2FLqB5GsREejIzbLk2BmLGxX0noaHWA8urwWiI 0TkYIRLTd6e8BFa+e3mh27dJaj9M57Y= X-Google-Smtp-Source: AMrXdXvsammvzVNmae9HMd1PJfoWNBE6Eq83n0hYX6ga586uP90UHBEEejOzZTlW5F0u2KG3FgXK+g== X-Received: by 2002:adf:ec4f:0:b0:27b:a73e:33ae with SMTP id w15-20020adfec4f000000b0027ba73e33aemr38816611wrn.8.1673475200125; Wed, 11 Jan 2023 14:13:20 -0800 (PST) Received: from [127.0.0.1] ([13.74.141.28]) by smtp.gmail.com with ESMTPSA id y15-20020adffa4f000000b002bbec19c8acsm9456893wrr.64.2023.01.11.14.13.19 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 11 Jan 2023 14:13:19 -0800 (PST) Message-Id: <65a620b08ef359e29d678497f1b529e3ce6477b1.1673475190.git.gitgitgadget@gmail.com> In-Reply-To: References: Date: Wed, 11 Jan 2023 22:13:08 +0000 Subject: [PATCH v5 08/10] strvec: expose strvec_push_nodup for external use Fcc: Sent MIME-Version: 1.0 To: git@vger.kernel.org Cc: Derrick Stolee , Lessley Dennington , Matthew John Cheetham , M Hickford , Jeff Hostetler , Glen Choo , Victoria Dye , Matthew John Cheetham , Matthew John Cheetham Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org From: Matthew John Cheetham From: Matthew John Cheetham Remove the static modifier from the existing `strvec_push_nodup` function and define the function is `strvec.h` to make it available for other callers, making it now possible to append to a `struct strvec` array without duplication. Signed-off-by: Matthew John Cheetham --- strvec.c | 2 +- strvec.h | 3 +++ 2 files changed, 4 insertions(+), 1 deletion(-) diff --git a/strvec.c b/strvec.c index 61a76ce6cb9..26e8751cae0 100644 --- a/strvec.c +++ b/strvec.c @@ -10,7 +10,7 @@ void strvec_init(struct strvec *array) memcpy(array, &blank, sizeof(*array)); } -static void strvec_push_nodup(struct strvec *array, const char *value) +void strvec_push_nodup(struct strvec *array, const char *value) { if (array->v == empty_strvec) array->v = NULL; diff --git a/strvec.h b/strvec.h index 9f55c8766ba..5d61dd73680 100644 --- a/strvec.h +++ b/strvec.h @@ -43,6 +43,9 @@ struct strvec { */ void strvec_init(struct strvec *); +/* Push a string onto the end of the array without copying. */ +void strvec_push_nodup(struct strvec *array, const char *value); + /* Push a copy of a string onto the end of the array. */ const char *strvec_push(struct strvec *, const char *); From patchwork Wed Jan 11 22:13:09 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Matthew John Cheetham X-Patchwork-Id: 13097288 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id B1308C54EBC for ; Wed, 11 Jan 2023 22:14:38 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S238942AbjAKWOV (ORCPT ); Wed, 11 Jan 2023 17:14:21 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:33492 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S236082AbjAKWNY (ORCPT ); Wed, 11 Jan 2023 17:13:24 -0500 Received: from mail-wr1-x434.google.com (mail-wr1-x434.google.com [IPv6:2a00:1450:4864:20::434]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id C8DEB431A5 for ; Wed, 11 Jan 2023 14:13:22 -0800 (PST) Received: by mail-wr1-x434.google.com with SMTP id bn26so16444948wrb.0 for ; Wed, 11 Jan 2023 14:13:22 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=cc:to:mime-version:content-transfer-encoding:fcc:subject:date:from :references:in-reply-to:message-id:from:to:cc:subject:date :message-id:reply-to; bh=sti4/YXuRxJdTVRBK4qF5FuZujpVj+P/dob0ruiB2Vw=; b=H4Ggie1RTCH65FW0JOkfA/KwIeZuX9tKMzOZmgC5F4qVQxZm8E8bfpD9/5uOEUhll3 DJ1yl2oulHm0v/qWMNMSYW/hQnjLPD6GaFsuvibCfjOeymyYZUcdmfQiDjlNOXXrDiRn 28LGsTVewVd72ooht9wPBRY41qAOsyJbWeaY9+vOsD316uYY2iH2gBKFb6TccXIJ54oQ sbTYTuLyhL7vd206LNMd+JuginXzN7mGyyCPGOb8TUG3eVVEQdtsmg7iLQKw1H7oTG7S Fy2bZ3AxdY/OQpVr/5yJf2Lz9YeCKl3eL2F5mPP2ht9MqY05iWcqXjsh/zzCH8p238Jh aXfw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=cc:to:mime-version:content-transfer-encoding:fcc:subject:date:from :references:in-reply-to:message-id:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=sti4/YXuRxJdTVRBK4qF5FuZujpVj+P/dob0ruiB2Vw=; b=OROJf+fGZKw8X9+uv3q9JQFfKOv4wfG9XsjZ7TSnGJS3lVlgMIPYntLLPJn8WUP0c1 JN6zB2FjTxPsXNrcRMTPuV8yySH1YVHvo+vCBcON0DjxFq//wmdA40XR5cz5DStIHBCZ s5vFdmP/9dNOEPD1Vy4yabGge7hX2gLHUB77fv5cPNCWeE6UKR3q2zmfT5XkOlKfTofx xbqJeoOsl6+KXXcnrBT2ipCv/ht5TUObNNAcjfukeNJSeCAGxCN31hYOCjsxJ2xthuu+ rUGHg9JUs751wVrooxteNRLN/SlwJ9OGqslJZXB0NqidwoJgRXqxeAmfs6mVHxgyC6Ix g9ng== X-Gm-Message-State: AFqh2kqSj1LwSkaQe5WvDiRE/fb9EzLdNZxVLV1uyv1l2bMbrbMfS6IN EFOcREoq1ES6cTEI8+eZh/Awu0X1KzI= X-Google-Smtp-Source: AMrXdXutMIaLnk9UEvzeLfY1PBtTvwzCWUhhY3HppQi7UdX/YFbz+/1iB/DUs6engF9lmXKFcWhy9A== X-Received: by 2002:adf:e449:0:b0:27c:fdb:cfef with SMTP id t9-20020adfe449000000b0027c0fdbcfefmr36254934wrm.49.1673475201147; Wed, 11 Jan 2023 14:13:21 -0800 (PST) Received: from [127.0.0.1] ([13.74.141.28]) by smtp.gmail.com with ESMTPSA id l24-20020a1ced18000000b003d99da8d30asm24012663wmh.46.2023.01.11.14.13.20 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 11 Jan 2023 14:13:20 -0800 (PST) Message-Id: In-Reply-To: References: Date: Wed, 11 Jan 2023 22:13:09 +0000 Subject: [PATCH v5 09/10] http: read HTTP WWW-Authenticate response headers Fcc: Sent MIME-Version: 1.0 To: git@vger.kernel.org Cc: Derrick Stolee , Lessley Dennington , Matthew John Cheetham , M Hickford , Jeff Hostetler , Glen Choo , Victoria Dye , Matthew John Cheetham , Matthew John Cheetham Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org From: Matthew John Cheetham From: Matthew John Cheetham Read and store the HTTP WWW-Authenticate response headers made for a particular request. This will allow us to pass important authentication challenge information to credential helpers or others that would otherwise have been lost. According to RFC2616 Section 4.2 [1], header field names are not case-sensitive meaning when collecting multiple values for the same field name, we can just use the case of the first observed instance of each field name and no normalisation is required. libcurl only provides us with the ability to read all headers recieved for a particular request, including any intermediate redirect requests or proxies. The lines returned by libcurl include HTTP status lines delinating any intermediate requests such as "HTTP/1.1 200". We use these lines to reset the strvec of WWW-Authenticate header values as we encounter them in order to only capture the final response headers. The collection of all header values matching the WWW-Authenticate header is complicated by the fact that it is legal for header fields to be continued over multiple lines, but libcurl only gives us one line at a time. In the future [2] we may be able to leverage functions to read headers from libcurl itself, but as of today we must do this ourselves. [1] https://datatracker.ietf.org/doc/html/rfc2616#section-4.2 [2] https://daniel.haxx.se/blog/2022/03/22/a-headers-api-for-libcurl/ Signed-off-by: Matthew John Cheetham --- credential.c | 1 + credential.h | 15 ++++++++ http.c | 98 ++++++++++++++++++++++++++++++++++++++++++++++++++++ 3 files changed, 114 insertions(+) diff --git a/credential.c b/credential.c index f6389a50684..897b4679333 100644 --- a/credential.c +++ b/credential.c @@ -22,6 +22,7 @@ void credential_clear(struct credential *c) free(c->username); free(c->password); string_list_clear(&c->helpers, 0); + strvec_clear(&c->wwwauth_headers); credential_init(c); } diff --git a/credential.h b/credential.h index f430e77fea4..6f2e5bc610b 100644 --- a/credential.h +++ b/credential.h @@ -2,6 +2,7 @@ #define CREDENTIAL_H #include "string-list.h" +#include "strvec.h" /** * The credentials API provides an abstracted way of gathering username and @@ -115,6 +116,19 @@ struct credential { */ struct string_list helpers; + /** + * A `strvec` of WWW-Authenticate header values. Each string + * is the value of a WWW-Authenticate header in an HTTP response, + * in the order they were received in the response. + */ + struct strvec wwwauth_headers; + + /** + * Internal use only. Used to keep track of split header fields + * in order to fold multiple lines into one value. + */ + unsigned header_is_last_match:1; + unsigned approved:1, configured:1, quit:1, @@ -130,6 +144,7 @@ struct credential { #define CREDENTIAL_INIT { \ .helpers = STRING_LIST_INIT_DUP, \ + .wwwauth_headers = STRVEC_INIT, \ } /* Initialize a credential structure, setting all fields to empty. */ diff --git a/http.c b/http.c index a2a80318bb2..10882034145 100644 --- a/http.c +++ b/http.c @@ -183,6 +183,102 @@ size_t fwrite_buffer(char *ptr, size_t eltsize, size_t nmemb, void *buffer_) return nmemb; } +static size_t fwrite_wwwauth(char *ptr, size_t eltsize, size_t nmemb, void *p) +{ + size_t size = st_mult(eltsize, nmemb); + struct strvec *values = &http_auth.wwwauth_headers; + struct strbuf buf = STRBUF_INIT; + const char *val; + + /* + * Header lines may not come NULL-terminated from libcurl so we must + * limit all scans to the maximum length of the header line, or leverage + * strbufs for all operations. + * + * In addition, it is possible that header values can be split over + * multiple lines as per RFC 2616 (even though this has since been + * deprecated in RFC 7230). A continuation header field value is + * identified as starting with a space or horizontal tab. + * + * The formal definition of a header field as given in RFC 2616 is: + * + * message-header = field-name ":" [ field-value ] + * field-name = token + * field-value = *( field-content | LWS ) + * field-content = + */ + + strbuf_add(&buf, ptr, size); + + /* Strip the CRLF that should be present at the end of each field */ + strbuf_trim_trailing_newline(&buf); + + /* Start of a new WWW-Authenticate header */ + if (skip_iprefix(buf.buf, "www-authenticate:", &val)) { + while (isspace(*val)) + val++; + + strvec_push(values, val); + http_auth.header_is_last_match = 1; + goto exit; + } + + /* + * This line could be a continuation of the previously matched header + * field. If this is the case then we should append this value to the + * end of the previously consumed value. + * Continuation lines start with at least one whitespace, maybe more, + * so we should collapse these down to a single SP (valid per the spec). + */ + if (http_auth.header_is_last_match && isspace(*buf.buf)) { + /* Trim leading whitespace from this continuation hdr line. */ + strbuf_ltrim(&buf); + + /* + * At this point we should always have at least one existing + * value, even if it is empty. Do not bother appending the new + * value if this continuation header is itself empty. + */ + if (!values->nr) { + BUG("should have at least one existing header value"); + } else if (buf.len) { + const char *prev = values->v[values->nr - 1]; + struct strbuf append = STRBUF_INIT; + strbuf_addstr(&append, prev); + + /* Join two non-empty values with a single space. */ + if (append.len) + strbuf_addch(&append, ' '); + + strbuf_addbuf(&append, &buf); + + strvec_pop(values); + strvec_push_nodup(values, strbuf_detach(&append, NULL)); + } + + goto exit; + } + + /* This is the start of a new header we don't care about */ + http_auth.header_is_last_match = 0; + + /* + * If this is a HTTP status line and not a header field, this signals + * a different HTTP response. libcurl writes all the output of all + * response headers of all responses, including redirects. + * We only care about the last HTTP request response's headers so clear + * the existing array. + */ + if (istarts_with(buf.buf, "http/")) + strvec_clear(values); + +exit: + strbuf_release(&buf); + return size; +} + size_t fwrite_null(char *ptr, size_t eltsize, size_t nmemb, void *strbuf) { return nmemb; @@ -1864,6 +1960,8 @@ static int http_request(const char *url, fwrite_buffer); } + curl_easy_setopt(slot->curl, CURLOPT_HEADERFUNCTION, fwrite_wwwauth); + accept_language = http_get_accept_language_header(); if (accept_language) From patchwork Wed Jan 11 22:13:10 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Matthew John Cheetham X-Patchwork-Id: 13097290 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id C0B50C46467 for ; Wed, 11 Jan 2023 22:14:38 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S238996AbjAKWOX (ORCPT ); Wed, 11 Jan 2023 17:14:23 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:33500 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S235873AbjAKWNZ (ORCPT ); Wed, 11 Jan 2023 17:13:25 -0500 Received: from mail-wr1-x435.google.com (mail-wr1-x435.google.com [IPv6:2a00:1450:4864:20::435]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id B4B6D431A6 for ; Wed, 11 Jan 2023 14:13:23 -0800 (PST) Received: by mail-wr1-x435.google.com with SMTP id bn26so16444972wrb.0 for ; Wed, 11 Jan 2023 14:13:23 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=cc:to:mime-version:content-transfer-encoding:fcc:subject:date:from :references:in-reply-to:message-id:from:to:cc:subject:date :message-id:reply-to; bh=7SXgnA4M5wKdobtw7SHt2vwTto2ayZwPSP6ipHKss68=; b=QvXlavGy82zxzsK4hbA8nNEhz/N8eHA/5N6rBiwwp33pOzyAOhJhhYbulW2Y62F5kj +f5EXOqOV6xCOmuE62AZ+ZwZFPMFp4wb0wPgJkxxrdKtVU+XoIFO3HHoAoMjt6b/DnDA Jd5gWSdHYSyyq9HKZNN3JxukJITvZtNx2xIW427qPIIbeoWp/zsc+8MveAWLIdvZWu8z oaavbKm0iuDoahc7tMI6VBD2+AaDCcby3E5EY6o8WSyAJgPE7y0vH47Knd4ZN8Jzfde0 PfOpB5X5CQqxc74rz6zh1vwulcjtR2GXnoCuMLPtQO+hvzB910F0Io/8n4X34EA3WmyP b3Sw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=cc:to:mime-version:content-transfer-encoding:fcc:subject:date:from :references:in-reply-to:message-id:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=7SXgnA4M5wKdobtw7SHt2vwTto2ayZwPSP6ipHKss68=; b=5NESSK6ONGTP8+v1reqaubOoRvfDxzPpQK8XzNAOC4vrBRS7oy3sb6SnTjE1kKuAty 9N3mBZj45RGCBO649IlfiNxumVhEtoph+1a5ytALQUyiIgoEbv99ZsQn6W3p19YcKvKa B/y4RwZzauPlHFO71GEAcimlCcFtZ1vH+MZTmisaAmfqAQ0Mv2BiAbRUIDHF5DoOLBvI jTMzbeD07mNeqcl3IVL4w2islDjiBNCO2V5RigPSqgYNelLwLpOT8GScvrSLhHBZnu+Y ZdoFNQLGPHtHQJ89i4uJVZEp6UYjQhM1L9T8Utlt2sJAj7ynqClsc1/zAyyn2myxr+o+ aDnQ== X-Gm-Message-State: AFqh2koZYJT6hJULd1aGxQRRY6afa1FV4L6AGIBnNMkf5zjuPmPBMxKw A6qiavzU0aei/8x2z1bcR2yFhLmAttg= X-Google-Smtp-Source: AMrXdXu9kkk/s7oCvN6HIOtVEJfLEKA6kNykrhsQ6zy9/gB4L9Dj8ThtqyL1TWlGxAp7/cFyLZC8cw== X-Received: by 2002:a05:6000:54f:b0:2bc:aa67:28fb with SMTP id b15-20020a056000054f00b002bcaa6728fbmr4998743wrf.49.1673475201975; Wed, 11 Jan 2023 14:13:21 -0800 (PST) Received: from [127.0.0.1] ([13.74.141.28]) by smtp.gmail.com with ESMTPSA id g1-20020a056000118100b002755e301eeasm14462334wrx.100.2023.01.11.14.13.21 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 11 Jan 2023 14:13:21 -0800 (PST) Message-Id: In-Reply-To: References: Date: Wed, 11 Jan 2023 22:13:10 +0000 Subject: [PATCH v5 10/10] credential: add WWW-Authenticate header to cred requests Fcc: Sent MIME-Version: 1.0 To: git@vger.kernel.org Cc: Derrick Stolee , Lessley Dennington , Matthew John Cheetham , M Hickford , Jeff Hostetler , Glen Choo , Victoria Dye , Matthew John Cheetham , Matthew John Cheetham Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org From: Matthew John Cheetham From: Matthew John Cheetham Add the value of the WWW-Authenticate response header to credential requests. Credential helpers that understand and support HTTP authentication and authorization can use this standard header (RFC 2616 Section 14.47 [1]) to generate valid credentials. WWW-Authenticate headers can contain information pertaining to the authority, authentication mechanism, or extra parameters/scopes that are required. The current I/O format for credential helpers only allows for unique names for properties/attributes, so in order to transmit multiple header values (with a specific order) we introduce a new convention whereby a C-style array syntax is used in the property name to denote multiple ordered values for the same property. In this case we send multiple `wwwauth[]` properties where the order that the repeated attributes appear in the conversation reflects the order that the WWW-Authenticate headers appeared in the HTTP response. Add a set of tests to exercise the HTTP authentication header parsing and the interop with credential helpers. Credential helpers will receive WWW-Authenticate information in credential requests. [1] https://datatracker.ietf.org/doc/html/rfc2616#section-14.47 Signed-off-by: Matthew John Cheetham --- Documentation/git-credential.txt | 19 +- credential.c | 12 + t/helper/test-credential-helper-replay.sh | 14 ++ t/t5556-http-auth.sh | 270 +++++++++++++++++++++- 4 files changed, 312 insertions(+), 3 deletions(-) create mode 100755 t/helper/test-credential-helper-replay.sh diff --git a/Documentation/git-credential.txt b/Documentation/git-credential.txt index ac2818b9f66..50759153ef1 100644 --- a/Documentation/git-credential.txt +++ b/Documentation/git-credential.txt @@ -113,7 +113,13 @@ separated by an `=` (equals) sign, followed by a newline. The key may contain any bytes except `=`, newline, or NUL. The value may contain any bytes except newline or NUL. -In both cases, all bytes are treated as-is (i.e., there is no quoting, +Attributes with keys that end with C-style array brackets `[]` can have +multiple values. Each instance of a multi-valued attribute forms an +ordered list of values - the order of the repeated attributes defines +the order of the values. An empty multi-valued attribute (`key[]=\n`) +acts to clear any previous entries and reset the list. + +In all cases, all bytes are treated as-is (i.e., there is no quoting, and one cannot transmit a value with newline or NUL in it). The list of attributes is terminated by a blank line or end-of-file. @@ -160,6 +166,17 @@ empty string. Components which are missing from the URL (e.g., there is no username in the example above) will be left unset. +`wwwauth[]`:: + + When an HTTP response is received by Git that includes one or more + 'WWW-Authenticate' authentication headers, these will be passed by Git + to credential helpers. ++ +Each 'WWW-Authenticate' header value is passed as a multi-valued +attribute 'wwwauth[]', where the order of the attributes is the same as +they appear in the HTTP response. This attribute is 'one-way' from Git +to pass additional information to credential helpers. + Unrecognised attributes are silently discarded. GIT diff --git a/credential.c b/credential.c index 897b4679333..8a3ad6c0ae2 100644 --- a/credential.c +++ b/credential.c @@ -263,6 +263,17 @@ static void credential_write_item(FILE *fp, const char *key, const char *value, fprintf(fp, "%s=%s\n", key, value); } +static void credential_write_strvec(FILE *fp, const char *key, + const struct strvec *vec) +{ + int i = 0; + const char *full_key = xstrfmt("%s[]", key); + for (; i < vec->nr; i++) { + credential_write_item(fp, full_key, vec->v[i], 0); + } + free((void*)full_key); +} + void credential_write(const struct credential *c, FILE *fp) { credential_write_item(fp, "protocol", c->protocol, 1); @@ -270,6 +281,7 @@ void credential_write(const struct credential *c, FILE *fp) credential_write_item(fp, "path", c->path, 0); credential_write_item(fp, "username", c->username, 0); credential_write_item(fp, "password", c->password, 0); + credential_write_strvec(fp, "wwwauth", &c->wwwauth_headers); } static int run_credential_helper(struct credential *c, diff --git a/t/helper/test-credential-helper-replay.sh b/t/helper/test-credential-helper-replay.sh new file mode 100755 index 00000000000..03e5e63dad6 --- /dev/null +++ b/t/helper/test-credential-helper-replay.sh @@ -0,0 +1,14 @@ +cmd=$1 +teefile=$cmd-actual.cred +catfile=$cmd-response.cred +rm -f $teefile +while read line; +do + if test -z "$line"; then + break; + fi + echo "$line" >> $teefile +done +if test "$cmd" = "get"; then + cat $catfile +fi diff --git a/t/t5556-http-auth.sh b/t/t5556-http-auth.sh index 65105a5a6a9..a8dbee6ca40 100755 --- a/t/t5556-http-auth.sh +++ b/t/t5556-http-auth.sh @@ -27,6 +27,8 @@ PID_FILE="$TRASH_DIRECTORY"/pid-file.pid SERVER_LOG="$TRASH_DIRECTORY"/OUT.server.log PATH="$GIT_BUILD_DIR/t/helper/:$PATH" && export PATH +CREDENTIAL_HELPER="$GIT_BUILD_DIR/t/helper/test-credential-helper-replay.sh" \ + && export CREDENTIAL_HELPER test_expect_success 'setup repos' ' test_create_repo "$REPO_DIR" && @@ -92,15 +94,279 @@ start_http_server () { per_test_cleanup () { stop_http_server && - rm -f OUT.* + rm -f OUT.* && + rm -f *.cred && + rm -f auth.config } test_expect_success 'http auth anonymous no challenge' ' test_when_finished "per_test_cleanup" && - start_http_server && + + cat >auth.config <<-EOF && + [auth] + allowAnonymous = true + EOF + + start_http_server --auth-config="$TRASH_DIRECTORY/auth.config" && # Attempt to read from a protected repository git ls-remote $ORIGIN_URL ' +test_expect_success 'http auth www-auth headers to credential helper basic valid' ' + test_when_finished "per_test_cleanup" && + # base64("alice:secret-passwd") + USERPASS64=YWxpY2U6c2VjcmV0LXBhc3N3ZA== && + export USERPASS64 && + + cat >auth.config <<-EOF && + [auth] + challenge = basic:realm=\"example.com\" + token = basic:$USERPASS64 + EOF + + start_http_server --auth-config="$TRASH_DIRECTORY/auth.config" && + + cat >get-expected.cred <<-EOF && + protocol=http + host=$HOST_PORT + wwwauth[]=basic realm="example.com" + EOF + + cat >store-expected.cred <<-EOF && + protocol=http + host=$HOST_PORT + username=alice + password=secret-passwd + EOF + + cat >get-response.cred <<-EOF && + protocol=http + host=$HOST_PORT + username=alice + password=secret-passwd + EOF + + git -c credential.helper="$CREDENTIAL_HELPER" ls-remote $ORIGIN_URL && + + test_cmp get-expected.cred get-actual.cred && + test_cmp store-expected.cred store-actual.cred +' + +test_expect_success 'http auth www-auth headers to credential helper ignore case valid' ' + test_when_finished "per_test_cleanup" && + # base64("alice:secret-passwd") + USERPASS64=YWxpY2U6c2VjcmV0LXBhc3N3ZA== && + export USERPASS64 && + + cat >auth.config <<-EOF && + [auth] + challenge = basic:realm=\"example.com\" + token = basic:$USERPASS64 + extraHeader = wWw-aUtHeNtIcAtE: bEaRer auThoRiTy=\"id.example.com\" + EOF + + start_http_server --auth-config="$TRASH_DIRECTORY/auth.config" && + + cat >get-expected.cred <<-EOF && + protocol=http + host=$HOST_PORT + wwwauth[]=basic realm="example.com" + wwwauth[]=bEaRer auThoRiTy="id.example.com" + EOF + + cat >store-expected.cred <<-EOF && + protocol=http + host=$HOST_PORT + username=alice + password=secret-passwd + EOF + + cat >get-response.cred <<-EOF && + protocol=http + host=$HOST_PORT + username=alice + password=secret-passwd + EOF + + git -c credential.helper="$CREDENTIAL_HELPER" ls-remote $ORIGIN_URL && + + test_cmp get-expected.cred get-actual.cred && + test_cmp store-expected.cred store-actual.cred +' + +test_expect_success 'http auth www-auth headers to credential helper continuation hdr' ' + test_when_finished "per_test_cleanup" && + # base64("alice:secret-passwd") + USERPASS64=YWxpY2U6c2VjcmV0LXBhc3N3ZA== && + export USERPASS64 && + + cat >auth.config <<-EOF && + [auth] + challenge = "bearer:authority=\"id.example.com\"\\n q=1\\n \\t p=0" + challenge = basic:realm=\"example.com\" + token = basic:$USERPASS64 + EOF + + start_http_server --auth-config="$TRASH_DIRECTORY/auth.config" && + + cat >get-expected.cred <<-EOF && + protocol=http + host=$HOST_PORT + wwwauth[]=bearer authority="id.example.com" q=1 p=0 + wwwauth[]=basic realm="example.com" + EOF + + cat >store-expected.cred <<-EOF && + protocol=http + host=$HOST_PORT + username=alice + password=secret-passwd + EOF + + cat >get-response.cred <<-EOF && + protocol=http + host=$HOST_PORT + username=alice + password=secret-passwd + EOF + + git -c credential.helper="$CREDENTIAL_HELPER" ls-remote $ORIGIN_URL && + + test_cmp get-expected.cred get-actual.cred && + test_cmp store-expected.cred store-actual.cred +' + +test_expect_success 'http auth www-auth headers to credential helper empty continuation hdrs' ' + test_when_finished "per_test_cleanup" && + # base64("alice:secret-passwd") + USERPASS64=YWxpY2U6c2VjcmV0LXBhc3N3ZA== && + export USERPASS64 && + + cat >auth.config <<-EOF && + [auth] + challenge = basic:realm=\"example.com\" + token = basic:$USERPASS64 + extraheader = "WWW-Authenticate:" + extraheader = " " + extraheader = " bearer authority=\"id.example.com\"" + EOF + + start_http_server --auth-config="$TRASH_DIRECTORY/auth.config" && + + cat >get-expected.cred <<-EOF && + protocol=http + host=$HOST_PORT + wwwauth[]=basic realm="example.com" + wwwauth[]=bearer authority="id.example.com" + EOF + + cat >store-expected.cred <<-EOF && + protocol=http + host=$HOST_PORT + username=alice + password=secret-passwd + EOF + + cat >get-response.cred <<-EOF && + protocol=http + host=$HOST_PORT + username=alice + password=secret-passwd + EOF + + git -c credential.helper="$CREDENTIAL_HELPER" ls-remote $ORIGIN_URL && + + test_cmp get-expected.cred get-actual.cred && + test_cmp store-expected.cred store-actual.cred +' + +test_expect_success 'http auth www-auth headers to credential helper custom schemes' ' + test_when_finished "per_test_cleanup" && + # base64("alice:secret-passwd") + USERPASS64=YWxpY2U6c2VjcmV0LXBhc3N3ZA== && + export USERPASS64 && + + cat >auth.config <<-EOF && + [auth] + challenge = "foobar:alg=test widget=1" + challenge = "bearer:authority=\"id.example.com\" q=1 p=0" + challenge = basic:realm=\"example.com\" + token = basic:$USERPASS64 + EOF + + start_http_server --auth-config="$TRASH_DIRECTORY/auth.config" && + + cat >get-expected.cred <<-EOF && + protocol=http + host=$HOST_PORT + wwwauth[]=foobar alg=test widget=1 + wwwauth[]=bearer authority="id.example.com" q=1 p=0 + wwwauth[]=basic realm="example.com" + EOF + + cat >store-expected.cred <<-EOF && + protocol=http + host=$HOST_PORT + username=alice + password=secret-passwd + EOF + + cat >get-response.cred <<-EOF && + protocol=http + host=$HOST_PORT + username=alice + password=secret-passwd + EOF + + git -c credential.helper="$CREDENTIAL_HELPER" ls-remote $ORIGIN_URL && + + test_cmp get-expected.cred get-actual.cred && + test_cmp store-expected.cred store-actual.cred +' + +test_expect_success 'http auth www-auth headers to credential helper invalid' ' + test_when_finished "per_test_cleanup" && + # base64("alice:secret-passwd") + USERPASS64=YWxpY2U6c2VjcmV0LXBhc3N3ZA== && + export USERPASS64 && + + cat >auth.config <<-EOF && + [auth] + challenge = "bearer:authority=\"id.example.com\" q=1 p=0" + challenge = basic:realm=\"example.com\" + token = basic:$USERPASS64 + EOF + + start_http_server --auth-config="$TRASH_DIRECTORY/auth.config" && + + cat >get-expected.cred <<-EOF && + protocol=http + host=$HOST_PORT + wwwauth[]=bearer authority="id.example.com" q=1 p=0 + wwwauth[]=basic realm="example.com" + EOF + + cat >erase-expected.cred <<-EOF && + protocol=http + host=$HOST_PORT + username=alice + password=invalid-passwd + wwwauth[]=bearer authority="id.example.com" q=1 p=0 + wwwauth[]=basic realm="example.com" + EOF + + cat >get-response.cred <<-EOF && + protocol=http + host=$HOST_PORT + username=alice + password=invalid-passwd + EOF + + test_must_fail git -c credential.helper="$CREDENTIAL_HELPER" ls-remote $ORIGIN_URL && + + test_cmp get-expected.cred get-actual.cred && + test_cmp erase-expected.cred erase-actual.cred +' + test_done