From patchwork Mon Dec 12 21:36:16 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Matthew John Cheetham X-Patchwork-Id: 13071455 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 8BC90C4167B for ; Mon, 12 Dec 2022 21:36:36 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233257AbiLLVgf (ORCPT ); Mon, 12 Dec 2022 16:36:35 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:35334 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229497AbiLLVga (ORCPT ); Mon, 12 Dec 2022 16:36:30 -0500 Received: from mail-wr1-x432.google.com (mail-wr1-x432.google.com [IPv6:2a00:1450:4864:20::432]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id F03D525E9 for ; Mon, 12 Dec 2022 13:36:28 -0800 (PST) Received: by mail-wr1-x432.google.com with SMTP id w15so13622867wrl.9 for ; Mon, 12 Dec 2022 13:36:28 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=cc:to:mime-version:content-transfer-encoding:fcc:subject:date:from :references:in-reply-to:message-id:from:to:cc:subject:date :message-id:reply-to; bh=MUdy5+lwLcyYzmn69jjG1A8V0jJQApX5scNAQaOGLjY=; b=pa5i9iaG64OE3IG2iZ0abwlhMBWk2YVuc3Mbufr32/2s5lT1Px9S9FMl0cOnjHnlcs R3v355CnYpT9rPZ9jPyxuXojGgvvQdQFRjo8TodupT9q+NzZJFX6k/Myf636MSVglNYt SIzhdKr2G3tEKoCYPHPxhZqCD8ji//u9HkQbbHGRuDk9D/kq9RUaTZnNt8E+OnIHbjwm 2qBcr7ZCdaEzWb2TBvrRB1FjYFEgNgA6VC0zblu6e2wuYXkCeM5uFRTJFh3QVI8VY0vp d+0FEcndpN2RSImWg0M84ZQ02har7FYVvoTrJYxzeiftaHkjzKkjhb6JdmhB79RJM52n Nwkw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=cc:to:mime-version:content-transfer-encoding:fcc:subject:date:from :references:in-reply-to:message-id:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=MUdy5+lwLcyYzmn69jjG1A8V0jJQApX5scNAQaOGLjY=; b=C/e54plf2X4hY5x/FzdihF1Ucrowo5Bbd9pGcavySpJgmk9s1stsx2zBb8VkFT6193 bx/zhSNMq5Md2Nj4sw1m6sfQ2S8XjBVX1WYUORjDQfqOgYdRyPZ/49rfofYJ8lJSKgIc Ab6aXcSJqR+7RhyM8jHSiZ3wfnMz7Es9zhq/EPkF6GY/xEGjiYcWTK0ru7Hg/Xtm/aBU m6T1tJ5axgdzpOwYAqAE1gwU7yV2ek8TpM+M+LUcn//wIfIHpjx4NcramYLqe6N8ASGH wrhfG517LZ1Be9LrzYr+RSlWbX+trNLDS9iMQzXOTjExpcux76o/EAJ3CbTJknRjsEfe nTBw== X-Gm-Message-State: ANoB5pmfa1ygki8yknBvEAC3thsK0Fce/x/PlKApqeadCM+r4vx16/A7 BOhVC0/D2Qf4qeBzVH3haMN34HmynMQ= X-Google-Smtp-Source: AA0mqf7mffJxlKsPk+9K0cgkXhU3Ho46qojpJGpLQrzVnX+0xgpdtz8kbO5UyLof++s2lf5xsPqfoA== X-Received: by 2002:a5d:4892:0:b0:241:6654:62f1 with SMTP id g18-20020a5d4892000000b00241665462f1mr9846410wrq.49.1670880987186; Mon, 12 Dec 2022 13:36:27 -0800 (PST) Received: from [127.0.0.1] ([13.74.141.28]) by smtp.gmail.com with ESMTPSA id x1-20020a5d4441000000b0023677081f3asm9762912wrr.42.2022.12.12.13.36.26 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 12 Dec 2022 13:36:26 -0800 (PST) Message-Id: In-Reply-To: References: Date: Mon, 12 Dec 2022 21:36:16 +0000 Subject: [PATCH v4 1/8] http: read HTTP WWW-Authenticate response headers Fcc: Sent MIME-Version: 1.0 To: git@vger.kernel.org Cc: Derrick Stolee , Lessley Dennington , Matthew John Cheetham , M Hickford , Jeff Hostetler , Glen Choo , Matthew John Cheetham , Matthew John Cheetham Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org From: Matthew John Cheetham From: Matthew John Cheetham Read and store the HTTP WWW-Authenticate response headers made for a particular request. This will allow us to pass important authentication challenge information to credential helpers or others that would otherwise have been lost. According to RFC2616 Section 4.2 [1], header field names are not case-sensitive meaning when collecting multiple values for the same field name, we can just use the case of the first observed instance of each field name and no normalisation is required. libcurl only provides us with the ability to read all headers recieved for a particular request, including any intermediate redirect requests or proxies. The lines returned by libcurl include HTTP status lines delinating any intermediate requests such as "HTTP/1.1 200". We use these lines to reset the strvec of WWW-Authenticate header values as we encounter them in order to only capture the final response headers. The collection of all header values matching the WWW-Authenticate header is complicated by the fact that it is legal for header fields to be continued over multiple lines, but libcurl only gives us one line at a time. In the future [2] we may be able to leverage functions to read headers from libcurl itself, but as of today we must do this ourselves. [1] https://datatracker.ietf.org/doc/html/rfc2616#section-4.2 [2] https://daniel.haxx.se/blog/2022/03/22/a-headers-api-for-libcurl/ Signed-off-by: Matthew John Cheetham --- credential.c | 1 + credential.h | 15 ++++++++++ http.c | 78 ++++++++++++++++++++++++++++++++++++++++++++++++++++ 3 files changed, 94 insertions(+) diff --git a/credential.c b/credential.c index f6389a50684..897b4679333 100644 --- a/credential.c +++ b/credential.c @@ -22,6 +22,7 @@ void credential_clear(struct credential *c) free(c->username); free(c->password); string_list_clear(&c->helpers, 0); + strvec_clear(&c->wwwauth_headers); credential_init(c); } diff --git a/credential.h b/credential.h index f430e77fea4..6f2e5bc610b 100644 --- a/credential.h +++ b/credential.h @@ -2,6 +2,7 @@ #define CREDENTIAL_H #include "string-list.h" +#include "strvec.h" /** * The credentials API provides an abstracted way of gathering username and @@ -115,6 +116,19 @@ struct credential { */ struct string_list helpers; + /** + * A `strvec` of WWW-Authenticate header values. Each string + * is the value of a WWW-Authenticate header in an HTTP response, + * in the order they were received in the response. + */ + struct strvec wwwauth_headers; + + /** + * Internal use only. Used to keep track of split header fields + * in order to fold multiple lines into one value. + */ + unsigned header_is_last_match:1; + unsigned approved:1, configured:1, quit:1, @@ -130,6 +144,7 @@ struct credential { #define CREDENTIAL_INIT { \ .helpers = STRING_LIST_INIT_DUP, \ + .wwwauth_headers = STRVEC_INIT, \ } /* Initialize a credential structure, setting all fields to empty. */ diff --git a/http.c b/http.c index 8a5ba3f4776..c4e9cd73e14 100644 --- a/http.c +++ b/http.c @@ -183,6 +183,82 @@ size_t fwrite_buffer(char *ptr, size_t eltsize, size_t nmemb, void *buffer_) return nmemb; } +static size_t fwrite_wwwauth(char *ptr, size_t eltsize, size_t nmemb, void *p) +{ + size_t size = eltsize * nmemb; + struct strvec *values = &http_auth.wwwauth_headers; + struct strbuf buf = STRBUF_INIT; + const char *val; + const char *z = NULL; + + /* + * Header lines may not come NULL-terminated from libcurl so we must + * limit all scans to the maximum length of the header line, or leverage + * strbufs for all operations. + * + * In addition, it is possible that header values can be split over + * multiple lines as per RFC 2616 (even though this has since been + * deprecated in RFC 7230). A continuation header field value is + * identified as starting with a space or horizontal tab. + * + * The formal definition of a header field as given in RFC 2616 is: + * + * message-header = field-name ":" [ field-value ] + * field-name = token + * field-value = *( field-content | LWS ) + * field-content = + */ + + strbuf_add(&buf, ptr, size); + + /* Strip the CRLF that should be present at the end of each field */ + strbuf_trim_trailing_newline(&buf); + + /* Start of a new WWW-Authenticate header */ + if (skip_iprefix(buf.buf, "www-authenticate:", &val)) { + while (isspace(*val)) + val++; + + strvec_push(values, val); + http_auth.header_is_last_match = 1; + goto exit; + } + + /* + * This line could be a continuation of the previously matched header + * field. If this is the case then we should append this value to the + * end of the previously consumed value. + */ + if (http_auth.header_is_last_match && isspace(*buf.buf)) { + const char **v = values->v + values->nr - 1; + char *append = xstrfmt("%s%.*s", *v, (int)(size - 1), ptr + 1); + + free((void*)*v); + *v = append; + + goto exit; + } + + /* This is the start of a new header we don't care about */ + http_auth.header_is_last_match = 0; + + /* + * If this is a HTTP status line and not a header field, this signals + * a different HTTP response. libcurl writes all the output of all + * response headers of all responses, including redirects. + * We only care about the last HTTP request response's headers so clear + * the existing array. + */ + if (skip_iprefix(buf.buf, "http/", &z)) + strvec_clear(values); + +exit: + strbuf_release(&buf); + return size; +} + size_t fwrite_null(char *ptr, size_t eltsize, size_t nmemb, void *strbuf) { return nmemb; @@ -1864,6 +1940,8 @@ static int http_request(const char *url, fwrite_buffer); } + curl_easy_setopt(slot->curl, CURLOPT_HEADERFUNCTION, fwrite_wwwauth); + accept_language = http_get_accept_language_header(); if (accept_language) From patchwork Mon Dec 12 21:36:17 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Matthew John Cheetham X-Patchwork-Id: 13071456 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 33DE6C4332F for ; Mon, 12 Dec 2022 21:36:39 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233282AbiLLVgg (ORCPT ); Mon, 12 Dec 2022 16:36:36 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:35336 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232318AbiLLVga (ORCPT ); Mon, 12 Dec 2022 16:36:30 -0500 Received: from mail-wr1-x431.google.com (mail-wr1-x431.google.com [IPv6:2a00:1450:4864:20::431]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id B36D765E5 for ; Mon, 12 Dec 2022 13:36:29 -0800 (PST) Received: by mail-wr1-x431.google.com with SMTP id h12so13622697wrv.10 for ; Mon, 12 Dec 2022 13:36:29 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=cc:to:mime-version:content-transfer-encoding:fcc:subject:date:from :references:in-reply-to:message-id:from:to:cc:subject:date :message-id:reply-to; bh=HcsLOR+gIEF9OyHWTKumTW6RBJ2+B1MMahHJ0eVDYgY=; b=N1c/hvAMn1fedz/oGz92qIndQvhgXfDZjsl4nxD3szMioNUyIeP04sJtQ1jKICozHl TIRHp6nmpa7BgBusupJqmmbR9hVKaZo0o7ZbNogc2H/7SrBZoUDg2bWeBLioHR60kfO5 hTtpv3g2RKdnndPiFxQ9BY0VOlSIYvN2hCjpNDUcquBiFZEJEcLWF7ZbJ49HyHnTZxSL lbBIr/u0/lws/ehbfo0sWRbIUfVEHDYt/hG2L1AwTZvlLrwLV83xQkPugOi/teyATJ2+ A2UnzbgzOHTTpb+F6VdP7Au4VZEHLqkvCHpGpvyixth/8hbveDBX6Vzs55V73rFoDJ6h /RZQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=cc:to:mime-version:content-transfer-encoding:fcc:subject:date:from :references:in-reply-to:message-id:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=HcsLOR+gIEF9OyHWTKumTW6RBJ2+B1MMahHJ0eVDYgY=; b=Ep3+R2WWEBEmzz9wUTOPJ89K/E+WpFbu09bGZ5SowAsaWLLbs5xG363IwA0Ly3KhYb sW+NeOu14b64oXZYqgjekdu66wJLcTQYRbm/konRX/O9Br494f2dykUlDFEPd19ViOzH QZlEKcOTGKEnM8gQLllsU41XNpS/8BePHKXGIp4WHNpR1sAF2+rxA+r9RCd9zOw+yXeq hIbU/e5WtAfFTIZ1WL8dq7D07B9gfHYuNcvyBnNZnAdYrcagh8PhVP47g6kI852MNWnw hYUYhqgWmL2Ac0cDZXscoKr7fdOuyV+Blp5XKE9uUjvxEtJHfYfLb9fyz7TEsWWNKsfm Bznw== X-Gm-Message-State: ANoB5pnsF7AB2HkCJv0uO88o902763Ck2zfNh2iespJRKYnVDZyxIaHe NOW2SchousyObp3s9OZCABNtMAX40+Q= X-Google-Smtp-Source: AA0mqf7NCJBhj67ofMIa1d3rat0giJZyJZH++BV6Ekd06OceMZryP86NUeXATwY1i4PLsdbGNsqvtw== X-Received: by 2002:adf:fd52:0:b0:242:7d25:5892 with SMTP id h18-20020adffd52000000b002427d255892mr10570392wrs.16.1670880988133; Mon, 12 Dec 2022 13:36:28 -0800 (PST) Received: from [127.0.0.1] ([13.74.141.28]) by smtp.gmail.com with ESMTPSA id l2-20020a056000022200b002420dba6447sm9757834wrz.59.2022.12.12.13.36.27 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 12 Dec 2022 13:36:27 -0800 (PST) Message-Id: In-Reply-To: References: Date: Mon, 12 Dec 2022 21:36:17 +0000 Subject: [PATCH v4 2/8] credential: add WWW-Authenticate header to cred requests Fcc: Sent MIME-Version: 1.0 To: git@vger.kernel.org Cc: Derrick Stolee , Lessley Dennington , Matthew John Cheetham , M Hickford , Jeff Hostetler , Glen Choo , Matthew John Cheetham , Matthew John Cheetham Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org From: Matthew John Cheetham From: Matthew John Cheetham Add the value of the WWW-Authenticate response header to credential requests. Credential helpers that understand and support HTTP authentication and authorization can use this standard header (RFC 2616 Section 14.47 [1]) to generate valid credentials. WWW-Authenticate headers can contain information pertaining to the authority, authentication mechanism, or extra parameters/scopes that are required. The current I/O format for credential helpers only allows for unique names for properties/attributes, so in order to transmit multiple header values (with a specific order) we introduce a new convention whereby a C-style array syntax is used in the property name to denote multiple ordered values for the same property. In this case we send multiple `wwwauth[]` properties where the order that the repeated attributes appear in the conversation reflects the order that the WWW-Authenticate headers appeared in the HTTP response. [1] https://datatracker.ietf.org/doc/html/rfc2616#section-14.47 Signed-off-by: Matthew John Cheetham --- Documentation/git-credential.txt | 18 +++++++++++++++++- credential.c | 12 ++++++++++++ 2 files changed, 29 insertions(+), 1 deletion(-) diff --git a/Documentation/git-credential.txt b/Documentation/git-credential.txt index ac2818b9f66..bf0de0e9408 100644 --- a/Documentation/git-credential.txt +++ b/Documentation/git-credential.txt @@ -113,7 +113,13 @@ separated by an `=` (equals) sign, followed by a newline. The key may contain any bytes except `=`, newline, or NUL. The value may contain any bytes except newline or NUL. -In both cases, all bytes are treated as-is (i.e., there is no quoting, +Attributes with keys that end with C-style array brackets `[]` can have +multiple values. Each instance of a multi-valued attribute forms an +ordered list of values - the order of the repeated attributes defines +the order of the values. An empty multi-valued attribute (`key[]=\n`) +acts to clear any previous entries and reset the list. + +In all cases, all bytes are treated as-is (i.e., there is no quoting, and one cannot transmit a value with newline or NUL in it). The list of attributes is terminated by a blank line or end-of-file. @@ -160,6 +166,16 @@ empty string. Components which are missing from the URL (e.g., there is no username in the example above) will be left unset. +`wwwauth[]`:: + + When an HTTP response is received by Git that includes one or more + 'WWW-Authenticate' authentication headers, these will be passed by Git + to credential helpers. + Each 'WWW-Authenticate' header value is passed as a multi-valued + attribute 'wwwauth[]', where the order of the attributes is the same as + they appear in the HTTP response. This attribute is 'one-way' from Git + to pass additional information to credential helpers. + Unrecognised attributes are silently discarded. GIT diff --git a/credential.c b/credential.c index 897b4679333..8a3ad6c0ae2 100644 --- a/credential.c +++ b/credential.c @@ -263,6 +263,17 @@ static void credential_write_item(FILE *fp, const char *key, const char *value, fprintf(fp, "%s=%s\n", key, value); } +static void credential_write_strvec(FILE *fp, const char *key, + const struct strvec *vec) +{ + int i = 0; + const char *full_key = xstrfmt("%s[]", key); + for (; i < vec->nr; i++) { + credential_write_item(fp, full_key, vec->v[i], 0); + } + free((void*)full_key); +} + void credential_write(const struct credential *c, FILE *fp) { credential_write_item(fp, "protocol", c->protocol, 1); @@ -270,6 +281,7 @@ void credential_write(const struct credential *c, FILE *fp) credential_write_item(fp, "path", c->path, 0); credential_write_item(fp, "username", c->username, 0); credential_write_item(fp, "password", c->password, 0); + credential_write_strvec(fp, "wwwauth", &c->wwwauth_headers); } static int run_credential_helper(struct credential *c, From patchwork Mon Dec 12 21:36:18 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Matthew John Cheetham X-Patchwork-Id: 13071458 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 1A951C4167B for ; Mon, 12 Dec 2022 21:36:44 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233320AbiLLVgn (ORCPT ); Mon, 12 Dec 2022 16:36:43 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:35348 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232871AbiLLVgd (ORCPT ); Mon, 12 Dec 2022 16:36:33 -0500 Received: from mail-wm1-x330.google.com (mail-wm1-x330.google.com [IPv6:2a00:1450:4864:20::330]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 27B03A1B6 for ; Mon, 12 Dec 2022 13:36:31 -0800 (PST) Received: by mail-wm1-x330.google.com with SMTP id k22-20020a05600c1c9600b003d1ee3a6289so6503394wms.2 for ; Mon, 12 Dec 2022 13:36:31 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=cc:to:mime-version:content-transfer-encoding:fcc:subject:date:from :references:in-reply-to:message-id:from:to:cc:subject:date :message-id:reply-to; bh=ivi5Q9Uwsih/vMfDfz8elbck+Im+cI6/2ViNodEJxPQ=; b=fJM1vAqx7UMu5RSVJuhVY+S/pmRJ+Tx+CpjfAmD+55uQw5Spf9Zt+KHatLeWZw1mYn H+uBgDBA1/2k6GKZi5tvECB7NDaqWs7aVNTxI8x8KsizwRbffHo3AzpUIuEvIGwxlTgC 1R32TBnF1CxkR7sO14MFuN/Lbw3TqDzOguki2AVQ3AbNrx0vYjvO3uetskrc2fyBVPpB JshXkOHOIPQXQNpzabaSlzZyA3cQMNCPFJ8vSYtkI2gtPyCOMf2cL+t5GEnrEdGPmijd /FWmI/nDH+Sh9aXQWezwzvgxI31KdWouHksrjogTLOPxY9iEZ6CX746r+YbpsVxnuCaD 0k6g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=cc:to:mime-version:content-transfer-encoding:fcc:subject:date:from :references:in-reply-to:message-id:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=ivi5Q9Uwsih/vMfDfz8elbck+Im+cI6/2ViNodEJxPQ=; b=y8oMGQx9QYEcSMSz6qY2N1wU4aisHduOGtsi53qth4ou8HDKd6HTCnuakoL5kKIOAI 7f2obGe+zsIk8zXJQaCbILZcgtf1giVbJvifctdALeHKAiBl2h/rAM75UZFbOBSehJF7 E7EOjOPyQ+a9aFGOJxKKg3Iu4pnasDNPzemDeiG7VLscTK0PByPVHVXxZZKk8w3YNYIp F9r/6S9qkdMoxureGuo7SJmQYVpN3crx2WdOvlSZT9ZC070KQ0IkS7Tgy01b160jL14s ODI6Vv8NtCujjtCgscIsYS0t2W5YqpoRb0SlKRg6jVf22VZy46y9nJcy3vTBUjwvzfw9 x2CQ== X-Gm-Message-State: ANoB5pnByC46qbyeBE1MbfyXga/I1MxC4ukP63kwK8RVLuXGbzOYmZeE 65J0nywQjN64d9nybuv40oftiGLQ9ZQ= X-Google-Smtp-Source: AA0mqf7qy3u1Eu76KmPp+rqzS3DMJ2k5sDc0bzt2UFNs0hHr2YQnDI14Ois3Tb/+yfUJW8BOcq/tyQ== X-Received: by 2002:a05:600c:4891:b0:3d1:fbf9:3bd4 with SMTP id j17-20020a05600c489100b003d1fbf93bd4mr13396568wmp.10.1670880989224; Mon, 12 Dec 2022 13:36:29 -0800 (PST) Received: from [127.0.0.1] ([13.74.141.28]) by smtp.gmail.com with ESMTPSA id m17-20020a05600c3b1100b003c6b7f5567csm58279wms.0.2022.12.12.13.36.28 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 12 Dec 2022 13:36:28 -0800 (PST) Message-Id: <07a1845ea5693fc8d3716e7f97e65d467f34a40e.1670880984.git.gitgitgadget@gmail.com> In-Reply-To: References: Date: Mon, 12 Dec 2022 21:36:18 +0000 Subject: [PATCH v4 3/8] test-http-server: add stub HTTP server test helper Fcc: Sent MIME-Version: 1.0 To: git@vger.kernel.org Cc: Derrick Stolee , Lessley Dennington , Matthew John Cheetham , M Hickford , Jeff Hostetler , Glen Choo , Matthew John Cheetham , Matthew John Cheetham Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org From: Matthew John Cheetham From: Matthew John Cheetham Introduce a mini HTTP server helper that in the future will be enhanced to provide a frontend for the git-http-backend, with support for arbitrary authentication schemes. Right now, test-http-server is a pared-down copy of the git-daemon that always returns a 501 Not Implemented response to all callers. Signed-off-by: Matthew John Cheetham --- Makefile | 2 + contrib/buildsystems/CMakeLists.txt | 13 + t/helper/.gitignore | 1 + t/helper/test-http-server.c | 685 ++++++++++++++++++++++++++++ 4 files changed, 701 insertions(+) create mode 100644 t/helper/test-http-server.c diff --git a/Makefile b/Makefile index b258fdbed86..1eb795bbfd4 100644 --- a/Makefile +++ b/Makefile @@ -1611,6 +1611,8 @@ else endif BASIC_CFLAGS += $(CURL_CFLAGS) + TEST_PROGRAMS_NEED_X += test-http-server + REMOTE_CURL_PRIMARY = git-remote-http$X REMOTE_CURL_ALIASES = git-remote-https$X git-remote-ftp$X git-remote-ftps$X REMOTE_CURL_NAMES = $(REMOTE_CURL_PRIMARY) $(REMOTE_CURL_ALIASES) diff --git a/contrib/buildsystems/CMakeLists.txt b/contrib/buildsystems/CMakeLists.txt index 2f6e0197ffa..e9b9bfbb437 100644 --- a/contrib/buildsystems/CMakeLists.txt +++ b/contrib/buildsystems/CMakeLists.txt @@ -989,6 +989,19 @@ set(wrapper_scripts set(wrapper_test_scripts test-fake-ssh test-tool) +if(CURL_FOUND) + list(APPEND wrapper_test_scripts test-http-server) + + add_executable(test-http-server ${CMAKE_SOURCE_DIR}/t/helper/test-http-server.c) + target_link_libraries(test-http-server common-main) + + if(MSVC) + set_target_properties(test-http-server + PROPERTIES RUNTIME_OUTPUT_DIRECTORY_DEBUG ${CMAKE_BINARY_DIR}/t/helper) + set_target_properties(test-http-server + PROPERTIES RUNTIME_OUTPUT_DIRECTORY_RELEASE ${CMAKE_BINARY_DIR}/t/helper) + endif() +endif() foreach(script ${wrapper_scripts}) file(STRINGS ${CMAKE_SOURCE_DIR}/wrap-for-bin.sh content NEWLINE_CONSUME) diff --git a/t/helper/.gitignore b/t/helper/.gitignore index 8c2ddcce95f..9aa9c752997 100644 --- a/t/helper/.gitignore +++ b/t/helper/.gitignore @@ -1,2 +1,3 @@ /test-tool /test-fake-ssh +/test-http-server diff --git a/t/helper/test-http-server.c b/t/helper/test-http-server.c new file mode 100644 index 00000000000..18f1f741305 --- /dev/null +++ b/t/helper/test-http-server.c @@ -0,0 +1,685 @@ +#include "config.h" +#include "run-command.h" +#include "strbuf.h" +#include "string-list.h" +#include "trace2.h" +#include "version.h" +#include "dir.h" +#include "date.h" + +#define TR2_CAT "test-http-server" + +static const char *pid_file; +static int verbose; +static int reuseaddr; + +static const char test_http_auth_usage[] = +"http-server [--verbose]\n" +" [--timeout=] [--init-timeout=] [--max-connections=]\n" +" [--reuseaddr] [--pid-file=]\n" +" [--listen=]* [--port=]\n" +; + +/* Timeout, and initial timeout */ +static unsigned int timeout; +static unsigned int init_timeout; + +static void logreport(const char *label, const char *err, va_list params) +{ + struct strbuf msg = STRBUF_INIT; + + strbuf_addf(&msg, "[%"PRIuMAX"] %s: ", (uintmax_t)getpid(), label); + strbuf_vaddf(&msg, err, params); + strbuf_addch(&msg, '\n'); + + fwrite(msg.buf, sizeof(char), msg.len, stderr); + fflush(stderr); + + strbuf_release(&msg); +} + +__attribute__((format (printf, 1, 2))) +static void logerror(const char *err, ...) +{ + va_list params; + va_start(params, err); + logreport("error", err, params); + va_end(params); +} + +__attribute__((format (printf, 1, 2))) +static void loginfo(const char *err, ...) +{ + va_list params; + if (!verbose) + return; + va_start(params, err); + logreport("info", err, params); + va_end(params); +} + +static void set_keep_alive(int sockfd) +{ + int ka = 1; + + if (setsockopt(sockfd, SOL_SOCKET, SO_KEEPALIVE, &ka, sizeof(ka)) < 0) { + if (errno != ENOTSOCK) + logerror("unable to set SO_KEEPALIVE on socket: %s", + strerror(errno)); + } +} + +/* + * The code in this section is used by "worker" instances to service + * a single connection from a client. The worker talks to the client + * on 0 and 1. + */ + +enum worker_result { + /* + * Operation successful. + * Caller *might* keep the socket open and allow keep-alive. + */ + WR_OK = 0, + + /* + * Various errors while processing the request and/or the response. + * Close the socket and clean up. + * Exit child-process with non-zero status. + */ + WR_IO_ERROR = 1<<0, + + /* + * Close the socket and clean up. Does not imply an error. + */ + WR_HANGUP = 1<<1, + + WR_STOP_THE_MUSIC = (WR_IO_ERROR | WR_HANGUP), +}; + +static enum worker_result worker(void) +{ + const char *response = "HTTP/1.1 501 Not Implemented\r\n"; + char *client_addr = getenv("REMOTE_ADDR"); + char *client_port = getenv("REMOTE_PORT"); + enum worker_result wr = WR_OK; + + if (client_addr) + loginfo("Connection from %s:%s", client_addr, client_port); + + set_keep_alive(0); + + while (1) { + if (write_in_full(1, response, strlen(response)) < 0) { + logerror("unable to write response"); + wr = WR_IO_ERROR; + } + + if (wr & WR_STOP_THE_MUSIC) + break; + } + + close(0); + close(1); + + return !!(wr & WR_IO_ERROR); +} + +/* + * This section contains the listener and child-process management + * code used by the primary instance to accept incoming connections + * and dispatch them to async child process "worker" instances. + */ + +static int addrcmp(const struct sockaddr_storage *s1, + const struct sockaddr_storage *s2) +{ + const struct sockaddr *sa1 = (const struct sockaddr*) s1; + const struct sockaddr *sa2 = (const struct sockaddr*) s2; + + if (sa1->sa_family != sa2->sa_family) + return sa1->sa_family - sa2->sa_family; + if (sa1->sa_family == AF_INET) + return memcmp(&((struct sockaddr_in *)s1)->sin_addr, + &((struct sockaddr_in *)s2)->sin_addr, + sizeof(struct in_addr)); +#ifndef NO_IPV6 + if (sa1->sa_family == AF_INET6) + return memcmp(&((struct sockaddr_in6 *)s1)->sin6_addr, + &((struct sockaddr_in6 *)s2)->sin6_addr, + sizeof(struct in6_addr)); +#endif + return 0; +} + +static int max_connections = 32; + +static unsigned int live_children; + +static struct child { + struct child *next; + struct child_process cld; + struct sockaddr_storage address; +} *firstborn; + +static void add_child(struct child_process *cld, struct sockaddr *addr, socklen_t addrlen) +{ + struct child *newborn, **cradle; + + newborn = xcalloc(1, sizeof(*newborn)); + live_children++; + memcpy(&newborn->cld, cld, sizeof(*cld)); + memcpy(&newborn->address, addr, addrlen); + for (cradle = &firstborn; *cradle; cradle = &(*cradle)->next) + if (!addrcmp(&(*cradle)->address, &newborn->address)) + break; + newborn->next = *cradle; + *cradle = newborn; +} + +/* + * This gets called if the number of connections grows + * past "max_connections". + * + * We kill the newest connection from a duplicate IP. + */ +static void kill_some_child(void) +{ + const struct child *blanket, *next; + + if (!(blanket = firstborn)) + return; + + for (; (next = blanket->next); blanket = next) + if (!addrcmp(&blanket->address, &next->address)) { + kill(blanket->cld.pid, SIGTERM); + break; + } +} + +static void check_dead_children(void) +{ + int status; + pid_t pid; + + struct child **cradle, *blanket; + for (cradle = &firstborn; (blanket = *cradle);) + if ((pid = waitpid(blanket->cld.pid, &status, WNOHANG)) > 1) { + const char *dead = ""; + if (status) + dead = " (with error)"; + loginfo("[%"PRIuMAX"] Disconnected%s", (uintmax_t)pid, dead); + + /* remove the child */ + *cradle = blanket->next; + live_children--; + child_process_clear(&blanket->cld); + free(blanket); + } else + cradle = &blanket->next; +} + +static struct strvec cld_argv = STRVEC_INIT; +static void handle(int incoming, struct sockaddr *addr, socklen_t addrlen) +{ + struct child_process cld = CHILD_PROCESS_INIT; + + if (max_connections && live_children >= max_connections) { + kill_some_child(); + sleep(1); /* give it some time to die */ + check_dead_children(); + if (live_children >= max_connections) { + close(incoming); + logerror("Too many children, dropping connection"); + return; + } + } + + if (addr->sa_family == AF_INET) { + char buf[128] = ""; + struct sockaddr_in *sin_addr = (void *) addr; + inet_ntop(addr->sa_family, &sin_addr->sin_addr, buf, sizeof(buf)); + strvec_pushf(&cld.env, "REMOTE_ADDR=%s", buf); + strvec_pushf(&cld.env, "REMOTE_PORT=%d", + ntohs(sin_addr->sin_port)); +#ifndef NO_IPV6 + } else if (addr->sa_family == AF_INET6) { + char buf[128] = ""; + struct sockaddr_in6 *sin6_addr = (void *) addr; + inet_ntop(AF_INET6, &sin6_addr->sin6_addr, buf, sizeof(buf)); + strvec_pushf(&cld.env, "REMOTE_ADDR=[%s]", buf); + strvec_pushf(&cld.env, "REMOTE_PORT=%d", + ntohs(sin6_addr->sin6_port)); +#endif + } + + strvec_pushv(&cld.args, cld_argv.v); + cld.in = incoming; + cld.out = dup(incoming); + + if (cld.out < 0) + logerror("could not dup() `incoming`"); + else if (start_command(&cld)) + logerror("unable to fork"); + else + add_child(&cld, addr, addrlen); +} + +static void child_handler(int signo) +{ + /* + * Otherwise empty handler because systemcalls will get interrupted + * upon signal receipt + * SysV needs the handler to be rearmed + */ + signal(SIGCHLD, child_handler); +} + +static int set_reuse_addr(int sockfd) +{ + int on = 1; + + if (!reuseaddr) + return 0; + return setsockopt(sockfd, SOL_SOCKET, SO_REUSEADDR, + &on, sizeof(on)); +} + +struct socketlist { + int *list; + size_t nr; + size_t alloc; +}; + +static const char *ip2str(int family, struct sockaddr *sin, socklen_t len) +{ +#ifdef NO_IPV6 + static char ip[INET_ADDRSTRLEN]; +#else + static char ip[INET6_ADDRSTRLEN]; +#endif + + switch (family) { +#ifndef NO_IPV6 + case AF_INET6: + inet_ntop(family, &((struct sockaddr_in6*)sin)->sin6_addr, ip, len); + break; +#endif + case AF_INET: + inet_ntop(family, &((struct sockaddr_in*)sin)->sin_addr, ip, len); + break; + default: + xsnprintf(ip, sizeof(ip), ""); + } + return ip; +} + +#ifndef NO_IPV6 + +static int setup_named_sock(char *listen_addr, int listen_port, struct socketlist *socklist) +{ + int socknum = 0; + char pbuf[NI_MAXSERV]; + struct addrinfo hints, *ai0, *ai; + int gai; + long flags; + + xsnprintf(pbuf, sizeof(pbuf), "%d", listen_port); + memset(&hints, 0, sizeof(hints)); + hints.ai_family = AF_UNSPEC; + hints.ai_socktype = SOCK_STREAM; + hints.ai_protocol = IPPROTO_TCP; + hints.ai_flags = AI_PASSIVE; + + gai = getaddrinfo(listen_addr, pbuf, &hints, &ai0); + if (gai) { + logerror("getaddrinfo() for %s failed: %s", listen_addr, gai_strerror(gai)); + return 0; + } + + for (ai = ai0; ai; ai = ai->ai_next) { + int sockfd; + + sockfd = socket(ai->ai_family, ai->ai_socktype, ai->ai_protocol); + if (sockfd < 0) + continue; + if (sockfd >= FD_SETSIZE) { + logerror("Socket descriptor too large"); + close(sockfd); + continue; + } + +#ifdef IPV6_V6ONLY + if (ai->ai_family == AF_INET6) { + int on = 1; + setsockopt(sockfd, IPPROTO_IPV6, IPV6_V6ONLY, + &on, sizeof(on)); + /* Note: error is not fatal */ + } +#endif + + if (set_reuse_addr(sockfd)) { + logerror("Could not set SO_REUSEADDR: %s", strerror(errno)); + close(sockfd); + continue; + } + + set_keep_alive(sockfd); + + if (bind(sockfd, ai->ai_addr, ai->ai_addrlen) < 0) { + logerror("Could not bind to %s: %s", + ip2str(ai->ai_family, ai->ai_addr, ai->ai_addrlen), + strerror(errno)); + close(sockfd); + continue; /* not fatal */ + } + if (listen(sockfd, 5) < 0) { + logerror("Could not listen to %s: %s", + ip2str(ai->ai_family, ai->ai_addr, ai->ai_addrlen), + strerror(errno)); + close(sockfd); + continue; /* not fatal */ + } + + flags = fcntl(sockfd, F_GETFD, 0); + if (flags >= 0) + fcntl(sockfd, F_SETFD, flags | FD_CLOEXEC); + + ALLOC_GROW(socklist->list, socklist->nr + 1, socklist->alloc); + socklist->list[socklist->nr++] = sockfd; + socknum++; + } + + freeaddrinfo(ai0); + + return socknum; +} + +#else /* NO_IPV6 */ + +static int setup_named_sock(char *listen_addr, int listen_port, struct socketlist *socklist) +{ + struct sockaddr_in sin; + int sockfd; + long flags; + + memset(&sin, 0, sizeof sin); + sin.sin_family = AF_INET; + sin.sin_port = htons(listen_port); + + if (listen_addr) { + /* Well, host better be an IP address here. */ + if (inet_pton(AF_INET, listen_addr, &sin.sin_addr.s_addr) <= 0) + return 0; + } else { + sin.sin_addr.s_addr = htonl(INADDR_ANY); + } + + sockfd = socket(AF_INET, SOCK_STREAM, 0); + if (sockfd < 0) + return 0; + + if (set_reuse_addr(sockfd)) { + logerror("Could not set SO_REUSEADDR: %s", strerror(errno)); + close(sockfd); + return 0; + } + + set_keep_alive(sockfd); + + if (bind(sockfd, (struct sockaddr *)&sin, sizeof sin) < 0) { + logerror("Could not bind to %s: %s", + ip2str(AF_INET, (struct sockaddr *)&sin, sizeof(sin)), + strerror(errno)); + close(sockfd); + return 0; + } + + if (listen(sockfd, 5) < 0) { + logerror("Could not listen to %s: %s", + ip2str(AF_INET, (struct sockaddr *)&sin, sizeof(sin)), + strerror(errno)); + close(sockfd); + return 0; + } + + flags = fcntl(sockfd, F_GETFD, 0); + if (flags >= 0) + fcntl(sockfd, F_SETFD, flags | FD_CLOEXEC); + + ALLOC_GROW(socklist->list, socklist->nr + 1, socklist->alloc); + socklist->list[socklist->nr++] = sockfd; + return 1; +} + +#endif + +static void socksetup(struct string_list *listen_addr, int listen_port, struct socketlist *socklist) +{ + if (!listen_addr->nr) + setup_named_sock("127.0.0.1", listen_port, socklist); + else { + int i, socknum; + for (i = 0; i < listen_addr->nr; i++) { + socknum = setup_named_sock(listen_addr->items[i].string, + listen_port, socklist); + + if (socknum == 0) + logerror("unable to allocate any listen sockets for host %s on port %u", + listen_addr->items[i].string, listen_port); + } + } +} + +static int service_loop(struct socketlist *socklist) +{ + struct pollfd *pfd; + int i; + + CALLOC_ARRAY(pfd, socklist->nr); + + for (i = 0; i < socklist->nr; i++) { + pfd[i].fd = socklist->list[i]; + pfd[i].events = POLLIN; + } + + signal(SIGCHLD, child_handler); + + for (;;) { + int i; + int nr_ready; + int timeout = (pid_file ? 100 : -1); + + check_dead_children(); + + nr_ready = poll(pfd, socklist->nr, timeout); + if (nr_ready < 0) { + if (errno != EINTR) { + logerror("Poll failed, resuming: %s", + strerror(errno)); + sleep(1); + } + continue; + } + else if (nr_ready == 0) { + /* + * If we have a pid_file, then we watch it. + * If someone deletes it, we shutdown the service. + * The shell scripts in the test suite will use this. + */ + if (!pid_file || file_exists(pid_file)) + continue; + goto shutdown; + } + + for (i = 0; i < socklist->nr; i++) { + if (pfd[i].revents & POLLIN) { + union { + struct sockaddr sa; + struct sockaddr_in sai; +#ifndef NO_IPV6 + struct sockaddr_in6 sai6; +#endif + } ss; + socklen_t sslen = sizeof(ss); + int incoming = accept(pfd[i].fd, &ss.sa, &sslen); + if (incoming < 0) { + switch (errno) { + case EAGAIN: + case EINTR: + case ECONNABORTED: + continue; + default: + die_errno("accept returned"); + } + } + handle(incoming, &ss.sa, sslen); + } + } + } + +shutdown: + loginfo("Starting graceful shutdown (pid-file gone)"); + for (i = 0; i < socklist->nr; i++) + close(socklist->list[i]); + + return 0; +} + +static int serve(struct string_list *listen_addr, int listen_port) +{ + struct socketlist socklist = { NULL, 0, 0 }; + + socksetup(listen_addr, listen_port, &socklist); + if (socklist.nr == 0) + die("unable to allocate any listen sockets on port %u", + listen_port); + + loginfo("Ready to rumble"); + + /* + * Wait to create the pid-file until we've setup the sockets + * and are open for business. + */ + if (pid_file) + write_file(pid_file, "%"PRIuMAX, (uintmax_t) getpid()); + + return service_loop(&socklist); +} + +/* + * This section is executed by both the primary instance and all + * worker instances. So, yes, each child-process re-parses the + * command line argument and re-discovers how it should behave. + */ + +int cmd_main(int argc, const char **argv) +{ + int listen_port = 0; + struct string_list listen_addr = STRING_LIST_INIT_NODUP; + int worker_mode = 0; + int i; + + trace2_cmd_name("test-http-server"); + setup_git_directory_gently(NULL); + + for (i = 1; i < argc; i++) { + const char *arg = argv[i]; + const char *v; + + if (skip_prefix(arg, "--listen=", &v)) { + string_list_append(&listen_addr, xstrdup_tolower(v)); + continue; + } + if (skip_prefix(arg, "--port=", &v)) { + char *end; + unsigned long n; + n = strtoul(v, &end, 0); + if (*v && !*end) { + listen_port = n; + continue; + } + } + if (!strcmp(arg, "--worker")) { + worker_mode = 1; + trace2_cmd_mode("worker"); + continue; + } + if (!strcmp(arg, "--verbose")) { + verbose = 1; + continue; + } + if (skip_prefix(arg, "--timeout=", &v)) { + timeout = atoi(v); + continue; + } + if (skip_prefix(arg, "--init-timeout=", &v)) { + init_timeout = atoi(v); + continue; + } + if (skip_prefix(arg, "--max-connections=", &v)) { + max_connections = atoi(v); + if (max_connections < 0) + max_connections = 0; /* unlimited */ + continue; + } + if (!strcmp(arg, "--reuseaddr")) { + reuseaddr = 1; + continue; + } + if (skip_prefix(arg, "--pid-file=", &v)) { + pid_file = v; + continue; + } + + fprintf(stderr, "error: unknown argument '%s'\n", arg); + usage(test_http_auth_usage); + } + + /* avoid splitting a message in the middle */ + setvbuf(stderr, NULL, _IOFBF, 4096); + + if (listen_port == 0) + listen_port = DEFAULT_GIT_PORT; + + /* + * If no --listen= args are given, the setup_named_sock() + * code will use receive a NULL address and set INADDR_ANY. + * This exposes both internal and external interfaces on the + * port. + * + * Disallow that and default to the internal-use-only loopback + * address. + */ + if (!listen_addr.nr) + string_list_append(&listen_addr, "127.0.0.1"); + + /* + * worker_mode is set in our own child process instances + * (that are bound to a connected socket from a client). + */ + if (worker_mode) + return worker(); + + /* + * `cld_argv` is a bit of a clever hack. The top-level instance + * of test-http-server does the normal bind/listen/accept stuff. + * For each incoming socket, the top-level process spawns + * a child instance of test-http-server *WITH* the additional + * `--worker` argument. This causes the child to set `worker_mode` + * and immediately call `worker()` using the connected socket (and + * without the usual need for fork() or threads). + * + * The magic here is made possible because `cld_argv` is static + * and handle() (called by service_loop()) knows about it. + */ + strvec_push(&cld_argv, argv[0]); + strvec_push(&cld_argv, "--worker"); + for (i = 1; i < argc; ++i) + strvec_push(&cld_argv, argv[i]); + + /* + * Setup primary instance to listen for connections. + */ + return serve(&listen_addr, listen_port); +} From patchwork Mon Dec 12 21:36:19 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Matthew John Cheetham X-Patchwork-Id: 13071457 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 9D220C4332F for ; Mon, 12 Dec 2022 21:36:41 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233314AbiLLVgk (ORCPT ); Mon, 12 Dec 2022 16:36:40 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:35346 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232792AbiLLVgc (ORCPT ); Mon, 12 Dec 2022 16:36:32 -0500 Received: from mail-wm1-x334.google.com (mail-wm1-x334.google.com [IPv6:2a00:1450:4864:20::334]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id DF1DD25E9 for ; Mon, 12 Dec 2022 13:36:31 -0800 (PST) Received: by mail-wm1-x334.google.com with SMTP id ay2-20020a05600c1e0200b003d22e3e796dso761694wmb.0 for ; Mon, 12 Dec 2022 13:36:31 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=cc:to:mime-version:content-transfer-encoding:fcc:subject:date:from :references:in-reply-to:message-id:from:to:cc:subject:date :message-id:reply-to; bh=Oppsb8WsLbEUxkPzx/oHb/EanaXnzcdNH3JS2XbniwU=; b=SIeb3jGdUp/n7SXESrkwlVURMSMU88LPhAaKoZHi7TBM6U1lr1M0Ezn1KlrXkDDn5p P2HQYQBPxuV6yPEP4erM9qXbyi4k9iDxZ8eBXDqgfnbF9VU+Cx3qqVg16WlxNNsafkiq JhMozByKv1uF8dR2iFrpzyvMt6ndgheKh8n5VWoW5zqI6nTr5dvu93sAvPh78WYRcdEi Kb1+b64fF5JlPJ9ZIYp/J4QbJWR3sYfgK9jqEmLuokFE5RFHLnh9tnBOlTx779szIdkg +GtlvT+I2Sj1E3CIk3pzgMqu9mEs2J9VMHW3fASW1af11GYsqV4u9JHSq4c4EoyEEIeB VYfA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=cc:to:mime-version:content-transfer-encoding:fcc:subject:date:from :references:in-reply-to:message-id:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=Oppsb8WsLbEUxkPzx/oHb/EanaXnzcdNH3JS2XbniwU=; b=E+tUZjfQBnviJvocCfTMSJjf2QbFxPtCE8Wk9RLfKalUWQgA1GC0+MmiO5jyTWLbF+ yIAY1+WzvQDMjWyD0O+TGvlwNFIYX4K0Zmg+l06B25dIXU6NX0Zjy2v6/YwYFPUE4Mxc MGTizRWOBlTl3Yr3QDIV5vgvxo1VbS8fx1WB79v8cbahpKWJjfTgh8d/8xemsDbZtAJT m+XGiHNfuFP9rBxb16/3akugz1NXGvx7gXevCP21a4pam6SECR1CZylltW7lCh6NV9FQ TFBsxQ2EHN63kNbmN1xNxLc8ZY2J/XlRPpbFzLLM14NziRn0wzEVib06Ys7D2rrNt8m+ T3YA== X-Gm-Message-State: ANoB5pn2w1XRjp1YLRrBwmLxJINLLsscZTCsWSo1bFvfyITBpCZ7vVn4 2xBCP8sBKR1wskP21l2Z2pFL5rUKtCU= X-Google-Smtp-Source: AA0mqf59c9/2zzO4ZukAmL3nAJz10YesJeQztPX+gLPJDTTQABH9xnfk2r06Tve2HnEowWPLeOC1wg== X-Received: by 2002:a05:600c:1d9f:b0:3cf:a80d:69ab with SMTP id p31-20020a05600c1d9f00b003cfa80d69abmr13755284wms.31.1670880990324; Mon, 12 Dec 2022 13:36:30 -0800 (PST) Received: from [127.0.0.1] ([13.74.141.28]) by smtp.gmail.com with ESMTPSA id g4-20020a5d46c4000000b00228d52b935asm9859938wrs.71.2022.12.12.13.36.29 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 12 Dec 2022 13:36:29 -0800 (PST) Message-Id: <98dd286db7c95b6401167c4a9b5e2336843d2629.1670880984.git.gitgitgadget@gmail.com> In-Reply-To: References: Date: Mon, 12 Dec 2022 21:36:19 +0000 Subject: [PATCH v4 4/8] test-http-server: add HTTP error response function Fcc: Sent MIME-Version: 1.0 To: git@vger.kernel.org Cc: Derrick Stolee , Lessley Dennington , Matthew John Cheetham , M Hickford , Jeff Hostetler , Glen Choo , Matthew John Cheetham , Matthew John Cheetham Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org From: Matthew John Cheetham From: Matthew John Cheetham Introduce a function to the test-http-server test helper to write more full and valid HTTP error responses, including all the standard response headers like `Server` and `Date`. Signed-off-by: Matthew John Cheetham --- t/helper/test-http-server.c | 59 +++++++++++++++++++++++++++++++++---- 1 file changed, 53 insertions(+), 6 deletions(-) diff --git a/t/helper/test-http-server.c b/t/helper/test-http-server.c index 18f1f741305..53508639714 100644 --- a/t/helper/test-http-server.c +++ b/t/helper/test-http-server.c @@ -97,9 +97,59 @@ enum worker_result { WR_STOP_THE_MUSIC = (WR_IO_ERROR | WR_HANGUP), }; +static enum worker_result send_http_error( + int fd, + int http_code, const char *http_code_name, + int retry_after_seconds, struct string_list *response_headers, + enum worker_result wr_in) +{ + struct strbuf response_header = STRBUF_INIT; + struct strbuf response_content = STRBUF_INIT; + struct string_list_item *h; + enum worker_result wr; + + strbuf_addf(&response_content, "Error: %d %s\r\n", + http_code, http_code_name); + if (retry_after_seconds > 0) + strbuf_addf(&response_content, "Retry-After: %d\r\n", + retry_after_seconds); + + strbuf_addf (&response_header, "HTTP/1.1 %d %s\r\n", http_code, http_code_name); + strbuf_addstr(&response_header, "Cache-Control: private\r\n"); + strbuf_addstr(&response_header, "Content-Type: text/plain\r\n"); + strbuf_addf (&response_header, "Content-Length: %d\r\n", (int)response_content.len); + if (retry_after_seconds > 0) + strbuf_addf(&response_header, "Retry-After: %d\r\n", retry_after_seconds); + strbuf_addf( &response_header, "Server: test-http-server/%s\r\n", git_version_string); + strbuf_addf( &response_header, "Date: %s\r\n", show_date(time(NULL), 0, DATE_MODE(RFC2822))); + if (response_headers) + for_each_string_list_item(h, response_headers) + strbuf_addf(&response_header, "%s\r\n", h->string); + strbuf_addstr(&response_header, "\r\n"); + + if (write_in_full(fd, response_header.buf, response_header.len) < 0) { + logerror("unable to write response header"); + wr = WR_IO_ERROR; + goto done; + } + + if (write_in_full(fd, response_content.buf, response_content.len) < 0) { + logerror("unable to write response content body"); + wr = WR_IO_ERROR; + goto done; + } + + wr = wr_in; + +done: + strbuf_release(&response_header); + strbuf_release(&response_content); + + return wr; +} + static enum worker_result worker(void) { - const char *response = "HTTP/1.1 501 Not Implemented\r\n"; char *client_addr = getenv("REMOTE_ADDR"); char *client_port = getenv("REMOTE_PORT"); enum worker_result wr = WR_OK; @@ -110,11 +160,8 @@ static enum worker_result worker(void) set_keep_alive(0); while (1) { - if (write_in_full(1, response, strlen(response)) < 0) { - logerror("unable to write response"); - wr = WR_IO_ERROR; - } - + wr = send_http_error(1, 501, "Not Implemented", -1, NULL, + WR_OK | WR_HANGUP); if (wr & WR_STOP_THE_MUSIC) break; } From patchwork Mon Dec 12 21:36:20 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Matthew John Cheetham X-Patchwork-Id: 13071459 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id D1D1BC4332F for ; Mon, 12 Dec 2022 21:36:46 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233333AbiLLVgo (ORCPT ); Mon, 12 Dec 2022 16:36:44 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:35362 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231708AbiLLVge (ORCPT ); Mon, 12 Dec 2022 16:36:34 -0500 Received: from mail-wm1-x335.google.com (mail-wm1-x335.google.com [IPv6:2a00:1450:4864:20::335]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 1E81F65A1 for ; Mon, 12 Dec 2022 13:36:33 -0800 (PST) Received: by mail-wm1-x335.google.com with SMTP id ay2-20020a05600c1e0200b003d22e3e796dso761724wmb.0 for ; Mon, 12 Dec 2022 13:36:33 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=cc:to:mime-version:content-transfer-encoding:fcc:subject:date:from :references:in-reply-to:message-id:from:to:cc:subject:date :message-id:reply-to; bh=Pys0iYEZZUGF1uUgWPkThf6R1S5uxxZxLTtL9ZuAB7M=; b=ogBhGSUoR9TzV7jALkUEXUOTv28fdLylfx+wZW37rfU66KDTov2tIDaQcXg9Juo2LF 4f3PlXzX/htquijAk0XMO5tuzNIle/vlC3g2Sd5HJcUb2uuXR/LcapykGkXndxCI5SHx z6RRl4EkAGY24KQgHM2CdAOySurh3PAhYrCJMf8PPKivLnDq6Pnbwe9UK6Ao9kHUVSxa S58bAaIp3ezBcOgr+TD9d5iIgPc+0kiuf151q8pS+xcaAOz5aivf7QBpf+Pb9ROBQo1c yN9WUgxNdhH68mayi7oZOC2mKYB63AsVQJYJI9srJ8WYlXC3vxcXkC76tOPC6V33mCte wK+w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=cc:to:mime-version:content-transfer-encoding:fcc:subject:date:from :references:in-reply-to:message-id:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=Pys0iYEZZUGF1uUgWPkThf6R1S5uxxZxLTtL9ZuAB7M=; b=6WdsKPI2wSyyqkQT7KDu5m34oM/KdzRNlcpj/v7Ta5wndC529wvlM+lGeAeR0zb1HY ViA73DtaSzIrbfec99XhBW7ivxzVT8cR78zkCAZfA2DReTHOVKRcqqsr++Lt6tLY4aQP q5cjPNP8nj5s0K5SILXlUT6NRMW3mV4XqHWKidBsO1Eq3g+X0k2IE0vxmTb8AWt5P76m N/E+d2+LHHiHRnqBTqfYAjQaO6Y+ffGBaDC5nZLVqBG0lcNV+sO8WR5+HZgvx0KfyWND FPVfLUJ1aLVpqoCAqg0wrpzhgF6RDbXn9ll14X5ceSuYCr9rnW9dbvy/gs5MPDu0jJ15 nRgg== X-Gm-Message-State: ANoB5pkr2gfX4BpG7g8MjsNLvRnxH85hzqIsVM4mKGMKP1clP3tUvSIw 7pQyCrHHI3m+y+sN+f/RCrM9C/OX1Aw= X-Google-Smtp-Source: AA0mqf5MjicRWvmLvjydqDoO4JfpoXFAoR30rcF7bFgueJLSDbWX0IZkCV87fYffui6b/2RBO9sEIQ== X-Received: by 2002:a05:600c:3ca5:b0:3cf:8957:a441 with SMTP id bg37-20020a05600c3ca500b003cf8957a441mr13625162wmb.12.1670880991308; Mon, 12 Dec 2022 13:36:31 -0800 (PST) Received: from [127.0.0.1] ([13.74.141.28]) by smtp.gmail.com with ESMTPSA id m16-20020a7bcb90000000b003cf37c5ddc0sm10250824wmi.22.2022.12.12.13.36.30 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 12 Dec 2022 13:36:30 -0800 (PST) Message-Id: <5c4e36e23eecbb7841078939a982b7150e2f4ab8.1670880984.git.gitgitgadget@gmail.com> In-Reply-To: References: Date: Mon, 12 Dec 2022 21:36:20 +0000 Subject: [PATCH v4 5/8] test-http-server: add HTTP request parsing Fcc: Sent MIME-Version: 1.0 To: git@vger.kernel.org Cc: Derrick Stolee , Lessley Dennington , Matthew John Cheetham , M Hickford , Jeff Hostetler , Glen Choo , Matthew John Cheetham , Matthew John Cheetham Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org From: Matthew John Cheetham From: Matthew John Cheetham Add ability to parse HTTP requests to the test-http-server test helper. Signed-off-by: Matthew John Cheetham --- t/helper/test-http-server.c | 176 +++++++++++++++++++++++++++++++++++- 1 file changed, 174 insertions(+), 2 deletions(-) diff --git a/t/helper/test-http-server.c b/t/helper/test-http-server.c index 53508639714..7bde678e264 100644 --- a/t/helper/test-http-server.c +++ b/t/helper/test-http-server.c @@ -97,6 +97,42 @@ enum worker_result { WR_STOP_THE_MUSIC = (WR_IO_ERROR | WR_HANGUP), }; +/* + * Fields from a parsed HTTP request. + */ +struct req { + struct strbuf start_line; + + const char *method; + const char *http_version; + + struct strbuf uri_path; + struct strbuf query_args; + + struct string_list header_list; + const char *content_type; + ssize_t content_length; +}; + +#define REQ__INIT { \ + .start_line = STRBUF_INIT, \ + .uri_path = STRBUF_INIT, \ + .query_args = STRBUF_INIT, \ + .header_list = STRING_LIST_INIT_NODUP, \ + .content_type = NULL, \ + .content_length = -1 \ + } + +static void req__release(struct req *req) +{ + strbuf_release(&req->start_line); + + strbuf_release(&req->uri_path); + strbuf_release(&req->query_args); + + string_list_clear(&req->header_list, 0); +} + static enum worker_result send_http_error( int fd, int http_code, const char *http_code_name, @@ -148,8 +184,136 @@ done: return wr; } +/* + * Read the HTTP request up to the start of the optional message-body. + * We do this byte-by-byte because we have keep-alive turned on and + * cannot rely on an EOF. + * + * https://tools.ietf.org/html/rfc7230 + * + * We cannot call die() here because our caller needs to properly + * respond to the client and/or close the socket before this + * child exits so that the client doesn't get a connection reset + * by peer error. + */ +static enum worker_result req__read(struct req *req, int fd) +{ + struct strbuf h = STRBUF_INIT; + struct string_list start_line_fields = STRING_LIST_INIT_DUP; + int nr_start_line_fields; + const char *uri_target; + const char *query; + char *hp; + const char *hv; + + enum worker_result result = WR_OK; + + /* + * Read line 0 of the request and split it into component parts: + * + * SP SP CRLF + * + */ + if (strbuf_getwholeline_fd(&req->start_line, fd, '\n') == EOF) { + result = WR_OK | WR_HANGUP; + goto done; + } + + strbuf_trim_trailing_newline(&req->start_line); + + nr_start_line_fields = string_list_split(&start_line_fields, + req->start_line.buf, + ' ', -1); + if (nr_start_line_fields != 3) { + logerror("could not parse request start-line '%s'", + req->start_line.buf); + result = WR_IO_ERROR; + goto done; + } + + req->method = xstrdup(start_line_fields.items[0].string); + req->http_version = xstrdup(start_line_fields.items[2].string); + + uri_target = start_line_fields.items[1].string; + + if (strcmp(req->http_version, "HTTP/1.1")) { + logerror("unsupported version '%s' (expecting HTTP/1.1)", + req->http_version); + result = WR_IO_ERROR; + goto done; + } + + query = strchr(uri_target, '?'); + + if (query) { + strbuf_add(&req->uri_path, uri_target, (query - uri_target)); + strbuf_trim_trailing_dir_sep(&req->uri_path); + strbuf_addstr(&req->query_args, query + 1); + } else { + strbuf_addstr(&req->uri_path, uri_target); + strbuf_trim_trailing_dir_sep(&req->uri_path); + } + + /* + * Read the set of HTTP headers into a string-list. + */ + while (1) { + if (strbuf_getwholeline_fd(&h, fd, '\n') == EOF) + goto done; + strbuf_trim_trailing_newline(&h); + + if (!h.len) + goto done; /* a blank line ends the header */ + + hp = strbuf_detach(&h, NULL); + string_list_append(&req->header_list, hp); + + /* store common request headers separately */ + if (skip_prefix(hp, "Content-Type: ", &hv)) { + req->content_type = hv; + } else if (skip_prefix(hp, "Content-Length: ", &hv)) { + req->content_length = strtol(hv, &hp, 10); + } + } + + /* + * We do not attempt to read the , if it exists. + * We let our caller read/chunk it in as appropriate. + */ + +done: + string_list_clear(&start_line_fields, 0); + + /* + * This is useful for debugging the request, but very noisy. + */ + if (trace2_is_enabled()) { + struct string_list_item *item; + trace2_printf("%s: %s", TR2_CAT, req->start_line.buf); + trace2_printf("%s: hver: %s", TR2_CAT, req->http_version); + trace2_printf("%s: hmth: %s", TR2_CAT, req->method); + trace2_printf("%s: path: %s", TR2_CAT, req->uri_path.buf); + trace2_printf("%s: qury: %s", TR2_CAT, req->query_args.buf); + if (req->content_length >= 0) + trace2_printf("%s: clen: %d", TR2_CAT, req->content_length); + if (req->content_type) + trace2_printf("%s: ctyp: %s", TR2_CAT, req->content_type); + for_each_string_list_item(item, &req->header_list) + trace2_printf("%s: hdrs: %s", TR2_CAT, item->string); + } + + return result; +} + +static enum worker_result dispatch(struct req *req) +{ + return send_http_error(1, 501, "Not Implemented", -1, NULL, + WR_OK | WR_HANGUP); +} + static enum worker_result worker(void) { + struct req req = REQ__INIT; char *client_addr = getenv("REMOTE_ADDR"); char *client_port = getenv("REMOTE_PORT"); enum worker_result wr = WR_OK; @@ -160,8 +324,16 @@ static enum worker_result worker(void) set_keep_alive(0); while (1) { - wr = send_http_error(1, 501, "Not Implemented", -1, NULL, - WR_OK | WR_HANGUP); + req__release(&req); + + alarm(init_timeout ? init_timeout : timeout); + wr = req__read(&req, 0); + alarm(0); + + if (wr & WR_STOP_THE_MUSIC) + break; + + wr = dispatch(&req); if (wr & WR_STOP_THE_MUSIC) break; } From patchwork Mon Dec 12 21:36:21 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Matthew John Cheetham X-Patchwork-Id: 13071460 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 335D2C4167B for ; Mon, 12 Dec 2022 21:36:48 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233346AbiLLVgr (ORCPT ); Mon, 12 Dec 2022 16:36:47 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:35370 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S233197AbiLLVge (ORCPT ); Mon, 12 Dec 2022 16:36:34 -0500 Received: from mail-wm1-x32f.google.com (mail-wm1-x32f.google.com [IPv6:2a00:1450:4864:20::32f]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id BD3B9A1B6 for ; Mon, 12 Dec 2022 13:36:33 -0800 (PST) Received: by mail-wm1-x32f.google.com with SMTP id c65-20020a1c3544000000b003cfffd00fc0so6420762wma.1 for ; Mon, 12 Dec 2022 13:36:33 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=cc:to:mime-version:content-transfer-encoding:fcc:subject:date:from :references:in-reply-to:message-id:from:to:cc:subject:date :message-id:reply-to; bh=OvXDn+rrjem7Hg0M8fnuF0bVApTee9PoBQCoroxyf5w=; b=HRzpbO+Gs7e0Wnbg0CGcEiwXrr+TFfSx5HUVfZsfuoJ1dv9OAz/7KP+iMblUTJo15b 5Z7Jy5sv8bFFzClVUrfVEOtY+4w2r/H6df7X4uHWxn+s6eFSMVlZyTAg4/qa5QcrI9xx vrKxSM6U2BzzzUyQyc1sKRHHjGzjexu9i7rTitSpbaGB27rzAOu+bnaTMEWizXyOAuMN GwKcKuj+7Lw0MCMNidmLErJpUHyaLyiyRLTf5OQgMl3SABQuL2gn0taIcjG3lV+1wRqk JqhYd8R0uCDjYiCt8LuZwhwx2XIrTwlIlohbPV8a4drygWXJbGSsmPhB0k10amKSM/fL I+Ug== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=cc:to:mime-version:content-transfer-encoding:fcc:subject:date:from :references:in-reply-to:message-id:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=OvXDn+rrjem7Hg0M8fnuF0bVApTee9PoBQCoroxyf5w=; b=raTxsiHOvHD5y+ANhcOKpM07fY4DV09LP8VeGaQby5QPHHBT8/EqGJlpSQQH8+6Liv 1TqlO6cm8WuophPRWk5W7nSLJbnIKgmjqOlzOx+BiSrLC/UVlvB/SYOtnUjpMK7XHfdD 0tu+rnKslfnTYVCXf4/jaq8kvpM2g6bYizdpE9pogRxMMXuiTTaIeOe4TmEaXNF8ewIH yw39iwyogQF4SG32QzCDVwiqXqSeKqsRx5kpnB/uT06THdI6rtiNyMKVoF2zPbKmZeqM sAo3id0ulLJiEhEdVWHYGZeNbV0KMDVGBN2lvUbEiNOeUNjhtN7g0ONPYTGX1AZVAvmH X/YA== X-Gm-Message-State: ANoB5pn+iPOwwt3IZQxpHaA1bEcIK1/OcsN2X/OlPAPZaA1UP7fPJOuH rjN3YpXIs99lhrbL6vbzXIFpfdSlZQs= X-Google-Smtp-Source: AA0mqf4pW3PtYDZqlX79RCGwigMyxv2OPOX4abXWflHCyyPWY3goXYFrNbQQB8rZC+Bed1O/kccATA== X-Received: by 2002:a1c:720f:0:b0:3c6:e63e:23e6 with SMTP id n15-20020a1c720f000000b003c6e63e23e6mr13509545wmc.21.1670880992056; Mon, 12 Dec 2022 13:36:32 -0800 (PST) Received: from [127.0.0.1] ([13.74.141.28]) by smtp.gmail.com with ESMTPSA id q13-20020a7bce8d000000b003cf71b1f66csm10049113wmj.0.2022.12.12.13.36.31 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 12 Dec 2022 13:36:31 -0800 (PST) Message-Id: <0a0f4fd10c8b29f327c35dadc7b17881f22b253a.1670880984.git.gitgitgadget@gmail.com> In-Reply-To: References: Date: Mon, 12 Dec 2022 21:36:21 +0000 Subject: [PATCH v4 6/8] test-http-server: pass Git requests to http-backend Fcc: Sent MIME-Version: 1.0 To: git@vger.kernel.org Cc: Derrick Stolee , Lessley Dennington , Matthew John Cheetham , M Hickford , Jeff Hostetler , Glen Choo , Matthew John Cheetham , Matthew John Cheetham Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org From: Matthew John Cheetham From: Matthew John Cheetham Teach the test-http-sever test helper to forward Git requests to the `git-http-backend`. Introduce a new test script t5556-http-auth.sh that spins up the test HTTP server and attempts an `ls-remote` on the served repository, without any authentication. Signed-off-by: Matthew John Cheetham --- t/helper/test-http-server.c | 56 +++++++++++++++++++ t/t5556-http-auth.sh | 105 ++++++++++++++++++++++++++++++++++++ 2 files changed, 161 insertions(+) create mode 100755 t/t5556-http-auth.sh diff --git a/t/helper/test-http-server.c b/t/helper/test-http-server.c index 7bde678e264..9f1d6b58067 100644 --- a/t/helper/test-http-server.c +++ b/t/helper/test-http-server.c @@ -305,8 +305,64 @@ done: return result; } +static int is_git_request(struct req *req) +{ + static regex_t *smart_http_regex; + static int initialized; + + if (!initialized) { + smart_http_regex = xmalloc(sizeof(*smart_http_regex)); + if (regcomp(smart_http_regex, "^/(HEAD|info/refs|" + "objects/info/[^/]+|git-(upload|receive)-pack)$", + REG_EXTENDED)) { + warning("could not compile smart HTTP regex"); + smart_http_regex = NULL; + } + initialized = 1; + } + + return smart_http_regex && + !regexec(smart_http_regex, req->uri_path.buf, 0, NULL, 0); +} + +static enum worker_result do__git(struct req *req, const char *user) +{ + const char *ok = "HTTP/1.1 200 OK\r\n"; + struct child_process cp = CHILD_PROCESS_INIT; + int res; + + if (write(1, ok, strlen(ok)) < 0) + return error(_("could not send '%s'"), ok); + + if (user) + strvec_pushf(&cp.env, "REMOTE_USER=%s", user); + + strvec_pushf(&cp.env, "REQUEST_METHOD=%s", req->method); + strvec_pushf(&cp.env, "PATH_TRANSLATED=%s", + req->uri_path.buf); + strvec_push(&cp.env, "SERVER_PROTOCOL=HTTP/1.1"); + if (req->query_args.len) + strvec_pushf(&cp.env, "QUERY_STRING=%s", + req->query_args.buf); + if (req->content_type) + strvec_pushf(&cp.env, "CONTENT_TYPE=%s", + req->content_type); + if (req->content_length >= 0) + strvec_pushf(&cp.env, "CONTENT_LENGTH=%" PRIdMAX, + (intmax_t)req->content_length); + cp.git_cmd = 1; + strvec_push(&cp.args, "http-backend"); + res = run_command(&cp); + close(1); + close(0); + return !!res; +} + static enum worker_result dispatch(struct req *req) { + if (is_git_request(req)) + return do__git(req, NULL); + return send_http_error(1, 501, "Not Implemented", -1, NULL, WR_OK | WR_HANGUP); } diff --git a/t/t5556-http-auth.sh b/t/t5556-http-auth.sh new file mode 100755 index 00000000000..78da151f122 --- /dev/null +++ b/t/t5556-http-auth.sh @@ -0,0 +1,105 @@ +#!/bin/sh + +test_description='test http auth header and credential helper interop' + +. ./test-lib.sh + +test_set_port GIT_TEST_HTTP_PROTOCOL_PORT + +# Setup a repository +# +REPO_DIR="$(pwd)"/repo + +# Setup some lookback URLs where test-http-server will be listening. +# We will spawn it directly inside the repo directory, so we avoid +# any need to configure directory mappings etc - we only serve this +# repository from the root '/' of the server. +# +HOST_PORT=127.0.0.1:$GIT_TEST_HTTP_PROTOCOL_PORT +ORIGIN_URL=http://$HOST_PORT/ + +# The pid-file is created by test-http-server when it starts. +# The server will shutdown if/when we delete it (this is easier than +# killing it by PID). +# +PID_FILE="$(pwd)"/pid-file.pid +SERVER_LOG="$(pwd)"/OUT.server.log + +PATH="$GIT_BUILD_DIR/t/helper/:$PATH" && export PATH + +test_expect_success 'setup repos' ' + test_create_repo "$REPO_DIR" && + git -C "$REPO_DIR" branch -M main +' + +stop_http_server () { + if ! test -f "$PID_FILE" + then + return 0 + fi + # + # The server will shutdown automatically when we delete the pid-file. + # + rm -f "$PID_FILE" + # + # Give it a few seconds to shutdown (mainly to completely release the + # port before the next test start another instance and it attempts to + # bind to it). + # + for k in 0 1 2 3 4 + do + if grep -q "Starting graceful shutdown" "$SERVER_LOG" + then + return 0 + fi + sleep 1 + done + + echo "stop_http_server: timeout waiting for server shutdown" + return 1 +} + +start_http_server () { + # + # Launch our server into the background in repo_dir. + # + ( + cd "$REPO_DIR" + test-http-server --verbose \ + --listen=127.0.0.1 \ + --port=$GIT_TEST_HTTP_PROTOCOL_PORT \ + --reuseaddr \ + --pid-file="$PID_FILE" \ + "$@" \ + 2>"$SERVER_LOG" & + ) + # + # Give it a few seconds to get started. + # + for k in 0 1 2 3 4 + do + if test -f "$PID_FILE" + then + return 0 + fi + sleep 1 + done + + echo "start_http_server: timeout waiting for server startup" + return 1 +} + +per_test_cleanup () { + stop_http_server && + rm -f OUT.* +} + +test_expect_success 'http auth anonymous no challenge' ' + test_when_finished "per_test_cleanup" && + start_http_server --allow-anonymous && + + # Attempt to read from a protected repository + git ls-remote $ORIGIN_URL +' + +test_done From patchwork Mon Dec 12 21:36:22 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Matthew John Cheetham X-Patchwork-Id: 13071461 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 8B523C4332F for ; Mon, 12 Dec 2022 21:37:05 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233461AbiLLVhD (ORCPT ); Mon, 12 Dec 2022 16:37:03 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:35388 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S233269AbiLLVgf (ORCPT ); Mon, 12 Dec 2022 16:36:35 -0500 Received: from mail-wr1-x429.google.com (mail-wr1-x429.google.com [IPv6:2a00:1450:4864:20::429]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id B9615B7D8 for ; Mon, 12 Dec 2022 13:36:34 -0800 (PST) Received: by mail-wr1-x429.google.com with SMTP id q7so13640357wrr.8 for ; Mon, 12 Dec 2022 13:36:34 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=cc:to:mime-version:content-transfer-encoding:fcc:subject:date:from :references:in-reply-to:message-id:from:to:cc:subject:date :message-id:reply-to; bh=G+u+0cAjz1+TrMNJ0vQOg929e3nZQu9IrtC17YCqzI0=; b=it4WNTznGVVO2GSvB9ifeJik6A5jqCxF3NYGtH0jZ/1bXGepkp5r8iE8SQRiBGxAeq TwuDSK6EdhJapw4kULXPpToGo6+eXevD7bES6H7gJjLd7W1evx2/OwuHI68+y15A/jeH 8awIgaAimMFimcXa1votJz21MOYgDv1Sq0gipRK1VSDJGeaV2gBM/RhqBixmsqz7uMpP /cHvFg1WeB+4HRi+EDFPqCOMW9LIAN/lRV4+cdvamVAxvyaWVQl4GdtiSbJixZptv2f5 ngqx1Gys2Ptu6Mk2EOg6ONhd91mXQ4Y8f5AkOqH0Me7cRbAsWRxjvUchv+xDuAIwjEJG heNw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=cc:to:mime-version:content-transfer-encoding:fcc:subject:date:from :references:in-reply-to:message-id:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=G+u+0cAjz1+TrMNJ0vQOg929e3nZQu9IrtC17YCqzI0=; b=NxrRFPWD8voRS2bG6KXEfTNc22+IrBHJiSVJtIebsloGSxjssC/BxSMswAlX8mmiCL m6hPmSoMGsvxBMtqPdwBbDA43ycUNtyRGxIN2MgHRLx4/1MYWawcgJ4d8F/PxbMex3iy RiYk7KWnm50ULCNzdGTyGYnn8qtvfQNs++qQulwhD2p6YBlKBKSDYhbMp8yo6NQhTBq0 14M2cGtUiBCRe6ymc/ajxXyp2442XLFFFXROnrs80R1djFkh/ug0Jvi1pwYNbBB0aG+M MpBh0d740FI9CE2Ipg7U3J4d5YMtrka+HQ2FM4d7JiySjPpfDX2l1Hr7QgFEWlJ0tUUg q55g== X-Gm-Message-State: ANoB5pmFVmlnyaeB9se9rQZ3tAIX+oIZ8jmuMroxC3qxWFMlDEqVVxZ5 /02RVNUVqC/44A8PzJMzIMMv9dydIJc= X-Google-Smtp-Source: AA0mqf7Faj4vERfMn6zc2rkGufL7aTYnUXwJq4P4TLm/XH6Mjl+xXx07XG9wHP3dvTgEZh2Q1cjICA== X-Received: by 2002:adf:ffc1:0:b0:232:be5d:5eb7 with SMTP id x1-20020adfffc1000000b00232be5d5eb7mr11671203wrs.14.1670880993113; Mon, 12 Dec 2022 13:36:33 -0800 (PST) Received: from [127.0.0.1] ([13.74.141.28]) by smtp.gmail.com with ESMTPSA id h20-20020adfaa94000000b002367ad808a9sm9869112wrc.30.2022.12.12.13.36.32 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 12 Dec 2022 13:36:32 -0800 (PST) Message-Id: <794256754c1f7d32e438dfb19a05444d423989aa.1670880984.git.gitgitgadget@gmail.com> In-Reply-To: References: Date: Mon, 12 Dec 2022 21:36:22 +0000 Subject: [PATCH v4 7/8] test-http-server: add simple authentication Fcc: Sent MIME-Version: 1.0 To: git@vger.kernel.org Cc: Derrick Stolee , Lessley Dennington , Matthew John Cheetham , M Hickford , Jeff Hostetler , Glen Choo , Matthew John Cheetham , Matthew John Cheetham Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org From: Matthew John Cheetham From: Matthew John Cheetham Add simple authentication to the test-http-server test helper. Authentication schemes and sets of valid tokens can be specified via command-line arguments. Incoming requests are compared against the set of valid schemes and tokens and only approved if a matching token is found, or if no auth was provided and anonymous auth is enabled. Signed-off-by: Matthew John Cheetham --- t/helper/test-http-server.c | 188 +++++++++++++++++++++++++++++++++++- 1 file changed, 187 insertions(+), 1 deletion(-) diff --git a/t/helper/test-http-server.c b/t/helper/test-http-server.c index 9f1d6b58067..9a458743d13 100644 --- a/t/helper/test-http-server.c +++ b/t/helper/test-http-server.c @@ -18,6 +18,8 @@ static const char test_http_auth_usage[] = " [--timeout=] [--init-timeout=] [--max-connections=]\n" " [--reuseaddr] [--pid-file=]\n" " [--listen=]* [--port=]\n" +" [--anonymous-allowed]\n" +" [--auth=[:] [--auth-token=:]]*\n" ; /* Timeout, and initial timeout */ @@ -358,10 +360,136 @@ static enum worker_result do__git(struct req *req, const char *user) return !!res; } +enum auth_result { + /* No auth module matches the request. */ + AUTH_UNKNOWN = 0, + + /* Auth module denied the request. */ + AUTH_DENY = 1, + + /* Auth module successfully validated the request. */ + AUTH_ALLOW = 2, +}; + +struct auth_module { + char *scheme; + char *challenge_params; + struct string_list *tokens; +}; + +static int allow_anonymous; +static struct auth_module **auth_modules = NULL; +static size_t auth_modules_nr = 0; +static size_t auth_modules_alloc = 0; + +static struct auth_module *get_auth_module(const char *scheme) +{ + int i; + struct auth_module *mod; + for (i = 0; i < auth_modules_nr; i++) { + mod = auth_modules[i]; + if (!strcasecmp(mod->scheme, scheme)) + return mod; + } + + return NULL; +} + +static void add_auth_module(struct auth_module *mod) +{ + ALLOC_GROW(auth_modules, auth_modules_nr + 1, auth_modules_alloc); + auth_modules[auth_modules_nr++] = mod; +} + +static int is_authed(struct req *req, const char **user, enum worker_result *wr) +{ + enum auth_result result = AUTH_UNKNOWN; + struct string_list hdrs = STRING_LIST_INIT_NODUP; + struct auth_module *mod; + + struct string_list_item *hdr; + struct string_list_item *token; + const char *v; + struct strbuf **split = NULL; + int i; + char *challenge; + + /* + * Check all auth modules and try to validate the request. + * The first module that matches a valid token approves the request. + * If no module is found, or if there is no valid token, then 401 error. + * Otherwise, only permit the request if anonymous auth is enabled. + */ + for_each_string_list_item(hdr, &req->header_list) { + if (skip_iprefix(hdr->string, "Authorization: ", &v)) { + split = strbuf_split_str(v, ' ', 2); + if (!split[0] || !split[1]) continue; + + /* trim trailing space ' ' */ + strbuf_setlen(split[0], split[0]->len - 1); + + mod = get_auth_module(split[0]->buf); + if (mod) { + result = AUTH_DENY; + + for_each_string_list_item(token, mod->tokens) { + if (!strcmp(split[1]->buf, token->string)) { + result = AUTH_ALLOW; + break; + } + } + + goto done; + } + } + } + +done: + switch (result) { + case AUTH_ALLOW: + trace2_printf("%s: auth '%s' ALLOW", TR2_CAT, mod->scheme); + *user = "VALID_TEST_USER"; + *wr = WR_OK; + break; + + case AUTH_DENY: + trace2_printf("%s: auth '%s' DENY", TR2_CAT, mod->scheme); + /* fall-through */ + + case AUTH_UNKNOWN: + if (result != AUTH_DENY && allow_anonymous) + break; + for (i = 0; i < auth_modules_nr; i++) { + mod = auth_modules[i]; + if (mod->challenge_params) + challenge = xstrfmt("WWW-Authenticate: %s %s", + mod->scheme, + mod->challenge_params); + else + challenge = xstrfmt("WWW-Authenticate: %s", + mod->scheme); + string_list_append(&hdrs, challenge); + } + *wr = send_http_error(1, 401, "Unauthorized", -1, &hdrs, *wr); + } + + strbuf_list_free(split); + string_list_clear(&hdrs, 0); + + return result == AUTH_ALLOW || + (result == AUTH_UNKNOWN && allow_anonymous); +} + static enum worker_result dispatch(struct req *req) { + enum worker_result wr = WR_OK; + const char *user = NULL; + + if (!is_authed(req, &user, &wr)) + return wr; + if (is_git_request(req)) - return do__git(req, NULL); + return do__git(req, user); return send_http_error(1, 501, "Not Implemented", -1, NULL, WR_OK | WR_HANGUP); @@ -854,6 +982,7 @@ int cmd_main(int argc, const char **argv) struct string_list listen_addr = STRING_LIST_INIT_NODUP; int worker_mode = 0; int i; + struct auth_module *mod = NULL; trace2_cmd_name("test-http-server"); setup_git_directory_gently(NULL); @@ -906,6 +1035,63 @@ int cmd_main(int argc, const char **argv) pid_file = v; continue; } + if (skip_prefix(arg, "--allow-anonymous", &v)) { + allow_anonymous = 1; + continue; + } + if (skip_prefix(arg, "--auth=", &v)) { + struct strbuf **p = strbuf_split_str(v, ':', 2); + + if (!p[0]) { + error("invalid argument '%s'", v); + usage(test_http_auth_usage); + } + + /* trim trailing ':' */ + if (p[1]) + strbuf_setlen(p[0], p[0]->len - 1); + + if (get_auth_module(p[0]->buf)) { + error("duplicate auth scheme '%s'\n", p[0]->buf); + usage(test_http_auth_usage); + } + + mod = xmalloc(sizeof(struct auth_module)); + mod->scheme = xstrdup(p[0]->buf); + mod->challenge_params = p[1] ? xstrdup(p[1]->buf) : NULL; + CALLOC_ARRAY(mod->tokens, 1); + string_list_init_dup(mod->tokens); + + add_auth_module(mod); + + strbuf_list_free(p); + continue; + } + if (skip_prefix(arg, "--auth-token=", &v)) { + struct strbuf **p = strbuf_split_str(v, ':', 2); + if (!p[0]) { + error("invalid argument '%s'", v); + usage(test_http_auth_usage); + } + + if (!p[1]) { + error("missing token value '%s'\n", v); + usage(test_http_auth_usage); + } + + /* trim trailing ':' */ + strbuf_setlen(p[0], p[0]->len - 1); + + mod = get_auth_module(p[0]->buf); + if (!mod) { + error("auth scheme not defined '%s'\n", p[0]->buf); + usage(test_http_auth_usage); + } + + string_list_append(mod->tokens, p[1]->buf); + strbuf_list_free(p); + continue; + } fprintf(stderr, "error: unknown argument '%s'\n", arg); usage(test_http_auth_usage); From patchwork Mon Dec 12 21:36:23 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Matthew John Cheetham X-Patchwork-Id: 13071462 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 27E05C4332F for ; Mon, 12 Dec 2022 21:37:11 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233462AbiLLVhH (ORCPT ); Mon, 12 Dec 2022 16:37:07 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:35402 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S233291AbiLLVgg (ORCPT ); Mon, 12 Dec 2022 16:36:36 -0500 Received: from mail-wm1-x333.google.com (mail-wm1-x333.google.com [IPv6:2a00:1450:4864:20::333]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 85685140C6 for ; Mon, 12 Dec 2022 13:36:35 -0800 (PST) Received: by mail-wm1-x333.google.com with SMTP id bg10so6807205wmb.1 for ; Mon, 12 Dec 2022 13:36:35 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=cc:to:mime-version:content-transfer-encoding:fcc:subject:date:from :references:in-reply-to:message-id:from:to:cc:subject:date :message-id:reply-to; bh=MtFGJSzuLCu7fgCvtvMWmTU5Ym8iaMyfw6igubjubJs=; b=icRxobRd+HQ4IhpTJGe4rHtFUAdIqP60j3ALmoNVcvBqL489vvI4+wsVPLkas6Lkxu 2gWUvK8Qj7oEdjsvOpQ5ElDlUldUKs5iIUmM/2uRmrM2XtslKWyJfYjRQB9bz9OaVH6x ZrhnbKVIeSmAGlPwkjvXEOklVdW061fzgnpxzLJ7auJM/NgrCKbz+/cnvCEK8dOoMmd4 W/I69BWsYroqRWQd0wHM/dLopR2KuL48Ieixmqng/LxIQFovgL2cWGcP8FkQxkgcGUbs p7SmxVDnrZ2YgOsAJ37aUUht6cOk+kdW8hJcC36JU+3SYaiASZcJJjsyEHHFU8JknKVK m+dg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=cc:to:mime-version:content-transfer-encoding:fcc:subject:date:from :references:in-reply-to:message-id:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=MtFGJSzuLCu7fgCvtvMWmTU5Ym8iaMyfw6igubjubJs=; b=w34w5PsNh4MubOfXNgv1Zunipq2ZeD9MY2XVlsqao8h1BL7WlKvNwR0Wfm8PeQr42n hUT31nQkoYWh78XEh1jaIZLVUlb6Oe/UzxqQcY3j74qATgzHoYqH0FoCj2l7s43wjKTq JjXkzuWCkXBadi/husbVs0Cspu9rPlPvHckdZIesuQT602S4XhJx9+Z2Yg+WPdfJpg4C d+GNfpPhC4ccgt5Xd7PF7CcFKmY8F8p+h4Y/+Ax0r6pc3xi2/PsVHY9ysbbNYapExH7z 9k47ClhVs6nKotI7ji7iaBBM1ePJ/xgXQqRWHGqsFIA5BCjRyIQd7rrH+H1RVy1XD1Yh si5g== X-Gm-Message-State: ANoB5plvWlhoyBnsXvFsexM32BlJ7rV0GCiaja7v/qz5YzEt5s/DWUKM fsKsu9lU/qL+gY9lOZLJOA2dbl0AGes= X-Google-Smtp-Source: AA0mqf79HxnflozjepUx43xXE0woc3TVCXp3jIDLzH3oejRXi6AlXpg8Q4oAMlr/n9Ddw3Gex0Tsbw== X-Received: by 2002:a1c:f614:0:b0:3cf:5502:8a0 with SMTP id w20-20020a1cf614000000b003cf550208a0mr13825712wmc.27.1670880993935; Mon, 12 Dec 2022 13:36:33 -0800 (PST) Received: from [127.0.0.1] ([13.74.141.28]) by smtp.gmail.com with ESMTPSA id c66-20020a1c3545000000b003d09150b339sm10745457wma.20.2022.12.12.13.36.33 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 12 Dec 2022 13:36:33 -0800 (PST) Message-Id: <8ecf63835229676677e3f7e33f634eb5d3a568b7.1670880984.git.gitgitgadget@gmail.com> In-Reply-To: References: Date: Mon, 12 Dec 2022 21:36:23 +0000 Subject: [PATCH v4 8/8] t5556: add HTTP authentication tests Fcc: Sent MIME-Version: 1.0 To: git@vger.kernel.org Cc: Derrick Stolee , Lessley Dennington , Matthew John Cheetham , M Hickford , Jeff Hostetler , Glen Choo , Matthew John Cheetham , Matthew John Cheetham Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org From: Matthew John Cheetham From: Matthew John Cheetham Add a series of tests to exercise the HTTP authentication header parsing and the interop with credential helpers. Credential helpers will receive WWW-Authenticate information in credential requests. Signed-off-by: Matthew John Cheetham --- t/helper/test-credential-helper-replay.sh | 14 +++ t/t5556-http-auth.sh | 120 +++++++++++++++++++++- 2 files changed, 133 insertions(+), 1 deletion(-) create mode 100755 t/helper/test-credential-helper-replay.sh diff --git a/t/helper/test-credential-helper-replay.sh b/t/helper/test-credential-helper-replay.sh new file mode 100755 index 00000000000..03e5e63dad6 --- /dev/null +++ b/t/helper/test-credential-helper-replay.sh @@ -0,0 +1,14 @@ +cmd=$1 +teefile=$cmd-actual.cred +catfile=$cmd-response.cred +rm -f $teefile +while read line; +do + if test -z "$line"; then + break; + fi + echo "$line" >> $teefile +done +if test "$cmd" = "get"; then + cat $catfile +fi diff --git a/t/t5556-http-auth.sh b/t/t5556-http-auth.sh index 78da151f122..541fa32bd77 100755 --- a/t/t5556-http-auth.sh +++ b/t/t5556-http-auth.sh @@ -26,6 +26,8 @@ PID_FILE="$(pwd)"/pid-file.pid SERVER_LOG="$(pwd)"/OUT.server.log PATH="$GIT_BUILD_DIR/t/helper/:$PATH" && export PATH +CREDENTIAL_HELPER="$GIT_BUILD_DIR/t/helper/test-credential-helper-replay.sh" \ + && export CREDENTIAL_HELPER test_expect_success 'setup repos' ' test_create_repo "$REPO_DIR" && @@ -91,7 +93,8 @@ start_http_server () { per_test_cleanup () { stop_http_server && - rm -f OUT.* + rm -f OUT.* && + rm -f *.cred } test_expect_success 'http auth anonymous no challenge' ' @@ -102,4 +105,119 @@ test_expect_success 'http auth anonymous no challenge' ' git ls-remote $ORIGIN_URL ' +test_expect_success 'http auth www-auth headers to credential helper basic valid' ' + test_when_finished "per_test_cleanup" && + # base64("alice:secret-passwd") + USERPASS64=YWxpY2U6c2VjcmV0LXBhc3N3ZA== && + export USERPASS64 && + + start_http_server \ + --auth=basic:realm=\"example.com\" \ + --auth-token=basic:$USERPASS64 && + + cat >get-expected.cred <<-EOF && + protocol=http + host=$HOST_PORT + wwwauth[]=basic realm="example.com" + EOF + + cat >store-expected.cred <<-EOF && + protocol=http + host=$HOST_PORT + username=alice + password=secret-passwd + EOF + + cat >get-response.cred <<-EOF && + protocol=http + host=$HOST_PORT + username=alice + password=secret-passwd + EOF + + git -c credential.helper="$CREDENTIAL_HELPER" ls-remote $ORIGIN_URL && + + test_cmp get-expected.cred get-actual.cred && + test_cmp store-expected.cred store-actual.cred +' + +test_expect_success 'http auth www-auth headers to credential helper custom schemes' ' + test_when_finished "per_test_cleanup" && + # base64("alice:secret-passwd") + USERPASS64=YWxpY2U6c2VjcmV0LXBhc3N3ZA== && + export USERPASS64 && + + start_http_server \ + --auth=foobar:alg=test\ widget=1 \ + --auth=bearer:authority=\"id.example.com\"\ q=1\ p=0 \ + --auth=basic:realm=\"example.com\" \ + --auth-token=basic:$USERPASS64 && + + cat >get-expected.cred <<-EOF && + protocol=http + host=$HOST_PORT + wwwauth[]=foobar alg=test widget=1 + wwwauth[]=bearer authority="id.example.com" q=1 p=0 + wwwauth[]=basic realm="example.com" + EOF + + cat >store-expected.cred <<-EOF && + protocol=http + host=$HOST_PORT + username=alice + password=secret-passwd + EOF + + cat >get-response.cred <<-EOF && + protocol=http + host=$HOST_PORT + username=alice + password=secret-passwd + EOF + + git -c credential.helper="$CREDENTIAL_HELPER" ls-remote $ORIGIN_URL && + + test_cmp get-expected.cred get-actual.cred && + test_cmp store-expected.cred store-actual.cred +' + +test_expect_success 'http auth www-auth headers to credential helper invalid' ' + test_when_finished "per_test_cleanup" && + # base64("alice:secret-passwd") + USERPASS64=YWxpY2U6c2VjcmV0LXBhc3N3ZA== && + export USERPASS64 && + start_http_server \ + --auth=bearer:authority=\"id.example.com\"\ q=1\ p=0 \ + --auth=basic:realm=\"example.com\" \ + --auth-token=basic:$USERPASS64 && + + cat >get-expected.cred <<-EOF && + protocol=http + host=$HOST_PORT + wwwauth[]=bearer authority="id.example.com" q=1 p=0 + wwwauth[]=basic realm="example.com" + EOF + + cat >erase-expected.cred <<-EOF && + protocol=http + host=$HOST_PORT + username=alice + password=invalid-passwd + wwwauth[]=bearer authority="id.example.com" q=1 p=0 + wwwauth[]=basic realm="example.com" + EOF + + cat >get-response.cred <<-EOF && + protocol=http + host=$HOST_PORT + username=alice + password=invalid-passwd + EOF + + test_must_fail git -c credential.helper="$CREDENTIAL_HELPER" ls-remote $ORIGIN_URL && + + test_cmp get-expected.cred get-actual.cred && + test_cmp erase-expected.cred erase-actual.cred +' + test_done