Message ID | pull.1474.v2.git.git.1679327330032.gitgitgadget@gmail.com (mailing list archive) |
---|---|
State | Accepted |
Commit | 0a01d41ee4ca7f8afb75219f46f4f1c573465075 |
Headers | show |
Series | [v2] http: add support for different sslcert and sslkey types. | expand |
On Mon, Mar 20, 2023 at 03:48:49PM +0000, Stanislav Malishevskiy via GitGitGadget wrote: > From: Stanislav Malishevskiy <s.malishevskiy@auriga.com> > > Basically git work with default curl ssl type - PEM. But for support > eTokens like SafeNet tokens via pksc11 need setup 'ENG' as sslcert type > and as sslkey type. So there added additional options for http to make > that possible. Seems like a reasonable thing to want, and the patch looks pretty clean. Two small points: > http.c | 12 ++++++++++++ > 1 file changed, 12 insertions(+) There are no tests here. I think it might be possible to add them, but our https test support is currently optional, and has bit-rotted a bit over the years. There's some discussion here: https://lore.kernel.org/git/Y9s7vyHKXP+TQPRm@pobox.com/ So I don't think it makes sense to block this patch over the lack of tests, but it's something we might keep in mind to add if the test situation improves. > @@ -1014,10 +1020,14 @@ static CURL *get_curl_handle(void) > > if (ssl_cert) > curl_easy_setopt(result, CURLOPT_SSLCERT, ssl_cert); > + if (ssl_cert_type) > + curl_easy_setopt(result, CURLOPT_SSLCERTTYPE, ssl_cert_type); We're just feeding curl whatever string the user gave us (which is good, since we don't know which ones are valid). But what happens with: GIT_SSL_CERT_TYPE=bogus git fetch ... Should we check for an error here, or will the actual request later complain properly? -Peff
"Stanislav Malishevskiy via GitGitGadget" <gitgitgadget@gmail.com> writes: > From: Stanislav Malishevskiy <s.malishevskiy@auriga.com> > > Basically git work with default curl ssl type - PEM. But for support > eTokens like SafeNet tokens via pksc11 need setup 'ENG' as sslcert type > and as sslkey type. So there added additional options for http to make > that possible. > > Signed-off-by: Stanislav Malishevskiy <stanislav.malishevskiy@gmail.com> > --- > http.c | 12 ++++++++++++ > 1 file changed, 12 insertions(+) Thanks. Is this something that can be protected from future breakage with a few new tests somewhere in t/t5559 (which may also involve touching t/lib-httpd.sh as well)?
> > @@ -1014,10 +1020,14 @@ static CURL *get_curl_handle(void) > > > > if (ssl_cert) > > curl_easy_setopt(result, CURLOPT_SSLCERT, ssl_cert); > > + if (ssl_cert_type) > > + curl_easy_setopt(result, CURLOPT_SSLCERTTYPE, ssl_cert_type); > > We're just feeding curl whatever string the user gave us (which is good, > since we don't know which ones are valid). But what happens with: > > GIT_SSL_CERT_TYPE=bogus git fetch ... > > Should we check for an error here, or will the actual request later > complain properly? Curl itself validates that string. And if we pass the wrong type or not pass 'ENG' in case of pkcs11: curl will return an error. In that case git do the same if GIT_SSL_CERT passed wrong ss 'ENG' in case of pkcs11: curl will return an error. In that case git do the same if GIT_SSL_CERT passed wrong пн, 20 мар. 2023 г. в 20:10, Jeff King <peff@peff.net>: > > On Mon, Mar 20, 2023 at 03:48:49PM +0000, Stanislav Malishevskiy via GitGitGadget wrote: > > > From: Stanislav Malishevskiy <s.malishevskiy@auriga.com> > > > > Basically git work with default curl ssl type - PEM. But for support > > eTokens like SafeNet tokens via pksc11 need setup 'ENG' as sslcert type > > and as sslkey type. So there added additional options for http to make > > that possible. > > Seems like a reasonable thing to want, and the patch looks pretty clean. > Two small points: > > > http.c | 12 ++++++++++++ > > 1 file changed, 12 insertions(+) > > There are no tests here. I think it might be possible to add them, but > our https test support is currently optional, and has bit-rotted a bit > over the years. There's some discussion here: > > https://lore.kernel.org/git/Y9s7vyHKXP+TQPRm@pobox.com/ > > So I don't think it makes sense to block this patch over the lack of > tests, but it's something we might keep in mind to add if the test > situation improves. > > > @@ -1014,10 +1020,14 @@ static CURL *get_curl_handle(void) > > > > if (ssl_cert) > > curl_easy_setopt(result, CURLOPT_SSLCERT, ssl_cert); > > + if (ssl_cert_type) > > + curl_easy_setopt(result, CURLOPT_SSLCERTTYPE, ssl_cert_type); > > We're just feeding curl whatever string the user gave us (which is good, > since we don't know which ones are valid). But what happens with: > > GIT_SSL_CERT_TYPE=bogus git fetch ... > > Should we check for an error here, or will the actual request later > complain properly? > > -Peff
> Is this something that can be protected from future breakage with a > few new tests somewhere in t/t5559 (which may also involve touching > t/lib-httpd.sh as well)? I am not sure about that. That required only for local access to the cert and key from curl пн, 20 мар. 2023 г. в 20:23, Junio C Hamano <gitster@pobox.com>: > > "Stanislav Malishevskiy via GitGitGadget" <gitgitgadget@gmail.com> > writes: > > > From: Stanislav Malishevskiy <s.malishevskiy@auriga.com> > > > > Basically git work with default curl ssl type - PEM. But for support > > eTokens like SafeNet tokens via pksc11 need setup 'ENG' as sslcert type > > and as sslkey type. So there added additional options for http to make > > that possible. > > > > Signed-off-by: Stanislav Malishevskiy <stanislav.malishevskiy@gmail.com> > > --- > > > http.c | 12 ++++++++++++ > > 1 file changed, 12 insertions(+) > > Thanks. > > Is this something that can be protected from future breakage with a > few new tests somewhere in t/t5559 (which may also involve touching > t/lib-httpd.sh as well)?
On Mon, Mar 20, 2023 at 10:23:41AM -0700, Junio C Hamano wrote: > Is this something that can be protected from future breakage with a > few new tests somewhere in t/t5559 (which may also involve touching > t/lib-httpd.sh as well)? I don't think we'd want to put it there. While it is the only test that requires SSL currently, it's also specific to HTTP/2, so it may not get run everywhere. The original design of the SSL support in the test suite was that you'd do a run of the whole suite with LIB_HTTPD_SSL set, and then it would run all of the usual tests with ssl. But experience has shown that nobody does that. So I think there are two paths forward here: 1. Add a mode to CI that runs with LIB_HTTPD_SSL set. We'd need to fix the bit-rotted tests that fail (usually due to things like expecting "http" in the output instead of "https", etc). I linked to earlier discussion there elsewhere in the thread. 2. Add a specific "test with https" script that covers some basic tests (possibly even just including t5551, in the same way t5559 does). If the platform apache doesn't support ssl, then it should fail gracefully. And then we could add more SSL specific tests to that script. -Peff
On Mon, Mar 20, 2023 at 09:21:44PM +0300, Stanislav M wrote: > > > @@ -1014,10 +1020,14 @@ static CURL *get_curl_handle(void) > > > > > > if (ssl_cert) > > > curl_easy_setopt(result, CURLOPT_SSLCERT, ssl_cert); > > > + if (ssl_cert_type) > > > + curl_easy_setopt(result, CURLOPT_SSLCERTTYPE, ssl_cert_type); > > > > We're just feeding curl whatever string the user gave us (which is good, > > since we don't know which ones are valid). But what happens with: > > > > GIT_SSL_CERT_TYPE=bogus git fetch ... > > > > Should we check for an error here, or will the actual request later > > complain properly? > > Curl itself validates that string. And if we pass the wrong type or > not pass 'ENG' in case of pkcs11: curl will return an error. In that > case git do the same if GIT_SSL_CERT passed wrong ss 'ENG' in case of > pkcs11: curl will return an error. In that case git do the same if > GIT_SSL_CERT passed wrong That sounds great. Thanks for confirming! -Peff
Jeff King <peff@peff.net> writes: > 2. Add a specific "test with https" script that covers some basic > tests (possibly even just including t5551, in the same way t5559 > does). If the platform apache doesn't support ssl, then it should > fail gracefully. And then we could add more SSL specific tests > to that script. Both choices sound sensible. This second one may be simpler to arrange. Thanks.
I want to do some clarification there. Those changes are not related to http or https directly. That only configures curl to know how to access the certificate and key locally on the client box, so if the way is wrong there is a simple error: Certificate or key not found. вт, 21 мар. 2023 г. в 20:43, Junio C Hamano <gitster@pobox.com>: > > Jeff King <peff@peff.net> writes: > > > 2. Add a specific "test with https" script that covers some basic > > tests (possibly even just including t5551, in the same way t5559 > > does). If the platform apache doesn't support ssl, then it should > > fail gracefully. And then we could add more SSL specific tests > > to that script. > > Both choices sound sensible. This second one may be simpler to > arrange. > > Thanks.
On Thu, Mar 23, 2023 at 12:33:59PM +0300, Stanislav M wrote: > I want to do some clarification there. Those changes are not related > to http or https directly. > That only configures curl to know how to access the certificate and > key locally on the client box, so if the way is wrong there is a > simple error: Certificate or key not found. Yes, but I'm not sure if there is a way for Git to trigger curl to look at the certificate that does not involve feeding it an https URL (and we want a valid one, because we want to see that it correctly speaks to the server). -Peff
In my opinion they need the same set of tests which is used as usual for https. But use the right certificate and key. But I don't have any idea how to do that with hardware usb eToken in my case. чт, 23 мар. 2023 г. в 21:02, Jeff King <peff@peff.net>: > > On Thu, Mar 23, 2023 at 12:33:59PM +0300, Stanislav M wrote: > > > I want to do some clarification there. Those changes are not related > > to http or https directly. > > That only configures curl to know how to access the certificate and > > key locally on the client box, so if the way is wrong there is a > > simple error: Certificate or key not found. > > Yes, but I'm not sure if there is a way for Git to trigger curl to look > at the certificate that does not involve feeding it an https URL (and we > want a valid one, because we want to see that it correctly speaks to the > server). > > -Peff
Stanislav M <stanislav.malishevskiy@gmail.com> writes: [administrivia: do not top-post] >> Yes, but I'm not sure if there is a way for Git to trigger curl to look >> at the certificate that does not involve feeding it an https URL (and we >> want a valid one, because we want to see that it correctly speaks to the >> server). > ... > In my opinion they need the same set of tests which is used as usual > for https. But use the right certificate and key. > But I don't have any idea how to do that with hardware usb eToken in my case. OK, so where does this put us, with respect to the change? We have some behaviour change that we do not know how to test? How would we know when we break it in the future? It is not like the new feature is not useful enough that nobody would care if it gets broken by accident or anything like that, so...? At least perhaps we can throw bogus strings in the environment and make sure cURL library gives complaints, or something? Thanks.
Yes. If you set bogus strings in the environment cURL should return an error the same as if you set the wrong file for certificate or key. So you can set GIT_SSL_CERT=some_real_pem_file - That should work (PEM type used by default) GIT_SSL_CERT=some_real_pem_file GIT_SSL_CERT_TYPE=PEM - That should work too GIT_SSL_CERT=some_real_pem_file GIT_SSL_CERT_TYPE=Bogus - That shouldn't work GIT_SSL_CERT=some_real_der_file GIT_SSL_CERT_TYPE=DER - I am not sure about that, because as I far remember there issue with DER in openssl I think that more detailed information there: https://curl.se/libcurl/c/CURLOPT_SSLKEYTYPE.html Basically that only a format of cert and key file or engine in case of pkcs11 url instead of file in others cases. So if you set it into right values, respect your ssl cert and ssl key - https should work. But if not, error from curl should returned ср, 29 мар. 2023 г. в 21:53, Junio C Hamano <gitster@pobox.com>: > > Stanislav M <stanislav.malishevskiy@gmail.com> writes: > > [administrivia: do not top-post] > > >> Yes, but I'm not sure if there is a way for Git to trigger curl to look > >> at the certificate that does not involve feeding it an https URL (and we > >> want a valid one, because we want to see that it correctly speaks to the > >> server). > > ... > > In my opinion they need the same set of tests which is used as usual > > for https. But use the right certificate and key. > > But I don't have any idea how to do that with hardware usb eToken in my case. > > OK, so where does this put us, with respect to the change? We have > some behaviour change that we do not know how to test? How would we > know when we break it in the future? It is not like the new feature > is not useful enough that nobody would care if it gets broken by > accident or anything like that, so...? > > At least perhaps we can throw bogus strings in the environment and > make sure cURL library gives complaints, or something? > > Thanks.
On Wed, Mar 29, 2023 at 11:53:11AM -0700, Junio C Hamano wrote: > > In my opinion they need the same set of tests which is used as usual > > for https. But use the right certificate and key. > > But I don't have any idea how to do that with hardware usb eToken in my case. > > OK, so where does this put us, with respect to the change? We have > some behaviour change that we do not know how to test? How would we > know when we break it in the future? It is not like the new feature > is not useful enough that nobody would care if it gets broken by > accident or anything like that, so...? > > At least perhaps we can throw bogus strings in the environment and > make sure cURL library gives complaints, or something? I would be OK taking the patches without any further tests. It is not really making anything worse in the sense that we already do not test any of the client-cert stuff. If we can cheaply add some tests that at least exercise the code and hand off to curl, that is better than nothing, I guess. I think the ideal would be a new t5565 that sets LIB_HTTPD_SSL unconditionally and actually tests various client-cert formats and requests using a made-on-the-fly cert. But I don't want to hold up a patch we otherwise think is OK on the basis of long-term technical debt we already have. -Peff
Jeff King <peff@peff.net> writes: > On Wed, Mar 29, 2023 at 11:53:11AM -0700, Junio C Hamano wrote: > > ... >> > But I don't have any idea how to do that with hardware usb eToken in my case. >> >> OK, so where does this put us, with respect to the change? ... > I would be OK taking the patches without any further tests. It is not > really making anything worse in the sense that we already do not test > any of the client-cert stuff. Alright. I think the patch has already been queued for some time, and it is OK to merge it down to 'next'. > If we can cheaply add some tests that at least exercise the code and > hand off to curl, that is better than nothing, I guess. > > I think the ideal would be a new t5565 that sets LIB_HTTPD_SSL > unconditionally and actually tests various client-cert formats and > requests using a made-on-the-fly cert. Thanks.
diff --git a/http.c b/http.c index dbe4d29ef7a..d5d82c5230f 100644 --- a/http.c +++ b/http.c @@ -40,6 +40,7 @@ static int curl_ssl_verify = -1; static int curl_ssl_try; static const char *curl_http_version = NULL; static const char *ssl_cert; +static const char *ssl_cert_type; static const char *ssl_cipherlist; static const char *ssl_version; static struct { @@ -59,6 +60,7 @@ static struct { #endif }; static const char *ssl_key; +static const char *ssl_key_type; static const char *ssl_capath; static const char *curl_no_proxy; #ifdef GIT_CURL_HAVE_CURLOPT_PINNEDPUBLICKEY @@ -374,8 +376,12 @@ static int http_options(const char *var, const char *value, void *cb) return git_config_string(&ssl_version, var, value); if (!strcmp("http.sslcert", var)) return git_config_pathname(&ssl_cert, var, value); + if (!strcmp("http.sslcerttype", var)) + return git_config_string(&ssl_cert_type, var, value); if (!strcmp("http.sslkey", var)) return git_config_pathname(&ssl_key, var, value); + if (!strcmp("http.sslkeytype", var)) + return git_config_string(&ssl_key_type, var, value); if (!strcmp("http.sslcapath", var)) return git_config_pathname(&ssl_capath, var, value); if (!strcmp("http.sslcainfo", var)) @@ -1014,10 +1020,14 @@ static CURL *get_curl_handle(void) if (ssl_cert) curl_easy_setopt(result, CURLOPT_SSLCERT, ssl_cert); + if (ssl_cert_type) + curl_easy_setopt(result, CURLOPT_SSLCERTTYPE, ssl_cert_type); if (has_cert_password()) curl_easy_setopt(result, CURLOPT_KEYPASSWD, cert_auth.password); if (ssl_key) curl_easy_setopt(result, CURLOPT_SSLKEY, ssl_key); + if (ssl_key_type) + curl_easy_setopt(result, CURLOPT_SSLKEYTYPE, ssl_key_type); if (ssl_capath) curl_easy_setopt(result, CURLOPT_CAPATH, ssl_capath); #ifdef GIT_CURL_HAVE_CURLOPT_PINNEDPUBLICKEY @@ -1252,7 +1262,9 @@ void http_init(struct remote *remote, const char *url, int proactive_auth) curl_ssl_verify = 0; set_from_env(&ssl_cert, "GIT_SSL_CERT"); + set_from_env(&ssl_cert_type, "GIT_SSL_CERT_TYPE"); set_from_env(&ssl_key, "GIT_SSL_KEY"); + set_from_env(&ssl_key_type, "GIT_SSL_KEY_TYPE"); set_from_env(&ssl_capath, "GIT_SSL_CAPATH"); set_from_env(&ssl_cainfo, "GIT_SSL_CAINFO");