From patchwork Tue Sep 13 19:25:49 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Matthew John Cheetham X-Patchwork-Id: 12975179 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 0139EC6FA82 for ; Tue, 13 Sep 2022 19:26:07 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229484AbiIMT0G (ORCPT ); Tue, 13 Sep 2022 15:26:06 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:50196 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229436AbiIMT0B (ORCPT ); Tue, 13 Sep 2022 15:26:01 -0400 Received: from mail-wr1-x429.google.com (mail-wr1-x429.google.com [IPv6:2a00:1450:4864:20::429]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id B6911726A9 for ; Tue, 13 Sep 2022 12:26:00 -0700 (PDT) Received: by mail-wr1-x429.google.com with SMTP id cc5so12531188wrb.6 for ; Tue, 13 Sep 2022 12:26:00 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=cc:to:mime-version:content-transfer-encoding:fcc:subject:date:from :references:in-reply-to:message-id:from:to:cc:subject:date; bh=nux1mDXV6UAP3b2Yui30zaM88WL6NE924cP9WN28Wz4=; b=KVNa+ziCJpWIbhXYmqBa0yFAYr/PMC2J7FubRNXq2VpZhkZ4gyRudZkn2iUZBANbIZ eaKPX/XhsUVPZCJG2Kz14WGXa1CFJiAHAZqX77ZnBoLAUQmASKRak0ZrUPxjlqkPsp+x 4fJ2U6Waa4A/TDAn6MXueCUcOnQfLJuKBdCA5HExoangs3xYRmRkzS8wBGrm1lFNKcLW RHSwbEzmg3/GXEzSb3twg/wfYAYCLo8FofJQIFeiVIatm5HUJF8fREHx4UYOPu5mpl+3 cB9cvT9KfqPJFToqBZS6F/OmPwfVZjtHjZmqqhJ1xiy0eokGcfoB7dkDZ3TGgKtQ5S15 eaCQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=cc:to:mime-version:content-transfer-encoding:fcc:subject:date:from :references:in-reply-to:message-id:x-gm-message-state:from:to:cc :subject:date; bh=nux1mDXV6UAP3b2Yui30zaM88WL6NE924cP9WN28Wz4=; b=Ya1sB11+5dPyccXzB4HgbOUQ2+0UOQpt6VI8IRGsYIqiSva2IfLjHL0rYD2xlPEwRY 5ETHE/HhBIw8JFDaB1yJQD49vKgOhUapRh+2kT1W/6USdl36QIseUE1euexIb5nKbmb6 M2wf4jHme4HSnU60/q7VPAHWYwA+6SEWpBGBcRKPtc11EI0DqXLiXviDl+iMKyOmRpvx Ri/BaCC7W/Kn/NLbPbHOBw8cI7kAl/rqPNShNRQBMZNzui4eQedvdob1IKLTyhyVVZCF uKN28Q2qPLaTePEA48e+XZzc83ZZhBmPSmvHb5s/peuSo5B3bxwVb2MUFK0Qx6X0BdMN pbFQ== X-Gm-Message-State: ACgBeo1ouNlvBNeGqjjfPzP/FNj5TD9bwEbFBOygiREEMJJh0Fau+xha rKoSWTa43sGVl58JBAtsTt+QwKn8WtM= X-Google-Smtp-Source: AA6agR6A76rThjpUken2JGLk57sc1KXbhnvzTuSMGZJSnPWAkjVrb0ZWPEkRuemGEBl5cYNomrVWcg== X-Received: by 2002:a5d:58da:0:b0:22a:c3cb:e3cb with SMTP id o26-20020a5d58da000000b0022ac3cbe3cbmr1563941wrf.34.1663097158966; Tue, 13 Sep 2022 12:25:58 -0700 (PDT) Received: from [127.0.0.1] ([13.74.141.28]) by smtp.gmail.com with ESMTPSA id r15-20020a05600c35cf00b003a4f08495b7sm15329742wmq.34.2022.09.13.12.25.58 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 13 Sep 2022 12:25:58 -0700 (PDT) Message-Id: <6426f9c3954866b3fd9259d1a58d2c41dc42e17f.1663097156.git.gitgitgadget@gmail.com> In-Reply-To: References: Date: Tue, 13 Sep 2022 19:25:49 +0000 Subject: [PATCH 1/8] wincred: ignore unknown lines (do not die) Fcc: Sent MIME-Version: 1.0 To: git@vger.kernel.org Cc: Matthew John Cheetham , Matthew John Cheetham Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org From: Matthew John Cheetham From: Matthew John Cheetham It is the expectation that credential helpers be liberal in what they accept and conservative in what they return, to allow for future growth and evolution of the protocol/interaction. All of the other helpers (store, cache, osxkeychain, libsecret, gnome-keyring) except `netrc` currently ignore any credential lines that are not recognised, whereas the Windows helper (wincred) instead dies. Fix the discrepancy and ignore unknown lines in the wincred helper. Signed-off-by: Matthew John Cheetham --- contrib/credential/wincred/git-credential-wincred.c | 7 +++++-- 1 file changed, 5 insertions(+), 2 deletions(-) diff --git a/contrib/credential/wincred/git-credential-wincred.c b/contrib/credential/wincred/git-credential-wincred.c index 5091048f9c6..ead6e267c78 100644 --- a/contrib/credential/wincred/git-credential-wincred.c +++ b/contrib/credential/wincred/git-credential-wincred.c @@ -278,8 +278,11 @@ static void read_credential(void) wusername = utf8_to_utf16_dup(v); } else if (!strcmp(buf, "password")) password = utf8_to_utf16_dup(v); - else - die("unrecognized input"); + /* + * Ignore other lines; we don't know what they mean, but + * this future-proofs us when later versions of git do + * learn new lines, and the helpers are updated to match. + */ } } From patchwork Tue Sep 13 19:25:50 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Matthew John Cheetham X-Patchwork-Id: 12975180 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 930C1ECAAD8 for ; Tue, 13 Sep 2022 19:26:09 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229489AbiIMT0H (ORCPT ); Tue, 13 Sep 2022 15:26:07 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:50226 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229449AbiIMT0C (ORCPT ); Tue, 13 Sep 2022 15:26:02 -0400 Received: from mail-wm1-x32c.google.com (mail-wm1-x32c.google.com [IPv6:2a00:1450:4864:20::32c]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id C52FE7962B for ; Tue, 13 Sep 2022 12:26:01 -0700 (PDT) Received: by mail-wm1-x32c.google.com with SMTP id r66-20020a1c4445000000b003b494ffc00bso1581951wma.0 for ; Tue, 13 Sep 2022 12:26:01 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=cc:to:mime-version:content-transfer-encoding:fcc:subject:date:from :references:in-reply-to:message-id:from:to:cc:subject:date; bh=+Gm0KiPEcdtvw2UZkAcrYNiFoXQRkfYRnk7dWz7jZJA=; b=YB910xYMB1acQe9deX3qPHIfWqM0o+JPfY0qU+u84szHEmI9l4tlhuaaYtNsdgD0f/ ChIJdz+0FkYmsgfz7dmZBO82bn5hhB1tM9wfC4FLuLiLZdpddxmS093XkIbzx7arJfc2 DQIpWsaPTPL+sU0GNikJK9F3CM43ydp07lEuQ+rueD5hlixMsOP8SC0D1QppngHDXRaT NNBIuAdG9x9nUF2sop+UqP3/XjKG811mTHZ5z0Rn3ydyFXFUvoZbGtvJ54t5n9lvgp64 iSy9kQm1wu3lMt0vfwaFtGa3QOruFkYeawi5rcYJlU6+2bI8/Ls+Os+nhVunQ0h7R1hZ GB7g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=cc:to:mime-version:content-transfer-encoding:fcc:subject:date:from :references:in-reply-to:message-id:x-gm-message-state:from:to:cc :subject:date; bh=+Gm0KiPEcdtvw2UZkAcrYNiFoXQRkfYRnk7dWz7jZJA=; b=TvqhuqerC/3mAxa4N71hwdAj2l4GQ+h6Hv+BOLfZ0re5uIFCkT4n6w1cGO00L//Oo7 LtjFNWT8YQDHVu6Sq8vqnT57kRdfwTresclS+KszD4SP27u1e8t7BUsms3ljbW/ry/ps S3/YcuKlV0/foVowsnSNOD5g13+723Up/ZBYwRTb6FWWw6OZ1cdLvuEUBwiut6UP+952 fljR9jBcVEyqWwKa5T12N+6mvQRvMGFGsg3DoqSCTt4tXieIa9isKKE1MMlUmsv1DuzP rUwOmznBKLmhlchpa4AzL2MEO8tDNQDlpL3PF2LhjVu0mstz9aIrfMqLbrOiZkOx3gls 10yw== X-Gm-Message-State: ACgBeo0EXgW2XM0nPDS7l3OA2D0XsfhjTpJemR4XVDlttf8Y1tWDy4WN IYGCmqLWeP0NEYRgE6x30BAGkVc2ix8= X-Google-Smtp-Source: AA6agR7oFomqg1OJD0aVKBjKS8bNDyBHm9OFeOfX/y23+wVj08E2lOdqNw5BxjRaVu3OQHuf72UBCA== X-Received: by 2002:a7b:c5d3:0:b0:3a8:3d5f:4562 with SMTP id n19-20020a7bc5d3000000b003a83d5f4562mr633152wmk.78.1663097159965; Tue, 13 Sep 2022 12:25:59 -0700 (PDT) Received: from [127.0.0.1] ([13.74.141.28]) by smtp.gmail.com with ESMTPSA id d10-20020a056000114a00b002253604bbefsm11775322wrx.75.2022.09.13.12.25.59 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 13 Sep 2022 12:25:59 -0700 (PDT) Message-Id: In-Reply-To: References: Date: Tue, 13 Sep 2022 19:25:50 +0000 Subject: [PATCH 2/8] netrc: ignore unknown lines (do not die) Fcc: Sent MIME-Version: 1.0 To: git@vger.kernel.org Cc: Matthew John Cheetham , Matthew John Cheetham Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org From: Matthew John Cheetham From: Matthew John Cheetham Contrary to the documentation on credential helpers, as well as the help text for git-credential-netrc itself, this helper will `die` when presented with an unknown property/attribute/token. Correct the behaviour here by skipping and ignoring any tokens that are unknown. This means all helpers in the tree are consistent and ignore any unknown credential properties/attributes. Signed-off-by: Matthew John Cheetham --- contrib/credential/netrc/git-credential-netrc.perl | 5 ++++- 1 file changed, 4 insertions(+), 1 deletion(-) diff --git a/contrib/credential/netrc/git-credential-netrc.perl b/contrib/credential/netrc/git-credential-netrc.perl index bc57cc65884..9fb998ae090 100755 --- a/contrib/credential/netrc/git-credential-netrc.perl +++ b/contrib/credential/netrc/git-credential-netrc.perl @@ -356,7 +356,10 @@ sub read_credential_data_from_stdin { next unless m/^([^=]+)=(.+)/; my ($token, $value) = ($1, $2); - die "Unknown search token $token" unless exists $q{$token}; + + # skip any unknown tokens + next unless exists $q{$token}; + $q{$token} = $value; log_debug("We were given search token $token and value $value"); } From patchwork Tue Sep 13 19:25:51 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Matthew John Cheetham X-Patchwork-Id: 12975181 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id DC795ECAAD8 for ; Tue, 13 Sep 2022 19:26:12 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229502AbiIMT0L (ORCPT ); Tue, 13 Sep 2022 15:26:11 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:50250 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229459AbiIMT0D (ORCPT ); Tue, 13 Sep 2022 15:26:03 -0400 Received: from mail-wm1-x330.google.com (mail-wm1-x330.google.com [IPv6:2a00:1450:4864:20::330]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 6C8BE7287F for ; Tue, 13 Sep 2022 12:26:02 -0700 (PDT) Received: by mail-wm1-x330.google.com with SMTP id n8so2610009wmr.5 for ; Tue, 13 Sep 2022 12:26:02 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=cc:to:mime-version:content-transfer-encoding:fcc:subject:date:from :references:in-reply-to:message-id:from:to:cc:subject:date; bh=+f6YTqMKgo/XauQ5HVwJ9Cjt9MJOXkiqZPvZQkkmKdM=; b=mspgyTb+hcDL9njD9Uk0Hvon3R7vpTiJi6+fRcUtZBDtZko3AibySE8OM8vmFaheJw 4tW+b6XVPh0+sF4/8JQohc3QmnPkYdd7wn0fJKihwa7Y00O3DhVAvjiBMtNXnmz75Unx 4o+6ssBJzjjDazmxxsQFN8qU0gQ5upmSnMPNcb11hJwwOPJeuyNsdyjtFiykFL6cRtPg xdXMsJ4IE3O01fc2Netzf5C4pu55gMKdHSU+uEJru/En5mKI+oS6fd1GeFDnmeJwleXY BumGZGzeCFbMEiRftIIofIpt8y1+i2WH4KZ5fVl2Pt6ra0fD3Ap3ib+rWrtH7SYyo1Ja 6wyw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=cc:to:mime-version:content-transfer-encoding:fcc:subject:date:from :references:in-reply-to:message-id:x-gm-message-state:from:to:cc :subject:date; bh=+f6YTqMKgo/XauQ5HVwJ9Cjt9MJOXkiqZPvZQkkmKdM=; b=d6Pfcdkjrn6El3iZdN4lfW8M+WHFuQX6TmQsPhudyAkJw2en1pmgxwfTB6o3cRxzDS 2tomJpj08Fw9V8XaHQ5OFlGWHfClQ9n8RNPLwiAHs31nHe61nDHvrZGfYTEfRdAnZju2 /3x1W8BUl2woWebfmgjx2HuSdq5pLDJGU6mVOk6ZAJewCW3t9l6r0a7KbCZHVY+W0kIO U2pch6igN2R9Hwgne7d+A45d6RjUYqce+8M2WIJptjBenq76742Iah3v0HBBMR5CiY7c IuSv0D2fwUZBFhR3SUMbwFV/L48fRdD2OP6mdKjNcOKUlYRD3RyIWwwDSJ7vkdXmM9vY TdpA== X-Gm-Message-State: ACgBeo2SKl4+yZxSyX+enPXVK9HxS/NwDgSZLEw291a/V3iYrtfjoOh0 0YwFZiEP4wXFsvG+Epud8Ugt7q83POA= X-Google-Smtp-Source: AA6agR7JuMNaHyN26xZtCiYNzs+RGqfc/a5fzRjjA+c2lrSdS7oi96N2Ka2g5O6HfEqf3f3C1oyVFw== X-Received: by 2002:a05:600c:4f10:b0:3a5:f8c8:a5b5 with SMTP id l16-20020a05600c4f1000b003a5f8c8a5b5mr632365wmq.34.1663097160811; Tue, 13 Sep 2022 12:26:00 -0700 (PDT) Received: from [127.0.0.1] ([13.74.141.28]) by smtp.gmail.com with ESMTPSA id y25-20020a05600c365900b003b483000583sm8977817wmq.48.2022.09.13.12.26.00 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 13 Sep 2022 12:26:00 -0700 (PDT) Message-Id: <2ece562a5952b5752de5040b17ec36076164c72f.1663097156.git.gitgitgadget@gmail.com> In-Reply-To: References: Date: Tue, 13 Sep 2022 19:25:51 +0000 Subject: [PATCH 3/8] osxkeychain: clarify that we ignore unknown lines Fcc: Sent MIME-Version: 1.0 To: git@vger.kernel.org Cc: Matthew John Cheetham , Matthew John Cheetham Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org From: Matthew John Cheetham From: Matthew John Cheetham Like in all the other credential helpers, the osxkeychain helper ignores unknown credential lines. Add a comment (a la the other helpers) to make it clear and explicit that this is the desired behaviour. Signed-off-by: Matthew John Cheetham --- contrib/credential/osxkeychain/git-credential-osxkeychain.c | 5 +++++ 1 file changed, 5 insertions(+) diff --git a/contrib/credential/osxkeychain/git-credential-osxkeychain.c b/contrib/credential/osxkeychain/git-credential-osxkeychain.c index bf77748d602..e29cc28779d 100644 --- a/contrib/credential/osxkeychain/git-credential-osxkeychain.c +++ b/contrib/credential/osxkeychain/git-credential-osxkeychain.c @@ -159,6 +159,11 @@ static void read_credential(void) username = xstrdup(v); else if (!strcmp(buf, "password")) password = xstrdup(v); + /* + * Ignore other lines; we don't know what they mean, but + * this future-proofs us when later versions of git do + * learn new lines, and the helpers are updated to match. + */ } } From patchwork Tue Sep 13 19:25:52 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Matthew John Cheetham X-Patchwork-Id: 12975183 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 91E75ECAAD8 for ; Tue, 13 Sep 2022 19:26:20 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229513AbiIMT0T (ORCPT ); Tue, 13 Sep 2022 15:26:19 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:50350 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229472AbiIMT0G (ORCPT ); Tue, 13 Sep 2022 15:26:06 -0400 Received: from mail-wr1-x42c.google.com (mail-wr1-x42c.google.com [IPv6:2a00:1450:4864:20::42c]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 61FF979634 for ; Tue, 13 Sep 2022 12:26:03 -0700 (PDT) Received: by mail-wr1-x42c.google.com with SMTP id h8so15386870wrf.3 for ; Tue, 13 Sep 2022 12:26:03 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=cc:to:mime-version:content-transfer-encoding:fcc:subject:date:from :references:in-reply-to:message-id:from:to:cc:subject:date; bh=9m3T1Nih1NsVSAkhHW78RmCfreuv3OwtdZEGKLpv2Q8=; b=gRJngYN3XSR2qRtHhq/tMq8xd0mCgbWQbGG3OTxr1xcJ8U+wkqpmlvnk/9RW96Tc68 UjuuTz1k2dExiZBYfF1Xnze6Q8IGun84B0JNqHLwWyAYq4+o84s13WvRbYvjPOJHswAc 1Xl+P9++RHEflAnzX9L8JnAMLFuWsibbEsxSbkqRTp+Cq4HuPN7YukNyEvwixP0n+G64 4pq+ll37VVqSNrMPqsFdxA2YYlE9hz5J/jWJM3pgxiqxrjXhpfViN8kbGKfegdJTZVB4 3I2kTpRaCoKJ7zlM1LEJVH9DQbGyBf9IqT4/BH8TjsFhNCGCcHt7NF6PnPb2tSP/VaZX jvGw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=cc:to:mime-version:content-transfer-encoding:fcc:subject:date:from :references:in-reply-to:message-id:x-gm-message-state:from:to:cc :subject:date; bh=9m3T1Nih1NsVSAkhHW78RmCfreuv3OwtdZEGKLpv2Q8=; b=af5fwBTX84y6Wpp6cJ9Y+BxCy4FrM6cOQWRMegv+bVXsiCELah9GK8gHMH8yskgo33 t/CbonXpaJAotJShUF4p7d7KpOZFSHVQRCTVCoq8C674n2hm2CxhvcZ8FbQ7Xqk/LBDc C8etvhORbllK9cNHsR7sjqhkkfrkZ0pc+gD+QOSk23sL6RVFa6SyJ5CkzAsfAurcYXSd xMRRLwvq6crd+0hyNFR9p8DXGo7H7TpncDUWQ7vY9T8c34z/2FpbhWP2hMLI9UFplQ6i p0tG143GK4BjXmg6e6DD2MsEeqAF4Q/onhLPV2X7/2uBK7VWg10DbskaLqSu3b7JgBsg /NiQ== X-Gm-Message-State: ACgBeo2Coz7dSqE9HYA07yyvVsB0yMMF92wn+KXDfkE25UA03JO1wZ62 nmpLl2Y+wENr5X1AxQtMdv7uFivD+jM= X-Google-Smtp-Source: AA6agR5E7U9bOzJvDwnIQ8kvXpXg8hDh0mZ7KIk/4jB/i0OQR/BvLrnGvZmpRMEX9d/RIYViyBdvvA== X-Received: by 2002:a05:6000:cf:b0:228:e37b:361b with SMTP id q15-20020a05600000cf00b00228e37b361bmr19098182wrx.374.1663097161660; Tue, 13 Sep 2022 12:26:01 -0700 (PDT) Received: from [127.0.0.1] ([13.74.141.28]) by smtp.gmail.com with ESMTPSA id h17-20020a05600c2cb100b003a5ca627333sm15714153wmc.8.2022.09.13.12.26.01 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 13 Sep 2022 12:26:01 -0700 (PDT) Message-Id: <78e66d56605cfb1c7000edf329ac16c05a5d69b0.1663097156.git.gitgitgadget@gmail.com> In-Reply-To: References: Date: Tue, 13 Sep 2022 19:25:52 +0000 Subject: [PATCH 4/8] http: read HTTP WWW-Authenticate response headers Fcc: Sent MIME-Version: 1.0 To: git@vger.kernel.org Cc: Matthew John Cheetham , Matthew John Cheetham Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org From: Matthew John Cheetham From: Matthew John Cheetham Read and store the HTTP WWW-Authenticate response headers made for a particular request. This will allow us to pass important authentication challenge information to credential helpers or others that would otherwise have been lost. According to RFC2616 Section 4.2 [1], header field names are not case-sensitive meaning when collecting multiple values for the same field name, we can just use the case of the first observed instance of each field name and no normalisation is required. libcurl only provides us with the ability to read all headers recieved for a particular request, including any intermediate redirect requests or proxies. The lines returned by libcurl include HTTP status lines delinating any intermediate requests such as "HTTP/1.1 200". We use these lines to reset the strvec of WWW-Authenticate header values as we encounter them in order to only capture the final response headers. The collection of all header values matching the WWW-Authenticate header is complicated by the fact that it is legal for header fields to be continued over multiple lines, but libcurl only gives us one line at a time. In the future [2] we may be able to leverage functions to read headers from libcurl itself, but as of today we must do this ourselves. [1] https://datatracker.ietf.org/doc/html/rfc2616#section-4.2 [2] https://daniel.haxx.se/blog/2022/03/22/a-headers-api-for-libcurl/ Signed-off-by: Matthew John Cheetham --- credential.c | 1 + credential.h | 10 +++++++ http.c | 77 ++++++++++++++++++++++++++++++++++++++++++++++++++++ 3 files changed, 88 insertions(+) diff --git a/credential.c b/credential.c index f6389a50684..897b4679333 100644 --- a/credential.c +++ b/credential.c @@ -22,6 +22,7 @@ void credential_clear(struct credential *c) free(c->username); free(c->password); string_list_clear(&c->helpers, 0); + strvec_clear(&c->wwwauth_headers); credential_init(c); } diff --git a/credential.h b/credential.h index f430e77fea4..6a9d4e3de07 100644 --- a/credential.h +++ b/credential.h @@ -2,6 +2,7 @@ #define CREDENTIAL_H #include "string-list.h" +#include "strvec.h" /** * The credentials API provides an abstracted way of gathering username and @@ -115,6 +116,14 @@ struct credential { */ struct string_list helpers; + /** + * A `strvec` of WWW-Authenticate header values. Each string + * is the value of a WWW-Authenticate header in an HTTP response, + * in the order they were received in the response. + */ + struct strvec wwwauth_headers; + unsigned header_is_last_match:1; + unsigned approved:1, configured:1, quit:1, @@ -130,6 +139,7 @@ struct credential { #define CREDENTIAL_INIT { \ .helpers = STRING_LIST_INIT_DUP, \ + .wwwauth_headers = STRVEC_INIT, \ } /* Initialize a credential structure, setting all fields to empty. */ diff --git a/http.c b/http.c index 5d0502f51fd..091321af98e 100644 --- a/http.c +++ b/http.c @@ -183,6 +183,81 @@ size_t fwrite_buffer(char *ptr, size_t eltsize, size_t nmemb, void *buffer_) return nmemb; } +static size_t fwrite_wwwauth(char *ptr, size_t eltsize, size_t nmemb, void *p) +{ + size_t size = eltsize * nmemb; + struct strvec *values = &http_auth.wwwauth_headers; + struct strbuf buf = STRBUF_INIT; + const char *val; + const char *z = NULL; + + /* + * Header lines may not come NULL-terminated from libcurl so we must + * limit all scans to the maximum length of the header line, or leverage + * strbufs for all operations. + * + * In addition, it is possible that header values can be split over + * multiple lines as per RFC 2616 (even though this has since been + * deprecated in RFC 7230). A continuation header field value is + * identified as starting with a space or horizontal tab. + * + * The formal definition of a header field as given in RFC 2616 is: + * + * message-header = field-name ":" [ field-value ] + * field-name = token + * field-value = *( field-content | LWS ) + * field-content = + */ + + strbuf_add(&buf, ptr, size); + + /* Strip the CRLF that should be present at the end of each field */ + strbuf_trim_trailing_newline(&buf); + + /* Start of a new WWW-Authenticate header */ + if (skip_iprefix(buf.buf, "www-authenticate:", &val)) { + while (isspace(*val)) val++; + + strvec_push(values, val); + http_auth.header_is_last_match = 1; + goto exit; + } + + /* + * This line could be a continuation of the previously matched header + * field. If this is the case then we should append this value to the + * end of the previously consumed value. + */ + if (http_auth.header_is_last_match && isspace(*buf.buf)) { + const char **v = values->v + values->nr - 1; + char *append = xstrfmt("%s%.*s", *v, (int)(size - 1), ptr + 1); + + free((void*)*v); + *v = append; + + goto exit; + } + + /* This is the start of a new header we don't care about */ + http_auth.header_is_last_match = 0; + + /* + * If this is a HTTP status line and not a header field, this signals + * a different HTTP response. libcurl writes all the output of all + * response headers of all responses, including redirects. + * We only care about the last HTTP request response's headers so clear + * the existing array. + */ + if (skip_iprefix(buf.buf, "http/", &z)) + strvec_clear(values); + +exit: + strbuf_release(&buf); + return size; +} + size_t fwrite_null(char *ptr, size_t eltsize, size_t nmemb, void *strbuf) { return nmemb; @@ -1829,6 +1904,8 @@ static int http_request(const char *url, fwrite_buffer); } + curl_easy_setopt(slot->curl, CURLOPT_HEADERFUNCTION, fwrite_wwwauth); + accept_language = http_get_accept_language_header(); if (accept_language) From patchwork Tue Sep 13 19:25:53 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Matthew John Cheetham X-Patchwork-Id: 12975182 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 4CD7DECAAD8 for ; Tue, 13 Sep 2022 19:26:16 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229506AbiIMT0N (ORCPT ); Tue, 13 Sep 2022 15:26:13 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:50306 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229480AbiIMT0E (ORCPT ); Tue, 13 Sep 2022 15:26:04 -0400 Received: from mail-wm1-x32c.google.com (mail-wm1-x32c.google.com [IPv6:2a00:1450:4864:20::32c]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 3817179632 for ; Tue, 13 Sep 2022 12:26:03 -0700 (PDT) Received: by mail-wm1-x32c.google.com with SMTP id r66-20020a1c4445000000b003b494ffc00bso1582018wma.0 for ; Tue, 13 Sep 2022 12:26:03 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=cc:to:mime-version:content-transfer-encoding:fcc:subject:date:from :references:in-reply-to:message-id:from:to:cc:subject:date; bh=77O6rPRJ2q8PuFyRFUJwKrCINBOGtDCnw4Thl805/gI=; b=gP3ZMjKpTgsfAbodCdyfwu13nlfRGnc24J1aLaZ2CCl3kiz/VzBr63OejEjR3ZEaKc mTQt2G04TCqPFjSUIjszkund3j/+DDho1ubyQPpABJAZ1zp9bezc/UAYuP/q5Kq1i3Ib JGssq9MdjCmMFGJV+laPsho+0beKAC5hUcEu1fEPjpH+w2f93wt4xQTrWxRqtdCsebX6 +MG7PFuiWza9afPI8MGjFvcW/pHQNFWSOJM8DPe+huvV0l1/fmf5IZyPCq0HE9o9X+HP hDafju6LR91bOlCFn3XsrpGJFhi6p3y1KgtuReOA5FaSaz9+tFMh/X44tKuOq6Fcmp6/ +d3Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=cc:to:mime-version:content-transfer-encoding:fcc:subject:date:from :references:in-reply-to:message-id:x-gm-message-state:from:to:cc :subject:date; bh=77O6rPRJ2q8PuFyRFUJwKrCINBOGtDCnw4Thl805/gI=; b=tnkCIVN59WhbMs3G2xijOSZa6sSAqygQy5NYdNRvCOzkGcpcnbGk91qIwi9CzWmTM6 ZTIV6mBd1fewShHUxF5FEgWlp9x7GfSqrtSErd6ExwURlsOC57vOo51jROsuuF2JBKNp y5V/eYDV7RxQppY34vujBCYP8iqp5RscS7k7LfBCcV7cdOB1472ogAW51K31HRh5SQ9u rDJOG149XLCTkMclyIXI10td99Cj+gYIJoiDbyPZUJl+ScCLNEv3Cnab2EuS2Dq3l5ue NTxXxghhHdi3dvSsdRs7vkBZYMztXRM0Cnhn5tuwNDhZje99+fH5bI6bLfl3T0A2fMWB 47WA== X-Gm-Message-State: ACgBeo2x4WFLww+o27Ly4ppWgCjQjjKiAia8OrK47K1v8xnpk4uiC+nU D2yn0Vnj5AFIK8dXf9fKQC08dlCL09A= X-Google-Smtp-Source: AA6agR41aZfJ8iQP/koCn7Taik6tne7eo2IcaKCZu7juVTPRWwR+531lcxg2ZFTtMtZootXrcvyTXw== X-Received: by 2002:a05:600c:1c8d:b0:3a6:9d60:faf0 with SMTP id k13-20020a05600c1c8d00b003a69d60faf0mr629370wms.82.1663097162487; Tue, 13 Sep 2022 12:26:02 -0700 (PDT) Received: from [127.0.0.1] ([13.74.141.28]) by smtp.gmail.com with ESMTPSA id cc17-20020a5d5c11000000b0022ac119fcc5sm2550028wrb.60.2022.09.13.12.26.01 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 13 Sep 2022 12:26:02 -0700 (PDT) Message-Id: <936545004b8b46cbe24d8069cfd95ae5b5f98593.1663097156.git.gitgitgadget@gmail.com> In-Reply-To: References: Date: Tue, 13 Sep 2022 19:25:53 +0000 Subject: [PATCH 5/8] credential: add WWW-Authenticate header to cred requests Fcc: Sent MIME-Version: 1.0 To: git@vger.kernel.org Cc: Matthew John Cheetham , Matthew John Cheetham Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org From: Matthew John Cheetham From: Matthew John Cheetham Add the value of the WWW-Authenticate response header to credential requests. Credential helpers that understand and support HTTP authentication and authorization can use this standard header (RFC 2616 Section 14.47 [1]) to generate valid credentials. WWW-Authenticate headers can contain information pertaining to the authority, authentication mechanism, or extra parameters/scopes that are required. The current I/O format for credential helpers only allows for unique names for properties/attributes, so in order to transmit multiple header values (with a specific order) we introduce a new convention whereby a C-style array syntax is used in the property name to denote multiple ordered values for the same property. In this case we send multiple `wwwauth[n]` properties where `n` is a zero-indexed number, reflecting the order the WWW-Authenticate headers appeared in the HTTP response. [1] https://datatracker.ietf.org/doc/html/rfc2616#section-14.47 Signed-off-by: Matthew John Cheetham --- Documentation/git-credential.txt | 9 +++++++ credential.c | 12 +++++++++ t/lib-httpd/apache.conf | 13 +++++++++ t/t5551-http-fetch-smart.sh | 46 ++++++++++++++++++++++++++++++++ 4 files changed, 80 insertions(+) diff --git a/Documentation/git-credential.txt b/Documentation/git-credential.txt index f18673017f5..7d4a788c63d 100644 --- a/Documentation/git-credential.txt +++ b/Documentation/git-credential.txt @@ -151,6 +151,15 @@ Git understands the following attributes: were read (e.g., `url=https://example.com` would behave as if `protocol=https` and `host=example.com` had been provided). This can help callers avoid parsing URLs themselves. + +`wwwauth[n]`:: + + When an HTTP response is received that includes one or more + 'WWW-Authenticate' authentication headers, these can be passed to Git + (and subsequent credential helpers) with these attributes. + Each 'WWW-Authenticate' header value should be passed as a separate + attribute 'wwwauth[n]' where 'n' is the zero-indexed order the headers + appear in the HTTP response. + Note that specifying a protocol is mandatory and if the URL doesn't specify a hostname (e.g., "cert:///path/to/file") the diff --git a/credential.c b/credential.c index 897b4679333..4ad40323fc7 100644 --- a/credential.c +++ b/credential.c @@ -263,6 +263,17 @@ static void credential_write_item(FILE *fp, const char *key, const char *value, fprintf(fp, "%s=%s\n", key, value); } +static void credential_write_strvec(FILE *fp, const char *key, + const struct strvec *vec) +{ + int i = 0; + for (; i < vec->nr; i++) { + const char *full_key = xstrfmt("%s[%d]", key, i); + credential_write_item(fp, full_key, vec->v[i], 0); + free((void*)full_key); + } +} + void credential_write(const struct credential *c, FILE *fp) { credential_write_item(fp, "protocol", c->protocol, 1); @@ -270,6 +281,7 @@ void credential_write(const struct credential *c, FILE *fp) credential_write_item(fp, "path", c->path, 0); credential_write_item(fp, "username", c->username, 0); credential_write_item(fp, "password", c->password, 0); + credential_write_strvec(fp, "wwwauth", &c->wwwauth_headers); } static int run_credential_helper(struct credential *c, diff --git a/t/lib-httpd/apache.conf b/t/lib-httpd/apache.conf index 497b9b9d927..fe118d76f98 100644 --- a/t/lib-httpd/apache.conf +++ b/t/lib-httpd/apache.conf @@ -235,6 +235,19 @@ SSLEngine On Require valid-user +# Advertise two additional auth methods above "Basic". +# Neither of them actually work but serve test cases showing these +# additional auth headers are consumed correctly. + + AuthType Basic + AuthName "git-auth" + AuthUserFile passwd + Require valid-user + SetEnvIf Authorization "^\S+" authz + Header always add WWW-Authenticate "Bearer authority=https://login.example.com" env=!authz + Header always add WWW-Authenticate "FooAuth foo=bar baz=1" env=!authz + + RewriteCond %{QUERY_STRING} service=git-receive-pack [OR] RewriteCond %{REQUEST_URI} /git-receive-pack$ RewriteRule ^/half-auth-complete/ - [E=AUTHREQUIRED:yes] diff --git a/t/t5551-http-fetch-smart.sh b/t/t5551-http-fetch-smart.sh index 6a38294a476..c99d8e253df 100755 --- a/t/t5551-http-fetch-smart.sh +++ b/t/t5551-http-fetch-smart.sh @@ -564,6 +564,52 @@ test_expect_success 'http auth forgets bogus credentials' ' expect_askpass both user@host ' +test_expect_success 'http auth sends www-auth headers to credential helper' ' + write_script git-credential-tee <<-\EOF && + cmd=$1 + teefile=credential-$cmd + if [ -f "$teefile" ]; then + rm $teefile + fi + ( + while read line; + do + if [ -z "$line" ]; then + exit 0 + fi + echo "$line" >> $teefile + echo $line + done + ) | git credential-store $cmd + EOF + + cat >expected-get <<-EOF && + protocol=http + host=127.0.0.1:5551 + wwwauth[0]=Bearer authority=https://login.example.com + wwwauth[1]=FooAuth foo=bar baz=1 + wwwauth[2]=Basic realm="git-auth" + EOF + + cat >expected-store <<-EOF && + protocol=http + host=127.0.0.1:5551 + username=user@host + password=pass@host + EOF + + rm -f .git-credentials && + test_config credential.helper tee && + set_askpass user@host pass@host && + ( + PATH="$PWD:$PATH" && + git ls-remote "$HTTPD_URL/auth-wwwauth/smart/repo.git" + ) && + expect_askpass both user@host && + test_cmp expected-get credential-get && + test_cmp expected-store credential-store +' + test_expect_success 'client falls back from v2 to v0 to match server' ' GIT_TRACE_PACKET=$PWD/trace \ GIT_TEST_PROTOCOL_VERSION=2 \ From patchwork Tue Sep 13 19:25:54 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Matthew John Cheetham X-Patchwork-Id: 12975186 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 76B67ECAAD8 for ; Tue, 13 Sep 2022 19:26:25 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229507AbiIMT0Y (ORCPT ); Tue, 13 Sep 2022 15:26:24 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:50476 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229493AbiIMT0J (ORCPT ); Tue, 13 Sep 2022 15:26:09 -0400 Received: from mail-wr1-x436.google.com (mail-wr1-x436.google.com [IPv6:2a00:1450:4864:20::436]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 96DAF72B4C for ; Tue, 13 Sep 2022 12:26:05 -0700 (PDT) Received: by mail-wr1-x436.google.com with SMTP id bj14so22333163wrb.12 for ; Tue, 13 Sep 2022 12:26:05 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=cc:to:mime-version:content-transfer-encoding:fcc:subject:date:from :references:in-reply-to:message-id:from:to:cc:subject:date; bh=vLw0qYalCooNhlK2Lxc4/zWY+hUhPGUErRGtlzuXuSI=; b=WtMThNMpKoQWibB0Ydfm/XK1Y1cuncy1vviloia11apuM4Tg5XCYkSNhrvSy1cQsJS VWakXZikZS/S0PryWBf0CieOydY4IK8q0heJs+0jmHOSUP8fNYJe2Mx2Vv8wljWrluIm JEOJJTx3Kux34RmtGio1WioLWkwwn/IFqTY4PgeL43iz0eD7iu5aGuW+OxfeMiGgxN9o RWSD0wbbsywEc6xKpfHfVbydY9pHfErPfYGQNOpXTb8THanITscOsLgEBaq7Vt5Qmf60 7HmAEQz5pYiQlun0PYbqrrrKtoM+6c1aJ+/Mq4IKGIJz41SV22sDRKMMKNae3XNszPK9 o36Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=cc:to:mime-version:content-transfer-encoding:fcc:subject:date:from :references:in-reply-to:message-id:x-gm-message-state:from:to:cc :subject:date; bh=vLw0qYalCooNhlK2Lxc4/zWY+hUhPGUErRGtlzuXuSI=; b=2Ww63wX7cK0/SZ7Yr0iU2IOlNrIIH0F7m4nNadVDtmUt2BBLCuEHvHD7D7RaIwGMC0 3xn5FKXoPzqHaHzmvirT8d/cvsx7YYzr7dEwI9ibY/AujA1Ujw+lccD1V98a7l50mtYF F5SptAmx9GGRWAiHcB3OFFlCK3hZnTaorK/5FttwxBuCNZ0Ne8JuUL/c8G5BWCILhdQZ N/fLN5tWCjYxLqyfDrC8r9YuD+KTaFmZhIERiZxSJGXB0Lo1xQVg2MPd8U709T0C3uln 09hLLy8Ta+H3kfN5UJ55YH0rIWOpVRjzeP8NfTnawjrCvacQkfl6vpeMN4yDx9c0Z+S7 A96Q== X-Gm-Message-State: ACgBeo0txzP5pIBn4DmxzViVPLffz+Gw05DbTrl2/+yOBxRmqEIxD8zZ 48vT8baZ/nvdTjUob6eV0pkBUxdW1Bw= X-Google-Smtp-Source: AA6agR7f1xQwCN6UFIjh7idLJq2FzXz2LaQTFsaYaysat0Eof8Dq4YUXFC2Fa1BpYSZfistamdsn6w== X-Received: by 2002:a05:6000:243:b0:226:d241:11e6 with SMTP id m3-20020a056000024300b00226d24111e6mr19136818wrz.187.1663097163556; Tue, 13 Sep 2022 12:26:03 -0700 (PDT) Received: from [127.0.0.1] ([13.74.141.28]) by smtp.gmail.com with ESMTPSA id i2-20020a5d4382000000b00228933481dbsm11197953wrq.47.2022.09.13.12.26.02 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 13 Sep 2022 12:26:03 -0700 (PDT) Message-Id: <20843e2051eeab71c5b7555f3e10383484e34b0e.1663097156.git.gitgitgadget@gmail.com> In-Reply-To: References: Date: Tue, 13 Sep 2022 19:25:54 +0000 Subject: [PATCH 6/8] http: store all request headers on active_request_slot Fcc: Sent MIME-Version: 1.0 To: git@vger.kernel.org Cc: Matthew John Cheetham , Matthew John Cheetham Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org From: Matthew John Cheetham From: Matthew John Cheetham Once a list of headers has been set on the curl handle, it is not possible to recover that `struct curl_slist` instance to add or modify headers. In future commits we will want to modify the set of request headers in response to an authentication challenge/401 response from the server, with information provided by a credential helper. There are a number of different places where curl is used for an HTTP request, and they do not have a common handling of request headers. However, given that they all do call the `start_active_slot()` function, either directly or indirectly via `run_slot()` or `run_one_slot()`, we use this as the point to set the `CURLOPT_HTTPHEADER` option just before the request is made. We collect all request headers in a `struct curl_slist` on the `struct active_request_slot` that is obtained from a call to `get_active_slot(int)`. This function now takes a single argument to define if the initial set of headers on the slot should include the "Pragma: no-cache" header, along with all extra headers specified via `http.extraHeader` config values. The active request slot obtained from `get_active_slot(int)` will always contain a fresh set of default headers and any headers set in previous usages of this slot will be freed. Signed-off-by: Matthew John Cheetham --- http-push.c | 103 ++++++++++++++++++++++---------------------------- http-walker.c | 2 +- http.c | 82 ++++++++++++++++++---------------------- http.h | 4 +- remote-curl.c | 36 +++++++++--------- 5 files changed, 101 insertions(+), 126 deletions(-) diff --git a/http-push.c b/http-push.c index 5f4340a36e6..2b40959b376 100644 --- a/http-push.c +++ b/http-push.c @@ -211,29 +211,29 @@ static void curl_setup_http(CURL *curl, const char *url, curl_easy_setopt(curl, CURLOPT_UPLOAD, 1); } -static struct curl_slist *get_dav_token_headers(struct remote_lock *lock, enum dav_header_flag options) +static struct curl_slist *append_dav_token_headers(struct curl_slist *headers, + struct remote_lock *lock, enum dav_header_flag options) { struct strbuf buf = STRBUF_INIT; - struct curl_slist *dav_headers = http_copy_default_headers(); if (options & DAV_HEADER_IF) { strbuf_addf(&buf, "If: (<%s>)", lock->token); - dav_headers = curl_slist_append(dav_headers, buf.buf); + headers = curl_slist_append(headers, buf.buf); strbuf_reset(&buf); } if (options & DAV_HEADER_LOCK) { strbuf_addf(&buf, "Lock-Token: <%s>", lock->token); - dav_headers = curl_slist_append(dav_headers, buf.buf); + headers = curl_slist_append(headers, buf.buf); strbuf_reset(&buf); } if (options & DAV_HEADER_TIMEOUT) { strbuf_addf(&buf, "Timeout: Second-%ld", lock->timeout); - dav_headers = curl_slist_append(dav_headers, buf.buf); + headers = curl_slist_append(headers, buf.buf); strbuf_reset(&buf); } strbuf_release(&buf); - return dav_headers; + return headers; } static void finish_request(struct transfer_request *request); @@ -281,7 +281,7 @@ static void start_mkcol(struct transfer_request *request) request->url = get_remote_object_url(repo->url, hex, 1); - slot = get_active_slot(); + slot = get_active_slot(0); slot->callback_func = process_response; slot->callback_data = request; curl_setup_http_get(slot->curl, request->url, DAV_MKCOL); @@ -399,7 +399,7 @@ static void start_put(struct transfer_request *request) strbuf_add(&buf, request->lock->tmpfile_suffix, the_hash_algo->hexsz + 1); request->url = strbuf_detach(&buf, NULL); - slot = get_active_slot(); + slot = get_active_slot(0); slot->callback_func = process_response; slot->callback_data = request; curl_setup_http(slot->curl, request->url, DAV_PUT, @@ -417,15 +417,13 @@ static void start_put(struct transfer_request *request) static void start_move(struct transfer_request *request) { struct active_request_slot *slot; - struct curl_slist *dav_headers = http_copy_default_headers(); - slot = get_active_slot(); + slot = get_active_slot(0); slot->callback_func = process_response; slot->callback_data = request; curl_setup_http_get(slot->curl, request->url, DAV_MOVE); - dav_headers = curl_slist_append(dav_headers, request->dest); - dav_headers = curl_slist_append(dav_headers, "Overwrite: T"); - curl_easy_setopt(slot->curl, CURLOPT_HTTPHEADER, dav_headers); + slot->headers = curl_slist_append(slot->headers, request->dest); + slot->headers = curl_slist_append(slot->headers, "Overwrite: T"); if (start_active_slot(slot)) { request->slot = slot; @@ -440,17 +438,16 @@ static int refresh_lock(struct remote_lock *lock) { struct active_request_slot *slot; struct slot_results results; - struct curl_slist *dav_headers; int rc = 0; lock->refreshing = 1; - dav_headers = get_dav_token_headers(lock, DAV_HEADER_IF | DAV_HEADER_TIMEOUT); - - slot = get_active_slot(); + slot = get_active_slot(0); slot->results = &results; + slot->headers = append_dav_token_headers(slot->headers, lock, + DAV_HEADER_IF | DAV_HEADER_TIMEOUT); + curl_setup_http_get(slot->curl, lock->url, DAV_LOCK); - curl_easy_setopt(slot->curl, CURLOPT_HTTPHEADER, dav_headers); if (start_active_slot(slot)) { run_active_slot(slot); @@ -464,7 +461,6 @@ static int refresh_lock(struct remote_lock *lock) } lock->refreshing = 0; - curl_slist_free_all(dav_headers); return rc; } @@ -838,7 +834,6 @@ static struct remote_lock *lock_remote(const char *path, long timeout) char *ep; char timeout_header[25]; struct remote_lock *lock = NULL; - struct curl_slist *dav_headers = http_copy_default_headers(); struct xml_ctx ctx; char *escaped; @@ -849,7 +844,7 @@ static struct remote_lock *lock_remote(const char *path, long timeout) while (ep) { char saved_character = ep[1]; ep[1] = '\0'; - slot = get_active_slot(); + slot = get_active_slot(0); slot->results = &results; curl_setup_http_get(slot->curl, url, DAV_MKCOL); if (start_active_slot(slot)) { @@ -875,14 +870,15 @@ static struct remote_lock *lock_remote(const char *path, long timeout) strbuf_addf(&out_buffer.buf, LOCK_REQUEST, escaped); free(escaped); + slot = get_active_slot(0); + slot->results = &results; + xsnprintf(timeout_header, sizeof(timeout_header), "Timeout: Second-%ld", timeout); - dav_headers = curl_slist_append(dav_headers, timeout_header); - dav_headers = curl_slist_append(dav_headers, "Content-Type: text/xml"); + slot->headers = curl_slist_append(slot->headers, timeout_header); + slot->headers = curl_slist_append(slot->headers, + "Content-Type: text/xml"); - slot = get_active_slot(); - slot->results = &results; curl_setup_http(slot->curl, url, DAV_LOCK, &out_buffer, fwrite_buffer); - curl_easy_setopt(slot->curl, CURLOPT_HTTPHEADER, dav_headers); curl_easy_setopt(slot->curl, CURLOPT_WRITEDATA, &in_buffer); CALLOC_ARRAY(lock, 1); @@ -921,7 +917,6 @@ static struct remote_lock *lock_remote(const char *path, long timeout) fprintf(stderr, "Unable to start LOCK request\n"); } - curl_slist_free_all(dav_headers); strbuf_release(&out_buffer.buf); strbuf_release(&in_buffer); @@ -945,15 +940,14 @@ static int unlock_remote(struct remote_lock *lock) struct active_request_slot *slot; struct slot_results results; struct remote_lock *prev = repo->locks; - struct curl_slist *dav_headers; int rc = 0; - dav_headers = get_dav_token_headers(lock, DAV_HEADER_LOCK); - - slot = get_active_slot(); + slot = get_active_slot(0); slot->results = &results; + slot->headers = append_dav_token_headers(slot->headers, lock, + DAV_HEADER_LOCK); + curl_setup_http_get(slot->curl, lock->url, DAV_UNLOCK); - curl_easy_setopt(slot->curl, CURLOPT_HTTPHEADER, dav_headers); if (start_active_slot(slot)) { run_active_slot(slot); @@ -966,8 +960,6 @@ static int unlock_remote(struct remote_lock *lock) fprintf(stderr, "Unable to start UNLOCK request\n"); } - curl_slist_free_all(dav_headers); - if (repo->locks == lock) { repo->locks = lock->next; } else { @@ -1121,7 +1113,6 @@ static void remote_ls(const char *path, int flags, struct slot_results results; struct strbuf in_buffer = STRBUF_INIT; struct buffer out_buffer = { STRBUF_INIT, 0 }; - struct curl_slist *dav_headers = http_copy_default_headers(); struct xml_ctx ctx; struct remote_ls_ctx ls; @@ -1134,14 +1125,14 @@ static void remote_ls(const char *path, int flags, strbuf_addstr(&out_buffer.buf, PROPFIND_ALL_REQUEST); - dav_headers = curl_slist_append(dav_headers, "Depth: 1"); - dav_headers = curl_slist_append(dav_headers, "Content-Type: text/xml"); - - slot = get_active_slot(); + slot = get_active_slot(0); slot->results = &results; + slot->headers = curl_slist_append(slot->headers, "Depth: 1"); + slot->headers = curl_slist_append(slot->headers, + "Content-Type: text/xml"); + curl_setup_http(slot->curl, url, DAV_PROPFIND, &out_buffer, fwrite_buffer); - curl_easy_setopt(slot->curl, CURLOPT_HTTPHEADER, dav_headers); curl_easy_setopt(slot->curl, CURLOPT_WRITEDATA, &in_buffer); if (start_active_slot(slot)) { @@ -1177,7 +1168,6 @@ static void remote_ls(const char *path, int flags, free(url); strbuf_release(&out_buffer.buf); strbuf_release(&in_buffer); - curl_slist_free_all(dav_headers); } static void get_remote_object_list(unsigned char parent) @@ -1199,7 +1189,6 @@ static int locking_available(void) struct slot_results results; struct strbuf in_buffer = STRBUF_INIT; struct buffer out_buffer = { STRBUF_INIT, 0 }; - struct curl_slist *dav_headers = http_copy_default_headers(); struct xml_ctx ctx; int lock_flags = 0; char *escaped; @@ -1208,14 +1197,14 @@ static int locking_available(void) strbuf_addf(&out_buffer.buf, PROPFIND_SUPPORTEDLOCK_REQUEST, escaped); free(escaped); - dav_headers = curl_slist_append(dav_headers, "Depth: 0"); - dav_headers = curl_slist_append(dav_headers, "Content-Type: text/xml"); - - slot = get_active_slot(); + slot = get_active_slot(0); slot->results = &results; + slot->headers = curl_slist_append(slot->headers, "Depth: 0"); + slot->headers = curl_slist_append(slot->headers, + "Content-Type: text/xml"); + curl_setup_http(slot->curl, repo->url, DAV_PROPFIND, &out_buffer, fwrite_buffer); - curl_easy_setopt(slot->curl, CURLOPT_HTTPHEADER, dav_headers); curl_easy_setopt(slot->curl, CURLOPT_WRITEDATA, &in_buffer); if (start_active_slot(slot)) { @@ -1257,7 +1246,6 @@ static int locking_available(void) strbuf_release(&out_buffer.buf); strbuf_release(&in_buffer); - curl_slist_free_all(dav_headers); return lock_flags; } @@ -1374,17 +1362,16 @@ static int update_remote(const struct object_id *oid, struct remote_lock *lock) struct active_request_slot *slot; struct slot_results results; struct buffer out_buffer = { STRBUF_INIT, 0 }; - struct curl_slist *dav_headers; - - dav_headers = get_dav_token_headers(lock, DAV_HEADER_IF); strbuf_addf(&out_buffer.buf, "%s\n", oid_to_hex(oid)); - slot = get_active_slot(); + slot = get_active_slot(0); slot->results = &results; + slot->headers = append_dav_token_headers(slot->headers, lock, + DAV_HEADER_IF); + curl_setup_http(slot->curl, lock->url, DAV_PUT, &out_buffer, fwrite_null); - curl_easy_setopt(slot->curl, CURLOPT_HTTPHEADER, dav_headers); if (start_active_slot(slot)) { run_active_slot(slot); @@ -1486,18 +1473,18 @@ static void update_remote_info_refs(struct remote_lock *lock) struct buffer buffer = { STRBUF_INIT, 0 }; struct active_request_slot *slot; struct slot_results results; - struct curl_slist *dav_headers; remote_ls("refs/", (PROCESS_FILES | RECURSIVE), add_remote_info_ref, &buffer.buf); if (!aborted) { - dav_headers = get_dav_token_headers(lock, DAV_HEADER_IF); - slot = get_active_slot(); + slot = get_active_slot(0); slot->results = &results; + slot->headers = append_dav_token_headers(slot->headers, lock, + DAV_HEADER_IF); + curl_setup_http(slot->curl, lock->url, DAV_PUT, &buffer, fwrite_null); - curl_easy_setopt(slot->curl, CURLOPT_HTTPHEADER, dav_headers); if (start_active_slot(slot)) { run_active_slot(slot); @@ -1652,7 +1639,7 @@ static int delete_remote_branch(const char *pattern, int force) if (dry_run) return 0; url = xstrfmt("%s%s", repo->url, remote_ref->name); - slot = get_active_slot(); + slot = get_active_slot(0); slot->results = &results; curl_setup_http_get(slot->curl, url, DAV_DELETE); if (start_active_slot(slot)) { diff --git a/http-walker.c b/http-walker.c index b8f0f98ae14..8747de2fcdb 100644 --- a/http-walker.c +++ b/http-walker.c @@ -373,7 +373,7 @@ static void fetch_alternates(struct walker *walker, const char *base) * Use a callback to process the result, since another request * may fail and need to have alternates loaded before continuing */ - slot = get_active_slot(); + slot = get_active_slot(0); slot->callback_func = process_alternates_response; alt_req.walker = walker; slot->callback_data = &alt_req; diff --git a/http.c b/http.c index 091321af98e..42616f746b1 100644 --- a/http.c +++ b/http.c @@ -124,8 +124,6 @@ static unsigned long empty_auth_useless = | CURLAUTH_DIGEST_IE | CURLAUTH_DIGEST; -static struct curl_slist *pragma_header; -static struct curl_slist *no_pragma_header; static struct string_list extra_http_headers = STRING_LIST_INIT_DUP; static struct curl_slist *host_resolutions; @@ -1132,11 +1130,6 @@ void http_init(struct remote *remote, const char *url, int proactive_auth) if (remote) var_override(&http_proxy_authmethod, remote->http_proxy_authmethod); - pragma_header = curl_slist_append(http_copy_default_headers(), - "Pragma: no-cache"); - no_pragma_header = curl_slist_append(http_copy_default_headers(), - "Pragma:"); - { char *http_max_requests = getenv("GIT_HTTP_MAX_REQUESTS"); if (http_max_requests) @@ -1198,6 +1191,8 @@ void http_cleanup(void) while (slot != NULL) { struct active_request_slot *next = slot->next; + if (slot->headers) + curl_slist_free_all(slot->headers); if (slot->curl) { xmulti_remove_handle(slot); curl_easy_cleanup(slot->curl); @@ -1214,12 +1209,6 @@ void http_cleanup(void) string_list_clear(&extra_http_headers, 0); - curl_slist_free_all(pragma_header); - pragma_header = NULL; - - curl_slist_free_all(no_pragma_header); - no_pragma_header = NULL; - curl_slist_free_all(host_resolutions); host_resolutions = NULL; @@ -1254,7 +1243,18 @@ void http_cleanup(void) FREE_AND_NULL(cached_accept_language); } -struct active_request_slot *get_active_slot(void) +static struct curl_slist *http_copy_default_headers(void) +{ + struct curl_slist *headers = NULL; + const struct string_list_item *item; + + for_each_string_list_item(item, &extra_http_headers) + headers = curl_slist_append(headers, item->string); + + return headers; +} + +struct active_request_slot *get_active_slot(int no_pragma_header) { struct active_request_slot *slot = active_queue_head; struct active_request_slot *newslot; @@ -1276,6 +1276,7 @@ struct active_request_slot *get_active_slot(void) newslot->curl = NULL; newslot->in_use = 0; newslot->next = NULL; + newslot->headers = NULL; slot = active_queue_head; if (!slot) { @@ -1293,6 +1294,15 @@ struct active_request_slot *get_active_slot(void) curl_session_count++; } + if (slot->headers) + curl_slist_free_all(slot->headers); + + slot->headers = http_copy_default_headers(); + + if (!no_pragma_header) + slot->headers = curl_slist_append(slot->headers, + "Pragma: no-cache"); + active_requests++; slot->in_use = 1; slot->results = NULL; @@ -1302,7 +1312,6 @@ struct active_request_slot *get_active_slot(void) curl_easy_setopt(slot->curl, CURLOPT_COOKIEFILE, curl_cookie_file); if (curl_save_cookies) curl_easy_setopt(slot->curl, CURLOPT_COOKIEJAR, curl_cookie_file); - curl_easy_setopt(slot->curl, CURLOPT_HTTPHEADER, pragma_header); curl_easy_setopt(slot->curl, CURLOPT_RESOLVE, host_resolutions); curl_easy_setopt(slot->curl, CURLOPT_ERRORBUFFER, curl_errorstr); curl_easy_setopt(slot->curl, CURLOPT_CUSTOMREQUEST, NULL); @@ -1334,9 +1343,12 @@ struct active_request_slot *get_active_slot(void) int start_active_slot(struct active_request_slot *slot) { - CURLMcode curlm_result = curl_multi_add_handle(curlm, slot->curl); + CURLMcode curlm_result; int num_transfers; + curl_easy_setopt(slot->curl, CURLOPT_HTTPHEADER, slot->headers); + curlm_result = curl_multi_add_handle(curlm, slot->curl); + if (curlm_result != CURLM_OK && curlm_result != CURLM_CALL_MULTI_PERFORM) { warning("curl_multi_add_handle failed: %s", @@ -1651,17 +1663,6 @@ int run_one_slot(struct active_request_slot *slot, return handle_curl_result(results); } -struct curl_slist *http_copy_default_headers(void) -{ - struct curl_slist *headers = NULL; - const struct string_list_item *item; - - for_each_string_list_item(item, &extra_http_headers) - headers = curl_slist_append(headers, item->string); - - return headers; -} - static CURLcode curlinfo_strbuf(CURL *curl, CURLINFO info, struct strbuf *buf) { char *ptr; @@ -1879,12 +1880,11 @@ static int http_request(const char *url, { struct active_request_slot *slot; struct slot_results results; - struct curl_slist *headers = http_copy_default_headers(); - struct strbuf buf = STRBUF_INIT; + int no_cache = options && options->no_cache; const char *accept_language; int ret; - slot = get_active_slot(); + slot = get_active_slot(!no_cache); curl_easy_setopt(slot->curl, CURLOPT_HTTPGET, 1); if (!result) { @@ -1909,27 +1909,23 @@ static int http_request(const char *url, accept_language = http_get_accept_language_header(); if (accept_language) - headers = curl_slist_append(headers, accept_language); + slot->headers = curl_slist_append(slot->headers, + accept_language); - strbuf_addstr(&buf, "Pragma:"); - if (options && options->no_cache) - strbuf_addstr(&buf, " no-cache"); if (options && options->initial_request && http_follow_config == HTTP_FOLLOW_INITIAL) curl_easy_setopt(slot->curl, CURLOPT_FOLLOWLOCATION, 1); - headers = curl_slist_append(headers, buf.buf); - /* Add additional headers here */ if (options && options->extra_headers) { const struct string_list_item *item; for_each_string_list_item(item, options->extra_headers) { - headers = curl_slist_append(headers, item->string); + slot->headers = curl_slist_append(slot->headers, + item->string); } } curl_easy_setopt(slot->curl, CURLOPT_URL, url); - curl_easy_setopt(slot->curl, CURLOPT_HTTPHEADER, headers); curl_easy_setopt(slot->curl, CURLOPT_ENCODING, ""); curl_easy_setopt(slot->curl, CURLOPT_FAILONERROR, 0); @@ -1947,9 +1943,6 @@ static int http_request(const char *url, curlinfo_strbuf(slot->curl, CURLINFO_EFFECTIVE_URL, options->effective_url); - curl_slist_free_all(headers); - strbuf_release(&buf); - return ret; } @@ -2310,12 +2303,10 @@ struct http_pack_request *new_direct_http_pack_request( goto abort; } - preq->slot = get_active_slot(); + preq->slot = get_active_slot(1); curl_easy_setopt(preq->slot->curl, CURLOPT_WRITEDATA, preq->packfile); curl_easy_setopt(preq->slot->curl, CURLOPT_WRITEFUNCTION, fwrite); curl_easy_setopt(preq->slot->curl, CURLOPT_URL, preq->url); - curl_easy_setopt(preq->slot->curl, CURLOPT_HTTPHEADER, - no_pragma_header); /* * If there is data present from a previous transfer attempt, @@ -2480,14 +2471,13 @@ struct http_object_request *new_http_object_request(const char *base_url, } } - freq->slot = get_active_slot(); + freq->slot = get_active_slot(1); curl_easy_setopt(freq->slot->curl, CURLOPT_WRITEDATA, freq); curl_easy_setopt(freq->slot->curl, CURLOPT_FAILONERROR, 0); curl_easy_setopt(freq->slot->curl, CURLOPT_WRITEFUNCTION, fwrite_sha1_file); curl_easy_setopt(freq->slot->curl, CURLOPT_ERRORBUFFER, freq->errorstr); curl_easy_setopt(freq->slot->curl, CURLOPT_URL, freq->url); - curl_easy_setopt(freq->slot->curl, CURLOPT_HTTPHEADER, no_pragma_header); /* * If we have successfully processed data from a previous fetch diff --git a/http.h b/http.h index 3c94c479100..a304cc408b2 100644 --- a/http.h +++ b/http.h @@ -22,6 +22,7 @@ struct slot_results { struct active_request_slot { CURL *curl; int in_use; + struct curl_slist *headers; CURLcode curl_result; long http_code; int *finished; @@ -43,7 +44,7 @@ size_t fwrite_null(char *ptr, size_t eltsize, size_t nmemb, void *strbuf); curlioerr ioctl_buffer(CURL *handle, int cmd, void *clientp); /* Slot lifecycle functions */ -struct active_request_slot *get_active_slot(void); +struct active_request_slot *get_active_slot(int no_pragma_header); int start_active_slot(struct active_request_slot *slot); void run_active_slot(struct active_request_slot *slot); void finish_all_active_slots(void); @@ -64,7 +65,6 @@ void step_active_slots(void); void http_init(struct remote *remote, const char *url, int proactive_auth); void http_cleanup(void); -struct curl_slist *http_copy_default_headers(void); extern long int git_curl_ipresolve; extern int active_requests; diff --git a/remote-curl.c b/remote-curl.c index 72dfb8fb86a..edbd4504beb 100644 --- a/remote-curl.c +++ b/remote-curl.c @@ -847,14 +847,13 @@ static int run_slot(struct active_request_slot *slot, static int probe_rpc(struct rpc_state *rpc, struct slot_results *results) { struct active_request_slot *slot; - struct curl_slist *headers = http_copy_default_headers(); struct strbuf buf = STRBUF_INIT; int err; - slot = get_active_slot(); + slot = get_active_slot(0); - headers = curl_slist_append(headers, rpc->hdr_content_type); - headers = curl_slist_append(headers, rpc->hdr_accept); + slot->headers = curl_slist_append(slot->headers, rpc->hdr_content_type); + slot->headers = curl_slist_append(slot->headers, rpc->hdr_accept); curl_easy_setopt(slot->curl, CURLOPT_NOBODY, 0); curl_easy_setopt(slot->curl, CURLOPT_POST, 1); @@ -862,13 +861,11 @@ static int probe_rpc(struct rpc_state *rpc, struct slot_results *results) curl_easy_setopt(slot->curl, CURLOPT_ENCODING, NULL); curl_easy_setopt(slot->curl, CURLOPT_POSTFIELDS, "0000"); curl_easy_setopt(slot->curl, CURLOPT_POSTFIELDSIZE, 4); - curl_easy_setopt(slot->curl, CURLOPT_HTTPHEADER, headers); curl_easy_setopt(slot->curl, CURLOPT_WRITEFUNCTION, fwrite_buffer); curl_easy_setopt(slot->curl, CURLOPT_WRITEDATA, &buf); err = run_slot(slot, results); - curl_slist_free_all(headers); strbuf_release(&buf); return err; } @@ -888,7 +885,6 @@ static curl_off_t xcurl_off_t(size_t len) static int post_rpc(struct rpc_state *rpc, int stateless_connect, int flush_received) { struct active_request_slot *slot; - struct curl_slist *headers = http_copy_default_headers(); int use_gzip = rpc->gzip_request; char *gzip_body = NULL; size_t gzip_size = 0; @@ -930,21 +926,23 @@ static int post_rpc(struct rpc_state *rpc, int stateless_connect, int flush_rece needs_100_continue = 1; } - headers = curl_slist_append(headers, rpc->hdr_content_type); - headers = curl_slist_append(headers, rpc->hdr_accept); - headers = curl_slist_append(headers, needs_100_continue ? +retry: + slot = get_active_slot(0); + + slot->headers = curl_slist_append(slot->headers, rpc->hdr_content_type); + slot->headers = curl_slist_append(slot->headers, rpc->hdr_accept); + slot->headers = curl_slist_append(slot->headers, needs_100_continue ? "Expect: 100-continue" : "Expect:"); /* Add Accept-Language header */ if (rpc->hdr_accept_language) - headers = curl_slist_append(headers, rpc->hdr_accept_language); + slot->headers = curl_slist_append(slot->headers, + rpc->hdr_accept_language); /* Add the extra Git-Protocol header */ if (rpc->protocol_header) - headers = curl_slist_append(headers, rpc->protocol_header); - -retry: - slot = get_active_slot(); + slot->headers = curl_slist_append(slot->headers, + rpc->protocol_header); curl_easy_setopt(slot->curl, CURLOPT_NOBODY, 0); curl_easy_setopt(slot->curl, CURLOPT_POST, 1); @@ -955,7 +953,8 @@ retry: /* The request body is large and the size cannot be predicted. * We must use chunked encoding to send it. */ - headers = curl_slist_append(headers, "Transfer-Encoding: chunked"); + slot->headers = curl_slist_append(slot->headers, + "Transfer-Encoding: chunked"); rpc->initial_buffer = 1; curl_easy_setopt(slot->curl, CURLOPT_READFUNCTION, rpc_out); curl_easy_setopt(slot->curl, CURLOPT_INFILE, rpc); @@ -1002,7 +1001,8 @@ retry: gzip_size = stream.total_out; - headers = curl_slist_append(headers, "Content-Encoding: gzip"); + slot->headers = curl_slist_append(slot->headers, + "Content-Encoding: gzip"); curl_easy_setopt(slot->curl, CURLOPT_POSTFIELDS, gzip_body); curl_easy_setopt(slot->curl, CURLOPT_POSTFIELDSIZE_LARGE, xcurl_off_t(gzip_size)); @@ -1025,7 +1025,6 @@ retry: } } - curl_easy_setopt(slot->curl, CURLOPT_HTTPHEADER, headers); curl_easy_setopt(slot->curl, CURLOPT_WRITEFUNCTION, rpc_in); rpc_in_data.rpc = rpc; rpc_in_data.slot = slot; @@ -1055,7 +1054,6 @@ retry: if (stateless_connect) packet_response_end(rpc->in); - curl_slist_free_all(headers); free(gzip_body); return err; } From patchwork Tue Sep 13 19:25:55 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Matthew John Cheetham X-Patchwork-Id: 12975184 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 04A6FC54EE9 for ; Tue, 13 Sep 2022 19:26:22 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229533AbiIMT0U (ORCPT ); Tue, 13 Sep 2022 15:26:20 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:50396 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229492AbiIMT0H (ORCPT ); Tue, 13 Sep 2022 15:26:07 -0400 Received: from mail-wm1-x32a.google.com (mail-wm1-x32a.google.com [IPv6:2a00:1450:4864:20::32a]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id AA23A72B55 for ; Tue, 13 Sep 2022 12:26:06 -0700 (PDT) Received: by mail-wm1-x32a.google.com with SMTP id ay36so562298wmb.0 for ; Tue, 13 Sep 2022 12:26:06 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=cc:to:mime-version:content-transfer-encoding:fcc:subject:date:from :references:in-reply-to:message-id:from:to:cc:subject:date; bh=SEIrIWSvchzYGJ1jiEGsC50LyAYDFjVOJpzFYRqqQrI=; b=DIRB6fg0XiYDzFjzLVd/XGOo7xz/92SBDxd8To16kMt7rCWaUTTw/JU0J+cmb3RtfH ELsxzoQOAK39Z9sBzdH27ZnABm2Fs8jG7YBthb9S6hU51o+voYpv/FmxDA7+h1bJ0gQy pSF78Vk5Lievq2Z5J8njAunonF7Qqm/IKWvEaet2SygoEgzJXUdQwxzYGbLPIrpMqO9X GSgPC35B4WpHkDtI8BdhV8+6T0iL/xGX1K5lDWzL+4Wd/E5UAxB/7YRr/tpP88+aH8nB Dr+Kt/ZaTJMIwnEOasHTLb35uuQlQ6uv6m4fHvZBrnhk6f/1iNSydtRnbcbQWoyDeAQK LlLA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=cc:to:mime-version:content-transfer-encoding:fcc:subject:date:from :references:in-reply-to:message-id:x-gm-message-state:from:to:cc :subject:date; bh=SEIrIWSvchzYGJ1jiEGsC50LyAYDFjVOJpzFYRqqQrI=; b=35AMgXb2C12XmR1FhhorsSNvKs6j1/jreCAnAvDTYZdRC7HznYdLZyNgoqen+p5H2n Zt4OSKrj1AhVKaKBvDUcJ+EFeyJt2cXM+XLGRH2xSbwa/8dHBtQ3sGYJ1rclniiSLJyM LEZ1N57NmqFqL6+/r6Uq+8xsopnTUwuZDaH11VpsMPNeVPG/x2Ultn0yd5+aOXUskwku vD93W7nVH6FnJqkbhgoiBld3k9U+GBNboXfrBJ2haoH9WqaU27FPTJZUnzsAoGEfAauZ twCQ9M797L6EmxFuWdP9mfQkz+wKOwXwayfQqtqR1fyczmWdt7xhdgJnCaSrNx1GG/X3 PGhA== X-Gm-Message-State: ACgBeo0S89glql/WbM53pMt4QxVbxZlv2nVxUmc3KpBFaxAsUM4AuFqy VnW6EnNDGJfdZ8g5ilRGoYzTqh+K4vQ= X-Google-Smtp-Source: AA6agR7QL5tTANTxJBX0xyZgYCzb/pu6rmy2MYRp+1DEyfUUhluoSEBaVYQGpYyjFTQ6y028BZv32Q== X-Received: by 2002:a05:600c:3c84:b0:3a6:9596:fa1f with SMTP id bg4-20020a05600c3c8400b003a69596fa1fmr602756wmb.162.1663097164482; Tue, 13 Sep 2022 12:26:04 -0700 (PDT) Received: from [127.0.0.1] ([13.74.141.28]) by smtp.gmail.com with ESMTPSA id h11-20020adff18b000000b00228daaa84aesm11433305wro.25.2022.09.13.12.26.03 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 13 Sep 2022 12:26:03 -0700 (PDT) Message-Id: In-Reply-To: References: Date: Tue, 13 Sep 2022 19:25:55 +0000 Subject: [PATCH 7/8] http: move proactive auth to first slot creation Fcc: Sent MIME-Version: 1.0 To: git@vger.kernel.org Cc: Matthew John Cheetham , Matthew John Cheetham Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org From: Matthew John Cheetham From: Matthew John Cheetham Rather than proactively seek credentials to authenticate a request at `http_init()` time, do it when the first `active_request_slot` is created. Because credential helpers may modify the headers used for a request we can only auth when a slot is created (when we can first start to gather request headers). Signed-off-by: Matthew John Cheetham --- http.c | 20 +++++++++++--------- 1 file changed, 11 insertions(+), 9 deletions(-) diff --git a/http.c b/http.c index 42616f746b1..8e107ff19b8 100644 --- a/http.c +++ b/http.c @@ -514,18 +514,18 @@ static int curl_empty_auth_enabled(void) return 0; } -static void init_curl_http_auth(CURL *result) +static void init_curl_http_auth(struct active_request_slot *slot) { if (!http_auth.username || !*http_auth.username) { if (curl_empty_auth_enabled()) - curl_easy_setopt(result, CURLOPT_USERPWD, ":"); + curl_easy_setopt(slot->curl, CURLOPT_USERPWD, ":"); return; } credential_fill(&http_auth); - curl_easy_setopt(result, CURLOPT_USERNAME, http_auth.username); - curl_easy_setopt(result, CURLOPT_PASSWORD, http_auth.password); + curl_easy_setopt(slot->curl, CURLOPT_USERNAME, http_auth.username); + curl_easy_setopt(slot->curl, CURLOPT_PASSWORD, http_auth.password); } /* *var must be free-able */ @@ -900,9 +900,6 @@ static CURL *get_curl_handle(void) #endif } - if (http_proactive_auth) - init_curl_http_auth(result); - if (getenv("GIT_SSL_VERSION")) ssl_version = getenv("GIT_SSL_VERSION"); if (ssl_version && *ssl_version) { @@ -1259,6 +1256,7 @@ struct active_request_slot *get_active_slot(int no_pragma_header) struct active_request_slot *slot = active_queue_head; struct active_request_slot *newslot; + int proactive_auth = 0; int num_transfers; /* Wait for a slot to open up if the queue is full */ @@ -1281,6 +1279,9 @@ struct active_request_slot *get_active_slot(int no_pragma_header) slot = active_queue_head; if (!slot) { active_queue_head = newslot; + + /* Auth first slot if asked for proactive auth */ + proactive_auth = http_proactive_auth; } else { while (slot->next != NULL) slot = slot->next; @@ -1335,8 +1336,9 @@ struct active_request_slot *get_active_slot(int no_pragma_header) curl_easy_setopt(slot->curl, CURLOPT_IPRESOLVE, git_curl_ipresolve); curl_easy_setopt(slot->curl, CURLOPT_HTTPAUTH, http_auth_methods); - if (http_auth.password || curl_empty_auth_enabled()) - init_curl_http_auth(slot->curl); + + if (http_auth.password || curl_empty_auth_enabled() || proactive_auth) + init_curl_http_auth(slot); return slot; } From patchwork Tue Sep 13 19:25:56 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Matthew John Cheetham X-Patchwork-Id: 12975185 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 84516C54EE9 for ; Tue, 13 Sep 2022 19:26:24 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229547AbiIMT0W (ORCPT ); Tue, 13 Sep 2022 15:26:22 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:50484 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229498AbiIMT0J (ORCPT ); Tue, 13 Sep 2022 15:26:09 -0400 Received: from mail-wr1-x432.google.com (mail-wr1-x432.google.com [IPv6:2a00:1450:4864:20::432]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id B83FB7963C for ; Tue, 13 Sep 2022 12:26:07 -0700 (PDT) Received: by mail-wr1-x432.google.com with SMTP id g3so1475014wrq.13 for ; Tue, 13 Sep 2022 12:26:07 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=cc:to:mime-version:content-transfer-encoding:fcc:subject:date:from :references:in-reply-to:message-id:from:to:cc:subject:date; bh=c+t8BXpaVe7agqU9QWWg3rEcJMGswM19s/qehasi/Dg=; b=W/AuDUcQoSKNwmJ0aAhXav075W0fbl87e1iZje3bsGthCQZt1SId/tcUFGWGIa0HnK eiQR2Q/vAJjbqkUG5hTnkXJ6rWJQjZUDZESpMgVJrqGcgU5isMX1e1C87eLfDpBeXFsq Ry6rVfZKTB1PBbNQ9AHPpJjvg1Y5oPkHRazqB9Y+Z8Qw/p9Lf3x+KzOoREzZsgNca2Pa jMzdqIgePT2HqkL6ZQSMMplqgW09X5vyQH7KMPG25dGYb630r9Cz0sI2/sq28PRoCmrt wXfTlskJBuoBP9fMeUR3gVNmjP/XbxPtV9d/sqSU80GfZaLnkwmn/Q/lzh5wDq+VvZcO SCFw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=cc:to:mime-version:content-transfer-encoding:fcc:subject:date:from :references:in-reply-to:message-id:x-gm-message-state:from:to:cc :subject:date; bh=c+t8BXpaVe7agqU9QWWg3rEcJMGswM19s/qehasi/Dg=; b=NKVZOWtJ5orqrSdlPA3/0NsDzHIoq6x7Y8oxy52jRWoEGOgLQrQbPfaKHIvByk8L5l dtyRPUL0xilOtke5qnLRrKdcnilZAtI/pBXK0EO9wpswf15j69R50z+ZCsL1yuqyBhYc 7wJvGLOvn2uL2Bt2fekc6G1S73b97IXhpXAAyn4Y8weYN/AcftCgEUmNGg2ORKXQVRHY USi/oSNQP+Drs1oSwEy6221T8YYO+2E6Ldw9XHDIFXSmgAlGGwbuwRjFDQyB6v9gIYtp 5g0DnXd40HPrvUq6ct/LKUkOFUXi+gc5rK7ZthU/KPGmj5/oqzh4t4z6AdPM/Gb4bdxG n04Q== X-Gm-Message-State: ACgBeo2EH+4tMPWoXEt2ehj5g436vYOIbVlahBFWXzXifVYVlgvOWHHG uf+JVHeGTnU64bqF0Thb07QjMB22+uQ= X-Google-Smtp-Source: AA6agR7sWgoDj2KvU9R/1/DFEDgOKyRAToPPeyLnGUYrZhMAr9v3/GSaeW3XVrTsDtlytCLsxlrX0Q== X-Received: by 2002:adf:fbd2:0:b0:21e:e983:165f with SMTP id d18-20020adffbd2000000b0021ee983165fmr19087841wrs.517.1663097165860; Tue, 13 Sep 2022 12:26:05 -0700 (PDT) Received: from [127.0.0.1] ([13.74.141.28]) by smtp.gmail.com with ESMTPSA id g7-20020adfa487000000b0022917d58603sm11562077wrb.32.2022.09.13.12.26.04 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 13 Sep 2022 12:26:05 -0700 (PDT) Message-Id: <7f827067f55d596284eb2ad764e59d402c75be18.1663097156.git.gitgitgadget@gmail.com> In-Reply-To: References: Date: Tue, 13 Sep 2022 19:25:56 +0000 Subject: [PATCH 8/8] http: set specific auth scheme depending on credential Fcc: Sent MIME-Version: 1.0 To: git@vger.kernel.org Cc: Matthew John Cheetham , Matthew John Cheetham Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org From: Matthew John Cheetham From: Matthew John Cheetham Introduce a new credential field `authtype` that can be used by credential helpers to indicate the type of the credential or authentication mechanism to use for a request. Modify http.c to now specify the correct authentication scheme or credential type when authenticating the curl handle. If the new `authtype` field in the credential structure is `NULL` or "Basic" then use the existing username/password options. If the field is "Bearer" then use the OAuth bearer token curl option. Otherwise, the `authtype` field is the authentication scheme and the `password` field is the raw, unencoded value. Signed-off-by: Matthew John Cheetham --- Documentation/git-credential.txt | 9 +++++++++ credential.c | 5 +++++ credential.h | 1 + git-curl-compat.h | 7 +++++++ http.c | 24 +++++++++++++++++++++--- 5 files changed, 43 insertions(+), 3 deletions(-) diff --git a/Documentation/git-credential.txt b/Documentation/git-credential.txt index 7d4a788c63d..3b6ef6f4906 100644 --- a/Documentation/git-credential.txt +++ b/Documentation/git-credential.txt @@ -152,6 +152,15 @@ Git understands the following attributes: `protocol=https` and `host=example.com` had been provided). This can help callers avoid parsing URLs themselves. +`authtype`:: + + Indicates the type of authentication scheme used. If this is not + present the default is "Basic". + Known values include "Basic", "Digest", and "Bearer". + If an unknown value is provided, this is taken as the authentication + scheme for the `Authorization` header, and the `password` field is + used as the raw unencoded authorization parameters of the same header. + `wwwauth[n]`:: When an HTTP response is received that includes one or more diff --git a/credential.c b/credential.c index 4ad40323fc7..9d4a0f3fd51 100644 --- a/credential.c +++ b/credential.c @@ -21,6 +21,7 @@ void credential_clear(struct credential *c) free(c->path); free(c->username); free(c->password); + free(c->authtype); string_list_clear(&c->helpers, 0); strvec_clear(&c->wwwauth_headers); @@ -235,6 +236,9 @@ int credential_read(struct credential *c, FILE *fp) } else if (!strcmp(key, "path")) { free(c->path); c->path = xstrdup(value); + } else if (!strcmp(key, "authtype")) { + free(c->authtype); + c->authtype = xstrdup(value); } else if (!strcmp(key, "url")) { credential_from_url(c, value); } else if (!strcmp(key, "quit")) { @@ -281,6 +285,7 @@ void credential_write(const struct credential *c, FILE *fp) credential_write_item(fp, "path", c->path, 0); credential_write_item(fp, "username", c->username, 0); credential_write_item(fp, "password", c->password, 0); + credential_write_item(fp, "authtype", c->authtype, 0); credential_write_strvec(fp, "wwwauth", &c->wwwauth_headers); } diff --git a/credential.h b/credential.h index 6a9d4e3de07..a6572aacf1d 100644 --- a/credential.h +++ b/credential.h @@ -135,6 +135,7 @@ struct credential { char *protocol; char *host; char *path; + char *authtype; }; #define CREDENTIAL_INIT { \ diff --git a/git-curl-compat.h b/git-curl-compat.h index 56a83b6bbd8..74732500a9f 100644 --- a/git-curl-compat.h +++ b/git-curl-compat.h @@ -126,4 +126,11 @@ #define GIT_CURL_HAVE_CURLSSLSET_NO_BACKENDS #endif +/** + * CURLAUTH_BEARER was added in 7.61.0, released in July 2018. + */ +#if LIBCURL_VERSION_NUM >= 0x073D00 +#define GIT_CURL_HAVE_CURLAUTH_BEARER +#endif + #endif diff --git a/http.c b/http.c index 8e107ff19b8..d8913b2c641 100644 --- a/http.c +++ b/http.c @@ -516,7 +516,8 @@ static int curl_empty_auth_enabled(void) static void init_curl_http_auth(struct active_request_slot *slot) { - if (!http_auth.username || !*http_auth.username) { + if (!http_auth.authtype && + (!http_auth.username || !*http_auth.username)) { if (curl_empty_auth_enabled()) curl_easy_setopt(slot->curl, CURLOPT_USERPWD, ":"); return; @@ -524,8 +525,25 @@ static void init_curl_http_auth(struct active_request_slot *slot) credential_fill(&http_auth); - curl_easy_setopt(slot->curl, CURLOPT_USERNAME, http_auth.username); - curl_easy_setopt(slot->curl, CURLOPT_PASSWORD, http_auth.password); + if (!http_auth.authtype || !strcasecmp(http_auth.authtype, "basic") + || !strcasecmp(http_auth.authtype, "digest")) { + curl_easy_setopt(slot->curl, CURLOPT_USERNAME, + http_auth.username); + curl_easy_setopt(slot->curl, CURLOPT_PASSWORD, + http_auth.password); +#ifdef GIT_CURL_HAVE_CURLAUTH_BEARER + } else if (!strcasecmp(http_auth.authtype, "bearer")) { + curl_easy_setopt(slot->curl, CURLOPT_HTTPAUTH, CURLAUTH_BEARER); + curl_easy_setopt(slot->curl, CURLOPT_XOAUTH2_BEARER, + http_auth.password); +#endif + } else { + struct strbuf auth = STRBUF_INIT; + strbuf_addf(&auth, "Authorization: %s %s", + http_auth.authtype, http_auth.password); + slot->headers = curl_slist_append(slot->headers, auth.buf); + strbuf_release(&auth); + } } /* *var must be free-able */