From patchwork Thu Dec 14 14:13:42 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jiang Xin X-Patchwork-Id: 13493112 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="QHVnJWwx" Received: from mail-pg1-x531.google.com (mail-pg1-x531.google.com [IPv6:2607:f8b0:4864:20::531]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id C9185A7 for ; Thu, 14 Dec 2023 06:13:49 -0800 (PST) Received: by mail-pg1-x531.google.com with SMTP id 41be03b00d2f7-5ca29c131ebso3300205a12.0 for ; Thu, 14 Dec 2023 06:13:49 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1702563229; x=1703168029; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=adGc6EGVmjIPNKTg+encN8li33DZ1VDLOfTu31s9f6E=; b=QHVnJWwx8wm17GV5IXYZcexRW5Wrouhe81fohmUM2kyiHMtdqcW2DX4kwLdoe95yCl yQMWWe2D23HZx8huxoq3GeRqVwwJUWTuxQFv9Ln6Go3AHhOqK0ThoSN/5sjvhTR4lJHl Oig+YhTZwabsOFP3temMXclVL5yogAjM18saCi+HDZGiY0GMN6Sthz1NnfA2KLwt5l6A J53uYRbozP7VeSQ1Vg9VHqN5jxnk7r8973wj75r/oIjocG1vGlgXN0Jkhe49lnvxX3V8 cX0hY9SJbkOXxoxvbb/lw4OqINg9OadCCpHCe9DeXsBwjg/8R/JO/L+FPFEmOgtoPfJa 9m3g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1702563229; x=1703168029; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=adGc6EGVmjIPNKTg+encN8li33DZ1VDLOfTu31s9f6E=; b=EXyoi7jAtvFuVreTK9M1/v5fhb8N/ejt5TLz7ROosbMPAXTtcOTDvj8gJEnF59ZyXX rQsgxtASg+y/lVIv0pX1S6ZrloAsV7/oe781JDYsbSyJqMjD5bYnn3Sdd7q19TBYw0gf RKY44BzvrDRJP/Mev1xSXXvze8lQmcGDhXvhL4slRUJQYmz/QaLEQPmVaazY6FQrwjw0 +ZpJ0q9UGwFiOII20+0YJ+2Tx2KUKaEwp8rqig2GYrRtwUJtmfhyQlm6cgyrxEcwmDkI lXePW1lNfLckmp9IaAFgeYu20OMoEcILHUEDuDWPTRBKLNBjEI2v3QoadFqQ7SLA3jnU ykrw== X-Gm-Message-State: AOJu0Ywq889ppEZMfXuu4sF2QLLWT6KQKMypY27EsPPow5AFwE2IAULw C6nsp7sXX+z+ZQ523cf9rtd/6H1INmQ= X-Google-Smtp-Source: AGHT+IFuaaHA+vwUPKzLEFhKLRkL4LQyjQQBD3ggFBrKSEb7+SuYzV591X4ia3achs+t+g/rCddGcw== X-Received: by 2002:a05:6a20:7f9b:b0:18f:97c:8a3b with SMTP id d27-20020a056a207f9b00b0018f097c8a3bmr13662317pzj.102.1702563229009; Thu, 14 Dec 2023 06:13:49 -0800 (PST) Received: from tigtog-proxy.localdomain.localdomain (144.34.163.219.16clouds.com. [144.34.163.219]) by smtp.gmail.com with ESMTPSA id 16-20020a056a00071000b006ce921da705sm11806232pfl.136.2023.12.14.06.13.48 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Thu, 14 Dec 2023 06:13:48 -0800 (PST) From: Jiang Xin To: Git List , Junio C Hamano Cc: Jiang Xin Subject: [PATCH v4 1/4] transport-helper: no connection restriction in connect_helper Date: Thu, 14 Dec 2023 22:13:42 +0800 Message-Id: X-Mailer: git-send-email 2.32.0.rc3 In-Reply-To: References: Precedence: bulk X-Mailing-List: git@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 From: Jiang Xin When commit b236752a (Support remote archive from all smart transports, 2009-12-09) added "remote archive" support for "smart transports", it was for transport that supports the ".connect" method. The "connect_helper()" function protected itself from getting called for a transport without the method before calling process_connect_service(), which did not work with such a transport. Later, commit edc9caf7 (transport-helper: introduce stateless-connect, 2018-03-15) added a way for a transport without the ".connect" method to establish a "stateless" connection in protocol-v2, which process_connect_service() was taught to handle the "stateless" connection, making the old safety valve in its caller that insisted that ".connect" method must be defined too strict, and forgot to loosen it. Remove the restriction in the "connect_helper()" function and give the function "process_connect_service()" the opportunity to establish a connection using ".connect" or ".stateless_connect" for protocol v2. So we can connect with a stateless-rpc and do something useful. E.g., in a later commit, implements remote archive for a repository over HTTP protocol. Helped-by: Junio C Hamano Signed-off-by: Jiang Xin --- transport-helper.c | 2 -- 1 file changed, 2 deletions(-) diff --git a/transport-helper.c b/transport-helper.c index 49811ef176..2e127d24a5 100644 --- a/transport-helper.c +++ b/transport-helper.c @@ -662,8 +662,6 @@ static int connect_helper(struct transport *transport, const char *name, /* Get_helper so connect is inited. */ get_helper(transport); - if (!data->connect) - die(_("operation not supported by protocol")); if (!process_connect_service(transport, name, exec)) die(_("can't connect to subservice %s"), name); From patchwork Thu Dec 14 14:13:43 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jiang Xin X-Patchwork-Id: 13493113 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="bMBppE0u" Received: from mail-pg1-x52c.google.com (mail-pg1-x52c.google.com [IPv6:2607:f8b0:4864:20::52c]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 988F4AF for ; Thu, 14 Dec 2023 06:13:50 -0800 (PST) Received: by mail-pg1-x52c.google.com with SMTP id 41be03b00d2f7-5c229dabbb6so4439018a12.0 for ; Thu, 14 Dec 2023 06:13:50 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1702563230; x=1703168030; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=2xAmt5WN8yEdP5hRVdxogyrRphFntMC1QeO0Ts32bmE=; b=bMBppE0ulPdRiYwQTifs/HuVy218Y84+yuf/IukzNr10AXBXPr+iLB4ie+lkl/rS/b 6hKHsWfysYpbVhmdLSBEd/kRFtKfprTg8fjrBReiJQXF3Q4Y9hwCq4dIICAVTj0+SGDW /WPxI4wiVCuySKnT144jK/TnWre8dLWnBMgRiHrCcmcP1pTGCF8txLN/NVePpR7/LuyZ /qbqSod3ZkxzXA61+kJssYVIqUI1/YTEefWmi/m4osRnlbKZ1dolp2z8jDARDL4gmgoQ 1RRAouRrWeBtHtATKRGdIucIo+MXr0EZD/pwZa6FIQCOn7KOJeA3i2IOdd6muqriWOPD M6zg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1702563230; x=1703168030; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=2xAmt5WN8yEdP5hRVdxogyrRphFntMC1QeO0Ts32bmE=; b=XqM2E+PYWwDHp6qmjdL4/1r9kN/YHj+w9NzKR9klB+ZEbBrFEdZnGksFblAF2uB/qA 1DF0liTvmlLi3rPqAxoqoenMHDG5rMvR6Rnx6fMPvllaSh5MT3s+GaOJuh4OSURYbAco lYWm/gBefc/glzPWcEKYoBiEjY/MGB9Vgj7GFemTuq6z2weETo5lmVvu/jCEyCxm7SJa 99EOtmEW9bnDhMv+12PPH2lpmI6pZqWmpnSIUvUaq/6x+1iGwt9NWWQsqFz9uBy0lT5B YNS6G7b/btG7yciLVWHu2D79u66VgTHvgXX74O5UPe3s/eICVmHb6Szdq/sdtvrMisih 4DUA== X-Gm-Message-State: AOJu0Yyl30i2H9hOpSZQDSp6TKxfeC+KlYMMQnCg23tuOObo0T/mxR+P wrA+uG8knJVzAk8AGUSbnb9fjV5S70o= X-Google-Smtp-Source: AGHT+IGUX7+FB9oB3RXhs0BUL9KeUvCLyyrnZyKnHvzMg15PgT9wNRO2GAHz4vGl2lpcjMzr4loI5g== X-Received: by 2002:a05:6a20:734f:b0:18c:5795:669e with SMTP id v15-20020a056a20734f00b0018c5795669emr5187137pzc.19.1702563229780; Thu, 14 Dec 2023 06:13:49 -0800 (PST) Received: from tigtog-proxy.localdomain.localdomain (144.34.163.219.16clouds.com. [144.34.163.219]) by smtp.gmail.com with ESMTPSA id 16-20020a056a00071000b006ce921da705sm11806232pfl.136.2023.12.14.06.13.49 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Thu, 14 Dec 2023 06:13:49 -0800 (PST) From: Jiang Xin To: Git List , Junio C Hamano Cc: Jiang Xin Subject: [PATCH v4 2/4] transport-helper: call do_take_over() in process_connect Date: Thu, 14 Dec 2023 22:13:43 +0800 Message-Id: <65fb67523c5c052fae466cbd8ce966e0f6265297.1702562879.git.zhiyou.jx@alibaba-inc.com> X-Mailer: git-send-email 2.32.0.rc3 In-Reply-To: References: Precedence: bulk X-Mailing-List: git@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 From: Jiang Xin The existing pattern among all callers of process_connect() seems to be if (process_connect(...)) { do_take_over(); ... dispatch to the underlying method ... } ... otherwise implement the fallback ... where the return value from process_connect() is the return value of the call it makes to process_connect_service(). Move the call of do_take_over() inside process_connect(), so that calling the process_connect() function is more concise and will not miss do_take_over(). Suggested-by: Junio C Hamano Signed-off-by: Jiang Xin --- transport-helper.c | 22 +++++++++------------- 1 file changed, 9 insertions(+), 13 deletions(-) diff --git a/transport-helper.c b/transport-helper.c index 2e127d24a5..51088cc03a 100644 --- a/transport-helper.c +++ b/transport-helper.c @@ -645,6 +645,7 @@ static int process_connect(struct transport *transport, struct helper_data *data = transport->data; const char *name; const char *exec; + int ret; name = for_push ? "git-receive-pack" : "git-upload-pack"; if (for_push) @@ -652,7 +653,10 @@ static int process_connect(struct transport *transport, else exec = data->transport_options.uploadpack; - return process_connect_service(transport, name, exec); + ret = process_connect_service(transport, name, exec); + if (ret) + do_take_over(transport); + return ret; } static int connect_helper(struct transport *transport, const char *name, @@ -682,10 +686,8 @@ static int fetch_refs(struct transport *transport, get_helper(transport); - if (process_connect(transport, 0)) { - do_take_over(transport); + if (process_connect(transport, 0)) return transport->vtable->fetch_refs(transport, nr_heads, to_fetch); - } /* * If we reach here, then the server, the client, and/or the transport @@ -1142,10 +1144,8 @@ static int push_refs(struct transport *transport, { struct helper_data *data = transport->data; - if (process_connect(transport, 1)) { - do_take_over(transport); + if (process_connect(transport, 1)) return transport->vtable->push_refs(transport, remote_refs, flags); - } if (!remote_refs) { fprintf(stderr, @@ -1186,11 +1186,9 @@ static struct ref *get_refs_list(struct transport *transport, int for_push, { get_helper(transport); - if (process_connect(transport, for_push)) { - do_take_over(transport); + if (process_connect(transport, for_push)) return transport->vtable->get_refs_list(transport, for_push, transport_options); - } return get_refs_list_using_list(transport, for_push); } @@ -1274,10 +1272,8 @@ static int get_bundle_uri(struct transport *transport) { get_helper(transport); - if (process_connect(transport, 0)) { - do_take_over(transport); + if (process_connect(transport, 0)) return transport->vtable->get_bundle_uri(transport); - } return -1; } From patchwork Thu Dec 14 14:13:44 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jiang Xin X-Patchwork-Id: 13493114 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="UXe/NTo4" Received: from mail-pf1-x431.google.com (mail-pf1-x431.google.com [IPv6:2607:f8b0:4864:20::431]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 7560ACF for ; Thu, 14 Dec 2023 06:13:51 -0800 (PST) Received: by mail-pf1-x431.google.com with SMTP id d2e1a72fcca58-6d0985c70ffso2951655b3a.2 for ; Thu, 14 Dec 2023 06:13:51 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1702563231; x=1703168031; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=Xaz8QUNp74xM8bl73p4VvtgDU7aS99bbuavivXhTyUs=; b=UXe/NTo4TL98JG80/lOi/Cq5NXKJ4PSEw+hEyG3mus1I4EjE1T5mNYKfSsFi50Ae1S Wr2++48nenyi2LAl3KOPHd5tKR5SwjXRUIGKBvogGNBD0VeVWrr3swrjxO7f9KbDPByB RO1lqIRIurqn0jEQk7p4EnbaEzv6jofSbJIVCqXj0nGo2yuMjmp7Jrw5BfILxWXOlv2P OLDQodq0mLNeqQF+WMXp+7moZvDLd7HyKLF/Kk/yl2g8JVLH3jZA4xQCIidNzenJwcyi +Q/34Gv3EwyZ2vcM9WoDkVZ2Gt6Ev7bR7Z9XFm/idbPbLDtQwBuFsFmkp7LjgNH1d79R cAQA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1702563231; x=1703168031; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=Xaz8QUNp74xM8bl73p4VvtgDU7aS99bbuavivXhTyUs=; b=bOx5tRs2Wl3Wo7N4ZDMWA2UHV+eZM8kpYLl+LEI8uNQeWtsUefDYG3eIeJaDAIhNYJ DWI9oO/ycNfqTg2yCsx7sLXY3+jV7MpvfEZzBenx36XmTuDnRu+4bg4FeeooSQJ4uSvC Tm5JaifEHQiz/C8CDeTzX2+VpO4RFf/WzxOK9QwOBRq9BCTV7c8GcE/pZJu0mwAOzoFX ryyTqcss1qjoDyBCIHkWew1F+QG3T5nFxbiMt/RPaVTNimfYw1EZU/MRzJLfPjj7OwJq +kGoSSVQhJZYkAFoOJmfsZBqs5sicXWhCvzZUaecWfF+Jm063aSaKVuGZaCpXz8+YIs+ 5gwA== X-Gm-Message-State: AOJu0YwaF61NDuJN/+nyZPL2ct23iOiNkw2dPaqN9EF5mvD822+8G/lH IqKlVm2EEiRtUuyGUf+izU2Iidpb51s= X-Google-Smtp-Source: AGHT+IGlNh7rKgAmQCwl9B+HJG9L0ufQVdYmrvqiuO9CAbA0kW0CiDWYiB0lD1yEgQcemyhxmu/ZKg== X-Received: by 2002:aa7:830f:0:b0:6ce:558e:de91 with SMTP id bk15-20020aa7830f000000b006ce558ede91mr4537557pfb.46.1702563230666; Thu, 14 Dec 2023 06:13:50 -0800 (PST) Received: from tigtog-proxy.localdomain.localdomain (144.34.163.219.16clouds.com. [144.34.163.219]) by smtp.gmail.com with ESMTPSA id 16-20020a056a00071000b006ce921da705sm11806232pfl.136.2023.12.14.06.13.49 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Thu, 14 Dec 2023 06:13:50 -0800 (PST) From: Jiang Xin To: Git List , Junio C Hamano Cc: Jiang Xin Subject: [PATCH v4 3/4] transport-helper: call do_take_over() in connect_helper Date: Thu, 14 Dec 2023 22:13:44 +0800 Message-Id: X-Mailer: git-send-email 2.32.0.rc3 In-Reply-To: References: Precedence: bulk X-Mailing-List: git@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 From: Jiang Xin After successfully connecting to the smart transport by calling process_connect_service() in connect_helper(), run do_take_over() to replace the old vtable with a new one which has methods ready for the smart transport connection. The connect_helper() function is used as the connect method of the vtable in "transport-helper.c", and it is called by transport_connect() in "transport.c" to setup a connection. The only place that we call transport_connect() so far is in "builtin/archive.c". Without running do_take_over(), it may fail to call transport_disconnect() in run_remote_archiver() of "builtin/archive.c". This is because for a stateless connection or a service like "git-upload-pack-archive", the remote helper may receive a SIGPIPE signal and exit early. To have a graceful disconnect method by calling do_take_over() will solve this issue. The subsequent commit will introduce remote archive over a stateless-rpc connection. Signed-off-by: Jiang Xin --- transport-helper.c | 2 ++ 1 file changed, 2 insertions(+) diff --git a/transport-helper.c b/transport-helper.c index 51088cc03a..3b036ae1ca 100644 --- a/transport-helper.c +++ b/transport-helper.c @@ -672,6 +672,8 @@ static int connect_helper(struct transport *transport, const char *name, fd[0] = data->helper->out; fd[1] = data->helper->in; + + do_take_over(transport); return 0; } From patchwork Thu Dec 14 14:13:45 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jiang Xin X-Patchwork-Id: 13493115 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="FBoDqtv9" Received: from mail-pf1-x436.google.com (mail-pf1-x436.google.com [IPv6:2607:f8b0:4864:20::436]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 49FCF9C for ; Thu, 14 Dec 2023 06:13:52 -0800 (PST) Received: by mail-pf1-x436.google.com with SMTP id d2e1a72fcca58-6ce72faf1e8so5182826b3a.0 for ; Thu, 14 Dec 2023 06:13:52 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1702563231; x=1703168031; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=d5buL3oD1qUEiKjt8Q2ZKcp2Nr+bw2svcEJVPZusC2g=; b=FBoDqtv9eyjweC5JPuyjg3LsoscmMZU4dq5IR3MEI5KwgDm9FTlGV4KGBiLzCqiGZZ ioHgDqAF0jcGrkOXucPtDCmhO4D9L2zF2GS3JayrAgUbMGJSMsd4EDmEDCytRPHHy4h4 g1IRXRFP+mQbl1k2er0stXPab/meObq+7X9+LgrVdj6gF2/4zWsMxuG4Pgk5FyxQAM/G fv/MJ2ZCKriG3C0nFtRUG2Mr/sxde7Rj2ELdw8JtPEgVnk78CfGiCi3Zhs+IBw0o8xvq GCZvCnYqFeYS9fTEHnZM4QVzmvLirE3QcsagfXMA9+XLuL8135sjccytn0+oz9Vf3+dG Ctjw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1702563231; x=1703168031; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=d5buL3oD1qUEiKjt8Q2ZKcp2Nr+bw2svcEJVPZusC2g=; b=PLErYkbJPMN0UwVZAKBpHmtH/fD6WWi7SB2Mi2wTppV05zmSZFqEHUK/X/FUq+Z66b fkbRuGqdlO/LgOzHt9ggpD1u4Eu2/GkD15S1UzH1eLuXnKNbNyznADN3JYYG2sWyMgo0 KdZ+gShYg+mZ3f05EcZdirFjuPHRz1fvyyhk3EPfZc5GFML3vy/9DO1CNNLYdo+tgErn q8ighbyfBUkuFn20um8Onzt+DYRe9GzTuUqyhneibSo4BxdHaMQYDb4U8GuWfTJtDE+X qUPMj9HnJ7bT1OQjAFm0NAAofTgn5eAxZPZgjlHKhdgLSbpsNgGf4V6csiZ1ZLXObt+y l5Vw== X-Gm-Message-State: AOJu0YyqbZqAWXtPAEk+x06YpwJJx5GgjSV3wjx1aIf/13gRbIUiBYDf iew76c5pOJ6DeU4HRblzTWBikBI2OC0= X-Google-Smtp-Source: AGHT+IFJKDGtUPttmdrP4juV5iHveS5i/MvwVIQzqLIm8D4Nm0Bjp4z2XQeI+AeLwXG94a12u2wWMQ== X-Received: by 2002:a05:6a00:230a:b0:6ce:f78e:1054 with SMTP id h10-20020a056a00230a00b006cef78e1054mr5572364pfh.14.1702563231465; Thu, 14 Dec 2023 06:13:51 -0800 (PST) Received: from tigtog-proxy.localdomain.localdomain (144.34.163.219.16clouds.com. [144.34.163.219]) by smtp.gmail.com with ESMTPSA id 16-20020a056a00071000b006ce921da705sm11806232pfl.136.2023.12.14.06.13.50 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Thu, 14 Dec 2023 06:13:51 -0800 (PST) From: Jiang Xin To: Git List , Junio C Hamano Cc: Jiang Xin , Eric Sunshine Subject: [PATCH v4 4/4] archive: support remote archive from stateless transport Date: Thu, 14 Dec 2023 22:13:45 +0800 Message-Id: <18b9a11d3be9d804e8d22d054ea881b8336d170c.1702562879.git.zhiyou.jx@alibaba-inc.com> X-Mailer: git-send-email 2.32.0.rc3 In-Reply-To: References: Precedence: bulk X-Mailing-List: git@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 From: Jiang Xin Even though we can establish a stateless connection, we still cannot archive the remote repository using a stateless HTTP protocol. Try the following steps to make it work. 1. Add support for "git-upload-archive" service in "http-backend". 2. Use the URL ".../info/refs?service=git-upload-pack" to detect the protocol version, instead of use the "git-upload-archive" service. 3. "git-archive" does not expect to see protocol version and capabilities when connecting to remote-helper, so do not send them in "remote-curl.c" for the "git-upload-archive" service. Helped-by: Eric Sunshine Signed-off-by: Jiang Xin --- http-backend.c | 15 +++++++++++---- remote-curl.c | 14 +++++++++++--- t/t5003-archive-zip.sh | 34 ++++++++++++++++++++++++++++++++++ transport-helper.c | 3 ++- 4 files changed, 58 insertions(+), 8 deletions(-) diff --git a/http-backend.c b/http-backend.c index ff07b87e64..6a2c919839 100644 --- a/http-backend.c +++ b/http-backend.c @@ -38,6 +38,7 @@ struct rpc_service { static struct rpc_service rpc_service[] = { { "upload-pack", "uploadpack", 1, 1 }, { "receive-pack", "receivepack", 0, -1 }, + { "upload-archive", "uploadarchive", 0, -1 }, }; static struct string_list *get_parameters(void) @@ -639,10 +640,15 @@ static void check_content_type(struct strbuf *hdr, const char *accepted_type) static void service_rpc(struct strbuf *hdr, char *service_name) { - const char *argv[] = {NULL, "--stateless-rpc", ".", NULL}; + struct strvec argv = STRVEC_INIT; struct rpc_service *svc = select_service(hdr, service_name); struct strbuf buf = STRBUF_INIT; + strvec_push(&argv, svc->name); + if (strcmp(service_name, "git-upload-archive")) + strvec_push(&argv, "--stateless-rpc"); + strvec_push(&argv, "."); + strbuf_reset(&buf); strbuf_addf(&buf, "application/x-git-%s-request", svc->name); check_content_type(hdr, buf.buf); @@ -655,9 +661,9 @@ static void service_rpc(struct strbuf *hdr, char *service_name) end_headers(hdr); - argv[0] = svc->name; - run_service(argv, svc->buffer_input); + run_service(argv.v, svc->buffer_input); strbuf_release(&buf); + strvec_clear(&argv); } static int dead; @@ -723,7 +729,8 @@ static struct service_cmd { {"GET", "/objects/pack/pack-[0-9a-f]{64}\\.idx$", get_idx_file}, {"POST", "/git-upload-pack$", service_rpc}, - {"POST", "/git-receive-pack$", service_rpc} + {"POST", "/git-receive-pack$", service_rpc}, + {"POST", "/git-upload-archive$", service_rpc} }; static int bad_request(struct strbuf *hdr, const struct service_cmd *c) diff --git a/remote-curl.c b/remote-curl.c index ef05752ca5..ce6cb8ac05 100644 --- a/remote-curl.c +++ b/remote-curl.c @@ -1447,8 +1447,14 @@ static int stateless_connect(const char *service_name) * establish a stateless connection, otherwise we need to tell the * client to fallback to using other transport helper functions to * complete their request. + * + * The "git-upload-archive" service is a read-only operation. Fallback + * to use "git-upload-pack" service to discover protocol version. */ - discover = discover_refs(service_name, 0); + if (!strcmp(service_name, "git-upload-archive")) + discover = discover_refs("git-upload-pack", 0); + else + discover = discover_refs(service_name, 0); if (discover->version != protocol_v2) { printf("fallback\n"); fflush(stdout); @@ -1486,9 +1492,11 @@ static int stateless_connect(const char *service_name) /* * Dump the capability listing that we got from the server earlier - * during the info/refs request. + * during the info/refs request. This does not work with the + * "git-upload-archive" service. */ - write_or_die(rpc.in, discover->buf, discover->len); + if (strcmp(service_name, "git-upload-archive")) + write_or_die(rpc.in, discover->buf, discover->len); /* Until we see EOF keep sending POSTs */ while (1) { diff --git a/t/t5003-archive-zip.sh b/t/t5003-archive-zip.sh index fc499cdff0..961c6aac25 100755 --- a/t/t5003-archive-zip.sh +++ b/t/t5003-archive-zip.sh @@ -239,4 +239,38 @@ check_zip with_untracked2 check_added with_untracked2 untracked one/untracked check_added with_untracked2 untracked two/untracked +# Test remote archive over HTTP protocol. +# +# Note: this should be the last part of this test suite, because +# by including lib-httpd.sh, the test may end early if httpd tests +# should not be run. +# +. "$TEST_DIRECTORY"/lib-httpd.sh +start_httpd + +test_expect_success "setup for HTTP protocol" ' + cp -R bare.git "$HTTPD_DOCUMENT_ROOT_PATH/bare.git" && + git -C "$HTTPD_DOCUMENT_ROOT_PATH/bare.git" \ + config http.uploadpack true && + set_askpass user@host pass@host +' + +setup_askpass_helper + +test_expect_success 'remote archive does not work with protocol v1' ' + test_must_fail git -c protocol.version=1 archive \ + --remote="$HTTPD_URL/auth/smart/bare.git" \ + --output=remote-http.zip HEAD >actual 2>&1 && + cat >expect <<-EOF && + fatal: can${SQ}t connect to subservice git-upload-archive + EOF + test_cmp expect actual +' + +test_expect_success 'archive remote http repository' ' + git archive --remote="$HTTPD_URL/auth/smart/bare.git" \ + --output=remote-http.zip HEAD && + test_cmp_bin d.zip remote-http.zip +' + test_done diff --git a/transport-helper.c b/transport-helper.c index 3b036ae1ca..566f7473df 100644 --- a/transport-helper.c +++ b/transport-helper.c @@ -628,7 +628,8 @@ static int process_connect_service(struct transport *transport, ret = run_connect(transport, &cmdbuf); } else if (data->stateless_connect && (get_protocol_version_config() == protocol_v2) && - !strcmp("git-upload-pack", name)) { + (!strcmp("git-upload-pack", name) || + !strcmp("git-upload-archive", name))) { strbuf_addf(&cmdbuf, "stateless-connect %s\n", name); ret = run_connect(transport, &cmdbuf); if (ret)