From patchwork Sat Jul 20 03:43:32 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Peijian Ju X-Patchwork-Id: 13737598 Received: from mail-qk1-f178.google.com (mail-qk1-f178.google.com [209.85.222.178]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 4409FC132 for ; Sat, 20 Jul 2024 03:43:55 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.222.178 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1721447036; cv=none; b=FO7kjk+1+NjkaCIU5e0bH/DATKQs/kl5rM4LVUYabQQxNvh0D9GK3mB5Ic7CMU2fHrzq999Q0P1uPivng73jh1ADLk3I/L6uaSX8TtdPhRDB2d5PCAr5z47xcP0FBnd3Bez+QCrH5Uof+cUysx8TXhAwcxpf9+GtcWRtJaDctNg= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1721447036; c=relaxed/simple; bh=68J8rWZytDNPK2UF0u450YUZZpabXDXIT8zbqD/y5nE=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=FStV1OEC+oX/kqKTEhjIfJ3+UhPn3wz1Nkvuwyb6mwnNmqZ0sYvqqL7qBvyk9+ZF45vnBwfbMPd9CR9TpHS3L0CNQYjJdgr8awLRiS7GKLr3uLz8Jj8Xvj+/WE96tQkIvJFIElSlms78UPD1joYAjC7N0n31Icoyza57N81t5tA= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=S4xQ3Gh2; arc=none smtp.client-ip=209.85.222.178 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="S4xQ3Gh2" Received: by mail-qk1-f178.google.com with SMTP id af79cd13be357-79f323f084eso135294385a.3 for ; Fri, 19 Jul 2024 20:43:54 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1721447034; x=1722051834; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=MJvl9xkzpCuXEpujzmUrficdZxEe3PBklsEQh6bXZKE=; b=S4xQ3Gh2FExdgy7YnC1biSNfyaX7sLWj/CDSNmDhiIa4sgzGKG+mpDi+7fmOEGgIte +L8XeKebeo8TonYYd7y2gAEhfhxk8ieUVoCiC6OPvb6aUEHT7Fky0JS3FOMTJOHJmo+L NSHIxHnNUFQS3m0lYvFR/B68/NoKs8MsulpIc5cs6tjiOSA5QxI6UgsRGNmkE00buqQD QGK2FipRFacY5ymeD8a6X/0ijtKV2/Hdfw5cy38sj00qSc2D54qoLUxsQHfTtxOc7wUD F179Rto6V+L7ijNXY7ApjPjno1UT+DYJ7IRHnE40PwMeqW+Vymp+B8Aze5Fa1HPMa3WK kLsQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1721447034; x=1722051834; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=MJvl9xkzpCuXEpujzmUrficdZxEe3PBklsEQh6bXZKE=; b=MLXL/QvuUbY4ngnWk5Oufll4lT7LwGJsFwhife9rCdjwQBKmvqNgiuHfBNnUEYbwwB M4xwGozNVNcofpMZcGZUJMaFaUHUkX6u5UHtwTB4taPTAD7yiqIPXVOgaWkuLDbP8qfp CW522cPI2MDwKSreaWhVKcleNC51tRsKuxrFS0dggx1yfwhI1/o4/rLNRffmnHbwkXwt /zRBCYV996xbaqEtpL3ysWHcoiYo65xMl8Mkq3hH1ExVlU7+XnuhWkTv83VMbtoC8fFR 9DH7h9DYjILJPmjT6/Y9CoiGrV76nlSlq9RhuHJN8ZNqHnh14WuIzRgvoU/nxiD4ksaf VnNw== X-Gm-Message-State: AOJu0Yy+XeWWWRE488qGX3i23htlPHN5pNzjlPVKz3L5ll7x0DLXG4Ag Vz1xxwByWcyHAtmqTeuGMvSCiZLxDo9JrGCyWQVKAOxhEwg3T4z1tW3dM+2OZYE= X-Google-Smtp-Source: AGHT+IG/+/ISRO3yy5FDyVzJMsKFlhdDgK2UIueLgLwaIVJAmWeZZhoFJi7f00Qob9g2h8MhxNz9nA== X-Received: by 2002:a05:620a:2889:b0:79f:41b:aaa8 with SMTP id af79cd13be357-7a1a133d645mr204794185a.29.1721447033758; Fri, 19 Jul 2024 20:43:53 -0700 (PDT) Received: from localhost.localdomain (bras-base-unvlon5586w-grc-14-76-71-94-205.dsl.bell.ca. [76.71.94.205]) by smtp.gmail.com with ESMTPSA id af79cd13be357-7a199073b49sm150487385a.100.2024.07.19.20.43.52 (version=TLS1_3 cipher=TLS_CHACHA20_POLY1305_SHA256 bits=256/256); Fri, 19 Jul 2024 20:43:53 -0700 (PDT) From: Eric Ju To: git@vger.kernel.org Cc: calvinwan@google.com, jonathantanmy@google.com, chriscool@tuxfamily.org, eric.peijian@gmail.com, karthik.188@gmail.com, toon@iotcl.com, jltobler@gmail.com Subject: [PATCH v2 1/6] fetch-pack: refactor packet writing Date: Fri, 19 Jul 2024 23:43:32 -0400 Message-ID: <20240720034337.57125-2-eric.peijian@gmail.com> X-Mailer: git-send-email 2.45.2 In-Reply-To: <20240720034337.57125-1-eric.peijian@gmail.com> References: <20240628190503.67389-1-eric.peijian@gmail.com> <20240720034337.57125-1-eric.peijian@gmail.com> Precedence: bulk X-Mailing-List: git@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 From: Calvin Wan A subsequent patch needs to write capabilities for another command. Refactor write_fetch_command_and_capabilities() to be a more general purpose function write_command_and_capabilities(), so that it can be used by both fetch and future command. Here "command" means the "operations" supported by Git’s wire protocol https://git-scm.com/docs/protocol-v2. An example would be a git's subcommand, such as git-fetch(1); or an operation supported by the server side such as "object-info" implemented in "a2ba162cda (object-info: support for retrieving object info, 2021-04-20)". Helped-by: Jonathan Tan Helped-by: Christian Couder Signed-off-by: Calvin Wan Signed-off-by: Eric Ju --- fetch-pack.c | 12 ++++++------ 1 file changed, 6 insertions(+), 6 deletions(-) diff --git a/fetch-pack.c b/fetch-pack.c index 732511604b..9c8cda0f9e 100644 --- a/fetch-pack.c +++ b/fetch-pack.c @@ -1312,13 +1312,13 @@ static int add_haves(struct fetch_negotiator *negotiator, return haves_added; } -static void write_fetch_command_and_capabilities(struct strbuf *req_buf, - const struct string_list *server_options) +static void write_command_and_capabilities(struct strbuf *req_buf, + const struct string_list *server_options, const char* command) { const char *hash_name; - ensure_server_supports_v2("fetch"); - packet_buf_write(req_buf, "command=fetch"); + ensure_server_supports_v2(command); + packet_buf_write(req_buf, "command=%s", command); if (server_supports_v2("agent")) packet_buf_write(req_buf, "agent=%s", git_user_agent_sanitized()); if (advertise_sid && server_supports_v2("session-id")) @@ -1354,7 +1354,7 @@ static int send_fetch_request(struct fetch_negotiator *negotiator, int fd_out, int done_sent = 0; struct strbuf req_buf = STRBUF_INIT; - write_fetch_command_and_capabilities(&req_buf, args->server_options); + write_command_and_capabilities(&req_buf, args->server_options, "fetch"); if (args->use_thin_pack) packet_buf_write(&req_buf, "thin-pack"); @@ -2172,7 +2172,7 @@ void negotiate_using_fetch(const struct oid_array *negotiation_tips, the_repository, "%d", negotiation_round); strbuf_reset(&req_buf); - write_fetch_command_and_capabilities(&req_buf, server_options); + write_command_and_capabilities(&req_buf, server_options, "fetch"); packet_buf_write(&req_buf, "wait-for-done"); From patchwork Sat Jul 20 03:43:33 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Peijian Ju X-Patchwork-Id: 13737599 Received: from mail-qk1-f179.google.com (mail-qk1-f179.google.com [209.85.222.179]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 2764DC133 for ; Sat, 20 Jul 2024 03:43:55 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.222.179 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1721447037; cv=none; b=hyHk57v2H/a3YQ7EX9/4J2gC9HhFeHTT17V/VugnszMaBVsbkLhVpzxM/6drmL/TkCXr4vCfkZe9osFLPI58mE0Tv6SOq/Mnz805szSzyvsLHr561Nws6k6uX/dFiKaHbp6HqhF0KkSwVu6VhINRP/nTJcQvh6vXDHYdTb/Ch2Q= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1721447037; c=relaxed/simple; bh=HWeG/ivjt1g5ClaOquTXlrAteavQUo29TeRLt8oSP4g=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=WE7ZGCD4I0QQA/bGficsCRzo/7U9nkIpr/NNeGv+Fsr5dq4PXp3UwAk/RTQByUZgOd+gIJkjiDMvHWk4wVJXtH/ulee/898AFMfnrnwDHTSht8sdfnR0cWxzZa+72NvwgotMHNUHHJZkptN4q4DVKUoXhUvy16Fkz80lxZrDHeo= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=fKkKfZji; arc=none smtp.client-ip=209.85.222.179 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="fKkKfZji" Received: by mail-qk1-f179.google.com with SMTP id af79cd13be357-79ef72bb8c8so101297385a.2 for ; Fri, 19 Jul 2024 20:43:55 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1721447035; x=1722051835; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=zXJpSg21NiWZjerNEeO54cx2aELRqlPQgLqFicUJJKQ=; b=fKkKfZji6o522Rd1Pm0+/IS81Ex6egKdn7itoV2UWdqnPayqvh23BJqcGAOSvw2uL4 18GSxIS+2UFoTKNSRfYcZmrL5Wo6ALrrJ2P9DK251nBhjgCRK9vBofofJDr/qV6pmJGR Hw/omeL+XwJLhBDYTihXMYI83yhZ9pJ5IAU+JKQm1Jvgzh4ReKH9y6Hc0Y9W+W3SSxio kCKPv/M+v6KMJd07mhEThLFi6CAH9tEx5mq/SV5vyh830pc0Ox7gdaQIXl6luanfKGBl T3pKcoRpR37cSOUsiRR4FehXVXWdwtf9oO0925QBq40ckmY6bqFDMPsf7Uum4blwbQLB Lggg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1721447035; x=1722051835; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=zXJpSg21NiWZjerNEeO54cx2aELRqlPQgLqFicUJJKQ=; b=qU2NYTy8AeAuB8uUPtXLHUNbbC5LRn7e4fFYCJIYT4tmVsD/aM7dlg2J6Irzuu1teM QRpg+v4Iu7YIpMIwZvdqh8Q25Cnsd7N6qd5MEuxn9qC+86h0xCTVKa5Qrjzt9edUn2bg GuLqh3L+1YwhKRvyAtplPeJxFNQfmdSjEFeWD/4tdmC8QN1eLiOOnXP0M5IgDSf3KNC6 CCTjW5fBEnhTcbqtYqSgjcVJHwD99fS0fxfW95onn/5hVwVQEoPsPkgU0aET95KCUyUE YwS9BczhFlHgMMgUlB1x73yf7st82AqHt7nlI0yD9iVgWDM+UrDkQbZiLxQaU6Kb/xNQ B8uQ== X-Gm-Message-State: AOJu0Yz/Hf8TKzAW48MYEP6yjzdsnefsNt7ZCk6VPRIwetvaucHol3x1 Bo14cM4P7O8xG2cUM4/yHJwV4uhxO+ktouvO9jM+9vYHVZDSShjii1Wa94+7LZM= X-Google-Smtp-Source: AGHT+IEozT0sDFtu+uemyHJL8o1apgQ9P953yY2Tdm9PA9W4GFLKB2+tO3HaIvrMJcIVvy+ZmlB5wg== X-Received: by 2002:a05:620a:4249:b0:79e:fd16:6b74 with SMTP id af79cd13be357-7a1a13bea4fmr259572485a.57.1721447034802; Fri, 19 Jul 2024 20:43:54 -0700 (PDT) Received: from localhost.localdomain (bras-base-unvlon5586w-grc-14-76-71-94-205.dsl.bell.ca. [76.71.94.205]) by smtp.gmail.com with ESMTPSA id af79cd13be357-7a199073b49sm150487385a.100.2024.07.19.20.43.53 (version=TLS1_3 cipher=TLS_CHACHA20_POLY1305_SHA256 bits=256/256); Fri, 19 Jul 2024 20:43:54 -0700 (PDT) From: Eric Ju To: git@vger.kernel.org Cc: calvinwan@google.com, jonathantanmy@google.com, chriscool@tuxfamily.org, eric.peijian@gmail.com, karthik.188@gmail.com, toon@iotcl.com, jltobler@gmail.com Subject: [PATCH v2 2/6] fetch-pack: move fetch initialization Date: Fri, 19 Jul 2024 23:43:33 -0400 Message-ID: <20240720034337.57125-3-eric.peijian@gmail.com> X-Mailer: git-send-email 2.45.2 In-Reply-To: <20240720034337.57125-1-eric.peijian@gmail.com> References: <20240628190503.67389-1-eric.peijian@gmail.com> <20240720034337.57125-1-eric.peijian@gmail.com> Precedence: bulk X-Mailing-List: git@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 From: Calvin Wan There are some variables initialized at the start of the do_fetch_pack_v2() state machine. Currently, they are initialized in FETCH_CHECK_LOCAL, which is the initial state set at the beginning of the function. However, a subsequent patch will allow for another initial state, while still requiring these initialized variables. Move the initialization to be before the state machine, so that they are set regardless of the initial state. Note that there is no change in behavior, because we're moving code from the beginning of the first state to just before the execution of the state machine. Helped-by: Jonathan Tan Helped-by: Christian Couder Signed-off-by: Calvin Wan Signed-off-by: Eric Ju --- fetch-pack.c | 12 ++++++------ 1 file changed, 6 insertions(+), 6 deletions(-) diff --git a/fetch-pack.c b/fetch-pack.c index 9c8cda0f9e..a605b9a499 100644 --- a/fetch-pack.c +++ b/fetch-pack.c @@ -1675,18 +1675,18 @@ static struct ref *do_fetch_pack_v2(struct fetch_pack_args *args, reader.me = "fetch-pack"; } + /* v2 supports these by default */ + allow_unadvertised_object_request |= ALLOW_REACHABLE_SHA1; + use_sideband = 2; + if (args->depth > 0 || args->deepen_since || args->deepen_not) + args->deepen = 1; + while (state != FETCH_DONE) { switch (state) { case FETCH_CHECK_LOCAL: sort_ref_list(&ref, ref_compare_name); QSORT(sought, nr_sought, cmp_ref_by_name); - /* v2 supports these by default */ - allow_unadvertised_object_request |= ALLOW_REACHABLE_SHA1; - use_sideband = 2; - if (args->depth > 0 || args->deepen_since || args->deepen_not) - args->deepen = 1; - /* Filter 'ref' by 'sought' and those that aren't local */ mark_complete_and_common_ref(negotiator, args, &ref); filter_refs(args, &ref, sought, nr_sought); From patchwork Sat Jul 20 03:43:34 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Peijian Ju X-Patchwork-Id: 13737600 Received: from mail-qk1-f176.google.com (mail-qk1-f176.google.com [209.85.222.176]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 3BF13BE7F for ; Sat, 20 Jul 2024 03:43:56 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.222.176 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1721447038; cv=none; b=i2753jnk91iJxSoainBwT6+G0/MdmeLbLCpjATdUXrDYMgJfOuhs5ulJ05H9DBjQqaO6+kj92wzHFp4BB/6i70X0qTlJfEX9fXXJRp20+OFzAIcp17gZiX0UQCTiLREHRek5PNZpEIWYNPCmcIX1F/f/1ud6rtTraUDvT8J59iA= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1721447038; c=relaxed/simple; bh=3TqlCere+U+JSxPVmJIJa/GjoI4TjZHgJ5VqdP41Qsk=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=ZisDmzHQHjAFaTxWYV17G4fXP++FR2oFPD0PS7cqh/uHk3RVRRKS9oGOo2CxJhjq1Ooyg0u4ifWhfVLLnRAqGXyvlrobQRU52dd5NUiSAMp+NGgW4qIauMQtjQ3S4QNxVgFCT6Pd5tNUd0B06lb2Lezjx+eRkXX5bBop3P58nq0= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=BjwW1B19; arc=none smtp.client-ip=209.85.222.176 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="BjwW1B19" Received: by mail-qk1-f176.google.com with SMTP id af79cd13be357-79f323f084eso135296085a.3 for ; Fri, 19 Jul 2024 20:43:56 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1721447036; x=1722051836; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=odAwMZhqrTfkfMaTE3JegaUPfYg8rBgF4fRaR28KiOw=; b=BjwW1B1906QzG+OG/H/op1YqhQOFfz4wx9aSJx3JoWn5ktnDBi875GOYnA6DxCk06B R8XM09LPxiebtbGfyg6LAxzhDeYmxCNyKA9BxGe16iQki4Y+tqt0a8lqinOGSgDeif5o /gi6d7MpZO4gn5IrPsbADLPNMjtfGcKa1wZHiUwrY25ag2YQBdF9IxXvjm96zHJRcSnl rGQ2ECGC89QeLwv1kBvV4KgMHobXrgG+j3FP4WVEFCC0qJc5dNrIIaXQ2Dy084fxMm9N TlwoU+rNJD6uR91hQcD8Uk1NDPpgJPnxkQ0mq/H1YFHRQCRcSWcWQHbUN336/VkMRsNh U9MQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1721447036; x=1722051836; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=odAwMZhqrTfkfMaTE3JegaUPfYg8rBgF4fRaR28KiOw=; b=C7o8CWVNzbeWEBKvLeAU3e69lhibmskVw37r9j2QfyDINyVL6UlBS2OLXq3ClS91pI pybXMo0ZJW4Gj2ZD8pZArxW4xAbSpMOR1sQUUQSbxN9DjeoMCU3PBYkzE6Y5gsBKxdE2 gzORjtB1o5Te7DgVzn0q2SPAcQMxdh04qyGZtcvcpIZgn6pGOzOzGrb+Yq7yGOiUt+kl nmfi5RFGB/HU00S7kHcdcChxlZtghB0RYbXsG6C2J5WyOdzpioSgc1dM+TCpHyVJseA0 5v2QqGyRb9uwUdILbYclP2QxLsOAHb6BfcccF28SGLlYs+yV3VRk1j0AkUy7Dsajx09i qaLQ== X-Gm-Message-State: AOJu0YyoLip9jRvaIH08MkOJJtMEv0ppy2S/UB25KD3NQimYH+ZKETkQ OOegCoTwyU/pS7ODZak/3JHx9lukCu1htRG+g5YMMLG1T0c2Vo6MOh/oftB/4i0= X-Google-Smtp-Source: AGHT+IHZJn+vetgMVCXEa2XVS4okto/+35EfLtf3+8c1LnUZQOQI2f8NsoZprLpGrULkYsqicuABHg== X-Received: by 2002:a05:620a:2a16:b0:79d:5b21:804a with SMTP id af79cd13be357-7a1a130c2b7mr251349985a.14.1721447035777; Fri, 19 Jul 2024 20:43:55 -0700 (PDT) Received: from localhost.localdomain (bras-base-unvlon5586w-grc-14-76-71-94-205.dsl.bell.ca. [76.71.94.205]) by smtp.gmail.com with ESMTPSA id af79cd13be357-7a199073b49sm150487385a.100.2024.07.19.20.43.54 (version=TLS1_3 cipher=TLS_CHACHA20_POLY1305_SHA256 bits=256/256); Fri, 19 Jul 2024 20:43:55 -0700 (PDT) From: Eric Ju To: git@vger.kernel.org Cc: calvinwan@google.com, jonathantanmy@google.com, chriscool@tuxfamily.org, eric.peijian@gmail.com, karthik.188@gmail.com, toon@iotcl.com, jltobler@gmail.com Subject: [PATCH v2 3/6] serve: advertise object-info feature Date: Fri, 19 Jul 2024 23:43:34 -0400 Message-ID: <20240720034337.57125-4-eric.peijian@gmail.com> X-Mailer: git-send-email 2.45.2 In-Reply-To: <20240720034337.57125-1-eric.peijian@gmail.com> References: <20240628190503.67389-1-eric.peijian@gmail.com> <20240720034337.57125-1-eric.peijian@gmail.com> Precedence: bulk X-Mailing-List: git@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 From: Calvin Wan In order for a client to know what object-info components a server can provide, advertise supported object-info features. This will allow a client to decide whether to query the server for object-info or fetch as a fallback. Helped-by: Jonathan Tan Helped-by: Christian Couder Signed-off-by: Calvin Wan Signed-off-by: Eric Ju --- serve.c | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/serve.c b/serve.c index 884cd84ca8..3aae03405b 100644 --- a/serve.c +++ b/serve.c @@ -70,7 +70,7 @@ static void session_id_receive(struct repository *r UNUSED, trace2_data_string("transfer", NULL, "client-sid", client_sid); } -static int object_info_advertise(struct repository *r, struct strbuf *value UNUSED) +static int object_info_advertise(struct repository *r, struct strbuf *value) { if (advertise_object_info == -1 && repo_config_get_bool(r, "transfer.advertiseobjectinfo", @@ -78,6 +78,8 @@ static int object_info_advertise(struct repository *r, struct strbuf *value UNUS /* disabled by default */ advertise_object_info = 0; } + if (value && advertise_object_info) + strbuf_addstr(value, "size"); return advertise_object_info; } From patchwork Sat Jul 20 03:43:35 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Peijian Ju X-Patchwork-Id: 13737601 Received: from mail-oi1-f180.google.com (mail-oi1-f180.google.com [209.85.167.180]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id E3F35D530 for ; Sat, 20 Jul 2024 03:43:57 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.167.180 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1721447039; cv=none; b=BDvbie5fRcyRB0j0v763/lJxzm/w8O3SphZK9Abw7NS4uqQYN5WDqjwPJ/EviWBuR+jh0Foj+N+NjCSOFz6TFv4KQlxxs5oQtpP4OLKIM5dwWuv0PJKi5D7PiFOCzEVjQCTGgyNggOvHp6ERkHpttTz8REzQF/LWRX7Ox42hDBU= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1721447039; c=relaxed/simple; bh=KlcsBUzDtx/sqVWyT/sRgryceaSyl1YaifWNObNw8ZY=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=BeUSIMfTWkosz5e8HzFqy9IF614L8/zUuQ/ywEQiINCkt6S2nOf3vSAqKOPbvaDR9O8zTelF9uPljtlLDV68MVJ0X/gCTuChr6to4te5AlN6RxVjM2hcC2QQW2/FtrC2M6tJJ3OwtkiLxsXlcaJmDJJHX+IVit1/7G/RmhpRK+o= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=EUDPSsoo; arc=none smtp.client-ip=209.85.167.180 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="EUDPSsoo" Received: by mail-oi1-f180.google.com with SMTP id 5614622812f47-3d9400a1ad9so1428675b6e.1 for ; Fri, 19 Jul 2024 20:43:57 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1721447037; x=1722051837; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=KWnaXqqHS5rotJarZ3nwzRv/FG6nNEysYZ00CFfc0Tk=; b=EUDPSsooIG4Zie7MSu64QaC0DudwQoQnnMhJvyw1yXDiGI6z3fop7ViZQxScGrUDzO NAs+SzsQlpAJPqB5/21qvYQFvj8qLpvDA60qgMWJuwYY48KsdFtPOK/tzj5oawIHASj9 HzqS5aa+8PM3dKYs71ImIfjxYQ2IEizvjdQCoTSclxnYbFE6PrEPuxa0o3Oh1nuijApA xGFWi0aTE26E+nShu1N64zPAyxCdOM/zXGUUfTRpWqevEomO+Ux3wBnXX19hjdYrNUls G0B7Y3YbhPA4sh2XhkD3OioIupzvned0zmxMqdEuDS1mxya/cDY1s4kf3af05zc/Xjww /mXw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1721447037; x=1722051837; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=KWnaXqqHS5rotJarZ3nwzRv/FG6nNEysYZ00CFfc0Tk=; b=h2RA4n5WKL76Ou3hhTD9QhKNTKa+Wa0SDvR/nw88pQ2/Ls8wKtCfQLfK9XPaSd8bdW gyW9ja0quZEoslqJiKNNn9+qzyf3MI5vx3MlSlA7zifWXfvjilSx3Ln7YE5LS+0wNzle tkqbhldHhm3LQVqb9Dd13HS6PBcRz0XqqPsySMpmgVZ+PSzAw7ECgU5jIaDp0MN0uiqC L2rFnWU+ujzjRmGIO0f9ziMVSftEH2GF8x0xiUHtTLxEo6MO0UPcwBXI3RXcmwcU1eW/ sv4spC5585xJZr+YGK4yhkB2zy3bRA6WGYm8bkbKIAvG7Fcj1QPQrzCeFE18wDcGvtHr gmQg== X-Gm-Message-State: AOJu0YwD3aO1DzP9bmR33ts4VCdwvrrS4rQQ4Z0aIUh8O5YjsZK/0aSB lrX7rR2YagmUed/l9UiNnI0VDGSV0uGEu8wAG6FcICxQ4TOTaL4SH3KtZ5v9RNg= X-Google-Smtp-Source: AGHT+IFXBePLXT5qke/KHp4ULR4YKruCFQBCqiExI9z2IJcNOuWpwoQ3XNux/7TenM1pfhRhMG4ghw== X-Received: by 2002:a05:6808:1404:b0:3d9:2e63:8330 with SMTP id 5614622812f47-3dae6078585mr1996132b6e.43.1721447036732; Fri, 19 Jul 2024 20:43:56 -0700 (PDT) Received: from localhost.localdomain (bras-base-unvlon5586w-grc-14-76-71-94-205.dsl.bell.ca. [76.71.94.205]) by smtp.gmail.com with ESMTPSA id af79cd13be357-7a199073b49sm150487385a.100.2024.07.19.20.43.55 (version=TLS1_3 cipher=TLS_CHACHA20_POLY1305_SHA256 bits=256/256); Fri, 19 Jul 2024 20:43:56 -0700 (PDT) From: Eric Ju To: git@vger.kernel.org Cc: calvinwan@google.com, jonathantanmy@google.com, chriscool@tuxfamily.org, eric.peijian@gmail.com, karthik.188@gmail.com, toon@iotcl.com, jltobler@gmail.com Subject: [PATCH v2 4/6] transport: add client support for object-info Date: Fri, 19 Jul 2024 23:43:35 -0400 Message-ID: <20240720034337.57125-5-eric.peijian@gmail.com> X-Mailer: git-send-email 2.45.2 In-Reply-To: <20240720034337.57125-1-eric.peijian@gmail.com> References: <20240628190503.67389-1-eric.peijian@gmail.com> <20240720034337.57125-1-eric.peijian@gmail.com> Precedence: bulk X-Mailing-List: git@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 From: Calvin Wan Sometimes it is useful to get information about an object without having to download it completely. The server logic has already been implemented in “a2ba162cda (object-info: support for retrieving object info, 2021-04-20)”. Add client functions to communicate with the server. The client currently supports requesting a list of object ids with feature 'size' from a v2 server. If a server does not advertise the feature, then the client falls back to making the request through 'fetch'. Helped-by: Jonathan Tan Helped-by: Christian Couder Signed-off-by: Calvin Wan Signed-off-by: Eric Ju --- fetch-pack.c | 24 +++++++++ fetch-pack.h | 10 ++++ transport-helper.c | 8 ++- transport.c | 118 +++++++++++++++++++++++++++++++++++++++++++-- transport.h | 11 +++++ 5 files changed, 164 insertions(+), 7 deletions(-) diff --git a/fetch-pack.c b/fetch-pack.c index a605b9a499..419450c8dd 100644 --- a/fetch-pack.c +++ b/fetch-pack.c @@ -1344,6 +1344,27 @@ static void write_command_and_capabilities(struct strbuf *req_buf, packet_buf_delim(req_buf); } +void send_object_info_request(int fd_out, struct object_info_args *args) +{ + struct strbuf req_buf = STRBUF_INIT; + + write_command_and_capabilities(&req_buf, args->server_options, "object-info"); + + if (unsorted_string_list_has_string(args->object_info_options, "size")) + packet_buf_write(&req_buf, "size"); + + if (args->oids) { + for (size_t i = 0; i < args->oids->nr; i++) + packet_buf_write(&req_buf, "oid %s", oid_to_hex(&args->oids->oid[i])); + } + + packet_buf_flush(&req_buf); + if (write_in_full(fd_out, req_buf.buf, req_buf.len) < 0) + die_errno(_("unable to write request to remote")); + + strbuf_release(&req_buf); +} + static int send_fetch_request(struct fetch_negotiator *negotiator, int fd_out, struct fetch_pack_args *args, const struct ref *wants, struct oidset *common, @@ -1681,6 +1702,9 @@ static struct ref *do_fetch_pack_v2(struct fetch_pack_args *args, if (args->depth > 0 || args->deepen_since || args->deepen_not) args->deepen = 1; + if (args->object_info) + state = FETCH_SEND_REQUEST; + while (state != FETCH_DONE) { switch (state) { case FETCH_CHECK_LOCAL: diff --git a/fetch-pack.h b/fetch-pack.h index b5c579cdae..5a5211e355 100644 --- a/fetch-pack.h +++ b/fetch-pack.h @@ -16,6 +16,7 @@ struct fetch_pack_args { const struct string_list *deepen_not; struct list_objects_filter_options filter_options; const struct string_list *server_options; + struct object_info *object_info_data; /* * If not NULL, during packfile negotiation, fetch-pack will send "have" @@ -42,6 +43,7 @@ struct fetch_pack_args { unsigned reject_shallow_remote:1; unsigned deepen:1; unsigned refetch:1; + unsigned object_info:1; /* * Indicate that the remote of this request is a promisor remote. The @@ -68,6 +70,12 @@ struct fetch_pack_args { unsigned connectivity_checked:1; }; +struct object_info_args { + struct string_list *object_info_options; + const struct string_list *server_options; + struct oid_array *oids; +}; + /* * sought represents remote references that should be updated from. * On return, the names that were found on the remote will have been @@ -106,4 +114,6 @@ int report_unmatched_refs(struct ref **sought, int nr_sought); */ int fetch_pack_fsck_objects(void); +void send_object_info_request(int fd_out, struct object_info_args *args); + #endif diff --git a/transport-helper.c b/transport-helper.c index 09b3560ffd..841a32e80a 100644 --- a/transport-helper.c +++ b/transport-helper.c @@ -699,13 +699,17 @@ static int fetch_refs(struct transport *transport, /* * If we reach here, then the server, the client, and/or the transport - * helper does not support protocol v2. --negotiate-only requires - * protocol v2. + * helper does not support protocol v2. --negotiate-only and cat-file remote-object-info + * require protocol v2. */ if (data->transport_options.acked_commits) { warning(_("--negotiate-only requires protocol v2")); return -1; } + if (transport->smart_options->object_info) { + // fail the command explicitly to avoid further commands input + die(_("remote-object-info requires protocol v2")); + } if (!data->get_refs_list_called) get_refs_list_using_list(transport, 0); diff --git a/transport.c b/transport.c index 12cc5b4d96..8f990fcba6 100644 --- a/transport.c +++ b/transport.c @@ -366,6 +366,80 @@ static struct ref *handshake(struct transport *transport, int for_push, return refs; } +static int fetch_object_info(struct transport *transport, struct object_info *object_info_data) +{ + int size_index = -1; + struct git_transport_data *data = transport->data; + struct object_info_args args = { 0 }; + struct packet_reader reader; + + args.server_options = transport->server_options; + args.object_info_options = transport->smart_options->object_info_options; + args.oids = transport->smart_options->object_info_oids; + + connect_setup(transport, 0); + packet_reader_init(&reader, data->fd[0], NULL, 0, + PACKET_READ_CHOMP_NEWLINE | + PACKET_READ_GENTLE_ON_EOF | + PACKET_READ_DIE_ON_ERR_PACKET); + data->version = discover_version(&reader); + + transport->hash_algo = reader.hash_algo; + + switch (data->version) { + case protocol_v2: + if (!server_supports_v2("object-info")) + return -1; + if (unsorted_string_list_has_string(args.object_info_options, "size") + && !server_supports_feature("object-info", "size", 0)) { + return -1; + } + send_object_info_request(data->fd[1], &args); + break; + case protocol_v1: + case protocol_v0: + die(_("wrong protocol version. expected v2")); + case protocol_unknown_version: + BUG("unknown protocol version"); + } + + for (size_t i = 0; i < args.object_info_options->nr; i++) { + if (packet_reader_read(&reader) != PACKET_READ_NORMAL) { + check_stateless_delimiter(transport->stateless_rpc, &reader, "stateless delimiter expected"); + return -1; + } + if (unsorted_string_list_has_string(args.object_info_options, reader.line)) { + if (!strcmp(reader.line, "size")) { + size_index = i; + for (size_t j = 0; j < args.oids->nr; j++) { + object_info_data[j].sizep = xcalloc(1, sizeof(long)); + } + } + continue; + } + return -1; + } + + for (size_t i = 0; packet_reader_read(&reader) == PACKET_READ_NORMAL && i < args.oids->nr; i++){ + struct string_list object_info_values = STRING_LIST_INIT_DUP; + + string_list_split(&object_info_values, reader.line, ' ', -1); + if (0 <= size_index) { + if (!strcmp(object_info_values.items[1 + size_index].string, "")) + die("object-info: not our ref %s", + object_info_values.items[0].string); + + *object_info_data[i].sizep = strtoul(object_info_values.items[1 + size_index].string, NULL, 10); + + } + + string_list_clear(&object_info_values, 0); + } + check_stateless_delimiter(transport->stateless_rpc, &reader, "stateless delimiter expected"); + + return 0; +} + static struct ref *get_refs_via_connect(struct transport *transport, int for_push, struct transport_ls_refs_options *options) { @@ -413,6 +487,7 @@ static int fetch_refs_via_pack(struct transport *transport, struct ref *refs = NULL; struct fetch_pack_args args; struct ref *refs_tmp = NULL; + struct ref *object_info_refs = NULL; memset(&args, 0, sizeof(args)); args.uploadpack = data->options.uploadpack; @@ -439,11 +514,36 @@ static int fetch_refs_via_pack(struct transport *transport, args.server_options = transport->server_options; args.negotiation_tips = data->options.negotiation_tips; args.reject_shallow_remote = transport->smart_options->reject_shallow; + args.object_info = transport->smart_options->object_info; + + if (transport->smart_options + && transport->smart_options->object_info + && transport->smart_options->object_info_oids->nr > 0) { + struct ref *ref_itr = object_info_refs = alloc_ref(""); + + if (!fetch_object_info(transport, data->options.object_info_data)) + goto cleanup; + + args.object_info_data = data->options.object_info_data; + args.quiet = 1; + args.no_progress = 1; + for (size_t i = 0; i < transport->smart_options->object_info_oids->nr; i++) { + ref_itr->old_oid = transport->smart_options->object_info_oids->oid[i]; + ref_itr->exact_oid = 1; + if (i == transport->smart_options->object_info_oids->nr - 1) + /* last element, no need to allocat to next */ + ref_itr -> next = NULL; + else + ref_itr->next = alloc_ref(""); - if (!data->finished_handshake) { - int i; + ref_itr = ref_itr->next; + } + + transport->remote_refs = object_info_refs; + + } else if (!data->finished_handshake) { int must_list_refs = 0; - for (i = 0; i < nr_heads; i++) { + for (int i = 0; i < nr_heads; i++) { if (!to_fetch[i]->exact_oid) { must_list_refs = 1; break; @@ -481,23 +581,31 @@ static int fetch_refs_via_pack(struct transport *transport, &transport->pack_lockfiles, data->version); data->finished_handshake = 0; + if (args.object_info) { + struct ref *ref_cpy_reader = object_info_refs; + for (int i = 0; ref_cpy_reader; i++) { + oid_object_info_extended(the_repository, &ref_cpy_reader->old_oid, &args.object_info_data[i], OBJECT_INFO_LOOKUP_REPLACE); + ref_cpy_reader = ref_cpy_reader->next; + } + } + data->options.self_contained_and_connected = args.self_contained_and_connected; data->options.connectivity_checked = args.connectivity_checked; - if (!refs) + if (!refs && !args.object_info) ret = -1; if (report_unmatched_refs(to_fetch, nr_heads)) ret = -1; cleanup: + free_refs(object_info_refs); close(data->fd[0]); if (data->fd[1] >= 0) close(data->fd[1]); if (finish_connect(data->conn)) ret = -1; data->conn = NULL; - free_refs(refs_tmp); free_refs(refs); list_objects_filter_release(&args.filter_options); diff --git a/transport.h b/transport.h index 6393cd9823..50ea2b05cf 100644 --- a/transport.h +++ b/transport.h @@ -5,6 +5,7 @@ #include "remote.h" #include "list-objects-filter-options.h" #include "string-list.h" +#include "object-store.h" struct git_transport_options { unsigned thin : 1; @@ -30,6 +31,12 @@ struct git_transport_options { */ unsigned connectivity_checked:1; + /* + * Transport will attempt to pull only object-info. Fallbacks + * to pulling entire object if object-info is not supported. + */ + unsigned object_info : 1; + int depth; const char *deepen_since; const struct string_list *deepen_not; @@ -53,6 +60,10 @@ struct git_transport_options { * common commits to this oidset instead of fetching any packfiles. */ struct oidset *acked_commits; + + struct oid_array *object_info_oids; + struct object_info *object_info_data; + struct string_list *object_info_options; }; enum transport_family { From patchwork Sat Jul 20 03:43:36 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Peijian Ju X-Patchwork-Id: 13737602 Received: from mail-qk1-f175.google.com (mail-qk1-f175.google.com [209.85.222.175]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id E099BFBFD for ; Sat, 20 Jul 2024 03:43:58 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.222.175 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1721447040; cv=none; b=bF+RB8E6a6dmE1W+OjhXC44ydiZWTRu+rnX4o5RPbikUW/XfIuff60ERs62m6TbofUW3tue+zOyFuHiDj3C6bi2mW+mF/Y8L3fKGWw4O5E9vxOCbcv22T/TzMtZrXnD/pp1JvweQdnsaLN+6T6nv9Y3gGcYvXUXZirhELyImhm8= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1721447040; c=relaxed/simple; bh=5CbIiOg08o6CSaEHGHpwptNW5+wXvDUAOC9uSxrrws4=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=Wips2368ZTf4QJf72WH1h43i0Wg9QLIHKzL3Ziw3bNf85YT7VHL/9Fjk9kI7yuLqZ74LWqJbKwIDUOuMYxStSCRblOHcyuEF61iGevyvbu8z/gf8sa3JN+AVmixf39LSGBgME1rhxccF48NAGAs6TuZ/dyWAZRLIDIZLA4+PK7I= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=lvBaG78d; arc=none smtp.client-ip=209.85.222.175 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="lvBaG78d" Received: by mail-qk1-f175.google.com with SMTP id af79cd13be357-79ef72bb8c8so101298385a.2 for ; Fri, 19 Jul 2024 20:43:58 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1721447038; x=1722051838; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=2wTinyu1GxajV0q7z55o9Gpr/Z8n0zsvzWOBu7mf1aQ=; b=lvBaG78dOf1/fQXjfjLYpzRbfHQMQTbSOtO4SJ4H6gsjSJkZa48w/ocU2PMGvmXcZQ CUL9II3P8kTkLoSqzosuUCS1vnaDepBm2I+afbUXJuQ6paaqWuo0ONZtSda9UQnLFzbz 8yBtLOMsdTHUrVwL3qwN/+DnbNk1AfwmFgyfKoWbnmTQ1sTa3zu1KK2638pJvXrQUNQs UWhwrNFnXq5sFEnoedUcir88gYEi4cZ3U+fSvCl1AewcUSc7Qd7vLIkiLdPGlzM5au12 /wOsZ7wyQ9kFZf3po0pulHMtZXwFfzlaMoc00CPr/Zh9rz90mW5cBsijaH1wmOLw1Jsu mMXA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1721447038; x=1722051838; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=2wTinyu1GxajV0q7z55o9Gpr/Z8n0zsvzWOBu7mf1aQ=; b=c8YhLoNPYLnpvnAiUNRq5WxM3l/0bpUWgb2oJq04niSfpjakYvRGvzvQ0LqwibD/wX /z9q1+pkCAbIIGP9Ri/Fv6dw4VReb7AYcl/aCC1ymJmRrmAgAINn0F+w5x7hjCi8sE5C 46hfGRlIjYlYLXGsUgjLyac87E3jYzG8F/bG6Sl2qXUgndwXEje1r/zkjOQ6n5qNOpDM 1P9sO0vNke1W4wFCDM0LclkZ/1dXH00PvLBE4WN9X4dXkELlTyw/EFwgmMnbdgP+Qyxk QgBAKO5tPruv/h1WE4I7Lgum+U9SDV/hBFSG9xabbxOHG/tcMbWzCFND8lMaIGuvJi8X lU5A== X-Gm-Message-State: AOJu0YwMJFjJpYRdYNJAQXUHIe78MGtVNwIPpb9LGdRkJKFa7Qmwa3Zz Xe7bsHTCisD7qc5SVNhF7u3Syh/WzN4627Xe/E0gnsRy/E6ON93W/8Ijal0EN5U= X-Google-Smtp-Source: AGHT+IEuOJmIEuAovgeunbhYe1s0176wdnqzuJ9+bibdVjn0ZSQnpckPULHFiaVkJPn8SrA0Lciw9Q== X-Received: by 2002:a05:620a:4442:b0:79e:ff79:c2bf with SMTP id af79cd13be357-7a1a13a754amr237466985a.50.1721447037686; Fri, 19 Jul 2024 20:43:57 -0700 (PDT) Received: from localhost.localdomain (bras-base-unvlon5586w-grc-14-76-71-94-205.dsl.bell.ca. [76.71.94.205]) by smtp.gmail.com with ESMTPSA id af79cd13be357-7a199073b49sm150487385a.100.2024.07.19.20.43.56 (version=TLS1_3 cipher=TLS_CHACHA20_POLY1305_SHA256 bits=256/256); Fri, 19 Jul 2024 20:43:57 -0700 (PDT) From: Eric Ju To: git@vger.kernel.org Cc: calvinwan@google.com, jonathantanmy@google.com, chriscool@tuxfamily.org, eric.peijian@gmail.com, karthik.188@gmail.com, toon@iotcl.com, jltobler@gmail.com Subject: [PATCH v2 5/6] cat-file: add declaration of variable i inside its for loop Date: Fri, 19 Jul 2024 23:43:36 -0400 Message-ID: <20240720034337.57125-6-eric.peijian@gmail.com> X-Mailer: git-send-email 2.45.2 In-Reply-To: <20240720034337.57125-1-eric.peijian@gmail.com> References: <20240628190503.67389-1-eric.peijian@gmail.com> <20240720034337.57125-1-eric.peijian@gmail.com> Precedence: bulk X-Mailing-List: git@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Some code declares variable i and only uses it in a for loop, not in any other logic outside the loop. Change the declaration of i to be inside the for loop for readability. Helped-by: Christian Couder Signed-off-by: Eric Ju --- builtin/cat-file.c | 11 +++-------- 1 file changed, 3 insertions(+), 8 deletions(-) diff --git a/builtin/cat-file.c b/builtin/cat-file.c index 18fe58d6b8..a5724667b1 100644 --- a/builtin/cat-file.c +++ b/builtin/cat-file.c @@ -673,12 +673,10 @@ static void dispatch_calls(struct batch_options *opt, struct queued_cmd *cmd, int nr) { - int i; - if (!opt->buffer_output) die(_("flush is only for --buffer mode")); - for (i = 0; i < nr; i++) + for (size_t i = 0; i < nr; i++) cmd[i].fn(opt, cmd[i].line, output, data); fflush(stdout); @@ -686,9 +684,7 @@ static void dispatch_calls(struct batch_options *opt, static void free_cmds(struct queued_cmd *cmd, size_t *nr) { - size_t i; - - for (i = 0; i < *nr; i++) + for (size_t i = 0; i < *nr; i++) FREE_AND_NULL(cmd[i].line); *nr = 0; @@ -714,7 +710,6 @@ static void batch_objects_command(struct batch_options *opt, size_t alloc = 0, nr = 0; while (strbuf_getdelim_strip_crlf(&input, stdin, opt->input_delim) != EOF) { - int i; const struct parse_cmd *cmd = NULL; const char *p = NULL, *cmd_end; struct queued_cmd call = {0}; @@ -724,7 +719,7 @@ static void batch_objects_command(struct batch_options *opt, if (isspace(*input.buf)) die(_("whitespace before command: '%s'"), input.buf); - for (i = 0; i < ARRAY_SIZE(commands); i++) { + for (size_t i = 0; i < ARRAY_SIZE(commands); i++) { if (!skip_prefix(input.buf, commands[i].name, &cmd_end)) continue; From patchwork Sat Jul 20 03:43:37 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Peijian Ju X-Patchwork-Id: 13737603 Received: from mail-qk1-f182.google.com (mail-qk1-f182.google.com [209.85.222.182]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 59FD3BE7F for ; Sat, 20 Jul 2024 03:44:00 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.222.182 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1721447043; cv=none; b=WnRbgTNNc5gVGOAc/MI1eKVJe8qlphvwPWNHGYHMD8rha89m4JQm8nv8togaSW4Tukswc4mtDpC485rHA/QJNgb2SHfC1tXdwUXb3p8xK8dVFyTVTWwig2A/viwDQcID6Hr+0OEy5S5jMecSlFRXT7qcTCDck/6oCj2sZb02mu8= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1721447043; c=relaxed/simple; bh=MXfjAYlH5q2iCYJh4U80VjOK/LX80OrtGj+H6lBRk3E=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=EPoTAiXS1fZU638QHhpoY5ckgYFBC9oT084bknGROOinTPkSGarWV++dvewvL5aWHSF+nqjUmBt8eqbYeaH8DaWOU8p2VuzRixdPNVZCGijrx0KOe4yn2SMfLGiFxNeDGBoEw4bR83wWMiuMuIRsNptnmg2xyO85dIV5yY4sBl0= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=fvR7LdHP; arc=none smtp.client-ip=209.85.222.182 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="fvR7LdHP" Received: by mail-qk1-f182.google.com with SMTP id af79cd13be357-79f18509e76so123706585a.1 for ; Fri, 19 Jul 2024 20:44:00 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1721447039; x=1722051839; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=eJevdIaZSCIDL9/JwgCg753RgsmvN7wLTuE2b/bi8Wg=; b=fvR7LdHPMXfOvabM4sKLtF4+zmJIn/L0oRnM3w+esRXljnkW7SIKCLqAFjCawh1uF2 0xiEQ+bUyBqT8V1UVMsMTCN+jVF/pOHoaWgRYfdxkX95o7Mm7x/euf/ZITeQfTjeVeTY hTNd0Ts7tYdkPb2zTwYJzs1jzSC8+rdDXReodVdEKQDg10+nZjoSRKuQQjJlNWriCgE9 LF3KUN/lnHAB83EUC/pdp0WS/Uc6djnY173golVSsMhl6tRSqTlYwaaSFvVin76VHLkT a0jWM8Qt9ob1wLhP5mJpR8/UI6EKkl1r7DcCFs/z6Xr05NRvFa9ThMwzLVF6jBoapQyB ufgg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1721447039; x=1722051839; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=eJevdIaZSCIDL9/JwgCg753RgsmvN7wLTuE2b/bi8Wg=; b=G81N7O/GMDxUiVtP9bHNl6edsO+naISCDVWAZ0gElM5bVrH302x2qsG7fWM89lNGga 4+uri2YqJ8e387UBDYh42vW9iOorBgCA7Io1zRTNFOyFly4ib5ZEK0hX2gOdiw+fK6nd 8a1H11p2rghHK5nMrUmyEI/Yw7PJi5OewEHSMZRpKaBsSYaYotJVw5LdcuIGeJndQaSY cTNVH1sm19LB3bNhSJYQFL61R8E72DJtfseE+Z7/rXtRl4pykDzHk5kotfeGg5jvFhbX MlX8UMR7KR1jjcAcSD6CLbDoG7RoLWbMJPMosXlhvRV/ssHMqbuB+95gDvRmUoTB5yqR rnHA== X-Gm-Message-State: AOJu0YzYs0Y+2VPbKpsL3viHmD8yPihztsu83/aQHZqNMjfjE/h5/ylk ClwU0LhPM1CEg2Y8qKdz1z0akkyTgp0+U7yH5rKwTN/swxxVVSdJPxAXYwO6fns= X-Google-Smtp-Source: AGHT+IHRnUuqv+KrZrK0507bBgvWGVmGe9AmbpC5FrTr3Pacq/TXJCIBlh8kM7cwviJnRExkXK7Cxg== X-Received: by 2002:a05:620a:19a1:b0:79d:7b02:ebbf with SMTP id af79cd13be357-7a1a6680444mr35764485a.70.1721447038705; Fri, 19 Jul 2024 20:43:58 -0700 (PDT) Received: from localhost.localdomain (bras-base-unvlon5586w-grc-14-76-71-94-205.dsl.bell.ca. [76.71.94.205]) by smtp.gmail.com with ESMTPSA id af79cd13be357-7a199073b49sm150487385a.100.2024.07.19.20.43.57 (version=TLS1_3 cipher=TLS_CHACHA20_POLY1305_SHA256 bits=256/256); Fri, 19 Jul 2024 20:43:58 -0700 (PDT) From: Eric Ju To: git@vger.kernel.org Cc: calvinwan@google.com, jonathantanmy@google.com, chriscool@tuxfamily.org, eric.peijian@gmail.com, karthik.188@gmail.com, toon@iotcl.com, jltobler@gmail.com Subject: [PATCH v2 6/6] cat-file: add remote-object-info to batch-command Date: Fri, 19 Jul 2024 23:43:37 -0400 Message-ID: <20240720034337.57125-7-eric.peijian@gmail.com> X-Mailer: git-send-email 2.45.2 In-Reply-To: <20240720034337.57125-1-eric.peijian@gmail.com> References: <20240628190503.67389-1-eric.peijian@gmail.com> <20240720034337.57125-1-eric.peijian@gmail.com> Precedence: bulk X-Mailing-List: git@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 From: Calvin Wan Since the `info` command in cat-file --batch-command prints object info for a given object, it is natural to add another command in cat-file --batch-command to print object info for a given object from a remote. Add `remote-object-info` to cat-file --batch-command. While `info` takes object ids one at a time, this creates overhead when making requests to a server so `remote-object-info` instead can take multiple object ids at once. cat-file --batch-command is generally implemented in the following manner: - Receive and parse input from user - Call respective function attached to command - Get object info, print object info In --buffer mode, this changes to: - Receive and parse input from user - Store respective function attached to command in a queue - After flush, loop through commands in queue - Call respective function attached to command - Get object info, print object info Notice how the getting and printing of object info is accomplished one at a time. As described above, this creates a problem for making requests to a server. Therefore, `remote-object-info` is implemented in the following manner: - Receive and parse input from user If command is `remote-object-info`: - Get object info from remote - Loop through and print each object info Else: - Call respective function attached to command - Parse input, get object info, print object info And finally for --buffer mode `remote-object-info`: - Receive and parse input from user - Store respective function attached to command in a queue - After flush, loop through commands in queue: If command is `remote-object-info`: - Get object info from remote - Loop through and print each object info Else: - Call respective function attached to command - Get object info, print object info To summarize, `remote-object-info` gets object info from the remote and then loop through the object info passed in, print the info. In order for remote-object-info to avoid remote communication overhead in the non-buffer mode, the objects are passed in as such: remote-object-info ... rather than remote-object-info remote-object-info ... remote-object-info Helped-by: Jonathan Tan Helped-by: Christian Couder Signed-off-by: Calvin Wan Signed-off-by: Eric Ju --- Documentation/git-cat-file.txt | 23 +- builtin/cat-file.c | 116 +++- object-file.c | 11 + object-store-ll.h | 3 + t/t1017-cat-file-remote-object-info.sh | 748 +++++++++++++++++++++++++ 5 files changed, 893 insertions(+), 8 deletions(-) create mode 100755 t/t1017-cat-file-remote-object-info.sh diff --git a/Documentation/git-cat-file.txt b/Documentation/git-cat-file.txt index bd95a6c10a..98375dab46 100644 --- a/Documentation/git-cat-file.txt +++ b/Documentation/git-cat-file.txt @@ -149,6 +149,13 @@ info :: Print object info for object reference ``. This corresponds to the output of `--batch-check`. +remote-object-info ...:: + Print object info for object references `` at specified without + downloading objects from remote. If the object-info capability is not + supported by the server, the objects will be downloaded instead. + Error when no object references is provided. + This command may be combined with `--buffer`. + flush:: Used with `--buffer` to execute all preceding commands that were issued since the beginning or since the last flush was issued. When `--buffer` @@ -290,7 +297,8 @@ newline. The available atoms are: The full hex representation of the object name. `objecttype`:: - The type of the object (the same as `cat-file -t` reports). + The type of the object (the same as `cat-file -t` reports). See + `CAVEATS` below. Not supported by `remote-object-info`. `objectsize`:: The size, in bytes, of the object (the same as `cat-file -s` @@ -298,13 +306,14 @@ newline. The available atoms are: `objectsize:disk`:: The size, in bytes, that the object takes up on disk. See the - note about on-disk sizes in the `CAVEATS` section below. + note about on-disk sizes in the `CAVEATS` section below. Not + supported by `remote-object-info`. `deltabase`:: If the object is stored as a delta on-disk, this expands to the full hex representation of the delta base object name. Otherwise, expands to the null OID (all zeroes). See `CAVEATS` - below. + below. Not supported by `remote-object-info`. `rest`:: If this atom is used in the output string, input lines are split @@ -314,7 +323,9 @@ newline. The available atoms are: line) are output in place of the `%(rest)` atom. If no format is specified, the default format is `%(objectname) -%(objecttype) %(objectsize)`. +%(objecttype) %(objectsize)`, except remote-object-info command who uses +`%(objectname) %(objectsize)` for now because "%(objecttype)" is not supported yet. +When "%(objecttype)" is supported, default format should be unified. If `--batch` is specified, or if `--batch-command` is used with the `contents` command, the object information is followed by the object contents (consisting @@ -396,6 +407,10 @@ scripting purposes. CAVEATS ------- +Note that since objecttype, objectsize:disk and deltabase are currently not supported by the +remote-object-info, git will error and exit when they are in the format string. + + Note that the sizes of objects on disk are reported accurately, but care should be taken in drawing conclusions about which refs or objects are responsible for disk usage. The size of a packed non-delta object may be diff --git a/builtin/cat-file.c b/builtin/cat-file.c index a5724667b1..ca6e05e769 100644 --- a/builtin/cat-file.c +++ b/builtin/cat-file.c @@ -24,6 +24,9 @@ #include "promisor-remote.h" #include "mailmap.h" #include "write-or-die.h" +#include "alias.h" +#include "remote.h" +#include "transport.h" enum batch_mode { BATCH_MODE_CONTENTS, @@ -42,9 +45,12 @@ struct batch_options { char input_delim; char output_delim; const char *format; + int use_remote_info; }; static const char *force_path; +static struct object_info *remote_object_info; +static struct oid_array object_info_oids = OID_ARRAY_INIT; static struct string_list mailmap = STRING_LIST_INIT_NODUP; static int use_mailmap; @@ -528,7 +534,7 @@ static void batch_one_object(const char *obj_name, enum get_oid_result result; result = get_oid_with_context(the_repository, obj_name, - flags, &data->oid, &ctx); + flags, &data->oid, &ctx); if (result != FOUND) { switch (result) { case MISSING_OBJECT: @@ -576,6 +582,61 @@ static void batch_one_object(const char *obj_name, object_context_release(&ctx); } +static int get_remote_info(struct batch_options *opt, int argc, const char **argv) +{ + int retval = 0; + struct remote *remote = NULL; + struct object_id oid; + struct string_list object_info_options = STRING_LIST_INIT_NODUP; + static struct transport *gtransport; + + /* + * Change the format to "%(objectname) %(objectsize)" when + * remote-object-info command is used. Once we start supporting objecttype + * the default format should change to DEFAULT_FORMAT + */ + if (!opt->format) + opt->format = "%(objectname) %(objectsize)"; + + remote = remote_get(argv[0]); + if (!remote) + die(_("must supply valid remote when using remote-object-info")); + + oid_array_clear(&object_info_oids); + for (size_t i = 1; i < argc; i++) { + if (get_oid_hex(argv[i], &oid)) + die(_("Not a valid object name %s"), argv[i]); + oid_array_append(&object_info_oids, &oid); + } + + gtransport = transport_get(remote, NULL); + if (gtransport->smart_options) { + CALLOC_ARRAY(remote_object_info, object_info_oids.nr); + gtransport->smart_options->object_info = 1; + gtransport->smart_options->object_info_oids = &object_info_oids; + /* + * 'size' is the only option currently supported. + * Other options that are passed in the format will exit with error. + */ + if (strstr(opt->format, "%(objectsize)")) { + string_list_append(&object_info_options, "size"); + } else { + die(_("%s is currently not supported with remote-object-info"), opt->format); + } + if (object_info_options.nr > 0) { + gtransport->smart_options->object_info_options = &object_info_options; + gtransport->smart_options->object_info_data = remote_object_info; + retval = transport_fetch_refs(gtransport, NULL); + } + } else { + retval = -1; + } + + string_list_clear(&object_info_options, 0); + transport_disconnect(gtransport); + return retval; +} + struct object_cb_data { struct batch_options *opt; struct expand_data *expand; @@ -667,6 +728,52 @@ static void parse_cmd_info(struct batch_options *opt, batch_one_object(line, output, opt, data); } +static void parse_cmd_remote_object_info(struct batch_options *opt, + const char *line, + struct strbuf *output, + struct expand_data *data) +{ + int count; + const char **argv; + + char *line_to_split = xstrdup_or_null(line); + count = split_cmdline(line_to_split, &argv); + if (get_remote_info(opt, count, argv)) + goto cleanup; + + opt->use_remote_info = 1; + data->skip_object_info = 1; + for (size_t i = 0; i < object_info_oids.nr; i++) { + + data->oid = object_info_oids.oid[i]; + + if (remote_object_info[i].sizep) { + data->size = *remote_object_info[i].sizep; + } else { + /* + * When reaching here, it means remote-object-info can't retrive + * infomation from server withoug downloading them, and the objects + * have been fetched to client already. + * Print the infomation using the logic for local objects. + */ + data->skip_object_info = 0; + } + + opt->batch_mode = BATCH_MODE_INFO; + batch_object_write(argv[i+1], output, opt, data, NULL, 0); + + } + opt->use_remote_info = 0; + data->skip_object_info = 0; + +cleanup: + for (size_t i = 0; i < object_info_oids.nr; i++) + free_object_info_contents(&remote_object_info[i]); + free(line_to_split); + free(argv); + free(remote_object_info); +} + static void dispatch_calls(struct batch_options *opt, struct strbuf *output, struct expand_data *data, @@ -696,9 +803,10 @@ static const struct parse_cmd { parse_cmd_fn_t fn; unsigned takes_args; } commands[] = { - { "contents", parse_cmd_contents, 1}, - { "info", parse_cmd_info, 1}, - { "flush", NULL, 0}, + { "contents", parse_cmd_contents, 1 }, + { "info", parse_cmd_info, 1 }, + { "remote-object-info", parse_cmd_remote_object_info, 1 }, + { "flush", NULL, 0 }, }; static void batch_objects_command(struct batch_options *opt, diff --git a/object-file.c b/object-file.c index 065103be3e..34c702ece5 100644 --- a/object-file.c +++ b/object-file.c @@ -2987,3 +2987,14 @@ int read_loose_object(const char *path, munmap(map, mapsize); return ret; } + +void free_object_info_contents(struct object_info *object_info) +{ + if (!object_info) + return; + free(object_info->typep); + free(object_info->sizep); + free(object_info->disk_sizep); + free(object_info->delta_base_oid); + free(object_info->type_name); +} diff --git a/object-store-ll.h b/object-store-ll.h index c5f2bb2fc2..333e19cd1e 100644 --- a/object-store-ll.h +++ b/object-store-ll.h @@ -533,4 +533,7 @@ int for_each_object_in_pack(struct packed_git *p, int for_each_packed_object(each_packed_object_fn, void *, enum for_each_object_flags flags); +/* Free pointers inside of object_info, but not object_info itself */ +void free_object_info_contents(struct object_info *object_info); + #endif /* OBJECT_STORE_LL_H */ diff --git a/t/t1017-cat-file-remote-object-info.sh b/t/t1017-cat-file-remote-object-info.sh new file mode 100755 index 0000000000..64eb55bd9e --- /dev/null +++ b/t/t1017-cat-file-remote-object-info.sh @@ -0,0 +1,748 @@ +#!/bin/sh + +test_description='git cat-file --batch-command with remote-object-info command' + +GIT_TEST_DEFAULT_INITIAL_BRANCH_NAME=main +export GIT_TEST_DEFAULT_INITIAL_BRANCH_NAME + +. ./test-lib.sh + +echo_without_newline () { + printf '%s' "$*" +} + +echo_without_newline_nul () { + echo_without_newline "$@" | tr '\n' '\0' +} + +strlen () { + echo_without_newline "$1" | wc -c | sed -e 's/^ *//' +} + +hello_content="Hello World" +hello_size=$(strlen "$hello_content") +hello_oid=$(echo_without_newline "$hello_content" | git hash-object --stdin) + +# This is how we get 13: +# 13 = + + + , where +# file mode is 100644, which is 6 characters; +# file name is hello, which is 5 characters +# a space is 1 character and a null is 1 character +tree_size=$(($(test_oid rawsz) + 13)) + +commit_message="Initial commit" + +# This is how we get 137: +# 137 = + + + +# + + +# + + +# + +# +# An easier way to calculate is: 1. use `git cat-file commit | wc -c`, +# to get 177, 2. then deduct 40 hex characters to get 137 +commit_size=$(($(test_oid hexsz) + 137)) + +tag_header_without_oid="type blob +tag hellotag +tagger $GIT_COMMITTER_NAME <$GIT_COMMITTER_EMAIL>" +tag_header_without_timestamp="object $hello_oid +$tag_header_without_oid" +tag_description="This is a tag" +tag_content="$tag_header_without_timestamp 0 +0000 + +$tag_description" + +tag_oid=$(echo_without_newline "$tag_content" | git hash-object -t tag --stdin -w) +tag_size=$(strlen "$tag_content") + +set_transport_variables () { + hello_sha1=$(echo_without_newline "$hello_content" | git hash-object --stdin) + tree_sha1=$(git -C "$1" write-tree) + commit_sha1=$(echo_without_newline "$commit_message" | git -C "$1" commit-tree $tree_sha1) + tag_sha1=$(echo_without_newline "$tag_content" | git -C "$1" hash-object -t tag --stdin -w) + tag_size=$(strlen "$tag_content") +} + +# This section tests --batch-command with remote-object-info command +# Since "%(objecttype)" is currently not supported by the command remote-object-info , +# the filters are set to "%(objectname) %(objectsize)" in some test cases. + +# Test --batch-command remote-object-info with 'git://' transport +. "$TEST_DIRECTORY"/lib-git-daemon.sh +start_git_daemon --export-all --enable=receive-pack +daemon_parent=$GIT_DAEMON_DOCUMENT_ROOT_PATH/parent + +test_expect_success 'create repo to be served by git-daemon' ' + git init "$daemon_parent" && + echo_without_newline "$hello_content" > $daemon_parent/hello && + git -C "$daemon_parent" update-index --add hello && + git -C "$daemon_parent" config transfer.advertiseobjectinfo true && + git clone "$GIT_DAEMON_URL/parent" -n "$daemon_parent/daemon_client_empty" +' + +test_expect_success 'batch-command remote-object-info git://' ' + ( + set_transport_variables "$daemon_parent" && + cd "$daemon_parent/daemon_client_empty" && + + # These results prove remote-object-info can get object info from the remote + echo "$hello_sha1 $hello_size" >expect && + echo "$tree_sha1 $tree_size" >>expect && + echo "$commit_sha1 $commit_size" >>expect && + echo "$tag_sha1 $tag_size" >>expect && + + # These results prove remote-object-info did not download objects from the remote + echo "$hello_sha1 missing" >>expect && + echo "$tree_sha1 missing" >>expect && + echo "$commit_sha1 missing" >>expect && + echo "$tag_sha1 missing" >>expect && + + git cat-file --batch-command="%(objectname) %(objectsize)" >actual <<-EOF && + remote-object-info "$GIT_DAEMON_URL/parent" $hello_sha1 + remote-object-info "$GIT_DAEMON_URL/parent" $tree_sha1 + remote-object-info "$GIT_DAEMON_URL/parent" $commit_sha1 + remote-object-info "$GIT_DAEMON_URL/parent" $tag_sha1 + info $hello_sha1 + info $tree_sha1 + info $commit_sha1 + info $tag_sha1 + EOF + test_cmp expect actual + ) +' + +test_expect_success 'batch-command remote-object-info git:// multiple sha1 per line' ' + ( + set_transport_variables "$daemon_parent" && + cd "$daemon_parent/daemon_client_empty" && + + # These results prove remote-object-info can get object info from the remote + echo "$hello_sha1 $hello_size" >expect && + echo "$tree_sha1 $tree_size" >>expect && + echo "$commit_sha1 $commit_size" >>expect && + echo "$tag_sha1 $tag_size" >>expect && + + # These results prove remote-object-info did not download objects from the remote + echo "$hello_sha1 missing" >>expect && + echo "$tree_sha1 missing" >>expect && + echo "$commit_sha1 missing" >>expect && + echo "$tag_sha1 missing" >>expect && + + git cat-file --batch-command="%(objectname) %(objectsize)" >actual <<-EOF && + remote-object-info "$GIT_DAEMON_URL/parent" $hello_sha1 $tree_sha1 $commit_sha1 $tag_sha1 + info $hello_sha1 + info $tree_sha1 + info $commit_sha1 + info $tag_sha1 + EOF + test_cmp expect actual + ) +' + +test_expect_success 'batch-command remote-object-info git:// default filter' ' + ( + set_transport_variables "$daemon_parent" && + cd "$daemon_parent/daemon_client_empty" && + + echo "$hello_sha1 $hello_size" >expect && + echo "$tree_sha1 $tree_size" >>expect && + echo "$commit_sha1 $commit_size" >>expect && + echo "$tag_sha1 $tag_size" >>expect && + GIT_TRACE_PACKET=1 git cat-file --batch-command >actual <<-EOF && + remote-object-info "$GIT_DAEMON_URL/parent" $hello_sha1 $tree_sha1 + remote-object-info "$GIT_DAEMON_URL/parent" $commit_sha1 $tag_sha1 + EOF + test_cmp expect actual + ) +' + +test_expect_success 'batch-command --buffer remote-object-info git://' ' + ( + set_transport_variables "$daemon_parent" && + cd "$daemon_parent/daemon_client_empty" && + + # These results prove remote-object-info can get object info from the remote + echo "$hello_sha1 $hello_size" >expect && + echo "$tree_sha1 $tree_size" >>expect && + echo "$commit_sha1 $commit_size" >>expect && + echo "$tag_sha1 $tag_size" >>expect && + + # These results prove remote-object-info did not download objects from the remote + echo "$hello_sha1 missing" >>expect && + echo "$tree_sha1 missing" >>expect && + echo "$commit_sha1 missing" >>expect && + echo "$tag_sha1 missing" >>expect && + + git cat-file --batch-command="%(objectname) %(objectsize)" --buffer >actual <<-EOF && + remote-object-info "$GIT_DAEMON_URL/parent" $hello_sha1 $tree_sha1 + remote-object-info "$GIT_DAEMON_URL/parent" $commit_sha1 $tag_sha1 + info $hello_sha1 + info $tree_sha1 + info $commit_sha1 + info $tag_sha1 + flush + EOF + test_cmp expect actual + ) +' + +test_expect_success 'batch-command -Z remote-object-info git:// default filter' ' + ( + set_transport_variables "$daemon_parent" && + cd "$daemon_parent/daemon_client_empty" && + + printf "%s\0" "$hello_sha1 $hello_size" >expect && + printf "%s\0" "$tree_sha1 $tree_size" >>expect && + printf "%s\0" "$commit_sha1 $commit_size" >>expect && + printf "%s\0" "$tag_sha1 $tag_size" >>expect && + + printf "%s\0" "$hello_sha1 missing" >>expect && + printf "%s\0" "$tree_sha1 missing" >>expect && + printf "%s\0" "$commit_sha1 missing" >>expect && + printf "%s\0" "$tag_sha1 missing" >>expect && + + batch_input="remote-object-info $GIT_DAEMON_URL/parent $hello_sha1 $tree_sha1 +remote-object-info $GIT_DAEMON_URL/parent $commit_sha1 $tag_sha1 +info $hello_sha1 +info $tree_sha1 +info $commit_sha1 +info $tag_sha1 +" && + echo_without_newline_nul "$batch_input" >commands_null_delimited && + + git cat-file --batch-command -Z < commands_null_delimited >actual && + test_cmp expect actual + ) +' + +# Test --batch-command remote-object-info with 'git://' and +# transfer.advertiseobjectinfo set to false, i.e. server does not have object-info capability + +test_expect_success 'remote-object-info fallback git://: fetch objects to client' ' + ( + git -C "$daemon_parent" config transfer.advertiseobjectinfo false && + set_transport_variables "$daemon_parent" && + cd "$daemon_parent/daemon_client_empty" && + + # Prove object is not on the client + echo "$hello_sha1 missing" >expect && + echo "$tree_sha1 missing" >>expect && + echo "$commit_sha1 missing" >>expect && + echo "$tag_sha1 missing" >>expect && + + # These results prove remote-object-info can retrieve object info + echo "$hello_sha1 $hello_size" >>expect && + echo "$tree_sha1 $tree_size" >>expect && + echo "$commit_sha1 $commit_size" >>expect && + echo "$tag_sha1 $tag_size" >>expect && + + # These results are for the info command + # They prove objects are downloaded + echo "$hello_sha1 $hello_size" >>expect && + echo "$tree_sha1 $tree_size" >>expect && + echo "$commit_sha1 $commit_size" >>expect && + echo "$tag_sha1 $tag_size" >>expect && + + git cat-file --batch-command="%(objectname) %(objectsize)" >actual <<-EOF && + info $hello_sha1 + info $tree_sha1 + info $commit_sha1 + info $tag_sha1 + remote-object-info $GIT_DAEMON_URL/parent $hello_sha1 $tree_sha1 $commit_sha1 $tag_sha1 + info $hello_sha1 + info $tree_sha1 + info $commit_sha1 + info $tag_sha1 + EOF + + # revert server state back + git -C "$daemon_parent" config transfer.advertiseobjectinfo true && + + test_cmp expect actual + ) +' + +stop_git_daemon + +# Test --batch-command remote-object-info with 'file://' transport +# shellcheck disable=SC2016 +test_expect_success 'create repo to be served by file:// transport' ' + git init server && + git -C server config protocol.version 2 && + git -C server config transfer.advertiseobjectinfo true && + echo_without_newline "$hello_content" > server/hello && + git -C server update-index --add hello && + git clone -n "file://$(pwd)/server" file_client_empty +' + +test_expect_success 'batch-command remote-object-info file://' ' + ( + set_transport_variables "server" && + server_path="$(pwd)/server" && + cd file_client_empty && + + # These results prove remote-object-info can get object info from the remote + echo "$hello_sha1 $hello_size" >expect && + echo "$tree_sha1 $tree_size" >>expect && + echo "$commit_sha1 $commit_size" >>expect && + echo "$tag_sha1 $tag_size" >>expect && + + # These results prove remote-object-info did not download objects from the remote + echo "$hello_sha1 missing" >>expect && + echo "$tree_sha1 missing" >>expect && + echo "$commit_sha1 missing" >>expect && + echo "$tag_sha1 missing" >>expect && + + git cat-file --batch-command="%(objectname) %(objectsize)" >actual <<-EOF && + remote-object-info "file://${server_path}" $hello_sha1 + remote-object-info "file://${server_path}" $tree_sha1 + remote-object-info "file://${server_path}" $commit_sha1 + remote-object-info "file://${server_path}" $tag_sha1 + info $hello_sha1 + info $tree_sha1 + info $commit_sha1 + info $tag_sha1 + EOF + test_cmp expect actual + ) +' + +test_expect_success 'batch-command remote-object-info file:// multiple sha1 per line' ' + ( + set_transport_variables "server" && + server_path="$(pwd)/server" && + cd file_client_empty && + + # These results prove remote-object-info can get object info from the remote + echo "$hello_sha1 $hello_size" >expect && + echo "$tree_sha1 $tree_size" >>expect && + echo "$commit_sha1 $commit_size" >>expect && + echo "$tag_sha1 $tag_size" >>expect && + + # These results prove remote-object-info did not download objects from the remote + echo "$hello_sha1 missing" >>expect && + echo "$tree_sha1 missing" >>expect && + echo "$commit_sha1 missing" >>expect && + echo "$tag_sha1 missing" >>expect && + + + git cat-file --batch-command="%(objectname) %(objectsize)" >actual <<-EOF && + remote-object-info "file://${server_path}" $hello_sha1 $tree_sha1 $commit_sha1 $tag_sha1 + info $hello_sha1 + info $tree_sha1 + info $commit_sha1 + info $tag_sha1 + EOF + test_cmp expect actual + ) +' + +test_expect_success 'batch-command --buffer remote-object-info file://' ' + ( + set_transport_variables "server" && + server_path="$(pwd)/server" && + cd file_client_empty && + + # These results prove remote-object-info can get object info from the remote + echo "$hello_sha1 $hello_size" >expect && + echo "$tree_sha1 $tree_size" >>expect && + echo "$commit_sha1 $commit_size" >>expect && + echo "$tag_sha1 $tag_size" >>expect && + + # These results prove remote-object-info did not download objects from the remote + echo "$hello_sha1 missing" >>expect && + echo "$tree_sha1 missing" >>expect && + echo "$commit_sha1 missing" >>expect && + echo "$tag_sha1 missing" >>expect && + + git cat-file --batch-command="%(objectname) %(objectsize)" --buffer >actual <<-EOF && + remote-object-info "file://${server_path}" $hello_sha1 $tree_sha1 + remote-object-info "file://${server_path}" $commit_sha1 $tag_sha1 + info $hello_sha1 + info $tree_sha1 + info $commit_sha1 + info $tag_sha1 + flush + EOF + test_cmp expect actual + ) +' + +test_expect_success 'batch-command remote-object-info file:// default filter' ' + ( + set_transport_variables "server" && + server_path="$(pwd)/server" && + cd file_client_empty && + + echo "$hello_sha1 $hello_size" >expect && + echo "$tree_sha1 $tree_size" >>expect && + echo "$commit_sha1 $commit_size" >>expect && + echo "$tag_sha1 $tag_size" >>expect && + + git cat-file --batch-command >actual <<-EOF && + remote-object-info "file://${server_path}" $hello_sha1 $tree_sha1 + remote-object-info "file://${server_path}" $commit_sha1 $tag_sha1 + EOF + test_cmp expect actual + ) +' + +test_expect_success 'batch-command -Z remote-object-info file:// default filter' ' + ( + set_transport_variables "server" && + server_path="$(pwd)/server" && + cd file_client_empty && + + printf "%s\0" "$hello_sha1 $hello_size" >expect && + printf "%s\0" "$tree_sha1 $tree_size" >>expect && + printf "%s\0" "$commit_sha1 $commit_size" >>expect && + printf "%s\0" "$tag_sha1 $tag_size" >>expect && + + printf "%s\0" "$hello_sha1 missing" >>expect && + printf "%s\0" "$tree_sha1 missing" >>expect && + printf "%s\0" "$commit_sha1 missing" >>expect && + printf "%s\0" "$tag_sha1 missing" >>expect && + + batch_input="remote-object-info \"file://${server_path}\" $hello_sha1 $tree_sha1 +remote-object-info \"file://${server_path}\" $commit_sha1 $tag_sha1 +info $hello_sha1 +info $tree_sha1 +info $commit_sha1 +info $tag_sha1 +" && + echo_without_newline_nul "$batch_input" >commands_null_delimited && + + git cat-file --batch-command -Z < commands_null_delimited >actual && + test_cmp expect actual + ) +' + +# Test --batch-command remote-object-info with 'file://' and +# transfer.advertiseobjectinfo set to false, i.e. server does not have object-info capability + +test_expect_success 'remote-object-info fallback file://: fetch objects to client' ' + ( + set_transport_variables "server" && + server_path="$(pwd)/server" && + git -C "${server_path}" config transfer.advertiseobjectinfo false && + cd file_client_empty && + + # Prove object is not on the client + echo "$hello_sha1 missing" >expect && + echo "$tree_sha1 missing" >>expect && + echo "$commit_sha1 missing" >>expect && + echo "$tag_sha1 missing" >>expect && + + # These results prove remote-object-info can retrieve object info + echo "$hello_sha1 $hello_size" >>expect && + echo "$tree_sha1 $tree_size" >>expect && + echo "$commit_sha1 $commit_size" >>expect && + echo "$tag_sha1 $tag_size" >>expect && + + # These results are for the info command + # They prove objects are downloaded + echo "$hello_sha1 $hello_size" >>expect && + echo "$tree_sha1 $tree_size" >>expect && + echo "$commit_sha1 $commit_size" >>expect && + echo "$tag_sha1 $tag_size" >>expect && + + git cat-file --batch-command="%(objectname) %(objectsize)" >actual <<-EOF && + info $hello_sha1 + info $tree_sha1 + info $commit_sha1 + info $tag_sha1 + remote-object-info "file://${server_path}" $hello_sha1 $tree_sha1 $commit_sha1 $tag_sha1 + info $hello_sha1 + info $tree_sha1 + info $commit_sha1 + info $tag_sha1 + EOF + + # revert server state back + git -C "${server_path}" config transfer.advertiseobjectinfo true && + test_cmp expect actual + ) +' + +# Test --batch-command remote-object-info with 'http://' transport with +# transfer.advertiseobjectinfo set to true, i.e. server has object-info capability + +. "$TEST_DIRECTORY"/lib-httpd.sh +start_httpd + +test_expect_success 'create repo to be served by http:// transport' ' + git init "$HTTPD_DOCUMENT_ROOT_PATH/http_parent" && + git -C "$HTTPD_DOCUMENT_ROOT_PATH/http_parent" config http.receivepack true && + git -C "$HTTPD_DOCUMENT_ROOT_PATH/http_parent" config transfer.advertiseobjectinfo true && + echo_without_newline "$hello_content" > $HTTPD_DOCUMENT_ROOT_PATH/http_parent/hello && + git -C "$HTTPD_DOCUMENT_ROOT_PATH/http_parent" update-index --add hello && + git clone "$HTTPD_URL/smart/http_parent" -n "$HTTPD_DOCUMENT_ROOT_PATH/http_client_empty" +' + +test_expect_success 'batch-command remote-object-info http://' ' + ( + set_transport_variables "$HTTPD_DOCUMENT_ROOT_PATH/http_parent" && + cd "$HTTPD_DOCUMENT_ROOT_PATH/http_client_empty" && + + # These results prove remote-object-info can get object info from the remote + echo "$hello_sha1 $hello_size" >expect && + echo "$tree_sha1 $tree_size" >>expect && + echo "$commit_sha1 $commit_size" >>expect && + echo "$tag_sha1 $tag_size" >>expect && + + # These results prove remote-object-info did not download objects from the remote + echo "$hello_sha1 missing" >>expect && + echo "$tree_sha1 missing" >>expect && + echo "$commit_sha1 missing" >>expect && + echo "$tag_sha1 missing" >>expect && + + git cat-file --batch-command="%(objectname) %(objectsize)" >actual <<-EOF && + remote-object-info "$HTTPD_URL/smart/http_parent" $hello_sha1 + remote-object-info "$HTTPD_URL/smart/http_parent" $tree_sha1 + remote-object-info "$HTTPD_URL/smart/http_parent" $commit_sha1 + remote-object-info "$HTTPD_URL/smart/http_parent" $tag_sha1 + info $hello_sha1 + info $tree_sha1 + info $commit_sha1 + info $tag_sha1 + EOF + test_cmp expect actual + ) +' + +test_expect_success 'batch-command remote-object-info http:// one line' ' + ( + set_transport_variables "$HTTPD_DOCUMENT_ROOT_PATH/http_parent" && + cd "$HTTPD_DOCUMENT_ROOT_PATH/http_client_empty" && + + # These results prove remote-object-info can get object info from the remote + echo "$hello_sha1 $hello_size" >expect && + echo "$tree_sha1 $tree_size" >>expect && + echo "$commit_sha1 $commit_size" >>expect && + echo "$tag_sha1 $tag_size" >>expect && + + # These results prove remote-object-info did not download objects from the remote + echo "$hello_sha1 missing" >>expect && + echo "$tree_sha1 missing" >>expect && + echo "$commit_sha1 missing" >>expect && + echo "$tag_sha1 missing" >>expect && + + git cat-file --batch-command="%(objectname) %(objectsize)" >actual <<-EOF && + remote-object-info "$HTTPD_URL/smart/http_parent" $hello_sha1 $tree_sha1 $commit_sha1 $tag_sha1 + info $hello_sha1 + info $tree_sha1 + info $commit_sha1 + info $tag_sha1 + EOF + test_cmp expect actual + ) +' + +test_expect_success 'batch-command --buffer remote-object-info http://' ' + ( + set_transport_variables "$HTTPD_DOCUMENT_ROOT_PATH/http_parent" && + cd "$HTTPD_DOCUMENT_ROOT_PATH/http_client_empty" && + + # These results prove remote-object-info can get object info from the remote + echo "$hello_sha1 $hello_size" >expect && + echo "$tree_sha1 $tree_size" >>expect && + echo "$commit_sha1 $commit_size" >>expect && + echo "$tag_sha1 $tag_size" >>expect && + + # These results prove remote-object-info did not download objects from the remote + echo "$hello_sha1 missing" >>expect && + echo "$tree_sha1 missing" >>expect && + echo "$commit_sha1 missing" >>expect && + echo "$tag_sha1 missing" >>expect && + + git cat-file --batch-command="%(objectname) %(objectsize)" --buffer >actual <<-EOF && + remote-object-info "$HTTPD_URL/smart/http_parent" $hello_sha1 $tree_sha1 + remote-object-info "$HTTPD_URL/smart/http_parent" $commit_sha1 $tag_sha1 + info $hello_sha1 + info $tree_sha1 + info $commit_sha1 + info $tag_sha1 + flush + EOF + test_cmp expect actual + ) +' + +test_expect_success 'batch-command remote-object-info http:// default filter' ' + ( + set_transport_variables "$HTTPD_DOCUMENT_ROOT_PATH/http_parent" && + cd "$HTTPD_DOCUMENT_ROOT_PATH/http_client_empty" && + + echo "$hello_sha1 $hello_size" >expect && + echo "$tree_sha1 $tree_size" >>expect && + echo "$commit_sha1 $commit_size" >>expect && + echo "$tag_sha1 $tag_size" >>expect && + + git cat-file --batch-command >actual <<-EOF && + remote-object-info "$HTTPD_URL/smart/http_parent" $hello_sha1 $tree_sha1 + remote-object-info "$HTTPD_URL/smart/http_parent" $commit_sha1 $tag_sha1 + EOF + test_cmp expect actual + ) +' + +test_expect_success 'batch-command -Z remote-object-info http:// default filter' ' + ( + set_transport_variables "$HTTPD_DOCUMENT_ROOT_PATH/http_parent" && + cd "$HTTPD_DOCUMENT_ROOT_PATH/http_client_empty" && + + printf "%s\0" "$hello_sha1 $hello_size" >expect && + printf "%s\0" "$tree_sha1 $tree_size" >>expect && + printf "%s\0" "$commit_sha1 $commit_size" >>expect && + printf "%s\0" "$tag_sha1 $tag_size" >>expect && + + batch_input="remote-object-info $HTTPD_URL/smart/http_parent $hello_sha1 $tree_sha1 +remote-object-info $HTTPD_URL/smart/http_parent $commit_sha1 $tag_sha1 +" && + echo_without_newline_nul "$batch_input" >commands_null_delimited && + + git cat-file --batch-command -Z < commands_null_delimited >actual && + test_cmp expect actual + ) +' + +test_expect_success 'remote-object-info fails on unspported filter option (objectsize:disk)' ' + ( + set_transport_variables "$HTTPD_DOCUMENT_ROOT_PATH/http_parent" && + cd "$HTTPD_DOCUMENT_ROOT_PATH/http_parent" && + + test_must_fail git cat-file --batch-command="%(objectsize:disk)" 2>err <<-EOF && + remote-object-info "$HTTPD_URL/smart/http_parent" $hello_sha1 + EOF + test_grep "%(objectsize:disk) is currently not supported with remote-object-info" err + ) +' + +test_expect_success 'remote-object-info fails on unspported filter option (deltabase)' ' + ( + set_transport_variables "$HTTPD_DOCUMENT_ROOT_PATH/http_parent" && + cd "$HTTPD_DOCUMENT_ROOT_PATH/http_parent" && + + test_must_fail git cat-file --batch-command="%(deltabase)" 2>err <<-EOF && + remote-object-info "$HTTPD_URL/smart/http_parent" $hello_sha1 + EOF + test_grep "%(deltabase) is currently not supported with remote-object-info" err + ) +' + +test_expect_success 'remote-object-info fails on server with legacy protocol' ' + ( + set_transport_variables "$HTTPD_DOCUMENT_ROOT_PATH/http_parent" && + cd "$HTTPD_DOCUMENT_ROOT_PATH/http_parent" && + + test_must_fail git -c protocol.version=0 cat-file --batch-command="%(objectname) %(objectsize)" 2>err <<-EOF && + remote-object-info "$HTTPD_URL/smart/http_parent" $hello_sha1 + EOF + test_grep "remote-object-info requires protocol v2" err + ) +' + +test_expect_success 'remote-object-info fails on server with legacy protocol fallback' ' + ( + set_transport_variables "$HTTPD_DOCUMENT_ROOT_PATH/http_parent" && + cd "$HTTPD_DOCUMENT_ROOT_PATH/http_parent" && + + test_must_fail git -c protocol.version=0 cat-file --batch-command 2>err <<-EOF && + remote-object-info "$HTTPD_URL/smart/http_parent" $hello_sha1 + EOF + test_grep "remote-object-info requires protocol v2" err + ) +' + +test_expect_success 'remote-object-info fails on malformed OID' ' + ( + set_transport_variables "$HTTPD_DOCUMENT_ROOT_PATH/http_parent" && + cd "$HTTPD_DOCUMENT_ROOT_PATH/http_parent" && + malformed_object_id="this_id_is_not_valid" && + + test_must_fail git cat-file --batch-command="%(objectname) %(objectsize)" 2>err <<-EOF && + remote-object-info "$HTTPD_URL/smart/http_parent" $malformed_object_id + EOF + test_grep "Not a valid object name '$malformed_object_id'" err + ) +' + +test_expect_success 'remote-object-info fails on malformed OID fallback' ' + ( + set_transport_variables "$HTTPD_DOCUMENT_ROOT_PATH/http_parent" && + cd "$HTTPD_DOCUMENT_ROOT_PATH/http_parent" && + malformed_object_id="this_id_is_not_valid" && + + test_must_fail git cat-file --batch-command 2>err <<-EOF && + remote-object-info "$HTTPD_URL/smart/http_parent" $malformed_object_id + EOF + test_grep "Not a valid object name '$malformed_object_id'" err + ) +' + +test_expect_success 'remote-object-info fails on missing OID' ' + ( + set_transport_variables "$HTTPD_DOCUMENT_ROOT_PATH/http_parent" && + git clone "$HTTPD_DOCUMENT_ROOT_PATH/http_parent" missing_oid_repo && + test_commit -C missing_oid_repo message1 c.txt && + cd missing_oid_repo && + + object_id=$(git rev-parse message1:c.txt) && + test_must_fail git cat-file --batch-command="%(objectname) %(objectsize)" 2>err <<-EOF && + remote-object-info "$HTTPD_URL/smart/http_parent" $object_id + EOF + test_grep "object-info: not our ref $object_id" err + ) +' + +# Test --batch-command remote-object-info with 'http://' transport and +# transfer.advertiseobjectinfo set to false, i.e. server does not have object-info capability + +test_expect_success 'remote-object-info fallback http://: fetch objects to client' ' + ( + set_transport_variables "$HTTPD_DOCUMENT_ROOT_PATH/http_parent" && + git -C "$HTTPD_DOCUMENT_ROOT_PATH/http_parent" config transfer.advertiseobjectinfo false && + cd "$HTTPD_DOCUMENT_ROOT_PATH/http_client_empty" && + + # Prove object is not on the client + echo "$hello_sha1 missing" >expect && + echo "$tree_sha1 missing" >>expect && + echo "$commit_sha1 missing" >>expect && + echo "$tag_sha1 missing" >>expect && + + # These results prove remote-object-info can retrieve object info + echo "$hello_sha1 $hello_size" >>expect && + echo "$tree_sha1 $tree_size" >>expect && + echo "$commit_sha1 $commit_size" >>expect && + echo "$tag_sha1 $tag_size" >>expect && + + # These results are for the info command + # They prove objects are downloaded + echo "$hello_sha1 $hello_size" >>expect && + echo "$tree_sha1 $tree_size" >>expect && + echo "$commit_sha1 $commit_size" >>expect && + echo "$tag_sha1 $tag_size" >>expect && + + git cat-file --batch-command >actual <<-EOF && + info $hello_sha1 + info $tree_sha1 + info $commit_sha1 + info $tag_sha1 + remote-object-info "$HTTPD_URL/smart/http_parent" $hello_sha1 $tree_sha1 $commit_sha1 $tag_sha1 + info $hello_sha1 + info $tree_sha1 + info $commit_sha1 + info $tag_sha1 + EOF + + # revert server state back + git -C "$HTTPD_DOCUMENT_ROOT_PATH/http_parent" config transfer.advertiseobjectinfo true && + + test_cmp expect actual + ) +' + +# DO NOT add non-httpd-specific tests here, because the last part of this +# test script is only executed when httpd is available and enabled. + +test_done