From patchwork Thu Aug 8 07:35:27 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Patrick Steinhardt X-Patchwork-Id: 13757056 Received: from fout3-smtp.messagingengine.com (fout3-smtp.messagingengine.com [103.168.172.146]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 38E14145A05 for ; Thu, 8 Aug 2024 07:35:31 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=103.168.172.146 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1723102534; cv=none; b=mH3c7UIGikd/eHHZ7cZCVBYiFyiNgAu8flTopFdMNy8BQfZ/HQgJ5YUlts+k5RW2DueSfV2N97k7Z6JtFZxicBJk0b0FYk+CuxY0Rjd9R1eHt3EZHsKoMcPSHHPn/NlNiVSc4LR5E0nt4ke0iLEzabMcIJWvibgvu6qprgqVJE4= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1723102534; c=relaxed/simple; bh=4V9T7K0FBuMjUEWo3mby++Uer+2U+vCPWhcPvPX31CY=; h=Date:From:To:Cc:Subject:Message-ID:References:MIME-Version: Content-Type:Content-Disposition:In-Reply-To; b=W3kqFyTYMmUUY4oo8XUNBa0p9VKrR+RjqwdRXIQ8bnfOTddvvXPuLEDKBlZ4SXwqOimE0FNFd7M+bE5d+nl391ETkq2H6bHuX9A1BY8ynHTajVrGtBU+kDw0BbVdXwA7+7e6kyc6rtTwdU6mYvvBZy2tRYDScLnG59vdVaFYo2w= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=pks.im; spf=pass smtp.mailfrom=pks.im; dkim=pass (2048-bit key) header.d=pks.im header.i=@pks.im header.b=V5LbdtD9; dkim=pass (2048-bit key) header.d=messagingengine.com header.i=@messagingengine.com header.b=NpV5DqJ+; arc=none smtp.client-ip=103.168.172.146 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=pks.im Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=pks.im Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=pks.im header.i=@pks.im header.b="V5LbdtD9"; dkim=pass (2048-bit key) header.d=messagingengine.com header.i=@messagingengine.com header.b="NpV5DqJ+" Received: from compute3.internal (compute3.nyi.internal [10.202.2.43]) by mailfout.nyi.internal (Postfix) with ESMTP id 47207138FD36; Thu, 8 Aug 2024 03:35:31 -0400 (EDT) Received: from mailfrontend1 ([10.202.2.162]) by compute3.internal (MEProxy); Thu, 08 Aug 2024 03:35:31 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=pks.im; h=cc:cc :content-type:content-type:date:date:from:from:in-reply-to :in-reply-to:message-id:mime-version:references:reply-to:subject :subject:to:to; s=fm3; t=1723102531; x=1723188931; bh=glvkNn8FMX Ynih0kSifQiLh8ZWcztl+ng1GKSCW8iZM=; b=V5LbdtD9EDcSnSghaRCXIzE7BP MwB8gRf8u6IC/oVUUQYFKftn4Bls30Os48X0g4uWe4vzZoYUqbgpDS35r8B2mAWv NHqvK+ehQudrjfKFp6gewN+625vEC4QPz+kXh8G0PoUqlzAQM90DJbsFSo5BMxeR //U/yZ0EKIFGIIJOdfR1KrOfX9Y1J4vwTVmY834M97UP1zcCRi9vJrjd9CTK15Ck GfyB87oZozBIyvzISXl90TwpSS+m042I/KdJza+qhg5l9r4yWVgTe9lhQbdranRW Prk5yni8ZYamNCRCTB9pDHYssUcw6w/N0TNXcKbCuW1EETcJt9hIuR5E3r8Q== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:cc:content-type:content-type:date:date :feedback-id:feedback-id:from:from:in-reply-to:in-reply-to :message-id:mime-version:references:reply-to:subject:subject:to :to:x-me-proxy:x-me-proxy:x-me-sender:x-me-sender:x-sasl-enc; s= fm3; t=1723102531; x=1723188931; bh=glvkNn8FMXYnih0kSifQiLh8ZWcz tl+ng1GKSCW8iZM=; b=NpV5DqJ+yHUJIhtp8YMFDQRofZfFXrZ7cR1ECcaR+pXe wAYHuFgCjBGpEq2+lvmofNtb12zt5miVJQC9VmQSah03nMabbZYajJ+FNvM9pSV2 V2Z4XwI6cSd+yic6l6MGe/aP/u9XprhhOdIKfOQ4kezfpHNw48y95PDYiMFyfmsP bnOPwifCVGcEEYCNhKN4T3pGpwmFgMQvMVXDlhOPKnbN2WdI82xUxAQ5bk7/5CBC 7ilfrxs2F4fVDkj/YrCxBBrxnTFlCf/crM5dmbSy+207VnO2Np6ubdFi30QWboRv f5ehGazAEA6AhHb0XpUeEq3Ww2QV9qUFXHgoif/pfg== X-ME-Sender: X-ME-Received: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgeeftddrledugdduvddvucetufdoteggodetrfdotf fvucfrrhhofhhilhgvmecuhfgrshhtofgrihhlpdggtfgfnhhsuhgsshgtrhhisggvpdfu rfetoffkrfgpnffqhgenuceurghilhhouhhtmecufedttdenucesvcftvggtihhpihgvnh htshculddquddttddmnecujfgurhepfffhvfevuffkfhggtggujgesghdtreertddtvden ucfhrhhomheprfgrthhrihgtkhcuufhtvghinhhhrghrughtuceophhssehpkhhsrdhimh eqnecuggftrfgrthhtvghrnhepueektdevtdffveeljeetgfehheeigeekleduvdeffeeg hefgledttdehjeelffetnecuvehluhhsthgvrhfuihiivgeptdenucfrrghrrghmpehmrg hilhhfrhhomhepphhssehpkhhsrdhimhdpnhgspghrtghpthhtohepgedpmhhouggvpehs mhhtphhouhhtpdhrtghpthhtohepghhithhsthgvrhesphhosghogidrtghomhdprhgtph htthhopehjohhlrghnugesghhmrghilhdrtghomhdprhgtphhtthhopehsuhhnshhhihhn vgesshhunhhshhhinhgvtghordgtohhmpdhrtghpthhtohepghhithesvhhgvghrrdhkvg hrnhgvlhdrohhrgh X-ME-Proxy: Feedback-ID: i197146af:Fastmail Received: by mail.messagingengine.com (Postfix) with ESMTPA; Thu, 8 Aug 2024 03:35:30 -0400 (EDT) Received: by vm-mail (OpenSMTPD) with ESMTPSA id df73f779 (TLSv1.3:TLS_AES_256_GCM_SHA384:256:NO); Thu, 8 Aug 2024 07:35:23 +0000 (UTC) Date: Thu, 8 Aug 2024 09:35:27 +0200 From: Patrick Steinhardt To: git@vger.kernel.org Cc: Eric Sunshine , Jeppe =?utf-8?b?w5hsYW5k?= , Junio C Hamano Subject: [PATCH v2 1/8] git-submodule.sh: break overly long command lines Message-ID: <6513c6b17d261ed5900aa904a438c8770ae026cd.1723102259.git.ps@pks.im> References: Precedence: bulk X-Mailing-List: git@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: For most of the subcommands of git-submodule(1), we end up passing a bunch of arguments to the submodule helper. This quickly leads to overly long lines, where it becomes hard to spot what has changed when one needs to modify them. Break up these lines into one argument per line, similarly to how it is done for the "clone" subcommand already. Signed-off-by: Patrick Steinhardt --- git-submodule.sh | 64 +++++++++++++++++++++++++++++++++++++++++------- 1 file changed, 55 insertions(+), 9 deletions(-) diff --git a/git-submodule.sh b/git-submodule.sh index 7f9582d923..fd588b1864 100755 --- a/git-submodule.sh +++ b/git-submodule.sh @@ -129,7 +129,17 @@ cmd_add() usage fi - git ${wt_prefix:+-C "$wt_prefix"} submodule--helper add ${quiet:+--quiet} ${force:+--force} ${progress:+"--progress"} ${branch:+--branch "$branch"} ${reference_path:+--reference "$reference_path"} ${dissociate:+--dissociate} ${custom_name:+--name "$custom_name"} ${depth:+"$depth"} -- "$@" + git ${wt_prefix:+-C "$wt_prefix"} submodule--helper add \ + ${quiet:+--quiet} \ + ${force:+--force} \ + ${progress:+"--progress"} \ + ${branch:+--branch "$branch"} \ + ${reference_path:+--reference "$reference_path"} \ + ${dissociate:+--dissociate} \ + ${custom_name:+--name "$custom_name"} \ + ${depth:+"$depth"} \ + -- \ + "$@" } # @@ -160,7 +170,11 @@ cmd_foreach() shift done - git ${wt_prefix:+-C "$wt_prefix"} submodule--helper foreach ${quiet:+--quiet} ${recursive:+--recursive} -- "$@" + git ${wt_prefix:+-C "$wt_prefix"} submodule--helper foreach \ + ${quiet:+--quiet} \ + ${recursive:+--recursive} \ + -- \ + "$@" } # @@ -191,7 +205,10 @@ cmd_init() shift done - git ${wt_prefix:+-C "$wt_prefix"} submodule--helper init ${quiet:+--quiet} -- "$@" + git ${wt_prefix:+-C "$wt_prefix"} submodule--helper init \ + ${quiet:+--quiet} \ + -- \ + "$@" } # @@ -227,7 +244,12 @@ cmd_deinit() shift done - git ${wt_prefix:+-C "$wt_prefix"} submodule--helper deinit ${quiet:+--quiet} ${force:+--force} ${deinit_all:+--all} -- "$@" + git ${wt_prefix:+-C "$wt_prefix"} submodule--helper deinit \ + ${quiet:+--quiet} \ + ${force:+--force} \ + ${deinit_all:+--all} \ + -- \ + "$@" } # @@ -399,7 +421,12 @@ cmd_set_branch() { shift done - git ${wt_prefix:+-C "$wt_prefix"} submodule--helper set-branch ${quiet:+--quiet} ${branch:+--branch "$branch"} ${default:+--default} -- "$@" + git ${wt_prefix:+-C "$wt_prefix"} submodule--helper set-branch \ + ${quiet:+--quiet} \ + ${branch:+--branch "$branch"} \ + ${default:+--default} \ + -- \ + "$@" } # @@ -428,7 +455,10 @@ cmd_set_url() { shift done - git ${wt_prefix:+-C "$wt_prefix"} submodule--helper set-url ${quiet:+--quiet} -- "$@" + git ${wt_prefix:+-C "$wt_prefix"} submodule--helper set-url \ + ${quiet:+--quiet} \ + -- \ + "$@" } # @@ -480,7 +510,13 @@ cmd_summary() { shift done - git ${wt_prefix:+-C "$wt_prefix"} submodule--helper summary ${files:+--files} ${cached:+--cached} ${for_status:+--for-status} ${summary_limit:+-n $summary_limit} -- "$@" + git ${wt_prefix:+-C "$wt_prefix"} submodule--helper summary \ + ${files:+--files} \ + ${cached:+--cached} \ + ${for_status:+--for-status} \ + ${summary_limit:+-n $summary_limit} \ + -- \ + "$@" } # # List all submodules, prefixed with: @@ -521,8 +557,14 @@ cmd_status() shift done - git ${wt_prefix:+-C "$wt_prefix"} submodule--helper status ${quiet:+--quiet} ${cached:+--cached} ${recursive:+--recursive} -- "$@" + git ${wt_prefix:+-C "$wt_prefix"} submodule--helper status \ + ${quiet:+--quiet} \ + ${cached:+--cached} \ + ${recursive:+--recursive} \ + -- \ + "$@" } + # # Sync remote urls for submodules # This makes the value for remote.$remote.url match the value @@ -554,7 +596,11 @@ cmd_sync() esac done - git ${wt_prefix:+-C "$wt_prefix"} submodule--helper sync ${quiet:+--quiet} ${recursive:+--recursive} -- "$@" + git ${wt_prefix:+-C "$wt_prefix"} submodule--helper sync \ + ${quiet:+--quiet} \ + ${recursive:+--recursive} \ + -- \ + "$@" } cmd_absorbgitdirs() From patchwork Thu Aug 8 07:35:32 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Patrick Steinhardt X-Patchwork-Id: 13757057 Received: from fout3-smtp.messagingengine.com (fout3-smtp.messagingengine.com [103.168.172.146]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 5E431839EB for ; Thu, 8 Aug 2024 07:35:37 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=103.168.172.146 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1723102539; cv=none; b=M59LkWKKgFRyyNM73ZW1V4K3LCjeuA3rwklX7oaA4/S/DX/nDBzNHchry9cdARsTeGKQo6IW/UUwbwBA/wURCN/36GP4Shn350DAljswRCPkddjK2XWs29ec/zV/uESCt2nrY1iBM47873+r2xbb+16jH7zsEVIxc0WxRX/EGJ8= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1723102539; c=relaxed/simple; bh=PfLweo+U7I48Z/d5SKrNgbqBNrYP8RpYdF92r0720IY=; h=Date:From:To:Cc:Subject:Message-ID:References:MIME-Version: Content-Type:Content-Disposition:In-Reply-To; b=IPPaDbMWeSAVAYPEmcOJUECMC6mZNjCqvHx/duI0euWJQ1LYgQJYxYu3jcDjdJ3en+aaD28KD3E9cCfXqgRVcYAfqbF/Vpe9PWnDwxhSx7QwKFGzJJKee+eM/0u0w3YWidgcr/pynOxmpVojpXucoAaN0gVBjzKvSxtAUB+xA1Q= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=pks.im; spf=pass smtp.mailfrom=pks.im; dkim=pass (2048-bit key) header.d=pks.im header.i=@pks.im header.b=Fi2aDgJP; dkim=pass (2048-bit key) header.d=messagingengine.com header.i=@messagingengine.com header.b=jQkD27Pb; arc=none smtp.client-ip=103.168.172.146 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=pks.im Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=pks.im Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=pks.im header.i=@pks.im header.b="Fi2aDgJP"; dkim=pass (2048-bit key) header.d=messagingengine.com header.i=@messagingengine.com header.b="jQkD27Pb" Received: from compute3.internal (compute3.nyi.internal [10.202.2.43]) by mailfout.nyi.internal (Postfix) with ESMTP id 933BA138FE1D; Thu, 8 Aug 2024 03:35:36 -0400 (EDT) Received: from mailfrontend1 ([10.202.2.162]) by compute3.internal (MEProxy); Thu, 08 Aug 2024 03:35:36 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=pks.im; h=cc:cc :content-type:content-type:date:date:from:from:in-reply-to :in-reply-to:message-id:mime-version:references:reply-to:subject :subject:to:to; s=fm3; t=1723102536; x=1723188936; bh=VhIZuX9zPV k3wp0/Dw/nV8JYey9A20+HjyRsJXG9NVw=; b=Fi2aDgJPrh6oPTVwQXBA462wNv oo5mBtUm9PC2W978BOC++B6rdhecJyVJ6ZfjydfVaig6AWhdXV18iUwNe4PRgaQ7 JzlO18WE6YH5HX+kiUgJlZ/dtBGRzn4X4BEi7NSqUSBITbzJjQriS23bpUG5t0lB iNvDQutFZXiAj9Z3vZ/NsuFSlmVUoGDcwm97hGCnvuHVrKLKddy0ykYGpfWaeMBB nGJncSYHpBo5tVTrBhezABXLLmPvU/IJuOTj/HB+NrJig+6NDi8l4idPAuvoK2Mn kflM+v6wLaKlGAfpSTZ+jl+g4FfBCC5kZUhY3nysTRpSMB4CMFJmHqyWW5yg== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:cc:content-type:content-type:date:date :feedback-id:feedback-id:from:from:in-reply-to:in-reply-to :message-id:mime-version:references:reply-to:subject:subject:to :to:x-me-proxy:x-me-proxy:x-me-sender:x-me-sender:x-sasl-enc; s= fm3; t=1723102536; x=1723188936; bh=VhIZuX9zPVk3wp0/Dw/nV8JYey9A 20+HjyRsJXG9NVw=; b=jQkD27PbB0aKWbhKTfKHpCPx50FV5GXwnCXboFhe7LvF EeMbwgYWRKwgcm22lEY1Ay4XtaTlIGRWD7Y/sK/SFnyiBLNvx7bI9HuyXOMlT5+v Xwhon3ApPkxSF6WOgLtzzczK1yCteaT/CmFbm63hA6KCpZwQSkuJcQ1cM0zuSsDj zCpIoa4xZu8+5DsIfsLtqW3JFOb/k8SgGRzkrBk9dsdIY3YG8Lh+lxOb1EE1ihR4 OXn3QPhnP1Og6NBakV10ihXMpXG/AMtrFt9ZFYmoOTazVQeZ1YyUux/clIkBNro6 J5rmZGvAnXW9PpcU1HpGATFGX7FQ4rJisq9NXNw+gA== X-ME-Sender: X-ME-Received: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgeeftddrledugdduvddvucetufdoteggodetrfdotf fvucfrrhhofhhilhgvmecuhfgrshhtofgrihhlpdggtfgfnhhsuhgsshgtrhhisggvpdfu rfetoffkrfgpnffqhgenuceurghilhhouhhtmecufedttdenucesvcftvggtihhpihgvnh htshculddquddttddmnecujfgurhepfffhvfevuffkfhggtggujgesghdtreertddtvden ucfhrhhomheprfgrthhrihgtkhcuufhtvghinhhhrghrughtuceophhssehpkhhsrdhimh eqnecuggftrfgrthhtvghrnhepueektdevtdffveeljeetgfehheeigeekleduvdeffeeg hefgledttdehjeelffetnecuvehluhhsthgvrhfuihiivgepudenucfrrghrrghmpehmrg hilhhfrhhomhepphhssehpkhhsrdhimhdpnhgspghrtghpthhtohepgedpmhhouggvpehs mhhtphhouhhtpdhrtghpthhtohepghhithesvhhgvghrrdhkvghrnhgvlhdrohhrghdprh gtphhtthhopehjohhlrghnugesghhmrghilhdrtghomhdprhgtphhtthhopehgihhtshht vghrsehpohgsohigrdgtohhmpdhrtghpthhtohepshhunhhshhhinhgvsehsuhhnshhhih hnvggtohdrtghomh X-ME-Proxy: Feedback-ID: i197146af:Fastmail Received: by mail.messagingengine.com (Postfix) with ESMTPA; Thu, 8 Aug 2024 03:35:35 -0400 (EDT) Received: by vm-mail (OpenSMTPD) with ESMTPSA id c3d0375a (TLSv1.3:TLS_AES_256_GCM_SHA384:256:NO); Thu, 8 Aug 2024 07:35:28 +0000 (UTC) Date: Thu, 8 Aug 2024 09:35:32 +0200 From: Patrick Steinhardt To: git@vger.kernel.org Cc: Eric Sunshine , Jeppe =?utf-8?b?w5hsYW5k?= , Junio C Hamano Subject: [PATCH v2 2/8] builtin/submodule: allow cloning with different ref storage format Message-ID: References: Precedence: bulk X-Mailing-List: git@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: As submodules are proper self-contained repositories, it is perfectly valid for them to have a different ref storage format than their parent repository. There is no obvious way for users to ask for the ref storage format when initializing submodules though. Whether the setup of such mixed-ref-storage-format constellations is all that useful remains to be seen. But there is no good reason to not expose such an option, and we will require it in a subsequent patch. Introduce a new `--ref-format=` option for git-submodule(1) that allows the user to pick the ref storage format. This option will also be used in a subsequent commit, where we start to propagate the same flag from git-clone(1) to cloning submodules with the `--recursive` switch. Signed-off-by: Patrick Steinhardt --- Documentation/git-submodule.txt | 5 +++- builtin/submodule--helper.c | 30 +++++++++++++++++++ git-submodule.sh | 9 ++++++ t/t7424-submodule-mixed-ref-formats.sh | 41 ++++++++++++++++++++++++++ 4 files changed, 84 insertions(+), 1 deletion(-) create mode 100755 t/t7424-submodule-mixed-ref-formats.sh diff --git a/Documentation/git-submodule.txt b/Documentation/git-submodule.txt index ca0347a37b..73ef8b9696 100644 --- a/Documentation/git-submodule.txt +++ b/Documentation/git-submodule.txt @@ -136,7 +136,7 @@ If you really want to remove a submodule from the repository and commit that use linkgit:git-rm[1] instead. See linkgit:gitsubmodules[7] for removal options. -update [--init] [--remote] [-N|--no-fetch] [--[no-]recommend-shallow] [-f|--force] [--checkout|--rebase|--merge] [--reference ] [--depth ] [--recursive] [--jobs ] [--[no-]single-branch] [--filter ] [--] [...]:: +update [--init] [--remote] [-N|--no-fetch] [--[no-]recommend-shallow] [-f|--force] [--checkout|--rebase|--merge] [--reference ] [--ref-format ] [--depth ] [--recursive] [--jobs ] [--[no-]single-branch] [--filter ] [--] [...]:: + -- Update the registered submodules to match what the superproject @@ -185,6 +185,9 @@ submodule with the `--init` option. If `--recursive` is specified, this command will recurse into the registered submodules, and update any nested submodules within. +If `--ref-format ` is specified, the ref storage format of newly +cloned submodules will be set accordingly. + If `--filter ` is specified, the given partial clone filter will be applied to the submodule. See linkgit:git-rev-list[1] for details on filter specifications. diff --git a/builtin/submodule--helper.c b/builtin/submodule--helper.c index f1218a1995..42a36bc2f7 100644 --- a/builtin/submodule--helper.c +++ b/builtin/submodule--helper.c @@ -1532,6 +1532,7 @@ struct module_clone_data { const char *url; const char *depth; struct list_objects_filter_options *filter_options; + enum ref_storage_format ref_storage_format; unsigned int quiet: 1; unsigned int progress: 1; unsigned int dissociate: 1; @@ -1540,6 +1541,7 @@ struct module_clone_data { }; #define MODULE_CLONE_DATA_INIT { \ .single_branch = -1, \ + .ref_storage_format = REF_STORAGE_FORMAT_UNKNOWN, \ } struct submodule_alternate_setup { @@ -1738,6 +1740,9 @@ static int clone_submodule(const struct module_clone_data *clone_data, strvec_pushl(&cp.args, "--reference", item->string, NULL); } + if (clone_data->ref_storage_format != REF_STORAGE_FORMAT_UNKNOWN) + strvec_pushf(&cp.args, "--ref-format=%s", + ref_storage_format_to_name(clone_data->ref_storage_format)); if (clone_data->dissociate) strvec_push(&cp.args, "--dissociate"); if (sm_gitdir && *sm_gitdir) @@ -1832,6 +1837,7 @@ static int module_clone(int argc, const char **argv, const char *prefix) struct string_list reference = STRING_LIST_INIT_NODUP; struct list_objects_filter_options filter_options = LIST_OBJECTS_FILTER_INIT; + const char *ref_storage_format = NULL; struct option module_clone_options[] = { OPT_STRING(0, "prefix", &clone_data.prefix, @@ -1849,6 +1855,8 @@ static int module_clone(int argc, const char **argv, const char *prefix) OPT_STRING_LIST(0, "reference", &reference, N_("repo"), N_("reference repository")), + OPT_STRING(0, "ref-format", &ref_storage_format, N_("format"), + N_("specify the reference format to use")), OPT_BOOL(0, "dissociate", &dissociate, N_("use --reference only while cloning")), OPT_STRING(0, "depth", &clone_data.depth, @@ -1875,6 +1883,11 @@ static int module_clone(int argc, const char **argv, const char *prefix) argc = parse_options(argc, argv, prefix, module_clone_options, git_submodule_helper_usage, 0); + if (ref_storage_format) { + clone_data.ref_storage_format = ref_storage_format_by_name(ref_storage_format); + if (clone_data.ref_storage_format == REF_STORAGE_FORMAT_UNKNOWN) + die(_("unknown ref storage format '%s'"), ref_storage_format); + } clone_data.dissociate = !!dissociate; clone_data.quiet = !!quiet; clone_data.progress = !!progress; @@ -1974,6 +1987,7 @@ struct update_data { struct submodule_update_strategy update_strategy; struct list_objects_filter_options *filter_options; struct module_list list; + enum ref_storage_format ref_storage_format; int depth; int max_jobs; int single_branch; @@ -1997,6 +2011,7 @@ struct update_data { #define UPDATE_DATA_INIT { \ .update_strategy = SUBMODULE_UPDATE_STRATEGY_INIT, \ .list = MODULE_LIST_INIT, \ + .ref_storage_format = REF_STORAGE_FORMAT_UNKNOWN, \ .recommend_shallow = -1, \ .references = STRING_LIST_INIT_DUP, \ .single_branch = -1, \ @@ -2132,6 +2147,9 @@ static int prepare_to_clone_next_submodule(const struct cache_entry *ce, expand_list_objects_filter_spec(suc->update_data->filter_options)); if (suc->update_data->require_init) strvec_push(&child->args, "--require-init"); + if (suc->update_data->ref_storage_format != REF_STORAGE_FORMAT_UNKNOWN) + strvec_pushf(&child->args, "--ref-format=%s", + ref_storage_format_to_name(suc->update_data->ref_storage_format)); strvec_pushl(&child->args, "--path", sub->path, NULL); strvec_pushl(&child->args, "--name", sub->name, NULL); strvec_pushl(&child->args, "--url", url, NULL); @@ -2562,6 +2580,9 @@ static void update_data_to_args(const struct update_data *update_data, for_each_string_list_item(item, &update_data->references) strvec_pushl(args, "--reference", item->string, NULL); } + if (update_data->ref_storage_format != REF_STORAGE_FORMAT_UNKNOWN) + strvec_pushf(args, "--ref-format=%s", + ref_storage_format_to_name(update_data->ref_storage_format)); if (update_data->filter_options && update_data->filter_options->choice) strvec_pushf(args, "--filter=%s", expand_list_objects_filter_spec( @@ -2737,6 +2758,7 @@ static int module_update(int argc, const char **argv, const char *prefix) struct update_data opt = UPDATE_DATA_INIT; struct list_objects_filter_options filter_options = LIST_OBJECTS_FILTER_INIT; + const char *ref_storage_format = NULL; int ret; struct option module_update_options[] = { OPT__SUPER_PREFIX(&opt.super_prefix), @@ -2760,6 +2782,8 @@ static int module_update(int argc, const char **argv, const char *prefix) SM_UPDATE_REBASE), OPT_STRING_LIST(0, "reference", &opt.references, N_("repo"), N_("reference repository")), + OPT_STRING(0, "ref-format", &ref_storage_format, N_("format"), + N_("specify the reference format to use")), OPT_BOOL(0, "dissociate", &opt.dissociate, N_("use --reference only while cloning")), OPT_INTEGER(0, "depth", &opt.depth, @@ -2803,6 +2827,12 @@ static int module_update(int argc, const char **argv, const char *prefix) module_update_options); } + if (ref_storage_format) { + opt.ref_storage_format = ref_storage_format_by_name(ref_storage_format); + if (opt.ref_storage_format == REF_STORAGE_FORMAT_UNKNOWN) + die(_("unknown ref storage format '%s'"), ref_storage_format); + } + opt.filter_options = &filter_options; opt.prefix = prefix; diff --git a/git-submodule.sh b/git-submodule.sh index fd588b1864..448d58b18b 100755 --- a/git-submodule.sh +++ b/git-submodule.sh @@ -290,6 +290,14 @@ cmd_update() -r|--rebase) rebase=1 ;; + --ref-format) + case "$2" in '') usage ;; esac + ref_format="--ref-format=$2" + shift + ;; + --ref-format=*) + ref_format="$1" + ;; --reference) case "$2" in '') usage ;; esac reference="--reference=$2" @@ -371,6 +379,7 @@ cmd_update() ${rebase:+--rebase} \ ${merge:+--merge} \ ${checkout:+--checkout} \ + ${ref_format:+"$ref_format"} \ ${reference:+"$reference"} \ ${dissociate:+"--dissociate"} \ ${depth:+"$depth"} \ diff --git a/t/t7424-submodule-mixed-ref-formats.sh b/t/t7424-submodule-mixed-ref-formats.sh new file mode 100755 index 0000000000..de84007554 --- /dev/null +++ b/t/t7424-submodule-mixed-ref-formats.sh @@ -0,0 +1,41 @@ +#!/bin/sh + +test_description='submodules handle mixed ref storage formats' + +. ./test-lib.sh + +test_ref_format () { + echo "$2" >expect && + git -C "$1" rev-parse --show-ref-format >actual && + test_cmp expect actual +} + +for OTHER_FORMAT in files reftable +do + if test "$OTHER_FORMAT" = "$GIT_DEFAULT_REF_FORMAT" + then + continue + fi + +test_expect_success 'setup' ' + git config set --global protocol.file.allow always +' + +test_expect_success 'clone submodules with different ref storage format' ' + test_when_finished "rm -rf submodule upstream downstream" && + + git init submodule && + test_commit -C submodule submodule-initial && + git init upstream && + git -C upstream submodule add "file://$(pwd)/submodule" && + git -C upstream commit -m "upstream submodule" && + + git clone --no-recurse-submodules "file://$(pwd)/upstream" downstream && + test_ref_format downstream "$GIT_DEFAULT_REF_FORMAT" && + git -C downstream submodule update --init --ref-format=$OTHER_FORMAT && + test_ref_format downstream/submodule "$OTHER_FORMAT" +' + +done + +test_done From patchwork Thu Aug 8 07:35:37 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Patrick Steinhardt X-Patchwork-Id: 13757058 Received: from fhigh8-smtp.messagingengine.com (fhigh8-smtp.messagingengine.com [103.168.172.159]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 9F138185628 for ; Thu, 8 Aug 2024 07:35:41 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=103.168.172.159 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1723102543; cv=none; b=ftL9C+J7OB0mViSmX4sQiaby12MNY9ZljGjxJ9aYiQUYbGQwS2h/6afcxq7ZxC6tAtVLKXV15qKEFBCENhWqE/HKRhednGxge4rNUYyvSNGzrYbYCascgSzq/n6kkhE0lKbq7Vf2ODCo34RztJ1xFB0Nu+uvTB4LSAGYcNO0jBA= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1723102543; c=relaxed/simple; bh=4J+b5fHqZ4mHFNnGQj2LifjBllveWGRcYpi3LzqbusI=; h=Date:From:To:Cc:Subject:Message-ID:References:MIME-Version: Content-Type:Content-Disposition:In-Reply-To; b=OMR4W26eWgy9QiF9f6jzz+H1l1FexAp2Z2kGN+2MjxwvGS949TEqY82a8vhUaXjdZCYLKYnXo6m+blv5cjT3kFU48iU/xAgo7ZDu5lHexducceotlyTne6oh3lwSYojj5mGx5bhiqiDRYsRhQLLjAZHi0WfiGpbLU3YdcOaA7TU= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=pks.im; spf=pass smtp.mailfrom=pks.im; dkim=pass (2048-bit key) header.d=pks.im header.i=@pks.im header.b=oYM/CYSn; dkim=pass (2048-bit key) header.d=messagingengine.com header.i=@messagingengine.com header.b=WqQqEx5v; arc=none smtp.client-ip=103.168.172.159 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=pks.im Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=pks.im Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=pks.im header.i=@pks.im header.b="oYM/CYSn"; dkim=pass (2048-bit key) header.d=messagingengine.com header.i=@messagingengine.com header.b="WqQqEx5v" Received: from compute4.internal (compute4.nyi.internal [10.202.2.44]) by mailfhigh.nyi.internal (Postfix) with ESMTP id CC8EE1151B7A; Thu, 8 Aug 2024 03:35:40 -0400 (EDT) Received: from mailfrontend1 ([10.202.2.162]) by compute4.internal (MEProxy); Thu, 08 Aug 2024 03:35:40 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=pks.im; h=cc:cc :content-type:content-type:date:date:from:from:in-reply-to :in-reply-to:message-id:mime-version:references:reply-to:subject :subject:to:to; s=fm3; t=1723102540; x=1723188940; bh=nmmneoxzFA fI1Disqv2spSVr+fORkAoQP0PHid2f8Zg=; b=oYM/CYSnXmo6iqnZD0YwCEEo2o WT6lVrZowNF27ZQIOPE23FfG0+OXPYDwrb6k+6THZA0mpKHIm4DiCMHZBAQbcX2F G9pCb3898GS5LJT+rG+D2nOsjzVl+YCWVPJy7CHgBFePSJNYkpJVXp0CPhYFk9IP Dv4F5u4NKdvnnKwx8vZgXJOcboJ8FoTEjXPqWtBID/L2ig+livnWC1Y7yJsYROj/ RRnJ9Ou/1X0zQB2Bdtu2hCtX1l5TDNGcqFoIya92BKVSY8bqPLPVv3CL0qX1nZ7i c7SkzT12sEGBLASxf/iXfg80Nm2uYo0ZR7+FVPZHEFqSgEH2evKXq4BsDmKQ== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:cc:content-type:content-type:date:date :feedback-id:feedback-id:from:from:in-reply-to:in-reply-to :message-id:mime-version:references:reply-to:subject:subject:to :to:x-me-proxy:x-me-proxy:x-me-sender:x-me-sender:x-sasl-enc; s= fm3; t=1723102540; x=1723188940; bh=nmmneoxzFAfI1Disqv2spSVr+fOR kAoQP0PHid2f8Zg=; b=WqQqEx5vKnl0vyZ5RdNYO1KrdEdxbGm9mAk6iR368gGg pFo6T+q9HPIcqvMTugokiu87NeeeFR7qCOn87DOXwEtuRAlzJz56fWfPfT5QEgMG zkZCIzF8GQeMNocaWYptTchxbQR/FxRwXlKfLEEwtHx2NxLpdZNcpbLFPxjg/Qon RyyVGnRB5Ox5xc3Is0o3IvsFN1ODMMg4m6BQ98eAg1Z+k5dp4Woj7dqJt9zpKHyz WJ0XFpkZOuyACH+r7WJRQ4Eh1u8Ha8wqF8xv/XzP45IEiguFm1/6zsrdMIHlny1R NFZzjDGQAhHMjRKky39ZQO2Q1ZCiHA9XG5/i+dF32Q== X-ME-Sender: X-ME-Received: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgeeftddrledugdduvdduucetufdoteggodetrfdotf fvucfrrhhofhhilhgvmecuhfgrshhtofgrihhlpdggtfgfnhhsuhgsshgtrhhisggvpdfu rfetoffkrfgpnffqhgenuceurghilhhouhhtmecufedttdenucesvcftvggtihhpihgvnh htshculddquddttddmnecujfgurhepfffhvfevuffkfhggtggujgesghdtreertddtvden ucfhrhhomheprfgrthhrihgtkhcuufhtvghinhhhrghrughtuceophhssehpkhhsrdhimh eqnecuggftrfgrthhtvghrnhepueektdevtdffveeljeetgfehheeigeekleduvdeffeeg hefgledttdehjeelffetnecuvehluhhsthgvrhfuihiivgeptdenucfrrghrrghmpehmrg hilhhfrhhomhepphhssehpkhhsrdhimhdpnhgspghrtghpthhtohepgedpmhhouggvpehs mhhtphhouhhtpdhrtghpthhtohepghhithhsthgvrhesphhosghogidrtghomhdprhgtph htthhopehgihhtsehvghgvrhdrkhgvrhhnvghlrdhorhhgpdhrtghpthhtohepjhholhgr nhgusehgmhgrihhlrdgtohhmpdhrtghpthhtohepshhunhhshhhinhgvsehsuhhnshhhih hnvggtohdrtghomh X-ME-Proxy: Feedback-ID: i197146af:Fastmail Received: by mail.messagingengine.com (Postfix) with ESMTPA; Thu, 8 Aug 2024 03:35:39 -0400 (EDT) Received: by vm-mail (OpenSMTPD) with ESMTPSA id 6c9ddd0e (TLSv1.3:TLS_AES_256_GCM_SHA384:256:NO); Thu, 8 Aug 2024 07:35:33 +0000 (UTC) Date: Thu, 8 Aug 2024 09:35:37 +0200 From: Patrick Steinhardt To: git@vger.kernel.org Cc: Eric Sunshine , Jeppe =?utf-8?b?w5hsYW5k?= , Junio C Hamano Subject: [PATCH v2 3/8] builtin/clone: propagate ref storage format to submodules Message-ID: References: Precedence: bulk X-Mailing-List: git@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: When recursively cloning a repository with a non-default ref storage format, e.g. by passing the `--ref-format=` option, then only the top-level repository will end up will end up using that ref storage format. All recursively cloned submodules will instead use the default format. While mixed-format constellations are expected to work alright, the outcome still is somewhat surprising as we have essentially ignored the user's request. Fix this by propagating the requested ref format to cloned submodules. Signed-off-by: Patrick Steinhardt --- builtin/clone.c | 10 ++++++++-- t/t7424-submodule-mixed-ref-formats.sh | 23 +++++++++++++++++++++++ 2 files changed, 31 insertions(+), 2 deletions(-) diff --git a/builtin/clone.c b/builtin/clone.c index af6017d41a..75b15b5773 100644 --- a/builtin/clone.c +++ b/builtin/clone.c @@ -729,7 +729,8 @@ static int git_sparse_checkout_init(const char *repo) return result; } -static int checkout(int submodule_progress, int filter_submodules) +static int checkout(int submodule_progress, int filter_submodules, + enum ref_storage_format ref_storage_format) { struct object_id oid; char *head; @@ -813,6 +814,10 @@ static int checkout(int submodule_progress, int filter_submodules) strvec_push(&cmd.args, "--no-fetch"); } + if (ref_storage_format != REF_STORAGE_FORMAT_UNKNOWN) + strvec_pushf(&cmd.args, "--ref-format=%s", + ref_storage_format_to_name(ref_storage_format)); + if (filter_submodules && filter_options.choice) strvec_pushf(&cmd.args, "--filter=%s", expand_list_objects_filter_spec(&filter_options)); @@ -1536,7 +1541,8 @@ int cmd_clone(int argc, const char **argv, const char *prefix) return 1; junk_mode = JUNK_LEAVE_REPO; - err = checkout(submodule_progress, filter_submodules); + err = checkout(submodule_progress, filter_submodules, + ref_storage_format); free(remote_name); strbuf_release(&reflog_msg); diff --git a/t/t7424-submodule-mixed-ref-formats.sh b/t/t7424-submodule-mixed-ref-formats.sh index de84007554..4e4991d04c 100755 --- a/t/t7424-submodule-mixed-ref-formats.sh +++ b/t/t7424-submodule-mixed-ref-formats.sh @@ -21,6 +21,29 @@ test_expect_success 'setup' ' git config set --global protocol.file.allow always ' +test_expect_success 'recursive clone propagates ref storage format' ' + test_when_finished "rm -rf submodule upstream downstream" && + + git init submodule && + test_commit -C submodule submodule-initial && + git init upstream && + git -C upstream submodule add "file://$(pwd)/submodule" && + git -C upstream commit -am "add submodule" && + + # The upstream repository and its submodule should be using the default + # ref format. + test_ref_format upstream "$GIT_DEFAULT_REF_FORMAT" && + test_ref_format upstream/submodule "$GIT_DEFAULT_REF_FORMAT" && + + # The cloned repositories should use the other ref format that we have + # specified via `--ref-format`. The option should propagate to cloned + # submodules. + git clone --ref-format=$OTHER_FORMAT --recurse-submodules \ + upstream downstream && + test_ref_format downstream "$OTHER_FORMAT" && + test_ref_format downstream/submodule "$OTHER_FORMAT" +' + test_expect_success 'clone submodules with different ref storage format' ' test_when_finished "rm -rf submodule upstream downstream" && From patchwork Thu Aug 8 07:35:42 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Patrick Steinhardt X-Patchwork-Id: 13757059 Received: from fout3-smtp.messagingengine.com (fout3-smtp.messagingengine.com [103.168.172.146]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id C2028188CB8 for ; Thu, 8 Aug 2024 07:35:46 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=103.168.172.146 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1723102548; cv=none; b=Yg27qxjHEuiX4HbwpKVERO7ZPHw4xDzxSxlnGb/JlHn8/kVYTc2JYPv2TbaLt6gRPCtoYvUnT8NtlvGsEc3ZUjnvzwIMg+Gk78Xk1rx6UOf/l91SsqXA68LSjkbuc+1R6k9gqkl41ee21XDRmjOHAW4m2HMIflMs/OtzfGcNAek= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1723102548; c=relaxed/simple; bh=rC3LhTJTmHX8qHAt3yx7I6mADhSYGOKRR71UscxdRC0=; h=Date:From:To:Cc:Subject:Message-ID:References:MIME-Version: Content-Type:Content-Disposition:In-Reply-To; b=sktQhp2Zbsc/dJdH8Tk+ScbscbUrpTBqiObqUyzuo+PrAvwtlK58u/aA1cTTdKGStXc+yIitRY/RLtAe4TG1W++3mwJu6MHi4Vy3Q3K6XBr1GhTKTvInBvtjzwhbK1LI+j7XQjRHlZTa52nB1iZ11HThw6dLfaMGwYF00ok7Gmo= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=pks.im; spf=pass smtp.mailfrom=pks.im; dkim=pass (2048-bit key) header.d=pks.im header.i=@pks.im header.b=JCSKnUUR; dkim=pass (2048-bit key) header.d=messagingengine.com header.i=@messagingengine.com header.b=n5tCBR8F; arc=none smtp.client-ip=103.168.172.146 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=pks.im Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=pks.im Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=pks.im header.i=@pks.im header.b="JCSKnUUR"; dkim=pass (2048-bit key) header.d=messagingengine.com header.i=@messagingengine.com header.b="n5tCBR8F" Received: from compute7.internal (compute7.nyi.internal [10.202.2.48]) by mailfout.nyi.internal (Postfix) with ESMTP id 12C9B138FD86; Thu, 8 Aug 2024 03:35:46 -0400 (EDT) Received: from mailfrontend1 ([10.202.2.162]) by compute7.internal (MEProxy); Thu, 08 Aug 2024 03:35:46 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=pks.im; h=cc:cc :content-type:content-type:date:date:from:from:in-reply-to :in-reply-to:message-id:mime-version:references:reply-to:subject :subject:to:to; s=fm3; t=1723102546; x=1723188946; bh=R7v3Vm+YWm xNLP0sku5H6xKucmhttEmV4oF6cpxssfI=; b=JCSKnUURcnM2nYdhoYc8YYtJSJ rTaXpWSYpJ/jPLExnIwWDtP6f/o8RAxBPGexygwjTR2nnS+Fh5ZjMwsBq/atox8s aYp5wvQrGqvT/WlnQTOvOU+U42h/bDrRIkPMYird4uYa+oWeaDrPW2i+Hl5BKzKr frIy0sZNAXIV4QIhXN7XDpaiqf6ghKYwkOL3yr/0m1abe3XVAXUAoImYUQzP4BW7 iVnu6T+5d7U+/WuZt7DiXvA09LQ50Gu2O3yMX8QNdubJnF9YloDl/2NtkgTT14AY I5E+BlyLvxL5+zm34JHlf8ymQOW7L93jlmku7f0TXHNJJJ3hcBjy3gPnOPjg== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:cc:content-type:content-type:date:date :feedback-id:feedback-id:from:from:in-reply-to:in-reply-to :message-id:mime-version:references:reply-to:subject:subject:to :to:x-me-proxy:x-me-proxy:x-me-sender:x-me-sender:x-sasl-enc; s= fm3; t=1723102546; x=1723188946; bh=R7v3Vm+YWmxNLP0sku5H6xKucmht tEmV4oF6cpxssfI=; b=n5tCBR8FEVihQwPtMyR2E7UJ2UdmEfvEBxFFBu4uFzB5 S/9q5oeSh0Pr0x6hTcI60h//cKzocXsG+pU3a7lsvis0B+diKsh2OCzv0a/TXAwu asAqrttutPvV1YRl8pBCj6Wfm37ozJ7LO5pwkHxgK5T+2Ajxy/CLu8MJ0Y7aB5G6 3vZPs9DqDbKb+KnMtdq9+Y7d8sCsBnJ/vPk6BvSpMRukzqmM3eIlGOui5Z+iZ6gB hGiIvRjY964MmrJsjzbgb0G5dDPUiIcuxFgpUA6jp00B64graja2ORmcKnWjPccn lfqf2wo2ZGdBZFkFKzefsaMV9xIDlS/Epcm+gkMHFg== X-ME-Sender: X-ME-Received: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgeeftddrledugdduvddvucetufdoteggodetrfdotf fvucfrrhhofhhilhgvmecuhfgrshhtofgrihhlpdggtfgfnhhsuhgsshgtrhhisggvpdfu rfetoffkrfgpnffqhgenuceurghilhhouhhtmecufedttdenucesvcftvggtihhpihgvnh htshculddquddttddmnecujfgurhepfffhvfevuffkfhggtggujgesghdtreertddtjeen ucfhrhhomheprfgrthhrihgtkhcuufhtvghinhhhrghrughtuceophhssehpkhhsrdhimh eqnecuggftrfgrthhtvghrnhepteeuvefhhfdufedvgeeiueeileegtdfhgeeftdeuveej jedtgfejhedujeeutddunecuvehluhhsthgvrhfuihiivgeptdenucfrrghrrghmpehmrg hilhhfrhhomhepphhssehpkhhsrdhimhdpnhgspghrtghpthhtohepgedpmhhouggvpehs mhhtphhouhhtpdhrtghpthhtohepjhholhgrnhgusehgmhgrihhlrdgtohhmpdhrtghpth htohepghhithesvhhgvghrrdhkvghrnhgvlhdrohhrghdprhgtphhtthhopehsuhhnshhh ihhnvgesshhunhhshhhinhgvtghordgtohhmpdhrtghpthhtohepghhithhsthgvrhesph hosghogidrtghomh X-ME-Proxy: Feedback-ID: i197146af:Fastmail Received: by mail.messagingengine.com (Postfix) with ESMTPA; Thu, 8 Aug 2024 03:35:44 -0400 (EDT) Received: by vm-mail (OpenSMTPD) with ESMTPSA id a21b1fa0 (TLSv1.3:TLS_AES_256_GCM_SHA384:256:NO); Thu, 8 Aug 2024 07:35:37 +0000 (UTC) Date: Thu, 8 Aug 2024 09:35:42 +0200 From: Patrick Steinhardt To: git@vger.kernel.org Cc: Eric Sunshine , Jeppe =?utf-8?b?w5hsYW5k?= , Junio C Hamano Subject: [PATCH v2 4/8] refs: fix ref storage format for submodule ref stores Message-ID: References: Precedence: bulk X-Mailing-List: git@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: When opening a submodule ref storage we accidentally use the ref storage format of the owning repository, not of the submodule repository. As submodules may have a different storage format than their parent repo this can lead to bugs when trying to access the submodule ref storage from the parent repository. One such bug was reported when performing a recursive pull with mixed ref stores, which fails with: $ git pull --recursive fatal: Unable to find current revision in submodule path 'path/to/sub' The same issue occurs when adding a repository contained in the working tree with a different ref storage format via `git submodule add`. Fix the bug by using the submodule repository's ref storage format instead and add some tests. Note that the test for `git submodule status` was included as a precaution, only. The command worked alright even without the bugfix. Reported-by: Jeppe Øland Signed-off-by: Patrick Steinhardt --- refs.c | 2 +- t/t7424-submodule-mixed-ref-formats.sh | 70 +++++++++++++++++++++++++- 2 files changed, 70 insertions(+), 2 deletions(-) diff --git a/refs.c b/refs.c index 915aeb4d1d..e4b1f4f8b1 100644 --- a/refs.c +++ b/refs.c @@ -2011,7 +2011,7 @@ struct ref_store *repo_get_submodule_ref_store(struct repository *repo, free(subrepo); goto done; } - refs = ref_store_init(subrepo, the_repository->ref_storage_format, + refs = ref_store_init(subrepo, subrepo->ref_storage_format, submodule_sb.buf, REF_STORE_READ | REF_STORE_ODB); register_ref_store_map(&repo->submodule_ref_stores, "submodule", diff --git a/t/t7424-submodule-mixed-ref-formats.sh b/t/t7424-submodule-mixed-ref-formats.sh index 4e4991d04c..d4e184970a 100755 --- a/t/t7424-submodule-mixed-ref-formats.sh +++ b/t/t7424-submodule-mixed-ref-formats.sh @@ -18,7 +18,23 @@ do fi test_expect_success 'setup' ' - git config set --global protocol.file.allow always + git config set --global protocol.file.allow always && + # Some tests migrate the ref storage format, which does not work with + # reflogs at the time of writing these tests. + git config set --global core.logAllRefUpdates false +' + +test_expect_success 'add existing repository with different ref storage format' ' + test_when_finished "rm -rf parent" && + + git init parent && + ( + cd parent && + test_commit parent && + git init --ref-format=$OTHER_FORMAT submodule && + test_commit -C submodule submodule && + git submodule add ./submodule + ) ' test_expect_success 'recursive clone propagates ref storage format' ' @@ -59,6 +75,58 @@ test_expect_success 'clone submodules with different ref storage format' ' test_ref_format downstream/submodule "$OTHER_FORMAT" ' +test_expect_success 'status with mixed submodule ref storages' ' + test_when_finished "rm -rf submodule main" && + + git init submodule && + test_commit -C submodule submodule-initial && + git init main && + git -C main submodule add "file://$(pwd)/submodule" && + git -C main commit -m "add submodule" && + git -C main/submodule refs migrate --ref-format=$OTHER_FORMAT && + + # The main repository should use the default ref format now, whereas + # the submodule should use the other format. + test_ref_format main "$GIT_DEFAULT_REF_FORMAT" && + test_ref_format main/submodule "$OTHER_FORMAT" && + + cat >expect <<-EOF && + $(git -C main/submodule rev-parse HEAD) submodule (submodule-initial) + EOF + git -C main submodule status >actual && + test_cmp expect actual +' + +test_expect_success 'recursive pull with mixed formats' ' + test_when_finished "rm -rf submodule upstream downstream" && + + # Set up the initial structure with an upstream repository that has a + # submodule, as well as a downstream clone of the upstream repository. + git init submodule && + test_commit -C submodule submodule-initial && + git init upstream && + git -C upstream submodule add "file://$(pwd)/submodule" && + git -C upstream commit -m "upstream submodule" && + + # Clone the upstream repository such that the main repo and its + # submodules have different formats. + git clone --no-recurse-submodules "file://$(pwd)/upstream" downstream && + git -C downstream submodule update --init --ref-format=$OTHER_FORMAT && + test_ref_format downstream "$GIT_DEFAULT_REF_FORMAT" && + test_ref_format downstream/submodule "$OTHER_FORMAT" && + + # Update the upstream submodule as well as the owning repository such + # that we can do a recursive pull. + test_commit -C submodule submodule-update && + git -C upstream/submodule pull && + git -C upstream commit -am "update the submodule" && + + git -C downstream pull --recurse-submodules && + git -C upstream/submodule rev-parse HEAD >expect && + git -C downstream/submodule rev-parse HEAD >actual && + test_cmp expect actual +' + done test_done From patchwork Thu Aug 8 07:35:47 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Patrick Steinhardt X-Patchwork-Id: 13757060 Received: from fout3-smtp.messagingengine.com (fout3-smtp.messagingengine.com [103.168.172.146]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 2EE9C144306 for ; Thu, 8 Aug 2024 07:35:51 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=103.168.172.146 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1723102552; cv=none; b=j4YuFGub1AcquN5Rkd00py5kIo3KL9q/POCTV3dPujkbwbjDQCYw27AisU3bBf0Etxzt+Alee+AqDbOtXOGt+JGooK9eRWgtu6MNiCTdZImAigWfmtNYxtuPrHh+giapYJ3oLl0S8uSspPB481CHDAMtJW+nzdWwun7aXQps3iE= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1723102552; c=relaxed/simple; bh=NK2xje7aGwQ8SXUPlRMQkjqvafEyLVMD8oOejojieZU=; h=Date:From:To:Cc:Subject:Message-ID:References:MIME-Version: Content-Type:Content-Disposition:In-Reply-To; b=Qm49PebkASymYGqdHTIXB7hTof5mCTSCpBNMDhep6rfbqrkxR6PCTgx/MzpTnP9VzpTGd5gChIxhe92xWkgBSxu+gYdKhMA1aOkqoHXK+5S34VLovCiuT+GiPfEv9aYZvEYV3Y4/z8uAx0m0nkHwEbMv7MudTBOme/X8QCeZhDY= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=pks.im; spf=pass smtp.mailfrom=pks.im; dkim=pass (2048-bit key) header.d=pks.im header.i=@pks.im header.b=qQAYxp2u; dkim=pass (2048-bit key) header.d=messagingengine.com header.i=@messagingengine.com header.b=W7YPH+IU; arc=none smtp.client-ip=103.168.172.146 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=pks.im Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=pks.im Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=pks.im header.i=@pks.im header.b="qQAYxp2u"; dkim=pass (2048-bit key) header.d=messagingengine.com header.i=@messagingengine.com header.b="W7YPH+IU" Received: from compute3.internal (compute3.nyi.internal [10.202.2.43]) by mailfout.nyi.internal (Postfix) with ESMTP id 84881138FE23; Thu, 8 Aug 2024 03:35:50 -0400 (EDT) Received: from mailfrontend1 ([10.202.2.162]) by compute3.internal (MEProxy); Thu, 08 Aug 2024 03:35:50 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=pks.im; h=cc:cc :content-type:content-type:date:date:from:from:in-reply-to :in-reply-to:message-id:mime-version:references:reply-to:subject :subject:to:to; s=fm3; t=1723102550; x=1723188950; bh=KxlkCDQu/F dkwNJEw7LaXbmWERXmzorW8xyIiI26aWM=; b=qQAYxp2uJjSv3vy08llqpvUMQn 2ABwcgw2T/uSUXDaypDUpJy1OzBI3j0bAZ/Ga6f4xEEXpKZWFa0bEmL1rvfqOZmh s++dVdyHofAERjqZ+qCbR0GBqbu4pfXt30HaRFmnkf0Uwseu3zdrYYumKm1AFdFD tEJcd1qFvX5HV7Wcteiu9pfDlo4xA57QIlBbjhZswq10DSEIXiHPzROYLzw48SNK aMFLPGNm7moX5PUekw/Gbhc6KlTYlkW016hvaTRJ7STnDfPdhi5ANohcAcUJNoJz 4Qa6rqZg4p99FNP8Uv2lZBi5npqH+AJzQO+k6lNSyDwtcFxG/7tZq+5uo/+g== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:cc:content-type:content-type:date:date :feedback-id:feedback-id:from:from:in-reply-to:in-reply-to :message-id:mime-version:references:reply-to:subject:subject:to :to:x-me-proxy:x-me-proxy:x-me-sender:x-me-sender:x-sasl-enc; s= fm3; t=1723102550; x=1723188950; bh=KxlkCDQu/FdkwNJEw7LaXbmWERXm zorW8xyIiI26aWM=; b=W7YPH+IUOzR9XCGdZRJjkBQrqbqvG6ZbErXAD4p4JCXv yds5NN4shFKqoHm91p9owsaPJmfnr423zY/HCtva0TdDlVygJnXeNsecd/mFpZ08 KSRvbSAaCYi83XynBh4oPtWOk7o+T4FZX7SSvRLhhxBIF84g7TTanPqCN5kKyDJ0 0yi9gBWPpUQqTLF1VxnfFdya7EPhWxqDrx8vrgneORqCaAgw5FxmSVL4xABlwtWs g1+yzEi+jv0tvo6qR2XvSzkWPiW92yxRoJwZvlaPBmxMAiAhruBC13z2GhZlcDm6 mRlmmrZyGFL2jH8sFyQj2wE4jr7GxvV71/qbAhH4ZA== X-ME-Sender: X-ME-Received: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgeeftddrledugdduvddvucetufdoteggodetrfdotf fvucfrrhhofhhilhgvmecuhfgrshhtofgrihhlpdggtfgfnhhsuhgsshgtrhhisggvpdfu rfetoffkrfgpnffqhgenuceurghilhhouhhtmecufedttdenucesvcftvggtihhpihgvnh htshculddquddttddmnecujfgurhepfffhvfevuffkfhggtggujgesghdtreertddtvden ucfhrhhomheprfgrthhrihgtkhcuufhtvghinhhhrghrughtuceophhssehpkhhsrdhimh eqnecuggftrfgrthhtvghrnhepueektdevtdffveeljeetgfehheeigeekleduvdeffeeg hefgledttdehjeelffetnecuvehluhhsthgvrhfuihiivgepvdenucfrrghrrghmpehmrg hilhhfrhhomhepphhssehpkhhsrdhimhdpnhgspghrtghpthhtohepgedpmhhouggvpehs mhhtphhouhhtpdhrtghpthhtohepjhholhgrnhgusehgmhgrihhlrdgtohhmpdhrtghpth htohepghhithhsthgvrhesphhosghogidrtghomhdprhgtphhtthhopehgihhtsehvghgv rhdrkhgvrhhnvghlrdhorhhgpdhrtghpthhtohepshhunhhshhhinhgvsehsuhhnshhhih hnvggtohdrtghomh X-ME-Proxy: Feedback-ID: i197146af:Fastmail Received: by mail.messagingengine.com (Postfix) with ESMTPA; Thu, 8 Aug 2024 03:35:49 -0400 (EDT) Received: by vm-mail (OpenSMTPD) with ESMTPSA id a4ed108d (TLSv1.3:TLS_AES_256_GCM_SHA384:256:NO); Thu, 8 Aug 2024 07:35:42 +0000 (UTC) Date: Thu, 8 Aug 2024 09:35:47 +0200 From: Patrick Steinhardt To: git@vger.kernel.org Cc: Eric Sunshine , Jeppe =?utf-8?b?w5hsYW5k?= , Junio C Hamano Subject: [PATCH v2 5/8] builtin/submodule: allow "add" to use different ref storage format Message-ID: <4ce17e44a16335adf9423a227047d3810608aae4.1723102259.git.ps@pks.im> References: Precedence: bulk X-Mailing-List: git@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: Same as with "clone", users may want to add a submodule to a repository with a non-default ref storage format. Wire up a new `--ref-format=` option that works the same as for `git submodule clone`. Signed-off-by: Patrick Steinhardt --- Documentation/git-submodule.txt | 5 ++++- builtin/submodule--helper.c | 16 +++++++++++++++- git-submodule.sh | 9 +++++++++ t/t7424-submodule-mixed-ref-formats.sh | 11 +++++++++++ 4 files changed, 39 insertions(+), 2 deletions(-) diff --git a/Documentation/git-submodule.txt b/Documentation/git-submodule.txt index 73ef8b9696..87d8e0f0c5 100644 --- a/Documentation/git-submodule.txt +++ b/Documentation/git-submodule.txt @@ -34,7 +34,7 @@ COMMANDS With no arguments, shows the status of existing submodules. Several subcommands are available to perform operations on the submodules. -add [-b ] [-f|--force] [--name ] [--reference ] [--depth ] [--] []:: +add [-b ] [-f|--force] [--name ] [--reference ] [--ref-format ] [--depth ] [--] []:: Add the given repository as a submodule at the given path to the changeset to be committed next to the current project: the current project is termed the "superproject". @@ -71,6 +71,9 @@ submodule repositories will be kept together in the same relative location, and only the superproject's URL needs to be provided. git-submodule will correctly locate the submodule using the relative URL in `.gitmodules`. ++ +If `--ref-format ` is specified, the ref storage format of newly +cloned submodules will be set accordingly. status [--cached] [--recursive] [--] [...]:: Show the status of the submodules. This will print the SHA-1 of the diff --git a/builtin/submodule--helper.c b/builtin/submodule--helper.c index 42a36bc2f7..48f4577b53 100644 --- a/builtin/submodule--helper.c +++ b/builtin/submodule--helper.c @@ -3128,13 +3128,17 @@ struct add_data { const char *sm_name; const char *repo; const char *realrepo; + enum ref_storage_format ref_storage_format; int depth; unsigned int force: 1; unsigned int quiet: 1; unsigned int progress: 1; unsigned int dissociate: 1; }; -#define ADD_DATA_INIT { .depth = -1 } +#define ADD_DATA_INIT { \ + .depth = -1, \ + .ref_storage_format = REF_STORAGE_FORMAT_UNKNOWN, \ +} static void append_fetch_remotes(struct strbuf *msg, const char *git_dir_path) { @@ -3228,6 +3232,7 @@ static int add_submodule(const struct add_data *add_data) string_list_append(&reference, p)->util = p; } + clone_data.ref_storage_format = add_data->ref_storage_format; clone_data.dissociate = add_data->dissociate; if (add_data->depth >= 0) clone_data.depth = xstrfmt("%d", add_data->depth); @@ -3392,6 +3397,7 @@ static int module_add(int argc, const char **argv, const char *prefix) { int force = 0, quiet = 0, progress = 0, dissociate = 0; struct add_data add_data = ADD_DATA_INIT; + const char *ref_storage_format = NULL; char *to_free = NULL; struct option options[] = { OPT_STRING('b', "branch", &add_data.branch, N_("branch"), @@ -3402,6 +3408,8 @@ static int module_add(int argc, const char **argv, const char *prefix) OPT_BOOL(0, "progress", &progress, N_("force cloning progress")), OPT_STRING(0, "reference", &add_data.reference_path, N_("repository"), N_("reference repository")), + OPT_STRING(0, "ref-format", &ref_storage_format, N_("format"), + N_("specify the reference format to use")), OPT_BOOL(0, "dissociate", &dissociate, N_("borrow the objects from reference repositories")), OPT_STRING(0, "name", &add_data.sm_name, N_("name"), N_("sets the submodule's name to the given string " @@ -3428,6 +3436,12 @@ static int module_add(int argc, const char **argv, const char *prefix) if (argc == 0 || argc > 2) usage_with_options(usage, options); + if (ref_storage_format) { + add_data.ref_storage_format = ref_storage_format_by_name(ref_storage_format); + if (add_data.ref_storage_format == REF_STORAGE_FORMAT_UNKNOWN) + die(_("unknown ref storage format '%s'"), ref_storage_format); + } + add_data.repo = argv[0]; if (argc == 1) add_data.sm_path = git_url_basename(add_data.repo, 0, 0); diff --git a/git-submodule.sh b/git-submodule.sh index 448d58b18b..03c5a220a2 100755 --- a/git-submodule.sh +++ b/git-submodule.sh @@ -94,6 +94,14 @@ cmd_add() --reference=*) reference_path="${1#--reference=}" ;; + --ref-format) + case "$2" in '') usage ;; esac + ref_format="--ref-format=$2" + shift + ;; + --ref-format=*) + ref_format="$1" + ;; --dissociate) dissociate=1 ;; @@ -135,6 +143,7 @@ cmd_add() ${progress:+"--progress"} \ ${branch:+--branch "$branch"} \ ${reference_path:+--reference "$reference_path"} \ + ${ref_format:+"$ref_format"} \ ${dissociate:+--dissociate} \ ${custom_name:+--name "$custom_name"} \ ${depth:+"$depth"} \ diff --git a/t/t7424-submodule-mixed-ref-formats.sh b/t/t7424-submodule-mixed-ref-formats.sh index d4e184970a..559713b607 100755 --- a/t/t7424-submodule-mixed-ref-formats.sh +++ b/t/t7424-submodule-mixed-ref-formats.sh @@ -37,6 +37,17 @@ test_expect_success 'add existing repository with different ref storage format' ) ' +test_expect_success 'add submodules with different ref storage format' ' + test_when_finished "rm -rf submodule upstream" && + + git init submodule && + test_commit -C submodule submodule-initial && + git init upstream && + test_ref_format upstream "$GIT_DEFAULT_REF_FORMAT" && + git -C upstream submodule add --ref-format="$OTHER_FORMAT" "file://$(pwd)/submodule" && + test_ref_format upstream/submodule "$OTHER_FORMAT" +' + test_expect_success 'recursive clone propagates ref storage format' ' test_when_finished "rm -rf submodule upstream downstream" && From patchwork Thu Aug 8 07:35:51 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Patrick Steinhardt X-Patchwork-Id: 13757061 Received: from fhigh8-smtp.messagingengine.com (fhigh8-smtp.messagingengine.com [103.168.172.159]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 787EE188CD7 for ; Thu, 8 Aug 2024 07:35:55 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=103.168.172.159 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1723102557; cv=none; b=Nrs0vqR4FfREgIGbsCxJChIsgeBK/gU41f/XtC75EEYJymeogFNnOATEwU5dxz45B1cnp7s9iYCpz0TjUlP4SmshcX5cRxlkEhFu+X7oHFjliQBpMy0FZZengooztgmxFUHHtYwkzmt5ypFFfwJ+Iq0orI97TyL7CvMfnK6WYRc= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1723102557; c=relaxed/simple; bh=/CxbifL3mtx9yVwBeniXWGaFTP7QDrvCeer9Y52Nsc4=; h=Date:From:To:Cc:Subject:Message-ID:References:MIME-Version: Content-Type:Content-Disposition:In-Reply-To; b=KxFgGjhoMzADsVhqIDUJ4bk636pKYxGZDxgfwHN/L8uiWWbUs8FgWD4RsCSaiyk+wlFZCpwHxPm9B6u+xP7RLHBX6W1MXALVLHCsVceYdRuemAnJ6vaZvyNeTXbFcZFfpNsloKNfvhZZhVV/elqCMa+OxqbzBgFQtz4gcEscDWo= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=pks.im; spf=pass smtp.mailfrom=pks.im; dkim=pass (2048-bit key) header.d=pks.im header.i=@pks.im header.b=UUTSTNRF; dkim=pass (2048-bit key) header.d=messagingengine.com header.i=@messagingengine.com header.b=MM+Lxfw/; arc=none smtp.client-ip=103.168.172.159 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=pks.im Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=pks.im Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=pks.im header.i=@pks.im header.b="UUTSTNRF"; dkim=pass (2048-bit key) header.d=messagingengine.com header.i=@messagingengine.com header.b="MM+Lxfw/" Received: from compute3.internal (compute3.nyi.internal [10.202.2.43]) by mailfhigh.nyi.internal (Postfix) with ESMTP id B638B1151C3B; Thu, 8 Aug 2024 03:35:54 -0400 (EDT) Received: from mailfrontend1 ([10.202.2.162]) by compute3.internal (MEProxy); Thu, 08 Aug 2024 03:35:54 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=pks.im; h=cc:cc :content-type:content-type:date:date:from:from:in-reply-to :in-reply-to:message-id:mime-version:references:reply-to:subject :subject:to:to; s=fm3; t=1723102554; x=1723188954; bh=rPIKWxe1g/ n96DafRJhm9L6URSYkbDWkBQ/J9bq0kfA=; b=UUTSTNRFjf/yMhDDFndW1wMV7Y qKI6OcYmLPNJFaHKcxTVGSZW+XUteU4tRTAmahC9k4VufVJYc7hRMFkCC9mekofC iqXscP23cVj/lUqSPM93PJPW5Eqtaq15/Rr5iYPpjT9/+Lxye7kTBEQnJ7W6vHkc Vl/NPu9QWWgIMJAHWzZD9vXdR3rG2fFMIvfy2wJO6iQ+VykIClveRJZASsftC9sF RNwRsLgHsE2Ic2CmPfL+f0XY73PQYI8xCzbgfnkz7n9EOPAQSoYdCxL5lTZjkycY 7q7vfkJkuJNEZaXvSKu/tG2WCS7htr2RS9Xk7fW5slLy7EGspowkKpvDg0kQ== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:cc:content-type:content-type:date:date :feedback-id:feedback-id:from:from:in-reply-to:in-reply-to :message-id:mime-version:references:reply-to:subject:subject:to :to:x-me-proxy:x-me-proxy:x-me-sender:x-me-sender:x-sasl-enc; s= fm3; t=1723102554; x=1723188954; bh=rPIKWxe1g/n96DafRJhm9L6URSYk bDWkBQ/J9bq0kfA=; b=MM+Lxfw/p0N2fIcQkCdx0RCZ0KK7X0A9ItqJjKj1S5r6 IQhk3iBGFVh2NorLiTQng6hRARHw0X7QEsZl4Pb0+JAPWGOxSeWDlytaAPuiVZQ4 S4Uny9fqSXIEZWvO+n5ccgyrQc6PawT0y5xDt+jLdEm3ueIY+O74yiJTjR0anm2p nbaW9uj9ILWpSAHQK/RFnM3VOX1AnYMIjeJW7KQMmzcKZ/GUrm7C/6uRAus9vr4G 1gzUavv/4pLrgaBEhBCjmgj5YfCimwbabfqzFwVyGonjsRYCYztpE0Cmvluhan8q 29z8qu+6zNMGlFsbrspLGYlW5imRywQHqLvjSbnOtQ== X-ME-Sender: X-ME-Received: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgeeftddrledugdduvddvucetufdoteggodetrfdotf fvucfrrhhofhhilhgvmecuhfgrshhtofgrihhlpdggtfgfnhhsuhgsshgtrhhisggvpdfu rfetoffkrfgpnffqhgenuceurghilhhouhhtmecufedttdenucesvcftvggtihhpihgvnh htshculddquddttddmnecujfgurhepfffhvfevuffkfhggtggujgesghdtreertddtvden ucfhrhhomheprfgrthhrihgtkhcuufhtvghinhhhrghrughtuceophhssehpkhhsrdhimh eqnecuggftrfgrthhtvghrnhepueektdevtdffveeljeetgfehheeigeekleduvdeffeeg hefgledttdehjeelffetnecuvehluhhsthgvrhfuihiivgepfeenucfrrghrrghmpehmrg hilhhfrhhomhepphhssehpkhhsrdhimhdpnhgspghrtghpthhtohepgedpmhhouggvpehs mhhtphhouhhtpdhrtghpthhtohepghhithesvhhgvghrrdhkvghrnhgvlhdrohhrghdprh gtphhtthhopehgihhtshhtvghrsehpohgsohigrdgtohhmpdhrtghpthhtohepshhunhhs hhhinhgvsehsuhhnshhhihhnvggtohdrtghomhdprhgtphhtthhopehjohhlrghnugesgh hmrghilhdrtghomh X-ME-Proxy: Feedback-ID: i197146af:Fastmail Received: by mail.messagingengine.com (Postfix) with ESMTPA; Thu, 8 Aug 2024 03:35:53 -0400 (EDT) Received: by vm-mail (OpenSMTPD) with ESMTPSA id b7a4ec17 (TLSv1.3:TLS_AES_256_GCM_SHA384:256:NO); Thu, 8 Aug 2024 07:35:47 +0000 (UTC) Date: Thu, 8 Aug 2024 09:35:51 +0200 From: Patrick Steinhardt To: git@vger.kernel.org Cc: Eric Sunshine , Jeppe =?utf-8?b?w5hsYW5k?= , Junio C Hamano Subject: [PATCH v2 6/8] submodule: fix leaking fetch tasks Message-ID: References: Precedence: bulk X-Mailing-List: git@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: When done with a fetch task used for parallel fetches of submodules, we need to both call `fetch_task_release()` to release the task's contents and `free()` to release the task itself. Most sites do this already, but some only call `fetch_task_release()` and thus leak memory. While we could trivially fix this by adding the two missing calls to free(3P), the result would be that we always call both functions. Let's thus refactor the code such that `fetch_task_release()` also frees the structure itself. Rename it to `fetch_task_free()` accordingly. Signed-off-by: Patrick Steinhardt --- submodule.c | 17 +++++++---------- 1 file changed, 7 insertions(+), 10 deletions(-) diff --git a/submodule.c b/submodule.c index ab99a30253..f027a6455e 100644 --- a/submodule.c +++ b/submodule.c @@ -1496,7 +1496,7 @@ static const struct submodule *get_non_gitmodules_submodule(const char *path) return (const struct submodule *) ret; } -static void fetch_task_release(struct fetch_task *p) +static void fetch_task_free(struct fetch_task *p) { if (p->free_sub) free((void*)p->sub); @@ -1508,6 +1508,7 @@ static void fetch_task_release(struct fetch_task *p) FREE_AND_NULL(p->repo); strvec_clear(&p->git_args); + free(p); } static struct repository *get_submodule_repo_for(struct repository *r, @@ -1576,8 +1577,7 @@ static struct fetch_task *fetch_task_create(struct submodule_parallel_fetch *spf return task; cleanup: - fetch_task_release(task); - free(task); + fetch_task_free(task); return NULL; } @@ -1607,8 +1607,7 @@ get_fetch_task_from_index(struct submodule_parallel_fetch *spf, } else { struct strbuf empty_submodule_path = STRBUF_INIT; - fetch_task_release(task); - free(task); + fetch_task_free(task); /* * An empty directory is normal, @@ -1654,8 +1653,7 @@ get_fetch_task_from_changed(struct submodule_parallel_fetch *spf, cs_data->path, repo_find_unique_abbrev(the_repository, cs_data->super_oid, DEFAULT_ABBREV)); - fetch_task_release(task); - free(task); + fetch_task_free(task); continue; } @@ -1763,7 +1761,7 @@ static int fetch_start_failure(struct strbuf *err UNUSED, spf->result = 1; - fetch_task_release(task); + fetch_task_free(task); return 0; } @@ -1828,8 +1826,7 @@ static int fetch_finish(int retvalue, struct strbuf *err UNUSED, } out: - fetch_task_release(task); - + fetch_task_free(task); return 0; } From patchwork Thu Aug 8 07:35:56 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Patrick Steinhardt X-Patchwork-Id: 13757062 Received: from fhigh8-smtp.messagingengine.com (fhigh8-smtp.messagingengine.com [103.168.172.159]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id CD8EA188CAE for ; Thu, 8 Aug 2024 07:35:59 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=103.168.172.159 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1723102561; cv=none; b=Ap4iX8jfsTu7XZJMraIEaS61M3T39wzWzvgUJ5/P6jk+i++OC5zNcgolJv5wzwAcDaJtprm/0sadbHv54yFp33rYSBfi8lsiJZ2TjcGYCRkh0/3vu+Bu0ppg1wLLuCo07YzWd/+fmBoOwvqS0f9MmbXyxdTmFq840BpArgr+ZiA= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1723102561; c=relaxed/simple; bh=LGMkl10FM+0W9xFnf9XcL5J6xATPimqHtR/5EGh/NwQ=; h=Date:From:To:Cc:Subject:Message-ID:References:MIME-Version: Content-Type:Content-Disposition:In-Reply-To; b=Gjcxru+oqWrOaCWCuyWNbKWzgS+Id0lTLuzuMw8Tt4LRoUzMDVlFqYssr3ulqmX/8iCIohg1cYhY7o/iiXxBeQoNPzgdLCuoY/7mC/toeaR0VHfN3a2HhvAkrAo+NTm3qeOAPyqURlr/of+BVThWvVN0xIwLQfdhuuwiqVamNsc= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=pks.im; spf=pass smtp.mailfrom=pks.im; dkim=pass (2048-bit key) header.d=pks.im header.i=@pks.im header.b=KtJkIKV+; dkim=pass (2048-bit key) header.d=messagingengine.com header.i=@messagingengine.com header.b=llYQNG4L; arc=none smtp.client-ip=103.168.172.159 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=pks.im Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=pks.im Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=pks.im header.i=@pks.im header.b="KtJkIKV+"; dkim=pass (2048-bit key) header.d=messagingengine.com header.i=@messagingengine.com header.b="llYQNG4L" Received: from compute4.internal (compute4.nyi.internal [10.202.2.44]) by mailfhigh.nyi.internal (Postfix) with ESMTP id EDA191151B80; Thu, 8 Aug 2024 03:35:58 -0400 (EDT) Received: from mailfrontend1 ([10.202.2.162]) by compute4.internal (MEProxy); Thu, 08 Aug 2024 03:35:58 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=pks.im; h=cc:cc :content-type:content-type:date:date:from:from:in-reply-to :in-reply-to:message-id:mime-version:references:reply-to:subject :subject:to:to; s=fm3; t=1723102558; x=1723188958; bh=hv1CIfxmPF 7BptPBkx+G6Dyrawma91txl0UC4/NUnzw=; b=KtJkIKV+uadlqyyGk0UYUsUJR1 QJj5l44C6Sh2yVOu9E6Bajd2u/HJ9rWWiqhmUvyqDGTSh6rsnRAlzAFZlCNOa8v0 Ap6YAEZkR7Epm8rvfuzuRAi+a7coUX4Zgu0GuC+OJPXZnSaF3B2vtnl23eCsqidv kM+OTvBpC9IwkUSjs5Tjfff+dSYs+ITNlT9VDAeSTuAp7ZGzom0tVxIztU/iad45 0KnHLwfdEXpubauX9I7/Vz2TDSG4n75Ed2Jx4r5VrEplaQn7nctZwr+qBREn8gpF nw2Y7cZQ+d8xLTggJsYwczG2CmL+fIohkAeszdaOuVZGct9ufU4hkoQ9PIWw== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:cc:content-type:content-type:date:date :feedback-id:feedback-id:from:from:in-reply-to:in-reply-to :message-id:mime-version:references:reply-to:subject:subject:to :to:x-me-proxy:x-me-proxy:x-me-sender:x-me-sender:x-sasl-enc; s= fm3; t=1723102558; x=1723188958; bh=hv1CIfxmPF7BptPBkx+G6Dyrawma 91txl0UC4/NUnzw=; b=llYQNG4LSzZlLz6ISKILWH+g2aFeimHnjrDgEUVKs/B7 XnGtH443v3v/HuaSE+n7rzKgrtuMLLrf5NozF4swG7cX8oAmHk0z3TFtsrt+Tkk/ boVlg6RH1MtLTNyOIlfXQFNbCLDu5ZM6zGBevmLy5ZtHtVy4LdDoXvU0tdGSnets pH9VdvkBclQsPoAUSrX8xnv0tuC+EbCp9XAsVjD1HuePEMCWBVTDzkHlyGjq6aCR l+7q+1qCTsKfs41n5I0+1qCL8P7I+KBCn59cZGZofjIvk05D4vkeZrIBsKJ/QGpz BUVGsQA+dFzbhrPjzv7H+nzQAcQPE5yAWtWHPekqMg== X-ME-Sender: X-ME-Received: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgeeftddrledugdduvdduucetufdoteggodetrfdotf fvucfrrhhofhhilhgvmecuhfgrshhtofgrihhlpdggtfgfnhhsuhgsshgtrhhisggvpdfu rfetoffkrfgpnffqhgenuceurghilhhouhhtmecufedttdenucesvcftvggtihhpihgvnh htshculddquddttddmnecujfgurhepfffhvfevuffkfhggtggujgesghdtreertddtvden ucfhrhhomheprfgrthhrihgtkhcuufhtvghinhhhrghrughtuceophhssehpkhhsrdhimh eqnecuggftrfgrthhtvghrnhepueektdevtdffveeljeetgfehheeigeekleduvdeffeeg hefgledttdehjeelffetnecuvehluhhsthgvrhfuihiivgepudenucfrrghrrghmpehmrg hilhhfrhhomhepphhssehpkhhsrdhimhdpnhgspghrtghpthhtohepgedpmhhouggvpehs mhhtphhouhhtpdhrtghpthhtohepghhithhsthgvrhesphhosghogidrtghomhdprhgtph htthhopehgihhtsehvghgvrhdrkhgvrhhnvghlrdhorhhgpdhrtghpthhtohepshhunhhs hhhinhgvsehsuhhnshhhihhnvggtohdrtghomhdprhgtphhtthhopehjohhlrghnugesgh hmrghilhdrtghomh X-ME-Proxy: Feedback-ID: i197146af:Fastmail Received: by mail.messagingengine.com (Postfix) with ESMTPA; Thu, 8 Aug 2024 03:35:57 -0400 (EDT) Received: by vm-mail (OpenSMTPD) with ESMTPSA id e562387d (TLSv1.3:TLS_AES_256_GCM_SHA384:256:NO); Thu, 8 Aug 2024 07:35:51 +0000 (UTC) Date: Thu, 8 Aug 2024 09:35:56 +0200 From: Patrick Steinhardt To: git@vger.kernel.org Cc: Eric Sunshine , Jeppe =?utf-8?b?w5hsYW5k?= , Junio C Hamano Subject: [PATCH v2 7/8] submodule: fix leaking seen submodule names Message-ID: References: Precedence: bulk X-Mailing-List: git@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: We keep track of submodules we have already seen via a string map such that we don't process the same submodule twice. We never free that map though, causing a memory leak. Fix this leak by clearing the map. Signed-off-by: Patrick Steinhardt --- submodule.c | 1 + t/t5572-pull-submodule.sh | 1 + t/t7418-submodule-sparse-gitmodules.sh | 1 + 3 files changed, 3 insertions(+) diff --git a/submodule.c b/submodule.c index f027a6455e..13b8f8c19c 100644 --- a/submodule.c +++ b/submodule.c @@ -1880,6 +1880,7 @@ int fetch_submodules(struct repository *r, strvec_clear(&spf.args); out: free_submodules_data(&spf.changed_submodule_names); + string_list_clear(&spf.seen_submodule_names, 0); return spf.result; } diff --git a/t/t5572-pull-submodule.sh b/t/t5572-pull-submodule.sh index 51744521f7..916e58c166 100755 --- a/t/t5572-pull-submodule.sh +++ b/t/t5572-pull-submodule.sh @@ -5,6 +5,7 @@ test_description='pull can handle submodules' GIT_TEST_FATAL_REGISTER_SUBMODULE_ODB=1 export GIT_TEST_FATAL_REGISTER_SUBMODULE_ODB +TEST_PASSES_SANITIZE_LEAK=true . ./test-lib.sh . "$TEST_DIRECTORY"/lib-submodule-update.sh diff --git a/t/t7418-submodule-sparse-gitmodules.sh b/t/t7418-submodule-sparse-gitmodules.sh index dde11ecce8..e1d9bf2ee3 100755 --- a/t/t7418-submodule-sparse-gitmodules.sh +++ b/t/t7418-submodule-sparse-gitmodules.sh @@ -15,6 +15,7 @@ also by committing .gitmodules and then just removing it from the filesystem. GIT_TEST_FATAL_REGISTER_SUBMODULE_ODB=1 export GIT_TEST_FATAL_REGISTER_SUBMODULE_ODB +TEST_PASSES_SANITIZE_LEAK=true . ./test-lib.sh test_expect_success 'setup' ' From patchwork Thu Aug 8 07:36:00 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Patrick Steinhardt X-Patchwork-Id: 13757063 Received: from fout3-smtp.messagingengine.com (fout3-smtp.messagingengine.com [103.168.172.146]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 08FCA18800F for ; Thu, 8 Aug 2024 07:36:04 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=103.168.172.146 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1723102569; cv=none; b=Ug6FSAmZlVj2o6fbdfJhUc0SjX0hpN6UZvHmc3DFS/UBopl44SZJbZxZlOQMrwEioXFVIigqSVCPVK988nx2+quURc7vfA6kQRRDRKBpb4yGDqhQOGNky0ALYLyvHOLZYd+B7h8mW9WzqYeV4DhJ1ZHboABeQgUkRtiWsqWqJJA= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1723102569; c=relaxed/simple; bh=xD9eN/spuTe11VT7QXfNzc4lWQw7RJ28zbj9cvczjaE=; h=Date:From:To:Cc:Subject:Message-ID:References:MIME-Version: Content-Type:Content-Disposition:In-Reply-To; b=kWvZaYWCrON1mZgCSUMqtpcuMANVxM95YiFi+qcTVHCqLqPCglHM3fVf3R2Y+6q/BY1E7cPx/c/4Li8zI5RKltv71MwhU46wR1wwSF8rZ08EZ8NXqEKmYOEn26gNVMHjRNwRcUrc5Q9FOgrs3eSQNEqrpgPKR7FUnaWr1Abqx2o= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=pks.im; spf=pass smtp.mailfrom=pks.im; dkim=pass (2048-bit key) header.d=pks.im header.i=@pks.im header.b=jORXIaDh; dkim=pass (2048-bit key) header.d=messagingengine.com header.i=@messagingengine.com header.b=WqbHXSdL; arc=none smtp.client-ip=103.168.172.146 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=pks.im Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=pks.im Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=pks.im header.i=@pks.im header.b="jORXIaDh"; dkim=pass (2048-bit key) header.d=messagingengine.com header.i=@messagingengine.com header.b="WqbHXSdL" Received: from compute2.internal (compute2.nyi.internal [10.202.2.46]) by mailfout.nyi.internal (Postfix) with ESMTP id 2C10A138FE27; Thu, 8 Aug 2024 03:36:04 -0400 (EDT) Received: from mailfrontend1 ([10.202.2.162]) by compute2.internal (MEProxy); Thu, 08 Aug 2024 03:36:04 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=pks.im; h=cc:cc :content-type:content-type:date:date:from:from:in-reply-to :in-reply-to:message-id:mime-version:references:reply-to:subject :subject:to:to; s=fm3; t=1723102564; x=1723188964; bh=lHHSFJrKQt I193Np657IeanxxPtYm+VPv/+5ErtfG+Y=; b=jORXIaDh67G0eJkfHtk6O6v/f1 Y5m8eP0O+KnZA+UnWwjBMAaIB9kNm/aXXv2YJDYxMB3Fxl7r09T+grcl6yHWE4jE 1NViisAeAW8xXmXoouGAYkYt1+twNu1UuiHnyGrvilfnVQZ7JQY12Kdfo/wx8EUq Zjeni21u14GMMaWnPSOkEatoX54v40hF1wDGXPj4kKIechZ9+mGKrE4VbE2chzxz 4KFVBjtD/52RHhTYxlmWjk5m7Gi1Tm/TrtjQLdWNgTqvDuML4p+4mHK9sIDitpAz E6us7qFSUhiBq7E1KQOHBc9jIjoxKXdcXKGxhsaTBhYnnMTsqursNGGrsHwA== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:cc:content-type:content-type:date:date :feedback-id:feedback-id:from:from:in-reply-to:in-reply-to :message-id:mime-version:references:reply-to:subject:subject:to :to:x-me-proxy:x-me-proxy:x-me-sender:x-me-sender:x-sasl-enc; s= fm3; t=1723102564; x=1723188964; bh=lHHSFJrKQtI193Np657IeanxxPtY m+VPv/+5ErtfG+Y=; b=WqbHXSdLX5FMmW8dfaqK5y2/5LgxMmMS8E3V/euO9cKZ Nju3vYZSCFXa73wmJxwGD9dRMshTbO467NzAZYLtG0Ljo5iCBA5g85UYrLh6JvqG z+6pOjfkF5EvqvqLmSROvSYUGWQJa1zfUoDHXY9sbX/TB0Evziqm4sKOKIAFkQ3K 0zUa8vbxWJlkffCjCthF7r2LDDTCS1DEj1ywQ+VKpoxgBHnynZOUzjePmZCF7C2o Hrnk3hkuez3e7DqFTfC0ZrVhnO3StpT8bIV7xgGi7I93RiRcKeddkYPN7TSEmhP+ Mf6wSOskKc5Xvp2lYKKovYDUiZNLGIhb/HaWVHJCKQ== X-ME-Sender: X-ME-Received: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgeeftddrledugdduvdduucetufdoteggodetrfdotf fvucfrrhhofhhilhgvmecuhfgrshhtofgrihhlpdggtfgfnhhsuhgsshgtrhhisggvpdfu rfetoffkrfgpnffqhgenuceurghilhhouhhtmecufedttdenucesvcftvggtihhpihgvnh htshculddquddttddmnecujfgurhepfffhvfevuffkfhggtggujgesghdtreertddtvden ucfhrhhomheprfgrthhrihgtkhcuufhtvghinhhhrghrughtuceophhssehpkhhsrdhimh eqnecuggftrfgrthhtvghrnhepueektdevtdffveeljeetgfehheeigeekleduvdeffeeg hefgledttdehjeelffetnecuvehluhhsthgvrhfuihiivgeptdenucfrrghrrghmpehmrg hilhhfrhhomhepphhssehpkhhsrdhimhdpnhgspghrtghpthhtohepgedpmhhouggvpehs mhhtphhouhhtpdhrtghpthhtohepshhunhhshhhinhgvsehsuhhnshhhihhnvggtohdrtg homhdprhgtphhtthhopehgihhtsehvghgvrhdrkhgvrhhnvghlrdhorhhgpdhrtghpthht ohepghhithhsthgvrhesphhosghogidrtghomhdprhgtphhtthhopehjohhlrghnugesgh hmrghilhdrtghomh X-ME-Proxy: Feedback-ID: i197146af:Fastmail Received: by mail.messagingengine.com (Postfix) with ESMTPA; Thu, 8 Aug 2024 03:36:03 -0400 (EDT) Received: by vm-mail (OpenSMTPD) with ESMTPSA id 8ea47cb5 (TLSv1.3:TLS_AES_256_GCM_SHA384:256:NO); Thu, 8 Aug 2024 07:35:56 +0000 (UTC) Date: Thu, 8 Aug 2024 09:36:00 +0200 From: Patrick Steinhardt To: git@vger.kernel.org Cc: Eric Sunshine , Jeppe =?utf-8?b?w5hsYW5k?= , Junio C Hamano Subject: [PATCH v2 8/8] object: fix leaking packfiles when closing object store Message-ID: References: Precedence: bulk X-Mailing-List: git@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: When calling `raw_object_store_clear()`, we close and free several resources associated with the object store. Part of that is to close and free all the packfiles, which is handled by `close_object_store()`. That function really only ends up closing the packfiles though, but it doesn't free them. And in fact it can't, as that function is being called via `run_command()` when `close_object_store = 1`, which is done e.g. when we execute git-maintenance(1). At that point, other structures may still have references on those packfiles, and thus we cannot free them here. So while it is in fact intentional that we really only close them, the result is a memory leak because `raw_object_store_clear()` does not free them, either. Fix the leak by freeing the packfiles in `raw_object_store_clear()`. Signed-off-by: Patrick Steinhardt --- object.c | 9 +++++++++ t/t7424-submodule-mixed-ref-formats.sh | 1 + 2 files changed, 10 insertions(+) diff --git a/object.c b/object.c index 0c0fcb76c0..d756c7f2ea 100644 --- a/object.c +++ b/object.c @@ -614,6 +614,15 @@ void raw_object_store_clear(struct raw_object_store *o) INIT_LIST_HEAD(&o->packed_git_mru); close_object_store(o); + + /* + * `close_object_store()` only closes the packfiles, but doesn't free + * them. We thus have to do this manually. + */ + for (struct packed_git *p = o->packed_git, *next; p; p = next) { + next = p->next; + free(p); + } o->packed_git = NULL; hashmap_clear(&o->pack_map); diff --git a/t/t7424-submodule-mixed-ref-formats.sh b/t/t7424-submodule-mixed-ref-formats.sh index 559713b607..b43ef2ba67 100755 --- a/t/t7424-submodule-mixed-ref-formats.sh +++ b/t/t7424-submodule-mixed-ref-formats.sh @@ -2,6 +2,7 @@ test_description='submodules handle mixed ref storage formats' +TEST_PASSES_SANITIZE_LEAK=true . ./test-lib.sh test_ref_format () {