From patchwork Tue Apr 18 20:40:32 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Taylor Blau X-Patchwork-Id: 13216143 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id A5C4DC77B75 for ; Tue, 18 Apr 2023 20:40:38 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232285AbjDRUkh (ORCPT ); Tue, 18 Apr 2023 16:40:37 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:43506 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230290AbjDRUkf (ORCPT ); Tue, 18 Apr 2023 16:40:35 -0400 Received: from mail-yw1-x1132.google.com (mail-yw1-x1132.google.com [IPv6:2607:f8b0:4864:20::1132]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 6B7E4A5CC for ; Tue, 18 Apr 2023 13:40:34 -0700 (PDT) Received: by mail-yw1-x1132.google.com with SMTP id 00721157ae682-54ee0b73e08so476888467b3.0 for ; Tue, 18 Apr 2023 13:40:34 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ttaylorr-com.20221208.gappssmtp.com; s=20221208; t=1681850434; x=1684442434; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:from:to:cc:subject:date:message-id:reply-to; bh=GowqkZqJzwz5Wl6gCYv13LFGatPuqu0IDG+u/xSPVpI=; b=UUocnVVD/irqrJ0hmKF2a5Ba8spZoDzlx5ZyBOflnGTuBtr2jI6jySS3Qk+hLt4FZc C1ZO/JO4WbrLq/Ovek57Fqolz4bR0TV2BOjv3FbsbamW74QQE3gzri9OxVwKe+ItL9JK LMixRRUkmuXwHVDZZkgSVwTLNCBmjNEYF7drRqpBD3RK4tvvol2uO7k98ZuVWUWZWu6c eK33VMGcxz7JEha7Rw+gm+X3iuO48WnpyWcAcN1MdXmk22ZTrX5VWQSIT/C/ZmFadjgY rR2kOaSfMsoiXCt6lB1fMtzuswFrZztJHVPl1TSuc9pYbQ/mgY6vPJAooQ9vJ1ICXAo2 dxEw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1681850434; x=1684442434; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:x-gm-message-state:from:to:cc:subject:date :message-id:reply-to; bh=GowqkZqJzwz5Wl6gCYv13LFGatPuqu0IDG+u/xSPVpI=; b=cx2SrWnutLpkQ/P5cOdA8b36JK7MQoDZZ25+RYeo+mPirJcEjoEhCm0WRmkaQ1tQC0 VJbLOhLZjR30gd0kP0AMB/j5Uwfs8xJsZdK2MOU7LzBstdeTQh/SgD5v3LTA1Pqw0D4S 6zZK8WYRyUmjtjteowQD6/ZR2wmA8RfXzjrRwjc9JlJy+wx+GbuKTHrZ++8hV2fCiG9U oynnho7MT6gFVnWgltbdtdAOXA3y3nJyX7S4SKCjcN+A4Cl+RPoAbMDlzzaM1wofLrNm 9edyZqwGZ+2McLQONt3xAfVF4hUk+RqCzP7ReZS9EVlk2A+xR7DxYsZSGsehsLkZjea2 k8VQ== X-Gm-Message-State: AAQBX9dE45LGedop6eTItYyubd825meec1w8r2kukHSY1Krm4fk3MgT/ Y81YCW/AuS49YeOwuobyqZ8N7tXrssMJFcqO0lsULQ== X-Google-Smtp-Source: AKy350a+ZDQBtoHCPAhypIiWwkrbC1+Or8qCDddLqqV23nDU1kmlZeKShu1mvjOFqcRQq4huhPZGkQ== X-Received: by 2002:a0d:ef87:0:b0:551:11e7:30cc with SMTP id y129-20020a0def87000000b0055111e730ccmr947020ywe.22.1681850433967; Tue, 18 Apr 2023 13:40:33 -0700 (PDT) Received: from localhost (104-178-186-189.lightspeed.milwwi.sbcglobal.net. [104.178.186.189]) by smtp.gmail.com with ESMTPSA id 4-20020a811204000000b0054f9e7fed7asm4054916yws.137.2023.04.18.13.40.33 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 18 Apr 2023 13:40:33 -0700 (PDT) Date: Tue, 18 Apr 2023 16:40:32 -0400 From: Taylor Blau To: git@vger.kernel.org Cc: Jeff King , Derrick Stolee , Junio C Hamano Subject: [PATCH v2 01/10] pack-write.c: plug a leak in stage_tmp_packfiles() Message-ID: References: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org The function `stage_tmp_packfiles()` generates a filename to use for staging the contents of what will become the pack's ".mtimes" file. The name is generated in `write_mtimes_file()` and the result is returned back to `stage_tmp_packfiles()` which uses it to rename the temporary file into place via `rename_tmp_packfiles()`. `write_mtimes_file()` returns a `const char *`, indicating that callers are not expected to free its result (similar to, e.g., `oid_to_hex()`). But callers are expected to free its result, so this return type is incorrect. Change the function's signature to return a non-const `char *`, and free it at the end of `stage_tmp_packfiles()`. Signed-off-by: Taylor Blau --- pack-write.c | 14 ++++++++------ 1 file changed, 8 insertions(+), 6 deletions(-) diff --git a/pack-write.c b/pack-write.c index f171405495..4da0ccc5f5 100644 --- a/pack-write.c +++ b/pack-write.c @@ -312,13 +312,13 @@ static void write_mtimes_trailer(struct hashfile *f, const unsigned char *hash) hashwrite(f, hash, the_hash_algo->rawsz); } -static const char *write_mtimes_file(struct packing_data *to_pack, - struct pack_idx_entry **objects, - uint32_t nr_objects, - const unsigned char *hash) +static char *write_mtimes_file(struct packing_data *to_pack, + struct pack_idx_entry **objects, + uint32_t nr_objects, + const unsigned char *hash) { struct strbuf tmp_file = STRBUF_INIT; - const char *mtimes_name; + char *mtimes_name; struct hashfile *f; int fd; @@ -544,7 +544,7 @@ void stage_tmp_packfiles(struct strbuf *name_buffer, char **idx_tmp_name) { const char *rev_tmp_name = NULL; - const char *mtimes_tmp_name = NULL; + char *mtimes_tmp_name = NULL; if (adjust_shared_perm(pack_tmp_name)) die_errno("unable to make temporary pack file readable"); @@ -568,6 +568,8 @@ void stage_tmp_packfiles(struct strbuf *name_buffer, rename_tmp_packfile(name_buffer, rev_tmp_name, "rev"); if (mtimes_tmp_name) rename_tmp_packfile(name_buffer, mtimes_tmp_name, "mtimes"); + + free(mtimes_tmp_name); } void write_promisor_file(const char *promisor_name, struct ref **sought, int nr_sought) From patchwork Tue Apr 18 20:40:35 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Taylor Blau X-Patchwork-Id: 13216144 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 89BA7C77B75 for ; Tue, 18 Apr 2023 20:40:42 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230290AbjDRUkl (ORCPT ); Tue, 18 Apr 2023 16:40:41 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:43556 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232391AbjDRUkk (ORCPT ); Tue, 18 Apr 2023 16:40:40 -0400 Received: from mail-yw1-x1134.google.com (mail-yw1-x1134.google.com [IPv6:2607:f8b0:4864:20::1134]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 21476AD1E for ; Tue, 18 Apr 2023 13:40:39 -0700 (PDT) Received: by mail-yw1-x1134.google.com with SMTP id 00721157ae682-54fc337a650so222873407b3.4 for ; Tue, 18 Apr 2023 13:40:39 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ttaylorr-com.20221208.gappssmtp.com; s=20221208; t=1681850438; x=1684442438; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:from:to:cc:subject:date:message-id:reply-to; bh=7Hs+3ywhbstovkOsQyBNUBwcoLP41kBSwzvbmhtSIqs=; b=4QhlGv1npoYSLa2CcxGInOijJYslrUEVfslk0V9JuK3j6X0asIS6BaprgNnVbWAYVk 79QZYZ4I6kyQN1kxk3huj0tiCJEuXwmHLOeTNxL+eUON2DvKqvIqsEPIjcHyfjyt9kzG cc3EQDBhVgg0KQdOvosRZkSSlmJnaMXjbq2QZXe6IIlFmo9l4gYSGDqJusr4L6dO07bT SiNsNTh/ticNsOtOHxI5zDBVQVv0Mt+C3M9inrLiuMlrUyzVAV0Ouq6awmjhhQzOy1hg FCQshCXe1wzHRlmg+6v4Z+JYt0cv8fKT2PZ/h6nL3I9zL2gQgIOtm6He0LDLa/srisoW iYNA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1681850438; x=1684442438; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:x-gm-message-state:from:to:cc:subject:date :message-id:reply-to; bh=7Hs+3ywhbstovkOsQyBNUBwcoLP41kBSwzvbmhtSIqs=; b=DDixPNgqdw/750YYMi75dGklGxu0WYdTBknHGj671HOhmjSQI15lgNPdNdWdXm8jbN FHebzSiw42FlZQEpxpO6CRtW1LnWzW4mfzGEqQN3hhqKVoQlox24md+ZwNluulR3OpK7 TJooYxeOXCZBccKkbIO6JH8Qks0GBknbPuFCJd4VocrC7Lzq0ovR7BiKU2UEmCAE3g3c uuWe9CfKr/ha9/uiX3troAD3j+M3h8WcjbVNzMwr5gsqPGFMMVm02zfyGblUB7N/f6gE Fi0y6S7MuUE1VpJx/Blz7dgN81UGrex9Ddzm00e/Q/Z3NH98KkSBiSX+GbrwiA9VS0QG Y9Qw== X-Gm-Message-State: AAQBX9css3tA+ES6tFeJQHJ1WuofjtGN+e2piRHwsvmqUtD5Dz19j/ur 2YMzFQFLdftyfyLVjnmjr4NKBtMc7Vt5k/3HfJghPg== X-Google-Smtp-Source: AKy350Y7eBIUo1YMtRWvWUYbG1kcFdu5E3Q1sMO+T5VATa9VCHRLghD1qjyWqqwIPya/diZkPL5kCg== X-Received: by 2002:a81:8701:0:b0:54f:881e:75bb with SMTP id x1-20020a818701000000b0054f881e75bbmr1041825ywf.45.1681850436770; Tue, 18 Apr 2023 13:40:36 -0700 (PDT) Received: from localhost (104-178-186-189.lightspeed.milwwi.sbcglobal.net. [104.178.186.189]) by smtp.gmail.com with ESMTPSA id a206-20020a814dd7000000b00545a081847bsm4057680ywb.11.2023.04.18.13.40.36 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 18 Apr 2023 13:40:36 -0700 (PDT) Date: Tue, 18 Apr 2023 16:40:35 -0400 From: Taylor Blau To: git@vger.kernel.org Cc: Jeff King , Derrick Stolee , Junio C Hamano Subject: [PATCH v2 02/10] builtin/repack.c: fix incorrect reference to '-C' Message-ID: <52fb61fa9c7398c7b412336b3ebe5724f5263108.1681850424.git.me@ttaylorr.com> References: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org When cruft packs were originally being developed, `-C` was designated as the short-form for `--cruft` (as in `git repack -C`). This was dropped due to confusion with Git's top-level `-C` option before submitting to the list. But the reference to it in `--cruft-expiration`'s help text was never updated. Fix that dangling reference in this patch. Signed-off-by: Taylor Blau --- builtin/repack.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/builtin/repack.c b/builtin/repack.c index df4d8e0f0b..d9eee15c2f 100644 --- a/builtin/repack.c +++ b/builtin/repack.c @@ -774,7 +774,7 @@ int cmd_repack(int argc, const char **argv, const char *prefix) N_("same as -a, pack unreachable cruft objects separately"), PACK_CRUFT), OPT_STRING(0, "cruft-expiration", &cruft_expiration, N_("approxidate"), - N_("with -C, expire objects older than this")), + N_("with --cruft, expire objects older than this")), OPT_BOOL('d', NULL, &delete_redundant, N_("remove redundant packs, and run git-prune-packed")), OPT_BOOL('f', NULL, &po_args.no_reuse_delta, From patchwork Tue Apr 18 20:40:38 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Taylor Blau X-Patchwork-Id: 13216145 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id DF304C77B75 for ; Tue, 18 Apr 2023 20:40:48 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232611AbjDRUkr (ORCPT ); Tue, 18 Apr 2023 16:40:47 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:43604 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230465AbjDRUko (ORCPT ); Tue, 18 Apr 2023 16:40:44 -0400 Received: from mail-yw1-x112c.google.com (mail-yw1-x112c.google.com [IPv6:2607:f8b0:4864:20::112c]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 07663C164 for ; Tue, 18 Apr 2023 13:40:41 -0700 (PDT) Received: by mail-yw1-x112c.google.com with SMTP id 00721157ae682-54c12009c30so581295537b3.9 for ; Tue, 18 Apr 2023 13:40:41 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ttaylorr-com.20221208.gappssmtp.com; s=20221208; t=1681850441; x=1684442441; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:from:to:cc:subject:date:message-id:reply-to; bh=LSULjQeZr6EERLMdQ3oz0fdM8+pujbGzD2Wwp37zZXA=; b=n1pZj1pJo2Ez96fc9nRf84dcFfUm1b2GRAJHdmi2MUCSTyK/M7K/AyDBLu2LYQ3gu/ 7udo0PtHnh8wp501fVJFsMqN06gxWcz7LPpJ8SaQCLYQ412UJ6tFOn46/Pq43G+vqCLG FVx4hUYyCeBFI0d9uAW0nKpTHs/UKMysZFnH091PvXD7G6H/vs1ClZ3g5msxpUqctaaf nrcwy/G2wHcvN82ObqbZ0K2HypimSrgqk0UkrhUERrdf+uRLqKZtdqNDirSYt+81p8zg Uqk2rPa9Pdv48kaw8cFKON2t7QUbLmKWnGQJMBECaOvqUgQ4h6ZNoulTKETyqV29K/gf imXg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1681850441; x=1684442441; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:x-gm-message-state:from:to:cc:subject:date :message-id:reply-to; bh=LSULjQeZr6EERLMdQ3oz0fdM8+pujbGzD2Wwp37zZXA=; b=QPYtKqaImm9IDG1m69zhEtO0W0R8SdAN4ZA+4FhqU44WjiuAgjBTNR8T2wy2710eNT 9YtYnisIQZYXugDm2rxg6RogBr7cWMvZrm6asDq3BhbgugSFRO2/WmdqVwoibCSfDp8w mMtVzfKoJ0ZXz22DKKP6aioBKYLJgMV8Uz+4CK0Kb5WW2vUzlN0fq+UQ+tSHA9eziW+k P8VMOTWQVwWJYtLpYzw7hsqiYPdAL50xdmJcqFnw7nb7tGf/6J4/S6yAi2osUefWGj4Z 5GYjBI84O+h/A1/68JNtNBwdNHCfSnf4J9bZ17hnbqj3hp7DRm5ZBoOWiNLfdWdPP3Wp 0ohg== X-Gm-Message-State: AAQBX9eVY1FKOi6Opz68NCbrT9C4/BaLzIgH66fi/ZYVGuByZ12+mnsg WLgg8uOl67bxvE6WCaWhGglPFvO6uBrx/aC/NcJpUw== X-Google-Smtp-Source: AKy350bIppEknWnRrph6K8w/Qye7NNeuev29y4CAGsHju8XKnTeF/4XQ4MXmjA8iGUQRL0vQdpmxhQ== X-Received: by 2002:a81:6c15:0:b0:54f:b6b6:1906 with SMTP id h21-20020a816c15000000b0054fb6b61906mr1210611ywc.24.1681850439463; Tue, 18 Apr 2023 13:40:39 -0700 (PDT) Received: from localhost (104-178-186-189.lightspeed.milwwi.sbcglobal.net. [104.178.186.189]) by smtp.gmail.com with ESMTPSA id d63-20020a814f42000000b0054f8b7b8514sm4074644ywb.128.2023.04.18.13.40.39 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 18 Apr 2023 13:40:39 -0700 (PDT) Date: Tue, 18 Apr 2023 16:40:38 -0400 From: Taylor Blau To: git@vger.kernel.org Cc: Jeff King , Derrick Stolee , Junio C Hamano Subject: [PATCH v2 03/10] builtin/gc.c: ignore cruft packs with `--keep-largest-pack` Message-ID: <63b9ee8e2e398abb4a7cb435589b080dd26c266d.1681850424.git.me@ttaylorr.com> References: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org When cruft packs were implemented, we never adjusted the code for `git gc`'s `--keep-largest-pack` and `gc.bigPackThreshold` to ignore cruft packs. This option and configuration option share a common implementation, but including cruft packs is wrong in both cases: - Running `git gc --keep-largest-pack` in a repository where the largest pack is the cruft pack itself will make it impossible for `git gc` to prune objects, since the cruft pack itself is kept. - The same is true for `gc.bigPackThreshold`, if the size of the cruft pack exceeds the limit set by the caller. In the future, it is possible that `gc.bigPackThreshold` could be used to write a separate cruft pack containing any new unreachable objects that entered the repository since the last time a cruft pack was written. There are some complexities to doing so, mainly around handling pruning objects that are in an existing cruft pack that is above the threshold (which would either need to be rewritten, or else delay pruning). Rewriting a substantially similar cruft pack isn't ideal, but it is significantly better than the status-quo. If users have large cruft packs that they don't want to rewrite, they can mark them as `*.keep` packs. But in general, if a repository has a cruft pack that is so large it is slowing down GC's, it should probably be pruned anyway. In the meantime, ignore cruft packs in the common implementation for both of these options, and add a pair of tests to prevent any future regressions here. Signed-off-by: Taylor Blau --- Documentation/config/gc.txt | 10 ++++----- Documentation/git-gc.txt | 7 +++--- builtin/gc.c | 2 +- t/t6500-gc.sh | 43 +++++++++++++++++++++++++++++++++++++ 4 files changed, 53 insertions(+), 9 deletions(-) diff --git a/Documentation/config/gc.txt b/Documentation/config/gc.txt index 38fea076a2..8d5353e9e0 100644 --- a/Documentation/config/gc.txt +++ b/Documentation/config/gc.txt @@ -43,11 +43,11 @@ gc.autoDetach:: if the system supports it. Default is true. gc.bigPackThreshold:: - If non-zero, all packs larger than this limit are kept when - `git gc` is run. This is very similar to `--keep-largest-pack` - except that all packs that meet the threshold are kept, not - just the largest pack. Defaults to zero. Common unit suffixes of - 'k', 'm', or 'g' are supported. + If non-zero, all non-cruft packs larger than this limit are kept + when `git gc` is run. This is very similar to + `--keep-largest-pack` except that all non-cruft packs that meet + the threshold are kept, not just the largest pack. Defaults to + zero. Common unit suffixes of 'k', 'm', or 'g' are supported. + Note that if the number of kept packs is more than gc.autoPackLimit, this configuration variable is ignored, all packs except the base pack diff --git a/Documentation/git-gc.txt b/Documentation/git-gc.txt index a65c9aa62d..fef382a70f 100644 --- a/Documentation/git-gc.txt +++ b/Documentation/git-gc.txt @@ -77,9 +77,10 @@ be performed as well. instance running on this repository. --keep-largest-pack:: - All packs except the largest pack and those marked with a - `.keep` files are consolidated into a single pack. When this - option is used, `gc.bigPackThreshold` is ignored. + All packs except the largest non-cruft pack, any packs marked + with a `.keep` file, and any cruft pack(s) are consolidated into + a single pack. When this option is used, `gc.bigPackThreshold` + is ignored. AGGRESSIVE ---------- diff --git a/builtin/gc.c b/builtin/gc.c index edd98d35a5..53ef137e1d 100644 --- a/builtin/gc.c +++ b/builtin/gc.c @@ -219,7 +219,7 @@ static struct packed_git *find_base_packs(struct string_list *packs, struct packed_git *p, *base = NULL; for (p = get_all_packs(the_repository); p; p = p->next) { - if (!p->pack_local) + if (!p->pack_local || p->is_cruft) continue; if (limit) { if (p->pack_size >= limit) diff --git a/t/t6500-gc.sh b/t/t6500-gc.sh index d9acb63951..df6f2e6e52 100755 --- a/t/t6500-gc.sh +++ b/t/t6500-gc.sh @@ -298,6 +298,49 @@ test_expect_success 'feature.experimental=false avoids cruft packs by default' ' ) ' +test_expect_success '--keep-largest-pack ignores cruft packs' ' + test_when_finished "rm -fr repo" && + git init repo && + ( + cd repo && + + # Generate a pack for reachable objects (of which there + # are 3), and one for unreachable objects (of which + # there are 6). + prepare_cruft_history && + git gc --cruft && + + mtimes="$(find .git/objects/pack -type f -name "pack-*.mtimes")" && + sz="$(test_file_size "${mtimes%.mtimes}.pack")" && + + # Ensure that the cruft pack gets removed (due to + # `--prune=now`) despite it being the largest pack. + git -c gc.bigPackThreshold=$sz gc --cruft --prune=now && + + assert_no_cruft_packs + ) +' + +test_expect_success 'gc.bigPackThreshold ignores cruft packs' ' + test_when_finished "rm -fr repo" && + git init repo && + ( + cd repo && + + # Generate a pack for reachable objects (of which there + # are 3), and one for unreachable objects (of which + # there are 6). + prepare_cruft_history && + git gc --cruft && + + # Ensure that the cruft pack gets removed (due to + # `--prune=now`) despite it being the largest pack. + git gc --cruft --prune=now --keep-largest-pack && + + assert_no_cruft_packs + ) +' + run_and_wait_for_auto_gc () { # We read stdout from gc for the side effect of waiting until the # background gc process exits, closing its fd 9. Furthermore, the From patchwork Tue Apr 18 20:40:41 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Taylor Blau X-Patchwork-Id: 13216146 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id DD0C4C6FD18 for ; Tue, 18 Apr 2023 20:40:49 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232479AbjDRUkt (ORCPT ); Tue, 18 Apr 2023 16:40:49 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:43650 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232474AbjDRUko (ORCPT ); Tue, 18 Apr 2023 16:40:44 -0400 Received: from mail-yw1-x1132.google.com (mail-yw1-x1132.google.com [IPv6:2607:f8b0:4864:20::1132]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id E1C8AC65F for ; Tue, 18 Apr 2023 13:40:42 -0700 (PDT) Received: by mail-yw1-x1132.google.com with SMTP id 00721157ae682-54ee0b73e08so476892887b3.0 for ; Tue, 18 Apr 2023 13:40:42 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ttaylorr-com.20221208.gappssmtp.com; s=20221208; t=1681850442; x=1684442442; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:from:to:cc:subject:date:message-id:reply-to; bh=kgMBQJmcf8c34ITq6py7jrYV3tEVfzPkI6ydstMTiAc=; b=B92wNuyYHIVGJMR3Rx9hIJlwZGZxpqS0c3N8y2wf2lU9baZH2iV9fZmzWFzvSOUIFW lLyTPsCG33x6Qcd3meexO+I+cjT2q9dyJNPIzXJcrX6N9hrvp8P0BEKptFkuO+JICf1g Xm1t0bNWDgEtZtQWLcrWZqgdV3kWb2PjsEL2N2NyIZc9WjEiMIOInoB61zOHUg1czCP7 8S1jNDnZo/zjQrruoArlv6kxcy2VtSjGzm2BLi4tjJRQqc4yXzndpgvPW9OoRk3QqaGi bbeyJxqwTJAGmEkks1FlIyt7QxB7RyQvkOHXCDwtrLKIpSjCD8TN1k1zgPpDh/tGI2WP ctJA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1681850442; x=1684442442; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:x-gm-message-state:from:to:cc:subject:date :message-id:reply-to; bh=kgMBQJmcf8c34ITq6py7jrYV3tEVfzPkI6ydstMTiAc=; b=ZncO8loLBwsIK75CrKPEk1ZawAzzRQt6MjNMMzWpagcQZYnJn9/wwN19V7K1zgzbkh F0skQfj1xvCoRggEPm2zZBl5Zh6KbXZsCUp1rvpP6jl/sOikMD7M2iqCmXAP6TRCK2RY bScEkU3Er6MdO4LrsCJfNcvWmlQs/1hIJ9hHnztEKcF8UjiCfvDMudhkfc3gQBl9djmg m6bEIr8eRiJcVqT8nqqk+e4Vh4Gqhq58UXD7mlGx+nymlP7A7Wwf9vkg184xvcggJE+R zJv/tND9l+REZ+qZAQ6Hd7lw4HlekGkJY/CyFjWony/0FcwyBIWG77Ezw4J22HOZqYpW Xj4g== X-Gm-Message-State: AAQBX9fM9lT2PwNEHXxLYTnoPI/Knecx+bpCf2SMblscnmiOdlhu40iP kDaK8N/BiLrQ7Rx6HB+pcIh1WsBnxHM7lE1CINLxTQ== X-Google-Smtp-Source: AKy350ZHzS6EJCmhYB4gR6wUJG02g4D3CXG8ANUqacerSJAcSJ9Cw4TsG8wKr8I5oCOoDB7rwy+eCw== X-Received: by 2002:a0d:d286:0:b0:54f:e7cd:bb1d with SMTP id u128-20020a0dd286000000b0054fe7cdbb1dmr1116558ywd.12.1681850442242; Tue, 18 Apr 2023 13:40:42 -0700 (PDT) Received: from localhost (104-178-186-189.lightspeed.milwwi.sbcglobal.net. [104.178.186.189]) by smtp.gmail.com with ESMTPSA id l18-20020a812512000000b0054f8b69889fsm4047611ywl.74.2023.04.18.13.40.41 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 18 Apr 2023 13:40:42 -0700 (PDT) Date: Tue, 18 Apr 2023 16:40:41 -0400 From: Taylor Blau To: git@vger.kernel.org Cc: Jeff King , Derrick Stolee , Junio C Hamano Subject: [PATCH v2 04/10] t/t5304-prune.sh: prepare for `gc --cruft` by default Message-ID: <905eeb90275178ac53adddb1abc17d149775dd60.1681850424.git.me@ttaylorr.com> References: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org Many of the tests in t5304 run `git gc`, and rely on its behavior that unreachable-but-recent objects are written out loose. This is sensible, since t5304 deals specifically with this kind of pruning. If left unattended, however, this test would break when the default behavior of a bare "git gc" is adjusted to generate a cruft pack by default. Ensure that these tests continue to work as-is (and continue to provide coverage of loose object pruning) by passing `--no-cruft` explicitly. Signed-off-by: Taylor Blau --- t/t5304-prune.sh | 28 ++++++++++++++-------------- 1 file changed, 14 insertions(+), 14 deletions(-) diff --git a/t/t5304-prune.sh b/t/t5304-prune.sh index 5500dd0842..662ae9b152 100755 --- a/t/t5304-prune.sh +++ b/t/t5304-prune.sh @@ -62,11 +62,11 @@ test_expect_success 'prune --expire' ' test_expect_success 'gc: implicit prune --expire' ' add_blob && test-tool chmtime =-$((2*$week-30)) $BLOB_FILE && - git gc && + git gc --no-cruft && verbose test $((1 + $before)) = $(git count-objects | sed "s/ .*//") && test_path_is_file $BLOB_FILE && test-tool chmtime =-$((2*$week+1)) $BLOB_FILE && - git gc && + git gc --no-cruft && verbose test $before = $(git count-objects | sed "s/ .*//") && test_path_is_missing $BLOB_FILE ' @@ -86,7 +86,7 @@ test_expect_success 'gc: refuse to start with invalid gc.pruneExpire' ' test_expect_success 'gc: start with ok gc.pruneExpire' ' git config gc.pruneExpire 2.days.ago && - git gc + git gc --no-cruft ' test_expect_success 'prune: prune nonsense parameters' ' @@ -137,44 +137,44 @@ test_expect_success 'gc --no-prune' ' add_blob && test-tool chmtime =-$((5001*$day)) $BLOB_FILE && git config gc.pruneExpire 2.days.ago && - git gc --no-prune && + git gc --no-prune --no-cruft && verbose test 1 = $(git count-objects | sed "s/ .*//") && test_path_is_file $BLOB_FILE ' test_expect_success 'gc respects gc.pruneExpire' ' git config gc.pruneExpire 5002.days.ago && - git gc && + git gc --no-cruft && test_path_is_file $BLOB_FILE && git config gc.pruneExpire 5000.days.ago && - git gc && + git gc --no-cruft && test_path_is_missing $BLOB_FILE ' test_expect_success 'gc --prune=' ' add_blob && test-tool chmtime =-$((5001*$day)) $BLOB_FILE && - git gc --prune=5002.days.ago && + git gc --prune=5002.days.ago --no-cruft && test_path_is_file $BLOB_FILE && - git gc --prune=5000.days.ago && + git gc --prune=5000.days.ago --no-cruft && test_path_is_missing $BLOB_FILE ' test_expect_success 'gc --prune=never' ' add_blob && - git gc --prune=never && + git gc --prune=never --no-cruft && test_path_is_file $BLOB_FILE && - git gc --prune=now && + git gc --prune=now --no-cruft && test_path_is_missing $BLOB_FILE ' test_expect_success 'gc respects gc.pruneExpire=never' ' git config gc.pruneExpire never && add_blob && - git gc && + git gc --no-cruft && test_path_is_file $BLOB_FILE && git config gc.pruneExpire now && - git gc && + git gc --no-cruft && test_path_is_missing $BLOB_FILE ' @@ -194,7 +194,7 @@ test_expect_success 'gc: prune old objects after local clone' ' cd aclone && verbose test 1 = $(git count-objects | sed "s/ .*//") && test_path_is_file $BLOB_FILE && - git gc --prune && + git gc --prune --no-cruft && verbose test 0 = $(git count-objects | sed "s/ .*//") && test_path_is_missing $BLOB_FILE ) @@ -237,7 +237,7 @@ test_expect_success 'clean pack garbage with gc' ' >.git/objects/pack/fake2.keep && >.git/objects/pack/fake2.idx && >.git/objects/pack/fake3.keep && - git gc && + git gc --no-cruft && git count-objects -v 2>stderr && grep "^warning:" stderr | sort >actual && cat >expected <<\EOF && From patchwork Tue Apr 18 20:40:43 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Taylor Blau X-Patchwork-Id: 13216147 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 0BBC9C77B75 for ; Tue, 18 Apr 2023 20:40:59 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232624AbjDRUk4 (ORCPT ); Tue, 18 Apr 2023 16:40:56 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:43776 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232391AbjDRUkr (ORCPT ); Tue, 18 Apr 2023 16:40:47 -0400 Received: from mail-yw1-x112c.google.com (mail-yw1-x112c.google.com [IPv6:2607:f8b0:4864:20::112c]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 7942ECC1B for ; Tue, 18 Apr 2023 13:40:45 -0700 (PDT) Received: by mail-yw1-x112c.google.com with SMTP id 00721157ae682-54c12009c30so581297737b3.9 for ; Tue, 18 Apr 2023 13:40:45 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ttaylorr-com.20221208.gappssmtp.com; s=20221208; t=1681850445; x=1684442445; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:from:to:cc:subject:date:message-id:reply-to; bh=7PD6rXkq70FqXYe/Y3zZYy5XGdBSlQTwqEFYec7m9f0=; b=2jwbGWn4ZxpYYGwbBNlSjEVPqDMJZIArqeZA/Q2jyf+tDuatfm2zGWXFQz8qj9qg7D 84HpmvujzQ9s34VvmMVl+L+LS2XirmUhZREs1MuVf4KueQLLLV6UvDKP4y5QvXH7yzCg TX7wYuIdQZDgOFX12CxyeqBBzBqm/ldlh4xzT6iGYKC9mrJC/iBhIB2xZhZmI2oqYXG9 9eABenf7d31I1/N5m9pxWccxq+0Z5U8DRWgi7MXR7y810I1nEbH1C1BnaVXmq6rQ21zs xJAHYmVeC6+IEWVz7+YrvvSwvQyZ6hk8rno3ZGp3sypqjj2/qpPT9fWOK59kn6kMEQmB QjfQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1681850445; x=1684442445; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:x-gm-message-state:from:to:cc:subject:date :message-id:reply-to; bh=7PD6rXkq70FqXYe/Y3zZYy5XGdBSlQTwqEFYec7m9f0=; b=izmZuoc04yGntw55W2n6zrGM5H6UM0gxnOcO/gDrXCEmRr/bEUgE+sh5ToKm1QTVy9 yQHceuMLEALxdeRNEQaG4zLULn+58Kk6sLQOoEeFTArhgdavkINOlgKLUJ/m5HtTMtv5 5I4uDgGi7Isi7gmYsNC1VjDXRuOfVxE+WTOUX85dYZz0hz/S+ymQG1z85a0vvc4ND63N 4hfZUP09trMvxYMBxxlV4F61MidqYy0VvzwRoCTpelvGhoYRk55VZ5+0t7NxpMjHk4DZ nPpBA7BDBdo5rbUjOOHCDT12jSgCe8cV8snMtCSh7OCDcrLMb8KxlxJsxHuviqOrL9RV 0Itg== X-Gm-Message-State: AAQBX9cU5IGPQ807+zTMe6uYHa+ACouesgavNtIbIlYz15VKs+ctODnZ e3wKmrzKhcmyAeS+sC3TN0G2ty65e9MIjiFDiu3S6Q== X-Google-Smtp-Source: AKy350azd9Dp4NHI7TbhcoBN4L/9exe2J6CrW0GL3mlF0c7fSEtZKMpsf0EWacGZsjJNnHH0cPpLhg== X-Received: by 2002:a0d:d6c9:0:b0:54f:c657:d80c with SMTP id y192-20020a0dd6c9000000b0054fc657d80cmr1014605ywd.50.1681850444978; Tue, 18 Apr 2023 13:40:44 -0700 (PDT) Received: from localhost (104-178-186-189.lightspeed.milwwi.sbcglobal.net. [104.178.186.189]) by smtp.gmail.com with ESMTPSA id e6-20020a811e06000000b0054640a352f2sm4040547ywe.20.2023.04.18.13.40.44 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 18 Apr 2023 13:40:44 -0700 (PDT) Date: Tue, 18 Apr 2023 16:40:43 -0400 From: Taylor Blau To: git@vger.kernel.org Cc: Jeff King , Derrick Stolee , Junio C Hamano Subject: [PATCH v2 05/10] t/t6501-freshen-objects.sh: prepare for `gc --cruft` by default Message-ID: References: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org In a similar spirit as previous commits, prepare for `gc --cruft` becoming the default by ensuring that the tests in t6501 explicitly cover the case of freshening loose objects not using cruft packs. We could run this test twice, once with `--cruft` and once with `--no-cruft`, but doing so is unnecessary, since we already test object rescuing, freshening, and dealing with corrupt parts of the unreachable object graph extensively via t5329. Signed-off-by: Taylor Blau --- t/t6501-freshen-objects.sh | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-) diff --git a/t/t6501-freshen-objects.sh b/t/t6501-freshen-objects.sh index 3968b47ed5..dbfa8a4d4c 100755 --- a/t/t6501-freshen-objects.sh +++ b/t/t6501-freshen-objects.sh @@ -101,7 +101,7 @@ do ' test_expect_success "simultaneous gc ($title)" ' - git gc --prune=12.hours.ago + git gc --no-cruft --prune=12.hours.ago ' test_expect_success "finish writing out commit ($title)" ' @@ -131,7 +131,7 @@ do ' test_expect_success "simultaneous gc ($title)" ' - git gc --prune=12.hours.ago + git gc --no-cruft --prune=12.hours.ago ' # tree should have been refreshed by write-tree @@ -151,7 +151,7 @@ test_expect_success 'do not complain about existing broken links (commit)' ' some message EOF commit=$(git hash-object -t commit -w broken-commit) && - git gc -q 2>stderr && + git gc --no-cruft -q 2>stderr && verbose git cat-file -e $commit && test_must_be_empty stderr ' @@ -161,7 +161,7 @@ test_expect_success 'do not complain about existing broken links (tree)' ' 100644 blob $(test_oid 003) foo EOF tree=$(git mktree --missing stderr && + git gc --no-cruft -q 2>stderr && git cat-file -e $tree && test_must_be_empty stderr ' @@ -176,7 +176,7 @@ test_expect_success 'do not complain about existing broken links (tag)' ' this is a broken tag EOF tag=$(git hash-object -t tag -w broken-tag) && - git gc -q 2>stderr && + git gc --no-cruft -q 2>stderr && git cat-file -e $tag && test_must_be_empty stderr ' From patchwork Tue Apr 18 20:40:46 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Taylor Blau X-Patchwork-Id: 13216148 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id B783BC77B75 for ; Tue, 18 Apr 2023 20:41:02 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232660AbjDRUlB (ORCPT ); Tue, 18 Apr 2023 16:41:01 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:43650 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232654AbjDRUky (ORCPT ); Tue, 18 Apr 2023 16:40:54 -0400 Received: from mail-yw1-x1129.google.com (mail-yw1-x1129.google.com [IPv6:2607:f8b0:4864:20::1129]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 38332C670 for ; Tue, 18 Apr 2023 13:40:49 -0700 (PDT) Received: by mail-yw1-x1129.google.com with SMTP id 00721157ae682-54fe3cd445aso168683327b3.5 for ; Tue, 18 Apr 2023 13:40:49 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ttaylorr-com.20221208.gappssmtp.com; s=20221208; t=1681850448; x=1684442448; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:from:to:cc:subject:date:message-id:reply-to; bh=4bLMfk3306UpekY3xnL2cHG7+MAMdF8giKuJ6YI8Hac=; b=q0300kfTqO0FffcZMaJOj3/Hgv53VWkcXPx8mdxEAIerCB2ymaGHAZ1xBcbmnZGueo 5L3gWaQz9qpJqBLVmKtFZrmSRx1WXyWso7ZtbB5DyiMQnbfm9iQVR+VMEDcOScYNxK7b JoQA1OOD/TPN+Hf+1MKnswy2lRRELDxWdH5QK8gjnXN0Zzr5tqR6EgvuNysRKXU91mO5 1/uJUkZKKNvSaqjWZoUxMbjzhP8rPBr/ZJUpeAaFn/FcrY11NwpG1eHlZhzHPR0ANrrq FnSDXo7FnikT8/FgJGVYwxDV3K3ZuQEtIB8cssI494I4LXCN3ufVwCf/eNYGfUsMnSPg 5jAw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1681850448; x=1684442448; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:x-gm-message-state:from:to:cc:subject:date :message-id:reply-to; bh=4bLMfk3306UpekY3xnL2cHG7+MAMdF8giKuJ6YI8Hac=; b=HnEBbShz+op9jkcn8iRWEbgGntLCEfGF6Tu6dI8adY4LqRv2F2Cct/LQcxFv3T64Kl 8dYAA7e68HbqAh5Aaf7KY4qd+XhhzF6H81jre31IbvJdrAqeRoUoo+HA5TVqlmpXVitR +NiNMuDkN1QDD2kruOh+TeVBi7yFL+7E+HfEw0fY4XqRoHSiU0HpkZ85nfuViPgV0SnX OVBfkuYqjBoVtfCUdDOWwEYzGF53OQO6UbLZChDTLdQn7wH8kIjv8ihgr0qL2LXXTLC2 zUWk1zw2sJr1Oog+qGwPCB2JCaSH+e5bDj2BrxiHl8thyn1wB57GoJAwFMk703YT0zaA gA6Q== X-Gm-Message-State: AAQBX9eC6iY6cFV3rf5+9NOt1wDhhNMvVQpFtE5YiVofvjgttGiW/+Cg SAe+KMxu9/MYYPXixJt1IWRGAW2rQmCNK9ONQcTBMA== X-Google-Smtp-Source: AKy350bw57QekciFqcuOxoTJnh+FXGa7osp3amC75i6N3+QBmDOnCCJDMYr1hSXSBEPu6YTwIB6nOA== X-Received: by 2002:a0d:c102:0:b0:54f:d7f7:9d61 with SMTP id c2-20020a0dc102000000b0054fd7f79d61mr1275589ywd.14.1681850448166; Tue, 18 Apr 2023 13:40:48 -0700 (PDT) Received: from localhost (104-178-186-189.lightspeed.milwwi.sbcglobal.net. [104.178.186.189]) by smtp.gmail.com with ESMTPSA id v132-20020a81488a000000b0054fa99bed5asm4048696ywa.129.2023.04.18.13.40.47 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 18 Apr 2023 13:40:47 -0700 (PDT) Date: Tue, 18 Apr 2023 16:40:46 -0400 From: Taylor Blau To: git@vger.kernel.org Cc: Jeff King , Derrick Stolee , Junio C Hamano Subject: [PATCH v2 06/10] t/t6500-gc.sh: refactor cruft pack tests Message-ID: References: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org In 12253ab6d0 (gc: add tests for --cruft and friends, 2022-10-26), we added a handful of tests to t6500 to ensure that `git gc` respected the value of `--cruft` and `gc.cruftPacks`. Then, in c695592850 (config: let feature.experimental imply gc.cruftPacks=true, 2022-10-26), another set of similar tests was added to ensure that `feature.experimental` correctly implied enabling cruft pack generation (or not). These tests are similar and could be consolidated. Do so in this patch to prepare for expanding the set of command-line invocations that enable or disable writing cruft packs. This makes it possible to easily test more combinations of arguments without being overly repetitive. Signed-off-by: Taylor Blau --- t/t6500-gc.sh | 126 ++++++++++++++++++-------------------------------- 1 file changed, 44 insertions(+), 82 deletions(-) diff --git a/t/t6500-gc.sh b/t/t6500-gc.sh index df6f2e6e52..a2f988c5c2 100755 --- a/t/t6500-gc.sh +++ b/t/t6500-gc.sh @@ -210,93 +210,55 @@ prepare_cruft_history () { git reset HEAD^^ } -assert_cruft_packs () { - find .git/objects/pack -name "*.mtimes" >mtimes && - sed -e 's/\.mtimes$/\.pack/g' mtimes >packs && - - test_file_not_empty packs && - while read pack - do - test_path_is_file "$pack" || return 1 - done mtimes && test_must_be_empty mtimes } -test_expect_success 'gc --cruft generates a cruft pack' ' - test_when_finished "rm -fr crufts" && - git init crufts && - ( - cd crufts && - - prepare_cruft_history && - git gc --cruft && - assert_cruft_packs - ) -' - -test_expect_success 'gc.cruftPacks=true generates a cruft pack' ' - test_when_finished "rm -fr crufts" && - git init crufts && - ( - cd crufts && - - prepare_cruft_history && - git -c gc.cruftPacks=true gc && - assert_cruft_packs - ) -' - -test_expect_success 'feature.experimental=true generates a cruft pack' ' - git init crufts && - test_when_finished "rm -fr crufts" && - ( - cd crufts && - - prepare_cruft_history && - git -c feature.experimental=true gc && - assert_cruft_packs - ) -' - -test_expect_success 'feature.experimental=false allows explicit cruft packs' ' - git init crufts && - test_when_finished "rm -fr crufts" && - ( - cd crufts && - - prepare_cruft_history && - git -c gc.cruftPacks=true -c feature.experimental=false gc && - assert_cruft_packs - ) -' - -test_expect_success 'feature.experimental=true can be overridden' ' - git init crufts && - test_when_finished "rm -fr crufts" && - ( - cd crufts && - - prepare_cruft_history && - git -c feature.expiremental=true -c gc.cruftPacks=false gc && - assert_no_cruft_packs - ) -' - -test_expect_success 'feature.experimental=false avoids cruft packs by default' ' - git init crufts && - test_when_finished "rm -fr crufts" && - ( - cd crufts && - - prepare_cruft_history && - git -c feature.experimental=false gc && - assert_no_cruft_packs - ) -' +for argv in \ + "gc --cruft" \ + "-c gc.cruftPacks=true gc" \ + "-c feature.experimental=true gc" \ + "-c gc.cruftPacks=true -c feature.experimental=false gc" +do + test_expect_success "git $argv generates a cruft pack" ' + test_when_finished "rm -fr repo" && + git init repo && + ( + cd repo && + + prepare_cruft_history && + git $argv && + + find .git/objects/pack -name "*.mtimes" >mtimes && + sed -e 's/\.mtimes$/\.pack/g' mtimes >packs && + + test_file_not_empty packs && + while read pack + do + test_path_is_file "$pack" || return 1 + done X-Patchwork-Id: 13216149 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 9A60EC6FD18 for ; Tue, 18 Apr 2023 20:41:05 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232768AbjDRUlE (ORCPT ); Tue, 18 Apr 2023 16:41:04 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:43690 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232613AbjDRUkz (ORCPT ); Tue, 18 Apr 2023 16:40:55 -0400 Received: from mail-ua1-x92c.google.com (mail-ua1-x92c.google.com [IPv6:2607:f8b0:4864:20::92c]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 26288CC17 for ; Tue, 18 Apr 2023 13:40:52 -0700 (PDT) Received: by mail-ua1-x92c.google.com with SMTP id ch24so199007uab.3 for ; Tue, 18 Apr 2023 13:40:51 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ttaylorr-com.20221208.gappssmtp.com; s=20221208; t=1681850451; x=1684442451; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:from:to:cc:subject:date:message-id:reply-to; bh=uwLXTEsltaly6sNxrv22iYhcf/YerZr3+fP45JUpHhQ=; b=EtFNphYvH+L6+S/Z0BiXdDIT2dFBPDfOgwTgY2HbD8vHu3Q1lohifR/MzNGIFW9W3Y zRY2+3Jvo96kiwY8j4i5W+FPBOqf8aXbCX8HR6J3HaM3INmfLYt34+oyzDBY3bDmxvip 5rMyaBqz3gN9Qel6M1enp6U8p7zREnl6DlDUGog/Z0eMbUfsW3V2ZEfLXuU4fsTKJPJo ouWx5k37u3WnW2iXXX6Vciu83by2/m2uW3M8n8TDTE1hywTK/HHfLfA+2GY9R9v8jHpK e+/E6CuAjLJOpBa3HaZfY///1/ZEAngdxFpwFXaHR9dik0TxJuExxMIdipsxgfaavYHP sWSw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1681850451; x=1684442451; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:x-gm-message-state:from:to:cc:subject:date :message-id:reply-to; bh=uwLXTEsltaly6sNxrv22iYhcf/YerZr3+fP45JUpHhQ=; b=IFm8lOtEAOXeKmiRmldfVXX7r90qfxUlO65sa1pRY4o4SW0skKbqSqJPU7qIERhWc4 J911zx0TOyVgKDovQ+3zQbm6x2zAMAOdqWOIvyws+kesZhBoEoktTac5vd2nLkVgTiA0 8vP7BHnlRN1qTdU2FP0kXWzpVAF/e2psfr+RQXWs5Mw+cMDepGTuqVagbawl8qJ6W312 CKk1D6Sjcs8N2RRR+HPl4EsOr5Lb+LxKFqp0K+o9fVzCgKg0E4AcaXOfJstaC1bYTCBk K6u2P7+ETTO1VwkZlc6slNCJ3xDnOyms3lQPhMwxBgRW3y67/Xj+z0rsPnQs+2ACnMdc FXwQ== X-Gm-Message-State: AAQBX9fLOuzeSkeaOKBM8ZnJL8q00x/Ykzcw+qyThnNp3tdebdwbI3U0 qZIGvi01PCJvICmwPH81L0wXfQgyxljGzLdamfanRw== X-Google-Smtp-Source: AKy350YSEkxVrir6awQZH4jVjn3OQlyweUufGTLans6BV79GZM/ojE14YHwDw20WV0eOW658P4Wg1A== X-Received: by 2002:a1f:5c58:0:b0:440:499:7747 with SMTP id q85-20020a1f5c58000000b0044004997747mr5971620vkb.10.1681850450909; Tue, 18 Apr 2023 13:40:50 -0700 (PDT) Received: from localhost (104-178-186-189.lightspeed.milwwi.sbcglobal.net. [104.178.186.189]) by smtp.gmail.com with ESMTPSA id df5-20020a05690c0f8500b0054f9a3b60a1sm3281018ywb.33.2023.04.18.13.40.50 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 18 Apr 2023 13:40:50 -0700 (PDT) Date: Tue, 18 Apr 2023 16:40:49 -0400 From: Taylor Blau To: git@vger.kernel.org Cc: Jeff King , Derrick Stolee , Junio C Hamano Subject: [PATCH v2 07/10] t/t6500-gc.sh: add additional test cases Message-ID: <9db3fa9e36dce3fc7819f7c93e2af7ac6afcb326.1681850424.git.me@ttaylorr.com> References: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org In the last commit, we refactored some of the tests in t6500 to make clearer when cruft packs will and won't be generated by `git gc`. Add the remaining cases not covered by the previous patch into this one, which enumerates all possible combinations of arguments that will produce (or not produce) a cruft pack. This prepares us for a future commit which will change the default value of `gc.cruftPacks` by ensuring that we understand which invocations do and do not change as a result. Signed-off-by: Taylor Blau --- t/t6500-gc.sh | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/t/t6500-gc.sh b/t/t6500-gc.sh index a2f988c5c2..3ba2ae5140 100755 --- a/t/t6500-gc.sh +++ b/t/t6500-gc.sh @@ -218,6 +218,7 @@ assert_no_cruft_packs () { for argv in \ "gc --cruft" \ "-c gc.cruftPacks=true gc" \ + "-c gc.cruftPacks=false gc --cruft" \ "-c feature.experimental=true gc" \ "-c gc.cruftPacks=true -c feature.experimental=false gc" do @@ -243,6 +244,9 @@ do done for argv in \ + "gc" \ + "-c gc.cruftPacks=false gc" \ + "-c gc.cruftPacks=true gc --no-cruft" \ "-c feature.expiremental=true -c gc.cruftPacks=false gc" \ "-c feature.experimental=false gc" do From patchwork Tue Apr 18 20:40:52 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Taylor Blau X-Patchwork-Id: 13216150 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id EC787C77B75 for ; Tue, 18 Apr 2023 20:41:14 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232793AbjDRUlN (ORCPT ); Tue, 18 Apr 2023 16:41:13 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:44000 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231510AbjDRUlA (ORCPT ); Tue, 18 Apr 2023 16:41:00 -0400 Received: from mail-yw1-x1135.google.com (mail-yw1-x1135.google.com [IPv6:2607:f8b0:4864:20::1135]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id D7CDDB77D for ; Tue, 18 Apr 2023 13:40:56 -0700 (PDT) Received: by mail-yw1-x1135.google.com with SMTP id 00721157ae682-555bc7f6746so29773397b3.6 for ; Tue, 18 Apr 2023 13:40:56 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ttaylorr-com.20221208.gappssmtp.com; s=20221208; t=1681850455; x=1684442455; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:from:to:cc:subject:date:message-id:reply-to; bh=Znitps7bkYZ+IufIXRXGt7XWOPXXw9/MsW2Wk7xDbyo=; b=GszwfOrmVveNexlGOMPIqP8DWYmuDchKIPinxZg0HP3APSHCf7N7+6p7Niz/UtAhS2 YwIvxTdQC5rRrn9BtcXuQ2jk5QVQJ+oJVypX160SuAD8/ZiuaciWIsMZqJW7tbJ+K+cM A7rtqMzXLdemGUoQX+k78pZeyZ+YQlQ9wNCVMJiRQdg012ahBMXXDrwwDOly5a3Cwj8w j3sLc8HimN+wX6QW5ffLGSM96CALpoQx1LYv+RkIpz6l9+iog1L9ANBy3ADhbK+7arKs w/CKgRyeMiyUgGOdti7IMWXtEMEStiaZVnWOKPEQ/tqzJFUOu6aLwNxjXF2O65f5IyEW 0iaQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1681850455; x=1684442455; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:x-gm-message-state:from:to:cc:subject:date :message-id:reply-to; bh=Znitps7bkYZ+IufIXRXGt7XWOPXXw9/MsW2Wk7xDbyo=; b=FBHTrKvUzIJjnmoP66CIocZT8Ovmme9WX/SRjxce0aAVa5Ju1VuWtxcyuuGuWzF0IY b/Xx44PpbJbB5ZI2whDDKnvjz6fvdcia8SMKytRPgUzdiWfrda4EaE63KN6sdaIhvqgF qbJxVEEdyowApKdomJq8AGQcarKaFc59j6AStn82x+gpcggcHIQvulFIUUs3an9HBqTM nQQ0vthE98sjCNF1m7WXJLK3UphBgsKttQwfuspi7Jb0KKsbqH8y8ooLHFDp/ZzjAmmR PcJzbvdvz1vMRCIXLD0j0ItzKCGAWUkGYlntrPqDRW4bZBgCPhHOATnzqZpdSgDJWO6b hMGw== X-Gm-Message-State: AAQBX9dp6QU2ggzM6uJjrT1pOqOcp556qoJN2V5Gltkm3WacYNd2Bv1g X5/rmQrUafzYcvzwIfpXZVWXbNVYDXyjfMxorVSRXg== X-Google-Smtp-Source: AKy350ac6X8La5pjmCvqXCI6VjpsBlCd+PCoinaR9QBf9jhfLWoRa8P2Dj7k0WgYhE5Uy3pOFeNyOQ== X-Received: by 2002:a0d:d203:0:b0:54f:cf3a:2604 with SMTP id u3-20020a0dd203000000b0054fcf3a2604mr1056835ywd.41.1681850455653; Tue, 18 Apr 2023 13:40:55 -0700 (PDT) Received: from localhost (104-178-186-189.lightspeed.milwwi.sbcglobal.net. [104.178.186.189]) by smtp.gmail.com with ESMTPSA id t22-20020a817816000000b00545a0818494sm4100020ywc.36.2023.04.18.13.40.53 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 18 Apr 2023 13:40:53 -0700 (PDT) Date: Tue, 18 Apr 2023 16:40:52 -0400 From: Taylor Blau To: git@vger.kernel.org Cc: Jeff King , Derrick Stolee , Junio C Hamano Subject: [PATCH v2 08/10] t/t9300-fast-import.sh: prepare for `gc --cruft` by default Message-ID: <894cf176ea8911e615cb59857b93a2765ca6ba9d.1681850424.git.me@ttaylorr.com> References: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org In a similar fashion as previous commits, adjust the fast-import tests to prepare for "git gc" generating a cruft pack by default. This adjustment is slightly different, however. Instead of relying on us writing out the objects loose, and then calling `git prune` to remove them, t9300 needs to be prepared to drop objects that would be moved into cruft packs. To do this, we can combine the `git gc` invocation with `git prune` into one `git gc --prune`, which handles pruning both loose objects, and objects that would otherwise be written to a cruft pack. Likely this pattern of "git gc && git prune" started all the way back in 03db4525d3 (Support gitlinks in fast-import., 2008-07-19), which happened after deprecating `git gc --prune` in 9e7d501990 (builtin-gc.c: deprecate --prune, it now really has no effect, 2008-05-09). After `--prune` was un-deprecated in 58e9d9d472 (gc: make --prune useful again by accepting an optional parameter, 2009-02-14), this script got a handful of new "git gc && git prune" instances via via 4cedb78cb5 (fast-import: add input format tests, 2011-08-11). These could have been `git gc --prune`, but weren't (likely taking after 03db4525d3). Signed-off-by: Taylor Blau --- t/t9300-fast-import.sh | 13 ++++--------- 1 file changed, 4 insertions(+), 9 deletions(-) diff --git a/t/t9300-fast-import.sh b/t/t9300-fast-import.sh index aa55b41b9a..ac237a1f90 100755 --- a/t/t9300-fast-import.sh +++ b/t/t9300-fast-import.sh @@ -388,9 +388,7 @@ test_expect_success 'B: accept branch name "TEMP_TAG"' ' INPUT_END - test_when_finished "rm -f .git/TEMP_TAG - git gc - git prune" && + test_when_finished "rm -f .git/TEMP_TAG && git gc --prune=now" && git fast-import X-Patchwork-Id: 13216151 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 7FEC9C6FD18 for ; Tue, 18 Apr 2023 20:41:17 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232795AbjDRUlQ (ORCPT ); Tue, 18 Apr 2023 16:41:16 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:44188 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232699AbjDRUlC (ORCPT ); Tue, 18 Apr 2023 16:41:02 -0400 Received: from mail-yb1-xb2e.google.com (mail-yb1-xb2e.google.com [IPv6:2607:f8b0:4864:20::b2e]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 80202CC3C for ; Tue, 18 Apr 2023 13:40:59 -0700 (PDT) Received: by mail-yb1-xb2e.google.com with SMTP id b10so2497489ybk.9 for ; Tue, 18 Apr 2023 13:40:59 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ttaylorr-com.20221208.gappssmtp.com; s=20221208; t=1681850458; x=1684442458; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:from:to:cc:subject:date:message-id:reply-to; bh=fP+L07cwQTzkf6Cs7rGppdhtqqim7Hezjnkkp2v4+CA=; b=WPdQ2d0+iwlHKbynB/dLvAM/8jHbuU214DQZY4RBCZocNHiIkTaUVMnCUU4h9sscCs fpJr/wPobYgQu4hY3eakIjuGOWmk+yFngwWSz/WeCTjVsBmBgzSqb8M8PfVcwH/QCw5H VCVTICJZzrUL/bNgh91K29N3dLD+Q5UY3+sFS86kEJjdB/NFxo6SF792LN62XdNpYfBO i2vBTGtb1CAXlEQrXtCDyLtHrjL9u3faWTTvp54GdiyapZqKRwyaLy9ev/7exzxhzijl 4BNhwIOXFpitkfneGwgcEGlUjxAmHvf7TMHBcfFlVD7b42AJys3h4lA41vt/Jsko2it6 xmjw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1681850458; x=1684442458; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:x-gm-message-state:from:to:cc:subject:date :message-id:reply-to; bh=fP+L07cwQTzkf6Cs7rGppdhtqqim7Hezjnkkp2v4+CA=; b=UMFgYsskgFsa13EL899eT3yRwtYCGelFlnjThZucJI347oUfr7DUJqPcmw7cMhpKee iiywZpFseO1z010mDHIy8V8UtbsjQu1Wm4OJnT3f9nfK5Picp4qmUmYkWtf4ij2llSQC zCdT9ru9KSjCgxWMNtUsRQKSJIFgGs6C++amnfAglg3R+y+X7tw3i8q6yt/wW6hid+Dl KLC6Gl7QR57SKSXZOvJnOvp8wbNoYHvntqwzdoP6jva3crn36O8kjXtv9+6PL65335zI cSi+C588sP8JsV2svy0GxWLb1zeMbL/W3A+qIdG2U63Axz87SSQ8ICLrfBkLfHi5qW4n JL7g== X-Gm-Message-State: AAQBX9f8HuS1hOMGNYQlmjAAMzpRRBavrrcNkNDxf2d06yvwbueqqfSD tnbeffLliwS+4aSF3hQF91spNgenW9hyGi6F7mAs2A== X-Google-Smtp-Source: AKy350YW7UNabT6OKkxJZ+NYWmbGurTFwC+/JTaF7vxLBwMAz7IJaMTmhYfXfUCppokjKX7BufjWog== X-Received: by 2002:a25:2407:0:b0:b95:ded1:7d99 with SMTP id k7-20020a252407000000b00b95ded17d99mr2258589ybk.27.1681850458506; Tue, 18 Apr 2023 13:40:58 -0700 (PDT) Received: from localhost (104-178-186-189.lightspeed.milwwi.sbcglobal.net. [104.178.186.189]) by smtp.gmail.com with ESMTPSA id b14-20020a252e4e000000b00b8f6ec5a955sm3198301ybn.49.2023.04.18.13.40.58 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 18 Apr 2023 13:40:58 -0700 (PDT) Date: Tue, 18 Apr 2023 16:40:57 -0400 From: Taylor Blau To: git@vger.kernel.org Cc: Jeff King , Derrick Stolee , Junio C Hamano Subject: [PATCH v2 09/10] builtin/gc.c: make `gc.cruftPacks` enabled by default Message-ID: References: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org Back in 5b92477f89 (builtin/gc.c: conditionally avoid pruning objects via loose, 2022-05-20), `git gc` learned the `--cruft` option and `gc.cruftPacks` configuration to opt-in to writing cruft packs when collecting or pruning unreachable objects. Cruft packs were introduced with the merge in a50036da1a (Merge branch 'tb/cruft-packs', 2022-06-03). They address the problem of "loose object explosions", where Git will write out many individual loose objects when there is a large number of unreachable objects that have not yet aged past `--prune=`. Instead of keeping track of those unreachable yet recent objects via their loose object file's mtime, cruft packs collect all unreachable objects into a single pack with a corresponding `*.mtimes` file that acts as a table to store the mtimes of all unreachable objects. This prevents the need to store unreachable objects as loose as they age out of the repository, and avoids the problem of loose object explosions. Beyond avoiding loose object explosions, cruft packs also act as a more efficient mechanism to store unreachable objects as they age out of a repository. This is because pairs of similar unreachable objects serve as delta bases for one another. In 5b92477f89, the feature was introduced as experimental. Since then, GitHub has been running these patches in every repository generating hundreds of millions of cruft packs along the way. The feature is battle-tested, and avoids many pathological cases such as above. Users who either run `git gc` manually, or via `git maintenance` can benefit from having cruft packs. As such, enable cruft pack generation to take place by default (by making `gc.cruftPacks` have the default of "true" rather than "false). Signed-off-by: Taylor Blau --- Documentation/config/feature.txt | 3 --- Documentation/config/gc.txt | 2 +- Documentation/git-gc.txt | 5 +++-- Documentation/gitformat-pack.txt | 4 ++-- builtin/gc.c | 6 +----- t/t6500-gc.sh | 12 ++++-------- 6 files changed, 11 insertions(+), 21 deletions(-) diff --git a/Documentation/config/feature.txt b/Documentation/config/feature.txt index e52bc6b858..17b4d39f89 100644 --- a/Documentation/config/feature.txt +++ b/Documentation/config/feature.txt @@ -14,9 +14,6 @@ feature.experimental:: + * `fetch.negotiationAlgorithm=skipping` may improve fetch negotiation times by skipping more commits at a time, reducing the number of round trips. -+ -* `gc.cruftPacks=true` reduces disk space used by unreachable objects during -garbage collection, preventing loose object explosions. feature.manyFiles:: Enable config options that optimize for repos with many files in the diff --git a/Documentation/config/gc.txt b/Documentation/config/gc.txt index 8d5353e9e0..7f95c866e1 100644 --- a/Documentation/config/gc.txt +++ b/Documentation/config/gc.txt @@ -84,7 +84,7 @@ gc.packRefs:: gc.cruftPacks:: Store unreachable objects in a cruft pack (see linkgit:git-repack[1]) instead of as loose objects. The default - is `false`. + is `true`. gc.pruneExpire:: When 'git gc' is run, it will call 'prune --expire 2.weeks.ago' diff --git a/Documentation/git-gc.txt b/Documentation/git-gc.txt index fef382a70f..90806fd26a 100644 --- a/Documentation/git-gc.txt +++ b/Documentation/git-gc.txt @@ -54,9 +54,10 @@ other housekeeping tasks (e.g. rerere, working trees, reflog...) will be performed as well. ---cruft:: +--[no-]cruft:: When expiring unreachable objects, pack them separately into a - cruft pack instead of storing them as loose objects. + cruft pack instead of storing them as loose objects. `--cruft` + is on by default. --prune=:: Prune loose objects older than date (default is 2 weeks ago, diff --git a/Documentation/gitformat-pack.txt b/Documentation/gitformat-pack.txt index e06af02f21..0c1be2dbe8 100644 --- a/Documentation/gitformat-pack.txt +++ b/Documentation/gitformat-pack.txt @@ -611,8 +611,8 @@ result of repeatedly resetting the objects' mtimes to the present time. If you are GC-ing repositories in a mixed version environment, consider omitting the `--cruft` option when using linkgit:git-repack[1] and linkgit:git-gc[1], and -leaving the `gc.cruftPacks` configuration unset until all writers understand -cruft packs. +setting the `gc.cruftPacks` configuration to "false" until all writers +understand cruft packs. === Alternatives diff --git a/builtin/gc.c b/builtin/gc.c index 53ef137e1d..ece01e966f 100644 --- a/builtin/gc.c +++ b/builtin/gc.c @@ -48,7 +48,7 @@ static const char * const builtin_gc_usage[] = { static int pack_refs = 1; static int prune_reflogs = 1; -static int cruft_packs = -1; +static int cruft_packs = 1; static int aggressive_depth = 50; static int aggressive_window = 250; static int gc_auto_threshold = 6700; @@ -608,10 +608,6 @@ int cmd_gc(int argc, const char **argv, const char *prefix) if (prune_expire && parse_expiry_date(prune_expire, &dummy)) die(_("failed to parse prune expiry value %s"), prune_expire); - prepare_repo_settings(the_repository); - if (cruft_packs < 0) - cruft_packs = the_repository->settings.gc_cruft_packs; - if (aggressive) { strvec_push(&repack, "-f"); if (aggressive_depth > 0) diff --git a/t/t6500-gc.sh b/t/t6500-gc.sh index 3ba2ae5140..69509d0c11 100755 --- a/t/t6500-gc.sh +++ b/t/t6500-gc.sh @@ -216,11 +216,9 @@ assert_no_cruft_packs () { } for argv in \ - "gc --cruft" \ + "gc" \ "-c gc.cruftPacks=true gc" \ - "-c gc.cruftPacks=false gc --cruft" \ - "-c feature.experimental=true gc" \ - "-c gc.cruftPacks=true -c feature.experimental=false gc" + "-c gc.cruftPacks=false gc --cruft" do test_expect_success "git $argv generates a cruft pack" ' test_when_finished "rm -fr repo" && @@ -244,11 +242,9 @@ do done for argv in \ - "gc" \ + "gc --no-cruft" \ "-c gc.cruftPacks=false gc" \ - "-c gc.cruftPacks=true gc --no-cruft" \ - "-c feature.expiremental=true -c gc.cruftPacks=false gc" \ - "-c feature.experimental=false gc" + "-c gc.cruftPacks=true gc --no-cruft" do test_expect_success "git $argv does not generate a cruft pack" ' test_when_finished "rm -fr repo" && From patchwork Tue Apr 18 20:41:00 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Taylor Blau X-Patchwork-Id: 13216152 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 2F762C77B75 for ; Tue, 18 Apr 2023 20:41:24 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232805AbjDRUlW (ORCPT ); Tue, 18 Apr 2023 16:41:22 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:44260 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232764AbjDRUlE (ORCPT ); Tue, 18 Apr 2023 16:41:04 -0400 Received: from mail-yw1-x1132.google.com (mail-yw1-x1132.google.com [IPv6:2607:f8b0:4864:20::1132]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 15B73CC06 for ; Tue, 18 Apr 2023 13:41:02 -0700 (PDT) Received: by mail-yw1-x1132.google.com with SMTP id 00721157ae682-54ee0b73e08so476902277b3.0 for ; Tue, 18 Apr 2023 13:41:02 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ttaylorr-com.20221208.gappssmtp.com; s=20221208; t=1681850461; x=1684442461; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:from:to:cc:subject:date:message-id:reply-to; bh=Yx+EmS5WT/vkIpWsHvjXnxBRriyE44ZY5B+ZfvWZpSo=; b=P7hTM9Xd9ducdpqRT2TP0N1d6lXvvfHxKMEkJc9z7Hs+SDvXevqhLI5q45uBo333N6 HZRnJ2YrHIzcDyyTR9hqEtXStmXEbPJ8bfkWeATqCBq8ad2rWqVyltYVGfoCvRzqZmba BY8Mxzn1nbpUxmafzkRhlF8aphQXfpDScF85WyiLqIXlisUtjr61sf6toD2DxXBq/eZT LFRTgnqux8Y7AjbF/myr+kHQdfWmwujbovYnI4/Ch7RsHw0qbw0evcWLyf5fckRmQHnP uhW2v6QnsmME6WxIkYveascqlu59iSA6VfoH0J7tCVJ3/3cKalmMxgf0ZQxFdWYJoq7o QRlg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1681850461; x=1684442461; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:x-gm-message-state:from:to:cc:subject:date :message-id:reply-to; bh=Yx+EmS5WT/vkIpWsHvjXnxBRriyE44ZY5B+ZfvWZpSo=; b=NxDHEPklXUfKDCwAhHld+3PlRBB+9kHETQxXqXFqEwxPQg1Xue5Aaz5IoTZc4voqlu GbpzMBFqLvdxcGyo5cNBaO+NW4G9QIZFEyg81j7k564Ak4HQ5hzgTXWRGc0amfM4hyIw 6pH3IN8W8/y6rDqIbIJG0FPKPATCIhatlwwFZo4wNCvvH/MaHVFK5e0LusEX8Q375vag vX2OwR3P5hDkkPHMwwIXV2N7CoI34oANelb92rV4NfkUVhebi3XfrRYOHGBnSlb2X0Vd ttBFxtAkz2TNCaTtsg10FnUKSWiMJwRaF7mM+zSSQbAbNqUga7cNIwooJuA493aJR6fr Zujw== X-Gm-Message-State: AAQBX9fGMe3HsVvME9EZUBV1FrVvNIi6AXwc1ucp+uwuqL1edbCJrTz7 2swNIayByFb9cn+p3KxOxy1fYQNDBuu0hb2eeX/8vw== X-Google-Smtp-Source: AKy350Z1X16fY4STaNXEJiodL/pa4FAGjTctSMoc/9lMblg4KRM0IGDX5kVV3xDhVog2SDVZY8A2Xw== X-Received: by 2002:a81:4e05:0:b0:54f:64d8:9c9 with SMTP id c5-20020a814e05000000b0054f64d809c9mr36536ywb.21.1681850461487; Tue, 18 Apr 2023 13:41:01 -0700 (PDT) Received: from localhost (104-178-186-189.lightspeed.milwwi.sbcglobal.net. [104.178.186.189]) by smtp.gmail.com with ESMTPSA id db15-20020a05690c0dcf00b00545a08184b8sm4042087ywb.72.2023.04.18.13.41.01 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 18 Apr 2023 13:41:01 -0700 (PDT) Date: Tue, 18 Apr 2023 16:41:00 -0400 From: Taylor Blau To: git@vger.kernel.org Cc: Jeff King , Derrick Stolee , Junio C Hamano Subject: [PATCH v2 10/10] repository.h: drop unused `gc_cruft_packs` Message-ID: References: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org As of the previous commit, all callers that need to read the value of `gc.cruftPacks` do so outside without using the `repo_settings` struct, making its `gc_cruft_packs` unused. Drop it accordingly. Signed-off-by: Taylor Blau --- repo-settings.c | 4 +--- repository.h | 1 - 2 files changed, 1 insertion(+), 4 deletions(-) diff --git a/repo-settings.c b/repo-settings.c index 0a6c0b381f..a8d0b98794 100644 --- a/repo-settings.c +++ b/repo-settings.c @@ -41,10 +41,8 @@ void prepare_repo_settings(struct repository *r) repo_cfg_bool(r, "feature.experimental", &experimental, 0); /* Defaults modified by feature.* */ - if (experimental) { + if (experimental) r->settings.fetch_negotiation_algorithm = FETCH_NEGOTIATION_SKIPPING; - r->settings.gc_cruft_packs = 1; - } if (manyfiles) { r->settings.index_version = 4; r->settings.index_skip_hash = 1; diff --git a/repository.h b/repository.h index 15a8afc5fb..50eb0ce391 100644 --- a/repository.h +++ b/repository.h @@ -33,7 +33,6 @@ struct repo_settings { int commit_graph_generation_version; int commit_graph_read_changed_paths; int gc_write_commit_graph; - int gc_cruft_packs; int fetch_write_commit_graph; int command_requires_full_index; int sparse_index;