From patchwork Tue Sep 5 20:36:40 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Taylor Blau X-Patchwork-Id: 13375049 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id E00B7CA100D for ; Tue, 5 Sep 2023 20:36:58 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S243821AbjIEUgx (ORCPT ); Tue, 5 Sep 2023 16:36:53 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:60238 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S243807AbjIEUgv (ORCPT ); Tue, 5 Sep 2023 16:36:51 -0400 Received: from mail-yw1-x1135.google.com (mail-yw1-x1135.google.com [IPv6:2607:f8b0:4864:20::1135]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id AAEBAD1 for ; Tue, 5 Sep 2023 13:36:42 -0700 (PDT) Received: by mail-yw1-x1135.google.com with SMTP id 00721157ae682-58caaedb20bso30072567b3.1 for ; Tue, 05 Sep 2023 13:36:42 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ttaylorr-com.20230601.gappssmtp.com; s=20230601; t=1693946201; x=1694551001; darn=vger.kernel.org; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:from:to:cc:subject:date:message-id:reply-to; bh=HLVY8doTHJIbak1Hn85Z1exI/3nf4NSHqrRG9Jm/GX8=; b=jTSxFdjEHJbti8S8JYFcxu6oUNdP13G0DNXs6egme9rRiXyNy/Rc7PRuKh2O5eTWpL UpLjaRkqMVuB3RJZLesrYlclRM2RUXujc96NB7zu+7FVc+8KQzxMbZFF7OaPEG0gQcsp i0EXKbmclJkAWGkxmNuxmG15azMq+i5iQG2bY/O0gBLv2B1DKpWDxsADAPFyRd0PTWce r4Lgd9tL/yOPl2fp73ypLTQjzakZ4HlLNTMzBNDQs00HuXU5xzqlJEa6oG8Tmjv6qj51 WbXQEgxNrf53BgAWclnICVuL+uozKCOvjhNNjTUOc5YBgAtI922fPFgSM5QqfYMo6nfb 4Zpg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1693946201; x=1694551001; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:x-gm-message-state:from:to:cc:subject:date :message-id:reply-to; bh=HLVY8doTHJIbak1Hn85Z1exI/3nf4NSHqrRG9Jm/GX8=; b=j7ZmAeultdpAEso72K7qAm9CufX9o4uo6KoIoZW3KefLlWHc3S8Jsl2x3ZfaE7TdzM v37bFywRQYANfmrOzCDv24hZtQgSFNSSVDfkHTNWHHk3usKRjr26OWEkDbtPBpQKGXqy kep4teNmGAPDzcBPpqEOdbITOvNxEaCt5/yf5E1yGB/H+NC7UXxnU2EoZHfhR5Gvse+Y lOJa7vYHjbGC8PZe2kVXDtzLpGCP/VEbph6scT0DqdUfhXhQBqdYJe7ZSsJRXvtIzLvI hOP7jY8GZbe50tCmzdjCvvTiOl+hZKlDmUneFm9vtO1LutnJsa03PAi1BOvtJCvrEu3w 0Ssg== X-Gm-Message-State: AOJu0YwRGYmik5f+kcai5W7p4D6dB7Mhjhy/pUDqawjJwJ2Zbo0puXNv iM+N/is57ssAbO4CFY7yutz5hHOng71fjhd7cU5iRQ== X-Google-Smtp-Source: AGHT+IGRku6H3obZKmLozXqHxC8FvNhZo25kBx0/yKZxjBd8O+s/NoYWqXNuxtiqBWG6dzm7jnlrrQ== X-Received: by 2002:a0d:d981:0:b0:589:f7ad:7702 with SMTP id b123-20020a0dd981000000b00589f7ad7702mr15497104ywe.36.1693946201657; Tue, 05 Sep 2023 13:36:41 -0700 (PDT) Received: from localhost (104-178-186-189.lightspeed.milwwi.sbcglobal.net. [104.178.186.189]) by smtp.gmail.com with ESMTPSA id cs3-20020a05690c0ec300b0059a34cfa2a5sm1838863ywb.67.2023.09.05.13.36.41 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 05 Sep 2023 13:36:41 -0700 (PDT) Date: Tue, 5 Sep 2023 16:36:40 -0400 From: Taylor Blau To: git@vger.kernel.org Cc: Junio C Hamano , Jeff King , Patrick Steinhardt Subject: [PATCH 1/8] builtin/repack.c: extract structure to store existing packs Message-ID: <5b48b7e3cc03c83465a3dcecaa98b9d2e9667084.1693946195.git.me@ttaylorr.com> References: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org The repack machinery needs to keep track of which packfiles were present in the repository at the beginning of a repack, segmented by whether or not each pack is marked as kept. The names of these packs are stored in two `string_list`s, corresponding to kept- and non-kept packs, respectively. As a consequence, many functions within the repack code need to take both `string_list`s as arguments, leading to code like this: ret = write_cruft_pack(&cruft_po_args, packtmp, pack_prefix, cruft_expiration, &names, &existing_nonkept_packs, /* <- */ &existing_kept_packs); /* <- */ Wrap up this pair of `string_list`s into a single structure that stores both. This saves us from having to pass both string lists separately, and prepares for adding additional fields to this structure. Signed-off-by: Taylor Blau --- builtin/repack.c | 90 ++++++++++++++++++++++++++---------------------- 1 file changed, 49 insertions(+), 41 deletions(-) diff --git a/builtin/repack.c b/builtin/repack.c index 2b43a5be08..c3ab89912e 100644 --- a/builtin/repack.c +++ b/builtin/repack.c @@ -95,14 +95,29 @@ static int repack_config(const char *var, const char *value, return git_default_config(var, value, ctx, cb); } +struct existing_packs { + struct string_list kept_packs; + struct string_list non_kept_packs; +}; + +#define EXISTING_PACKS_INIT { \ + .kept_packs = STRING_LIST_INIT_DUP, \ + .non_kept_packs = STRING_LIST_INIT_DUP, \ +} + +static void existing_packs_release(struct existing_packs *existing) +{ + string_list_clear(&existing->kept_packs, 0); + string_list_clear(&existing->non_kept_packs, 0); +} + /* - * Adds all packs hex strings (pack-$HASH) to either fname_nonkept_list - * or fname_kept_list based on whether each pack has a corresponding + * Adds all packs hex strings (pack-$HASH) to either packs->non_kept + * or packs->kept based on whether each pack has a corresponding * .keep file or not. Packs without a .keep file are not to be kept * if we are going to pack everything into one file. */ -static void collect_pack_filenames(struct string_list *fname_nonkept_list, - struct string_list *fname_kept_list, +static void collect_pack_filenames(struct existing_packs *existing, const struct string_list *extra_keep) { struct packed_git *p; @@ -126,16 +141,16 @@ static void collect_pack_filenames(struct string_list *fname_nonkept_list, strbuf_strip_suffix(&buf, ".pack"); if ((extra_keep->nr > 0 && i < extra_keep->nr) || p->pack_keep) - string_list_append(fname_kept_list, buf.buf); + string_list_append(&existing->kept_packs, buf.buf); else { struct string_list_item *item; - item = string_list_append(fname_nonkept_list, buf.buf); + item = string_list_append(&existing->non_kept_packs, buf.buf); if (p->is_cruft) item->util = (void*)(uintptr_t)CRUFT_PACK; } } - string_list_sort(fname_kept_list); + string_list_sort(&existing->kept_packs); strbuf_release(&buf); } @@ -327,7 +342,7 @@ static int geometry_cmp(const void *va, const void *vb) } static void init_pack_geometry(struct pack_geometry *geometry, - struct string_list *existing_kept_packs, + struct existing_packs *existing, const struct pack_objects_args *args) { struct packed_git *p; @@ -344,23 +359,24 @@ static void init_pack_geometry(struct pack_geometry *geometry, if (!pack_kept_objects) { /* - * Any pack that has its pack_keep bit set will appear - * in existing_kept_packs below, but this saves us from - * doing a more expensive check. + * Any pack that has its pack_keep bit set will + * appear in existing->kept_packs below, but + * this saves us from doing a more expensive + * check. */ if (p->pack_keep) continue; /* - * The pack may be kept via the --keep-pack option; - * check 'existing_kept_packs' to determine whether to - * ignore it. + * The pack may be kept via the --keep-pack + * option; check 'existing->kept_packs' to + * determine whether to ignore it. */ strbuf_reset(&buf); strbuf_addstr(&buf, pack_basename(p)); strbuf_strip_suffix(&buf, ".pack"); - if (string_list_has_string(existing_kept_packs, buf.buf)) + if (string_list_has_string(&existing->kept_packs, buf.buf)) continue; } if (p->is_cruft) @@ -565,14 +581,13 @@ static void midx_snapshot_refs(struct tempfile *f) } static void midx_included_packs(struct string_list *include, - struct string_list *existing_nonkept_packs, - struct string_list *existing_kept_packs, + struct existing_packs *existing, struct string_list *names, struct pack_geometry *geometry) { struct string_list_item *item; - for_each_string_list_item(item, existing_kept_packs) + for_each_string_list_item(item, &existing->kept_packs) string_list_insert(include, xstrfmt("%s.idx", item->string)); for_each_string_list_item(item, names) string_list_insert(include, xstrfmt("pack-%s.idx", item->string)); @@ -600,7 +615,7 @@ static void midx_included_packs(struct string_list *include, string_list_insert(include, strbuf_detach(&buf, NULL)); } - for_each_string_list_item(item, existing_nonkept_packs) { + for_each_string_list_item(item, &existing->non_kept_packs) { if (!((uintptr_t)item->util & CRUFT_PACK)) { /* * no need to check DELETE_PACK, since we're not @@ -611,7 +626,7 @@ static void midx_included_packs(struct string_list *include, string_list_insert(include, xstrfmt("%s.idx", item->string)); } } else { - for_each_string_list_item(item, existing_nonkept_packs) { + for_each_string_list_item(item, &existing->non_kept_packs) { if ((uintptr_t)item->util & DELETE_PACK) continue; string_list_insert(include, xstrfmt("%s.idx", item->string)); @@ -700,8 +715,7 @@ static int write_cruft_pack(const struct pack_objects_args *args, const char *pack_prefix, const char *cruft_expiration, struct string_list *names, - struct string_list *existing_packs, - struct string_list *existing_kept_packs) + struct existing_packs *existing) { struct child_process cmd = CHILD_PROCESS_INIT; struct strbuf line = STRBUF_INIT; @@ -744,9 +758,9 @@ static int write_cruft_pack(const struct pack_objects_args *args, in = xfdopen(cmd.in, "w"); for_each_string_list_item(item, names) fprintf(in, "%s-%s.pack\n", pack_prefix, item->string); - for_each_string_list_item(item, existing_packs) + for_each_string_list_item(item, &existing->non_kept_packs) fprintf(in, "-%s.pack\n", item->string); - for_each_string_list_item(item, existing_kept_packs) + for_each_string_list_item(item, &existing->kept_packs) fprintf(in, "%s.pack\n", item->string); fclose(in); @@ -778,8 +792,7 @@ int cmd_repack(int argc, const char **argv, const char *prefix) struct child_process cmd = CHILD_PROCESS_INIT; struct string_list_item *item; struct string_list names = STRING_LIST_INIT_DUP; - struct string_list existing_nonkept_packs = STRING_LIST_INIT_DUP; - struct string_list existing_kept_packs = STRING_LIST_INIT_DUP; + struct existing_packs existing = EXISTING_PACKS_INIT; struct pack_geometry geometry = { 0 }; struct strbuf line = STRBUF_INIT; struct tempfile *refs_snapshot = NULL; @@ -915,13 +928,12 @@ int cmd_repack(int argc, const char **argv, const char *prefix) packtmp_name = xstrfmt(".tmp-%d-pack", (int)getpid()); packtmp = mkpathdup("%s/%s", packdir, packtmp_name); - collect_pack_filenames(&existing_nonkept_packs, &existing_kept_packs, - &keep_pack_list); + collect_pack_filenames(&existing, &keep_pack_list); if (geometry.split_factor) { if (pack_everything) die(_("options '%s' and '%s' cannot be used together"), "--geometric", "-A/-a"); - init_pack_geometry(&geometry, &existing_kept_packs, &po_args); + init_pack_geometry(&geometry, &existing, &po_args); split_pack_geometry(&geometry); } @@ -965,7 +977,7 @@ int cmd_repack(int argc, const char **argv, const char *prefix) if (pack_everything & ALL_INTO_ONE) { repack_promisor_objects(&po_args, &names); - if (existing_nonkept_packs.nr && delete_redundant && + if (existing.non_kept_packs.nr && delete_redundant && !(pack_everything & PACK_CRUFT)) { for_each_string_list_item(item, &names) { strvec_pushf(&cmd.args, "--keep-pack=%s-%s.pack", @@ -1054,8 +1066,7 @@ int cmd_repack(int argc, const char **argv, const char *prefix) ret = write_cruft_pack(&cruft_po_args, packtmp, pack_prefix, cruft_expiration, &names, - &existing_nonkept_packs, - &existing_kept_packs); + &existing); if (ret) goto cleanup; @@ -1086,8 +1097,7 @@ int cmd_repack(int argc, const char **argv, const char *prefix) pack_prefix, NULL, &names, - &existing_nonkept_packs, - &existing_kept_packs); + &existing); if (ret) goto cleanup; } @@ -1133,7 +1143,7 @@ int cmd_repack(int argc, const char **argv, const char *prefix) if (delete_redundant && pack_everything & ALL_INTO_ONE) { const int hexsz = the_hash_algo->hexsz; - for_each_string_list_item(item, &existing_nonkept_packs) { + for_each_string_list_item(item, &existing.non_kept_packs) { char *sha1; size_t len = strlen(item->string); if (len < hexsz) @@ -1152,8 +1162,7 @@ int cmd_repack(int argc, const char **argv, const char *prefix) if (write_midx) { struct string_list include = STRING_LIST_INIT_NODUP; - midx_included_packs(&include, &existing_nonkept_packs, - &existing_kept_packs, &names, &geometry); + midx_included_packs(&include, &existing, &names, &geometry); ret = write_midx_included_packs(&include, &geometry, refs_snapshot ? get_tempfile_path(refs_snapshot) : NULL, @@ -1172,7 +1181,7 @@ int cmd_repack(int argc, const char **argv, const char *prefix) if (delete_redundant) { int opts = 0; - for_each_string_list_item(item, &existing_nonkept_packs) { + for_each_string_list_item(item, &existing.non_kept_packs) { if (!((uintptr_t)item->util & DELETE_PACK)) continue; remove_redundant_pack(packdir, item->string); @@ -1193,7 +1202,7 @@ int cmd_repack(int argc, const char **argv, const char *prefix) strbuf_strip_suffix(&buf, ".pack"); if ((p->pack_keep) || - (string_list_has_string(&existing_kept_packs, + (string_list_has_string(&existing.kept_packs, buf.buf))) continue; @@ -1224,8 +1233,7 @@ int cmd_repack(int argc, const char **argv, const char *prefix) cleanup: string_list_clear(&names, 1); - string_list_clear(&existing_nonkept_packs, 0); - string_list_clear(&existing_kept_packs, 0); + existing_packs_release(&existing); free_pack_geometry(&geometry); return ret; From patchwork Tue Sep 5 20:36:43 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Taylor Blau X-Patchwork-Id: 13375046 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 539C1CA1008 for ; Tue, 5 Sep 2023 20:36:55 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S243815AbjIEUgw (ORCPT ); Tue, 5 Sep 2023 16:36:52 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:60210 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S236319AbjIEUgv (ORCPT ); Tue, 5 Sep 2023 16:36:51 -0400 Received: from mail-oo1-xc31.google.com (mail-oo1-xc31.google.com [IPv6:2607:f8b0:4864:20::c31]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 55C94CE for ; Tue, 5 Sep 2023 13:36:45 -0700 (PDT) Received: by mail-oo1-xc31.google.com with SMTP id 006d021491bc7-57361de8878so1903992eaf.0 for ; Tue, 05 Sep 2023 13:36:45 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ttaylorr-com.20230601.gappssmtp.com; s=20230601; t=1693946204; x=1694551004; darn=vger.kernel.org; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:from:to:cc:subject:date:message-id:reply-to; bh=LDlao44r7BPaMovJ24otVVwLkpGqsqayNnvJXl85O/s=; b=yykd89nHImCG3piD1Ai6oV1mBdW5RuvAbzjEoo9dYXBgnEjmtk4lkPj5X3eVHZD1Jx HwMEoavoM9fezChs24QCdcqq8pjAGnQkF/4gpKPBkkznt7bSzOAbkRPUcvCYOwCRSPdv ukuDZiD+S2cjwnh0WuagakXT8kDcHThPaa+XvIBG73nZaOT+jOWg07LQwSSrVyBlyRYN z89hauwAQsgBVTEqyI+2dSecHsSdAAgtg9coZ8AlJVpvJdOgDpsYc9KSxUypsbXO/J5S WNi9jvJZnsNfI9FSVFydwsAdEc2uqcwXRsamgRrpnFCo7k6M++453B5mGQBPnQnkC1Gm Jf/Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1693946204; x=1694551004; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:x-gm-message-state:from:to:cc:subject:date :message-id:reply-to; bh=LDlao44r7BPaMovJ24otVVwLkpGqsqayNnvJXl85O/s=; b=Xa7RFVBYkA+Xh7CQwZyLHVN30L8Lnh27xRWiunLplBdAOAj5/3sf4UEez+fGax2quX 8vZFpN7bol6DYZ+Us2rmLOEtgwzgvOFkPnbjBMzl4IxRRlqikLyEXTMNvBxWezGjioIr xUvFWWvv0X0elg2tVNYc9LlsizzdTLkjHwy/YXDRGliBVG3sbMGGQulJGV9n8EkgltnT zjQdZY859Q4biLjVA+GnICHi1UWMN7mRjkRjjSG7b5e/4qbCx2aGqfst7OoQLE+MKZAp +8HnP5zlD0WuRtxsJHgkJBpuP6T1hsWBVWDT8U6ohxdbQkVwEe0PMMa7SKr7IQ4CtJ5O NmmA== X-Gm-Message-State: AOJu0YwgYkXTxTwAwR7NbMqd2NKyN+rAcNrhZqKWTygjDaQhCVSWQ/gK B2RMFnm85xtxFAJSxVJddKq/dKpP/ljBl9pfpigf9w== X-Google-Smtp-Source: AGHT+IGFmKeQM38En3FjNI8swUZSJCwxDilmQ3W/a3UMbuVw5ElR9YyuphMjSj4VUa2bNlHDDTaQuw== X-Received: by 2002:a05:6358:9499:b0:139:688e:c73e with SMTP id i25-20020a056358949900b00139688ec73emr739863rwb.32.1693946204314; Tue, 05 Sep 2023 13:36:44 -0700 (PDT) Received: from localhost (104-178-186-189.lightspeed.milwwi.sbcglobal.net. [104.178.186.189]) by smtp.gmail.com with ESMTPSA id r11-20020a25b98b000000b00d7745e2bb19sm2828559ybg.29.2023.09.05.13.36.43 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 05 Sep 2023 13:36:44 -0700 (PDT) Date: Tue, 5 Sep 2023 16:36:43 -0400 From: Taylor Blau To: git@vger.kernel.org Cc: Junio C Hamano , Jeff King , Patrick Steinhardt Subject: [PATCH 2/8] builtin/repack.c: extract marking packs for deletion Message-ID: <313537ef68892b15e772eaad8937a4a8c7ebbe61.1693946195.git.me@ttaylorr.com> References: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org At the end of a repack (when given `-d`), Git attempts to remove any packs which have been made "redundant" as a result of the repacking operation. For example, an all-into-one (`-A` or `-a`) repack makes every pre-existing pack which is not marked as kept redundant. Geometric repacks (with `--geometric=`) make any packs which were rolled up redundant, and so on. But before deleting the set of packs we think are redundant, we first check to see whether or not we just wrote a pack which is identical to any one of the packs we were going to delete. When this is the case, Git must avoid deleting that pack, since it matches a pack we just wrote (so deleting it may cause the repository to become corrupt). Right now we only process the list of non-kept packs in a single pass. But a future change will split the existing non-kept packs further into two lists: one for cruft packs, and another for non-cruft packs. Factor out this routine to prepare for calling it twice on two separate lists in a future patch. Signed-off-by: Taylor Blau --- builtin/repack.c | 50 +++++++++++++++++++++++++++++++----------------- 1 file changed, 32 insertions(+), 18 deletions(-) diff --git a/builtin/repack.c b/builtin/repack.c index c3ab89912e..708556836e 100644 --- a/builtin/repack.c +++ b/builtin/repack.c @@ -105,6 +105,36 @@ struct existing_packs { .non_kept_packs = STRING_LIST_INIT_DUP, \ } +static void mark_packs_for_deletion_1(struct string_list *names, + struct string_list *list) +{ + struct string_list_item *item; + const int hexsz = the_hash_algo->hexsz; + + for_each_string_list_item(item, list) { + char *sha1; + size_t len = strlen(item->string); + if (len < hexsz) + continue; + sha1 = item->string + len - hexsz; + /* + * Mark this pack for deletion, which ensures that this + * pack won't be included in a MIDX (if `--write-midx` + * was given) and that we will actually delete this pack + * (if `-d` was given). + */ + if (!string_list_has_string(names, sha1)) + item->util = (void*)(uintptr_t)((size_t)item->util | DELETE_PACK); + } +} + +static void mark_packs_for_deletion(struct existing_packs *existing, + struct string_list *names) + +{ + mark_packs_for_deletion_1(names, &existing->non_kept_packs); +} + static void existing_packs_release(struct existing_packs *existing) { string_list_clear(&existing->kept_packs, 0); @@ -1141,24 +1171,8 @@ int cmd_repack(int argc, const char **argv, const char *prefix) } /* End of pack replacement. */ - if (delete_redundant && pack_everything & ALL_INTO_ONE) { - const int hexsz = the_hash_algo->hexsz; - for_each_string_list_item(item, &existing.non_kept_packs) { - char *sha1; - size_t len = strlen(item->string); - if (len < hexsz) - continue; - sha1 = item->string + len - hexsz; - /* - * Mark this pack for deletion, which ensures that this - * pack won't be included in a MIDX (if `--write-midx` - * was given) and that we will actually delete this pack - * (if `-d` was given). - */ - if (!string_list_has_string(&names, sha1)) - item->util = (void*)(uintptr_t)((size_t)item->util | DELETE_PACK); - } - } + if (delete_redundant && pack_everything & ALL_INTO_ONE) + mark_packs_for_deletion(&existing, &names); if (write_midx) { struct string_list include = STRING_LIST_INIT_NODUP; From patchwork Tue Sep 5 20:36:46 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Taylor Blau X-Patchwork-Id: 13375048 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id D0FA3CA1008 for ; Tue, 5 Sep 2023 20:36:58 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S239033AbjIEUgx (ORCPT ); Tue, 5 Sep 2023 16:36:53 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:60252 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S243808AbjIEUgv (ORCPT ); Tue, 5 Sep 2023 16:36:51 -0400 Received: from mail-yw1-x112d.google.com (mail-yw1-x112d.google.com [IPv6:2607:f8b0:4864:20::112d]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 1EAAEF4 for ; Tue, 5 Sep 2023 13:36:48 -0700 (PDT) Received: by mail-yw1-x112d.google.com with SMTP id 00721157ae682-59231a1ca9eso28718657b3.1 for ; Tue, 05 Sep 2023 13:36:48 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ttaylorr-com.20230601.gappssmtp.com; s=20230601; t=1693946207; x=1694551007; darn=vger.kernel.org; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:from:to:cc:subject:date:message-id:reply-to; bh=FGrKSX49akuTfpqvvhFRP39KLO/xfNXDlJ7hWeKhRJg=; b=cQHlI5+jgczaRygDp4hJXLRL82qOeiuTDBYxR3z0KqUQSp8lPcLhoobe0XkWPANuer Oc4Rc4hTbOHa4vr8R/SOrYY6i2LHHBgofJCqR5RRwLfQe7y5F6mAa5I0EFBrdIxdrXyC XXZ1hlSIkrtMqoDa5eBlZjRJZsD15AQ5Gv/E3n1BCp7qWnVbLYF/ljRi0vA10DszwBPv HVmUBemr/o+c4JJWvopsZpcqes2fIaPx8vra5GhGBW9QWJFjjomaCwMryqmMwr68kXUH D4gwBU5UQPeqLj53jvxUC+HAzHmtdJo8JyLwBQUUfv4oz3/ww4S/Nb5u3Ds91C+c5zGX UXFA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1693946207; x=1694551007; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:x-gm-message-state:from:to:cc:subject:date :message-id:reply-to; bh=FGrKSX49akuTfpqvvhFRP39KLO/xfNXDlJ7hWeKhRJg=; b=Y6lT7yP/tEUjNRPp9Lwp5cTaTMzb5nSkW2v8aWm2Ut52/0OnOy1YV7oXeY7mK8fzc8 9TYO1DNyuvsYjV+K1ScUTZUKA5EpUp1Ttocniy84zfuS7eOlmLyIMwnqFfRvyq7f34J+ jkhp3nMkYSkYV4wx2rFSHwlHwWgQx0Jnyex5MwUViBfFOd25Se6WF2LKQd5VWWnIZeBH AH4ls1XsCTRkUBUtZUI9k0CdcekcquXnFtZzhliZloDusQBxGpMQwrxJ0mcjGAZUh/wV FytcXVACKwaj5CKOK5Wfrg8np5kzkFjZ0JPrKyfc7TmDs0NCOsY9J5gTREil7e/4Ma29 avfg== X-Gm-Message-State: AOJu0Yzg/TlOMkx3o+ve1TdHBCklK+kBXxu8rMKJmUIj11dUyOWXoe6+ NHZVzj3rx1Z1Y0fAMmsuFQo1wS82JxjEY8g6JMNRpQ== X-Google-Smtp-Source: AGHT+IEBz46Br7cN9HI93sHXlhdfReiUyIYpyThLh7BFX+pPumeeh4LxqtbiiysCWVtrhdzjrKbjIA== X-Received: by 2002:a25:3751:0:b0:d7e:8175:4fa9 with SMTP id e78-20020a253751000000b00d7e81754fa9mr12529218yba.4.1693946207145; Tue, 05 Sep 2023 13:36:47 -0700 (PDT) Received: from localhost (104-178-186-189.lightspeed.milwwi.sbcglobal.net. [104.178.186.189]) by smtp.gmail.com with ESMTPSA id d1-20020a25bc41000000b00d74c9618c6fsm2848824ybk.1.2023.09.05.13.36.46 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 05 Sep 2023 13:36:46 -0700 (PDT) Date: Tue, 5 Sep 2023 16:36:46 -0400 From: Taylor Blau To: git@vger.kernel.org Cc: Junio C Hamano , Jeff King , Patrick Steinhardt Subject: [PATCH 3/8] builtin/repack.c: extract redundant pack cleanup for --geometric Message-ID: <5c25ef87c1430e012a2e48b738b3b5aa760b4b0f.1693946195.git.me@ttaylorr.com> References: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org To reduce the complexity of the already quite-long `cmd_repack()` implementation, extract out the parts responsible for deleting redundant packs from a geometric repack out into its own sub-routine. Signed-off-by: Taylor Blau --- builtin/repack.c | 52 +++++++++++++++++++++++++++--------------------- 1 file changed, 29 insertions(+), 23 deletions(-) diff --git a/builtin/repack.c b/builtin/repack.c index 708556836e..d3e6326bb9 100644 --- a/builtin/repack.c +++ b/builtin/repack.c @@ -540,6 +540,32 @@ static struct packed_git *get_preferred_pack(struct pack_geometry *geometry) return NULL; } +static void geometry_remove_redundant_packs(struct pack_geometry *geometry, + struct string_list *names, + struct existing_packs *existing) +{ + struct strbuf buf = STRBUF_INIT; + uint32_t i; + + for (i = 0; i < geometry->split; i++) { + struct packed_git *p = geometry->pack[i]; + if (string_list_has_string(names, hash_to_hex(p->hash))) + continue; + + strbuf_reset(&buf); + strbuf_addstr(&buf, pack_basename(p)); + strbuf_strip_suffix(&buf, ".pack"); + + if ((p->pack_keep) || + (string_list_has_string(&existing->kept_packs, buf.buf))) + continue; + + remove_redundant_pack(packdir, buf.buf); + } + + strbuf_release(&buf); +} + static void free_pack_geometry(struct pack_geometry *geometry) { if (!geometry) @@ -1201,29 +1227,9 @@ int cmd_repack(int argc, const char **argv, const char *prefix) remove_redundant_pack(packdir, item->string); } - if (geometry.split_factor) { - struct strbuf buf = STRBUF_INIT; - - uint32_t i; - for (i = 0; i < geometry.split; i++) { - struct packed_git *p = geometry.pack[i]; - if (string_list_has_string(&names, - hash_to_hex(p->hash))) - continue; - - strbuf_reset(&buf); - strbuf_addstr(&buf, pack_basename(p)); - strbuf_strip_suffix(&buf, ".pack"); - - if ((p->pack_keep) || - (string_list_has_string(&existing.kept_packs, - buf.buf))) - continue; - - remove_redundant_pack(packdir, buf.buf); - } - strbuf_release(&buf); - } + if (geometry.split_factor) + geometry_remove_redundant_packs(&geometry, &names, + &existing); if (show_progress) opts |= PRUNE_PACKED_VERBOSE; prune_packed_objects(opts); From patchwork Tue Sep 5 20:36:48 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Taylor Blau X-Patchwork-Id: 13375050 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 81611CA100D for ; Tue, 5 Sep 2023 20:37:01 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S243827AbjIEUg4 (ORCPT ); Tue, 5 Sep 2023 16:36:56 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:43204 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S243807AbjIEUgy (ORCPT ); Tue, 5 Sep 2023 16:36:54 -0400 Received: from mail-oo1-xc2a.google.com (mail-oo1-xc2a.google.com [IPv6:2607:f8b0:4864:20::c2a]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id E56A5E6 for ; Tue, 5 Sep 2023 13:36:50 -0700 (PDT) Received: by mail-oo1-xc2a.google.com with SMTP id 006d021491bc7-57359e85e9bso1612561eaf.2 for ; Tue, 05 Sep 2023 13:36:50 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ttaylorr-com.20230601.gappssmtp.com; s=20230601; t=1693946210; x=1694551010; darn=vger.kernel.org; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:from:to:cc:subject:date:message-id:reply-to; bh=FqSUCweIV5Zb8kL18Y6iJm3DZkvivOofgc4CKrPN9Ho=; b=Wjmp0XvMvNnjDVrbma7ttprd/5/93XDzPgcSd79dv9qhKFKQR+FyTd6mGXFjqExn9X 2hdRkFFMLl+eMyqNbMyQzlCwxoFxr0zdmCJyUhN/4U1NkdwcWpdTdpCUuX7YI5AGyEne gUcdQltDvXQ0qptNj1ykMhUeJpQcxVLDme5E4Sh+WiOc2V7E68wzva2QAeObQi0WEfhR zMXrGZ1A9h+pZ6Vyss/HFabR149dh09EFpSqYWGUq1LQ6XkOCD7Zo6swABtFzr/K2Ebp uqWcW727vgrnTW6TBNENacxmYGKClVA0xC3TX50FeA+waMBsM8SxYNoLUFMLuKALDDBp 64iQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1693946210; x=1694551010; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:x-gm-message-state:from:to:cc:subject:date :message-id:reply-to; bh=FqSUCweIV5Zb8kL18Y6iJm3DZkvivOofgc4CKrPN9Ho=; b=X7+mJoFn+C30NwhhfiXcMMHWL1irN+WSm05gUrT6Md8TLfQ4mfhmlzZqey1GGdb/8F 80+9mlZk+W3UTEyA442ReQ2uODAiLKkaYoWjR+FvEMBltRiWNp1f2FzFiAjrvG3EYcpj YKfmWMk8VhauqDC2HEL8VnSZt4KSm7J4+BOBXgf344MKPggTLJzTNz7oEXLdMtYHP9sl oSWkw74ZF28qSInTIlCVsbQ6H3/RjIaTNWBoaTIDsTRHyFwXoU/WLv2HFQc0cxSnjgN0 /t0vboRtfjBp4DInGCmZ5g9p8WlZwXuKVJidXNR8CjkKv92VYE6DChRrzcxIGS7A58c6 6Vpw== X-Gm-Message-State: AOJu0YyJ/rucL0Ou5CbtoxhUOaqwxtDXeUAjdMz7ojMcqMA0XtcHaTJr KP+MWZfxpYhdG+VHNtYHXnDV+AZvwwdBXaSo7rvOwg== X-Google-Smtp-Source: AGHT+IF0VTypAE/liRH8hlDa+yxGjp/ab2gVC32SK+sMQ7O999J4idA9ccHc5SR0OiUYsD3XGRkerg== X-Received: by 2002:a05:6358:590d:b0:135:4c43:f4df with SMTP id g13-20020a056358590d00b001354c43f4dfmr983234rwf.3.1693946209837; Tue, 05 Sep 2023 13:36:49 -0700 (PDT) Received: from localhost (104-178-186-189.lightspeed.milwwi.sbcglobal.net. [104.178.186.189]) by smtp.gmail.com with ESMTPSA id s6-20020a0dd006000000b005924fb1be44sm3367958ywd.139.2023.09.05.13.36.49 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 05 Sep 2023 13:36:49 -0700 (PDT) Date: Tue, 5 Sep 2023 16:36:48 -0400 From: Taylor Blau To: git@vger.kernel.org Cc: Junio C Hamano , Jeff King , Patrick Steinhardt Subject: [PATCH 4/8] builtin/repack.c: extract redundant pack cleanup for existing packs Message-ID: <7bb543fef8b140726b6b3efc2bb2cb1a6384fcd9.1693946195.git.me@ttaylorr.com> References: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org To remove redundant packs at the end of a repacking operation, Git uses its `remove_redundant_pack()` function in a loop over the set of pre-existing, non-kept packs. In a later commit, we will split this list into two, one for pre-existing cruft pack(s), and another for non-cruft pack(s). Prepare for this by factoring out the routine to loop over and delete redundant packs into its own function. Instead of calling `remove_redundant_pack()` directly, we now will call `remove_redundant_existing_packs()`, which itself dispatches a call to `remove_redundant_packs_1()`. Note that the geometric repacking code will still call `remove_redundant_pack()` directly, but see the previous commit for more details. Having `remove_redundant_packs_1()` exist as a separate function may seem like overkill in this patch. However, a later patch will call `remove_redundant_packs_1()` once over two separate lists, so this refactoring sets us up for that. Signed-off-by: Taylor Blau --- builtin/repack.c | 45 ++++++++++++++++++++++++++++----------------- 1 file changed, 28 insertions(+), 17 deletions(-) diff --git a/builtin/repack.c b/builtin/repack.c index d3e6326bb9..f6717e334c 100644 --- a/builtin/repack.c +++ b/builtin/repack.c @@ -135,6 +135,33 @@ static void mark_packs_for_deletion(struct existing_packs *existing, mark_packs_for_deletion_1(names, &existing->non_kept_packs); } +static void remove_redundant_pack(const char *dir_name, const char *base_name) +{ + struct strbuf buf = STRBUF_INIT; + struct multi_pack_index *m = get_local_multi_pack_index(the_repository); + strbuf_addf(&buf, "%s.pack", base_name); + if (m && midx_contains_pack(m, buf.buf)) + clear_midx_file(the_repository); + strbuf_insertf(&buf, 0, "%s/", dir_name); + unlink_pack_path(buf.buf, 1); + strbuf_release(&buf); +} + +static void remove_redundant_packs_1(struct string_list *packs) +{ + struct string_list_item *item; + for_each_string_list_item(item, packs) { + if (!((uintptr_t)item->util & DELETE_PACK)) + continue; + remove_redundant_pack(packdir, item->string); + } +} + +static void remove_redundant_existing_packs(struct existing_packs *existing) +{ + remove_redundant_packs_1(&existing->non_kept_packs); +} + static void existing_packs_release(struct existing_packs *existing) { string_list_clear(&existing->kept_packs, 0); @@ -184,18 +211,6 @@ static void collect_pack_filenames(struct existing_packs *existing, strbuf_release(&buf); } -static void remove_redundant_pack(const char *dir_name, const char *base_name) -{ - struct strbuf buf = STRBUF_INIT; - struct multi_pack_index *m = get_local_multi_pack_index(the_repository); - strbuf_addf(&buf, "%s.pack", base_name); - if (m && midx_contains_pack(m, buf.buf)) - clear_midx_file(the_repository); - strbuf_insertf(&buf, 0, "%s/", dir_name); - unlink_pack_path(buf.buf, 1); - strbuf_release(&buf); -} - static void prepare_pack_objects(struct child_process *cmd, const struct pack_objects_args *args, const char *out) @@ -1221,11 +1236,7 @@ int cmd_repack(int argc, const char **argv, const char *prefix) if (delete_redundant) { int opts = 0; - for_each_string_list_item(item, &existing.non_kept_packs) { - if (!((uintptr_t)item->util & DELETE_PACK)) - continue; - remove_redundant_pack(packdir, item->string); - } + remove_redundant_existing_packs(&existing); if (geometry.split_factor) geometry_remove_redundant_packs(&geometry, &names, From patchwork Tue Sep 5 20:36:51 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Taylor Blau X-Patchwork-Id: 13375047 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id F30E5CA100E for ; Tue, 5 Sep 2023 20:36:55 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S243829AbjIEUg5 (ORCPT ); Tue, 5 Sep 2023 16:36:57 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:43342 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S237898AbjIEUg5 (ORCPT ); Tue, 5 Sep 2023 16:36:57 -0400 Received: from mail-yb1-xb31.google.com (mail-yb1-xb31.google.com [IPv6:2607:f8b0:4864:20::b31]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 77D01197 for ; Tue, 5 Sep 2023 13:36:53 -0700 (PDT) Received: by mail-yb1-xb31.google.com with SMTP id 3f1490d57ef6-d66f105634eso2209995276.1 for ; Tue, 05 Sep 2023 13:36:53 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ttaylorr-com.20230601.gappssmtp.com; s=20230601; t=1693946212; x=1694551012; darn=vger.kernel.org; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:from:to:cc:subject:date:message-id:reply-to; bh=6IlrSxUII3vDULyN+O9gyrIKxMgvND139bZuetSHgFk=; b=Qp/RtoBPNngT9gzFch8xHjdvFHLQ0H/0AvXEcPc/LMgZ3POFuuTgWzrd0vUG+nQyzE CAriGQ284q6Rm9bdTFuBaDXCLECH8MVFFI4Zcg2pwxJ09o881o59F/82yLILwIdhCIpg ihW+f63EanTOaXrB3C4irjzJz7vG3e9Y9MAoE1sLewDdGqIwz+Ecow2wHLyKNd4HkEah qzLHagCdpv9u78amytT3bZHjm6d8+X3YOK94kc3CfSQYjRQ1bR/d997drZMhGuNr/dJF FAEu0ZIMvBE1ptRHlhMj3YRu6ojRcRHFxTWQSkR3p1WTtreJG7APzDLDaBdGUSjp9zOb d60g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1693946212; x=1694551012; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:x-gm-message-state:from:to:cc:subject:date :message-id:reply-to; bh=6IlrSxUII3vDULyN+O9gyrIKxMgvND139bZuetSHgFk=; b=PPff+F2WNgM50CBliZ9DavfbwjsexampAsyGedWhi8f/JszOS/5bZPmBLMZO4gIr09 GG8TQB4adqHRmnvyz7CWkaOSx7TzAoeeX3YntAqOPIFwrQx2/3bUA4bheRNgM0I8GKwA bvNHDmeR7iVyVDKgf7ifTHSF1Vu6nBy8jZx8QCYW9bELZBWsbLbpXLo6Z/J48YPRjpdA zbCWiRRv4urB5647lI8EeiW8hxPK9mma7dy07bgz1dPTbj6kgp1YarxkyMtCRGmETSjH /VBmImn0swpz6DsnsBhBmAlLuiN/NhKz6DmFg/XVcIFm+ucy7FrfwDrSSzspYb/rlUxt nooQ== X-Gm-Message-State: AOJu0YzHrb7Fvb1VHoN1n5EPq/jGYLAGG0d7VIqKXH5I+42Sdjs/3D+s d/Adbk1uYfnZUPJ9vcyHI0+Li+s1WV9/gOchUVhLUA== X-Google-Smtp-Source: AGHT+IH6erNmy2/2+yEx/TalcV/0hQFEqT6WoMG5scJ5HacErBXgG6in2wg33RyrDLnjPxeuFpaypA== X-Received: by 2002:a25:bcc6:0:b0:d7b:985b:278f with SMTP id l6-20020a25bcc6000000b00d7b985b278fmr14670164ybm.16.1693946212557; Tue, 05 Sep 2023 13:36:52 -0700 (PDT) Received: from localhost (104-178-186-189.lightspeed.milwwi.sbcglobal.net. [104.178.186.189]) by smtp.gmail.com with ESMTPSA id u15-20020a258f8f000000b00d7823cc7d62sm2861521ybl.19.2023.09.05.13.36.52 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 05 Sep 2023 13:36:52 -0700 (PDT) Date: Tue, 5 Sep 2023 16:36:51 -0400 From: Taylor Blau To: git@vger.kernel.org Cc: Junio C Hamano , Jeff King , Patrick Steinhardt Subject: [PATCH 5/8] builtin/repack.c: extract `has_existing_non_kept_packs()` Message-ID: References: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org When there is: - at least one pre-existing packfile (which is not marked as kept), - repacking with the `-d` flag, and - not doing a cruft repack , then we pass a handful of additional options to the inner `pack-objects` process, like `--unpack-unreachable`, `--keep-unreachable`, and `--pack-loose-unreachable`, in addition to marking any packs we just wrote for promisor remotes as kept in-core (with `--keep-pack`, as opposed to the presence of a ".keep" file on disk). Because we store both cruft and non-cruft packs together in the same `existing.non_kept_packs` list, it suffices to check its `nr` member to see if it is zero or not. But a following change will store cruft- and non-cruft packs separately, meaning this check would break as a result. Prepare for this by extracting this part of the check into a new helper function called `has_existing_non_kept_packs()`. This patch does not introduce any functional changes, but prepares us to make a more isolated change in a subsequent patch. Signed-off-by: Taylor Blau --- builtin/repack.c | 8 +++++++- 1 file changed, 7 insertions(+), 1 deletion(-) diff --git a/builtin/repack.c b/builtin/repack.c index f6717e334c..3f0789ff89 100644 --- a/builtin/repack.c +++ b/builtin/repack.c @@ -105,6 +105,11 @@ struct existing_packs { .non_kept_packs = STRING_LIST_INIT_DUP, \ } +static int has_existing_non_kept_packs(const struct existing_packs *existing) +{ + return existing->non_kept_packs.nr; +} + static void mark_packs_for_deletion_1(struct string_list *names, struct string_list *list) { @@ -1048,7 +1053,8 @@ int cmd_repack(int argc, const char **argv, const char *prefix) if (pack_everything & ALL_INTO_ONE) { repack_promisor_objects(&po_args, &names); - if (existing.non_kept_packs.nr && delete_redundant && + if (has_existing_non_kept_packs(&existing) && + delete_redundant && !(pack_everything & PACK_CRUFT)) { for_each_string_list_item(item, &names) { strvec_pushf(&cmd.args, "--keep-pack=%s-%s.pack", From patchwork Tue Sep 5 20:36:54 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Taylor Blau X-Patchwork-Id: 13375051 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 7D7A9CA1008 for ; Tue, 5 Sep 2023 20:37:31 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S243906AbjIEUhd (ORCPT ); Tue, 5 Sep 2023 16:37:33 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:43488 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S243878AbjIEUhD (ORCPT ); Tue, 5 Sep 2023 16:37:03 -0400 Received: from mail-yw1-x1134.google.com (mail-yw1-x1134.google.com [IPv6:2607:f8b0:4864:20::1134]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 1C6A0197 for ; Tue, 5 Sep 2023 13:36:56 -0700 (PDT) Received: by mail-yw1-x1134.google.com with SMTP id 00721157ae682-58fc4eaa04fso30087517b3.0 for ; Tue, 05 Sep 2023 13:36:56 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ttaylorr-com.20230601.gappssmtp.com; s=20230601; t=1693946215; x=1694551015; darn=vger.kernel.org; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:from:to:cc:subject:date:message-id:reply-to; bh=ASFF6ACfQmXR2E/jS2JSQg2gWuVPyob+iPpnASfyzyk=; b=kYWeqPsdGXyLzIeccoANNFtSrx3wXhQ+U/9oZ71bREQ+Jrt52vRLL4cE5mnIdT2DUi l2xK6JaaFjAU63/JXZntqF3uCCOJJ99wC5cVw5EzRKg6gzKWRqzxOD03Ca/Jtj0XBJAw hhWq5czR1X2LfhJRMR9OmkH5tvTb7KOOw6J76DFXMRaCZuqVcHVpw5eMG4LL5T8K2bQO flQrWh8tXXV8pAWn5Hhz3NgjvQ1WJ1XM0Ot+j4f7zL8DQzkCTvzArMDGYm7361xHlu9D X0Ytv1mCiGtAZ3fX+ongdywimC9JjK1dvOoFMYPRtlMJ4Q4OtlPUWXfi4p57kcp7fSdw Dxgg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1693946215; x=1694551015; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:x-gm-message-state:from:to:cc:subject:date :message-id:reply-to; bh=ASFF6ACfQmXR2E/jS2JSQg2gWuVPyob+iPpnASfyzyk=; b=TuHsD2yLWdbKtVVbNeRrQRD75KtiG6WPrRll0XgMSjukjTTSH+yA/j5xNe7LEhIwRL V5TRJEJnC2BeDMp7buMhYLdGSkx7nwDsUsEX5x7TlHqM5g/zW4PBju95v/NV9+hniuKz 1gW1JwbevCdfDArRoRNU7Vv5kCpReOAxcAstBpTwUWgV1V4U1h19sqDtFfwffpsyjzxI vbTIVFNiF84FHF+y9+DZJwLbMFdpq9rIOQhT0lxOW9TX1T5g3K5vu5AJjVB/V68UJQNI 5PkVDUL+dqJBXDnpZ0vm/7JNda+DKnASz1e1e6o1sY42k4b0tW2zXc4/Tkn79JeOMLK2 mUWw== X-Gm-Message-State: AOJu0Yz0kdI6l8Owihbo6Vo9NGa6nRhirwU6Ha8EFXm50jrGIF/pgRlt UQKKiwz/Jwml3ZfSafFN4SMvX/8pONwdTwmmx2o/OQ== X-Google-Smtp-Source: AGHT+IFiZxARE9AJGfqdrp8RfR4yEaS8jLWJl1S/UMZ/CK8+bn53anUzGh8RuCMm8LFWYZKu5M/+uw== X-Received: by 2002:a81:9988:0:b0:59a:e672:5a03 with SMTP id q130-20020a819988000000b0059ae6725a03mr5961009ywg.44.1693946215190; Tue, 05 Sep 2023 13:36:55 -0700 (PDT) Received: from localhost (104-178-186-189.lightspeed.milwwi.sbcglobal.net. [104.178.186.189]) by smtp.gmail.com with ESMTPSA id o4-20020a817304000000b0059b1e1b6e5dsm174973ywc.91.2023.09.05.13.36.54 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 05 Sep 2023 13:36:55 -0700 (PDT) Date: Tue, 5 Sep 2023 16:36:54 -0400 From: Taylor Blau To: git@vger.kernel.org Cc: Junio C Hamano , Jeff King , Patrick Steinhardt Subject: [PATCH 6/8] builtin/repack.c: store existing cruft packs separately Message-ID: <414a558883830a29924710126960074b37ab97fc.1693946195.git.me@ttaylorr.com> References: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org When repacking with the `--write-midx` option, we invoke the function `midx_included_packs()` in order to produce the list of packs we want to include in the resulting MIDX. This list is comprised of: - existing .keep packs - any pack(s) which were written earlier in the same process - any unchanged packs when doing a `--geometric` repack - any cruft packs Prior to this patch, we stored pre-existing cruft and non-cruft packs together (provided those packs are non-kept). This meant we needed an additional bit to indicate which non-kept pack(s) were cruft versus those that aren't. But alternatively we can store cruft packs in a separate list, avoiding the need for this extra bit, and simplifying the code below. Signed-off-by: Taylor Blau --- builtin/repack.c | 39 +++++++++++++++++++++++---------------- 1 file changed, 23 insertions(+), 16 deletions(-) diff --git a/builtin/repack.c b/builtin/repack.c index 3f0789ff89..478fab96c9 100644 --- a/builtin/repack.c +++ b/builtin/repack.c @@ -27,7 +27,6 @@ #define PACK_CRUFT 4 #define DELETE_PACK 1 -#define CRUFT_PACK 2 static int pack_everything; static int delta_base_offset = 1; @@ -98,16 +97,18 @@ static int repack_config(const char *var, const char *value, struct existing_packs { struct string_list kept_packs; struct string_list non_kept_packs; + struct string_list cruft_packs; }; #define EXISTING_PACKS_INIT { \ .kept_packs = STRING_LIST_INIT_DUP, \ .non_kept_packs = STRING_LIST_INIT_DUP, \ + .cruft_packs = STRING_LIST_INIT_DUP, \ } static int has_existing_non_kept_packs(const struct existing_packs *existing) { - return existing->non_kept_packs.nr; + return existing->non_kept_packs.nr || existing->cruft_packs.nr; } static void mark_packs_for_deletion_1(struct string_list *names, @@ -138,6 +139,7 @@ static void mark_packs_for_deletion(struct existing_packs *existing, { mark_packs_for_deletion_1(names, &existing->non_kept_packs); + mark_packs_for_deletion_1(names, &existing->cruft_packs); } static void remove_redundant_pack(const char *dir_name, const char *base_name) @@ -165,12 +167,14 @@ static void remove_redundant_packs_1(struct string_list *packs) static void remove_redundant_existing_packs(struct existing_packs *existing) { remove_redundant_packs_1(&existing->non_kept_packs); + remove_redundant_packs_1(&existing->cruft_packs); } static void existing_packs_release(struct existing_packs *existing) { string_list_clear(&existing->kept_packs, 0); string_list_clear(&existing->non_kept_packs, 0); + string_list_clear(&existing->cruft_packs, 0); } /* @@ -204,12 +208,10 @@ static void collect_pack_filenames(struct existing_packs *existing, if ((extra_keep->nr > 0 && i < extra_keep->nr) || p->pack_keep) string_list_append(&existing->kept_packs, buf.buf); - else { - struct string_list_item *item; - item = string_list_append(&existing->non_kept_packs, buf.buf); - if (p->is_cruft) - item->util = (void*)(uintptr_t)CRUFT_PACK; - } + else if (p->is_cruft) + string_list_append(&existing->cruft_packs, buf.buf); + else + string_list_append(&existing->non_kept_packs, buf.buf); } string_list_sort(&existing->kept_packs); @@ -691,14 +693,11 @@ static void midx_included_packs(struct string_list *include, string_list_insert(include, strbuf_detach(&buf, NULL)); } - for_each_string_list_item(item, &existing->non_kept_packs) { - if (!((uintptr_t)item->util & CRUFT_PACK)) { - /* - * no need to check DELETE_PACK, since we're not - * doing an ALL_INTO_ONE repack - */ - continue; - } + for_each_string_list_item(item, &existing->cruft_packs) { + /* + * no need to check DELETE_PACK, since we're not + * doing an ALL_INTO_ONE repack + */ string_list_insert(include, xstrfmt("%s.idx", item->string)); } } else { @@ -707,6 +706,12 @@ static void midx_included_packs(struct string_list *include, continue; string_list_insert(include, xstrfmt("%s.idx", item->string)); } + + for_each_string_list_item(item, &existing->cruft_packs) { + if ((uintptr_t)item->util & DELETE_PACK) + continue; + string_list_insert(include, xstrfmt("%s.idx", item->string)); + } } } @@ -836,6 +841,8 @@ static int write_cruft_pack(const struct pack_objects_args *args, fprintf(in, "%s-%s.pack\n", pack_prefix, item->string); for_each_string_list_item(item, &existing->non_kept_packs) fprintf(in, "-%s.pack\n", item->string); + for_each_string_list_item(item, &existing->cruft_packs) + fprintf(in, "-%s.pack\n", item->string); for_each_string_list_item(item, &existing->kept_packs) fprintf(in, "%s.pack\n", item->string); fclose(in); From patchwork Tue Sep 5 20:36:56 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Taylor Blau X-Patchwork-Id: 13375052 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 0232DCA1008 for ; Tue, 5 Sep 2023 20:37:36 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S243872AbjIEUhf (ORCPT ); Tue, 5 Sep 2023 16:37:35 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:43478 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S243866AbjIEUhX (ORCPT ); Tue, 5 Sep 2023 16:37:23 -0400 Received: from mail-yw1-x112f.google.com (mail-yw1-x112f.google.com [IPv6:2607:f8b0:4864:20::112f]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id C7570F4 for ; Tue, 5 Sep 2023 13:36:58 -0700 (PDT) Received: by mail-yw1-x112f.google.com with SMTP id 00721157ae682-59234aaca15so29995447b3.3 for ; Tue, 05 Sep 2023 13:36:58 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ttaylorr-com.20230601.gappssmtp.com; s=20230601; t=1693946218; x=1694551018; darn=vger.kernel.org; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:from:to:cc:subject:date:message-id:reply-to; bh=6JDb3tzUE1sg6L9+uU868gPJRyDrC/J8mq2wo/rsO8Q=; b=Hf2F2c9BV4OuYYJoiR7fyvOIywdg1aUqo5ZWPzdTyqTXvM1SSaYJIV1WmtBEIpEJgc 0XHZgBMgxLFbHijspxfvA5iLLVEjrAJlH3KNWXa0rQDLuWKsxyoNfmZZVnfSDpoV8Zbr wSYsEa4G3bb3XF1ZJd+IrNNJ/3KhzYtVqwkPPK2Gfj7QmEWfEdWDLCDilATCe97jvEwB pDxvDmyMDJr0qNXxMoPhykOmP49fpt9mRPy5fibFWsicesYoGSJH+pm6Q70O5p7ecofT I8K9c3YnzlmKyLQg2hE7nXBjUSBgArNll3obSS3K0sff4R6gvpUja/s+dlSQl6gxrKNM 7xNQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1693946218; x=1694551018; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:x-gm-message-state:from:to:cc:subject:date :message-id:reply-to; bh=6JDb3tzUE1sg6L9+uU868gPJRyDrC/J8mq2wo/rsO8Q=; b=TUgQSq7Y6Ob1fY0OYLyBXLlWaxIHbT/0yXhaS6OxkaQhn+2Dqy9tq2y0oJX/CBm6Ke hQmZeZP/fTUp0VB+6X9uJ2xNHuQL9mIpp9OtRftMgUOBCxmPRodCdJim8v0STuniL4zN e/lzsFVFeTtxZcu+qnl+aGDFO54AWT7ZJ7LWrlQ52/18rJPtssc+dJYoevFGy3lDeXQr FBWpOUcmfyaeTOiNMgV27qrs94igZUegwQr5rUBQ+pJ4T5HcxlQgCBwyfmNwf/kAwk5e H3jpFAE6P0KG3W/s2hVRLKUTo6cJrqbbeHHnIsNKoP9YP6vBirwDEDi79fkAmC/kEsLa yDcA== X-Gm-Message-State: AOJu0YyC59BjHT0/Wx4Nh46+NyOWBNPH6VsgAaCxoV2+UBwPZg14ta3M vX6FYP6POIFF8k4xWOpmq5yKSe3BlScemQhbkB2i/w== X-Google-Smtp-Source: AGHT+IENWzp1sBblsb7foqurCBfAQPM8xrzwRNcyjn3wMxtsTvuGMeNutoBkGB+HPs7icDyIbBXxZw== X-Received: by 2002:a81:8315:0:b0:589:8b56:15f with SMTP id t21-20020a818315000000b005898b56015fmr15503358ywf.24.1693946217907; Tue, 05 Sep 2023 13:36:57 -0700 (PDT) Received: from localhost (104-178-186-189.lightspeed.milwwi.sbcglobal.net. [104.178.186.189]) by smtp.gmail.com with ESMTPSA id w125-20020a817b83000000b0058427045833sm3388225ywc.133.2023.09.05.13.36.57 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 05 Sep 2023 13:36:57 -0700 (PDT) Date: Tue, 5 Sep 2023 16:36:56 -0400 From: Taylor Blau To: git@vger.kernel.org Cc: Junio C Hamano , Jeff King , Patrick Steinhardt Subject: [PATCH 7/8] builtin/repack.c: drop `DELETE_PACK` macro Message-ID: <559b487e2ab056c79367a673188764e4cdce3c96.1693946195.git.me@ttaylorr.com> References: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org Signed-off-by: Taylor Blau Signed-off-by: Taylor Blau --- builtin/repack.c | 18 ++++++++++-------- 1 file changed, 10 insertions(+), 8 deletions(-) diff --git a/builtin/repack.c b/builtin/repack.c index 478fab96c9..6110598a69 100644 --- a/builtin/repack.c +++ b/builtin/repack.c @@ -26,8 +26,6 @@ #define LOOSEN_UNREACHABLE 2 #define PACK_CRUFT 4 -#define DELETE_PACK 1 - static int pack_everything; static int delta_base_offset = 1; static int pack_kept_objects = -1; @@ -96,6 +94,10 @@ static int repack_config(const char *var, const char *value, struct existing_packs { struct string_list kept_packs; + /* + * for both non_kept_packs, and cruft_packs, a non-NULL + * 'util' field indicates the pack should be deleted. + */ struct string_list non_kept_packs; struct string_list cruft_packs; }; @@ -130,7 +132,7 @@ static void mark_packs_for_deletion_1(struct string_list *names, * (if `-d` was given). */ if (!string_list_has_string(names, sha1)) - item->util = (void*)(uintptr_t)((size_t)item->util | DELETE_PACK); + item->util = (void*)1; } } @@ -158,7 +160,7 @@ static void remove_redundant_packs_1(struct string_list *packs) { struct string_list_item *item; for_each_string_list_item(item, packs) { - if (!((uintptr_t)item->util & DELETE_PACK)) + if (!item->util) continue; remove_redundant_pack(packdir, item->string); } @@ -695,20 +697,20 @@ static void midx_included_packs(struct string_list *include, for_each_string_list_item(item, &existing->cruft_packs) { /* - * no need to check DELETE_PACK, since we're not - * doing an ALL_INTO_ONE repack + * no need to check for deleted packs, since we're + * not doing an ALL_INTO_ONE repack */ string_list_insert(include, xstrfmt("%s.idx", item->string)); } } else { for_each_string_list_item(item, &existing->non_kept_packs) { - if ((uintptr_t)item->util & DELETE_PACK) + if (item->util) continue; string_list_insert(include, xstrfmt("%s.idx", item->string)); } for_each_string_list_item(item, &existing->cruft_packs) { - if ((uintptr_t)item->util & DELETE_PACK) + if (item->util) continue; string_list_insert(include, xstrfmt("%s.idx", item->string)); } From patchwork Tue Sep 5 20:37:00 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Taylor Blau X-Patchwork-Id: 13375053 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 108AECA100D for ; Tue, 5 Sep 2023 20:37:38 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S239188AbjIEUhk (ORCPT ); Tue, 5 Sep 2023 16:37:40 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:33072 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S243847AbjIEUhd (ORCPT ); Tue, 5 Sep 2023 16:37:33 -0400 Received: from mail-yw1-x1135.google.com (mail-yw1-x1135.google.com [IPv6:2607:f8b0:4864:20::1135]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id CEBA0CFE for ; Tue, 5 Sep 2023 13:37:02 -0700 (PDT) Received: by mail-yw1-x1135.google.com with SMTP id 00721157ae682-58d70c441d5so29106737b3.2 for ; Tue, 05 Sep 2023 13:37:02 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ttaylorr-com.20230601.gappssmtp.com; s=20230601; t=1693946222; x=1694551022; darn=vger.kernel.org; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:from:to:cc:subject:date:message-id:reply-to; bh=NFdUY2x58Zha1WVlnRoV5I3qvQr2jZIt6GVF3cd/ntQ=; b=Foo5gsvZdvXtVfMZkOCWoEaah3/HqmEQN/5khbGsHLH4FHxdGjQ9Q81lMyIIhOV85J FwR7hTIjHIMaEuPGeiK//vkaGXpHFvuULQwxvyer3SKsH/oWuxA2+iDILzxgqhC4Q2bK Lip99ypDW/UAMoj2TNzZy0B6ba6stQfnUsIJbNMof2yoh4K4hrbqUva1mZrwocjGjn7R 3ugaLAfGoScw1U2HFI3XVAXrxyu08+JHIIUWiD2oHJDF/Sptp8eOts+wKDRolYMsKIOh vWJ43eGbZxpy5t2LC3G8qj3/65U4UxSzv+zN8iPzoavxrjo8W5ntEX5/OyzECpif6d8l nDFw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1693946222; x=1694551022; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:x-gm-message-state:from:to:cc:subject:date :message-id:reply-to; bh=NFdUY2x58Zha1WVlnRoV5I3qvQr2jZIt6GVF3cd/ntQ=; b=lp82V4zRtIJxPSVKSupi80k3bAesmcu1OeRwS9Dz+vQcnCSn3tG6A+z45jaA8tjek2 zV+3F++TeGY1J/A1at26RTWykPVAJIV4Zaevje3YdoQvtmF9WFsS+sQifI4leoNUuByX Zunk38gtNgsfYkT6CuVhOZdjdKm05P7+2Eox4eJk165G1Oif7OHaNpMGzqe0Bd4J7yBN pmXSY5bs+ZOhgEbFgIZGPn3ITujsABvUNyzUQkG9q8/79YvHQWfc5D4zR91x3N4v25CD pTmiphZxh832bNWA8sDqSVxv9NG3ROt/1nmGbAxP6aSqY/R9i7RKegoTY5fQvDxaDZ+s WMNQ== X-Gm-Message-State: AOJu0YzCxd/TtQFM24xBokNhUmY+KYKl/C2fX9SXQtncbBIcwvJK+laN wFmn/eIPBXA2byk+ONSlPeuKM3s8864PLV5AqCAeJA== X-Google-Smtp-Source: AGHT+IGDRk46qZv36slJWtlXK73oPVGdXOJOIjtN17J5FoKmLBe9W/e8+7XfMwPx2Q2qmS94GiXCTA== X-Received: by 2002:a81:8541:0:b0:57a:cf8:5b4 with SMTP id v62-20020a818541000000b0057a0cf805b4mr13924282ywf.51.1693946221909; Tue, 05 Sep 2023 13:37:01 -0700 (PDT) Received: from localhost (104-178-186-189.lightspeed.milwwi.sbcglobal.net. [104.178.186.189]) by smtp.gmail.com with ESMTPSA id v139-20020a814891000000b0058c55d40765sm3328532ywa.106.2023.09.05.13.37.01 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 05 Sep 2023 13:37:01 -0700 (PDT) Date: Tue, 5 Sep 2023 16:37:00 -0400 From: Taylor Blau To: git@vger.kernel.org Cc: Junio C Hamano , Jeff King , Patrick Steinhardt Subject: [PATCH 8/8] builtin/repack.c: extract common cruft pack loop Message-ID: References: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org When generating the list of packs to store in a MIDX (when given the `--write-midx` option), we include any cruft packs both during --geometric and non-geometric repacks. But the rules for when we do and don't have to check whether any of those cruft packs were queued for deletion differ slightly between the two cases. But the two can be unified, provided there is a little bit of extra detail added in the comment to clarify when it is safe to avoid checking for any pending deletions (and why it is OK to do so even when not required). Signed-off-by: Taylor Blau --- builtin/repack.c | 30 ++++++++++++++++++------------ 1 file changed, 18 insertions(+), 12 deletions(-) diff --git a/builtin/repack.c b/builtin/repack.c index 6110598a69..e6a1ef9a09 100644 --- a/builtin/repack.c +++ b/builtin/repack.c @@ -695,25 +695,31 @@ static void midx_included_packs(struct string_list *include, string_list_insert(include, strbuf_detach(&buf, NULL)); } - for_each_string_list_item(item, &existing->cruft_packs) { - /* - * no need to check for deleted packs, since we're - * not doing an ALL_INTO_ONE repack - */ - string_list_insert(include, xstrfmt("%s.idx", item->string)); - } } else { for_each_string_list_item(item, &existing->non_kept_packs) { if (item->util) continue; string_list_insert(include, xstrfmt("%s.idx", item->string)); } + } - for_each_string_list_item(item, &existing->cruft_packs) { - if (item->util) - continue; - string_list_insert(include, xstrfmt("%s.idx", item->string)); - } + for_each_string_list_item(item, &existing->cruft_packs) { + /* + * When doing a --geometric repack, there is no need to check + * for deleted packs, since we're by definition not doing an + * ALL_INTO_ONE repack (hence no packs will be deleted). + * Otherwise we must check for and exclude any packs which are + * enqueued for deletion. + * + * So we could omit the conditional below in the --geometric + * case, but doing so is unnecessary since no packs are marked + * as pending deletion (since we only call + * `mark_packs_for_deletion()` when doing an all-into-one + * repack). + */ + if (item->util) + continue; + string_list_insert(include, xstrfmt("%s.idx", item->string)); } }