From patchwork Wed Oct 12 13:51:12 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Christian Couder X-Patchwork-Id: 13005073 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 2A045C4332F for ; Wed, 12 Oct 2022 13:52:24 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229987AbiJLNwW (ORCPT ); Wed, 12 Oct 2022 09:52:22 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:48714 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229809AbiJLNwP (ORCPT ); Wed, 12 Oct 2022 09:52:15 -0400 Received: from mail-wr1-x436.google.com (mail-wr1-x436.google.com [IPv6:2a00:1450:4864:20::436]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 8A520CF86A for ; Wed, 12 Oct 2022 06:52:12 -0700 (PDT) Received: by mail-wr1-x436.google.com with SMTP id u10so26357369wrq.2 for ; Wed, 12 Oct 2022 06:52:12 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=yNRIoDjURwK5311I9SWzJpQEc3gjGouBtnKfgO4kKPk=; b=qi1Za0HyRFDLwj7AXklTCRLHzBI2wvbQL5IjY03Awyxm8/SuIpFP1+m8DwYRLN6AuD aDfKaJU3TuaGnmqF+TJ3Y5tdMzbzArQeGlO5wXm+RN36IXFYF8p4QFfsKG5op3es6UKG iIWOircdLTuwHSfkWIWx8FS4oeK0wM/xyYE8mz1w1MVyHhZKyo0olZh7RiGDC2yk+1S+ hOCoOiEUCBMprkV//ybvZJ7MUmVTTZA0FFQQ3wHDGh4HQwtWZpEFJQW8/AOADAsP7J6B cDbMX6FTjNNw7lB49XVFiY+Wv1eKQnhBQRIflb6rJjj94NxUdgQtPwKSsUg06yL7139R x1zQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=yNRIoDjURwK5311I9SWzJpQEc3gjGouBtnKfgO4kKPk=; b=lVm8X4RmLTUPXH9z0xXThGf39wdO5msaO0uxprEotrFYjBGHYF87D46l59luwn7jVE H2w0RKeYXhMKBeuvl5rT/4+mhpw9AqpVsUivNC7i3CItpZAYECjkIbO4PhYzZ5AKxkus fsi9iWD/hwYdb+fKAED6dMBY30TnfDGsOZkXdN068+I1GFjr0yCPFYyEzBCXEJWFU9ZM jKGVrI4g1jQ+sk8IA30YeI32kjI1xIdieATfS7TzMrjeYr/DMFGqTkoVyvawU3nfNc9e j3XqWXLhY7d8MVDJqMwhdqcwvor7xaBGs7bsWFtDnSQGzzQF081sFKzQi8L5P2cRcgfq KZ5g== X-Gm-Message-State: ACrzQf0RQOOqrDWk7CV6JtaOSASI0iFbORq5/w/jkekCz+Y3E6ZrxqAU mIkZIBgKK9YgFI/DOAbr++ZTCuOmqKA= X-Google-Smtp-Source: AMsMyM646df/KmBfpEuqlGvzHRTyL4NXNoj3GgMosDcTCoDIwiEcGRVdMVTkPm/Mqk4AGhbievGStw== X-Received: by 2002:adf:e381:0:b0:22e:335b:7c60 with SMTP id e1-20020adfe381000000b0022e335b7c60mr18663345wrm.332.1665582730647; Wed, 12 Oct 2022 06:52:10 -0700 (PDT) Received: from Precision-5550.. ([2a01:e0a:595:6400::ae22:851d]) by smtp.gmail.com with ESMTPSA id m5-20020a056000180500b0022f40a2d06esm11396284wrh.35.2022.10.12.06.52.09 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 12 Oct 2022 06:52:10 -0700 (PDT) From: Christian Couder To: git@vger.kernel.org Cc: Junio C Hamano , John Cai , Jonathan Tan , Jonathan Nieder , Taylor Blau , Christian Couder Subject: [PATCH 1/3] pack-objects: allow --filter without --stdout Date: Wed, 12 Oct 2022 15:51:12 +0200 Message-Id: <20221012135114.294680-2-christian.couder@gmail.com> X-Mailer: git-send-email 2.38.0.4.g7f9724c7bf.dirty In-Reply-To: <20221012135114.294680-1-christian.couder@gmail.com> References: <20221012135114.294680-1-christian.couder@gmail.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org From: Christian Couder 9535ce7337 (pack-objects: add list-objects filtering, 2017-11-21) taught pack-objects to use --filter, but required the use of --stdout since a partial clone mechanism was not yet in place to handle missing objects. Since then, changes like 9e27beaa23 (promisor-remote: implement promisor_remote_get_direct(), 2019-06-25) and others added support to dynamically fetch objects that were missing. Remove the --stdout requirement so that in a follow-up commit, repack can pass --filter to pack-objects to omit certain objects from the resulting packfile. Signed-off-by: John Cai Signed-off-by: Christian Couder --- builtin/pack-objects.c | 6 +----- 1 file changed, 1 insertion(+), 5 deletions(-) diff --git a/builtin/pack-objects.c b/builtin/pack-objects.c index 3658c05caf..d707e6230d 100644 --- a/builtin/pack-objects.c +++ b/builtin/pack-objects.c @@ -4385,12 +4385,8 @@ int cmd_pack_objects(int argc, const char **argv, const char *prefix) if (!rev_list_all || !rev_list_reflog || !rev_list_index) unpack_unreachable_expiration = 0; - if (pfd.have_revs && pfd.revs.filter.choice) { - if (!pack_to_stdout) - die(_("cannot use --filter without --stdout")); - if (stdin_packs) + if (pfd.have_revs && pfd.revs.filter.choice && stdin_packs) die(_("cannot use --filter with --stdin-packs")); - } if (stdin_packs && use_internal_rev_list) die(_("cannot use internal rev list with --stdin-packs")); From patchwork Wed Oct 12 13:51:13 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Christian Couder X-Patchwork-Id: 13005074 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 25402C4332F for ; Wed, 12 Oct 2022 13:52:27 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229997AbiJLNwZ (ORCPT ); Wed, 12 Oct 2022 09:52:25 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:48752 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229989AbiJLNwR (ORCPT ); Wed, 12 Oct 2022 09:52:17 -0400 Received: from mail-wm1-x32e.google.com (mail-wm1-x32e.google.com [IPv6:2a00:1450:4864:20::32e]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id B77FB61B11 for ; Wed, 12 Oct 2022 06:52:15 -0700 (PDT) Received: by mail-wm1-x32e.google.com with SMTP id fn7-20020a05600c688700b003b4fb113b86so1286741wmb.0 for ; Wed, 12 Oct 2022 06:52:15 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=53spqZpDiPruAYD3KZryzlHvG7q1FXbKidPR7awV7jE=; b=lkFlRgs3q89NdN0GHvSjfy7lijjtpeV6EptSVjeBpKjzetX0356g0xkCfPWcdOQyz4 RrcISoGRhEzN/pOiymNfLnnsSjLTzRdNqthjPZ8a0/IBYBDyLFPG75C8aH0F1Ip/Mka6 d/ugRm5TkoB4iawYbU0keP2I5yZ53Luek7d6ROIma9sdhBlWyWDsqg8Gdp1wwNdlUBtk FFTWp7lytFAdsKPrrSnuVZzpNrqP9dTxT2eZZ5qirhmG44p88Zdf93N9d9ETtzNiX3X1 jBL//7Gcv8wl8LRJIWLmWH7OY2ru6qI2ICxm93uwMdJaXShgH/L4kkRwXAuCfuDVl2Cx zxRw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=53spqZpDiPruAYD3KZryzlHvG7q1FXbKidPR7awV7jE=; b=YOrJTfz0BIdm5E7UaMNQqaJ6H/4SML70v2UfD03NoU2slrHUo1wt3rxiqKK88bykX2 ScU2Bi05ldWj5PBcbW4DDi2n6zIjFfXlO+m5F77PGRvTtpimeVE0f3e9ql7vHKQteaPj 8Px6fDtcoYs/L4ujiJni4LGr4uouefHha1u2riyXROTReMyfuTMlKBT9MBTNgP1a3HWV f0nLcHs959fasuXPwhioJ0rVlHgSt790uHpJnV+AViN7NVgO1JBedCQPMmz7YswcDwXE avr7qOT0lSEnk/Xuhjq2iQEgsOjMCCuOJOWSAqvxy+CYuSiK814MGlKF9qQndXrQSARo j8vg== X-Gm-Message-State: ACrzQf2KYqhTp358pSfRp+9+EZrloMAXsKb280QT7c8+fVkP7BHUupU6 bUUrINV7WzefG0N/X7X598Jgcq4hCls= X-Google-Smtp-Source: AMsMyM7fd1qBqp6CQjI7u2zIe42btcRTiaduMJRb9v33gn6HA8Cb5RrMaLfJt3lIO9vBcgIJhbtzEA== X-Received: by 2002:a05:600c:3585:b0:3b4:a308:1581 with SMTP id p5-20020a05600c358500b003b4a3081581mr2937319wmq.77.1665582733910; Wed, 12 Oct 2022 06:52:13 -0700 (PDT) Received: from Precision-5550.. ([2a01:e0a:595:6400::ae22:851d]) by smtp.gmail.com with ESMTPSA id m5-20020a056000180500b0022f40a2d06esm11396284wrh.35.2022.10.12.06.52.12 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 12 Oct 2022 06:52:13 -0700 (PDT) From: Christian Couder To: git@vger.kernel.org Cc: Junio C Hamano , John Cai , Jonathan Tan , Jonathan Nieder , Taylor Blau , Christian Couder Subject: [PATCH 2/3] repack: add --filter= option Date: Wed, 12 Oct 2022 15:51:13 +0200 Message-Id: <20221012135114.294680-3-christian.couder@gmail.com> X-Mailer: git-send-email 2.38.0.4.g7f9724c7bf.dirty In-Reply-To: <20221012135114.294680-1-christian.couder@gmail.com> References: <20221012135114.294680-1-christian.couder@gmail.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org From: Christian Couder After cloning with --filter=, for example to avoid getting unneeded large files on a user machine, it's possible that some of these large files still get fetched for some reasons (like checking out old branches) over time. In this case the repo size could grow too much for no good reason and `git repack --filter=` would be useful to remove the unneeded large files. This command could be dangerous to use though, as it might remove local objects that haven't been pushed which would lose data and corrupt the repo. On a server this command could also corrupt a repo unless ALL the removed objects aren't already available in another remote that clients can access. To avoid as much as possible data to be lost and repos to be corrupted, let's warn users and ask them to confirm that they really want to use this command. If no terminal is used, let's just die() for now. A follow-up commit will introduce a --force option that will allow using this option when no terminal is available. It will be easier to test that --filter is working correctly in the follow-up commit that adds --force, so let's just test that we die() if no terminal is used for now. Signed-off-by: John Cai Signed-off-by: Christian Couder --- Documentation/git-repack.txt | 7 ++++++ builtin/repack.c | 41 ++++++++++++++++++++++++++++++------ t/t7700-repack.sh | 5 +++++ 3 files changed, 46 insertions(+), 7 deletions(-) diff --git a/Documentation/git-repack.txt b/Documentation/git-repack.txt index 0bf13893d8..230f176e10 100644 --- a/Documentation/git-repack.txt +++ b/Documentation/git-repack.txt @@ -137,6 +137,13 @@ depth is 4095. a larger and slower repository; see the discussion in `pack.packSizeLimit`. +--filter=:: + Omits certain objects (usually blobs) from the resulting + packfile. WARNING: this could easily corrupt the current repo + and lose data if ANY of the omitted objects hasn't been + already pushed to a remote. See linkgit:git-rev-list[1] for + valid `` forms. + -b:: --write-bitmap-index:: Write a reachability bitmap index as part of the repack. This diff --git a/builtin/repack.c b/builtin/repack.c index a5bacc7797..0a93f38da4 100644 --- a/builtin/repack.c +++ b/builtin/repack.c @@ -17,6 +17,7 @@ #include "pack.h" #include "pack-bitmap.h" #include "refs.h" +#include "prompt.h" #define ALL_INTO_ONE 1 #define LOOSEN_UNREACHABLE 2 @@ -50,6 +51,7 @@ struct pack_objects_args { const char *depth; const char *threads; const char *max_pack_size; + const char *filter; int no_reuse_delta; int no_reuse_object; int quiet; @@ -201,6 +203,8 @@ static void prepare_pack_objects(struct child_process *cmd, strvec_pushf(&cmd->args, "--threads=%s", args->threads); if (args->max_pack_size) strvec_pushf(&cmd->args, "--max-pack-size=%s", args->max_pack_size); + if (args->filter) + strvec_pushf(&cmd->args, "--filter=%s", args->filter); if (args->no_reuse_delta) strvec_pushf(&cmd->args, "--no-reuse-delta"); if (args->no_reuse_object) @@ -268,6 +272,13 @@ static unsigned populate_pack_exts(char *name) return ret; } +static void write_promisor_file_1(char *p) +{ + char *promisor_name = mkpathdup("%s-%s.promisor", packtmp, p); + write_promisor_file(promisor_name, NULL, 0); + free(promisor_name); +} + static void repack_promisor_objects(const struct pack_objects_args *args, struct string_list *names) { @@ -299,7 +310,6 @@ static void repack_promisor_objects(const struct pack_objects_args *args, out = xfdopen(cmd.out, "r"); while (strbuf_getline_lf(&line, out) != EOF) { struct string_list_item *item; - char *promisor_name; if (line.len != the_hash_algo->hexsz) die(_("repack: Expecting full hex object ID lines only from pack-objects.")); @@ -316,13 +326,8 @@ static void repack_promisor_objects(const struct pack_objects_args *args, * concatenate the contents of all .promisor files instead of * just creating a new empty file. */ - promisor_name = mkpathdup("%s-%s.promisor", packtmp, - line.buf); - write_promisor_file(promisor_name, NULL, 0); - + write_promisor_file_1(line.buf); item->util = (void *)(uintptr_t)populate_pack_exts(item->string); - - free(promisor_name); } fclose(out); if (finish_command(&cmd)) @@ -786,6 +791,8 @@ int cmd_repack(int argc, const char **argv, const char *prefix) N_("limits the maximum number of threads")), OPT_STRING(0, "max-pack-size", &po_args.max_pack_size, N_("bytes"), N_("maximum size of each packfile")), + OPT_STRING(0, "filter", &po_args.filter, N_("args"), + N_("object filtering")), OPT_BOOL(0, "pack-kept-objects", &pack_kept_objects, N_("repack objects in packs marked with .keep")), OPT_STRING_LIST(0, "keep-pack", &keep_pack_list, N_("name"), @@ -818,6 +825,24 @@ int cmd_repack(int argc, const char **argv, const char *prefix) die(_("options '%s' and '%s' cannot be used together"), "--cruft", "-k"); } + if (po_args.filter) { + const char *yesno; + + if (!isatty(STDIN_FILENO)) + die (_("Repacking with a filter is not allowed " + "yet unless a terminal is used!")); + + /* + * TRANSLATORS: Make sure to include [y] and [N] in your translation. + * The program will only accept English input at this point. + */ + yesno = git_prompt(_("Repacking with a filter will lose data and corrupt the repo\n" + "if ANY of the filtered out object hasn't been already pushed!\n" + "Repack with a filter anyway [y/N]? "), PROMPT_ECHO); + if (tolower(*yesno) != 'y') + exit(1); + } + if (write_bitmaps < 0) { if (!write_midx && (!(pack_everything & ALL_INTO_ONE) || !is_bare_repository())) @@ -955,6 +980,8 @@ int cmd_repack(int argc, const char **argv, const char *prefix) if (line.len != the_hash_algo->hexsz) die(_("repack: Expecting full hex object ID lines only from pack-objects.")); string_list_append(&names, line.buf); + if (po_args.filter) + write_promisor_file_1(line.buf); } fclose(out); ret = finish_command(&cmd); diff --git a/t/t7700-repack.sh b/t/t7700-repack.sh index ca45c4cd2c..f8764d1dd9 100755 --- a/t/t7700-repack.sh +++ b/t/t7700-repack.sh @@ -237,6 +237,11 @@ test_expect_success 'auto-bitmaps do not complain if unavailable' ' test_must_be_empty actual ' +test_expect_success 'repacking with a filter is not allowed' ' + test_must_fail git repack -a -d --filter=blob:none 2>actual && + test_i18ngrep "Repacking with a filter is not allowed" actual +' + objdir=.git/objects midx=$objdir/pack/multi-pack-index From patchwork Wed Oct 12 13:51:14 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Christian Couder X-Patchwork-Id: 13005075 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 57C53C433FE for ; Wed, 12 Oct 2022 13:52:36 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230025AbiJLNwe (ORCPT ); Wed, 12 Oct 2022 09:52:34 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:48724 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229703AbiJLNwU (ORCPT ); Wed, 12 Oct 2022 09:52:20 -0400 Received: from mail-wr1-x433.google.com (mail-wr1-x433.google.com [IPv6:2a00:1450:4864:20::433]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 44C4FCD5D3 for ; Wed, 12 Oct 2022 06:52:19 -0700 (PDT) Received: by mail-wr1-x433.google.com with SMTP id bp11so13596910wrb.9 for ; Wed, 12 Oct 2022 06:52:19 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=+BuWtt731hYxc+21YZv1Oh27qLFhiG6Rg/CdXLit2ck=; b=PlJ1LdogroCQ08TbNp2zJfsA90Ic3HcIwcR0aCzmNfeyi7NoJsL8jpYljvm3vLnGfo 5fEYE59RgyXVJW7fjQ2DsO6rePvryKKFbTGAbwcoi5IK3/gyyUubruFeUqGSh5tLQnBj s4kx3c6gU5BOWoobOCE1DzHh2L3Ozxe21gcN44Ggrp8bpUxV9E32JxeLIeCxUlqYCUIu GjE652KJAA+s7fC9I14erayq76pfDOOxC+fd8E3EztKp0Y7I53o4IkIgsP+mZMYk+BmS pnK/6SgIH6cMefGUgyfU8rh7vQeIlToccN98J5XNyEYtw62e9RFl3PFclLmxnSdHpPL+ FGDQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=+BuWtt731hYxc+21YZv1Oh27qLFhiG6Rg/CdXLit2ck=; b=EMF2iJh0MG2Y3qRtmbEw5bmvA6W7XpTiww+uhyW21YrjWav1aY6vT38WkTuBVHaxg2 AzkvRCiMoaHYyV1umQmh1fH9DqbhQuKnLoT1dNUjHBr2ApMrRJrHrxF8Pe/5IOnyh3w7 F9N/1cNKot9e3v4Y+xMlJhpdSrs6Yre5+yUXYVi2UrrlfXSyt2DqV6f9M4figPR861li PkF3EGcREd6EnwcGflZViXeOPVeUjV0ZgeVxkJTzCsB313ec0tHTBiLfMMhnSzMVY+h3 YXFIq2fhkKo8hbEWMZ5gwDn0PLDydornjOMiCA9JmYMnNjXrsL6iZUXPlKnQXMAx3jQz aCiw== X-Gm-Message-State: ACrzQf1XIx3GSpd7gmFHyea0N41lr5hmTMLKbj6EMVyKEeEXcBw3sNPD GfYVsRlcQ6v7P1V7rCdA5QfMhVbluSY= X-Google-Smtp-Source: AMsMyM67Kt3NXOxR9a8JJZZSrrvMgtvcDyUmiZTHQC0bZ3XGBvXiqDNsbBPPnFYbdwYwKPHIJ4CiUg== X-Received: by 2002:a05:6000:1002:b0:22e:131c:4249 with SMTP id a2-20020a056000100200b0022e131c4249mr19649175wrx.50.1665582737479; Wed, 12 Oct 2022 06:52:17 -0700 (PDT) Received: from Precision-5550.. ([2a01:e0a:595:6400::ae22:851d]) by smtp.gmail.com with ESMTPSA id m5-20020a056000180500b0022f40a2d06esm11396284wrh.35.2022.10.12.06.52.15 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 12 Oct 2022 06:52:17 -0700 (PDT) From: Christian Couder To: git@vger.kernel.org Cc: Junio C Hamano , John Cai , Jonathan Tan , Jonathan Nieder , Taylor Blau , Christian Couder , Christian Couder Subject: [PATCH 3/3] repack: introduce --force to force filtering Date: Wed, 12 Oct 2022 15:51:14 +0200 Message-Id: <20221012135114.294680-4-christian.couder@gmail.com> X-Mailer: git-send-email 2.38.0.4.g7f9724c7bf.dirty In-Reply-To: <20221012135114.294680-1-christian.couder@gmail.com> References: <20221012135114.294680-1-christian.couder@gmail.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org A previous commit introduced --filter=, but disallowed it when stdin doesn't refer to a terminal, and asked the user when it does refer to a terminal. This is because this option is dangerous as it could lose data and corrupt the repo. In some cases people might want to run this command automatically when stdin desn't refer to a terminal or when no question should be asked. Let's introduce --force for this purpose. Unfortunately, we cannot use OPT__FORCE() to implement this because -f is already a repack option (to pass --no-reuse-delta to git-pack-objects), so we use OPT_BOOL() instead. This is also a good time to test that --filter works properly as it wasn't done in the previous commit that introduced --filter. Signed-off-by: Christian Couder --- Documentation/git-repack.txt | 12 +++++++++++- builtin/repack.c | 7 +++++-- t/t7700-repack.sh | 15 +++++++++++++++ 3 files changed, 31 insertions(+), 3 deletions(-) diff --git a/Documentation/git-repack.txt b/Documentation/git-repack.txt index 230f176e10..574f569fbe 100644 --- a/Documentation/git-repack.txt +++ b/Documentation/git-repack.txt @@ -142,7 +142,17 @@ depth is 4095. packfile. WARNING: this could easily corrupt the current repo and lose data if ANY of the omitted objects hasn't been already pushed to a remote. See linkgit:git-rev-list[1] for - valid `` forms. + valid `` forms. See also `--force` option below. + +--force:: + By default when using `--filter=` and stdin + refers to a terminal, the user will be warned and asked if the + filtering repack should really be launched. This tries to + avoid possible data loss and repo corruption. See `--filter` + option above. If stdin doesn't refer to a terminal, the repack + is aborted. `--force` allows the repack to go on anyway when + stdin doesn't refer to a terminal or when no question should + be asked. -b:: --write-bitmap-index:: diff --git a/builtin/repack.c b/builtin/repack.c index 0a93f38da4..3660dbb7a7 100644 --- a/builtin/repack.c +++ b/builtin/repack.c @@ -56,6 +56,7 @@ struct pack_objects_args { int no_reuse_object; int quiet; int local; + int force; }; static int repack_config(const char *var, const char *value, void *cb) @@ -793,6 +794,8 @@ int cmd_repack(int argc, const char **argv, const char *prefix) N_("maximum size of each packfile")), OPT_STRING(0, "filter", &po_args.filter, N_("args"), N_("object filtering")), + OPT_BOOL(0, "force", &po_args.force, + N_("force object filtering")), OPT_BOOL(0, "pack-kept-objects", &pack_kept_objects, N_("repack objects in packs marked with .keep")), OPT_STRING_LIST(0, "keep-pack", &keep_pack_list, N_("name"), @@ -825,12 +828,12 @@ int cmd_repack(int argc, const char **argv, const char *prefix) die(_("options '%s' and '%s' cannot be used together"), "--cruft", "-k"); } - if (po_args.filter) { + if (po_args.filter && !po_args.force) { const char *yesno; if (!isatty(STDIN_FILENO)) die (_("Repacking with a filter is not allowed " - "yet unless a terminal is used!")); + "unless a terminal is used or --force is passed!")); /* * TRANSLATORS: Make sure to include [y] and [N] in your translation. diff --git a/t/t7700-repack.sh b/t/t7700-repack.sh index f8764d1dd9..20727112b5 100755 --- a/t/t7700-repack.sh +++ b/t/t7700-repack.sh @@ -242,6 +242,21 @@ test_expect_success 'repacking with a filter is not allowed' ' test_i18ngrep "Repacking with a filter is not allowed" actual ' +test_expect_success 'repacking with a filter works when --force is passed' ' + test_when_finished "rm -rf server client" && + test_create_repo server && + git -C server config uploadpack.allowFilter true && + git -C server config uploadpack.allowAnySHA1InWant true && + test_commit -C server 1 && + git clone --bare --no-local server client && + git -C client config remote.origin.promisor true && + git -C client rev-list --objects --all --missing=print >objects && + test $(grep "^?" objects | wc -l) = 0 && + git -C client -c repack.writebitmaps=false repack -a -d --filter=blob:none --force && + git -C client rev-list --objects --all --missing=print >objects && + test $(grep "^?" objects | wc -l) = 1 +' + objdir=.git/objects midx=$objdir/pack/multi-pack-index