From patchwork Mon Mar 17 23:00:16 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Taylor Blau X-Patchwork-Id: 14020020 Received: from mail-qk1-f182.google.com (mail-qk1-f182.google.com [209.85.222.182]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 6968D14A4C6 for ; Mon, 17 Mar 2025 23:00:19 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.222.182 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1742252421; cv=none; b=ujaHFEfA3S6gYEcLjoa7U8npjyyJgQ7KqdgyULGnOaChEjY4YfPuWaW/So1i/NdPcyqR4Uwb+qUDqYdgf/3PR9W4w/jWUiDtBPLXULln8jLdh4s0QTnPpcktIwjXPqXphUNn7hwdbS0D5mR0uCrHXw3L+K8miKaRfrqfGq5Vyec= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1742252421; c=relaxed/simple; bh=43A7NGtKlP6KbXiN+qMR9+HF7Pp08wMfPX4fMM3Sm5I=; h=Date:From:To:Cc:Subject:Message-ID:References:MIME-Version: Content-Type:Content-Disposition:In-Reply-To; b=NuZlS4MF9Dtnkt/xbejep/zsW8po+KK67CKvebUawfYHB/zncmSAhPksxrev9O8or0/sVXgJ8w0HWdckpvE17tqeNlSPvSTfY4xK4g4/spuzMJqwu3YfUbq924EgtLNsX8tRRlZkVZ7IEnxZ3IE3gzOQxxYwDePp3sVOO97rllA= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=ttaylorr.com; spf=pass smtp.mailfrom=ttaylorr.com; dkim=pass (2048-bit key) header.d=ttaylorr-com.20230601.gappssmtp.com header.i=@ttaylorr-com.20230601.gappssmtp.com header.b=OdtXZUPT; arc=none smtp.client-ip=209.85.222.182 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=ttaylorr.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=ttaylorr.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=ttaylorr-com.20230601.gappssmtp.com header.i=@ttaylorr-com.20230601.gappssmtp.com header.b="OdtXZUPT" Received: by mail-qk1-f182.google.com with SMTP id af79cd13be357-7c56321b22cso509541385a.1 for ; Mon, 17 Mar 2025 16:00:19 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ttaylorr-com.20230601.gappssmtp.com; s=20230601; t=1742252418; x=1742857218; darn=vger.kernel.org; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:from:to:cc:subject:date:message-id:reply-to; bh=RLBmvjU4HjUgiBFZ3WorYsYR47/baowBM/NikWq7mF4=; b=OdtXZUPT6VrQm8omtAkcKs1QuGO/CtTvi9FhvPauZSIP0TEg+IBH9lQ5mBuCux6BkT 0YyA6xgjwDEu06E0496AR7PdFjc0LYgTFUSlFiXE/8d78xhpjg0JoG8dQL0OcJi/XA1g XVlSDM+syMpLgmCWyfJkBQsrm/k7l4eDw2hR/jpRnAYfnydWqoKyFEER4JhPu0Jy/msL t0ZlqaiqgIUc7GpWLlUGU6otOzNxZksdAwbnmNJqtU51IzGl6hzh2SqtDdXfQvYGRSry dHjvRpjAOu8fI4lg3RVNfG8PW0RR4aPYpUoqepLzBIq+VJB6x81ZE/VDPC9EksvJ5Vew wdMA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1742252418; x=1742857218; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:x-gm-message-state:from:to:cc:subject:date :message-id:reply-to; bh=RLBmvjU4HjUgiBFZ3WorYsYR47/baowBM/NikWq7mF4=; b=PSwCQsoK7Vge503C7OpbxD7nu1Dr3yDt1IsIy6CYWhDQ8jsBP7lZpfhk3MuphJS13I 2M0Ba4IS2o3EgsY02mV81QUVBh24aZBNs/+e3d1VXgb2p1wnAlV8eeyux0+nZMP3Jd8H 8+2bkRAMGs1c4RfltHBayE0yncixu+cUaeM/VmCyBeWuHKSpkjpxqa93siFd4bFJcUOM GB+AisH5sGFV49tM0lYQdL7/HjCnxqq2MsRGMt4J8lTBHVP7H9WERoq43xZxssvz8O3E BkgXTXWO8ydGR4Y2r0EZvlT25h076vI/C/PuOtetwRmfpYw3/2uNRy/fBa6nc/+nItaI XWhA== X-Gm-Message-State: AOJu0YzdNi3ftnm+AyJUZqTUSGXpiY0i347Gdt4Rv+2Sn+oT6NfI+7AA uMsuiMKGNjw+Uc8WaCpd+589YqammRIbRa7gbnfJAlpsnxrmW1TIgoxWQjeOwezlJuVIfWMjohp SstE= X-Gm-Gg: ASbGncuvYdnZBPfN7hWWFZo0Pxl8tCP2P0izFrsFnQQucOam5wnrGraI+bNuV4S4gJB 7oBbKA5IvNeqjxneHCEwLcLJ4jMN386NJmLrZDk2kPQfFX1EL4kYairJJhOp0YD50XMsedI0iqG ZSnUsGh7KkKrrqJVuvmgUgWtRd54KaL7N3MgwHDr77jb+5dVLh1nxXfpE595wAlDN8VUO2BRT5J GYUKg7+LDRx8P2BV5fPiHEy7vpe43Or3XQ70QO5BsCTRU8wyhOao+vErZxq6Gk9R8eNzW8l6rV9 aDfXI4uvaJ0TkkDT3N/a2bePGBHUhHiEA4zRhOJFImI15yRDX5LdE3OR61+1rCCtwZ9qGizVA1n YgDLT3oElWrkYkxET X-Google-Smtp-Source: AGHT+IG27kSjDOcMBcs7RYRRmKwFhaWxP/XesqpKheX2hv/CKJqGTX/jiGb/dTBQ94CAMP3LoS90Dg== X-Received: by 2002:a05:620a:1a83:b0:7c5:95cb:59ac with SMTP id af79cd13be357-7c59b88b89dmr212128585a.28.1742252417871; Mon, 17 Mar 2025 16:00:17 -0700 (PDT) Received: from localhost (104-178-186-189.lightspeed.milwwi.sbcglobal.net. [104.178.186.189]) by smtp.gmail.com with UTF8SMTPSA id af79cd13be357-7c573d9250asm636392985a.113.2025.03.17.16.00.17 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 17 Mar 2025 16:00:17 -0700 (PDT) Date: Mon, 17 Mar 2025 19:00:16 -0400 From: Taylor Blau To: git@vger.kernel.org Cc: Jeff King , Junio C Hamano , Elijah Newren Subject: [PATCH 1/5] t/t5329-pack-objects-cruft.sh: evict 'repack'-related tests Message-ID: <0aa8aa65c130fd62d38b4944c1ce2a97451b0064.1742252411.git.me@ttaylorr.com> References: Precedence: bulk X-Mailing-List: git@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: The cruft pack feature has two primary test scripts which exercise various parts of it, which are: - t5329-pack-objects-cruft.sh - t7704-repack-cruft.sh The former is designed to test low-level pack generation mechanics at the 'git pack-objects --cruft'-level, which is plumbing. The latter, on the other hand, is designed to test the user-facing behavior through 'git repack --cruft', which is porcelain (under the "ancillary manipulators" sub-section). At some point a handful of tests which should have been added to the latter script were instead written to the former. This isn't a huge deal, but rectifying it is straightforward. Move a handful of 'repack'-related tests out of t5329 and into their rightful home in t7704. Signed-off-by: Taylor Blau --- t/t5329-pack-objects-cruft.sh | 250 ---------------------------------- t/t7704-repack-cruft.sh | 250 ++++++++++++++++++++++++++++++++++ 2 files changed, 250 insertions(+), 250 deletions(-) diff --git a/t/t5329-pack-objects-cruft.sh b/t/t5329-pack-objects-cruft.sh index b71a0aef40..60dac8312d 100755 --- a/t/t5329-pack-objects-cruft.sh +++ b/t/t5329-pack-objects-cruft.sh @@ -360,43 +360,6 @@ test_expect_success 'expired objects are pruned' ' ) ' -test_expect_success 'repack --cruft generates a cruft pack' ' - git init repo && - test_when_finished "rm -fr repo" && - ( - cd repo && - - test_commit reachable && - git branch -M main && - git checkout --orphan other && - test_commit unreachable && - - git checkout main && - git branch -D other && - git tag -d unreachable && - # objects are not cruft if they are contained in the reflogs - git reflog expire --all --expire=all && - - git rev-list --objects --all --no-object-names >reachable.raw && - git cat-file --batch-all-objects --batch-check="%(objectname)" >objects && - sort reachable && - comm -13 reachable objects >unreachable && - - git repack --cruft -d && - - cruft=$(basename $(ls $packdir/pack-*.mtimes) .mtimes) && - pack=$(basename $(ls $packdir/pack-*.pack | grep -v $cruft) .pack) && - - git show-index <$packdir/$pack.idx >actual.raw && - cut -f2 -d" " actual.raw | sort >actual && - test_cmp reachable actual && - - git show-index <$packdir/$cruft.idx >actual.raw && - cut -f2 -d" " actual.raw | sort >actual && - test_cmp unreachable actual - ) -' - test_expect_success 'loose objects mtimes upsert others' ' git init repo && test_when_finished "rm -fr repo" && @@ -470,219 +433,6 @@ test_expect_success 'expiring cruft objects with git gc' ' ) ' -test_expect_success 'cruft packs are not included in geometric repack' ' - git init repo && - test_when_finished "rm -fr repo" && - ( - cd repo && - - test_commit reachable && - git repack -Ad && - git branch -M main && - - git checkout --orphan other && - test_commit cruft && - git repack -d && - - git checkout main && - git branch -D other && - git tag -d cruft && - git reflog expire --all --expire=all && - - git repack --cruft && - - find $packdir -type f | sort >before && - git repack --geometric=2 -d && - find $packdir -type f | sort >after && - - test_cmp before after - ) -' - -test_expect_success 'repack --geometric collects once-cruft objects' ' - git init repo && - test_when_finished "rm -fr repo" && - ( - cd repo && - - test_commit reachable && - git repack -Ad && - git branch -M main && - - git checkout --orphan other && - git rm -rf . && - test_commit --no-tag cruft && - cruft="$(git rev-parse HEAD)" && - - git checkout main && - git branch -D other && - git reflog expire --all --expire=all && - - # Pack the objects created in the previous step into a cruft - # pack. Intentionally leave loose copies of those objects - # around so we can pick them up in a subsequent --geometric - # reapack. - git repack --cruft && - - # Now make those objects reachable, and ensure that they are - # packed into the new pack created via a --geometric repack. - git update-ref refs/heads/other $cruft && - - # Without this object, the set of unpacked objects is exactly - # the set of objects already in the cruft pack. Tweak that set - # to ensure we do not overwrite the cruft pack entirely. - test_commit reachable2 && - - find $packdir -name "pack-*.idx" | sort >before && - git repack --geometric=2 -d && - find $packdir -name "pack-*.idx" | sort >after && - - { - git rev-list --objects --no-object-names $cruft && - git rev-list --objects --no-object-names reachable..reachable2 - } >want.raw && - sort want.raw >want && - - pack=$(comm -13 before after) && - git show-index <$pack >objects.raw && - - cut -d" " -f2 objects.raw | sort >got && - - test_cmp want got - ) -' - -test_expect_success 'cruft repack with no reachable objects' ' - git init repo && - test_when_finished "rm -fr repo" && - ( - cd repo && - - test_commit base && - git repack -ad && - - base="$(git rev-parse base)" && - - git for-each-ref --format="delete %(refname)" >in && - git update-ref --stdin in && - git hash-object -w -t blob in -} - -find_pack () { - for idx in $(ls $packdir/pack-*.idx) - do - git show-index <$idx >out && - if grep -q "$1" out - then - echo $idx - fi || return 1 - done -} - -test_expect_success 'cruft repack with --max-pack-size' ' - git init max-pack-size && - ( - cd max-pack-size && - test_commit base && - - # two cruft objects which exceed the maximum pack size - foo=$(write_blob foo 1048576) && - bar=$(write_blob bar 1048576) && - test-tool chmtime --get -1000 \ - "$objdir/$(test_oid_to_path $foo)" >foo.mtime && - test-tool chmtime --get -2000 \ - "$objdir/$(test_oid_to_path $bar)" >bar.mtime && - git repack --cruft --max-pack-size=1M && - find $packdir -name "*.mtimes" >cruft && - test_line_count = 2 cruft && - - foo_mtimes="$(basename $(find_pack $foo) .idx).mtimes" && - bar_mtimes="$(basename $(find_pack $bar) .idx).mtimes" && - test-tool pack-mtimes $foo_mtimes >foo.actual && - test-tool pack-mtimes $bar_mtimes >bar.actual && - - echo "$foo $(cat foo.mtime)" >foo.expect && - echo "$bar $(cat bar.mtime)" >bar.expect && - - test_cmp foo.expect foo.actual && - test_cmp bar.expect bar.actual && - test "$foo_mtimes" != "$bar_mtimes" - ) -' - -test_expect_success 'cruft repack with pack.packSizeLimit' ' - ( - cd max-pack-size && - # repack everything back together to remove the existing cruft - # pack (but to keep its objects) - git repack -adk && - git -c pack.packSizeLimit=1M repack --cruft && - # ensure the same post condition is met when --max-pack-size - # would otherwise be inferred from the configuration - find $packdir -name "*.mtimes" >cruft && - test_line_count = 2 cruft && - for pack in $(cat cruft) - do - test-tool pack-mtimes "$(basename $pack)" >objects && - test_line_count = 1 objects || return 1 - done - ) -' - -test_expect_success 'cruft repack respects repack.cruftWindow' ' - git init repo && - test_when_finished "rm -fr repo" && - ( - cd repo && - - test_commit base && - - GIT_TRACE2_EVENT=$(pwd)/event.trace \ - git -c pack.window=1 -c repack.cruftWindow=2 repack \ - --cruft --window=3 && - - grep "pack-objects.*--window=2.*--cruft" event.trace - ) -' - -test_expect_success 'cruft repack respects --window by default' ' - git init repo && - test_when_finished "rm -fr repo" && - ( - cd repo && - - test_commit base && - - GIT_TRACE2_EVENT=$(pwd)/event.trace \ - git -c pack.window=2 repack --cruft --window=3 && - - grep "pack-objects.*--window=3.*--cruft" event.trace - ) -' - -test_expect_success 'cruft repack respects --quiet' ' - git init repo && - test_when_finished "rm -fr repo" && - ( - cd repo && - - test_commit base && - GIT_PROGRESS_DELAY=0 git repack --cruft --quiet 2>err && - test_must_be_empty err - ) -' - test_expect_success 'cruft --local drops unreachable objects' ' git init alternate && git init repo && diff --git a/t/t7704-repack-cruft.sh b/t/t7704-repack-cruft.sh index 43d2947d28..cd452040ea 100755 --- a/t/t7704-repack-cruft.sh +++ b/t/t7704-repack-cruft.sh @@ -477,4 +477,254 @@ test_expect_success 'reachable packs are preferred over cruft ones' ' ) ' +test_expect_success 'repack --cruft generates a cruft pack' ' + git init repo && + test_when_finished "rm -fr repo" && + ( + cd repo && + + test_commit reachable && + git branch -M main && + git checkout --orphan other && + test_commit unreachable && + + git checkout main && + git branch -D other && + git tag -d unreachable && + # objects are not cruft if they are contained in the reflogs + git reflog expire --all --expire=all && + + git rev-list --objects --all --no-object-names >reachable.raw && + git cat-file --batch-all-objects --batch-check="%(objectname)" >objects && + sort reachable && + comm -13 reachable objects >unreachable && + + git repack --cruft -d && + + cruft=$(basename $(ls $packdir/pack-*.mtimes) .mtimes) && + pack=$(basename $(ls $packdir/pack-*.pack | grep -v $cruft) .pack) && + + git show-index <$packdir/$pack.idx >actual.raw && + cut -f2 -d" " actual.raw | sort >actual && + test_cmp reachable actual && + + git show-index <$packdir/$cruft.idx >actual.raw && + cut -f2 -d" " actual.raw | sort >actual && + test_cmp unreachable actual + ) +' + +test_expect_success 'cruft packs are not included in geometric repack' ' + git init repo && + test_when_finished "rm -fr repo" && + ( + cd repo && + + test_commit reachable && + git repack -Ad && + git branch -M main && + + git checkout --orphan other && + test_commit cruft && + git repack -d && + + git checkout main && + git branch -D other && + git tag -d cruft && + git reflog expire --all --expire=all && + + git repack --cruft && + + find $packdir -type f | sort >before && + git repack --geometric=2 -d && + find $packdir -type f | sort >after && + + test_cmp before after + ) +' + +test_expect_success 'repack --geometric collects once-cruft objects' ' + git init repo && + test_when_finished "rm -fr repo" && + ( + cd repo && + + test_commit reachable && + git repack -Ad && + git branch -M main && + + git checkout --orphan other && + git rm -rf . && + test_commit --no-tag cruft && + cruft="$(git rev-parse HEAD)" && + + git checkout main && + git branch -D other && + git reflog expire --all --expire=all && + + # Pack the objects created in the previous step into a cruft + # pack. Intentionally leave loose copies of those objects + # around so we can pick them up in a subsequent --geometric + # reapack. + git repack --cruft && + + # Now make those objects reachable, and ensure that they are + # packed into the new pack created via a --geometric repack. + git update-ref refs/heads/other $cruft && + + # Without this object, the set of unpacked objects is exactly + # the set of objects already in the cruft pack. Tweak that set + # to ensure we do not overwrite the cruft pack entirely. + test_commit reachable2 && + + find $packdir -name "pack-*.idx" | sort >before && + git repack --geometric=2 -d && + find $packdir -name "pack-*.idx" | sort >after && + + { + git rev-list --objects --no-object-names $cruft && + git rev-list --objects --no-object-names reachable..reachable2 + } >want.raw && + sort want.raw >want && + + pack=$(comm -13 before after) && + git show-index <$pack >objects.raw && + + cut -d" " -f2 objects.raw | sort >got && + + test_cmp want got + ) +' + +test_expect_success 'cruft repack with no reachable objects' ' + git init repo && + test_when_finished "rm -fr repo" && + ( + cd repo && + + test_commit base && + git repack -ad && + + base="$(git rev-parse base)" && + + git for-each-ref --format="delete %(refname)" >in && + git update-ref --stdin in && + git hash-object -w -t blob in +} + +find_pack () { + for idx in $(ls $packdir/pack-*.idx) + do + git show-index <$idx >out && + if grep -q "$1" out + then + echo $idx + fi || return 1 + done +} + +test_expect_success 'cruft repack with --max-pack-size' ' + git init max-pack-size && + ( + cd max-pack-size && + test_commit base && + + # two cruft objects which exceed the maximum pack size + foo=$(write_blob foo 1048576) && + bar=$(write_blob bar 1048576) && + test-tool chmtime --get -1000 \ + "$objdir/$(test_oid_to_path $foo)" >foo.mtime && + test-tool chmtime --get -2000 \ + "$objdir/$(test_oid_to_path $bar)" >bar.mtime && + git repack --cruft --max-pack-size=1M && + find $packdir -name "*.mtimes" >cruft && + test_line_count = 2 cruft && + + foo_mtimes="$(basename $(find_pack $foo) .idx).mtimes" && + bar_mtimes="$(basename $(find_pack $bar) .idx).mtimes" && + test-tool pack-mtimes $foo_mtimes >foo.actual && + test-tool pack-mtimes $bar_mtimes >bar.actual && + + echo "$foo $(cat foo.mtime)" >foo.expect && + echo "$bar $(cat bar.mtime)" >bar.expect && + + test_cmp foo.expect foo.actual && + test_cmp bar.expect bar.actual && + test "$foo_mtimes" != "$bar_mtimes" + ) +' + +test_expect_success 'cruft repack with pack.packSizeLimit' ' + ( + cd max-pack-size && + # repack everything back together to remove the existing cruft + # pack (but to keep its objects) + git repack -adk && + git -c pack.packSizeLimit=1M repack --cruft && + # ensure the same post condition is met when --max-pack-size + # would otherwise be inferred from the configuration + find $packdir -name "*.mtimes" >cruft && + test_line_count = 2 cruft && + for pack in $(cat cruft) + do + test-tool pack-mtimes "$(basename $pack)" >objects && + test_line_count = 1 objects || return 1 + done + ) +' + +test_expect_success 'cruft repack respects repack.cruftWindow' ' + git init repo && + test_when_finished "rm -fr repo" && + ( + cd repo && + + test_commit base && + + GIT_TRACE2_EVENT=$(pwd)/event.trace \ + git -c pack.window=1 -c repack.cruftWindow=2 repack \ + --cruft --window=3 && + + grep "pack-objects.*--window=2.*--cruft" event.trace + ) +' + +test_expect_success 'cruft repack respects --window by default' ' + git init repo && + test_when_finished "rm -fr repo" && + ( + cd repo && + + test_commit base && + + GIT_TRACE2_EVENT=$(pwd)/event.trace \ + git -c pack.window=2 repack --cruft --window=3 && + + grep "pack-objects.*--window=3.*--cruft" event.trace + ) +' + +test_expect_success 'cruft repack respects --quiet' ' + git init repo && + test_when_finished "rm -fr repo" && + ( + cd repo && + + test_commit base && + GIT_PROGRESS_DELAY=0 git repack --cruft --quiet 2>err && + test_must_be_empty err + ) +' + test_done From patchwork Mon Mar 17 23:00:19 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Taylor Blau X-Patchwork-Id: 14020021 Received: from mail-qk1-f177.google.com (mail-qk1-f177.google.com [209.85.222.177]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 4CE662054F2 for ; Mon, 17 Mar 2025 23:00:22 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.222.177 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1742252424; cv=none; b=DY8fx1xXIUE/Nr8VtboduEjIOpKxli/cLb5mwWxu6F9uXBiHuo0aFiuxvhsK1egRfe1jPI2WFjWjhXyhNX4XjhTDZgJC/HMNa1Tc444Jt1mqWkz9/ACY28zm2yqKx57GUyiDO5PiKnXgMlw/H339GIFwSSQ1sYX8fWrm/Lf3G4Q= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1742252424; c=relaxed/simple; bh=3D7qwAvdfdHfo7HJiMh5wPn2Rn3aPkDMmZRYZn2DqFs=; h=Date:From:To:Cc:Subject:Message-ID:References:MIME-Version: Content-Type:Content-Disposition:In-Reply-To; b=RGc5zx8nZloJ9JjZMwfeoM3JyHiAEYubxgwofZZt7ZF/LUATncO9NwaMjRkLardxLiL2483pGC7fb4kpLDf+EAoyUqIRdO6Cbr8i9lPQYAmZs1YsjTD1ja9KDomVMwryQrdfNAFQfdoFlaFOkwRKyD0QfOdgNM4Cv158cCT7YWI= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=ttaylorr.com; spf=pass smtp.mailfrom=ttaylorr.com; dkim=pass (2048-bit key) header.d=ttaylorr-com.20230601.gappssmtp.com header.i=@ttaylorr-com.20230601.gappssmtp.com header.b=bpY4U2MI; arc=none smtp.client-ip=209.85.222.177 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=ttaylorr.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=ttaylorr.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=ttaylorr-com.20230601.gappssmtp.com header.i=@ttaylorr-com.20230601.gappssmtp.com header.b="bpY4U2MI" Received: by mail-qk1-f177.google.com with SMTP id af79cd13be357-7c54f67db99so519094285a.1 for ; Mon, 17 Mar 2025 16:00:21 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ttaylorr-com.20230601.gappssmtp.com; s=20230601; t=1742252421; x=1742857221; darn=vger.kernel.org; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:from:to:cc:subject:date:message-id:reply-to; bh=7g+31hGIC6UafQoO2326PvHRLMcOv7XFl6ZWDT5n0ts=; b=bpY4U2MIRQflsKBGnsGbVWVlU5pyF9DluGFGrezK1NKrIwnxCnLt6z8NYBQdXO0glr Fu+QWt2k/nEF04MyjU1gfLy0voX03DFlqJ6B6/PkMR1q8DNhfs+PHiuwZU/N1x81wkFN jcYfKZSMeuqJv5E1lxS5H9Bw63pvzkUE/vGxBkcw7Jz4PHLPj4jsrlDBt/PzGw2KLbRd PInWUBjF1zk9vVQJUXgVIodtT9ViWUtiGvIoqn90I/VGsZX+SebqEggntpbs9HDaEkKb IyZha1dVTdoabNrD1AWsnOxTJtZzsOAkabJSrXM3cmqdJ9MYUz4bf+8tE7owm6Rb+dUp YDuA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1742252421; x=1742857221; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:x-gm-message-state:from:to:cc:subject:date :message-id:reply-to; bh=7g+31hGIC6UafQoO2326PvHRLMcOv7XFl6ZWDT5n0ts=; b=IfTFBfC9xMokgmCqIddtQIwV/G9UxEl1GltqO3O9oOd/cc5YZpX339r+dVPZt4cM7l v6/uMVBk3rqWIdnv954xD/3MebtMZvs72eB0NYI8LNgw/m0PtLztH7HnzIvBXsTLJoCT WKBOj+YLONcOyFaJ8HBsrO++e4SRv1ImjbIxYIcsmSePkmiDt1l6qKtKH8Do54PuGbEn rXTK97dlAEYoGce/QloleC9tZaw6RKu/twZ5bmlhmXLJ7nBljZqd4sAjcqWZXMjYKaH+ W/VGkl8oCYVm5Qo7ntX6m0QFhwJcDbTLDGzupaPIf/dZOC8UuABuW1cDnJwqrA0HKEDo y4dw== X-Gm-Message-State: AOJu0YwCMrLW4D0/snzRJ2ZxlYzYPKu6dCOOiezmxOlDE8fPRF0MFOZP 0ma26GuT+pe+XVF3Io6qOptSCjH8r2m3n2PO7w+b72zJ0WgLL6gkBS7GiAksWp5b3C5fypySuts ShwU= X-Gm-Gg: ASbGncs5SYMwhtQMMq4RIA6/5nrSPWLEXetbqbL3vB2RrBQWPsMFx0uE54kpfsCVX+7 zZBFNpoEqCRnyt3y8rBhRbfRjGme9dEnFsc9QnonicsFUGCgufYWTljix1QMdCRcH1RYmEQ4bSp lpuaAGzjr5Gak5NS/FWxnage+UuzpizPEJNYjt/gy7Wxs/otS75fek8nt+oXcCBGQl/zrvdNBKN IB/9XNwkYdOiOOAfZPx13pmxBJfcL9YCchzBzc04dfs38gVqqc648tApQziVDhLYwV1H6jqEeCa r5pcBoyRUK2V6Zko14YlpVjLGAsNnOx4J1CHabjyYY2FKDUs2TJL5JyxbempGSjFmxMCVRWUx5P QIgyHZQQV8SL0rZ7XkScJl6ND4bU= X-Google-Smtp-Source: AGHT+IGcjcvCIbsz0LYAovqNKASORcs8mVnigvAPo2m79eAhPitK4bk/owuNjW4xxmDMsg0XeoN84w== X-Received: by 2002:a05:620a:801c:b0:7c5:438b:78 with SMTP id af79cd13be357-7c59b5be82dmr242009685a.6.1742252420899; Mon, 17 Mar 2025 16:00:20 -0700 (PDT) Received: from localhost (104-178-186-189.lightspeed.milwwi.sbcglobal.net. [104.178.186.189]) by smtp.gmail.com with UTF8SMTPSA id af79cd13be357-7c573c9d8absm640194785a.55.2025.03.17.16.00.20 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 17 Mar 2025 16:00:20 -0700 (PDT) Date: Mon, 17 Mar 2025 19:00:19 -0400 From: Taylor Blau To: git@vger.kernel.org Cc: Jeff King , Junio C Hamano , Elijah Newren Subject: [PATCH 2/5] t/t7704-repack-cruft.sh: clarify wording in --max-cruft-size tests Message-ID: <5e8bd3e66e304f42d4e84ec2aa9f06b82321d64b.1742252411.git.me@ttaylorr.com> References: Precedence: bulk X-Mailing-List: git@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: Now that a number of new tests have landed in t7704, make sure that they all make sense and are testing the things they say they are. Things are mostly OK, but a handful of tests needed tweaks. Those tweaks are as follows: - Use the terms "too large" or "too small" in tests that exercise the '--max-cruft-size' behavior. This has historically been treated as a threshold beneath which to combine cruft packs, but that will change in a subsequent commit. Prepare for that by using a more generic term. - Remove references to "--max-cruft-size" in the freshening tests. These tests provide coverage of our ability to record updated mtimes for objects already in cruft packs whose mtimes are upserted from various sources (loose objects, finding that object in a new pack, another cruft pack, etc.). These have nothing to do with the '--max-cruft-size' feature, and in fact none of the tests even *use* '--max-cruft-size'. Name them appropriately to make it clear that these tests exercise freshening behavior, not '--max-cruft-size' behavior. Signed-off-by: Taylor Blau --- t/t7704-repack-cruft.sh | 20 ++++++++++---------- 1 file changed, 10 insertions(+), 10 deletions(-) diff --git a/t/t7704-repack-cruft.sh b/t/t7704-repack-cruft.sh index cd452040ea..e6e4c2fad8 100755 --- a/t/t7704-repack-cruft.sh +++ b/t/t7704-repack-cruft.sh @@ -149,7 +149,7 @@ generate_cruft_pack () { echo "$packdir/pack-$pack.mtimes" } -test_expect_success '--max-cruft-size creates new packs when above threshold' ' +test_expect_success '--max-cruft-size creates new packs when too large' ' git init max-cruft-size-large && ( cd max-cruft-size-large && @@ -173,7 +173,7 @@ test_expect_success '--max-cruft-size creates new packs when above threshold' ' ) ' -test_expect_success '--max-cruft-size combines existing packs when below threshold' ' +test_expect_success '--max-cruft-size combines existing packs when not too large' ' git init max-cruft-size-small && ( cd max-cruft-size-small && @@ -236,10 +236,10 @@ test_expect_success '--max-cruft-size combines smaller packs first' ' ) ' -test_expect_success 'setup --max-cruft-size with freshened objects' ' - git init max-cruft-size-freshen && +test_expect_success 'setup cruft with freshened objects' ' + git init cruft-freshen && ( - cd max-cruft-size-freshen && + cd cruft-freshen && test_commit base && git repack -ad && @@ -257,9 +257,9 @@ test_expect_success 'setup --max-cruft-size with freshened objects' ' ) ' -test_expect_success '--max-cruft-size with freshened objects (loose)' ' +test_expect_success 'cruft with freshened objects (loose)' ' ( - cd max-cruft-size-freshen && + cd cruft-freshen && # regenerate the object, setting its mtime to be more recent foo="$(generate_random_blob foo 64)" && @@ -275,9 +275,9 @@ test_expect_success '--max-cruft-size with freshened objects (loose)' ' ) ' -test_expect_success '--max-cruft-size with freshened objects (packed)' ' +test_expect_success 'cruft with freshened objects (packed)' ' ( - cd max-cruft-size-freshen && + cd cruft-freshen && # regenerate the object and store it in a packfile, # setting its mtime to be more recent @@ -304,7 +304,7 @@ test_expect_success '--max-cruft-size with freshened objects (packed)' ' ) ' -test_expect_success '--max-cruft-size with freshened objects (previously cruft)' ' +test_expect_success 'multi-cruft with freshened objects (previously cruft)' ' repo="max-cruft-size-threshold" && test_when_finished "rm -fr $repo" && From patchwork Mon Mar 17 23:00:22 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Taylor Blau X-Patchwork-Id: 14020022 Received: from mail-qt1-f180.google.com (mail-qt1-f180.google.com [209.85.160.180]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 19AD8205519 for ; Mon, 17 Mar 2025 23:00:24 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.160.180 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1742252426; cv=none; b=Tg2KxF+ybv9+NBecQLvyQdsFdHnlJ1Gfxuj7Ei51B5b5kUFXReL4/0PuVsrcXZDxnGuRgHceMhGNGB1WCIBGNgMp9QlIVLUGyNb2v15BfSGbdnQVB8q0bSm0K1M8XcyL5MciM+PmdjlPP/C8YlcI2ZyU3LsmnxeGMgPlfe7ImdY= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1742252426; c=relaxed/simple; bh=jxIPEvQvN8rP1sEFzCnUAj5g/QONuIOnqem0GIk/KcA=; h=Date:From:To:Cc:Subject:Message-ID:References:MIME-Version: Content-Type:Content-Disposition:In-Reply-To; b=P1ZgNpN9oyhOz9tf3r0bPx11E3n+FoO83nkkidEEsvZu+ryd6PzhtdD/ZMQveqAeOxxstLWBuSZe+DWr0j2anVQTpoMjy0C+c9dQcjHfXL/AJVYMN9R4RoIW616wcLHJESzq/3ylaAJjTimTnBX0BR6YJg6yzUUAfV92bA4kBxA= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=ttaylorr.com; spf=pass smtp.mailfrom=ttaylorr.com; dkim=pass (2048-bit key) header.d=ttaylorr-com.20230601.gappssmtp.com header.i=@ttaylorr-com.20230601.gappssmtp.com header.b=ESHTx1Mc; arc=none smtp.client-ip=209.85.160.180 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=ttaylorr.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=ttaylorr.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=ttaylorr-com.20230601.gappssmtp.com header.i=@ttaylorr-com.20230601.gappssmtp.com header.b="ESHTx1Mc" Received: by mail-qt1-f180.google.com with SMTP id d75a77b69052e-474f0c1e1c6so44521161cf.1 for ; Mon, 17 Mar 2025 16:00:24 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ttaylorr-com.20230601.gappssmtp.com; s=20230601; t=1742252424; x=1742857224; darn=vger.kernel.org; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:from:to:cc:subject:date:message-id:reply-to; bh=/HOUuAcgjzlJgRpBo8OBeUrq/KUuEh+Kj5YMB4slWRk=; b=ESHTx1McqI8nvUupUwYlu2y4Z7KdsoUU4nPGl7iMiegpc8E8Jf2jWsjRAOqs5RkhVy IYvU7IzP3MHTwKyeoteb+iblNOQOBDWoYuaLqH9HzRysECIJ39NTr4LlmcwSL37jAQhl 4REF1q9cc3nK9aQfMg/o5AmAmn8a8q5XYj3R8q7+yCbQZh33g4AiyN4iO+Qd74EN5dBI CtTIsyti5kWmz0HM/PArLGybQIXfn5cXLceupoTu8oPDiPgOB9yUMKp8lua7rvT9stD0 FAYG5DrUwbQbmxzTJPnbUUkoD81TqeJ4BU0WQ5COK2vNpa3fbxdwsk6bT81UwvyPxEhE zZag== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1742252424; x=1742857224; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:x-gm-message-state:from:to:cc:subject:date :message-id:reply-to; bh=/HOUuAcgjzlJgRpBo8OBeUrq/KUuEh+Kj5YMB4slWRk=; b=NhzlwLyA0sD+7CMBFKyA8tmdQfeh8EeqCRgK1FIDwc3eLxcenZ2xRTKGvpBNgUzIRL I0zXDxNx9I/fNQgWZCPwH4D20g3bTP4KfKo4/Jwe0sw8X9iXpd7A+vsBFTGLrka6QTCr SpEUmfjfWug4V6jK8wseGdBeLe9hYc2+Y1N0R6j/hnBoxOfUhWhPlrTnUngVD5eKYdIv mIVXsFjd3qNWZGcYU5TO24r1aQmyUpRXxWlxebL1uoGJhdfIJv0Pp1e75KgHBPaRsali BN3FK3B+BGay2PFk2sMY2rbRdTqkvgbHqGa2XeR+t2EKA2N68Yl1jPBMhOBpojyNJ6j+ 4tQw== X-Gm-Message-State: AOJu0YyYCRHjHlvG10FCIaEbr4D4yErkseMOfYACn4EqHAr82qAlAmHg i2y7jYSC5GV3YUgSXw+w/Vt+qsmUXKmYGGlM4n7/qWD/EF+PysFbTfptKhiIv08YNpK3HTD0GSe ttUk= X-Gm-Gg: ASbGnctES8X+SX1TKHUvqxxEhswQQkGyBZiXPS+XPaj7PmpPNOj7wV+mCuJpy/fHpp6 VxZPFtLgF8kJUJxKtcw0tJ2uCIAolTQMEvFPHHhv9rLVDQVZdjvOtMBYSFzRw5xWRbVYEvrq4mo +JCNxkBptmdGFa+YpbI9+++abHChbMNPAIgG0lf09obvN9tG7bQ/Mb0+u4Pc3EEx7p6ddi8cWvZ pNSzuBSY0iHbz3iTp+ZRs5jyBVAorc3X8nuGfFPFYJ8YCRn+2p10cTlSDaPUBJommxnzSQ4o+BG JU42FebDwdf2r31wE5bS1KtIsir1CACcptblqru+y6qxIH75xbRCwNWpWYTDvCwBt8LqLCZoxDH 4a/AulkKpIVV2BBOk X-Google-Smtp-Source: AGHT+IFhEwbUDnWhZ0lB1ZotUIYbwn9QP6w5O6dQPJSZ2E6Ou5WXjSpKoYJRRAil2Z01D+2YF348Jw== X-Received: by 2002:a05:622a:548a:b0:474:fe31:a239 with SMTP id d75a77b69052e-476c812361amr250339221cf.3.1742252423891; Mon, 17 Mar 2025 16:00:23 -0700 (PDT) Received: from localhost (104-178-186-189.lightspeed.milwwi.sbcglobal.net. [104.178.186.189]) by smtp.gmail.com with UTF8SMTPSA id d75a77b69052e-476bb7f21d9sm60724851cf.49.2025.03.17.16.00.23 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 17 Mar 2025 16:00:23 -0700 (PDT) Date: Mon, 17 Mar 2025 19:00:22 -0400 From: Taylor Blau To: git@vger.kernel.org Cc: Jeff King , Junio C Hamano , Elijah Newren Subject: [PATCH 3/5] t/t7704-repack-cruft.sh: consolidate `write_blob()` Message-ID: References: Precedence: bulk X-Mailing-List: git@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: A previous commit moved a handful of tests from a different script into t7704, including one that relies on generating random blobs. Incidentally, the original home of this test defined its own helper "write_blob" for doing so, which is identical in function to our "generate_random_blob" (and is slightly inferior to the latter, which cleans up after itself). Rewrite the test that uses "write_blob" to no longer do so and then remove the function. Signed-off-by: Taylor Blau --- t/t7704-repack-cruft.sh | 9 ++------- 1 file changed, 2 insertions(+), 7 deletions(-) diff --git a/t/t7704-repack-cruft.sh b/t/t7704-repack-cruft.sh index e6e4c2fad8..3fd5aa6089 100755 --- a/t/t7704-repack-cruft.sh +++ b/t/t7704-repack-cruft.sh @@ -618,11 +618,6 @@ test_expect_success 'cruft repack with no reachable objects' ' ) ' -write_blob () { - test-tool genrandom "$@" >in && - git hash-object -w -t blob in -} - find_pack () { for idx in $(ls $packdir/pack-*.idx) do @@ -641,8 +636,8 @@ test_expect_success 'cruft repack with --max-pack-size' ' test_commit base && # two cruft objects which exceed the maximum pack size - foo=$(write_blob foo 1048576) && - bar=$(write_blob bar 1048576) && + foo=$(generate_random_blob foo 1048576) && + bar=$(generate_random_blob bar 1048576) && test-tool chmtime --get -1000 \ "$objdir/$(test_oid_to_path $foo)" >foo.mtime && test-tool chmtime --get -2000 \ From patchwork Mon Mar 17 23:00:25 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Taylor Blau X-Patchwork-Id: 14020023 Received: from mail-qk1-f173.google.com (mail-qk1-f173.google.com [209.85.222.173]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 68AEA204F80 for ; Mon, 17 Mar 2025 23:00:28 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.222.173 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1742252430; cv=none; b=hGuzV13IrO3LbmLGcKKBzj918HM+7skmRX9H71HjYbdEBkRhA36ZaAWAldt4Vo+g3t6cJG+RsPRJ8Lq1rKDCClJKHmhb6REwRdyqiIwML2bYksNFx1yRdjtWmj5fTRpSAt6LiS97NSImlCXe8JHCgHbFqHbzWIG0iwRVCNZVjPI= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1742252430; c=relaxed/simple; bh=28VXWY3SMzWU/0pd146WHNXLrh7IynzXWysJfFj8lZo=; h=Date:From:To:Cc:Subject:Message-ID:References:MIME-Version: Content-Type:Content-Disposition:In-Reply-To; b=U5MGER9M8g0un3GxGjFLlD9nlgD8hAgOgLKK6+9OY6RNOsUa7DMR/bCwwVUDFOBRQotmHIdcaQCoOpp4L3a1VtdoKwXqHoxbZmdoScU2yNxng5NFf0SWuDBgH1ob4eel4+R8QP2pfTD9E/uORJdHxfRvJhMy3ZxH5ibrJnx3nV4= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=ttaylorr.com; spf=pass smtp.mailfrom=ttaylorr.com; dkim=pass (2048-bit key) header.d=ttaylorr-com.20230601.gappssmtp.com header.i=@ttaylorr-com.20230601.gappssmtp.com header.b=UMW734dC; arc=none smtp.client-ip=209.85.222.173 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=ttaylorr.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=ttaylorr.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=ttaylorr-com.20230601.gappssmtp.com header.i=@ttaylorr-com.20230601.gappssmtp.com header.b="UMW734dC" Received: by mail-qk1-f173.google.com with SMTP id af79cd13be357-7c5675dec99so548216285a.0 for ; Mon, 17 Mar 2025 16:00:28 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ttaylorr-com.20230601.gappssmtp.com; s=20230601; t=1742252427; x=1742857227; darn=vger.kernel.org; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:from:to:cc:subject:date:message-id:reply-to; bh=D14CWGS87GWsH4eZ7PAOHfxNDssTh4b/MuV9m1hgmKk=; b=UMW734dCtQIF83bpWMJ4foE21DDoldPBMgmJ7YLJImkqWo2+W7nPC51G6uPEpLfNps 25651ZuYdTnO+TjDPKEF5xP+dP+q8Up61RjSFv85pOpXOS0OERhrNEW3zezcb0oPUTvH tZAsp/VcL15ZQHCAE6a/aGgfB5jIMEV7H4bMWiu3Lgle8hfnO0HH/eSD2Gu8RlusAe1x Cpj3b9KWZR97Uh0fUy3ngfKSQls/Ne+WVKVxMBGbNDEKGHH6IxChHr497RylTCynoKHr Mo+HC50mqSElKeuRtiSq76FpJBy66F77o2xZHjWfG6ZiIDv7PRpxDP+XWUuPoREP8+W7 X1mQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1742252427; x=1742857227; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:x-gm-message-state:from:to:cc:subject:date :message-id:reply-to; bh=D14CWGS87GWsH4eZ7PAOHfxNDssTh4b/MuV9m1hgmKk=; b=O7TX/5E5oNhZchskY0yX5Wa59C4hFMB4tGIwNs/YJsSu3tbTeDIk0DG+KHSIXBMB4O xaUWiX6uUQQAlojJJZDGJW1OoJqDLwH/g+FXXKyMvsOcqrVYP5hIZiU2EOUjIFJgeum2 uf5UOuQTgs5uckyFzP/0FNfMv3TtW8L2z1DKNmztAq4oPsnd3Ml487d6UtzVOZX0ONO9 ltokpeo1LY7bBPf0OTroGFdhzvBqpFT46iZJtD+CPCxhU0NxZwfhFSpC4DKJGkfDUw0l AkwCKvBiitMdxKXcTkzXZhe5yfK+RDVbH4FoevkW/GYrpR3gTbhtQhbNwr45ttHaPejq xUMQ== X-Gm-Message-State: AOJu0YxHhiCr0Jmm2z92evDRn0FCOevMm8eeiTrAFTWmywQ7jJy6fGMB rJsIGaTKesC1TJuv2qDMve7ygWA14IX9byb4YauHAAnmUaP10ZT2lwJiqPfXEcZ7FXGeWj2XhCu ewnk= X-Gm-Gg: ASbGncvBlwGeYLNUxWOIYcEqPDU/mR8VUaMClmRMQXG4gWnNpaN0VcxLHAUZcbTWd0Q dCIucThzHnuhc6CM9qRjWEhizMyW24B219QwKbn5Sefbu7u0f6iFr2E5iWMped59/OlYD8EpQFa JBZ0DloGAJSRPl3J2NNnTaJZkFaS4nn59gF3pfw805+kPO0YdwgdegVuhoTmTZvjY1a9Nna8OLq 07A39PamZSaw7gp6qbEMUiTjunQ9i9hzpY0iiD+QTxyGYm14g5w+UxP30Xe2cVXw6Byivw4as08 NUqoGNpOgZDgqzVyDclLINXIIVUZEQe0mevOjmAiRacL03miNiq7Nw3l00bechdfkrjzEqNQ3KN uqkCf/EfX2WplF9Ha X-Google-Smtp-Source: AGHT+IHYLLX0HDwzk8yZGp8uqDu1IYKELDQRp3CUbVFXqhcJ/8dE/j3oAy9Azif574GeCb6EzWfoKA== X-Received: by 2002:a05:620a:1a8d:b0:7c5:6410:3a6 with SMTP id af79cd13be357-7c57c80f093mr2033018985a.27.1742252426920; Mon, 17 Mar 2025 16:00:26 -0700 (PDT) Received: from localhost (104-178-186-189.lightspeed.milwwi.sbcglobal.net. [104.178.186.189]) by smtp.gmail.com with UTF8SMTPSA id af79cd13be357-7c573d6fcf4sm641759685a.75.2025.03.17.16.00.26 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 17 Mar 2025 16:00:26 -0700 (PDT) Date: Mon, 17 Mar 2025 19:00:25 -0400 From: Taylor Blau To: git@vger.kernel.org Cc: Jeff King , Junio C Hamano , Elijah Newren Subject: [PATCH 4/5] repack: avoid combining cruft packs with `--max-cruft-size` Message-ID: <7941997e332828f0005acf97f925efc1e7251a33.1742252411.git.me@ttaylorr.com> References: Precedence: bulk X-Mailing-List: git@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: In 37dc6d8104 (builtin/repack.c: implement support for `--max-cruft-size`, 2023-10-02), we exposed new functionality that allowed repositories to specify the behavior of when we should combine multiple cruft packs together. This feature was designed to ensure that we never repacked cruft packs which were larger than the given threshold in order to provide tighter I/O bounds for repositories that have many unreachable objects. In essence, specifying '--max-cruft-size=N' instructed 'repack' to aggregate cruft packs together (in order of ascending size) until the combine size grows past 'N', and then make a new cruft pack whose contents includes the packs we rolled up. But this isn't quite how it works in practice. Suppose for example that we have two cruft packs which are each 100MiB in size. One might expect specifying "--max-cruft-size=200M" would combine these two packs together, and then avoid repacking them until a pruning GC takes place. In reality, 'repack' would try and aggregate these together, but writing a pack that is strictly smaller than 200 MiB (since pack-objects' "--max-pack-size" provides a strict bound for packs containing more than one object). So instead we'll write out a pack that is, say, 199 MiB in size, and then another 1 MiB pack containing the balance. If we later repack the repository without adding any new unreachable objects, we'll repeat the same exercise again, making the same 199 MiB and 1 MiB packs each time. This happens because of a poor choice to bolt the '--max-cruft-size' functionality onto pack-objects' '--max-pack-size', forcing us to generate packs which are always smaller than the provided threshold and thus subject to repacking. The following commit will introduce a new flag that implements something similar to the behavior above. Let's prepare for that by making repack's '--max-cruft-size' flag behave as an cruft pack-specific override for '--max-pack-size'. Do so by temporarily repurposing the 'collapse_small_cruft_packs()' function to instead generate a cruft pack using the same instructions as if we didn't specify any maximum pack size. The calling code looks something like: if (args->max_pack_size && !cruft_expiration) { collapse_small_cruft_packs(in, args->max_pack_size, existing); } else { for_each_string_list_item(item, &existing->non_kept_packs) fprintf(in, "-%s.pack\n", item->string); for_each_string_list_item(item, &existing->cruft_packs) fprintf(in, "-%s.pack\n", item->string); } This patch makes collapse_small_cruft_packs() behave identically to the 'else' arm of the conditional above. This repurposing of 'collapse_small_cruft_packs()' is intentional, since it will set us up nicely to introduce the new behavior in the following commit. Naturally, there is some test fallout in the test which exercises the old meaning of '--max-cruft-size'. Mark that test as failing for now to be dealt with in the following commit. Likewise, add a new test which explicitly tests the behavior of '--max-cruft-size' to place a hard limit on the size of any generated cruft pack(s). Note that this is a breaking change, as it alters the user-visible behavior of '--max-cruft-size'. But I'm OK changing this behavior in this instance, since the behavior wasn't accurate to begin with. Signed-off-by: Taylor Blau --- Documentation/git-repack.adoc | 12 ++------ builtin/repack.c | 50 ++++++--------------------------- t/t5329-pack-objects-cruft.sh | 52 +++++++++++++++++++++++++++++++++++ t/t7704-repack-cruft.sh | 8 ++---- 4 files changed, 67 insertions(+), 55 deletions(-) diff --git a/Documentation/git-repack.adoc b/Documentation/git-repack.adoc index 5852a5c973..11db43b1c5 100644 --- a/Documentation/git-repack.adoc +++ b/Documentation/git-repack.adoc @@ -77,15 +77,9 @@ to the new separate pack will be written. Only useful with `--cruft -d`. --max-cruft-size=:: - Repack cruft objects into packs as large as `` bytes before - creating new packs. As long as there are enough cruft packs - smaller than ``, repacking will cause a new cruft pack to - be created containing objects from any combined cruft packs, - along with any new unreachable objects. Cruft packs larger than - `` will not be modified. When the new cruft pack is larger - than `` bytes, it will be split into multiple packs, all of - which are guaranteed to be at most `` bytes in size. Only - useful with `--cruft -d`. + Overrides `--max-pack-size` for cruft packs. Inherits the value of + `--max-pack-size` (if any) by default. See the documentation for + `--max-pack-size` for more details. --expire-to=:: Write a cruft pack containing pruned objects (if any) to the diff --git a/builtin/repack.c b/builtin/repack.c index 75e3752353..9658f6b354 100644 --- a/builtin/repack.c +++ b/builtin/repack.c @@ -1022,28 +1022,19 @@ static int write_filtered_pack(const struct pack_objects_args *args, return finish_pack_objects_cmd(&cmd, names, local); } -static int existing_cruft_pack_cmp(const void *va, const void *vb) -{ - struct packed_git *a = *(struct packed_git **)va; - struct packed_git *b = *(struct packed_git **)vb; - - if (a->pack_size < b->pack_size) - return -1; - if (a->pack_size > b->pack_size) - return 1; - return 0; -} - -static void collapse_small_cruft_packs(FILE *in, size_t max_size, +static void collapse_small_cruft_packs(FILE *in, size_t max_size UNUSED, struct existing_packs *existing) { - struct packed_git **existing_cruft, *p; + struct packed_git *p; struct strbuf buf = STRBUF_INIT; - size_t total_size = 0; - size_t existing_cruft_nr = 0; size_t i; - ALLOC_ARRAY(existing_cruft, existing->cruft_packs.nr); + /* + * Squelch a -Wunused-function warning while we rationalize + * the behavior of --max-cruft-size. This function will become + * used again in a future commit. + */ + (void)retain_cruft_pack; for (p = get_all_packs(the_repository); p; p = p->next) { if (!(p->is_cruft && p->pack_local)) @@ -1056,29 +1047,7 @@ static void collapse_small_cruft_packs(FILE *in, size_t max_size, if (!string_list_has_string(&existing->cruft_packs, buf.buf)) continue; - if (existing_cruft_nr >= existing->cruft_packs.nr) - BUG("too many cruft packs (found %"PRIuMAX", but knew " - "of %"PRIuMAX")", - (uintmax_t)existing_cruft_nr + 1, - (uintmax_t)existing->cruft_packs.nr); - existing_cruft[existing_cruft_nr++] = p; - } - - QSORT(existing_cruft, existing_cruft_nr, existing_cruft_pack_cmp); - - for (i = 0; i < existing_cruft_nr; i++) { - size_t proposed; - - p = existing_cruft[i]; - proposed = st_add(total_size, p->pack_size); - - if (proposed <= max_size) { - total_size = proposed; - fprintf(in, "-%s\n", pack_basename(p)); - } else { - retain_cruft_pack(existing, p); - fprintf(in, "%s\n", pack_basename(p)); - } + fprintf(in, "-%s.pack\n", buf.buf); } for (i = 0; i < existing->non_kept_packs.nr; i++) @@ -1086,7 +1055,6 @@ static void collapse_small_cruft_packs(FILE *in, size_t max_size, existing->non_kept_packs.items[i].string); strbuf_release(&buf); - free(existing_cruft); } static int write_cruft_pack(const struct pack_objects_args *args, diff --git a/t/t5329-pack-objects-cruft.sh b/t/t5329-pack-objects-cruft.sh index 60dac8312d..25ddda5cf3 100755 --- a/t/t5329-pack-objects-cruft.sh +++ b/t/t5329-pack-objects-cruft.sh @@ -695,4 +695,56 @@ test_expect_success 'additional cruft blobs via gc.recentObjectsHook' ' ) ' +test_expect_success 'split cruft packs with --max-cruft-size' ' + repo=cruft-with--max-cruft-size && + test_when_finished "rm -fr $repo" && + + git init "$repo" && + + ( + cd "$repo" && + + git config core.compression 0 && + + sz=$((1024 * 1024)) && # 1MiB + test-tool genrandom foo $sz >foo && + test-tool genrandom bar $sz >bar && + foo="$(git hash-object -w -t blob foo)" && + bar="$(git hash-object -w -t blob bar)" && + + to=$packdir/pack && + # Pack together foo and bar into a single 2MiB pack. + pack="$(git pack-objects $to <<-EOF + $foo + $bar + EOF + )" && + + # Then generate a cruft pack containing foo and bar. + # + # Generate the pack with --max-pack-size equal to the + # size of one object, forcing us to write two cruft + # packs. + git pack-objects --cruft --max-pack-size=$sz $to <<-EOF && + -pack-$pack.pack + EOF + + ls $packdir/pack-*.mtimes >crufts && + test_line_count = 2 crufts && + + for cruft in $(cat crufts) + do + test-tool pack-mtimes "$(basename "$cruft")" || return 1 + done >actual.raw && + + cut -d" " -f1 actual && + sort >expect <<-EOF && + $foo + $bar + EOF + + test_cmp expect actual + ) +' + test_done diff --git a/t/t7704-repack-cruft.sh b/t/t7704-repack-cruft.sh index 3fd5aa6089..6debad368d 100755 --- a/t/t7704-repack-cruft.sh +++ b/t/t7704-repack-cruft.sh @@ -194,7 +194,7 @@ test_expect_success '--max-cruft-size combines existing packs when not too large ) ' -test_expect_success '--max-cruft-size combines smaller packs first' ' +test_expect_failure '--max-cruft-size combines smaller packs first' ' git init max-cruft-size-consume-small && ( cd max-cruft-size-consume-small && @@ -354,13 +354,11 @@ test_expect_success 'multi-cruft with freshened objects (previously cruft)' ' done >actual.raw && sort actual.raw >actual && - # Among the set of all cruft packs, we should see both - # mtimes for object $foo and $bar, as well as the + # Among the set of all cruft packs, we should see the + # new mtimes for object $foo and $bar, as well as the # single new copy of $baz. sort >expect <<-EOF && - $foo $(cat foo.old) $foo $(cat foo.new) - $bar $(cat bar.old) $bar $(cat bar.new) $baz $(cat baz.old) $quux $(cat quux.new) From patchwork Mon Mar 17 23:00:28 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Taylor Blau X-Patchwork-Id: 14020024 Received: from mail-qv1-f53.google.com (mail-qv1-f53.google.com [209.85.219.53]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 6F197205AC2 for ; Mon, 17 Mar 2025 23:00:31 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.219.53 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1742252433; cv=none; b=aiCHqAO5tad0V2bcdxzmv3NZYjT/EGx2h/iRqruoqt9a12traKwDR8xH33cn0cMKwq1pAOgfNe94MurNeAGppjegrSVKasXG5VBcNYiMAWCOpCUHhBipER1T1gM9+bXqp6Hfhp4brmbh3Dw/RRr9J3bMfMOl1FcdLoMmGyvdLpU= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1742252433; c=relaxed/simple; bh=KBf/0OcdRkDRxvreY2JmLVY3K/QSuqE1VHjOKYJV1DQ=; h=Date:From:To:Cc:Subject:Message-ID:References:MIME-Version: Content-Type:Content-Disposition:In-Reply-To; b=FY48dWeUrfylzPIcD1pV9IfjKT1qBzzDPLzWXnXSBtJnYOWjJgc8vDTY7RVODJQX7TOf+ybN8rAhhcRQmZAbySVoUaznJ8H1u8YyvgH2Z1Q7HdFiXFxfOYqHtgQbQndPvcXAbHL0g0oa9Jf8vZyiRRB7t2IxVhW5U8CY9yhUqBM= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=ttaylorr.com; spf=pass smtp.mailfrom=ttaylorr.com; dkim=pass (2048-bit key) header.d=ttaylorr-com.20230601.gappssmtp.com header.i=@ttaylorr-com.20230601.gappssmtp.com header.b=Ad/xCeiZ; arc=none smtp.client-ip=209.85.219.53 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=ttaylorr.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=ttaylorr.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=ttaylorr-com.20230601.gappssmtp.com header.i=@ttaylorr-com.20230601.gappssmtp.com header.b="Ad/xCeiZ" Received: by mail-qv1-f53.google.com with SMTP id 6a1803df08f44-6eaf1b6ce9aso37519196d6.2 for ; Mon, 17 Mar 2025 16:00:31 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ttaylorr-com.20230601.gappssmtp.com; s=20230601; t=1742252430; x=1742857230; darn=vger.kernel.org; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:from:to:cc:subject:date:message-id:reply-to; bh=v2u4leih2MyhNfvd/YPXw0KSe+aYUm91j040RvPk4Po=; b=Ad/xCeiZYUihvjvi4zB5ZiWSK8cxKGIU+hNN1+Eavpkif0QQOewWT15EQFnzwvJN+/ v/SnktIJ9sYJiCbAOKfmq0WzKJ4uzrqMSOvnjwxFcFod40Srdb3l6ikQvCDxZKeJlqXv qKnbgCBcfR718y1a+d+Vxoyc+zdmay2m4pr8lwg8LuS+e322ZehY8qlTGTbnZ/Gux0FL uN9dE1aoJMBhLDeKu+MklmNqWk45boQA+56wq/7ujgytMwSSRFFL4NG3AFmgmzBXV06P qFBwg/iPNb/PNq5TByNA1GpmG2Qv8YmMcnkXAId1wX2IxVSn2/XhdAsFd46FCXcd4ghH RgFQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1742252430; x=1742857230; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:x-gm-message-state:from:to:cc:subject:date :message-id:reply-to; bh=v2u4leih2MyhNfvd/YPXw0KSe+aYUm91j040RvPk4Po=; b=G78qme3Ju6jwgVfu3gKOH+amBBPUx6uvh/s8qezORqQpYhhDlj/v7lVFa4+YsGgckX 4bcUSdm06MFm4E+kkyqvvuZ86CYOojDE1ZEbU0VqvQi6f9EUvkUuBB0jv8YFkgw1vo0N nK9LgG1ghQ3CIqByaGBRW1gIf+G3NfrsaJ/VhCKAa95kOHcrfhhsiF+ElSfdeH2j0PWO fxN7VrdIKlsyZiWODqpIsUBZ3kDnRoq/uxY1yvj3Vlrot3j6V1VZel4TnPtpDw4trTdR 8T2sbN84A4/JZstdQU45URafblZisMA3iNOAwYJGLUKIwSAlIz275l6blk6ecKkvFJeU kagQ== X-Gm-Message-State: AOJu0YzAxHWdHKRPsJA9uiEhukGJIK1+pNO1gxfLZB83vQT2SzEsuYxe HJ2gRTFyzRb9VVADV31H+wBG99EveghuQlFyL44/CfOlCLtAX19TlDfEkTUOQLNBUxpJXYQFniX w3SY= X-Gm-Gg: ASbGnct/KO9k5lnVoJGsy7IIsJDtQv+OJU+LPpkemH2UxuYjkRs2MRekJOjGtsA5eTw q+MRxSn74v+cyB6FeeuklIoK+AfPC0QmplS8zHMta/OlWWQcx5EPhs4pYHBMJGvV2AEDL+vadi5 a3lfh3tHIQ60SO6wq4IEaEfz4PQIU8zJtu3lZDe1opeZYZ2WeguETVCdO0z9dgRzNeUS2zy/ZSc ivN3nBMgj8CcH7cx2Cv1xHtV61lPAgdSeb87nG9BYdq9rvMya/t7SrUhyY78lW5kSZ8i7aSfaR6 Su0njuEF+XjKim0795E2GPN2CHSlKVBYLWu195Zk5pmAS50+u5j5XnTXAl7ktUae2UwvQgYiAUa 0g48SDda55CnQOxAr8Msx+F0VZy4= X-Google-Smtp-Source: AGHT+IHWktikGnVxD9sbPg6XSn2VeNOv44QU4Q2cGS0PF0DDbwekibh24nc/YVBbLdas0IjNlDPokA== X-Received: by 2002:a05:620a:31a5:b0:7c5:4348:1847 with SMTP id af79cd13be357-7c59b1bbb89mr179552785a.19.1742252430063; Mon, 17 Mar 2025 16:00:30 -0700 (PDT) Received: from localhost (104-178-186-189.lightspeed.milwwi.sbcglobal.net. [104.178.186.189]) by smtp.gmail.com with UTF8SMTPSA id af79cd13be357-7c573c73273sm641963185a.27.2025.03.17.16.00.29 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 17 Mar 2025 16:00:29 -0700 (PDT) Date: Mon, 17 Mar 2025 19:00:28 -0400 From: Taylor Blau To: git@vger.kernel.org Cc: Jeff King , Junio C Hamano , Elijah Newren Subject: [PATCH 5/5] repack: begin combining cruft packs with `--combine-cruft-below-size` Message-ID: <7f120c35e95dcf41282c87dc2d1b2640ecdc5d84.1742252411.git.me@ttaylorr.com> References: Precedence: bulk X-Mailing-List: git@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: The previous commit changed the behavior of repack's '--max-cruft-size' to specify a cruft pack-specific override for '--max-pack-size'. Introduce a new flag, '--combine-cruft-below-size' which is a replacement for the old behavior of '--max-cruft-size'. This new flag does explicitly what it says: it combines together cruft packs which are smaller than a given threshold, and prohibits repacking ones which are larger. This accomplishes the original intent of '--max-cruft-size', which was to avoid repacking cruft packs larger than the given threshold. The new behavior is slightly different. Instead of building up small packs together until the threshold is met, '--combine-cruft-below-size' packs up *all* cruft packs smaller than the threshold. This means that we may make a pack much larger than the given threshold (e.g., if you aggregate 5 packs which are each 99 MiB in size with a threshold of 100 MiB). But that's OK: the point isn't to restrict the size of the cruft packs we generate, it's to avoid working with ones that have already grown too large. If repositories still want to limit the size of the generated cruft pack(s), they may use '--max-cruft-size' instead. There's some minor test fallout as a result of the slight differences in behavior between the old meaning of '--max-cruft-size' and the behavior of '--combine-cruft-below-size'. In the test which is now called "--combine-cruft-below-size combines packs", we need to use the new flag over the old one to exercise that test's intended behavior. The remainder of the changes there are to improve the clarity of the comments. Suggested-by: Elijah Newren Signed-off-by: Taylor Blau --- Documentation/git-repack.adoc | 8 ++++++++ builtin/repack.c | 38 +++++++++++++++++++++++------------ t/t7704-repack-cruft.sh | 22 +++++++++++--------- 3 files changed, 46 insertions(+), 22 deletions(-) diff --git a/Documentation/git-repack.adoc b/Documentation/git-repack.adoc index 11db43b1c5..8e6d61aa2f 100644 --- a/Documentation/git-repack.adoc +++ b/Documentation/git-repack.adoc @@ -81,6 +81,14 @@ to the new separate pack will be written. `--max-pack-size` (if any) by default. See the documentation for `--max-pack-size` for more details. +--combine-cruft-below-size=:: + When generating cruft packs without pruning, only repack + existing cruft packs whose size is strictly less than ``. + Cruft packs whose size is greater than or equal to `` are + left as-is and not repacked. Useful when you want to avoid + repacking large cruft pack(s) in repositories that have many + and/or large unreachable objects. + --expire-to=:: Write a cruft pack containing pruned objects (if any) to the directory ``. This option is useful for keeping a copy of diff --git a/builtin/repack.c b/builtin/repack.c index 9658f6b354..f3330ade7b 100644 --- a/builtin/repack.c +++ b/builtin/repack.c @@ -1022,20 +1022,13 @@ static int write_filtered_pack(const struct pack_objects_args *args, return finish_pack_objects_cmd(&cmd, names, local); } -static void collapse_small_cruft_packs(FILE *in, size_t max_size UNUSED, - struct existing_packs *existing) +static void combine_small_cruft_packs(FILE *in, size_t combine_cruft_below_size, + struct existing_packs *existing) { struct packed_git *p; struct strbuf buf = STRBUF_INIT; size_t i; - /* - * Squelch a -Wunused-function warning while we rationalize - * the behavior of --max-cruft-size. This function will become - * used again in a future commit. - */ - (void)retain_cruft_pack; - for (p = get_all_packs(the_repository); p; p = p->next) { if (!(p->is_cruft && p->pack_local)) continue; @@ -1047,7 +1040,12 @@ static void collapse_small_cruft_packs(FILE *in, size_t max_size UNUSED, if (!string_list_has_string(&existing->cruft_packs, buf.buf)) continue; - fprintf(in, "-%s.pack\n", buf.buf); + if (p->pack_size < combine_cruft_below_size) { + fprintf(in, "-%s\n", pack_basename(p)); + } else { + retain_cruft_pack(existing, p); + fprintf(in, "%s\n", pack_basename(p)); + } } for (i = 0; i < existing->non_kept_packs.nr; i++) @@ -1061,6 +1059,7 @@ static int write_cruft_pack(const struct pack_objects_args *args, const char *destination, const char *pack_prefix, const char *cruft_expiration, + unsigned long combine_cruft_below_size, struct string_list *names, struct existing_packs *existing) { @@ -1103,8 +1102,9 @@ static int write_cruft_pack(const struct pack_objects_args *args, in = xfdopen(cmd.in, "w"); for_each_string_list_item(item, names) fprintf(in, "%s-%s.pack\n", pack_prefix, item->string); - if (args->max_pack_size && !cruft_expiration) { - collapse_small_cruft_packs(in, args->max_pack_size, existing); + if (combine_cruft_below_size && !cruft_expiration) { + combine_small_cruft_packs(in, combine_cruft_below_size, + existing); } else { for_each_string_list_item(item, &existing->non_kept_packs) fprintf(in, "-%s.pack\n", item->string); @@ -1158,6 +1158,7 @@ int cmd_repack(int argc, const char *opt_window_memory = NULL; const char *opt_depth = NULL; const char *opt_threads = NULL; + unsigned long combine_cruft_below_size = 0ul; struct option builtin_repack_options[] = { OPT_BIT('a', NULL, &pack_everything, @@ -1170,6 +1171,9 @@ int cmd_repack(int argc, PACK_CRUFT), OPT_STRING(0, "cruft-expiration", &cruft_expiration, N_("approxidate"), N_("with --cruft, expire objects older than this")), + OPT_MAGNITUDE(0, "combine-cruft-below-size", + &combine_cruft_below_size, + N_("with --cruft, only repack cruft packs smaller than this")), OPT_MAGNITUDE(0, "max-cruft-size", &cruft_po_args.max_pack_size, N_("with --cruft, limit the size of new cruft packs")), OPT_BOOL('d', NULL, &delete_redundant, @@ -1413,7 +1417,8 @@ int cmd_repack(int argc, cruft_po_args.quiet = po_args.quiet; ret = write_cruft_pack(&cruft_po_args, packtmp, pack_prefix, - cruft_expiration, &names, + cruft_expiration, + combine_cruft_below_size, &names, &existing); if (ret) goto cleanup; @@ -1440,10 +1445,17 @@ int cmd_repack(int argc, * generate an empty pack (since every object not in the * cruft pack generated above will have an mtime older * than the expiration). + * + * Pretend we don't have a `--combine-cruft-below-size` + * argument, since we're not selectively combining + * anything based on size to generate the limbo cruft + * pack, but rather removing all cruft packs from the + * main repository regardless of size. */ ret = write_cruft_pack(&cruft_po_args, expire_to, pack_prefix, NULL, + 0ul, &names, &existing); if (ret) diff --git a/t/t7704-repack-cruft.sh b/t/t7704-repack-cruft.sh index 6debad368d..8aebfb45f5 100755 --- a/t/t7704-repack-cruft.sh +++ b/t/t7704-repack-cruft.sh @@ -194,10 +194,13 @@ test_expect_success '--max-cruft-size combines existing packs when not too large ) ' -test_expect_failure '--max-cruft-size combines smaller packs first' ' - git init max-cruft-size-consume-small && +test_expect_success '--combine-cruft-below-size combines packs' ' + repo=combine-cruft-below-size && + test_when_finished "rm -fr $repo" && + + git init "$repo" && ( - cd max-cruft-size-consume-small && + cd "$repo" && test_commit base && git repack -ad && @@ -211,11 +214,11 @@ test_expect_failure '--max-cruft-size combines smaller packs first' ' test-tool pack-mtimes "$(basename $cruft_bar)" >>expect.raw && sort expect.raw >expect.objects && - # repacking with `--max-cruft-size=2M` should combine - # both 0.5 MiB packs together, instead of, say, one of - # the 0.5 MiB packs with the 1.0 MiB pack + # Repacking with `--combine-cruft-below-size=1M` + # should combine both 0.5 MiB packs together, but + # ignore the two packs which are >= 1.0 MiB. ls $packdir/pack-*.mtimes | sort >cruft.before && - git repack -d --cruft --max-cruft-size=2M && + git repack -d --cruft --combine-cruft-below-size=1M && ls $packdir/pack-*.mtimes | sort >cruft.after && comm -13 cruft.before cruft.after >cruft.new && @@ -224,11 +227,12 @@ test_expect_failure '--max-cruft-size combines smaller packs first' ' test_line_count = 1 cruft.new && test_line_count = 2 cruft.removed && - # the two smaller packs should be rolled up first + # The two packs smaller than 1.0MiB should be repacked + # together. printf "%s\n" $cruft_foo $cruft_bar | sort >expect.removed && test_cmp expect.removed cruft.removed && - # ...and contain the set of objects rolled up + # ...and contain the set of objects rolled up. test-tool pack-mtimes "$(basename $(cat cruft.new))" >actual.raw && sort actual.raw >actual.objects &&