[v10,0/8] filter: support for excluding all trees and blobs

Commit Message

Matthew DeVore Oct. 3, 2018, 7:52 p.m. UTC

This is a minor change to the previous rollup. It moves positional
arguments to the end of git-rev-list invocations. Here is an interdiff
from v9:


Thank you,

Matthew DeVore (8):
  list-objects: store common func args in struct
  list-objects: refactor to process_tree_contents
  list-objects: always parse trees gently
  rev-list: handle missing tree objects properly
  revision: mark non-user-given objects instead
  list-objects-filter: use BUG rather than die
  list-objects-filter-options: do not over-strbuf_init
  list-objects-filter: implement filter tree:0

 Documentation/rev-list-options.txt     |   5 +
 builtin/rev-list.c                     |  11 +-
 list-objects-filter-options.c          |  19 +-
 list-objects-filter-options.h          |   1 +
 list-objects-filter.c                  |  60 ++++++-
 list-objects.c                         | 232 +++++++++++++------------
 revision.c                             |   1 -
 revision.h                             |  26 ++-
 t/t0410-partial-clone.sh               |  45 +++++
 t/t5317-pack-objects-filter-objects.sh |  41 +++++
 t/t5616-partial-clone.sh               |  42 +++++
 t/t6112-rev-list-filters-objects.sh    |  43 +++++
 12 files changed, 398 insertions(+), 128 deletions(-)

Comments

Matthew DeVore Oct. 3, 2018, 11:08 p.m. UTC | #1

On Wed, Oct 3, 2018 at 12:52 PM Matthew DeVore <[e-mail]> wrote:
>
> This is a minor change to the previous rollup. It moves positional
> arguments to the end of git-rev-list invocations. Here is an interdiff
> from v9:
...
There is another problem with this patchset related to dropped exit
codes and pipelines. In t6112, we run "git cat-file -t" on an object
with was rm'd without being promised. It was printing an error and
going undetected because it was upstream in a pipeline. The file was
removed in the previous test.

So I fixed the previous test to clone the repository before
manipulating it, and I fixed the latter test to not mask Git exit
codes :) (This is a really insidious pattern and I should have taken
it more seriously.) Below is an interdiff. The two tests are added in
different commits, so each commit had to be fixed up.

I'll send a re-roll in two days or so if there are no more comments.

diff --git a/t/t6112-rev-list-filters-objects.sh
b/t/t6112-rev-list-filters-objects.sh
index 5a61614b1..c8e3d87c4 100755
--- a/t/t6112-rev-list-filters-objects.sh
+++ b/t/t6112-rev-list-filters-objects.sh
@@ -210,16 +210,21 @@ test_expect_success 'verify sparse:oid=oid-ish
omits top-level files' '
 test_expect_success 'rev-list W/ --missing=print and
--missing=allow-any for trees' '
         TREE=$(git -C r3 rev-parse HEAD:dir1) &&

-        rm r3/.git/objects/$(echo $TREE | sed "s|^..|&/|") &&
+        # Create a spare repo because we will be deleting objects
from this one.
+        git clone r3 r3.b &&

-        git -C r3 rev-list --quiet --missing=print --objects HEAD
>missing_objs 2>rev_list_err &&
+        rm r3.b/.git/objects/$(echo $TREE | sed "s|^..|&/|") &&
+
+        git -C r3.b rev-list --quiet --missing=print --objects HEAD \
+                >missing_objs 2>rev_list_err &&
         echo "?$TREE" >expected &&
         test_cmp expected missing_objs &&

         # do not complain when a missing tree cannot be parsed
         test_must_be_empty rev_list_err &&

-        git -C r3 rev-list --missing=allow-any --objects HEAD >objs
2>rev_list_err &&
+        git -C r3.b rev-list --missing=allow-any --objects HEAD \
+                >objs 2>rev_list_err &&
         ! grep $TREE objs &&
         test_must_be_empty rev_list_err
 '
@@ -228,12 +233,13 @@ test_expect_success 'rev-list W/ --missing=print
and --missing=allow-any for tre

 test_expect_success 'verify tree:0 includes trees in "filtered" output' '
         git -C r3 rev-list --quiet --objects --filter-print-omitted \
-                --filter=tree:0 HEAD |
-        awk -f print_1.awk |
+                --filter=tree:0 HEAD >revs &&
+
+        awk -f print_1.awk revs |
         sed s/~// |
-        xargs -n1 git -C r3 cat-file -t |
-        sort -u >filtered_types &&
+        xargs -n1 git -C r3 cat-file -t >unsorted_filtered_types &&

+        sort -u unsorted_filtered_types >filtered_types &&
         printf "blob\ntree\n" >expected &&
         test_cmp expected filtered_types
 '

Patch

diff --git a/t/t5616-partial-clone.sh b/t/t5616-partial-clone.sh
index 7b6294ca5..53fbf7db8 100755
--- a/t/t5616-partial-clone.sh
+++ b/t/t5616-partial-clone.sh
@@ -168,7 +168,8 @@  test_expect_success 'use fsck before and after manually fetching a missing subtr
 	git -C dst fsck &&
 
 	# Make sure we only have commits, and all trees and blobs are missing.
-	git -C dst rev-list master --missing=allow-any --objects >fetched_objects &&
+	git -C dst rev-list --missing=allow-any --objects master \
+		>fetched_objects &&
 	awk -f print_1.awk fetched_objects |
 	xargs -n1 git -C dst cat-file -t >fetched_types &&
 
@@ -184,7 +185,7 @@  test_expect_success 'use fsck before and after manually fetching a missing subtr
 	git -C dst fsck &&
 
 	# Auto-fetch all remaining trees and blobs with --missing=error
-	git -C dst rev-list master --missing=error --objects >fetched_objects &&
+	git -C dst rev-list --missing=error --objects master >fetched_objects &&
 	test_line_count = 70 fetched_objects &&
 
 	awk -f print_1.awk fetched_objects |
diff --git a/t/t6112-rev-list-filters-objects.sh b/t/t6112-rev-list-filters-objects.sh
index 6e5c41a68..5a61614b1 100755
--- a/t/t6112-rev-list-filters-objects.sh
+++ b/t/t6112-rev-list-filters-objects.sh
@@ -227,7 +227,8 @@  test_expect_success 'rev-list W/ --missing=print and --missing=allow-any for tre
 # Test tree:0 filter.
 
 test_expect_success 'verify tree:0 includes trees in "filtered" output' '
-	git -C r3 rev-list HEAD --quiet --objects --filter-print-omitted --filter=tree:0 |
+	git -C r3 rev-list --quiet --objects --filter-print-omitted \
+		--filter=tree:0 HEAD |
 	awk -f print_1.awk |
 	sed s/~// |
 	xargs -n1 git -C r3 cat-file -t |