[v3,00/16] refs: implement jump lists for packed backend

Message

Taylor Blau June 7, 2023, 10:40 a.m. UTC

Here is a reroll of my series to implement jump (née skip) lists for the
packed refs backend, based on top of the current 'master'.

Nothing substantive has changed since the last version, where review had
stabilized. This version just resolves a couple of merge conflicts with
633390bd08 (Merge branch 'bc/clone-empty-repo-via-protocol-v0',
2023-05-19).

As usual, a range-diff is included below for convenience.

Thanks in advance for a hopefully final look ;-).

Jeff King (5):
  refs.c: rename `ref_filter`
  ref-filter.h: provide `REF_FILTER_INIT`
  ref-filter: clear reachable list pointers after freeing
  ref-filter: add `ref_filter_clear()`
  ref-filter.c: parameterize match functions over patterns

Taylor Blau (11):
  builtin/for-each-ref.c: add `--exclude` option
  refs: plumb `exclude_patterns` argument throughout
  refs/packed-backend.c: refactor `find_reference_location()`
  refs/packed-backend.c: implement jump lists to avoid excluded
    pattern(s)
  refs/packed-backend.c: add trace2 counters for jump list
  revision.h: store hidden refs in a `strvec`
  refs/packed-backend.c: ignore complicated hidden refs rules
  refs.h: let `for_each_namespaced_ref()` take excluded patterns
  builtin/receive-pack.c: avoid enumerating hidden references
  upload-pack.c: avoid enumerating hidden refs where possible
  ls-refs.c: avoid enumerating hidden refs where possible

 Documentation/git-for-each-ref.txt |   6 +
 builtin/branch.c                   |   4 +-
 builtin/for-each-ref.c             |   7 +-
 builtin/receive-pack.c             |   7 +-
 builtin/tag.c                      |   4 +-
 http-backend.c                     |   2 +-
 ls-refs.c                          |   8 +-
 ref-filter.c                       |  63 ++++++--
 ref-filter.h                       |  12 ++
 refs.c                             |  61 ++++----
 refs.h                             |  15 +-
 refs/debug.c                       |   5 +-
 refs/files-backend.c               |   5 +-
 refs/packed-backend.c              | 226 ++++++++++++++++++++++++++---
 refs/refs-internal.h               |   7 +-
 revision.c                         |   4 +-
 revision.h                         |   5 +-
 t/helper/test-reach.c              |   2 +-
 t/helper/test-ref-store.c          |  10 ++
 t/t0041-usage.sh                   |   1 +
 t/t1419-exclude-refs.sh            | 131 +++++++++++++++++
 t/t3402-rebase-merge.sh            |   1 +
 t/t6300-for-each-ref.sh            |  35 +++++
 trace2.h                           |   2 +
 trace2/tr2_ctr.c                   |   5 +
 upload-pack.c                      |  43 ++++--
 26 files changed, 565 insertions(+), 106 deletions(-)
 create mode 100755 t/t1419-exclude-refs.sh

Range-diff against v2:
 1:  6cac42e70e =  1:  afac948f04 refs.c: rename `ref_filter`
 2:  8dda7db738 =  2:  b9336e3b77 ref-filter.h: provide `REF_FILTER_INIT`
 3:  bf21df783d =  3:  fc28b5caaa ref-filter: clear reachable list pointers after freeing
 4:  85ecb70957 =  4:  bc5356fe4b ref-filter: add `ref_filter_clear()`
 5:  385890b459 =  5:  1988ca4c0a ref-filter.c: parameterize match functions over patterns
 6:  1a3371a0a7 =  6:  60d85aa4ad builtin/for-each-ref.c: add `--exclude` option
 7:  aa05549b6e =  7:  c4fe9a1893 refs: plumb `exclude_patterns` argument throughout
 8:  6002c568b5 =  8:  9cab5e0699 refs/packed-backend.c: refactor `find_reference_location()`
 9:  8c78f49a8d =  9:  8066858bf5 refs/packed-backend.c: implement jump lists to avoid excluded pattern(s)
10:  5059f5dd42 = 10:  3c045076a9 refs/packed-backend.c: add trace2 counters for jump list
11:  f765b50a84 = 11:  0ff542eaef revision.h: store hidden refs in a `strvec`
12:  254bcc4361 = 12:  ca006b2c3f refs/packed-backend.c: ignore complicated hidden refs rules
13:  50e7df7dc0 ! 13:  cae703a425 refs.h: let `for_each_namespaced_ref()` take excluded patterns
    @@ upload-pack.c: void upload_pack(const int advertise_refs, const int stateless_rp
      		head_ref_namespaced(send_ref, &data);
     -		for_each_namespaced_ref(send_ref, &data);
     +		for_each_namespaced_ref(NULL, send_ref, &data);
    - 		/*
    - 		 * fflush stdout before calling advertise_shallow_grafts because send_ref
    - 		 * uses stdio.
    + 		if (!data.sent_capabilities) {
    + 			const char *refname = "capabilities^{}";
    + 			write_v0_ref(&data, refname, refname, null_oid());
     @@ upload-pack.c: void upload_pack(const int advertise_refs, const int stateless_rpc,
      		packet_flush(1);
      	} else {
14:  f6a3a5a6ba = 14:  1db10b76ea builtin/receive-pack.c: avoid enumerating hidden references
15:  2331fa7a4d ! 15:  014243ebe6 upload-pack.c: avoid enumerating hidden refs where possible
    @@ upload-pack.c: void upload_pack(const int advertise_refs, const int stateless_rp
      		head_ref_namespaced(send_ref, &data);
     -		for_each_namespaced_ref(NULL, send_ref, &data);
     +		for_each_namespaced_ref_1(send_ref, &data);
    - 		/*
    - 		 * fflush stdout before calling advertise_shallow_grafts because send_ref
    - 		 * uses stdio.
    + 		if (!data.sent_capabilities) {
    + 			const char *refname = "capabilities^{}";
    + 			write_v0_ref(&data, refname, refname, null_oid());
     @@ upload-pack.c: void upload_pack(const int advertise_refs, const int stateless_rpc,
      		packet_flush(1);
      	} else {
16:  2c6b89d64a = 16:  e02fe93379 ls-refs.c: avoid enumerating hidden refs where possible

Comments

Junio C Hamano June 12, 2023, 9:05 p.m. UTC | #1

Taylor Blau <[e-mail]> writes:

> Here is a reroll of my series to implement jump (née skip) lists for the
> packed refs backend, based on top of the current 'master'.

Hmph.  I kind of liked Patrick's suggestion to split this into two
series to make it easier for the earlier half to graduate faster,
but perhaps it didn't make much difference?  I certainly did not get
the impression that "review had stabilized".  During my review of
the initial round, for example, I lost steam in the middle because
it was simply too long a series and didn't have a chance to give the
remainder a proper review.  I do not know about others.