[0/8] commit-graph: drop CHECK_OIDS, peel in callers

Message

Taylor Blau May 5, 2020, 1:13 a.m. UTC

Hi,

These are a few patches to address [1] by getting rid of the
'COMMIT_GRAPH_WRITE_CHECK_OIDS' flag. A more complete discussion about
why we might want to get rid of that flag can be found in the second
patch, as well as in [1], but the basic gist is as follows:

  - It is often convenient to pipe the output of 'git for-each-ref' into
    'git commit-graph write --stdin-commits', but handling multi-layer
    tags is frustrating (at the very least, you have to peel them
    yourself in 'for-each-ref', incurring a higher-than-necessary cost
    outside of the commit-graph invocation)

  - It would be much more convenient if 'git commit-graph write
    --stdin-commits' ignored OIDs that peel down to non-commit objects
    silently, while still warning the caller about objects that (a)
    don't exist, or (b) invalid hash inputs (for e.g., "HEAD", or
    "refs/heads/foo").

  - By making this change, we are allowing *more* possible behaviors
    than in previous versions, and the only case that we are breaking
    (that 'git commit-graph write --stdin-commits' complains on
    non-commit OIDs) can be addressed by peeling outside of
    'commit-graph write' [2].

The first six patches move peeling refs out from the commit-graph
internals, and into the callers.

This has no impact on the '--stdin-commits' case, but is a potential
improvement for '--reachable', where it is more efficient to call
'peel_ref()' than 'lookup_commit_reference_gently()'. There is an
intermediate state (that is resolved in the final patch) where we can
have more than one progress meter (since the progress meter moved with
the perhaps-expensive operations out of the commit-graph internals,
too), but the final patch addresses this.

The last two patches lift the restriction on input to '--stdin-commits'
resolving to commit objects, instead making the behavior to silently
drop non-commit OIDs, and continue complain about invalid OIDs, and
valid-but-missing OIDs.

Finally, this topic isn't based on anything that isn't already in
'master', so I think the days of many tangle commit-graph topics are
behind us.

Thanks in advance for your review.

Thanks,
Taylor

Taylor Blau (8):
  commit-graph.c: extract 'refs_cb_data'
  commit-graph.c: show progress of finding reachable commits
  commit-graph.c: peel refs in 'add_ref_to_set'
  builtin/commit-graph.c: extract 'read_one_commit()'
  builtin/commit-graph.c: dereference tags in builtin
  commit-graph.c: simplify 'fill_oids_from_commits'
  t5318: reorder test below 'graph_read_expect'
  commit-graph: drop COMMIT_GRAPH_WRITE_CHECK_OIDS flag

 Documentation/git-commit-graph.txt |  6 ++-
 builtin/commit-graph.c             | 73 ++++++++++++++++++++----------
 commit-graph.c                     | 61 +++++++++++--------------
 commit-graph.h                     | 10 ++--
 t/t5318-commit-graph.sh            | 25 ++++++----
 5 files changed, 98 insertions(+), 77 deletions(-)

--
2.26.0.113.ge9739cdccc

Comments

Derrick Stolee May 5, 2020, 11:35 a.m. UTC | #1

On 5/4/2020 9:13 PM, Taylor Blau wrote:
> Hi,
> 
> These are a few patches to address [1]

We seem to be missing links [1] and [2].

>   - It is often convenient to pipe the output of 'git for-each-ref' into
>     'git commit-graph write --stdin-commits', but handling multi-layer
>     tags is frustrating (at the very least, you have to peel them
>     yourself in 'for-each-ref', incurring a higher-than-necessary cost
>     outside of the commit-graph invocation)
> 
>   - It would be much more convenient if 'git commit-graph write
>     --stdin-commits' ignored OIDs that peel down to non-commit objects
>     silently, while still warning the caller about objects that (a)
>     don't exist, or (b) invalid hash inputs (for e.g., "HEAD", or
>     "refs/heads/foo").
> 
>   - By making this change, we are allowing *more* possible behaviors
>     than in previous versions, and the only case that we are breaking
>     (that 'git commit-graph write --stdin-commits' complains on
>     non-commit OIDs) can be addressed by peeling outside of
>     'commit-graph write' [2].

I agree that these are worthy goals. If someone was _depending_ on
the "git commit-graph write" command to _fail_, then they chose a
strange way to discover a ref that points to a non-commit.

> The first six patches move peeling refs out from the commit-graph
> internals, and into the callers.
> 
> This has no impact on the '--stdin-commits' case, but is a potential
> improvement for '--reachable', where it is more efficient to call
> 'peel_ref()' than 'lookup_commit_reference_gently()'. There is an
> intermediate state (that is resolved in the final patch) where we can
> have more than one progress meter (since the progress meter moved with
> the perhaps-expensive operations out of the commit-graph internals,
> too), but the final patch addresses this.

Interesting. I'll be sure to look carefully at this progress handling
as this has potential to lose our progress options. I expect that you
were careful about this.

> The last two patches lift the restriction on input to '--stdin-commits'
> resolving to commit objects, instead making the behavior to silently
> drop non-commit OIDs, and continue complain about invalid OIDs, and
> valid-but-missing OIDs.
> 
> Finally, this topic isn't based on anything that isn't already in
> 'master', so I think the days of many tangle commit-graph topics are
> behind us.

Woo!

-Stolee

Taylor Blau May 5, 2020, 4:11 p.m. UTC | #2

On Tue, May 05, 2020 at 07:35:29AM -0400, Derrick Stolee wrote:
> On 5/4/2020 9:13 PM, Taylor Blau wrote:
> > Hi,
> >
> > These are a few patches to address [1]
>
> We seem to be missing links [1] and [2].

Whoops. I wrote the cover letter and then postponed it for an hour or
two while I shuffled (what are now) the last two patches around. It
looks like I dropped the hyperlinks in the meantime, but here they are
now:

[1]: https://lore.kernel.org/git/[e-mail]/
[2]: https://lore.kernel.org/git/[e-mail]/

Thanks,
Taylor