[3/3] builtin/commit-graph.c: support '--input=none'
diff mbox series

Message ID c329a8ddc087c9b74f673005f8b1e8769816ab9a.1580430057.git.me@ttaylorr.com
State New
Headers show
Series
  • builtin/commit-graph.c: new split/merge options
Related show

Commit Message

Taylor Blau Jan. 31, 2020, 12:28 a.m. UTC
In the previous commit, we introduced '--[no-]merge', and alluded to the
fact that '--merge' would be useful for callers who wish to always
trigger a merge of an incremental chain.

There is a problem with the above approach, which is that there is no
way to specify to the commit-graph builtin that a caller only wants to
include commits already in the graph. One can specify '--input=append'
to include all commits in the existing graphs, but the absence of
'--input=stdin-{commits,packs}' causes the builtin to call
'fill_oids_from_all_packs()'.

Passing '--input=reachable' (as in 'git commit-graph write
--split=merge-all --input=reachable --input=append') works around this
issue by making '--input=reachable' effectively a no-op, but this can be
prohibitively expensive in large repositories, making it an undesirable
choice for some users.

Teach '--input=none' as an option to behave as if '--input=append' were
given, but to consider no other sources in addition.

This, in conjunction with the option introduced in the previous patch
offers the convenient way to force the commit-graph machinery to
condense a chain of incrementals without requiring any new commits:

  $ git commit-graph write --split=merge-all --input=none

Signed-off-by: Taylor Blau <me@ttaylorr.com>
---
 Documentation/git-commit-graph.txt | 21 +++++++++++++--------
 builtin/commit-graph.c             | 13 ++++++++++---
 commit-graph.c                     |  6 ++++--
 commit-graph.h                     |  3 ++-
 t/t5324-split-commit-graph.sh      | 26 ++++++++++++++++++++++++++
 5 files changed, 55 insertions(+), 14 deletions(-)

Comments

Derrick Stolee Jan. 31, 2020, 2:40 p.m. UTC | #1
On 1/30/2020 7:28 PM, Taylor Blau wrote:
> In the previous commit, we introduced '--[no-]merge', and alluded to the
> fact that '--merge' would be useful for callers who wish to always
> trigger a merge of an incremental chain.
> 
> There is a problem with the above approach, which is that there is no
> way to specify to the commit-graph builtin that a caller only wants to
> include commits already in the graph. One can specify '--input=append'
> to include all commits in the existing graphs, but the absence of
> '--input=stdin-{commits,packs}' causes the builtin to call
> 'fill_oids_from_all_packs()'.
> 
> Passing '--input=reachable' (as in 'git commit-graph write
> --split=merge-all --input=reachable --input=append') works around this
> issue by making '--input=reachable' effectively a no-op, but this can be
> prohibitively expensive in large repositories, making it an undesirable
> choice for some users.
> 
> Teach '--input=none' as an option to behave as if '--input=append' were
> given, but to consider no other sources in addition.

The code change looks good.

> +test_expect_success '--split=no-merge, --input=none writes nothing' '
> +	test_when_finished rm -rf a graphs.before graphs.after &&
> +	rm -rf $graphdir &&
> +	git reset --hard commits/2 &&
> +	git rev-list -1 HEAD~1 >a &&
> +	git commit-graph write --split=no-merge --input=stdin-commits <a &&
> +	ls $graphdir/graph-*.graph >graphs.before &&
> +	test_line_count = 1 $graphdir/commit-graph-chain &&
> +	git commit-graph write --split --input=none &&
> +	ls $graphdir/graph-*.graph >graphs.after &&
> +	test_cmp graphs.before graphs.after
> +'
> +
> +test_expect_success '--split=merge-all, --input=none merges the chain' '
> +	test_when_finished rm -rf a b &&
> +	rm -rf $graphdir &&
> +	git reset --hard commits/2 &&
> +	git rev-list -1 HEAD~1 >a &&
> +	git rev-list -1 HEAD >b &&
> +	git commit-graph write --split=no-merge --input=stdin-commits <a &&
> +	git commit-graph write --split=no-merge --input=stdin-commits <b &&
> +	test_line_count = 2 $graphdir/commit-graph-chain &&
> +	git commit-graph write --split=merge-all --input=none &&
> +	test_line_count = 1 $graphdir/commit-graph-chain
> +'

And these tests demonstrate the value quite clearly. Thanks!

-Stolee
Martin Ågren Jan. 31, 2020, 7:45 p.m. UTC | #2
On Fri, 31 Jan 2020 at 01:30, Taylor Blau <me@ttaylorr.com> wrote:
> In the previous commit, we introduced '--[no-]merge', and alluded to the
> fact that '--merge' would be useful for callers who wish to always
> trigger a merge of an incremental chain.

Hmmm. So it looks like you've already had similar thoughts as I did
about patch 1/3. At some point, you had a separate `--merge=...` option,
then later made that `--split=...`. :-) Could you say something about why
you changed your mind?

> There is a problem with the above approach, which is that there is no
> way to specify to the commit-graph builtin that a caller only wants to
> include commits already in the graph. One can specify '--input=append'
> to include all commits in the existing graphs, but the absence of
> '--input=stdin-{commits,packs}' causes the builtin to call
> 'fill_oids_from_all_packs()'.

(Use one of those options with an empty stdin? Anyway, let's read on.)

> Passing '--input=reachable' (as in 'git commit-graph write
> --split=merge-all --input=reachable --input=append') works around this
> issue by making '--input=reachable' effectively a no-op, but this can be
> prohibitively expensive in large repositories, making it an undesirable
> choice for some users.
>
> Teach '--input=none' as an option to behave as if '--input=append' were
> given, but to consider no other sources in addition.

`--input=none` almost makes me wonder if it should produce an empty
commit-graph. But there wouldn't be much point in that... I guess
another way of defining this would be that it "uses no input, and
implies `--append`".

> This, in conjunction with the option introduced in the previous patch
> offers the convenient way to force the commit-graph machinery to
> condense a chain of incrementals without requiring any new commits:
>
>   $ git commit-graph write --split=merge-all --input=none

Right.

> --- a/Documentation/git-commit-graph.txt
> +++ b/Documentation/git-commit-graph.txt
> @@ -39,24 +39,29 @@ COMMANDS
>  --------
>  'write'::
>
> -Write a commit-graph file based on the commits found in packfiles.
> +Write a commit-graph file based on the commits specified:
> +* With the `--input=stdin-packs` option, generate the new commit graph
> +by walking objects only in the specified pack-indexes. (Cannot be
> +combined with `--input=stdin-commits` or `--input=reachable`.)
>  +
> -With the `--input=stdin-packs` option, generate the new commit graph by
> -walking objects only in the specified pack-indexes. (Cannot be combined
> -with `--input=stdin-commits` or `--input=reachable`.)
> -+
> -With the `--input=stdin-commits` option, generate the new commit graph
> +* With the `--input=stdin-commits` option, generate the new commit graph
>  by walking commits starting at the commits specified in stdin as a list
>  of OIDs in hex, one OID per line. (Cannot be combined with
>  `--input=stdin-packs` or `--input=reachable`.)
>  +
> -With the `--input=reachable` option, generate the new commit graph by
> +* With the `--input=reachable` option, generate the new commit graph by
>  walking commits starting at all refs. (Cannot be combined with
>  `--input=stdin-commits` or `--input=stdin-packs`.)
>  +
> -With the `--input=append` option, include all commits that are present
> +* With the `--input=append` option, include all commits that are present
>  in the existing commit-graph file.

Do these changes above really belong in this commit?

> +* With the `--input=none` option, behave as if `input=append` were
> +given, but do not walk other packs to find additional commits.
> +
> +If none of the above options are given, then commits found in
> +packfiles are specified.

"specified"? Plus, that also happens for `--input=append` right? (It
really seems like "append" is an odd one among all the inputs.)

>         N_("git commit-graph write [--object-dir <objdir>] [--append] "
> -          "[--split[=<strategy>]] [--input=<reachable|stdin-packs|stdin-commits>] "
> +          "[--split[=<strategy>]] "
> +          "[--input=<reachable|stdin-packs|stdin-commits|none>] "
>            "[--[no-]progress] <split options>"),

Hmm, you've left "--append" the old way.


Martin
Taylor Blau Feb. 4, 2020, 5:01 a.m. UTC | #3
On Fri, Jan 31, 2020 at 08:45:59PM +0100, Martin Ågren wrote:
> On Fri, 31 Jan 2020 at 01:30, Taylor Blau <me@ttaylorr.com> wrote:
> > In the previous commit, we introduced '--[no-]merge', and alluded to the
> > fact that '--merge' would be useful for callers who wish to always
> > trigger a merge of an incremental chain.
>
> Hmmm. So it looks like you've already had similar thoughts as I did
> about patch 1/3. At some point, you had a separate `--merge=...` option,
> then later made that `--split=...`. :-) Could you say something about why
> you changed your mind?

Heh :-). Left overs from an earlier version of this series. I think that
I already talked about why this was changed further up in the thread.

> > There is a problem with the above approach, which is that there is no
> > way to specify to the commit-graph builtin that a caller only wants to
> > include commits already in the graph. One can specify '--input=append'
> > to include all commits in the existing graphs, but the absence of
> > '--input=stdin-{commits,packs}' causes the builtin to call
> > 'fill_oids_from_all_packs()'.
>
> (Use one of those options with an empty stdin? Anyway, let's read on.)
>
> > Passing '--input=reachable' (as in 'git commit-graph write
> > --split=merge-all --input=reachable --input=append') works around this
> > issue by making '--input=reachable' effectively a no-op, but this can be
> > prohibitively expensive in large repositories, making it an undesirable
> > choice for some users.
> >
> > Teach '--input=none' as an option to behave as if '--input=append' were
> > given, but to consider no other sources in addition.
>
> `--input=none` almost makes me wonder if it should produce an empty
> commit-graph. But there wouldn't be much point in that... I guess
> another way of defining this would be that it "uses no input, and
> implies `--append`".

I suppose, although (like you) I can't imagine why anybody would want to
do that.

> > This, in conjunction with the option introduced in the previous patch
> > offers the convenient way to force the commit-graph machinery to
> > condense a chain of incrementals without requiring any new commits:
> >
> >   $ git commit-graph write --split=merge-all --input=none
>
> Right.
>
> > --- a/Documentation/git-commit-graph.txt
> > +++ b/Documentation/git-commit-graph.txt
> > @@ -39,24 +39,29 @@ COMMANDS
> >  --------
> >  'write'::
> >
> > -Write a commit-graph file based on the commits found in packfiles.
> > +Write a commit-graph file based on the commits specified:
> > +* With the `--input=stdin-packs` option, generate the new commit graph
> > +by walking objects only in the specified pack-indexes. (Cannot be
> > +combined with `--input=stdin-commits` or `--input=reachable`.)
> >  +
> > -With the `--input=stdin-packs` option, generate the new commit graph by
> > -walking objects only in the specified pack-indexes. (Cannot be combined
> > -with `--input=stdin-commits` or `--input=reachable`.)
> > -+
> > -With the `--input=stdin-commits` option, generate the new commit graph
> > +* With the `--input=stdin-commits` option, generate the new commit graph
> >  by walking commits starting at the commits specified in stdin as a list
> >  of OIDs in hex, one OID per line. (Cannot be combined with
> >  `--input=stdin-packs` or `--input=reachable`.)
> >  +
> > -With the `--input=reachable` option, generate the new commit graph by
> > +* With the `--input=reachable` option, generate the new commit graph by
> >  walking commits starting at all refs. (Cannot be combined with
> >  `--input=stdin-commits` or `--input=stdin-packs`.)
> >  +
> > -With the `--input=append` option, include all commits that are present
> > +* With the `--input=append` option, include all commits that are present
> >  in the existing commit-graph file.
>
> Do these changes above really belong in this commit?

I think so. My thought here was to leave this documentation as-is until
this patch, when adding '--input=none' would... somehow change this, but
trying to construct a reply, I can't seem to come up with why I thought
that this was a good idea in the first place ;-).

> > +* With the `--input=none` option, behave as if `input=append` were
> > +given, but do not walk other packs to find additional commits.
> > +
> > +If none of the above options are given, then commits found in
> > +packfiles are specified.
>
> "specified"? Plus, that also happens for `--input=append` right? (It
> really seems like "append" is an odd one among all the inputs.)

I reworded this slightly to not use "specified", which I agree is indeed
weird.

>
> >         N_("git commit-graph write [--object-dir <objdir>] [--append] "
> > -          "[--split[=<strategy>]] [--input=<reachable|stdin-packs|stdin-commits>] "
> > +          "[--split[=<strategy>]] "
> > +          "[--input=<reachable|stdin-packs|stdin-commits|none>] "
> >            "[--[no-]progress] <split options>"),
>
> Hmm, you've left "--append" the old way.

Fixed, and thanks for noticing.
>
> Martin

Thanks,
Taylor

Patch
diff mbox series

diff --git a/Documentation/git-commit-graph.txt b/Documentation/git-commit-graph.txt
index cbf80226e9..d380c42e82 100644
--- a/Documentation/git-commit-graph.txt
+++ b/Documentation/git-commit-graph.txt
@@ -39,24 +39,29 @@  COMMANDS
 --------
 'write'::
 
-Write a commit-graph file based on the commits found in packfiles.
+Write a commit-graph file based on the commits specified:
+* With the `--input=stdin-packs` option, generate the new commit graph
+by walking objects only in the specified pack-indexes. (Cannot be
+combined with `--input=stdin-commits` or `--input=reachable`.)
 +
-With the `--input=stdin-packs` option, generate the new commit graph by
-walking objects only in the specified pack-indexes. (Cannot be combined
-with `--input=stdin-commits` or `--input=reachable`.)
-+
-With the `--input=stdin-commits` option, generate the new commit graph
+* With the `--input=stdin-commits` option, generate the new commit graph
 by walking commits starting at the commits specified in stdin as a list
 of OIDs in hex, one OID per line. (Cannot be combined with
 `--input=stdin-packs` or `--input=reachable`.)
 +
-With the `--input=reachable` option, generate the new commit graph by
+* With the `--input=reachable` option, generate the new commit graph by
 walking commits starting at all refs. (Cannot be combined with
 `--input=stdin-commits` or `--input=stdin-packs`.)
 +
-With the `--input=append` option, include all commits that are present
+* With the `--input=append` option, include all commits that are present
 in the existing commit-graph file.
 +
+* With the `--input=none` option, behave as if `input=append` were
+given, but do not walk other packs to find additional commits.
+
+If none of the above options are given, then commits found in
+packfiles are specified.
++
 With the `--split[=<strategy>]` option, write the commit-graph as a
 chain of multiple commit-graph files stored in
 `<dir>/info/commit-graphs`. Commit-graph layers are merged based on the
diff --git a/builtin/commit-graph.c b/builtin/commit-graph.c
index 03d815e652..937b98e99e 100644
--- a/builtin/commit-graph.c
+++ b/builtin/commit-graph.c
@@ -10,7 +10,8 @@ 
 static char const * const builtin_commit_graph_usage[] = {
 	N_("git commit-graph verify [--object-dir <objdir>] [--shallow] [--[no-]progress]"),
 	N_("git commit-graph write [--object-dir <objdir>] [--append] "
-	   "[--split[=<strategy>]] [--input=<reachable|stdin-packs|stdin-commits>] "
+	   "[--split[=<strategy>]] "
+	   "[--input=<reachable|stdin-packs|stdin-commits|none>] "
 	   "[--[no-]progress] <split options>"),
 	NULL
 };
@@ -22,7 +23,8 @@  static const char * const builtin_commit_graph_verify_usage[] = {
 
 static const char * const builtin_commit_graph_write_usage[] = {
 	N_("git commit-graph write [--object-dir <objdir>] [--append] "
-	   "[--split[=<strategy>]] [--input=<reachable|stdin-packs|stdin-commits>] "
+	   "[--split[=<strategy>]] "
+	   "[--input=<reachable|stdin-packs|stdin-commits|none>] "
 	   "[--[no-]progress] <split options>"),
 	NULL
 };
@@ -31,7 +33,8 @@  enum commit_graph_input {
 	COMMIT_GRAPH_INPUT_REACHABLE     = (1 << 1),
 	COMMIT_GRAPH_INPUT_STDIN_PACKS   = (1 << 2),
 	COMMIT_GRAPH_INPUT_STDIN_COMMITS = (1 << 3),
-	COMMIT_GRAPH_INPUT_APPEND        = (1 << 4)
+	COMMIT_GRAPH_INPUT_APPEND        = (1 << 4),
+	COMMIT_GRAPH_INPUT_NONE          = (1 << 5)
 };
 
 static struct opts_commit_graph {
@@ -59,6 +62,8 @@  static int option_parse_input(const struct option *opt, const char *arg,
 		*to |= COMMIT_GRAPH_INPUT_STDIN_COMMITS;
 	else if (!strcmp(arg, "append"))
 		*to |= COMMIT_GRAPH_INPUT_APPEND;
+	else if (!strcmp(arg, "none"))
+		*to |= (COMMIT_GRAPH_INPUT_APPEND | COMMIT_GRAPH_INPUT_NONE);
 	else
 		die(_("unrecognized --input source, %s"), arg);
 	return 0;
@@ -211,6 +216,8 @@  static int graph_write(int argc, const char **argv)
 		opts.obj_dir = get_object_directory();
 	if (opts.input & COMMIT_GRAPH_INPUT_APPEND)
 		flags |= COMMIT_GRAPH_WRITE_APPEND;
+	if (opts.input & COMMIT_GRAPH_INPUT_NONE)
+		flags |= COMMIT_GRAPH_WRITE_NO_INPUT;
 	if (opts.split)
 		flags |= COMMIT_GRAPH_WRITE_SPLIT;
 	if (opts.progress)
diff --git a/commit-graph.c b/commit-graph.c
index 02e6ad9d1f..a5d7624073 100644
--- a/commit-graph.c
+++ b/commit-graph.c
@@ -808,7 +808,8 @@  struct write_commit_graph_context {
 	unsigned append:1,
 		 report_progress:1,
 		 split:1,
-		 check_oids:1;
+		 check_oids:1,
+		 no_input:1;
 
 	const struct split_commit_graph_opts *split_opts;
 };
@@ -1802,6 +1803,7 @@  int write_commit_graph(struct object_directory *odb,
 	ctx->split = flags & COMMIT_GRAPH_WRITE_SPLIT ? 1 : 0;
 	ctx->check_oids = flags & COMMIT_GRAPH_WRITE_CHECK_OIDS ? 1 : 0;
 	ctx->split_opts = split_opts;
+	ctx->no_input = flags & COMMIT_GRAPH_WRITE_NO_INPUT ? 1 : 0;
 
 	if (ctx->split) {
 		struct commit_graph *g;
@@ -1860,7 +1862,7 @@  int write_commit_graph(struct object_directory *odb,
 			goto cleanup;
 	}
 
-	if (!pack_indexes && !commit_hex)
+	if (!ctx->no_input && !pack_indexes && !commit_hex)
 		fill_oids_from_all_packs(ctx);
 
 	close_reachable(ctx);
diff --git a/commit-graph.h b/commit-graph.h
index dadcc03808..dd8c00a2d8 100644
--- a/commit-graph.h
+++ b/commit-graph.h
@@ -81,7 +81,8 @@  enum commit_graph_write_flags {
 	COMMIT_GRAPH_WRITE_PROGRESS   = (1 << 1),
 	COMMIT_GRAPH_WRITE_SPLIT      = (1 << 2),
 	/* Make sure that each OID in the input is a valid commit OID. */
-	COMMIT_GRAPH_WRITE_CHECK_OIDS = (1 << 3)
+	COMMIT_GRAPH_WRITE_CHECK_OIDS = (1 << 3),
+	COMMIT_GRAPH_WRITE_NO_INPUT   = (1 << 4)
 };
 
 enum commit_graph_split_flags {
diff --git a/t/t5324-split-commit-graph.sh b/t/t5324-split-commit-graph.sh
index dd74295885..296b5a9185 100755
--- a/t/t5324-split-commit-graph.sh
+++ b/t/t5324-split-commit-graph.sh
@@ -369,4 +369,30 @@  test_expect_success '--split=no-merge always writes an incremental' '
 	test_line_count = 2 $graphdir/commit-graph-chain
 '
 
+test_expect_success '--split=no-merge, --input=none writes nothing' '
+	test_when_finished rm -rf a graphs.before graphs.after &&
+	rm -rf $graphdir &&
+	git reset --hard commits/2 &&
+	git rev-list -1 HEAD~1 >a &&
+	git commit-graph write --split=no-merge --input=stdin-commits <a &&
+	ls $graphdir/graph-*.graph >graphs.before &&
+	test_line_count = 1 $graphdir/commit-graph-chain &&
+	git commit-graph write --split --input=none &&
+	ls $graphdir/graph-*.graph >graphs.after &&
+	test_cmp graphs.before graphs.after
+'
+
+test_expect_success '--split=merge-all, --input=none merges the chain' '
+	test_when_finished rm -rf a b &&
+	rm -rf $graphdir &&
+	git reset --hard commits/2 &&
+	git rev-list -1 HEAD~1 >a &&
+	git rev-list -1 HEAD >b &&
+	git commit-graph write --split=no-merge --input=stdin-commits <a &&
+	git commit-graph write --split=no-merge --input=stdin-commits <b &&
+	test_line_count = 2 $graphdir/commit-graph-chain &&
+	git commit-graph write --split=merge-all --input=none &&
+	test_line_count = 1 $graphdir/commit-graph-chain
+'
+
 test_done