| Message ID | 892933752c638a9ed938e046e466dd1e927058a6.1584169893.git.gitgitgadget@gmail.com (mailing list archive) |
|---|---|
| State | New, archived |
| Headers | show |
| Series | Sparse checkout improvements -- improved sparsity updating | expand |
On 3/14/2020 3:11 AM, Elijah Newren via GitGitGadget wrote: > From: Elijah Newren <newren@gmail.com> > > Remove the equivalent of 'git read-tree -mu HEAD' in the sparse-checkout > codepaths for setting the SKIP_WORKTREE bits and instead use the new > update_sparsity() function. > > Signed-off-by: Elijah Newren <newren@gmail.com> > --- > builtin/sparse-checkout.c | 39 +++++++----------------------- > t/t1091-sparse-checkout-builtin.sh | 32 +++++++++++++++++------- > 2 files changed, 32 insertions(+), 39 deletions(-) > > diff --git a/builtin/sparse-checkout.c b/builtin/sparse-checkout.c > index 740da4b6d54..5d3ec2e6be9 100644 > --- a/builtin/sparse-checkout.c > +++ b/builtin/sparse-checkout.c > @@ -97,47 +97,32 @@ static int update_working_directory(struct pattern_list *pl) > int result = 0; > struct unpack_trees_options o; > struct lock_file lock_file = LOCK_INIT; > - struct object_id oid; > - struct tree *tree; > - struct tree_desc t; > struct repository *r = the_repository; > > - if (repo_read_index_unmerged(r)) > - die(_("you need to resolve your current index first")); > - > - if (get_oid("HEAD", &oid)) > - return 0; > - > - tree = parse_tree_indirect(&oid); > - parse_tree(tree); > - init_tree_desc(&t, tree->buffer, tree->size); > - > memset(&o, 0, sizeof(o)); > o.verbose_update = isatty(2); > - o.merge = 1; > o.update = 1; > - o.fn = oneway_merge; I'm glad this is getting much simpler. > o.head_idx = -1; > o.src_index = r->index; > o.dst_index = r->index; > o.skip_sparse_checkout = 0; > o.pl = pl; > - o.keep_pattern_list = !!pl; I'm making a mental note to see if this is necessary or not. I'm guessing that update_sparsity() expects a pattern list as input and will never free it, so this option is useless. > - resolve_undo_clear_index(r->index); > setup_work_tree(); > > - cache_tree_free(&r->index->cache_tree); > - > repo_hold_locked_index(r, &lock_file, LOCK_DIE_ON_ERROR); > > - core_apply_sparse_checkout = 1; > - result = unpack_trees(1, &t, &o); > + result = update_sparsity(&o); > > - if (!result) { > - prime_cache_tree(r, r->index, tree); > + if (result == 1) I wonder if it is appropriate to return an enum in update_sparsity() so this statement is clearer out-of-context. For example, I'm looking at this patch before looking closely at the update_sparsity() patch so I know how it will be used before I check its implementation. > + /* > + * We don't do any special handling of warnings from untracked > + * files in the way or dirty entries that can't be removed. > + */ > + result = 0; > + if (!result) > write_locked_index(r->index, &lock_file, COMMIT_LOCK); > - } else > + else > rollback_lock_file(&lock_file); > > return result; > @@ -304,8 +289,6 @@ static int sparse_checkout_init(int argc, const char **argv) > }; > > repo_read_index(the_repository); > - require_clean_work_tree(the_repository, > - N_("initialize sparse-checkout"), NULL, 1, 0); > > argc = parse_options(argc, argv, NULL, > builtin_sparse_checkout_init_options, > @@ -560,8 +543,6 @@ static int sparse_checkout_set(int argc, const char **argv, const char *prefix, > }; > > repo_read_index(the_repository); > - require_clean_work_tree(the_repository, > - N_("set sparse-checkout patterns"), NULL, 1, 0); > > argc = parse_options(argc, argv, prefix, > builtin_sparse_checkout_set_options, > @@ -577,8 +558,6 @@ static int sparse_checkout_disable(int argc, const char **argv) > struct strbuf match_all = STRBUF_INIT; > > repo_read_index(the_repository); > - require_clean_work_tree(the_repository, > - N_("disable sparse-checkout"), NULL, 1, 0); In the three hunks above, do we still need repo_read_index()? > memset(&pl, 0, sizeof(pl)); > hashmap_init(&pl.recursive_hashmap, pl_hashmap_cmp, NULL, 0); > diff --git a/t/t1091-sparse-checkout-builtin.sh b/t/t1091-sparse-checkout-builtin.sh > index 8607a8e6d1a..0d93d3983e0 100755 > --- a/t/t1091-sparse-checkout-builtin.sh > +++ b/t/t1091-sparse-checkout-builtin.sh > @@ -277,16 +277,23 @@ test_expect_success 'cone mode: add parent path' ' > check_files repo a deep folder1 > ' > > -test_expect_success 'revert to old sparse-checkout on bad update' ' > +test_expect_success 'not-up-to-date does not block rest of sparsification' ' > test_when_finished git -C repo sparse-checkout disable && > test_when_finished git -C repo reset --hard && > git -C repo sparse-checkout set deep && > + > echo update >repo/deep/deeper2/a && > cp repo/.git/info/sparse-checkout expect && > - test_must_fail git -C repo sparse-checkout set deep/deeper1 2>err && > - test_i18ngrep "cannot set sparse-checkout patterns" err && > - test_cmp repo/.git/info/sparse-checkout expect && > - check_files repo/deep a deeper1 deeper2 > + test_write_lines "!/deep/*/" "/deep/deeper1/" >>expect && > + > + git -C repo sparse-checkout set deep/deeper1 2>err && > + > + test_i18ngrep "Cannot update sparse checkout" err && > + test_cmp expect repo/.git/info/sparse-checkout && > + check_files repo/deep a deeper1 deeper2 && > + check_files repo/deep/deeper1 a deepest && > + check_files repo/deep/deeper1/deepest a && > + check_files repo/deep/deeper2 a > ' > > test_expect_success 'revert to old sparse-checkout on empty update' ' > @@ -316,12 +323,19 @@ test_expect_success '.gitignore should not warn about cone mode' ' > test_i18ngrep ! "disabling cone patterns" err > ' > > -test_expect_success 'sparse-checkout (init|set|disable) fails with dirty status' ' > +test_expect_success 'sparse-checkout (init|set|disable) warns with dirty status' ' > git clone repo dirty && > echo dirty >dirty/folder1/a && > - test_must_fail git -C dirty sparse-checkout init && > - test_must_fail git -C dirty sparse-checkout set /folder2/* /deep/deeper1/* && > - test_must_fail git -C dirty sparse-checkout disable && > + > + git -C dirty sparse-checkout init 2>err && > + test_i18ngrep "warning.*Cannot update sparse checkout" err && > + > + git -C dirty sparse-checkout set /folder2/* /deep/deeper1/* && > + test_i18ngrep "warning.*Cannot update sparse checkout" err && > + > + git -C dirty sparse-checkout disable && > + test_i18ngrep "warning.*Cannot update sparse checkout" err && > + > git -C dirty reset --hard && > git -C dirty sparse-checkout init && > git -C dirty sparse-checkout set /folder2/* /deep/deeper1/* && I like that the test changes here are limited to updating existing tests. You mentioned adding more tests, which I would also like, but those can come in a follow-up patch that is dedicated to all the fun situations we can get into, and how we should respond. Here are a few that I've tested manually: 1. Start: no sparse-checkout a. Get into rebase conflict in a directory. b. "git sparse-checkout init --cone" c. See that the file is on-disk with conflict markers. d. Resolve conflict and "git add" e. "git sparse-checkout set <dir-without-conflict>" f. Sparse-checkout is set as expected, file is not on disk, but still staged. g. "git rebase --continue" succeeds. 2. Start: sparse-checkout set to "dir1" a. Get into merge conflict in "dir2" b. Resolve conflict, "git add" c. "git sparse-checkout update" (available after next patch) d. dir1 gone, dir1/file staged. e. git merge --continue succeeds Those are the really big ones to me, but I'm sure we will find others. I'm really happy that this is working as we expect. Thanks! -Stolee
On Sun, Mar 15, 2020 at 9:19 AM Derrick Stolee <stolee@gmail.com> wrote: > > On 3/14/2020 3:11 AM, Elijah Newren via GitGitGadget wrote: > > From: Elijah Newren <newren@gmail.com> > > > > Remove the equivalent of 'git read-tree -mu HEAD' in the sparse-checkout > > codepaths for setting the SKIP_WORKTREE bits and instead use the new > > update_sparsity() function. > > > > Signed-off-by: Elijah Newren <newren@gmail.com> > > --- > > builtin/sparse-checkout.c | 39 +++++++----------------------- > > t/t1091-sparse-checkout-builtin.sh | 32 +++++++++++++++++------- > > 2 files changed, 32 insertions(+), 39 deletions(-) > > > > diff --git a/builtin/sparse-checkout.c b/builtin/sparse-checkout.c > > index 740da4b6d54..5d3ec2e6be9 100644 > > --- a/builtin/sparse-checkout.c > > +++ b/builtin/sparse-checkout.c > > @@ -97,47 +97,32 @@ static int update_working_directory(struct pattern_list *pl) > > int result = 0; > > struct unpack_trees_options o; > > struct lock_file lock_file = LOCK_INIT; > > - struct object_id oid; > > - struct tree *tree; > > - struct tree_desc t; > > struct repository *r = the_repository; > > > > - if (repo_read_index_unmerged(r)) > > - die(_("you need to resolve your current index first")); > > - > > - if (get_oid("HEAD", &oid)) > > - return 0; > > - > > - tree = parse_tree_indirect(&oid); > > - parse_tree(tree); > > - init_tree_desc(&t, tree->buffer, tree->size); > > - > > memset(&o, 0, sizeof(o)); > > o.verbose_update = isatty(2); > > - o.merge = 1; > > o.update = 1; > > - o.fn = oneway_merge; > > I'm glad this is getting much simpler. > > > o.head_idx = -1; > > o.src_index = r->index; > > o.dst_index = r->index; > > o.skip_sparse_checkout = 0; > > o.pl = pl; > > - o.keep_pattern_list = !!pl; > > I'm making a mental note to see if this is necessary or not. I'm > guessing that update_sparsity() expects a pattern list as input and > will never free it, so this option is useless. Yes, although your comment made me realize that I've got a memory leak if we pass o->pl==NULL to update_sparsity(), which happens whenever we pass NULL to update_working_directory(). I should fix that. > > - resolve_undo_clear_index(r->index); > > setup_work_tree(); > > > > - cache_tree_free(&r->index->cache_tree); > > - > > repo_hold_locked_index(r, &lock_file, LOCK_DIE_ON_ERROR); > > > > - core_apply_sparse_checkout = 1; > > - result = unpack_trees(1, &t, &o); > > + result = update_sparsity(&o); > > > > - if (!result) { > > - prime_cache_tree(r, r->index, tree); > > + if (result == 1) > I wonder if it is appropriate to return an enum in update_sparsity() > so this statement is clearer out-of-context. For example, I'm looking > at this patch before looking closely at the update_sparsity() patch > so I know how it will be used before I check its implementation. Yes, that'd be better. > > + /* > > + * We don't do any special handling of warnings from untracked > > + * files in the way or dirty entries that can't be removed. > > + */ > > + result = 0; > > + if (!result) > > write_locked_index(r->index, &lock_file, COMMIT_LOCK); > > - } else > > + else > > rollback_lock_file(&lock_file); > > > > return result; > > @@ -304,8 +289,6 @@ static int sparse_checkout_init(int argc, const char **argv) > > }; > > > > repo_read_index(the_repository); > > - require_clean_work_tree(the_repository, > > - N_("initialize sparse-checkout"), NULL, 1, 0); > > > > argc = parse_options(argc, argv, NULL, > > builtin_sparse_checkout_init_options, > > @@ -560,8 +543,6 @@ static int sparse_checkout_set(int argc, const char **argv, const char *prefix, > > }; > > > > repo_read_index(the_repository); > > - require_clean_work_tree(the_repository, > > - N_("set sparse-checkout patterns"), NULL, 1, 0); > > > > argc = parse_options(argc, argv, prefix, > > builtin_sparse_checkout_set_options, > > @@ -577,8 +558,6 @@ static int sparse_checkout_disable(int argc, const char **argv) > > struct strbuf match_all = STRBUF_INIT; > > > > repo_read_index(the_repository); > > - require_clean_work_tree(the_repository, > > - N_("disable sparse-checkout"), NULL, 1, 0); > > In the three hunks above, do we still need repo_read_index()? > > > memset(&pl, 0, sizeof(pl)); > > hashmap_init(&pl.recursive_hashmap, pl_hashmap_cmp, NULL, 0); > > diff --git a/t/t1091-sparse-checkout-builtin.sh b/t/t1091-sparse-checkout-builtin.sh > > index 8607a8e6d1a..0d93d3983e0 100755 > > --- a/t/t1091-sparse-checkout-builtin.sh > > +++ b/t/t1091-sparse-checkout-builtin.sh > > @@ -277,16 +277,23 @@ test_expect_success 'cone mode: add parent path' ' > > check_files repo a deep folder1 > > ' > > > > -test_expect_success 'revert to old sparse-checkout on bad update' ' > > +test_expect_success 'not-up-to-date does not block rest of sparsification' ' > > test_when_finished git -C repo sparse-checkout disable && > > test_when_finished git -C repo reset --hard && > > git -C repo sparse-checkout set deep && > > + > > echo update >repo/deep/deeper2/a && > > cp repo/.git/info/sparse-checkout expect && > > - test_must_fail git -C repo sparse-checkout set deep/deeper1 2>err && > > - test_i18ngrep "cannot set sparse-checkout patterns" err && > > - test_cmp repo/.git/info/sparse-checkout expect && > > - check_files repo/deep a deeper1 deeper2 > > + test_write_lines "!/deep/*/" "/deep/deeper1/" >>expect && > > + > > + git -C repo sparse-checkout set deep/deeper1 2>err && > > + > > + test_i18ngrep "Cannot update sparse checkout" err && > > + test_cmp expect repo/.git/info/sparse-checkout && > > + check_files repo/deep a deeper1 deeper2 && > > + check_files repo/deep/deeper1 a deepest && > > + check_files repo/deep/deeper1/deepest a && > > + check_files repo/deep/deeper2 a > > ' > > > > test_expect_success 'revert to old sparse-checkout on empty update' ' > > @@ -316,12 +323,19 @@ test_expect_success '.gitignore should not warn about cone mode' ' > > test_i18ngrep ! "disabling cone patterns" err > > ' > > > > -test_expect_success 'sparse-checkout (init|set|disable) fails with dirty status' ' > > +test_expect_success 'sparse-checkout (init|set|disable) warns with dirty status' ' > > git clone repo dirty && > > echo dirty >dirty/folder1/a && > > - test_must_fail git -C dirty sparse-checkout init && > > - test_must_fail git -C dirty sparse-checkout set /folder2/* /deep/deeper1/* && > > - test_must_fail git -C dirty sparse-checkout disable && > > + > > + git -C dirty sparse-checkout init 2>err && > > + test_i18ngrep "warning.*Cannot update sparse checkout" err && > > + > > + git -C dirty sparse-checkout set /folder2/* /deep/deeper1/* && > > + test_i18ngrep "warning.*Cannot update sparse checkout" err && > > + > > + git -C dirty sparse-checkout disable && > > + test_i18ngrep "warning.*Cannot update sparse checkout" err && > > + > > git -C dirty reset --hard && > > git -C dirty sparse-checkout init && > > git -C dirty sparse-checkout set /folder2/* /deep/deeper1/* && > > I like that the test changes here are limited to updating existing > tests. You mentioned adding more tests, which I would also like, but > those can come in a follow-up patch that is dedicated to all the fun > situations we can get into, and how we should respond. > > Here are a few that I've tested manually: > > 1. Start: no sparse-checkout > a. Get into rebase conflict in a directory. > b. "git sparse-checkout init --cone" > c. See that the file is on-disk with conflict markers. > d. Resolve conflict and "git add" > e. "git sparse-checkout set <dir-without-conflict>" > f. Sparse-checkout is set as expected, file is not on disk, but still staged. > g. "git rebase --continue" succeeds. > > 2. Start: sparse-checkout set to "dir1" > a. Get into merge conflict in "dir2" > b. Resolve conflict, "git add" > c. "git sparse-checkout update" (available after next patch) > d. dir1 gone, dir1/file staged. > e. git merge --continue succeeds > > Those are the really big ones to me, but I'm sure we will find others. > I'm really happy that this is working as we expect. Thanks for taking a look.
diff --git a/builtin/sparse-checkout.c b/builtin/sparse-checkout.c index 740da4b6d54..5d3ec2e6be9 100644 --- a/builtin/sparse-checkout.c +++ b/builtin/sparse-checkout.c @@ -97,47 +97,32 @@ static int update_working_directory(struct pattern_list *pl) int result = 0; struct unpack_trees_options o; struct lock_file lock_file = LOCK_INIT; - struct object_id oid; - struct tree *tree; - struct tree_desc t; struct repository *r = the_repository; - if (repo_read_index_unmerged(r)) - die(_("you need to resolve your current index first")); - - if (get_oid("HEAD", &oid)) - return 0; - - tree = parse_tree_indirect(&oid); - parse_tree(tree); - init_tree_desc(&t, tree->buffer, tree->size); - memset(&o, 0, sizeof(o)); o.verbose_update = isatty(2); - o.merge = 1; o.update = 1; - o.fn = oneway_merge; o.head_idx = -1; o.src_index = r->index; o.dst_index = r->index; o.skip_sparse_checkout = 0; o.pl = pl; - o.keep_pattern_list = !!pl; - resolve_undo_clear_index(r->index); setup_work_tree(); - cache_tree_free(&r->index->cache_tree); - repo_hold_locked_index(r, &lock_file, LOCK_DIE_ON_ERROR); - core_apply_sparse_checkout = 1; - result = unpack_trees(1, &t, &o); + result = update_sparsity(&o); - if (!result) { - prime_cache_tree(r, r->index, tree); + if (result == 1) + /* + * We don't do any special handling of warnings from untracked + * files in the way or dirty entries that can't be removed. + */ + result = 0; + if (!result) write_locked_index(r->index, &lock_file, COMMIT_LOCK); - } else + else rollback_lock_file(&lock_file); return result; @@ -304,8 +289,6 @@ static int sparse_checkout_init(int argc, const char **argv) }; repo_read_index(the_repository); - require_clean_work_tree(the_repository, - N_("initialize sparse-checkout"), NULL, 1, 0); argc = parse_options(argc, argv, NULL, builtin_sparse_checkout_init_options, @@ -560,8 +543,6 @@ static int sparse_checkout_set(int argc, const char **argv, const char *prefix, }; repo_read_index(the_repository); - require_clean_work_tree(the_repository, - N_("set sparse-checkout patterns"), NULL, 1, 0); argc = parse_options(argc, argv, prefix, builtin_sparse_checkout_set_options, @@ -577,8 +558,6 @@ static int sparse_checkout_disable(int argc, const char **argv) struct strbuf match_all = STRBUF_INIT; repo_read_index(the_repository); - require_clean_work_tree(the_repository, - N_("disable sparse-checkout"), NULL, 1, 0); memset(&pl, 0, sizeof(pl)); hashmap_init(&pl.recursive_hashmap, pl_hashmap_cmp, NULL, 0); diff --git a/t/t1091-sparse-checkout-builtin.sh b/t/t1091-sparse-checkout-builtin.sh index 8607a8e6d1a..0d93d3983e0 100755 --- a/t/t1091-sparse-checkout-builtin.sh +++ b/t/t1091-sparse-checkout-builtin.sh @@ -277,16 +277,23 @@ test_expect_success 'cone mode: add parent path' ' check_files repo a deep folder1 ' -test_expect_success 'revert to old sparse-checkout on bad update' ' +test_expect_success 'not-up-to-date does not block rest of sparsification' ' test_when_finished git -C repo sparse-checkout disable && test_when_finished git -C repo reset --hard && git -C repo sparse-checkout set deep && + echo update >repo/deep/deeper2/a && cp repo/.git/info/sparse-checkout expect && - test_must_fail git -C repo sparse-checkout set deep/deeper1 2>err && - test_i18ngrep "cannot set sparse-checkout patterns" err && - test_cmp repo/.git/info/sparse-checkout expect && - check_files repo/deep a deeper1 deeper2 + test_write_lines "!/deep/*/" "/deep/deeper1/" >>expect && + + git -C repo sparse-checkout set deep/deeper1 2>err && + + test_i18ngrep "Cannot update sparse checkout" err && + test_cmp expect repo/.git/info/sparse-checkout && + check_files repo/deep a deeper1 deeper2 && + check_files repo/deep/deeper1 a deepest && + check_files repo/deep/deeper1/deepest a && + check_files repo/deep/deeper2 a ' test_expect_success 'revert to old sparse-checkout on empty update' ' @@ -316,12 +323,19 @@ test_expect_success '.gitignore should not warn about cone mode' ' test_i18ngrep ! "disabling cone patterns" err ' -test_expect_success 'sparse-checkout (init|set|disable) fails with dirty status' ' +test_expect_success 'sparse-checkout (init|set|disable) warns with dirty status' ' git clone repo dirty && echo dirty >dirty/folder1/a && - test_must_fail git -C dirty sparse-checkout init && - test_must_fail git -C dirty sparse-checkout set /folder2/* /deep/deeper1/* && - test_must_fail git -C dirty sparse-checkout disable && + + git -C dirty sparse-checkout init 2>err && + test_i18ngrep "warning.*Cannot update sparse checkout" err && + + git -C dirty sparse-checkout set /folder2/* /deep/deeper1/* && + test_i18ngrep "warning.*Cannot update sparse checkout" err && + + git -C dirty sparse-checkout disable && + test_i18ngrep "warning.*Cannot update sparse checkout" err && + git -C dirty reset --hard && git -C dirty sparse-checkout init && git -C dirty sparse-checkout set /folder2/* /deep/deeper1/* &&