[00/16] mktree: support more flexible usage

Message

Philippe Blain via GitGitGadget June 11, 2024, 6:24 p.m. UTC

The goal of this series is to make 'git mktree' a much more flexible and
powerful tool for constructing arbitrary trees in memory without the use of
an index or worktree. The main additions are:

 * Using an optional "base tree" to add or replace entries in an existing
   tree rather than creating a new one from scratch.
   * Building off of this, having entries with mode "0" indicate "remove
     this entry, if it exists, from the tree"
 * Handling tree entries inside of subtrees (e.g., folder1/my-file.txt)

It also introduces some quality-of-life updates:

 * Using the same input parsing as 'update-index' to allow a wider variety
   of tree entry formats.
 * Adding deduplication of input entries & more thorough validation of
   inputs (with an option to disable both - plus input sorting - if desired
   with '--literally').

The implementation change underpinning the new features is completely
revamping how the tree is constructed in memory. Instead of writing a single
tree object into a strbuf and hashing it into the object database, we
construct an in-core sparse index and write out the root tree, as well as
any new subtrees, using the cache tree infrastructure.

The series is organized as follows:

 * Commits 1-3 contain miscellaneous small renames/refactors to make the
   code more readable & prepare for larger refactoring later.
 * Commits 4-7 generalize the input parsing performed by 'read_index_info()'
   in 'update-index' and update 'mktree' to use it.
 * Commit 8 adds the '--literally' option to 'mktree'. Practically, this
   option allows tests that currently use 'mktree' to generate corrupt trees
   to continue functioning after we strengthen input validations.
 * Commits 9 & 10 add input path validation & entry deduplication,
   respectively.
 * Commit 11 replaces the strbuf-to-object tree creation with construction
   of an in-core index & writing out the cache tree.
 * Commits 12-14 add the ability to add tree entries to an existing "base"
   tree. Takes 3 commits to do it because it requires a bit of finesse
   around directory/file deduplication and iterating over a tree with
   'read_tree()' with a parallel iteration over the input tree entries.
 * Commit 15 allows for deeper paths in the input.
 * Commit 16 adds handling for mode '0' as "removal" entries.

I also plan to add a '--strict' option that runs 'fsck' checks on the new
tree(s) before writing to the object database (similar to 'mkttag
--strict'), but this series is pretty long as it is and that part can easily
be separated out into its own series.

Thanks!

 * Victoria

Victoria Dye (16):
  mktree: use OPT_BOOL
  mktree: rename treeent to tree_entry
  mktree: use non-static tree_entry array
  update-index: generalize 'read_index_info'
  index-info.c: identify empty input lines in read_index_info
  index-info.c: parse object type in provided in read_index_info
  mktree: use read_index_info to read stdin lines
  mktree: add a --literally option
  mktree: validate paths more carefully
  mktree: overwrite duplicate entries
  mktree: create tree using an in-core index
  mktree: use iterator struct to add tree entries to index
  mktree: add directory-file conflict hashmap
  mktree: optionally add to an existing tree
  mktree: allow deeper paths in input
  mktree: remove entries when mode is 0

 Documentation/git-mktree.txt       |  42 +-
 Makefile                           |   1 +
 builtin/mktree.c                   | 595 +++++++++++++++++++++++------
 builtin/update-index.c             | 119 ++----
 index-info.c                       | 104 +++++
 index-info.h                       |  14 +
 t/t1010-mktree.sh                  | 354 ++++++++++++++++-
 t/t1014-read-tree-confusing.sh     |   6 +-
 t/t1450-fsck.sh                    |   4 +-
 t/t1601-index-bogus.sh             |   2 +-
 t/t1700-split-index.sh             |   6 +-
 t/t2107-update-index-basic.sh      |  32 ++
 t/t7008-filter-branch-null-sha1.sh |   6 +-
 t/t7417-submodule-path-url.sh      |   2 +-
 t/t7450-bad-git-dotfiles.sh        |   8 +-
 15 files changed, 1055 insertions(+), 240 deletions(-)
 create mode 100644 index-info.c
 create mode 100644 index-info.h


base-commit: 8d94cfb54504f2ec9edc7ca3eb5c29a3dd3675ae
Published-As: https://github.com/gitgitgadget/git/releases/tag/pr-1746%2Fvdye%2Fvdye%2Fmktree-recursive-v1
Fetch-It-Via: git fetch https://github.com/gitgitgadget/git pr-1746/vdye/vdye/mktree-recursive-v1
Pull-Request: https://github.com/gitgitgadget/git/pull/1746