mbox series

[RFC,00/22] SHA-256 stage 4 implementation, part 1/3

Message ID 20200113124729.3684846-1-sandals@crustytoothpaste.net
Headers show
Series SHA-256 stage 4 implementation, part 1/3 | expand


brian m. carlson Jan. 13, 2020, 12:47 p.m. UTC
This is an RFC series for part 1 of 3 of a SHA-256 implementation.  It
contains a few interesting pieces, and there are also some pieces it
does not contain.

First, it contains the pieces necessary to set up repositories and write
_but not read_ extensions.objectFormat.  In other words, you can create
a SHA-256 repository, but will be unable to read it.  We also
intentionally fail any use of SHA-256 in a repository except when
DEVELOPER=1 is set to discourage anyone from wiring it up, since at this
point it is nonfunctional.

It additionally contains code to adjust the setup code as necessary,
handle repository version 1 in worktrees, properly handle signatures in
tags and commits, and import and export submodules using SHA-256
(required for converting the Git repository).

The setup code now learns an environment variable to specify a default
hash.  This is useful for us because otherwise the test suite would need
every call to "git init" (which is a lot) to add a command line
argument, which would be untidy and burdensome.  It is not recommended
for everyday use.

Furthermore, this contains direct calls to test_oid_init in the test
suite setup code, since the alternative is duplicating those values.
I've opted not to remove the calls in the tests because we have other
topics in flight that may conflict with doing that, but I plan to send a
follow-up patch which does that at the end.  The _z40 variable is
persisted there for compatibility with master and will be dropped once
my current test series in next hits master.

There are also things it does not contain.  As mentioned, it lacks
support for reading extensions.objectFormat at all.  It lacks support
for cloning, fetching, and pushing, which while considered non-goals in
the transition plan, are required for the test suite to even come close
to passing.  That code, both for the original protocol and v2, would be
in part 2.  That code does not provide interoperability between SHA-1
and SHA-256 repositories, which will be the subject of the next major
chunk I do.  Part 3 contains the portions making the
extensions.objectFormat option functional and permitting us to run tests
against the new version.

This series, of course, lacks the test suite fixes which will be
required for the test suite to pass.  Those will be coming in two
further series, one of which I plan to send out soon.

Because this series sets up (and documents!) a useless option (which is
a large footgun) and because I'd like feedback about this approach, this
series is RFC.  I'd also like to know if you think anything is missing
outside of the items I've mentioned, because part 3 will result in the
test suite depending on SHA-256 support and therefore any structural
changes will be difficult to make at that point.  And of course, any
other feedback about this series is certainly welcome.

There may be other things that are interesting about this series, but
this cover letter is too small to contain them, so I encourage you to
look at the series for yourself.

If you'd like to see what the entire series looks like when complete,
you're welcome to inspect the transition-stage-4 branch at

brian m. carlson (22):
  hex: introduce parsing variants taking hash algorithms
  hex: add functions to parse hex object IDs in any algorithm
  repository: require a build flag to use SHA-256
  t: use hash-specific lookup tables to define test constants
  t6300: abstract away SHA-1-specific constants
  t6300: make hash algorithm independent
  t/helper/test-dump-split-index: initialize git repository
  t/helper: initialize repository if necessary
  t/helper: make repository tests hash independent
  setup: allow check_repository_format to read repository format
  builtin/init-db: allow specifying hash algorithm on command line
  builtin/init-db: add environment variable for new repo hash
  init-db: move writing repo version into a function
  worktree: allow repository version 1
  commit: use expected signature header for SHA-256
  gpg-interface: improve interface for parsing tags
  tag: store SHA-256 signatures in a header
  fast-import: permit reading multiple marks files
  fast-import: add helper function for inserting mark object entries
  fast-import: make find_marks work on any mark set
  fast-import: add a generic function to iterate over marks
  fast-import: add options for rewriting submodules

 Documentation/git-fast-import.txt |  20 +++
 Documentation/git-init.txt        |   7 +-
 Documentation/git.txt             |   6 +
 builtin/clone.c                   |   2 +-
 builtin/commit.c                  |   2 +-
 builtin/fmt-merge-msg.c           |  26 +++-
 builtin/init-db.c                 |  70 +++++++--
 builtin/mktag.c                   |  14 ++
 builtin/receive-pack.c            |   4 +-
 builtin/tag.c                     |  20 ++-
 cache.h                           |  25 ++-
 commit.c                          |  58 +++++--
 commit.h                          |   8 +
 config.mak.dev                    |   2 +
 fast-import.c                     | 246 ++++++++++++++++++++++--------
 gpg-interface.c                   |  17 ++-
 gpg-interface.h                   |   9 +-
 hex.c                             |  57 ++++++-
 log-tree.c                        |  14 +-
 path.c                            |   2 +-
 ref-filter.c                      |  23 ++-
 repository.c                      |   4 +
 sequencer.c                       |   2 +-
 setup.c                           |   6 +-
 t/helper/test-dump-split-index.c  |   2 +
 t/helper/test-repository.c        |  14 +-
 t/t1450-fsck.sh                   |  24 +++
 t/t5801-remote-helpers.sh         |   4 +-
 t/t6300-for-each-ref.sh           |  61 +++++---
 t/t7004-tag.sh                    |   8 +-
 t/t7030-verify-tag.sh             |  17 +++
 t/t7510-signed-commit.sh          |  16 +-
 t/t9300-fast-import.sh            | 109 +++++++++++++
 t/test-lib.sh                     |  29 ++--
 tag.c                             |  15 +-
 worktree.c                        |  10 +-
 36 files changed, 758 insertions(+), 195 deletions(-)