[00/31] Hash function transition part 16

Message

brian m. carlson Feb. 12, 2019, 1:22 a.m. UTC

This is the sixteenth series of hash function transition patches. This
series contains various fixes, mostly focused around the pack bitmap
code, the HTTP code, the archive code, the index, and parts of our Perl
code.

This is the second to last series required for a "stage 0" Git; that is,
one that can operate only with SHA-256, but not SHA-1. Observers will
notice a focus on getting rid of sha1_to_hex and null_sha1 as well as
the normal types of transforms; the next series will remove both of
these.

This series modifies the index code such that it can work with a hash
algorithm of any length. In order to do so, the structs involved were
changed to use flex array members and not store the hash in a fixed
array member. This design was chosen over a multiple struct approach
because it ensures that we have one consistent, well-tested code path
that works for both algorithms, as well as any algorithms in the future.
Comments on the approach or arguments for other designs are welcome.

This is a rather long series, but most of it is concentrated in a few
small areas, so hopefully it's a little easier to review because of
that.

To preview the series that come after this, there is an additional
series for stage 0 Git (object-id-part17 plus part of sha256-fixes),
plus potentially several series of test fixes (test-fixes-part4 and part
of sha256-fixes). Following that, I plan to introduce, under the
DEVELOPER Makefile flag, the actual code which supports
extensions.objectFormat and makes it so that it works
(transition-stage-4).

brian m. carlson (31):
  t/lib-submodule-update: use appropriate length constant
  pack-bitmap: make bitmap header handling hash agnostic
  pack-bitmap: convert struct stored_bitmap to object_id
  pack-bitmap: replace sha1_to_hex
  pack-bitmap: switch hard-coded constants to the_hash_algo
  submodule: avoid hard-coded constants
  notes-merge: switch to use the_hash_algo
  notes: make hash size independent
  notes: replace sha1_to_hex
  object-store: rename and expand packed_git's sha1 member
  builtin/name-rev: make hash-size independent
  fast-import: make hash-size independent
  fast-import: replace sha1_to_hex
  builtin/am: make hash size independent
  builtin/pull: make hash-size independent
  http-push: convert to use the_hash_algo
  http-backend: allow 64-character hex names
  http-push: remove remaining uses of sha1_to_hex
  http-walker: replace sha1_to_hex
  http: replace hard-coded constant with the_hash_algo
  http: compute hash of downloaded objects using the_hash_algo
  http: replace sha1_to_hex
  remote-curl: make hash size independent
  archive-tar: make hash size independent
  archive: convert struct archiver_args to object_id
  refspec: make hash size independent
  builtin/difftool: use parse_oid_hex
  dir: make untracked cache extension hash size independent
  read-cache: read data in a hash-independent way
  Git.pm: make hash size independent
  gitweb: make hash size independent

 archive-tar.c               |  7 ++--
 archive-zip.c               | 10 ++---
 archive.c                   |  8 ++--
 archive.h                   |  2 +-
 builtin/am.c                |  9 +++--
 builtin/difftool.c          |  6 +--
 builtin/get-tar-commit-id.c | 11 +++++-
 builtin/name-rev.c          | 14 ++++---
 builtin/pack-redundant.c    |  2 +-
 builtin/pull.c              |  5 ++-
 dir.c                       | 28 +++++++-------
 fast-import.c               | 48 +++++++++++++-----------
 gitweb/gitweb.perl          | 63 ++++++++++++++++---------------
 http-backend.c              |  3 ++
 http-push.c                 | 29 ++++++++-------
 http-walker.c               | 18 ++++-----
 http.c                      | 33 +++++++++--------
 http.h                      |  2 +-
 merge-recursive.c           |  2 +-
 notes-merge.c               |  6 +--
 notes.c                     | 44 +++++++++++-----------
 object-store.h              |  2 +-
 pack-bitmap-write.c         |  8 ++--
 pack-bitmap.c               | 20 +++++-----
 pack-bitmap.h               |  2 +-
 packfile.c                  |  6 +--
 perl/Git.pm                 |  2 +-
 read-cache.c                | 74 +++++++++++++++----------------------
 refspec.c                   |  2 +-
 remote-curl.c               | 11 +++---
 submodule.c                 |  2 +-
 t/lib-submodule-update.sh   |  3 +-
 32 files changed, 246 insertions(+), 236 deletions(-)

Comments

Ævar Arnfjörð Bjarmason Feb. 12, 2019, 11:15 a.m. UTC | #1

On Tue, Feb 12 2019, brian m. carlson wrote:

> This is the sixteenth series of hash function transition patches. This
> series contains various fixes, mostly focused around the pack bitmap
> code, the HTTP code, the archive code, the index, and parts of our Perl
> code.
>
> This is the second to last series required for a "stage 0" Git; that is,

I skimmed most of this, but decided to stop when I came to "am" since I
was just going to start repeating the same question I had in other
patches, i.e. for the parts of this that deal with on-disk formats how
does just e.g. search-replacing s/40/64/ interact with needing to read
existing files (bitmaps, "am" patches, untracked cache etc.) which may
be in the "old" format.