[00/22] reftable: handle allocation errors

Message

Patrick Steinhardt Sept. 16, 2024, 12:28 p.m. UTC

Hi,

the reftable library provides pluggable allocators that can be switched
out via `reftable_set_alloc()`. These pluggable allocators are not
expected to behave like `xmalloc()` and friends -- in other words, they
are allowed to fail. This is done such that the reftable library can
truly behave like a library and let its callers handle such conditions.

It has been true since the inception of the reftable library that
allocations may fail. But regardless of that we do not have any error
handling for this scenario at all. Consequently, we can easily cause a
segfault when running into an out-of-memory situation.

While the easy solution would be to plug in `xmalloc()` and friends such
that we abort on these cases, that would contradict the very notion of
what this library should be. Most importantly, I want to soonish revive
the effort to implement a reftable backend in libgit2, and thus I have
to make sure that the reftable library (of which we are the canonical
upstream now) can be used in such contexts.

This patch series here is the first one in a set of series that will
detangle the reftable library from the rest of the Git codebase again so
that it becomes a true standalone library that other projects can use.

The series is based on top of ed155187b4 (Sync with Git 2.46.1,
2024-09-13) with the following two depedencies merged into it to avoid
conflicts:

  - cp/unit-test-reftable-stack at 2b14ced370 (t-reftable-stack: add
    test for stack iterators, 2024-09-08).

  - ps/reftable-exclude at adf7a0ceef (refs/reftable: wire up support
    for exclude patterns, 2024-09-09).

Thanks!

Patrick

Patrick Steinhardt (22):
  reftable/error: introduce out-of-memory error code
  reftable/basics: merge "publicbasics" into "basics"
  reftable: introduce `reftable_strdup()`
  reftable/basics: handle allocation failures in `reftable_calloc()`
  reftable/basics: handle allocation failures in `parse_names()`
  reftable/record: handle allocation failures on copy
  reftable/record: handle allocation failures when decoding records
  reftable/writer: handle allocation failures in `writer_index_hash()`
  reftable/writer: handle allocation failures in `reftable_new_writer()`
  reftable/merged: handle allocation failures in
    `merged_table_init_iter()`
  reftable/reader: handle allocation failures for unindexed reader
  reftable/reader: handle allocation failures in `reader_init_iter()`
  reftable/stack: handle allocation failures on reload
  reftable/stack: handle allocation failures in `reftable_new_stack()`
  reftable/stack: handle allocation failures in `stack_compact_range()`
  reftable/stack: handle allocation failures in auto compaction
  reftable/iter: handle allocation failures when creating indexed table
    iter
  reftable/blocksource: handle allocation failures
  reftable/block: handle allocation failures
  reftable/pq: handle allocation failures when adding entries
  reftable/tree: handle allocation failures
  reftable: handle trivial allocation failures

 Makefile                            |   1 -
 refs/reftable-backend.c             |  39 ++++--
 reftable/basics.c                   |  87 +++++++++++++-
 reftable/basics.h                   |  13 +-
 reftable/block.c                    |  23 +++-
 reftable/block.h                    |   4 +-
 reftable/blocksource.c              |  25 +++-
 reftable/error.c                    |   2 +
 reftable/iter.c                     |  22 +++-
 reftable/iter.h                     |   2 +-
 reftable/merged.c                   |  84 ++++++++-----
 reftable/merged.h                   |   6 +-
 reftable/pq.c                       |   7 +-
 reftable/pq.h                       |   2 +-
 reftable/publicbasics.c             |  66 -----------
 reftable/reader.c                   |  68 ++++++++---
 reftable/reader.h                   |   6 +-
 reftable/record.c                   | 164 +++++++++++++++++++-------
 reftable/record.h                   |   6 +-
 reftable/reftable-basics.h          |  18 +++
 reftable/reftable-error.h           |   3 +
 reftable/reftable-malloc.h          |  18 ---
 reftable/reftable-merged.h          |   8 +-
 reftable/reftable-reader.h          |   8 +-
 reftable/reftable-stack.h           |   8 +-
 reftable/reftable-writer.h          |  12 +-
 reftable/stack.c                    | 177 ++++++++++++++++++++++------
 reftable/tree.c                     |  42 +++++--
 reftable/tree.h                     |  21 +++-
 reftable/writer.c                   | 150 +++++++++++++++--------
 t/helper/test-reftable.c            |  10 +-
 t/unit-tests/lib-reftable.c         |   8 +-
 t/unit-tests/t-reftable-basics.c    |  11 +-
 t/unit-tests/t-reftable-block.c     |  24 ++--
 t/unit-tests/t-reftable-merged.c    |  16 ++-
 t/unit-tests/t-reftable-readwrite.c |  61 ++++++----
 t/unit-tests/t-reftable-stack.c     |   4 +-
 t/unit-tests/t-reftable-tree.c      |  10 +-
 38 files changed, 848 insertions(+), 388 deletions(-)
 delete mode 100644 reftable/publicbasics.c
 create mode 100644 reftable/reftable-basics.h
 delete mode 100644 reftable/reftable-malloc.h


base-commit: 1755182f5a2e12ff27c230f34f0040a8afee58e4