| Message ID | cover.1709640322.git.ps@pks.im (mailing list archive) |
|---|---|
| Headers | show |
| Series | reftable: memory optimizations for reflog iteration | expand |
On 2024.03.05 13:10, Patrick Steinhardt wrote: > Hi, > > this patch series does the same as all the preceding patch series that > optimized how the reftable library iterates through refs, but for > reflogs instead. > > The goal of this patch series is to arrive at a constant number of > allocations when iterating refs. This is achieved in mostly the same way > we did it for ref iteration, namely by reusing already-allocated memory. > Overall, this brings us down from 8 allocations per reflog record to > essentially 0 allocations per reflog. Iterating through 1 million > reflogs with `git reflog list` thus goes down from 8.068m allocations to > only around 68.5k. > > This series is built on top of "master" at b387623c12 (The third batch, > 2024-03-01) with Junio's "ps/reftable-iteration-perf-part2" at > 43f70eaea0 (refs/reftable: precompute prefix length, 2024-03-04) merged > into it. > > Patrick > > Patrick Steinhardt (7): > refs/reftable: reload correct stack when creating reflog iter > reftable/record: convert old and new object IDs to arrays > reftable/record: avoid copying author info > reftable/record: reuse refnames when decoding log records > reftable/record: reuse message when decoding log records > reftable/record: use scratch buffer when decoding records > refs/reftable: track last log record name via strbuf > > refs/reftable-backend.c | 52 +++++---------- > reftable/block.c | 4 +- > reftable/block.h | 2 + > reftable/merged_test.c | 11 ++-- > reftable/readwrite_test.c | 62 +++++++----------- > reftable/record.c | 129 ++++++++++++++----------------------- > reftable/record.h | 5 +- > reftable/record_test.c | 68 ++++++++++--------- > reftable/reftable-record.h | 6 +- > reftable/stack_test.c | 26 ++++---- > 10 files changed, 154 insertions(+), 211 deletions(-) > > -- > 2.44.0 > This series looks good to me (with one request that we add a test for patch #1 if possible). Reviewed-by: Josh Steadmon <steadmon@google.com>
On Mon, Mar 11, 2024 at 12:41:19PM -0700, Josh Steadmon wrote: > On 2024.03.05 13:10, Patrick Steinhardt wrote: > > Hi, > > > > this patch series does the same as all the preceding patch series that > > optimized how the reftable library iterates through refs, but for > > reflogs instead. > > > > The goal of this patch series is to arrive at a constant number of > > allocations when iterating refs. This is achieved in mostly the same way > > we did it for ref iteration, namely by reusing already-allocated memory. > > Overall, this brings us down from 8 allocations per reflog record to > > essentially 0 allocations per reflog. Iterating through 1 million > > reflogs with `git reflog list` thus goes down from 8.068m allocations to > > only around 68.5k. > > > > This series is built on top of "master" at b387623c12 (The third batch, > > 2024-03-01) with Junio's "ps/reftable-iteration-perf-part2" at > > 43f70eaea0 (refs/reftable: precompute prefix length, 2024-03-04) merged > > into it. > > > > Patrick > > > > Patrick Steinhardt (7): > > refs/reftable: reload correct stack when creating reflog iter > > reftable/record: convert old and new object IDs to arrays > > reftable/record: avoid copying author info > > reftable/record: reuse refnames when decoding log records > > reftable/record: reuse message when decoding log records > > reftable/record: use scratch buffer when decoding records > > refs/reftable: track last log record name via strbuf > > > > refs/reftable-backend.c | 52 +++++---------- > > reftable/block.c | 4 +- > > reftable/block.h | 2 + > > reftable/merged_test.c | 11 ++-- > > reftable/readwrite_test.c | 62 +++++++----------- > > reftable/record.c | 129 ++++++++++++++----------------------- > > reftable/record.h | 5 +- > > reftable/record_test.c | 68 ++++++++++--------- > > reftable/reftable-record.h | 6 +- > > reftable/stack_test.c | 26 ++++---- > > 10 files changed, 154 insertions(+), 211 deletions(-) > > > > -- > > 2.44.0 > > > > This series looks good to me (with one request that we add a test for > patch #1 if possible). > > Reviewed-by: Josh Steadmon <steadmon@google.com> Thanks for your review! Patrick