[WIP,0/3] Repack on fetch

Message

Jonathan Tan Oct. 21, 2024, 10:29 p.m. UTC

I think that ultimately we want something like repack on fetch, so I
made some effort to implement it. There were some details that needed
to be ironed out, but here's a WIP of the repack-on-fetch solution. In
particular, note that we do not need to create the expensive set used by
is_promisor_object().

As you can see from the patches, some polishing still needs to be
done, but I'm sending them out now to check if other people have
opinions about the solution. In particular, Han Young reported that an
alternative solution (repack on GC) takes too long [1], so I would be
interested to see if the time taken by this solution is good enough for
Han Young's use case.

[1] https://lore.kernel.org/git/[e-mail]/

Jonathan Tan (3):
  move variable
  pack-objects
  record local links and call pack-objects

 builtin/index-pack.c     | 116 ++++++++++++++++++++++++++++++++++++++-
 builtin/pack-objects.c   |  31 ++++++++++-
 t/t0410-partial-clone.sh |  11 ++--
 t/t5300-pack-object.sh   |   8 +--
 t/t5616-partial-clone.sh |  30 ++++++++++
 5 files changed, 183 insertions(+), 13 deletions(-)

Comments

Han Young Oct. 23, 2024, 7 a.m. UTC | #1

On Tue, Oct 22, 2024 at 6:29 AM Jonathan Tan <[e-mail]> wrote:
> As you can see from the patches, some polishing still needs to be
> done, but I'm sending them out now to check if other people have
> opinions about the solution. In particular, Han Young reported that an
> alternative solution (repack on GC) takes too long [1], so I would be
> interested to see if the time taken by this solution is good enough for
> Han Young's use case.

Thanks, I've tested the patches on our internal repos, the fetching time
increase isn't noticeable.

Jonathan Tan Oct. 23, 2024, 5:03 p.m. UTC | #2

Han Young <[e-mail]> writes:
> On Tue, Oct 22, 2024 at 6:29 AM Jonathan Tan <[e-mail]> wrote:
> > As you can see from the patches, some polishing still needs to be
> > done, but I'm sending them out now to check if other people have
> > opinions about the solution. In particular, Han Young reported that an
> > alternative solution (repack on GC) takes too long [1], so I would be
> > interested to see if the time taken by this solution is good enough for
> > Han Young's use case.
> 
> Thanks, I've tested the patches on our internal repos, the fetching time
> increase isn't noticeable.

Ah, thanks. I'm looking into why the tests are failing now - I have a
solution for t0410, but am still looking into the other two. I'll send
out non-WIP patches once I have them.