mbox series

[00/22] Memory leak fixes (pt.4)

Message ID cover.1722933642.git.ps@pks.im (mailing list archive)
Headers show
Series Memory leak fixes (pt.4) | expand

Message

Patrick Steinhardt Aug. 6, 2024, 8:59 a.m. UTC
Hi,

the third set of memory leak fixes was merged to `next`, so this is the
next part of more or less random memory leak fixes all over the place.
With this series, we're at ~155 leaking test suites. Naturally, I've
already got v5 in the pipeline, which brings us down to ~120.

The series is built on top of 406f326d27 (The second batch, 2024-08-01)
with ps/leakfixes-part-3 at f30bfafcd4 (commit-reach: fix trivial memory
leak when computing reachability, 2024-08-01) merged into it.

Thanks!

Patrick

Patrick Steinhardt (22):
  remote: plug memory leak when aliasing URLs
  git: fix leaking system paths
  object-file: fix memory leak when reading corrupted headers
  object-name: fix leaking symlink paths in object context
  bulk-checkin: fix leaking state TODO
  read-cache: fix leaking hashfile when writing index fails
  submodule-config: fix leaking name enrty when traversing submodules
  config: fix leaking comment character config
  builtin/rebase: fix leaking `commit.gpgsign` value
  builtin/notes: fix leaking `struct notes_tree` when merging notes
  builtin/fast-import: plug trivial memory leaks
  builtin/fast-export: fix leaking diff options
  builtin/fast-export: plug leaking tag names
  merge-ort: unconditionally release attributes index
  sequencer: release todo list on error paths
  unpack-trees: clear index when not propagating it
  diff: fix leak when parsing invalid ignore regex option
  builtin/format-patch: fix various trivial memory leaks
  userdiff: fix leaking memory for configured diff drivers
  builtin/log: fix leak when showing converted blob contents
  diff: free state populated via options
  builtin/diff: free symmetric diff members

 builtin/diff.c                        | 10 ++-
 builtin/fast-export.c                 | 19 ++++--
 builtin/fast-import.c                 |  8 ++-
 builtin/log.c                         | 13 +++-
 builtin/notes.c                       |  9 ++-
 builtin/rebase.c                      |  8 +++
 bulk-checkin.c                        |  2 +
 config.c                              |  2 +
 csum-file.c                           |  2 +-
 csum-file.h                           | 10 +++
 diff.c                                | 16 ++++-
 environment.c                         |  3 +-
 environment.h                         |  1 +
 git.c                                 | 12 +++-
 merge-ort.c                           |  3 +-
 object-file.c                         |  1 +
 object-name.c                         |  1 +
 range-diff.c                          |  6 +-
 read-cache.c                          | 97 ++++++++++++++++-----------
 remote.c                              |  2 +
 sequencer.c                           | 65 +++++++++++++-----
 submodule-config.c                    | 18 +++--
 t/t0210-trace2-normal.sh              |  2 +-
 t/t1006-cat-file.sh                   |  1 +
 t/t1050-large.sh                      |  1 +
 t/t1450-fsck.sh                       |  1 +
 t/t1601-index-bogus.sh                |  2 +
 t/t2107-update-index-basic.sh         |  1 +
 t/t3310-notes-merge-manual-resolve.sh |  1 +
 t/t3311-notes-merge-fanout.sh         |  1 +
 t/t3404-rebase-interactive.sh         |  1 +
 t/t3435-rebase-gpg-sign.sh            |  1 +
 t/t3507-cherry-pick-conflict.sh       |  1 +
 t/t3510-cherry-pick-sequence.sh       |  1 +
 t/t3705-add-sparse-checkout.sh        |  1 +
 t/t4013-diff-various.sh               |  1 +
 t/t4014-format-patch.sh               |  1 +
 t/t4018-diff-funcname.sh              |  1 +
 t/t4030-diff-textconv.sh              |  2 +
 t/t4042-diff-textconv-caching.sh      |  2 +
 t/t4048-diff-combined-binary.sh       |  1 +
 t/t4064-diff-oidfind.sh               |  2 +
 t/t4065-diff-anchored.sh              |  1 +
 t/t4068-diff-symmetric-merge-base.sh  |  1 +
 t/t4069-remerge-diff.sh               |  1 +
 t/t4108-apply-threeway.sh             |  1 +
 t/t4209-log-pickaxe.sh                |  2 +
 t/t6421-merge-partial-clone.sh        |  1 +
 t/t6428-merge-conflicts-sparse.sh     |  1 +
 t/t7008-filter-branch-null-sha1.sh    |  1 +
 t/t7030-verify-tag.sh                 |  1 +
 t/t7817-grep-sparse-checkout.sh       |  1 +
 t/t9300-fast-import.sh                |  1 +
 t/t9304-fast-import-marks.sh          |  2 +
 t/t9351-fast-export-anonymize.sh      |  1 +
 unpack-trees.c                        |  2 +
 userdiff.c                            | 38 ++++++++---
 userdiff.h                            |  4 ++
 58 files changed, 289 insertions(+), 103 deletions(-)

Comments

James Liu Aug. 7, 2024, 9:27 a.m. UTC | #1
On Tue Aug 6, 2024 at 6:59 PM AEST, Patrick Steinhardt wrote:
> Hi,
>
> the third set of memory leak fixes was merged to `next`, so this is the
> next part of more or less random memory leak fixes all over the place.
> With this series, we're at ~155 leaking test suites. Naturally, I've
> already got v5 in the pipeline, which brings us down to ~120.
>
> The series is built on top of 406f326d27 (The second batch, 2024-08-01)
> with ps/leakfixes-part-3 at f30bfafcd4 (commit-reach: fix trivial memory
> leak when computing reachability, 2024-08-01) merged into it.
>
> Thanks!
>
> Patrick

Thanks Patrick, most of these fixes make sense to me! I appreciate that
even the minor changes are accompanied by context.

Cheers,
James
Junio C Hamano Aug. 7, 2024, 4:59 p.m. UTC | #2
Patrick Steinhardt <ps@pks.im> writes:

> The series is built on top of 406f326d27 (The second batch, 2024-08-01)
> with ps/leakfixes-part-3 at f30bfafcd4 (commit-reach: fix trivial memory
> leak when computing reachability, 2024-08-01) merged into it.

A quick question.  Is it on your radar that transport_get() leaks
the helper name when "foo::bar" is given as a remote?

  https://github.com/git/git/actions/runs/10274435719/job/28431161208#step:5:893

If not, I'll handle it separately, whose fix should look something
like the attached.

Thanks.

 transport.c | 4 +++-
 1 file changed, 3 insertions(+), 1 deletion(-)

diff --git c/transport.c w/transport.c
index 12cc5b4d96..13bf8183b7 100644
--- c/transport.c
+++ w/transport.c
@@ -1115,6 +1115,7 @@ static struct transport_vtable builtin_smart_vtable = {
 struct transport *transport_get(struct remote *remote, const char *url)
 {
 	const char *helper;
+	char *helper_to_free = NULL;
 	const char *p;
 	struct transport *ret = xcalloc(1, sizeof(*ret));
 
@@ -1139,10 +1140,11 @@ struct transport *transport_get(struct remote *remote, const char *url)
 	while (is_urlschemechar(p == url, *p))
 		p++;
 	if (starts_with(p, "::"))
-		helper = xstrndup(url, p - url);
+		helper_to_free = helper = xstrndup(url, p - url);
 
 	if (helper) {
 		transport_helper_init(ret, helper);
+		free(helper_to_free);
 	} else if (starts_with(url, "rsync:")) {
 		die(_("git-over-rsync is no longer supported"));
 	} else if (url_is_local_not_ssh(url) && is_file(url) && is_bundle(url, 1)) {
Patrick Steinhardt Aug. 7, 2024, 5:03 p.m. UTC | #3
On Wed, Aug 07, 2024 at 09:59:39AM -0700, Junio C Hamano wrote:
> Patrick Steinhardt <ps@pks.im> writes:
> 
> > The series is built on top of 406f326d27 (The second batch, 2024-08-01)
> > with ps/leakfixes-part-3 at f30bfafcd4 (commit-reach: fix trivial memory
> > leak when computing reachability, 2024-08-01) merged into it.
> 
> A quick question.  Is it on your radar that transport_get() leaks
> the helper name when "foo::bar" is given as a remote?
> 
>   https://github.com/git/git/actions/runs/10274435719/job/28431161208#step:5:893
> 
> If not, I'll handle it separately, whose fix should look something
> like the attached.
> 
> Thanks.

Yeah, it's in part 5 [1], 97613b9cb9 (transport-helper: fix leaking
helper name, 2024-05-27). Feel free to handle it separately though, I'll
wait for part 4 to land first anyway, which likely takes a couple of
days.

Patrick

[1]: https://gitlab.com/gitlab-org/git/-/tree/pks-leak-fixes-pt5
Junio C Hamano Aug. 8, 2024, 12:32 a.m. UTC | #4
Patrick Steinhardt <ps@pks.im> writes:

> On Wed, Aug 07, 2024 at 09:59:39AM -0700, Junio C Hamano wrote:
>> Patrick Steinhardt <ps@pks.im> writes:
>> 
>> > The series is built on top of 406f326d27 (The second batch, 2024-08-01)
>> > with ps/leakfixes-part-3 at f30bfafcd4 (commit-reach: fix trivial memory
>> > leak when computing reachability, 2024-08-01) merged into it.
>> 
>> A quick question.  Is it on your radar that transport_get() leaks
>> the helper name when "foo::bar" is given as a remote?
>> 
>>   https://github.com/git/git/actions/runs/10274435719/job/28431161208#step:5:893
>> 
>> If not, I'll handle it separately, whose fix should look something
>> like the attached.
>> 
>> Thanks.
>
> Yeah, it's in part 5 [1], 97613b9cb9 (transport-helper: fix leaking
> helper name, 2024-05-27). Feel free to handle it separately though, I'll
> wait for part 4 to land first anyway, which likely takes a couple of
> days.

OK, will do, as this seems to break CI.
Patrick Steinhardt Aug. 8, 2024, 5:05 a.m. UTC | #5
On Wed, Aug 07, 2024 at 07:27:32PM +1000, James Liu wrote:
> On Tue Aug 6, 2024 at 6:59 PM AEST, Patrick Steinhardt wrote:
> > Hi,
> >
> > the third set of memory leak fixes was merged to `next`, so this is the
> > next part of more or less random memory leak fixes all over the place.
> > With this series, we're at ~155 leaking test suites. Naturally, I've
> > already got v5 in the pipeline, which brings us down to ~120.
> >
> > The series is built on top of 406f326d27 (The second batch, 2024-08-01)
> > with ps/leakfixes-part-3 at f30bfafcd4 (commit-reach: fix trivial memory
> > leak when computing reachability, 2024-08-01) merged into it.
> >
> > Thanks!
> >
> > Patrick
> 
> Thanks Patrick, most of these fixes make sense to me! I appreciate that
> even the minor changes are accompanied by context.

Thanks for your review!

Patrick
James Liu Aug. 8, 2024, 6 a.m. UTC | #6
On Thu Aug 8, 2024 at 3:05 PM AEST, Patrick Steinhardt wrote:
> On Wed, Aug 07, 2024 at 07:27:32PM +1000, James Liu wrote:
> > On Tue Aug 6, 2024 at 6:59 PM AEST, Patrick Steinhardt wrote:
> > > Hi,
> > >
> > > the third set of memory leak fixes was merged to `next`, so this is the
> > > next part of more or less random memory leak fixes all over the place.
> > > With this series, we're at ~155 leaking test suites. Naturally, I've
> > > already got v5 in the pipeline, which brings us down to ~120.
> > >
> > > The series is built on top of 406f326d27 (The second batch, 2024-08-01)
> > > with ps/leakfixes-part-3 at f30bfafcd4 (commit-reach: fix trivial memory
> > > leak when computing reachability, 2024-08-01) merged into it.
> > >
> > > Thanks!
> > >
> > > Patrick
> > 
> > Thanks Patrick, most of these fixes make sense to me! I appreciate that
> > even the minor changes are accompanied by context.
>
> Thanks for your review!
>
> Patrick

Thanks for responding to my questions! I don't have anything further to
add.

Cheers,
James