diff mbox series

[4/9] repository: add repo reference to index_state

Message ID d4ff0468fc0e22fe3ef82f74603ac781fd7711e1.1611161639.git.gitgitgadget@gmail.com (mailing list archive)
State New
Headers show
Series More index cleanups | expand

Commit Message

Derrick Stolee Jan. 20, 2021, 4:53 p.m. UTC
From: Derrick Stolee <dstolee@microsoft.com>

It will be helpful to add behavior to index opertations that might
trigger an object lookup. Since each index belongs to a specific
repository, add a 'repo' pointer to struct index_state that allows
access to this repository.

This will prevent future changes from needing to pass an additional
'struct repository *repo' parameter and instead rely only on the 'struct
index_state *istate' parameter.

Signed-off-by: Derrick Stolee <dstolee@microsoft.com>
---
 cache.h      | 1 +
 repository.c | 4 ++++
 2 files changed, 5 insertions(+)

Comments

Elijah Newren Jan. 20, 2021, 5:46 p.m. UTC | #1
On Wed, Jan 20, 2021 at 8:54 AM Derrick Stolee via GitGitGadget
<gitgitgadget@gmail.com> wrote:
>
> From: Derrick Stolee <dstolee@microsoft.com>
>
> It will be helpful to add behavior to index opertations that might

s/opertations/operations/

> trigger an object lookup. Since each index belongs to a specific
> repository, add a 'repo' pointer to struct index_state that allows
> access to this repository.
>
> This will prevent future changes from needing to pass an additional
> 'struct repository *repo' parameter and instead rely only on the 'struct
> index_state *istate' parameter.
>
> Signed-off-by: Derrick Stolee <dstolee@microsoft.com>
> ---
>  cache.h      | 1 +
>  repository.c | 4 ++++
>  2 files changed, 5 insertions(+)
>
> diff --git a/cache.h b/cache.h
> index 71097657489..f9c7a603841 100644
> --- a/cache.h
> +++ b/cache.h
> @@ -328,6 +328,7 @@ struct index_state {
>         struct ewah_bitmap *fsmonitor_dirty;
>         struct mem_pool *ce_mem_pool;
>         struct progress *progress;
> +       struct repository *repo;
>  };
>
>  /* Name hashing */
> diff --git a/repository.c b/repository.c
> index a4174ddb062..67a4c1da2d9 100644
> --- a/repository.c
> +++ b/repository.c
> @@ -264,6 +264,10 @@ int repo_read_index(struct repository *repo)
>         if (!repo->index)
>                 repo->index = xcalloc(1, sizeof(*repo->index));
>
> +       /* Complete the double-reference */
> +       if (!repo->index->repo)
> +               repo->index->repo = repo;
> +
>         return read_index_from(repo->index, repo->index_file, repo->gitdir);
>  }
>
> --
> gitgitgadget

Since we have repo->index and we have index->repo, which are intended
to be circular...what if they aren't?  Do we want or need to add
assertions anywhere that repo == repo->index->repo or that index ==
index->repo->index ?

My initial implementations of --remerge-diff[1] played around with
creating a second repo, with a different primary object store but
everything else the same.  The index for the two repository objects
was thus the same, and thus clearly would have violated this assumed
invariant for one of the two repos.  I discarded that initial
implementation (which I didn't quite have working) because I
discovered tmp-objdir.h and was able to add some
tmp_objdir_make_primary() and tmp_objdir_remove_as_primary() functions
that merely altered the existing repo's primary object store, but I'm
curious if there might be other cases of folks doing stuff that might
have weird failures with this new invariant.

It's entirely possible that --remerge-diff was just so different, and
I was so unfamiliar with repo objects (and still kind of am) that I
was just doing weird stuff no one has done before, so perhaps no
additional checks are needed -- I'm just throwing my gut question out
there as food for thought.



[1] I have not yet submitted `--remerge-diff` to the list; you haven't
missed anything.  I'm waiting for merge-ort to be submitted, reviewed,
and merged first.  It's the remerge-diff branch in my fork on GitHub
if anyone is curious, though.
Derrick Stolee Jan. 20, 2021, 7:16 p.m. UTC | #2
On 1/20/2021 12:46 PM, Elijah Newren wrote:
> On Wed, Jan 20, 2021 at 8:54 AM Derrick Stolee via GitGitGadget
> <gitgitgadget@gmail.com> wrote:
>>
>> From: Derrick Stolee <dstolee@microsoft.com>
>>
>> It will be helpful to add behavior to index opertations that might
> 
> s/opertations/operations/

Thanks.

>> trigger an object lookup. Since each index belongs to a specific
>> repository, add a 'repo' pointer to struct index_state that allows
>> access to this repository.
>>
>> This will prevent future changes from needing to pass an additional
>> 'struct repository *repo' parameter and instead rely only on the 'struct
>> index_state *istate' parameter.
>>
>> Signed-off-by: Derrick Stolee <dstolee@microsoft.com>
>> ---
>>  cache.h      | 1 +
>>  repository.c | 4 ++++
>>  2 files changed, 5 insertions(+)
>>
>> diff --git a/cache.h b/cache.h
>> index 71097657489..f9c7a603841 100644
>> --- a/cache.h
>> +++ b/cache.h
>> @@ -328,6 +328,7 @@ struct index_state {
>>         struct ewah_bitmap *fsmonitor_dirty;
>>         struct mem_pool *ce_mem_pool;
>>         struct progress *progress;
>> +       struct repository *repo;
>>  };
>>
>>  /* Name hashing */
>> diff --git a/repository.c b/repository.c
>> index a4174ddb062..67a4c1da2d9 100644
>> --- a/repository.c
>> +++ b/repository.c
>> @@ -264,6 +264,10 @@ int repo_read_index(struct repository *repo)
>>         if (!repo->index)
>>                 repo->index = xcalloc(1, sizeof(*repo->index));
>>
>> +       /* Complete the double-reference */
>> +       if (!repo->index->repo)
>> +               repo->index->repo = repo;
>> +
>>         return read_index_from(repo->index, repo->index_file, repo->gitdir);
>>  }
>>
>> --
>> gitgitgadget
> 
> Since we have repo->index and we have index->repo, which are intended
> to be circular...what if they aren't?  Do we want or need to add
> assertions anywhere that repo == repo->index->repo or that index ==
> index->repo->index ?

Here, we are pairing them together and the loop is complete. I don't
view that as a permanent thing. This only initializes istate->repo
when we are parsing an index from a file, but not when we create one
in memory.

I imagine it will be likely in some cases to have multiple index_state
instances for a single repository. However, having the pointer "this
index belongs to this repository" seems helpful (to me).

> My initial implementations of --remerge-diff[1] played around with
> creating a second repo, with a different primary object store but
> everything else the same.  The index for the two repository objects
> was thus the same, and thus clearly would have violated this assumed
> invariant for one of the two repos.  I discarded that initial
> implementation (which I didn't quite have working) because I
> discovered tmp-objdir.h and was able to add some
> tmp_objdir_make_primary() and tmp_objdir_remove_as_primary() functions
> that merely altered the existing repo's primary object store, but I'm
> curious if there might be other cases of folks doing stuff that might
> have weird failures with this new invariant.

This is an interesting concept, and definitely violates my expectations
that an index belongs to only one repository. I'd need to know more
about why this was a good design decision before being convinced that
the relationship should not be many-to-one (index-to-repo).

> It's entirely possible that --remerge-diff was just so different, and
> I was so unfamiliar with repo objects (and still kind of am) that I
> was just doing weird stuff no one has done before, so perhaps no
> additional checks are needed -- I'm just throwing my gut question out
> there as food for thought.
> 
> [1] I have not yet submitted `--remerge-diff` to the list; you haven't
> missed anything.  I'm waiting for merge-ort to be submitted, reviewed,
> and merged first.  It's the remerge-diff branch in my fork on GitHub
> if anyone is curious, though.
 
I'm interested in what others might say about this idea. I'd be able
to do most of what I want to do without this patch, but it just gets
a lot messier. (istate->repo is used in the very next patch in a way
that would be less clean without it.)

Thanks,
-Stolee
Elijah Newren Jan. 20, 2021, 7:50 p.m. UTC | #3
On Wed, Jan 20, 2021 at 11:16 AM Derrick Stolee <stolee@gmail.com> wrote:
>
> On 1/20/2021 12:46 PM, Elijah Newren wrote:
> > On Wed, Jan 20, 2021 at 8:54 AM Derrick Stolee via GitGitGadget
> > <gitgitgadget@gmail.com> wrote:
> >>
> >> From: Derrick Stolee <dstolee@microsoft.com>
> >>
> >> It will be helpful to add behavior to index opertations that might
> >
> > s/opertations/operations/
>
> Thanks.
>
> >> trigger an object lookup. Since each index belongs to a specific
> >> repository, add a 'repo' pointer to struct index_state that allows
> >> access to this repository.
> >>
> >> This will prevent future changes from needing to pass an additional
> >> 'struct repository *repo' parameter and instead rely only on the 'struct
> >> index_state *istate' parameter.
> >>
> >> Signed-off-by: Derrick Stolee <dstolee@microsoft.com>
> >> ---
> >>  cache.h      | 1 +
> >>  repository.c | 4 ++++
> >>  2 files changed, 5 insertions(+)
> >>
> >> diff --git a/cache.h b/cache.h
> >> index 71097657489..f9c7a603841 100644
> >> --- a/cache.h
> >> +++ b/cache.h
> >> @@ -328,6 +328,7 @@ struct index_state {
> >>         struct ewah_bitmap *fsmonitor_dirty;
> >>         struct mem_pool *ce_mem_pool;
> >>         struct progress *progress;
> >> +       struct repository *repo;
> >>  };
> >>
> >>  /* Name hashing */
> >> diff --git a/repository.c b/repository.c
> >> index a4174ddb062..67a4c1da2d9 100644
> >> --- a/repository.c
> >> +++ b/repository.c
> >> @@ -264,6 +264,10 @@ int repo_read_index(struct repository *repo)
> >>         if (!repo->index)
> >>                 repo->index = xcalloc(1, sizeof(*repo->index));
> >>
> >> +       /* Complete the double-reference */
> >> +       if (!repo->index->repo)
> >> +               repo->index->repo = repo;
> >> +
> >>         return read_index_from(repo->index, repo->index_file, repo->gitdir);
> >>  }
> >>
> >> --
> >> gitgitgadget
> >
> > Since we have repo->index and we have index->repo, which are intended
> > to be circular...what if they aren't?  Do we want or need to add
> > assertions anywhere that repo == repo->index->repo or that index ==
> > index->repo->index ?
>
> Here, we are pairing them together and the loop is complete. I don't
> view that as a permanent thing. This only initializes istate->repo
> when we are parsing an index from a file, but not when we create one
> in memory.
>
> I imagine it will be likely in some cases to have multiple index_state
> instances for a single repository. However, having the pointer "this
> index belongs to this repository" seems helpful (to me).
>
> > My initial implementations of --remerge-diff[1] played around with
> > creating a second repo, with a different primary object store but
> > everything else the same.  The index for the two repository objects
> > was thus the same, and thus clearly would have violated this assumed
> > invariant for one of the two repos.  I discarded that initial
> > implementation (which I didn't quite have working) because I
> > discovered tmp-objdir.h and was able to add some
> > tmp_objdir_make_primary() and tmp_objdir_remove_as_primary() functions
> > that merely altered the existing repo's primary object store, but I'm
> > curious if there might be other cases of folks doing stuff that might
> > have weird failures with this new invariant.
>
> This is an interesting concept, and definitely violates my expectations
> that an index belongs to only one repository. I'd need to know more
> about why this was a good design decision before being convinced that
> the relationship should not be many-to-one (index-to-repo).

I'm not sure what I did was a good design decision; I was kind of
exploring and trying to figure things out.  In retrospect, I think it
was probably a bad idea.  But we have various guard rails in the form
of BUG() calls and such when basic assumptions are violated, and here
it seems that you are now making a new basic assumption that an index
belongs to only one repository.  (Even if all current callers happen
to satisfy that assumption, it's not clear to me that git previously
cared if this condition were satisfied or not).  Hence my question
about safety checks.

> > It's entirely possible that --remerge-diff was just so different, and
> > I was so unfamiliar with repo objects (and still kind of am) that I
> > was just doing weird stuff no one has done before, so perhaps no
> > additional checks are needed -- I'm just throwing my gut question out
> > there as food for thought.
> >
> > [1] I have not yet submitted `--remerge-diff` to the list; you haven't
> > missed anything.  I'm waiting for merge-ort to be submitted, reviewed,
> > and merged first.  It's the remerge-diff branch in my fork on GitHub
> > if anyone is curious, though.
>
> I'm interested in what others might say about this idea. I'd be able
> to do most of what I want to do without this patch, but it just gets
> a lot messier. (istate->repo is used in the very next patch in a way
> that would be less clean without it.)

I'm less concerned with your patch as-is (I think your assumption
seems reasonable and I'm fine with labelling my former unsubmitted
patches as erroneous), and more wondering whether others in the future
will accidentally violate assumptions your patch starts encoding...and
whether we can or should do anything about it.  If there's a simple
place we can add a check for such an error, then it probably makes
sense to add one.  If there isn't...then at least we considered it?
diff mbox series

Patch

diff --git a/cache.h b/cache.h
index 71097657489..f9c7a603841 100644
--- a/cache.h
+++ b/cache.h
@@ -328,6 +328,7 @@  struct index_state {
 	struct ewah_bitmap *fsmonitor_dirty;
 	struct mem_pool *ce_mem_pool;
 	struct progress *progress;
+	struct repository *repo;
 };
 
 /* Name hashing */
diff --git a/repository.c b/repository.c
index a4174ddb062..67a4c1da2d9 100644
--- a/repository.c
+++ b/repository.c
@@ -264,6 +264,10 @@  int repo_read_index(struct repository *repo)
 	if (!repo->index)
 		repo->index = xcalloc(1, sizeof(*repo->index));
 
+	/* Complete the double-reference */
+	if (!repo->index->repo)
+		repo->index->repo = repo;
+
 	return read_index_from(repo->index, repo->index_file, repo->gitdir);
 }