rebase --abort: cleanup refs/rewritten
diff mbox series

Message ID 20190426103212.8097-1-phillip.wood123@gmail.com
State New
Headers show
Series
  • rebase --abort: cleanup refs/rewritten
Related show

Commit Message

Phillip Wood April 26, 2019, 10:32 a.m. UTC
From: Phillip Wood <phillip.wood@dunelm.org.uk>

When `rebase -r` finishes it removes any refs under refs/rewritten
that it has created. However if the rebase is aborted these refs are
not removed. This can cause problems for future rebases. For example I
recently wanted to merge a updated version of a topic branch into an
integration branch so ran `rebase -ir` and removed the picks and label
for the topic branch from the todo list so that
    merge -C <old-merge> topic
would pick up the new version of topic. Unfortunately
refs/rewritten/topic already existed from a previous rebase that had
been aborted so the rebase just used the old topic, not the new one.

Signed-off-by: Phillip Wood <phillip.wood@dunelm.org.uk>
---

Notes:
    This is based on pw/rebase-i-internal, it would be nicer to base it on
    maint but there are function name clashes adding sequencer.h to rebase.c
    an maint. Those clashes are fixed in pw/rebase-i-internal

 builtin/rebase.c         | 13 ++++++++++---
 t/t3430-rebase-merges.sh |  8 ++++++++
 2 files changed, 18 insertions(+), 3 deletions(-)

Comments

Johannes Schindelin April 29, 2019, 4:07 p.m. UTC | #1
Hi Phillip,

On Fri, 26 Apr 2019, Phillip Wood wrote:

> From: Phillip Wood <phillip.wood@dunelm.org.uk>
>
> When `rebase -r` finishes it removes any refs under refs/rewritten
> that it has created. However if the rebase is aborted these refs are
> not removed. This can cause problems for future rebases. For example I
> recently wanted to merge a updated version of a topic branch into an
> integration branch so ran `rebase -ir` and removed the picks and label
> for the topic branch from the todo list so that
>     merge -C <old-merge> topic
> would pick up the new version of topic. Unfortunately
> refs/rewritten/topic already existed from a previous rebase that had
> been aborted so the rebase just used the old topic, not the new one.
>
> Signed-off-by: Phillip Wood <phillip.wood@dunelm.org.uk>
> ---

Makes a ton of sense, and I feel a bit embarrassed that I forgot about
that item on my TODO list. The patch looks obviously correct!

Thanks,
Dscho
Phillip Wood April 30, 2019, 8:54 a.m. UTC | #2
Hi Dscho

On 29/04/2019 17:07, Johannes Schindelin wrote:
> Hi Phillip,
> 
> On Fri, 26 Apr 2019, Phillip Wood wrote:
> 
>> From: Phillip Wood <phillip.wood@dunelm.org.uk>
>>
>> When `rebase -r` finishes it removes any refs under refs/rewritten
>> that it has created. However if the rebase is aborted these refs are
>> not removed. This can cause problems for future rebases. For example I
>> recently wanted to merge a updated version of a topic branch into an
>> integration branch so ran `rebase -ir` and removed the picks and label
>> for the topic branch from the todo list so that
>>      merge -C <old-merge> topic
>> would pick up the new version of topic. Unfortunately
>> refs/rewritten/topic already existed from a previous rebase that had
>> been aborted so the rebase just used the old topic, not the new one.
>>
>> Signed-off-by: Phillip Wood <phillip.wood@dunelm.org.uk>
>> ---
> 
> Makes a ton of sense, and I feel a bit embarrassed that I forgot about
> that item on my TODO list. The patch looks obviously correct!

Thanks, after I sent it I realized that --quit should probably clear 
refs/rewritten as well, so I'll re-roll with that added. (One could 
argue that a user might want them after quitting the rebase but there is 
no way to clean them up safely once we've deleted the state files and I 
suspect most users would be suprised if they were left laying around)

Best Wishes

Phillip

> 
> Thanks,
> Dscho
>
Johannes Schindelin April 30, 2019, 10:49 p.m. UTC | #3
Hi Phillip,

On Tue, 30 Apr 2019, Phillip Wood wrote:

> On 29/04/2019 17:07, Johannes Schindelin wrote:
> >
> > On Fri, 26 Apr 2019, Phillip Wood wrote:
> >
> > > From: Phillip Wood <phillip.wood@dunelm.org.uk>
> > >
> > > When `rebase -r` finishes it removes any refs under refs/rewritten
> > > that it has created. However if the rebase is aborted these refs are
> > > not removed. This can cause problems for future rebases. For example I
> > > recently wanted to merge a updated version of a topic branch into an
> > > integration branch so ran `rebase -ir` and removed the picks and label
> > > for the topic branch from the todo list so that
> > >      merge -C <old-merge> topic
> > > would pick up the new version of topic. Unfortunately
> > > refs/rewritten/topic already existed from a previous rebase that had
> > > been aborted so the rebase just used the old topic, not the new one.
> > >
> > > Signed-off-by: Phillip Wood <phillip.wood@dunelm.org.uk>
> > > ---
> >
> > Makes a ton of sense, and I feel a bit embarrassed that I forgot about
> > that item on my TODO list. The patch looks obviously correct!
>
> Thanks, after I sent it I realized that --quit should probably clear
> refs/rewritten as well, so I'll re-roll with that added. (One could argue that
> a user might want them after quitting the rebase but there is no way to clean
> them up safely once we've deleted the state files and I suspect most users
> would be suprised if they were left laying around)

I am not so sure. `--quit` is essentially all about "leave the state
as-is, but still abort the rebase".

So if I were you, I would *not* remove the `refs/rewritten/` refs in the
`--quit` case.

Ciao,
Dscho
Phillip Wood May 1, 2019, 3:36 p.m. UTC | #4
Hi Dscho

On 30/04/2019 23:49, Johannes Schindelin wrote:
> Hi Phillip,
> 
> On Tue, 30 Apr 2019, Phillip Wood wrote:
> 
>> On 29/04/2019 17:07, Johannes Schindelin wrote:
>>>
>>> On Fri, 26 Apr 2019, Phillip Wood wrote:
>>>
>>>> From: Phillip Wood <phillip.wood@dunelm.org.uk>
>>>>
>>>> When `rebase -r` finishes it removes any refs under refs/rewritten
>>>> that it has created. However if the rebase is aborted these refs are
>>>> not removed. This can cause problems for future rebases. For example I
>>>> recently wanted to merge a updated version of a topic branch into an
>>>> integration branch so ran `rebase -ir` and removed the picks and label
>>>> for the topic branch from the todo list so that
>>>>       merge -C <old-merge> topic
>>>> would pick up the new version of topic. Unfortunately
>>>> refs/rewritten/topic already existed from a previous rebase that had
>>>> been aborted so the rebase just used the old topic, not the new one.
>>>>
>>>> Signed-off-by: Phillip Wood <phillip.wood@dunelm.org.uk>
>>>> ---
>>>
>>> Makes a ton of sense, and I feel a bit embarrassed that I forgot about
>>> that item on my TODO list. The patch looks obviously correct!
>>
>> Thanks, after I sent it I realized that --quit should probably clear
>> refs/rewritten as well, so I'll re-roll with that added. (One could argue that
>> a user might want them after quitting the rebase but there is no way to clean
>> them up safely once we've deleted the state files and I suspect most users
>> would be suprised if they were left laying around)
> 
> I am not so sure. `--quit` is essentially all about "leave the state
> as-is, but still abort the rebase".

I think it depends on what you mean by "state" `--quit` is about 
removing state specific to rebases while preserving HEAD, the index and 
worktree. When "rebase --quit" was introduced in 9512177b68 ("rebase: 
add --quit to cleanup rebase, leave everything else untouched", 
2016-11-12) the start of the log message reads

     rebase: add --quit to cleanup rebase, leave everything else untouched

     There are occasions when you decide to abort an in-progress rebase and
     move on to do something else but you forget to do "git rebase --abort"
     first. Or the rebase has been in progress for so long you forgot about
     it. By the time you realize that (e.g. by starting another rebase)
     it's already too late to retrace your steps. The solution is normally

         rm -r .git/<some rebase dir>

     and continue with your life.


So `--quit` is used when the user has forgotten to run "rebase --abort". 
They have moved onto something else and want to remove the rebase state 
without changing the current HEAD, index or worktree, they are not 
looking to use the refs under refs/rewritten. I think the refs rebase 
creates under refs/rewritten is an implementation detail of "rebase -r" 
and should be treated like files under .git/rebase-merge. I'm worried 
that if we leave them lying around after --quit they will cause trouble 
for future rebases in the same way they have after "rebase --abort"

Best Wishes

Phillip

> So if I were you, I would *not* remove the `refs/rewritten/` refs in the
> `--quit` case.
> 
> Ciao,
> Dscho
>
Johannes Schindelin May 3, 2019, 9:21 a.m. UTC | #5
Hi Phillip,

On Wed, 1 May 2019, Phillip Wood wrote:

> On 30/04/2019 23:49, Johannes Schindelin wrote:
> >
> > On Tue, 30 Apr 2019, Phillip Wood wrote:
> >
> > > On 29/04/2019 17:07, Johannes Schindelin wrote:
> > > >
> > > > On Fri, 26 Apr 2019, Phillip Wood wrote:
> > > >
> > > > > From: Phillip Wood <phillip.wood@dunelm.org.uk>
> > > > >
> > > > > When `rebase -r` finishes it removes any refs under refs/rewritten
> > > > > that it has created. However if the rebase is aborted these refs are
> > > > > not removed. This can cause problems for future rebases. For example I
> > > > > recently wanted to merge a updated version of a topic branch into an
> > > > > integration branch so ran `rebase -ir` and removed the picks and label
> > > > > for the topic branch from the todo list so that
> > > > >       merge -C <old-merge> topic
> > > > > would pick up the new version of topic. Unfortunately
> > > > > refs/rewritten/topic already existed from a previous rebase that had
> > > > > been aborted so the rebase just used the old topic, not the new one.
> > > > >
> > > > > Signed-off-by: Phillip Wood <phillip.wood@dunelm.org.uk>
> > > > > ---
> > > >
> > > > Makes a ton of sense, and I feel a bit embarrassed that I forgot about
> > > > that item on my TODO list. The patch looks obviously correct!
> > >
> > > Thanks, after I sent it I realized that --quit should probably clear
> > > refs/rewritten as well, so I'll re-roll with that added. (One could argue
> > > that
> > > a user might want them after quitting the rebase but there is no way to
> > > clean
> > > them up safely once we've deleted the state files and I suspect most users
> > > would be suprised if they were left laying around)
> >
> > I am not so sure. `--quit` is essentially all about "leave the state
> > as-is, but still abort the rebase".
>
> I think it depends on what you mean by "state" `--quit` is about removing
> state specific to rebases while preserving HEAD, the index and worktree.

I guess the fault is mine for bleeding out internal rebase state into the
refs namespace.

While I cannot really imagine any harm from this patch in practice, it is
slightly worrisome that deleting refs also deletes their reflogs, which
makes it an unrecoverable problem *iff* any user runs into trouble with
this.

Ciao,
Dscho
Phillip Wood May 3, 2019, 10:06 a.m. UTC | #6
Hi Dscho

On 03/05/2019 10:21, Johannes Schindelin wrote:
> Hi Phillip,
> 
> On Wed, 1 May 2019, Phillip Wood wrote:
> 
>> On 30/04/2019 23:49, Johannes Schindelin wrote:
>>>
>>> On Tue, 30 Apr 2019, Phillip Wood wrote:
>>>
>>>> On 29/04/2019 17:07, Johannes Schindelin wrote:
>>>>>
>>>>> On Fri, 26 Apr 2019, Phillip Wood wrote:
>>>>>
>>>>>> From: Phillip Wood <phillip.wood@dunelm.org.uk>
>>>>>>
>>>>>> When `rebase -r` finishes it removes any refs under refs/rewritten
>>>>>> that it has created. However if the rebase is aborted these refs are
>>>>>> not removed. This can cause problems for future rebases. For example I
>>>>>> recently wanted to merge a updated version of a topic branch into an
>>>>>> integration branch so ran `rebase -ir` and removed the picks and label
>>>>>> for the topic branch from the todo list so that
>>>>>>        merge -C <old-merge> topic
>>>>>> would pick up the new version of topic. Unfortunately
>>>>>> refs/rewritten/topic already existed from a previous rebase that had
>>>>>> been aborted so the rebase just used the old topic, not the new one.
>>>>>>
>>>>>> Signed-off-by: Phillip Wood <phillip.wood@dunelm.org.uk>
>>>>>> ---
>>>>>
>>>>> Makes a ton of sense, and I feel a bit embarrassed that I forgot about
>>>>> that item on my TODO list. The patch looks obviously correct!
>>>>
>>>> Thanks, after I sent it I realized that --quit should probably clear
>>>> refs/rewritten as well, so I'll re-roll with that added. (One could argue
>>>> that
>>>> a user might want them after quitting the rebase but there is no way to
>>>> clean
>>>> them up safely once we've deleted the state files and I suspect most users
>>>> would be suprised if they were left laying around)
>>>
>>> I am not so sure. `--quit` is essentially all about "leave the state
>>> as-is, but still abort the rebase".
>>
>> I think it depends on what you mean by "state" `--quit` is about removing
>> state specific to rebases while preserving HEAD, the index and worktree.
> 
> I guess the fault is mine for bleeding out internal rebase state into the
> refs namespace.

I wouldn't feel bad about that, I guess it would be possible to get gc 
to read a list of objects not to collect from a file in 
.git/rebase-merge but creating a refs for labels seems like a sensible 
way to stop them from being collect by gc.

> While I cannot really imagine any harm from this patch in practice, it is
> slightly worrisome that deleting refs also deletes their reflogs,

Yes it's a shame there's no way to get a ref back once it's been deleted 
(though I'm not sure how long we'd want to keep any reflog of a deleted 
ref before gc'ing the objects). In any case refs/rewritten only has a 
reflog if the user has explicitly enabled it.

> which
> makes it an unrecoverable problem *iff* any user runs into trouble with
> this.

I guess "rebase --quit" could print a warning listing the refs that are 
being deleted so the user can cut and paste if they need to. I'm not 
sure how likely they are to need that though.

Best Wishes

Phillip

> Ciao,
> Dscho
>
SZEDER Gábor May 7, 2019, 3:15 p.m. UTC | #7
On Fri, Apr 26, 2019 at 11:32:12AM +0100, Phillip Wood wrote:
> From: Phillip Wood <phillip.wood@dunelm.org.uk>
> 
> When `rebase -r` finishes it removes any refs under refs/rewritten
> that it has created. However if the rebase is aborted these refs are
> not removed. This can cause problems for future rebases. For example I
> recently wanted to merge a updated version of a topic branch into an
> integration branch so ran `rebase -ir` and removed the picks and label
> for the topic branch from the todo list so that
>     merge -C <old-merge> topic
> would pick up the new version of topic. Unfortunately
> refs/rewritten/topic already existed from a previous rebase that had
> been aborted so the rebase just used the old topic, not the new one.
> 
> Signed-off-by: Phillip Wood <phillip.wood@dunelm.org.uk>
> ---
> 
> Notes:
>     This is based on pw/rebase-i-internal, it would be nicer to base it on
>     maint but there are function name clashes adding sequencer.h to rebase.c
>     an maint. Those clashes are fixed in pw/rebase-i-internal
> 
>  builtin/rebase.c         | 13 ++++++++++---
>  t/t3430-rebase-merges.sh |  8 ++++++++
>  2 files changed, 18 insertions(+), 3 deletions(-)
> 
> diff --git a/builtin/rebase.c b/builtin/rebase.c
> index 82bd50a1b4..e2e49c8239 100644
> --- a/builtin/rebase.c
> +++ b/builtin/rebase.c
> @@ -761,9 +761,16 @@ static int finish_rebase(struct rebase_options *opts)
>  	 * user should see them.
>  	 */
>  	run_command_v_opt(argv_gc_auto, RUN_GIT_CMD);
> -	strbuf_addstr(&dir, opts->state_dir);
> -	remove_dir_recursively(&dir, 0);
> -	strbuf_release(&dir);
> +	if (opts->type == REBASE_INTERACTIVE) {
> +		struct replay_opts replay = REPLAY_OPTS_INIT;

This patch and the topic 'pw/rebase-abort-clean-rewritten' can't be
compiled on its own, because it starts using 'struct replay_opts'
here, which is defined in 'sequencer.h', but 'builtin/rebase.c'
doesn't include that header yet.  (Though 'pu' already builds fine,
because commit 0609b741a4 (rebase -i: combine rebase--interactive.c
with rebase.c, 2019-04-17) in the parallel topic
'pw/rebase-i-internal' adds the necessary #include.)

So, to keep future bisects from potentially tipping over the compiler
error, this patch should either #include "sequencer.h", or be applied
on top of 'pw/rebase-i-internal'.

> +
> +		replay.action = REPLAY_INTERACTIVE_REBASE;
> +		sequencer_remove_state(&replay);
> +	} else {
> +		strbuf_addstr(&dir, opts->state_dir);
> +		remove_dir_recursively(&dir, 0);
> +		strbuf_release(&dir);
> +	}
>  
>  	return 0;
>  }
> diff --git a/t/t3430-rebase-merges.sh b/t/t3430-rebase-merges.sh
> index 4c69255ee6..6ebebf7098 100755
> --- a/t/t3430-rebase-merges.sh
> +++ b/t/t3430-rebase-merges.sh
> @@ -224,6 +224,14 @@ test_expect_success 'refs/rewritten/* is worktree-local' '
>  	test_cmp_rev HEAD "$(cat wt/b)"
>  '
>  
> +test_expect_success '--abort cleans up refs/rewritten' '
> +	git checkout -b abort-cleans-refs-rewritten H &&
> +	GIT_SEQUENCE_EDITOR="echo break >>" git rebase -ir @^ &&
> +	git rev-parse --verify refs/rewritten/onto &&
> +	git rebase --abort &&
> +	test_must_fail git rev-parse --verify refs/rewritten/onto
> +'
> +
>  test_expect_success 'post-rewrite hook and fixups work for merges' '
>  	git checkout -b post-rewrite &&
>  	test_commit same1 &&
> -- 
> 2.21.0
>
Junio C Hamano May 7, 2019, 4:07 p.m. UTC | #8
SZEDER Gábor <szeder.dev@gmail.com> writes:

> This patch and the topic 'pw/rebase-abort-clean-rewritten' can't be
> compiled on its own, because it starts using 'struct replay_opts'
> here, which is defined in 'sequencer.h', but 'builtin/rebase.c'
> doesn't include that header yet.  (Though 'pu' already builds fine,
> because commit 0609b741a4 (rebase -i: combine rebase--interactive.c
> with rebase.c, 2019-04-17) in the parallel topic
> 'pw/rebase-i-internal' adds the necessary #include.)

Thanks; that's entirely my fault.  I needed to find a good fork
point and failed to do so.  FTR, when there are too many topics
I need to queue on a given day, I may not have time to compile
check individual topic branches before merging them to the
integration branches, testing the integration branches and pushing
them out.  That was what happened here.

> So, to keep future bisects from potentially tipping over the compiler
> error, this patch should either #include "sequencer.h", or be applied
> on top of 'pw/rebase-i-internal'.

I suspect that the latter was how the patch originally was
developed.
Phillip Wood May 7, 2019, 8:06 p.m. UTC | #9
On 07/05/2019 17:07, Junio C Hamano wrote:
> SZEDER Gábor <szeder.dev@gmail.com> writes:
> 
>> This patch and the topic 'pw/rebase-abort-clean-rewritten' can't be
>> compiled on its own, because it starts using 'struct replay_opts'
>> here, which is defined in 'sequencer.h', but 'builtin/rebase.c'
>> doesn't include that header yet.  (Though 'pu' already builds fine,
>> because commit 0609b741a4 (rebase -i: combine rebase--interactive.c
>> with rebase.c, 2019-04-17) in the parallel topic
>> 'pw/rebase-i-internal' adds the necessary #include.)
> 
> Thanks; that's entirely my fault.  I needed to find a good fork
> point and failed to do so.  FTR, when there are too many topics
> I need to queue on a given day, I may not have time to compile
> check individual topic branches before merging them to the
> integration branches, testing the integration branches and pushing
> them out.  That was what happened here.
> 
>> So, to keep future bisects from potentially tipping over the compiler
>> error, this patch should either #include "sequencer.h", or be applied
>> on top of 'pw/rebase-i-internal'.
> 
> I suspect that the latter was how the patch originally was
> developed.

Yes that's right, there is a note on the original patch [1] explaining 
why - you cannot just add '#include "sequencer.h"' as there are function 
name conflicts between a static function in builtin/rebase.c and a 
global function in sequencer.c which are fixed in pw/rebase-i-internal

Best Wishes

Phillip

[1] 
https://public-inbox.org/git/xmqqpnoujlj4.fsf@gitster-ct.c.googlers.com/T/#mb431b81731798388e12f3852747c560f8ce7c6ec

Patch
diff mbox series

diff --git a/builtin/rebase.c b/builtin/rebase.c
index 82bd50a1b4..e2e49c8239 100644
--- a/builtin/rebase.c
+++ b/builtin/rebase.c
@@ -761,9 +761,16 @@  static int finish_rebase(struct rebase_options *opts)
 	 * user should see them.
 	 */
 	run_command_v_opt(argv_gc_auto, RUN_GIT_CMD);
-	strbuf_addstr(&dir, opts->state_dir);
-	remove_dir_recursively(&dir, 0);
-	strbuf_release(&dir);
+	if (opts->type == REBASE_INTERACTIVE) {
+		struct replay_opts replay = REPLAY_OPTS_INIT;
+
+		replay.action = REPLAY_INTERACTIVE_REBASE;
+		sequencer_remove_state(&replay);
+	} else {
+		strbuf_addstr(&dir, opts->state_dir);
+		remove_dir_recursively(&dir, 0);
+		strbuf_release(&dir);
+	}
 
 	return 0;
 }
diff --git a/t/t3430-rebase-merges.sh b/t/t3430-rebase-merges.sh
index 4c69255ee6..6ebebf7098 100755
--- a/t/t3430-rebase-merges.sh
+++ b/t/t3430-rebase-merges.sh
@@ -224,6 +224,14 @@  test_expect_success 'refs/rewritten/* is worktree-local' '
 	test_cmp_rev HEAD "$(cat wt/b)"
 '
 
+test_expect_success '--abort cleans up refs/rewritten' '
+	git checkout -b abort-cleans-refs-rewritten H &&
+	GIT_SEQUENCE_EDITOR="echo break >>" git rebase -ir @^ &&
+	git rev-parse --verify refs/rewritten/onto &&
+	git rebase --abort &&
+	test_must_fail git rev-parse --verify refs/rewritten/onto
+'
+
 test_expect_success 'post-rewrite hook and fixups work for merges' '
 	git checkout -b post-rewrite &&
 	test_commit same1 &&