[RFC,2/5] t3427: accelerate this test by using fast-export and fast-import
diff mbox series

Message ID 20190826235226.15386-3-newren@gmail.com
State New
Headers show
Series
  • Remove git-filter-branch from git.git; host it elsewhere
Related show

Commit Message

Elijah Newren Aug. 26, 2019, 11:52 p.m. UTC
fast-export and fast-import can easily handle the simple rewrite that
was being done by filter-branch, and should be significantly faster on
systems with a slow fork.  Timings from before and after on two laptops
that I have access to (measured via `time ./t3427-rebase-subtree.sh`,
i.e. including everything in this test -- not just the filter-branch or
fast-export/fast-import pair):

   Linux:  4.305s -> 3.684s (~17% speedup)
   Mac:   10.128s -> 7.038s (~30% speedup)

Signed-off-by: Elijah Newren <newren@gmail.com>
---
 t/t3427-rebase-subtree.sh | 32 ++++++++++++++++++++++++--------
 1 file changed, 24 insertions(+), 8 deletions(-)

Comments

Derrick Stolee Aug. 27, 2019, 1:25 a.m. UTC | #1
On 8/26/2019 7:52 PM, Elijah Newren wrote:
> fast-export and fast-import can easily handle the simple rewrite that
> was being done by filter-branch, and should be significantly faster on
> systems with a slow fork.  Timings from before and after on two laptops
> that I have access to (measured via `time ./t3427-rebase-subtree.sh`,
> i.e. including everything in this test -- not just the filter-branch or
> fast-export/fast-import pair):
> 
>    Linux:  4.305s -> 3.684s (~17% speedup)
>    Mac:   10.128s -> 7.038s (~30% speedup)

Again, impressive speedup!

> Signed-off-by: Elijah Newren <newren@gmail.com>
> ---
>  t/t3427-rebase-subtree.sh | 32 ++++++++++++++++++++++++--------
>  1 file changed, 24 insertions(+), 8 deletions(-)
> 
> diff --git a/t/t3427-rebase-subtree.sh b/t/t3427-rebase-subtree.sh
> index d8640522a0..d05fcce5dc 100755
> --- a/t/t3427-rebase-subtree.sh
> +++ b/t/t3427-rebase-subtree.sh
> @@ -42,7 +42,9 @@ test_expect_failure REBASE_P \
>  	'Rebase -Xsubtree --preserve-merges --onto commit 4' '
>  	reset_rebase &&
>  	git checkout -b rebase-preserve-merges-4 master &&
> -	git filter-branch --prune-empty -f --subdirectory-filter files_subtree &&
> +	git fast-export --no-data HEAD -- files_subtree/ \
> +		| sed -e "s%\([0-9a-f]\{40\} \)files_subtree/%\1%" \
> +		| git fast-import --force --quiet &&
>  	git commit -m "Empty commit" --allow-empty &&

There's a lot of repetition in these changes. Is this a good time
to introduce a helper method? This trio of commands happens to be
difficult to parse, so I'd rather do it just once.

>  	git rebase -Xsubtree=files_subtree --preserve-merges --onto files-master master &&
>  	verbose test "$(commit_message HEAD~)" = "files_subtree/master4"
> @@ -53,7 +55,9 @@ test_expect_failure REBASE_P \
>  	'Rebase -Xsubtree --preserve-merges --onto commit 5' '
>  	reset_rebase &&
>  	git checkout -b rebase-preserve-merges-5 master &&
> -	git filter-branch --prune-empty -f --subdirectory-filter files_subtree &&
> +	git fast-export --no-data HEAD -- files_subtree/ \
> +		| sed -e "s%\([0-9a-f]\{40\} \)files_subtree/%\1%" \
> +		| git fast-import --force --quiet &&
>  	git commit -m "Empty commit" --allow-empty &&
>  	git rebase -Xsubtree=files_subtree --preserve-merges --onto files-master master &&
>  	verbose test "$(commit_message HEAD)" = "files_subtree/master5"
> @@ -64,7 +68,9 @@ test_expect_failure REBASE_P \
>  	'Rebase -Xsubtree --keep-empty --preserve-merges --onto commit 4' '
>  	reset_rebase &&
>  	git checkout -b rebase-keep-empty-4 master &&
> -	git filter-branch --prune-empty -f --subdirectory-filter files_subtree &&
> +	git fast-export --no-data HEAD -- files_subtree/ \
> +		| sed -e "s%\([0-9a-f]\{40\} \)files_subtree/%\1%" \
> +		| git fast-import --force --quiet &&
>  	git commit -m "Empty commit" --allow-empty &&
>  	git rebase -Xsubtree=files_subtree --keep-empty --preserve-merges --onto files-master master &&
>  	verbose test "$(commit_message HEAD~2)" = "files_subtree/master4"
> @@ -75,7 +81,9 @@ test_expect_failure REBASE_P \
>  	'Rebase -Xsubtree --keep-empty --preserve-merges --onto commit 5' '
>  	reset_rebase &&
>  	git checkout -b rebase-keep-empty-5 master &&
> -	git filter-branch --prune-empty -f --subdirectory-filter files_subtree &&
> +	git fast-export --no-data HEAD -- files_subtree/ \
> +		| sed -e "s%\([0-9a-f]\{40\} \)files_subtree/%\1%" \
> +		| git fast-import --force --quiet &&
>  	git commit -m "Empty commit" --allow-empty &&
>  	git rebase -Xsubtree=files_subtree --keep-empty --preserve-merges --onto files-master master &&
>  	verbose test "$(commit_message HEAD~)" = "files_subtree/master5"
> @@ -86,7 +94,9 @@ test_expect_failure REBASE_P \
>  	'Rebase -Xsubtree --keep-empty --preserve-merges --onto empty commit' '
>  	reset_rebase &&
>  	git checkout -b rebase-keep-empty-empty master &&
> -	git filter-branch --prune-empty -f --subdirectory-filter files_subtree &&
> +	git fast-export --no-data HEAD -- files_subtree/ \
> +		| sed -e "s%\([0-9a-f]\{40\} \)files_subtree/%\1%" \
> +		| git fast-import --force --quiet &&
>  	git commit -m "Empty commit" --allow-empty &&
>  	git rebase -Xsubtree=files_subtree --keep-empty --preserve-merges --onto files-master master &&
>  	verbose test "$(commit_message HEAD)" = "Empty commit"
> @@ -96,7 +106,9 @@ test_expect_failure REBASE_P \
>  test_expect_failure 'Rebase -Xsubtree --onto commit 4' '
>  	reset_rebase &&
>  	git checkout -b rebase-onto-4 master &&
> -	git filter-branch --prune-empty -f --subdirectory-filter files_subtree &&
> +	git fast-export --no-data HEAD -- files_subtree/ \
> +		| sed -e "s%\([0-9a-f]\{40\} \)files_subtree/%\1%" \
> +		| git fast-import --force --quiet &&
>  	git commit -m "Empty commit" --allow-empty &&
>  	git rebase -Xsubtree=files_subtree --onto files-master master &&
>  	verbose test "$(commit_message HEAD~2)" = "files_subtree/master4"
> @@ -106,7 +118,9 @@ test_expect_failure 'Rebase -Xsubtree --onto commit 4' '
>  test_expect_failure 'Rebase -Xsubtree --onto commit 5' '
>  	reset_rebase &&
>  	git checkout -b rebase-onto-5 master &&
> -	git filter-branch --prune-empty -f --subdirectory-filter files_subtree &&
> +	git fast-export --no-data HEAD -- files_subtree/ \
> +		| sed -e "s%\([0-9a-f]\{40\} \)files_subtree/%\1%" \
> +		| git fast-import --force --quiet &&
>  	git commit -m "Empty commit" --allow-empty &&
>  	git rebase -Xsubtree=files_subtree --onto files-master master &&
>  	verbose test "$(commit_message HEAD~)" = "files_subtree/master5"
> @@ -115,7 +129,9 @@ test_expect_failure 'Rebase -Xsubtree --onto commit 5' '
>  test_expect_failure 'Rebase -Xsubtree --onto empty commit' '
>  	reset_rebase &&
>  	git checkout -b rebase-onto-empty master &&
> -	git filter-branch --prune-empty -f --subdirectory-filter files_subtree &&
> +	git fast-export --no-data HEAD -- files_subtree/ \
> +		| sed -e "s%\([0-9a-f]\{40\} \)files_subtree/%\1%" \
> +		| git fast-import --force --quiet &&
>  	git commit -m "Empty commit" --allow-empty &&
>  	git rebase -Xsubtree=files_subtree --onto files-master master &&
>  	verbose test "$(commit_message HEAD)" = "Empty commit"
>

Patch
diff mbox series

diff --git a/t/t3427-rebase-subtree.sh b/t/t3427-rebase-subtree.sh
index d8640522a0..d05fcce5dc 100755
--- a/t/t3427-rebase-subtree.sh
+++ b/t/t3427-rebase-subtree.sh
@@ -42,7 +42,9 @@  test_expect_failure REBASE_P \
 	'Rebase -Xsubtree --preserve-merges --onto commit 4' '
 	reset_rebase &&
 	git checkout -b rebase-preserve-merges-4 master &&
-	git filter-branch --prune-empty -f --subdirectory-filter files_subtree &&
+	git fast-export --no-data HEAD -- files_subtree/ \
+		| sed -e "s%\([0-9a-f]\{40\} \)files_subtree/%\1%" \
+		| git fast-import --force --quiet &&
 	git commit -m "Empty commit" --allow-empty &&
 	git rebase -Xsubtree=files_subtree --preserve-merges --onto files-master master &&
 	verbose test "$(commit_message HEAD~)" = "files_subtree/master4"
@@ -53,7 +55,9 @@  test_expect_failure REBASE_P \
 	'Rebase -Xsubtree --preserve-merges --onto commit 5' '
 	reset_rebase &&
 	git checkout -b rebase-preserve-merges-5 master &&
-	git filter-branch --prune-empty -f --subdirectory-filter files_subtree &&
+	git fast-export --no-data HEAD -- files_subtree/ \
+		| sed -e "s%\([0-9a-f]\{40\} \)files_subtree/%\1%" \
+		| git fast-import --force --quiet &&
 	git commit -m "Empty commit" --allow-empty &&
 	git rebase -Xsubtree=files_subtree --preserve-merges --onto files-master master &&
 	verbose test "$(commit_message HEAD)" = "files_subtree/master5"
@@ -64,7 +68,9 @@  test_expect_failure REBASE_P \
 	'Rebase -Xsubtree --keep-empty --preserve-merges --onto commit 4' '
 	reset_rebase &&
 	git checkout -b rebase-keep-empty-4 master &&
-	git filter-branch --prune-empty -f --subdirectory-filter files_subtree &&
+	git fast-export --no-data HEAD -- files_subtree/ \
+		| sed -e "s%\([0-9a-f]\{40\} \)files_subtree/%\1%" \
+		| git fast-import --force --quiet &&
 	git commit -m "Empty commit" --allow-empty &&
 	git rebase -Xsubtree=files_subtree --keep-empty --preserve-merges --onto files-master master &&
 	verbose test "$(commit_message HEAD~2)" = "files_subtree/master4"
@@ -75,7 +81,9 @@  test_expect_failure REBASE_P \
 	'Rebase -Xsubtree --keep-empty --preserve-merges --onto commit 5' '
 	reset_rebase &&
 	git checkout -b rebase-keep-empty-5 master &&
-	git filter-branch --prune-empty -f --subdirectory-filter files_subtree &&
+	git fast-export --no-data HEAD -- files_subtree/ \
+		| sed -e "s%\([0-9a-f]\{40\} \)files_subtree/%\1%" \
+		| git fast-import --force --quiet &&
 	git commit -m "Empty commit" --allow-empty &&
 	git rebase -Xsubtree=files_subtree --keep-empty --preserve-merges --onto files-master master &&
 	verbose test "$(commit_message HEAD~)" = "files_subtree/master5"
@@ -86,7 +94,9 @@  test_expect_failure REBASE_P \
 	'Rebase -Xsubtree --keep-empty --preserve-merges --onto empty commit' '
 	reset_rebase &&
 	git checkout -b rebase-keep-empty-empty master &&
-	git filter-branch --prune-empty -f --subdirectory-filter files_subtree &&
+	git fast-export --no-data HEAD -- files_subtree/ \
+		| sed -e "s%\([0-9a-f]\{40\} \)files_subtree/%\1%" \
+		| git fast-import --force --quiet &&
 	git commit -m "Empty commit" --allow-empty &&
 	git rebase -Xsubtree=files_subtree --keep-empty --preserve-merges --onto files-master master &&
 	verbose test "$(commit_message HEAD)" = "Empty commit"
@@ -96,7 +106,9 @@  test_expect_failure REBASE_P \
 test_expect_failure 'Rebase -Xsubtree --onto commit 4' '
 	reset_rebase &&
 	git checkout -b rebase-onto-4 master &&
-	git filter-branch --prune-empty -f --subdirectory-filter files_subtree &&
+	git fast-export --no-data HEAD -- files_subtree/ \
+		| sed -e "s%\([0-9a-f]\{40\} \)files_subtree/%\1%" \
+		| git fast-import --force --quiet &&
 	git commit -m "Empty commit" --allow-empty &&
 	git rebase -Xsubtree=files_subtree --onto files-master master &&
 	verbose test "$(commit_message HEAD~2)" = "files_subtree/master4"
@@ -106,7 +118,9 @@  test_expect_failure 'Rebase -Xsubtree --onto commit 4' '
 test_expect_failure 'Rebase -Xsubtree --onto commit 5' '
 	reset_rebase &&
 	git checkout -b rebase-onto-5 master &&
-	git filter-branch --prune-empty -f --subdirectory-filter files_subtree &&
+	git fast-export --no-data HEAD -- files_subtree/ \
+		| sed -e "s%\([0-9a-f]\{40\} \)files_subtree/%\1%" \
+		| git fast-import --force --quiet &&
 	git commit -m "Empty commit" --allow-empty &&
 	git rebase -Xsubtree=files_subtree --onto files-master master &&
 	verbose test "$(commit_message HEAD~)" = "files_subtree/master5"
@@ -115,7 +129,9 @@  test_expect_failure 'Rebase -Xsubtree --onto commit 5' '
 test_expect_failure 'Rebase -Xsubtree --onto empty commit' '
 	reset_rebase &&
 	git checkout -b rebase-onto-empty master &&
-	git filter-branch --prune-empty -f --subdirectory-filter files_subtree &&
+	git fast-export --no-data HEAD -- files_subtree/ \
+		| sed -e "s%\([0-9a-f]\{40\} \)files_subtree/%\1%" \
+		| git fast-import --force --quiet &&
 	git commit -m "Empty commit" --allow-empty &&
 	git rebase -Xsubtree=files_subtree --onto files-master master &&
 	verbose test "$(commit_message HEAD)" = "Empty commit"