[v6,2/4] fetch: allow adding a filter after initial clone
diff mbox series

Message ID 20200605091004.208668-3-delphij@google.com
State New
Headers show
Series
  • fetch: allow adding a filter after initial clone
Related show

Commit Message

Xin Li June 5, 2020, 9:10 a.m. UTC
Retroactively adding a filter can be useful for existing shallow clones as
they allow users to see earlier change histories without downloading all
git objects in a regular --unshallow fetch.

Without this patch, users can make a clone partial by editing the
repository configuration to convert the remote into a promisor, like:

  git config core.repositoryFormatVersion 1
  git config extensions.partialClone origin
  git fetch --unshallow --filter=blob:none origin

Since the hard part of making this work is already in place and such
edits can be error-prone, teach Git to perform the required configuration
change automatically instead.

Note that this change does not modify the existing git behavior which
recognizes setting extensions.partialClone without changing
repositoryFormatVersion.

Signed-off-by: Xin Li <delphij@google.com>
---
 builtin/fetch.c                  |  3 ---
 t/t0410-partial-clone.sh         | 12 ++++++++++++
 t/t1090-sparse-checkout-scope.sh |  1 -
 t/t5500-fetch-pack.sh            |  1 -
 t/t5702-protocol-v2.sh           |  1 -
 5 files changed, 12 insertions(+), 6 deletions(-)

Comments

Junio C Hamano June 5, 2020, 7:15 p.m. UTC | #1
Xin Li <delphij@google.com> writes:

> Retroactively adding a filter can be useful for existing shallow clones as
> they allow users to see earlier change histories without downloading all
> git objects in a regular --unshallow fetch.
>
> Without this patch, users can make a clone partial by editing the
> repository configuration to convert the remote into a promisor, like:
>
>   git config core.repositoryFormatVersion 1
>   git config extensions.partialClone origin
>   git fetch --unshallow --filter=blob:none origin
>
> Since the hard part of making this work is already in place and such
> edits can be error-prone, teach Git to perform the required configuration
> change automatically instead.
>
> Note that this change does not modify the existing git behavior which
> recognizes setting extensions.partialClone without changing
> repositoryFormatVersion.

Hmph, I am not sure which part of the above corresponds to the
removal of "in a repository without extensions.partialClone you
cannot use the --filter option" logic.  With or without this patch,
users can butcher $GIT_DIR/config and pretend that the repository
was created with the pertial clone extension from the beginning, but
this step is not about automating those tasks, it seems.

>
> Signed-off-by: Xin Li <delphij@google.com>
> ---
>  builtin/fetch.c                  |  3 ---
>  t/t0410-partial-clone.sh         | 12 ++++++++++++
>  t/t1090-sparse-checkout-scope.sh |  1 -
>  t/t5500-fetch-pack.sh            |  1 -
>  t/t5702-protocol-v2.sh           |  1 -
>  5 files changed, 12 insertions(+), 6 deletions(-)
>
> diff --git a/builtin/fetch.c b/builtin/fetch.c
> index b5788c16bf..3347d578ea 100644
> --- a/builtin/fetch.c
> +++ b/builtin/fetch.c
> @@ -1790,9 +1790,6 @@ int cmd_fetch(int argc, const char **argv, const char *prefix)
>  	if (depth || deepen_since || deepen_not.nr)
>  		deepen = 1;
>  
> -	if (filter_options.choice && !has_promisor_remote())
> -		die("--filter can only be used when extensions.partialClone is set");
> -
>  	if (all) {
>  		if (argc == 1)
>  			die(_("fetch --all does not take a repository argument"));
> diff --git a/t/t0410-partial-clone.sh b/t/t0410-partial-clone.sh
> index a3988bd4b8..16ad000382 100755
> --- a/t/t0410-partial-clone.sh
> +++ b/t/t0410-partial-clone.sh
> @@ -30,6 +30,18 @@ test_expect_success 'extensions.partialclone without filter' '
>  	git -C client fetch origin
>  '
>  
> +test_expect_success 'convert shallow clone to partial clone' '
> +	rm -fr server client &&
> +	test_create_repo server &&
> +	test_commit -C server my_commit 1 &&
> +	test_commit -C server my_commit2 1 &&
> +	git clone --depth=1 "file://$(pwd)/server" client &&
> +	git -C client fetch --unshallow --filter="blob:none" &&
> +	test_cmp_config -C client true remote.origin.promisor &&
> +	test_cmp_config -C client blob:none remote.origin.partialclonefilter &&
> +	test_cmp_config -C client 1 core.repositoryformatversion
> +'
> +
>  test_expect_success 'missing reflog object, but promised by a commit, passes fsck' '
>  	rm -rf repo &&
>  	test_create_repo repo &&
> diff --git a/t/t1090-sparse-checkout-scope.sh b/t/t1090-sparse-checkout-scope.sh
> index 40cc004326..f35a73dd20 100755
> --- a/t/t1090-sparse-checkout-scope.sh
> +++ b/t/t1090-sparse-checkout-scope.sh
> @@ -63,7 +63,6 @@ test_expect_success 'in partial clone, sparse checkout only fetches needed blobs
>  	git -C server commit -m message &&
>  
>  	test_config -C client core.sparsecheckout 1 &&
> -	test_config -C client extensions.partialclone origin &&
>  	echo "!/*" >client/.git/info/sparse-checkout &&
>  	echo "/a" >>client/.git/info/sparse-checkout &&
>  	git -C client fetch --filter=blob:none origin &&
> diff --git a/t/t5500-fetch-pack.sh b/t/t5500-fetch-pack.sh
> index 8c54e34ef1..0f5ff25179 100755
> --- a/t/t5500-fetch-pack.sh
> +++ b/t/t5500-fetch-pack.sh
> @@ -999,7 +999,6 @@ fetch_filter_blob_limit_zero () {
>  	test_config -C "$SERVER" uploadpack.allowfilter 1 &&
>  
>  	git clone "$URL" client &&
> -	test_config -C client extensions.partialclone origin &&
>  
>  	test_commit -C "$SERVER" two &&
>  
> diff --git a/t/t5702-protocol-v2.sh b/t/t5702-protocol-v2.sh
> index 5039e66dc4..8b27fad6cd 100755
> --- a/t/t5702-protocol-v2.sh
> +++ b/t/t5702-protocol-v2.sh
> @@ -348,7 +348,6 @@ test_expect_success 'partial fetch' '
>  	rm -rf client "$(pwd)/trace" &&
>  	git init client &&
>  	SERVER="file://$(pwd)/server" &&
> -	test_config -C client extensions.partialClone "$SERVER" &&
>  
>  	GIT_TRACE_PACKET="$(pwd)/trace" git -C client -c protocol.version=2 \
>  		fetch --filter=blob:none "$SERVER" master:refs/heads/other &&

Patch
diff mbox series

diff --git a/builtin/fetch.c b/builtin/fetch.c
index b5788c16bf..3347d578ea 100644
--- a/builtin/fetch.c
+++ b/builtin/fetch.c
@@ -1790,9 +1790,6 @@  int cmd_fetch(int argc, const char **argv, const char *prefix)
 	if (depth || deepen_since || deepen_not.nr)
 		deepen = 1;
 
-	if (filter_options.choice && !has_promisor_remote())
-		die("--filter can only be used when extensions.partialClone is set");
-
 	if (all) {
 		if (argc == 1)
 			die(_("fetch --all does not take a repository argument"));
diff --git a/t/t0410-partial-clone.sh b/t/t0410-partial-clone.sh
index a3988bd4b8..16ad000382 100755
--- a/t/t0410-partial-clone.sh
+++ b/t/t0410-partial-clone.sh
@@ -30,6 +30,18 @@  test_expect_success 'extensions.partialclone without filter' '
 	git -C client fetch origin
 '
 
+test_expect_success 'convert shallow clone to partial clone' '
+	rm -fr server client &&
+	test_create_repo server &&
+	test_commit -C server my_commit 1 &&
+	test_commit -C server my_commit2 1 &&
+	git clone --depth=1 "file://$(pwd)/server" client &&
+	git -C client fetch --unshallow --filter="blob:none" &&
+	test_cmp_config -C client true remote.origin.promisor &&
+	test_cmp_config -C client blob:none remote.origin.partialclonefilter &&
+	test_cmp_config -C client 1 core.repositoryformatversion
+'
+
 test_expect_success 'missing reflog object, but promised by a commit, passes fsck' '
 	rm -rf repo &&
 	test_create_repo repo &&
diff --git a/t/t1090-sparse-checkout-scope.sh b/t/t1090-sparse-checkout-scope.sh
index 40cc004326..f35a73dd20 100755
--- a/t/t1090-sparse-checkout-scope.sh
+++ b/t/t1090-sparse-checkout-scope.sh
@@ -63,7 +63,6 @@  test_expect_success 'in partial clone, sparse checkout only fetches needed blobs
 	git -C server commit -m message &&
 
 	test_config -C client core.sparsecheckout 1 &&
-	test_config -C client extensions.partialclone origin &&
 	echo "!/*" >client/.git/info/sparse-checkout &&
 	echo "/a" >>client/.git/info/sparse-checkout &&
 	git -C client fetch --filter=blob:none origin &&
diff --git a/t/t5500-fetch-pack.sh b/t/t5500-fetch-pack.sh
index 8c54e34ef1..0f5ff25179 100755
--- a/t/t5500-fetch-pack.sh
+++ b/t/t5500-fetch-pack.sh
@@ -999,7 +999,6 @@  fetch_filter_blob_limit_zero () {
 	test_config -C "$SERVER" uploadpack.allowfilter 1 &&
 
 	git clone "$URL" client &&
-	test_config -C client extensions.partialclone origin &&
 
 	test_commit -C "$SERVER" two &&
 
diff --git a/t/t5702-protocol-v2.sh b/t/t5702-protocol-v2.sh
index 5039e66dc4..8b27fad6cd 100755
--- a/t/t5702-protocol-v2.sh
+++ b/t/t5702-protocol-v2.sh
@@ -348,7 +348,6 @@  test_expect_success 'partial fetch' '
 	rm -rf client "$(pwd)/trace" &&
 	git init client &&
 	SERVER="file://$(pwd)/server" &&
-	test_config -C client extensions.partialClone "$SERVER" &&
 
 	GIT_TRACE_PACKET="$(pwd)/trace" git -C client -c protocol.version=2 \
 		fetch --filter=blob:none "$SERVER" master:refs/heads/other &&