diff mbox series

[v5,2/2] am: reload .gitattributes after patching it

Message ID 20190825233340.10894-3-sandals@crustytoothpaste.net (mailing list archive)
State New, archived
Headers show
Series Honor .gitattributes with rebase --am | expand

Commit Message

brian m. carlson Aug. 25, 2019, 11:33 p.m. UTC
When applying multiple patches with git am, or when rebasing using the
am backend, it's possible that one of our patches has updated a
gitattributes file. Currently, we cache this information, so if a
file in a subsequent patch has attributes applied, the file will be
written out with the attributes in place as of the time we started the
rebase or am operation, not with the attributes applied by the previous
patch. This problem does not occur when using the -m or -i flags to
rebase.

To ensure we write the correct data into the working tree, expire the
cache after each patch that touches a path ending in ".gitattributes".
Since we load these attributes in multiple separate files, we must
expire them accordingly.

Verify that both the am and rebase code paths work correctly, including
the conflict marker size with am -3.

Signed-off-by: brian m. carlson <sandals@crustytoothpaste.net>
---
 apply.c           | 11 ++++++++++
 convert.c         | 11 +++++++++-
 convert.h         |  6 ++++++
 ll-merge.c        | 19 +++++++++++++----
 ll-merge.h        |  1 +
 t/t3400-rebase.sh | 36 ++++++++++++++++++++++++++++++++
 t/t4150-am.sh     | 52 +++++++++++++++++++++++++++++++++++++++++++++++
 7 files changed, 131 insertions(+), 5 deletions(-)

Comments

Johannes Schindelin Aug. 28, 2019, 11:30 a.m. UTC | #1
Hi brian,

[chiming in from the peanut gallery; if my comments don't make any
sense, please do feel free to completely ignore me.]

On Sun, 25 Aug 2019, brian m. carlson wrote:

> diff --git a/convert.c b/convert.c
> index 94ff837649..0e6e9d2d75 100644
> --- a/convert.c
> +++ b/convert.c
> @@ -8,6 +8,7 @@
>  #include "pkt-line.h"
>  #include "sub-process.h"
>  #include "utf8.h"
> +#include "ll-merge.h"
>
>  /*
>   * convert.c - convert a file when checking it out and checking it in.
> @@ -1293,10 +1294,11 @@ struct conv_attrs {
>  	const char *working_tree_encoding; /* Supported encoding or default encoding if NULL */
>  };
>
> +static struct attr_check *check;
> +
>  static void convert_attrs(const struct index_state *istate,
>  			  struct conv_attrs *ca, const char *path)
>  {
> -	static struct attr_check *check;
>  	struct attr_check_item *ccheck = NULL;
>
>  	if (!check) {

After this line:

                check = attr_check_initl("crlf", "ident", "filter",
				"eol", "text", "working-tree-encoding",
				NULL);
		user_convert_tail = &user_convert;
		git_config(read_convert_config, NULL);
	}

I am a bit worried about `user_convert`: it seems never to be re-set.

Also, how thread-safe do we need `reset_parsed_attributes()` to be?

Ciao,
Dscho
brian m. carlson Aug. 29, 2019, 11:09 p.m. UTC | #2
On 2019-08-28 at 11:30:53, Johannes Schindelin wrote:
> > diff --git a/convert.c b/convert.c
> > index 94ff837649..0e6e9d2d75 100644
> > --- a/convert.c
> > +++ b/convert.c
> > @@ -8,6 +8,7 @@
> >  #include "pkt-line.h"
> >  #include "sub-process.h"
> >  #include "utf8.h"
> > +#include "ll-merge.h"
> >
> >  /*
> >   * convert.c - convert a file when checking it out and checking it in.
> > @@ -1293,10 +1294,11 @@ struct conv_attrs {
> >  	const char *working_tree_encoding; /* Supported encoding or default encoding if NULL */
> >  };
> >
> > +static struct attr_check *check;
> > +
> >  static void convert_attrs(const struct index_state *istate,
> >  			  struct conv_attrs *ca, const char *path)
> >  {
> > -	static struct attr_check *check;
> >  	struct attr_check_item *ccheck = NULL;
> >
> >  	if (!check) {
> 
> After this line:
> 
>                 check = attr_check_initl("crlf", "ident", "filter",
> 				"eol", "text", "working-tree-encoding",
> 				NULL);
> 		user_convert_tail = &user_convert;
> 		git_config(read_convert_config, NULL);
> 	}
> 
> I am a bit worried about `user_convert`: it seems never to be re-set.

Yeah, it looks like I'll need to reset that as well.  The only
consequence is that we leak a small amount of memory if there are filter
attributes, but it's better to avoid that leak if we can.

> Also, how thread-safe do we need `reset_parsed_attributes()` to be?

Since patch application isn't thread safe, it doesn't need to be
thread safe at all.  The original wasn't thread safe, either, since it
used a static variable without a mutex.
Johannes Schindelin Aug. 30, 2019, 7:46 p.m. UTC | #3
Hi brian,

On Thu, 29 Aug 2019, brian m. carlson wrote:

> On 2019-08-28 at 11:30:53, Johannes Schindelin wrote:
> > > diff --git a/convert.c b/convert.c
> > > index 94ff837649..0e6e9d2d75 100644
> > > --- a/convert.c
> > > +++ b/convert.c
> > > @@ -8,6 +8,7 @@
> > >  #include "pkt-line.h"
> > >  #include "sub-process.h"
> > >  #include "utf8.h"
> > > +#include "ll-merge.h"
> > >
> > >  /*
> > >   * convert.c - convert a file when checking it out and checking it in.
> > > @@ -1293,10 +1294,11 @@ struct conv_attrs {
> > >  	const char *working_tree_encoding; /* Supported encoding or default encoding if NULL */
> > >  };
> > >
> > > +static struct attr_check *check;
> > > +
> > >  static void convert_attrs(const struct index_state *istate,
> > >  			  struct conv_attrs *ca, const char *path)
> > >  {
> > > -	static struct attr_check *check;
> > >  	struct attr_check_item *ccheck = NULL;
> > >
> > >  	if (!check) {
> >
> > After this line:
> >
> >                 check = attr_check_initl("crlf", "ident", "filter",
> > 				"eol", "text", "working-tree-encoding",
> > 				NULL);
> > 		user_convert_tail = &user_convert;
> > 		git_config(read_convert_config, NULL);
> > 	}
> >
> > I am a bit worried about `user_convert`: it seems never to be re-set.
>
> Yeah, it looks like I'll need to reset that as well.  The only
> consequence is that we leak a small amount of memory if there are filter
> attributes, but it's better to avoid that leak if we can.

Okay, good, so my suggestion was not so completely off the mark.
>
> > Also, how thread-safe do we need `reset_parsed_attributes()` to be?
>
> Since patch application isn't thread safe, it doesn't need to be
> thread safe at all.  The original wasn't thread safe, either, since it
> used a static variable without a mutex.

Thank you for clarifying! Much appreciated.

Ciao,
Dscho
diff mbox series

Patch

diff --git a/apply.c b/apply.c
index cde95369bb..57a61f2881 100644
--- a/apply.c
+++ b/apply.c
@@ -4643,6 +4643,7 @@  static int apply_patch(struct apply_state *state,
 	struct patch *list = NULL, **listp = &list;
 	int skipped_patch = 0;
 	int res = 0;
+	int flush_attributes = 0;
 
 	state->patch_input_file = filename;
 	if (read_patch_file(&buf, fd) < 0)
@@ -4670,6 +4671,14 @@  static int apply_patch(struct apply_state *state,
 			patch_stats(state, patch);
 			*listp = patch;
 			listp = &patch->next;
+
+			if ((patch->new_name &&
+			     ends_with_path_components(patch->new_name,
+						       GITATTRIBUTES_FILE)) ||
+			    (patch->old_name &&
+			     ends_with_path_components(patch->old_name,
+						       GITATTRIBUTES_FILE)))
+				flush_attributes = 1;
 		}
 		else {
 			if (state->apply_verbosity > verbosity_normal)
@@ -4746,6 +4755,8 @@  static int apply_patch(struct apply_state *state,
 	if (state->summary && state->apply_verbosity > verbosity_silent)
 		summary_patch_list(list);
 
+	if (flush_attributes)
+		reset_parsed_attributes();
 end:
 	free_patch_list(list);
 	strbuf_release(&buf);
diff --git a/convert.c b/convert.c
index 94ff837649..0e6e9d2d75 100644
--- a/convert.c
+++ b/convert.c
@@ -8,6 +8,7 @@ 
 #include "pkt-line.h"
 #include "sub-process.h"
 #include "utf8.h"
+#include "ll-merge.h"
 
 /*
  * convert.c - convert a file when checking it out and checking it in.
@@ -1293,10 +1294,11 @@  struct conv_attrs {
 	const char *working_tree_encoding; /* Supported encoding or default encoding if NULL */
 };
 
+static struct attr_check *check;
+
 static void convert_attrs(const struct index_state *istate,
 			  struct conv_attrs *ca, const char *path)
 {
-	static struct attr_check *check;
 	struct attr_check_item *ccheck = NULL;
 
 	if (!check) {
@@ -1339,6 +1341,13 @@  static void convert_attrs(const struct index_state *istate,
 		ca->crlf_action = CRLF_AUTO_INPUT;
 }
 
+void reset_parsed_attributes(void)
+{
+	attr_check_free(check);
+	check = NULL;
+	reset_merge_attributes();
+}
+
 int would_convert_to_git_filter_fd(const struct index_state *istate, const char *path)
 {
 	struct conv_attrs ca;
diff --git a/convert.h b/convert.h
index 831559f10d..3710969d43 100644
--- a/convert.h
+++ b/convert.h
@@ -94,6 +94,12 @@  void convert_to_git_filter_fd(const struct index_state *istate,
 int would_convert_to_git_filter_fd(const struct index_state *istate,
 				   const char *path);
 
+/*
+ * Reset the internal list of attributes used by convert_to_git and
+ * convert_to_working_tree.
+ */
+void reset_parsed_attributes(void);
+
 /*****************************************************************
  *
  * Streaming conversion support
diff --git a/ll-merge.c b/ll-merge.c
index 5b8d46aede..d65a8971db 100644
--- a/ll-merge.c
+++ b/ll-merge.c
@@ -32,6 +32,20 @@  struct ll_merge_driver {
 	char *cmdline;
 };
 
+static struct attr_check *merge_attributes;
+static struct attr_check *load_merge_attributes(void)
+{
+	if (!merge_attributes)
+		merge_attributes = attr_check_initl("merge", "conflict-marker-size", NULL);
+	return merge_attributes;
+}
+
+void reset_merge_attributes(void)
+{
+	attr_check_free(merge_attributes);
+	merge_attributes = NULL;
+}
+
 /*
  * Built-in low-levels
  */
@@ -354,7 +368,7 @@  int ll_merge(mmbuffer_t *result_buf,
 	     struct index_state *istate,
 	     const struct ll_merge_options *opts)
 {
-	static struct attr_check *check;
+	struct attr_check *check = load_merge_attributes();
 	static const struct ll_merge_options default_opts;
 	const char *ll_driver_name = NULL;
 	int marker_size = DEFAULT_CONFLICT_MARKER_SIZE;
@@ -369,9 +383,6 @@  int ll_merge(mmbuffer_t *result_buf,
 		normalize_file(theirs, path, istate);
 	}
 
-	if (!check)
-		check = attr_check_initl("merge", "conflict-marker-size", NULL);
-
 	git_check_attr(istate, path, check);
 	ll_driver_name = check->items[0].value;
 	if (check->items[1].value) {
diff --git a/ll-merge.h b/ll-merge.h
index b9e2af1c88..e78973dd55 100644
--- a/ll-merge.h
+++ b/ll-merge.h
@@ -26,5 +26,6 @@  int ll_merge(mmbuffer_t *result_buf,
 	     const struct ll_merge_options *opts);
 
 int ll_merge_marker_size(struct index_state *istate, const char *path);
+void reset_merge_attributes(void);
 
 #endif
diff --git a/t/t3400-rebase.sh b/t/t3400-rebase.sh
index 80b23fd326..23469cc789 100755
--- a/t/t3400-rebase.sh
+++ b/t/t3400-rebase.sh
@@ -301,6 +301,42 @@  test_expect_success 'rebase --am and --show-current-patch' '
 	)
 '
 
+test_expect_success 'rebase --am and .gitattributes' '
+	test_create_repo attributes &&
+	(
+		cd attributes &&
+		test_commit init &&
+		git config filter.test.clean "sed -e '\''s/smudged/clean/g'\''" &&
+		git config filter.test.smudge "sed -e '\''s/clean/smudged/g'\''" &&
+
+		test_commit second &&
+		git checkout -b test HEAD^ &&
+
+		echo "*.txt filter=test" >.gitattributes &&
+		git add .gitattributes &&
+		test_commit third &&
+
+		echo "This text is smudged." >a.txt &&
+		git add a.txt &&
+		test_commit fourth &&
+
+		git checkout -b removal HEAD^ &&
+		git rm .gitattributes &&
+		git add -u &&
+		test_commit fifth &&
+		git cherry-pick test &&
+
+		git checkout test &&
+		git rebase master &&
+		grep "smudged" a.txt &&
+
+		git checkout removal &&
+		git reset --hard &&
+		git rebase master &&
+		grep "clean" a.txt
+	)
+'
+
 test_expect_success 'rebase--merge.sh and --show-current-patch' '
 	test_create_repo conflict-merge &&
 	(
diff --git a/t/t4150-am.sh b/t/t4150-am.sh
index 3f7f750cc8..4f1e24ecbe 100755
--- a/t/t4150-am.sh
+++ b/t/t4150-am.sh
@@ -1061,4 +1061,56 @@  test_expect_success 'am --quit keeps HEAD where it is' '
 	test_cmp expected actual
 '
 
+test_expect_success 'am and .gitattibutes' '
+	test_create_repo attributes &&
+	(
+		cd attributes &&
+		test_commit init &&
+		git config filter.test.clean "sed -e '\''s/smudged/clean/g'\''" &&
+		git config filter.test.smudge "sed -e '\''s/clean/smudged/g'\''" &&
+
+		test_commit second &&
+		git checkout -b test HEAD^ &&
+
+		echo "*.txt filter=test conflict-marker-size=10" >.gitattributes &&
+		git add .gitattributes &&
+		test_commit third &&
+
+		echo "This text is smudged." >a.txt &&
+		git add a.txt &&
+		test_commit fourth &&
+
+		git checkout -b removal HEAD^ &&
+		git rm .gitattributes &&
+		git add -u &&
+		test_commit fifth &&
+		git cherry-pick test &&
+
+		git checkout -b conflict third &&
+		echo "This text is different." >a.txt &&
+		git add a.txt &&
+		test_commit sixth &&
+
+		git checkout test &&
+		git format-patch --stdout master..HEAD >patches &&
+		git reset --hard master &&
+		git am patches &&
+		grep "smudged" a.txt &&
+
+		git checkout removal &&
+		git reset --hard &&
+		git format-patch --stdout master..HEAD >patches &&
+		git reset --hard master &&
+		git am patches &&
+		grep "clean" a.txt &&
+
+		git checkout conflict &&
+		git reset --hard &&
+		git format-patch --stdout master..HEAD >patches &&
+		git reset --hard fourth &&
+		test_must_fail git am -3 patches &&
+		grep "<<<<<<<<<<" a.txt
+	)
+'
+
 test_done