mbox series

[0/2] add trace2 regions to fetch & push

Message ID cover.1570059953.git.steadmon@google.com (mailing list archive)
Headers show
Series add trace2 regions to fetch & push | expand

Message

Josh Steadmon Oct. 2, 2019, 11:49 p.m. UTC
We'd like to collect better statistics about where the time is spent in
fetches and pushes so that we can hopefully identify some areas for
future optimization. So let's add some trace2 regions around some of the
fetch/push phases so we can break down their timing.

Josh Steadmon (2):
  fetch: add trace2 instrumentation
  push: add trace2 instrumentation

 builtin/fetch.c | 22 +++++++++++++++-------
 builtin/push.c  |  2 ++
 fetch-pack.c    | 13 ++++++++++++-
 transport.c     | 14 ++++++++++++--
 4 files changed, 41 insertions(+), 10 deletions(-)

Comments

Jonathan Tan Oct. 7, 2019, 9:46 p.m. UTC | #1
> We'd like to collect better statistics about where the time is spent in
> fetches and pushes so that we can hopefully identify some areas for
> future optimization. So let's add some trace2 regions around some of the
> fetch/push phases so we can break down their timing.

Thanks.

Patch 1 looks good to me - different regions at the same level
(builtin/fetch.c, so it will be just for "git fetch") and one specific
one just for negotiation, which has to be in fetch-pack.c because only
that file operates at that level.

Patch 2 mostly looks good to me too - unlike fetch, a lot happens in
transport.c, so it's reasonable to put most of the regions there. One
comment: in transport_push(), should trace2_region_{enter,leave} take
"r" instead of "the_repository"?
Josh Steadmon Oct. 7, 2019, 10:36 p.m. UTC | #2
On 2019.10.07 14:46, Jonathan Tan wrote:
> > We'd like to collect better statistics about where the time is spent in
> > fetches and pushes so that we can hopefully identify some areas for
> > future optimization. So let's add some trace2 regions around some of the
> > fetch/push phases so we can break down their timing.
> 
> Thanks.
> 
> Patch 1 looks good to me - different regions at the same level
> (builtin/fetch.c, so it will be just for "git fetch") and one specific
> one just for negotiation, which has to be in fetch-pack.c because only
> that file operates at that level.
> 
> Patch 2 mostly looks good to me too - unlike fetch, a lot happens in
> transport.c, so it's reasonable to put most of the regions there. One
> comment: in transport_push(), should trace2_region_{enter,leave} take
> "r" instead of "the_repository"?

Ah yeah, thanks for the catch. Fixed in V2.
Junio C Hamano Oct. 8, 2019, 3:53 a.m. UTC | #3
Josh Steadmon <steadmon@google.com> writes:

> On 2019.10.07 14:46, Jonathan Tan wrote:
>> > We'd like to collect better statistics about where the time is spent in
>> > fetches and pushes so that we can hopefully identify some areas for
>> > future optimization. So let's add some trace2 regions around some of the
>> > fetch/push phases so we can break down their timing.
>> 
>> Thanks.
>> 
>> Patch 1 looks good to me - different regions at the same level
>> (builtin/fetch.c, so it will be just for "git fetch") and one specific
>> one just for negotiation, which has to be in fetch-pack.c because only
>> that file operates at that level.
>> 
>> Patch 2 mostly looks good to me too - unlike fetch, a lot happens in
>> transport.c, so it's reasonable to put most of the regions there. One
>> comment: in transport_push(), should trace2_region_{enter,leave} take
>> "r" instead of "the_repository"?
>
> Ah yeah, thanks for the catch. Fixed in V2.

Yuck.  It's already in 'next', isn't it?
Junio C Hamano Oct. 8, 2019, 4:18 a.m. UTC | #4
Junio C Hamano <gitster@pobox.com> writes:

> Yuck.  It's already in 'next', isn't it?

Yuck, indeed.  Here is your v2 in an incremental form to be queued
on top.

Thanks.

-- >8 --
Subject: [PATCH 3/2] transport: push codepath can take arbitrary repository

The previous step added annotations with "the_repository" to various
functions in the push codepath in the transport layer, but they all
can take arbitrary repository pointer, and may be working on a
repository that is not the_repository.  Fix them.

Signed-off-by: Josh Steadmon <steadmon@google.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
---
 transport.c | 20 ++++++++++----------
 1 file changed, 10 insertions(+), 10 deletions(-)

diff --git a/transport.c b/transport.c
index f313f288de..83379a037d 100644
--- a/transport.c
+++ b/transport.c
@@ -1145,10 +1145,10 @@ int transport_push(struct repository *r,
 
 		refspec_ref_prefixes(rs, &ref_prefixes);
 
-		trace2_region_enter("transport_push", "get_refs_list", the_repository);
+		trace2_region_enter("transport_push", "get_refs_list", r);
 		remote_refs = transport->vtable->get_refs_list(transport, 1,
 							       &ref_prefixes);
-		trace2_region_leave("transport_push", "get_refs_list", the_repository);
+		trace2_region_leave("transport_push", "get_refs_list", r);
 
 		argv_array_clear(&ref_prefixes);
 
@@ -1184,7 +1184,7 @@ int transport_push(struct repository *r,
 			struct ref *ref = remote_refs;
 			struct oid_array commits = OID_ARRAY_INIT;
 
-			trace2_region_enter("transport_push", "push_submodules", the_repository);
+			trace2_region_enter("transport_push", "push_submodules", r);
 			for (; ref; ref = ref->next)
 				if (!is_null_oid(&ref->new_oid))
 					oid_array_append(&commits,
@@ -1197,11 +1197,11 @@ int transport_push(struct repository *r,
 						      transport->push_options,
 						      pretend)) {
 				oid_array_clear(&commits);
-				trace2_region_leave("transport_push", "push_submodules", the_repository);
+				trace2_region_leave("transport_push", "push_submodules", r);
 				die(_("failed to push all needed submodules"));
 			}
 			oid_array_clear(&commits);
-			trace2_region_leave("transport_push", "push_submodules", the_repository);
+			trace2_region_leave("transport_push", "push_submodules", r);
 		}
 
 		if (((flags & TRANSPORT_RECURSE_SUBMODULES_CHECK) ||
@@ -1212,7 +1212,7 @@ int transport_push(struct repository *r,
 			struct string_list needs_pushing = STRING_LIST_INIT_DUP;
 			struct oid_array commits = OID_ARRAY_INIT;
 
-			trace2_region_enter("transport_push", "check_submodules", the_repository);
+			trace2_region_enter("transport_push", "check_submodules", r);
 			for (; ref; ref = ref->next)
 				if (!is_null_oid(&ref->new_oid))
 					oid_array_append(&commits,
@@ -1223,18 +1223,18 @@ int transport_push(struct repository *r,
 						     transport->remote->name,
 						     &needs_pushing)) {
 				oid_array_clear(&commits);
-				trace2_region_leave("transport_push", "check_submodules", the_repository);
+				trace2_region_leave("transport_push", "check_submodules", r);
 				die_with_unpushed_submodules(&needs_pushing);
 			}
 			string_list_clear(&needs_pushing, 0);
 			oid_array_clear(&commits);
-			trace2_region_leave("transport_push", "check_submodules", the_repository);
+			trace2_region_leave("transport_push", "check_submodules", r);
 		}
 
 		if (!(flags & TRANSPORT_RECURSE_SUBMODULES_ONLY)) {
-			trace2_region_enter("transport_push", "push_refs", the_repository);
+			trace2_region_enter("transport_push", "push_refs", r);
 			push_ret = transport->vtable->push_refs(transport, remote_refs, flags);
-			trace2_region_leave("transport_push", "push_refs", the_repository);
+			trace2_region_leave("transport_push", "push_refs", r);
 		} else
 			push_ret = 0;
 		err = push_had_errors(remote_refs);