| Message ID | 17ec237ba7498251a3ff64eec259d6f61c8f5ccc.1552519463.git.steadmon@google.com (mailing list archive) |
|---|---|
| State | New, archived |
| Headers | show |
| Series | Randomize / timestamp trace2 targets | expand |
On Thu, Mar 14 2019, Josh Steadmon wrote: > When the value of a trace2 environment variable contains instances of > the string "%ISO8601%", expand them into the current UTC timestamp in > ISO 8601 format. Any reason not to just support feeding the path to strbuf_addftime(), to e.g. support a daily/hourly log? > When the value of a trace2 environment variable is an absolute path > referring to an existing directory, write output to randomly-named > files under the given directory. If the value is an absolute path > referring to a non-existent file and ends with a dash, use the value as > a prefix for randomly named files. > > The random filenames will consist of the value of the environment > variable (after potential timestamp expansion), followed by a 6 > character random string such as would be produced by mkstemp(3). > > This makes it more convenient to collect traces for every git > invocation by unconditionally setting the relevant trace2 envvar to a > constant directory name. Hrm, api-trace2.txt already specifies that the "sid" is going to be unique, couldn't we just have some mode where we use that? But then of course when we have nested processes will contain slashes, so we'd either run into deep nesting or need to munge the slashes, in which case we might bump against a file length limit (although I haven't seen process trees deeper than 3-4). Just to pry about the use-case since I'm doing similar collecting, why are you finding this easier to process? With the current O_APPEND semantics you're (unless I've missed something) guaranteed to get a single process tree in nested order, whereas with this they'll all end up in separate files and you'll need to slurp them up, sort the whole thing and stitch it together yourself without the benefit of stream-parsing it where you can cheat a bit knowing that e.g. a "reflog expire" entry is always coming after the corresponding "gc" that invoked it.
On Wed, Mar 13, 2019 at 04:33:29PM -0700, Josh Steadmon wrote: > When the value of a trace2 environment variable contains instances of > the string "%ISO8601%", expand them into the current UTC timestamp in > ISO 8601 format. This definitely seems useful. Could we drop the final "%" and make it either a single-character "%t" or "%(iso8601)" to match our other formatting strings? There's no _technical_ reason to do that, but it just seems like there's not much point in being unnecessarily inconsistent. > When the value of a trace2 environment variable is an absolute path > referring to an existing directory, write output to randomly-named > files under the given directory. If the value is an absolute path > referring to a non-existent file and ends with a dash, use the value as > a prefix for randomly named files. > > The random filenames will consist of the value of the environment > variable (after potential timestamp expansion), followed by a 6 > character random string such as would be produced by mkstemp(3). Doing this automatically for directories feels kind of magical. I'd have expected it to be just another placeholder. And in fact, I'd think using the process id as a placeholder would be pretty common. Between timestamp and pid, that's generally unique (though I'm not opposed to the random placeholder if somebody really wants to be thorough). That leaves the door open for being able to append to or overwrite existing trace files later, if we want to make that a possibility. -Peff
Jeff King <peff@peff.net> writes: > This definitely seems useful. Could we drop the final "%" and make it > either a single-character "%t" or "%(iso8601)" to match our other > formatting strings? There's no _technical_ reason to do that, but it > just seems like there's not much point in being unnecessarily > inconsistent. I do agree that %TOKEN% is not among the patterns we've used before, and I fully anticipated that it would attract paintbrushes when the proposal hits the public list ;-) A big question is what we want to be consisten with, though. From the readability point-of-view, for-each-ref language %(token) is easier to read and extend. A secondary question is that there likely are things other than the timestamp of the time process started that may want to be interpolated, so we may want to pick some useful vocabulary upfront. If we can declare interpolation will be done ONLY for timestamps, as Ævar suggests, taking strftime pattern directly from the caller may be sufficiently simple to explain and useful, but I am not sure "only for timestamps" is a limitation we would want to adopt this early in the design process. > Doing this automatically for directories feels kind of magical. I'd have > expected it to be just another placeholder. And in fact, I'd think using > the process id as a placeholder would be pretty common. Between > timestamp and pid, that's generally unique (though I'm not opposed to > the random placeholder if somebody really wants to be thorough). > > That leaves the door open for being able to append to or overwrite > existing trace files later, if we want to make that a possibility. Thanks.
On 3/13/2019 7:49 PM, Ævar Arnfjörð Bjarmason wrote: > > On Thu, Mar 14 2019, Josh Steadmon wrote: > >> When the value of a trace2 environment variable contains instances of >> the string "%ISO8601%", expand them into the current UTC timestamp in >> ISO 8601 format. > > Any reason not to just support feeding the path to strbuf_addftime(), to > e.g. support a daily/hourly log? > >> When the value of a trace2 environment variable is an absolute path >> referring to an existing directory, write output to randomly-named >> files under the given directory. If the value is an absolute path >> referring to a non-existent file and ends with a dash, use the value as >> a prefix for randomly named files. >> >> The random filenames will consist of the value of the environment >> variable (after potential timestamp expansion), followed by a 6 >> character random string such as would be produced by mkstemp(3). >> >> This makes it more convenient to collect traces for every git >> invocation by unconditionally setting the relevant trace2 envvar to a >> constant directory name. > > Hrm, api-trace2.txt already specifies that the "sid" is going to be > unique, couldn't we just have some mode where we use that? > > But then of course when we have nested processes will contain slashes, > so we'd either run into deep nesting or need to munge the slashes, in > which case we might bump against a file length limit (although I haven't > seen process trees deeper than 3-4). Using the "sid" would be a good place to start. Just take the final component in the string (after the last slash or the whole sid if there are no slashes). That will give you a filename with microseconds since epoch of the command's start time and the PID. That should be unique, should not require random strings, and not go deep in the filesystem. And it will let you correlate files between child and parent commands, if you need to. So maybe if GIT_TR2_* is set to a directory, we append the final portion of the "sid" and create a file inside that directory. > > Just to pry about the use-case since I'm doing similar collecting, why > are you finding this easier to process? > > With the current O_APPEND semantics you're (unless I've missed > something) guaranteed to get a single process tree in nested order, > whereas with this they'll all end up in separate files and you'll need > to slurp them up, sort the whole thing and stitch it together yourself > without the benefit of stream-parsing it where you can cheat a bit > knowing that e.g. a "reflog expire" entry is always coming after the > corresponding "gc" that invoked it. > Yes, with O_APPEND, you should get a series of events as they happen on the system all properly interleaved. And see concurrent activity. This file should let you grep to see individual processes if you want to. Routing each command to a different file is fine if you want, but that opens you up to having to manage and delete them. Whether to have 1 file (with occasional rotation) or 1 file-per-command depends, I guess, on how you want to process them. I'm routing the Trace2 data to a named-pipe/socket and have a daemon collecting and filtering, so I have a single pathname for output and yet get the per-file stream handling that I think Josh is looking for. Thanks, Jeff
On Fri, Mar 15 2019, Jeff Hostetler wrote: > On 3/13/2019 7:49 PM, Ævar Arnfjörð Bjarmason wrote: >> >> On Thu, Mar 14 2019, Josh Steadmon wrote: >> >>> When the value of a trace2 environment variable contains instances of >>> the string "%ISO8601%", expand them into the current UTC timestamp in >>> ISO 8601 format. >> >> Any reason not to just support feeding the path to strbuf_addftime(), to >> e.g. support a daily/hourly log? >> >>> When the value of a trace2 environment variable is an absolute path >>> referring to an existing directory, write output to randomly-named >>> files under the given directory. If the value is an absolute path >>> referring to a non-existent file and ends with a dash, use the value as >>> a prefix for randomly named files. >>> >>> The random filenames will consist of the value of the environment >>> variable (after potential timestamp expansion), followed by a 6 >>> character random string such as would be produced by mkstemp(3). >>> >>> This makes it more convenient to collect traces for every git >>> invocation by unconditionally setting the relevant trace2 envvar to a >>> constant directory name. >> >> Hrm, api-trace2.txt already specifies that the "sid" is going to be >> unique, couldn't we just have some mode where we use that? >> >> But then of course when we have nested processes will contain slashes, >> so we'd either run into deep nesting or need to munge the slashes, in >> which case we might bump against a file length limit (although I haven't >> seen process trees deeper than 3-4). > > Using the "sid" would be a good place to start. Just take the final > component in the string (after the last slash or the whole sid if there > are no slashes). That will give you a filename with microseconds since > epoch of the command's start time and the PID. > > That should be unique, should not require random strings, and not go > deep in the filesystem. And it will let you correlate files between > child and parent commands, if you need to. > > So maybe if GIT_TR2_* is set to a directory, we append the final portion > of the "sid" and create a file inside that directory. > >> >> Just to pry about the use-case since I'm doing similar collecting, why >> are you finding this easier to process? >> >> With the current O_APPEND semantics you're (unless I've missed >> something) guaranteed to get a single process tree in nested order, >> whereas with this they'll all end up in separate files and you'll need >> to slurp them up, sort the whole thing and stitch it together yourself >> without the benefit of stream-parsing it where you can cheat a bit >> knowing that e.g. a "reflog expire" entry is always coming after the >> corresponding "gc" that invoked it. >> > > Yes, with O_APPEND, you should get a series of events as they happen > on the system all properly interleaved. And see concurrent activity. > This file should let you grep to see individual processes if you want > to. > > Routing each command to a different file is fine if you want, but > that opens you up to having to manage and delete them. > > Whether to have 1 file (with occasional rotation) or 1 file-per-command > depends, I guess, on how you want to process them. > > I'm routing the Trace2 data to a named-pipe/socket and have a daemon > collecting and filtering, so I have a single pathname for output and > yet get the per-file stream handling that I think Josh is looking for. Is the collecting code something you can share & general enough that it might be useful for others?
On 3/15/2019 3:26 PM, Ævar Arnfjörð Bjarmason wrote: > > On Fri, Mar 15 2019, Jeff Hostetler wrote: > [...] >> >> I'm routing the Trace2 data to a named-pipe/socket and have a daemon >> collecting and filtering, so I have a single pathname for output and >> yet get the per-file stream handling that I think Josh is looking for. > > Is the collecting code something you can share & general enough that it > might be useful for others? > Yes, we're currently talking about releasing the source for that too. It probably won't happen this month, maybe early next quarter. Jeff
On 2019.03.14 00:49, Ævar Arnfjörð Bjarmason wrote: > > On Thu, Mar 14 2019, Josh Steadmon wrote: > > > When the value of a trace2 environment variable contains instances of > > the string "%ISO8601%", expand them into the current UTC timestamp in > > ISO 8601 format. > > Any reason not to just support feeding the path to strbuf_addftime(), to > e.g. support a daily/hourly log? No reason not to. Seems reasonable to me. > > When the value of a trace2 environment variable is an absolute path > > referring to an existing directory, write output to randomly-named > > files under the given directory. If the value is an absolute path > > referring to a non-existent file and ends with a dash, use the value as > > a prefix for randomly named files. > > > > The random filenames will consist of the value of the environment > > variable (after potential timestamp expansion), followed by a 6 > > character random string such as would be produced by mkstemp(3). > > > > This makes it more convenient to collect traces for every git > > invocation by unconditionally setting the relevant trace2 envvar to a > > constant directory name. > > Hrm, api-trace2.txt already specifies that the "sid" is going to be > unique, couldn't we just have some mode where we use that? > > But then of course when we have nested processes will contain slashes, > so we'd either run into deep nesting or need to munge the slashes, in > which case we might bump against a file length limit (although I haven't > seen process trees deeper than 3-4). > > Just to pry about the use-case since I'm doing similar collecting, why > are you finding this easier to process? Basically, our collection setup prefers smaller files that are "finished" earlier, rather than long-lived files that are constantly appended to. > With the current O_APPEND semantics you're (unless I've missed > something) guaranteed to get a single process tree in nested order, > whereas with this they'll all end up in separate files and you'll need > to slurp them up, sort the whole thing and stitch it together yourself > without the benefit of stream-parsing it where you can cheat a bit > knowing that e.g. a "reflog expire" entry is always coming after the > corresponding "gc" that invoked it. Yeah, that is not an issue for us, although I can see why others would prefer single file. I suppose we can just modify the envvar to point to our newly-generated file before we spawn any child processes?
On 2019.03.15 13:43, Josh Steadmon wrote: > On 2019.03.14 00:49, Ævar Arnfjörð Bjarmason wrote: > > > > On Thu, Mar 14 2019, Josh Steadmon wrote: > > > > > When the value of a trace2 environment variable contains instances of > > > the string "%ISO8601%", expand them into the current UTC timestamp in > > > ISO 8601 format. > > > > Any reason not to just support feeding the path to strbuf_addftime(), to > > e.g. support a daily/hourly log? > > No reason not to. Seems reasonable to me. Although as Junio says elsewhere in this thread, it's possible that we may want to support fields other than timestamps.
Josh Steadmon <steadmon@google.com> writes: > On 2019.03.15 13:43, Josh Steadmon wrote: >> On 2019.03.14 00:49, Ævar Arnfjörð Bjarmason wrote: >> > >> > On Thu, Mar 14 2019, Josh Steadmon wrote: >> > >> > > When the value of a trace2 environment variable contains instances of >> > > the string "%ISO8601%", expand them into the current UTC timestamp in >> > > ISO 8601 format. >> > >> > Any reason not to just support feeding the path to strbuf_addftime(), to >> > e.g. support a daily/hourly log? >> >> No reason not to. Seems reasonable to me. > > Although as Junio says elsewhere in this thread, it's possible that we > may want to support fields other than timestamps. Yup. On the other hand. After seeing that the possibilities got discussed on the list, and that nobody seems to be very much demanding customizability (I am taking Ævar's mention of strftime as a mere "if we were doing an optional timestamp, why not do so in an even more customizable way?" nice-to-have, not as a "we must allow hourly or daily log, adjusting for each host's needs" must-have), I actually am fine if we declare that we've chosen the hard-coded "if it is a directory, use the last portion of sid to create with O_EXCL (and if we fail, append a '.%d' counter to retry)" or something simple. Which I think takes us closer to your earlier and unpublished draft, but this time we can say that we omitted customizability after making sure that there is not much interest---so I think it was worth it. People who really want customizability can and are welcome to argue otherwise and then I may change my assessment of the level of interest in customizability, but the above is my current feeling. Thanks.
On Mon, Mar 18, 2019 at 10:40:00AM +0900, Junio C Hamano wrote: > After seeing that the possibilities got discussed on the list, and > that nobody seems to be very much demanding customizability (I am > taking Ævar's mention of strftime as a mere "if we were doing an > optional timestamp, why not do so in an even more customizable way?" > nice-to-have, not as a "we must allow hourly or daily log, adjusting > for each host's needs" must-have), I actually am fine if we declare > that we've chosen the hard-coded "if it is a directory, use the last > portion of sid to create with O_EXCL (and if we fail, append a '.%d' > counter to retry)" or something simple. Which I think takes us > closer to your earlier and unpublished draft, but this time we can > say that we omitted customizability after making sure that there is > not much interest---so I think it was worth it. > > People who really want customizability can and are welcome to argue > otherwise and then I may change my assessment of the level of > interest in customizability, but the above is my current feeling. I do not really care that much about this particular issue (and I haven't even really use trace2 for anything yet). My main concern was just painting ourselves into a corner, and making things explicit rather than implicit helps with that (i.e., having the user give us a placeholder that tells us what to do instead of selecting one of several reasonable behaviors based on whether the path exists). -Peff
diff --git a/Documentation/technical/api-trace2.txt b/Documentation/technical/api-trace2.txt index 2de565fa3d..1362bf7d0b 100644 --- a/Documentation/technical/api-trace2.txt +++ b/Documentation/technical/api-trace2.txt @@ -109,6 +109,16 @@ values are recognized. Enables the target, opens and writes to the file in append mode. + If the path includes any instances of the string "%ISO8601%", they will + be replaced with the current UTC timestamp in ISO 8601 format with + dashes and colons removed, e.g., "20190315T143059Z". + + If (after potential timestamp expansion) the path already exists and is + a directory, the traces will write to randomly-named files (one per + process) under the given directory. If the pathname does not already + exist and ends with a dash, it will be used as a prefix for + randomly-named files (one per process). + `af_unix:[<socket_type>:]<absolute-pathname>`:: Enables the target, opens and writes to a Unix Domain Socket diff --git a/t/t0210-trace2-normal.sh b/t/t0210-trace2-normal.sh index 03a0aedb1d..1b992c3e61 100755 --- a/t/t0210-trace2-normal.sh +++ b/t/t0210-trace2-normal.sh @@ -80,6 +80,99 @@ test_expect_success 'normal stream, return code 1' ' test_cmp expect actual ' +test_expect_success 'randomized filename' ' + test_when_finished "rm -r traces actual expect" && + mkdir traces && + GIT_TR2="$(pwd)/traces" test-tool trace2 001return 0 && + perl "$TEST_DIRECTORY/t0210/scrub_normal.perl" <"$(ls traces/??????)" >actual && + cat >expect <<-EOF && + version $V + start _EXE_ trace2 001return 0 + cmd_name trace2 (trace2) + exit elapsed:_TIME_ code:0 + atexit elapsed:_TIME_ code:0 + EOF + test_cmp expect actual +' + +test_expect_success 'randomized filename with prefix' ' + test_when_finished "rm -r traces actual expect" && + mkdir traces && + GIT_TR2="$(pwd)/traces/trace-" test-tool trace2 001return 0 && + perl "$TEST_DIRECTORY/t0210/scrub_normal.perl" <"$(ls traces/trace-??????)" >actual && + cat >expect <<-EOF && + version $V + start _EXE_ trace2 001return 0 + cmd_name trace2 (trace2) + exit elapsed:_TIME_ code:0 + atexit elapsed:_TIME_ code:0 + EOF + test_cmp expect actual +' + +test_expect_success 'timestamped filename' ' + test_when_finished "rm -r traces actual expect" && + mkdir traces && + GIT_TEST_DATE_NOW=1552658399 GIT_TR2="$(pwd)/traces/trace.%ISO8601%" \ + test-tool trace2 001return 0 && + perl "$TEST_DIRECTORY/t0210/scrub_normal.perl" <traces/trace.20190315T135959Z >actual && + cat >expect <<-EOF && + version $V + start _EXE_ trace2 001return 0 + cmd_name trace2 (trace2) + exit elapsed:_TIME_ code:0 + atexit elapsed:_TIME_ code:0 + EOF + test_cmp expect actual +' + +test_expect_success 'multiple timestamps' ' + test_when_finished "rm -r traces actual expect" && + mkdir -p traces/20190315T135959Z && + GIT_TEST_DATE_NOW=1552658399 GIT_TR2="$(pwd)/traces/%ISO8601%/trace.%ISO8601%" \ + test-tool trace2 001return 0 && + perl "$TEST_DIRECTORY/t0210/scrub_normal.perl" <traces/20190315T135959Z/trace.20190315T135959Z >actual && + cat >expect <<-EOF && + version $V + start _EXE_ trace2 001return 0 + cmd_name trace2 (trace2) + exit elapsed:_TIME_ code:0 + atexit elapsed:_TIME_ code:0 + EOF + test_cmp expect actual +' + +test_expect_success 'timestamp plus randomization' ' + test_when_finished "rm -r traces actual expect" && + mkdir traces && + GIT_TEST_DATE_NOW=1552658399 GIT_TR2="$(pwd)/traces/trace-%ISO8601%-" test-tool trace2 001return 0 && + perl "$TEST_DIRECTORY/t0210/scrub_normal.perl" <"$(ls traces/trace-20190315T135959Z-??????)" >actual && + cat >expect <<-EOF && + version $V + start _EXE_ trace2 001return 0 + cmd_name trace2 (trace2) + exit elapsed:_TIME_ code:0 + atexit elapsed:_TIME_ code:0 + EOF + test_cmp expect actual +' +test_expect_success 'no randomization if target exists' ' + test_when_finished "rm -r traces actual expect" && + mkdir traces && + touch traces/trace- && + GIT_TR2="$(pwd)/traces/trace-" test-tool trace2 001return 0 && + perl "$TEST_DIRECTORY/t0210/scrub_normal.perl" <traces/trace- >actual && + cat >expect <<-EOF && + version $V + start _EXE_ trace2 001return 0 + cmd_name trace2 (trace2) + exit elapsed:_TIME_ code:0 + atexit elapsed:_TIME_ code:0 + EOF + test_cmp expect actual +' + + # Verb 002exit # # Explicit exit(code) from within cmd_<verb> propagates <code>. diff --git a/trace2/tr2_dst.c b/trace2/tr2_dst.c index fd490a43ad..27405a92b4 100644 --- a/trace2/tr2_dst.c +++ b/trace2/tr2_dst.c @@ -12,6 +12,11 @@ */ #define TR2_ENVVAR_DST_DEBUG "GIT_TR2_DST_DEBUG" +/* Constant string used for timestamp replacement in output destination + * filenames. See tr2_replace_timestamp_templates() below. + */ +static const char *iso_timestamp_tmpl = "%ISO8601%"; + static int tr2_dst_want_warning(void) { static int tr2env_dst_debug = -1; @@ -55,6 +60,64 @@ static int tr2_dst_try_path(struct tr2_dst *dst, const char *tgt_value) return dst->fd; } +static void tr2_replace_timestamp_templates(const char *path, + struct strbuf *modified_path) +{ + char *iso_ptr; + + strbuf_addstr(modified_path, path); + + iso_ptr = strstr(modified_path->buf, iso_timestamp_tmpl); + if (iso_ptr) { + struct timeval tv; + struct tm tm; + size_t iso_len = strlen(iso_timestamp_tmpl); + struct strbuf timestamp = STRBUF_INIT; + + get_time(&tv); + gmtime_r(&tv.tv_sec, &tm); + strbuf_addftime(×tamp, "%Y%m%dT%H%M%SZ", &tm, 0, 0); + + while (iso_ptr) { + strbuf_splice(modified_path, + iso_ptr - modified_path->buf, iso_len, + timestamp.buf, timestamp.len); + + iso_ptr = strstr(modified_path->buf, + iso_timestamp_tmpl); + } + + strbuf_release(×tamp); + } +} + +static int tr2_dst_try_random_path(struct tr2_dst *dst, struct strbuf *path) +{ + int fd; + char last_path_char; + + last_path_char = path->buf[path->len - 1]; + if (!is_dir_sep(last_path_char) && last_path_char != '-') + strbuf_addch(path, '/'); + + strbuf_addstr(path, "XXXXXX"); + + fd = mkstemp(path->buf); + if (fd == -1) { + if (tr2_dst_want_warning()) + warning("trace2: could not open '%s' for '%s' tracing: %s", + path->buf, dst->env_var_name, strerror(errno)); + tr2_dst_trace_disable(dst); + return 0; + } + + dst->fd = fd; + dst->need_close = 1; + dst->initialized = 1; + + return dst->fd; +} + #ifndef NO_UNIX_SOCKETS #define PREFIX_AF_UNIX "af_unix:" #define PREFIX_AF_UNIX_STREAM "af_unix:stream:" @@ -177,6 +240,7 @@ static void tr2_dst_malformed_warning(struct tr2_dst *dst, int tr2_dst_get_trace_fd(struct tr2_dst *dst) { const char *tgt_value; + struct stat st; /* don't open twice */ if (dst->initialized) @@ -202,8 +266,26 @@ int tr2_dst_get_trace_fd(struct tr2_dst *dst) return dst->fd; } - if (is_absolute_path(tgt_value)) - return tr2_dst_try_path(dst, tgt_value); + if (is_absolute_path(tgt_value)) { + int fd; + struct strbuf modified_path = STRBUF_INIT; + + tr2_replace_timestamp_templates(tgt_value, &modified_path); + + /* + * Randomize the path if it is an existing directory, or if the + * path does not exist and ends with '-'. + */ + if (is_directory(modified_path.buf) || + (stat(modified_path.buf, &st) == -1 && errno == ENOENT && + modified_path.buf[modified_path.len - 1] == '-')) + fd = tr2_dst_try_random_path(dst, &modified_path); + else + fd = tr2_dst_try_path(dst, modified_path.buf); + + strbuf_release(&modified_path); + return fd; + } #ifndef NO_UNIX_SOCKETS if (starts_with(tgt_value, PREFIX_AF_UNIX))
When the value of a trace2 environment variable contains instances of the string "%ISO8601%", expand them into the current UTC timestamp in ISO 8601 format. When the value of a trace2 environment variable is an absolute path referring to an existing directory, write output to randomly-named files under the given directory. If the value is an absolute path referring to a non-existent file and ends with a dash, use the value as a prefix for randomly named files. The random filenames will consist of the value of the environment variable (after potential timestamp expansion), followed by a 6 character random string such as would be produced by mkstemp(3). This makes it more convenient to collect traces for every git invocation by unconditionally setting the relevant trace2 envvar to a constant directory name. Signed-off-by: Josh Steadmon <steadmon@google.com> --- Documentation/technical/api-trace2.txt | 10 +++ t/t0210-trace2-normal.sh | 93 ++++++++++++++++++++++++++ trace2/tr2_dst.c | 86 +++++++++++++++++++++++- 3 files changed, 187 insertions(+), 2 deletions(-)