| Message ID | 20200821175451.GB3263141@coredump.intra.peff.net (mailing list archive) |
|---|---|
| State | Accepted |
| Commit | 218389b9f3d1b5c8583cc0e944692f7c66af8a03 |
| Headers | show |
| Series | index-pack threading defaults | expand |
On Fri, Aug 21, 2020 at 01:54:51PM -0400, Jeff King wrote: > t/perf/p5302-pack-index.sh | 47 +++++++++++++++++++------------------- > 1 file changed, 24 insertions(+), 23 deletions(-) I meant to generate this diff with --patience, which is much more readable: diff --git a/t/perf/p5302-pack-index.sh b/t/perf/p5302-pack-index.sh index 23011ab739..228593d9ad 100755 --- a/t/perf/p5302-pack-index.sh +++ b/t/perf/p5302-pack-index.sh @@ -13,35 +13,36 @@ test_expect_success 'repack' ' export PACK ' +# Rather than counting up and doubling each time, count down from the endpoint, +# halving each time. That ensures that our final test uses as many threads as +# CPUs, even if it isn't a power of 2. +test_expect_success 'set up thread-counting tests' ' + t=$(test-tool online-cpus) && + threads= && + while test $t -gt 0 + do + threads="$t $threads" + t=$((t / 2)) + done +' + test_perf PERF_EXTRA 'index-pack 0 threads' ' rm -rf repo.git && git init --bare repo.git && GIT_DIR=repo.git git index-pack --threads=1 --stdin < $PACK ' -test_perf PERF_EXTRA 'index-pack 1 thread ' ' - rm -rf repo.git && - git init --bare repo.git && - GIT_DIR=repo.git GIT_FORCE_THREADS=1 git index-pack --threads=1 --stdin < $PACK -' - -test_perf PERF_EXTRA 'index-pack 2 threads' ' - rm -rf repo.git && - git init --bare repo.git && - GIT_DIR=repo.git git index-pack --threads=2 --stdin < $PACK -' - -test_perf PERF_EXTRA 'index-pack 4 threads' ' - rm -rf repo.git && - git init --bare repo.git && - GIT_DIR=repo.git git index-pack --threads=4 --stdin < $PACK -' - -test_perf PERF_EXTRA 'index-pack 8 threads' ' - rm -rf repo.git && - git init --bare repo.git && - GIT_DIR=repo.git git index-pack --threads=8 --stdin < $PACK -' +for t in $threads +do + THREADS=$t + export THREADS + test_perf PERF_EXTRA "index-pack $t threads" ' + rm -rf repo.git && + git init --bare repo.git && + GIT_DIR=repo.git GIT_FORCE_THREADS=1 \ + git index-pack --threads=$THREADS --stdin <$PACK + ' +done test_perf 'index-pack default number of threads' ' rm -rf repo.git &&
diff --git a/t/perf/p5302-pack-index.sh b/t/perf/p5302-pack-index.sh index 23011ab739..228593d9ad 100755 --- a/t/perf/p5302-pack-index.sh +++ b/t/perf/p5302-pack-index.sh @@ -13,35 +13,36 @@ test_expect_success 'repack' ' export PACK ' -test_perf PERF_EXTRA 'index-pack 0 threads' ' - rm -rf repo.git && - git init --bare repo.git && - GIT_DIR=repo.git git index-pack --threads=1 --stdin < $PACK +# Rather than counting up and doubling each time, count down from the endpoint, +# halving each time. That ensures that our final test uses as many threads as +# CPUs, even if it isn't a power of 2. +test_expect_success 'set up thread-counting tests' ' + t=$(test-tool online-cpus) && + threads= && + while test $t -gt 0 + do + threads="$t $threads" + t=$((t / 2)) + done ' -test_perf PERF_EXTRA 'index-pack 1 thread ' ' - rm -rf repo.git && - git init --bare repo.git && - GIT_DIR=repo.git GIT_FORCE_THREADS=1 git index-pack --threads=1 --stdin < $PACK -' - -test_perf PERF_EXTRA 'index-pack 2 threads' ' - rm -rf repo.git && - git init --bare repo.git && - GIT_DIR=repo.git git index-pack --threads=2 --stdin < $PACK -' - -test_perf PERF_EXTRA 'index-pack 4 threads' ' +test_perf PERF_EXTRA 'index-pack 0 threads' ' rm -rf repo.git && git init --bare repo.git && - GIT_DIR=repo.git git index-pack --threads=4 --stdin < $PACK + GIT_DIR=repo.git git index-pack --threads=1 --stdin < $PACK ' -test_perf PERF_EXTRA 'index-pack 8 threads' ' - rm -rf repo.git && - git init --bare repo.git && - GIT_DIR=repo.git git index-pack --threads=8 --stdin < $PACK -' +for t in $threads +do + THREADS=$t + export THREADS + test_perf PERF_EXTRA "index-pack $t threads" ' + rm -rf repo.git && + git init --bare repo.git && + GIT_DIR=repo.git GIT_FORCE_THREADS=1 \ + git index-pack --threads=$THREADS --stdin <$PACK + ' +done test_perf 'index-pack default number of threads' ' rm -rf repo.git &&
When PERF_EXTRA is enabled, p5302 checks the performance of index-pack with various numbers of threads. This can be useful for deciding what the default should be (which is currently capped at 3 threads based on the results of this script). However, we only go up to 8 threads, and modern machines may have more. Let's get the number of CPUs from test-tool, and test various numbers of threads between one and that maximum. Note that the current tests aren't all identical, as we have to set GIT_FORCE_THREADS for the --threads=1 test (which measures the overhead of starting a single worker thread versus the "0" case of using the main thread). To keep the loop simple, we'll keep the "0" case out of it, and set GIT_FORCE_THREADS=1 for all of the other cases (it's a noop for all but the "1" case, since numbers higher than 1 would always need threads). Note also that we could skip running "test-tool" if PERF_EXTRA isn't set. However, there's some small value in knowing the number of threads, so that we can mark each test as skipped in the output. Signed-off-by: Jeff King <peff@peff.net> --- t/perf/p5302-pack-index.sh | 47 +++++++++++++++++++------------------- 1 file changed, 24 insertions(+), 23 deletions(-)