From patchwork Wed Oct 11 17:52:37 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: George Dunlap X-Patchwork-Id: 10000259 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork.web.codeaurora.org (Postfix) with ESMTP id 8309960244 for ; Wed, 11 Oct 2017 17:55:29 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 7875428ADE for ; Wed, 11 Oct 2017 17:55:29 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 6D10728B03; Wed, 11 Oct 2017 17:55:29 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-4.2 required=2.0 tests=BAYES_00, RCVD_IN_DNSWL_MED autolearn=ham version=3.3.1 Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by mail.wl.linuxfoundation.org (Postfix) with ESMTPS id 0843D28ADE for ; Wed, 11 Oct 2017 17:55:29 +0000 (UTC) Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.84_2) (envelope-from ) id 1e2LBg-0000a3-AJ; Wed, 11 Oct 2017 17:52:56 +0000 Received: from mail6.bemta5.messagelabs.com ([195.245.231.135]) by lists.xenproject.org with esmtp (Exim 4.84_2) (envelope-from ) id 1e2LBe-0000Yl-UO for xen-devel@lists.xenproject.org; Wed, 11 Oct 2017 17:52:55 +0000 Received: from [85.158.139.211] by server-3.bemta-5.messagelabs.com id 37/F1-02748-67A5ED95; Wed, 11 Oct 2017 17:52:54 +0000 X-Brightmail-Tracker: H4sIAAAAAAAAA+NgFprGIsWRWlGSWpSXmKPExsXitHRDpG5p1L1 Ig9u9chbft0xmcmD0OPzhCksAYxRrZl5SfkUCa8adQ5PYCnpVKra8S2hg3CLdxcjJISHgL7Hi xU5GEJtNQE9i3vGvLF2MHBwiAioSt/cadDFycTAL7GeU6P36gw2kRhio/t3D90wgNouAqsSSn jawXl4BW4lN676zQ8yUl3i/4D5YnFPATuLN4/usILYQUM2KpXsZIWxVicUPjrJD9ApKnJz5hA XEZhaQkDj44gXzBEbeWUhSs5CkFjAyrWLUKE4tKkst0jUy0ksqykzPKMlNzMzRNTQw1ctNLS5 OTE/NSUwq1kvOz93ECAydegYGxh2Me9r9DjFKcjApifIG692LFOJLyk+pzEgszogvKs1JLT7E KMPBoSTBqxgJlBMsSk1PrUjLzAEGMUxagoNHSYS3HyTNW1yQmFucmQ6ROsVozHHoxe0/TBwdN +/+YRJiycvPS5US520DKRUAKc0ozYMbBIuuS4yyUsK8jAwMDEI8BalFuZklqPKvGMU5GJWEef 9HAE3hycwrgdv3CugUJqBTRNPugJxSkoiQkmpglPZ6x1PrcU9B5/qqgKlXTXev+sDm01vUYdX q6PcuQHay4JJDz50ZzxcbrVsUIvdQ6ZauVk9LEZvurJdbp/nZrzw5mWvHMXkv46XW7htyvqgW rkjL6hfkbqm1nWJfHuFxLnPlvs1rJZ7NeSE52fzx3Y1LN2ldWePg9mHx7l9PcqY+S6pnzYvkV GIpzkg01GIuKk4EAPgmK3SpAgAA X-Env-Sender: prvs=450528267=George.Dunlap@citrix.com X-Msg-Ref: server-8.tower-206.messagelabs.com!1507744369!106410454!3 X-Originating-IP: [66.165.176.89] X-SpamReason: No, hits=0.0 required=7.0 tests=sa_preprocessor: VHJ1c3RlZCBJUDogNjYuMTY1LjE3Ni44OSA9PiAyMDMwMDc=\n, received_headers: No Received headers X-StarScan-Received: X-StarScan-Version: 9.4.45; banners=-,-,- X-VirusChecked: Checked Received: (qmail 27592 invoked from network); 11 Oct 2017 17:52:52 -0000 Received: from smtp.citrix.com (HELO SMTP.CITRIX.COM) (66.165.176.89) by server-8.tower-206.messagelabs.com with RC4-SHA encrypted SMTP; 11 Oct 2017 17:52:52 -0000 X-IronPort-AV: E=Sophos;i="5.43,362,1503360000"; d="scan'208";a="445492265" From: George Dunlap To: Date: Wed, 11 Oct 2017 18:52:37 +0100 Message-ID: <20171011175243.19871-6-george.dunlap@citrix.com> X-Mailer: git-send-email 2.14.2 In-Reply-To: <20171011175243.19871-1-george.dunlap@citrix.com> References: <20171011175243.19871-1-george.dunlap@citrix.com> MIME-Version: 1.0 Cc: Ian Jackson , Wei Liu , George Dunlap , Jan Beulich , Andrew Cooper Subject: [Xen-devel] [PATCH v4 06/12] fuzz/x86_emulate: Take multiple test files for inputs X-BeenThere: xen-devel@lists.xen.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xen.org Sender: "Xen-devel" X-Virus-Scanned: ClamAV using ClamSMTP Finding aggregate coverage for a set of test files means running each afl-generated test case through the harness. At the moment, this is done by re-executing afl-harness-cov with each input file. When a large number of test cases have been generated, this can take a significant amonut of time; a recent test with 30k total files generated by 4 parallel fuzzers took over 7 minutes. The vast majority of this time is taken up with 'exec', however. Since the harness is already designed to loop over multiple inputs for llvm "persistent mode", just allow it to take a large number of inputs on the same when *not* running in llvm "persistent mode".. Then the command can be efficiently executed like this: ls */queue/id* | xargs $path/afl-harness-cov For the above-mentioned test on 30k files, the time to generate coverage data was reduced from 7 minutes to under 30 seconds. Signed-off-by: George Dunlap Acked-by: Jan Beulich Acked-by: Andrew Cooper --- v4: - Fix printf to print the right filename v3: - Combine some variable declarations - Make sure that count is set only once no matter how it's compiled v2: - Make check for batch processing more clear Jan: I took the liberty of retaining your Ack on this, since it was a simple and obvious fix (which I think you had also suggested). CC: Ian Jackson CC: Wei Liu CC: Andrew Cooper CC: Jan Beulich --- tools/fuzz/README.afl | 7 +++++++ tools/fuzz/x86_instruction_emulator/afl-harness.c | 25 +++++++++++++++-------- 2 files changed, 24 insertions(+), 8 deletions(-) diff --git a/tools/fuzz/README.afl b/tools/fuzz/README.afl index 8b58b8cdea..a59564985a 100644 --- a/tools/fuzz/README.afl +++ b/tools/fuzz/README.afl @@ -49,6 +49,13 @@ generate coverage data. To do this, use the target `afl-cov`: $ make afl-cov #produces afl-harness-cov +In order to speed up the process of checking total coverage, +`afl-harness-cov` can take several test inputs on its command-line; +the speed-up effect should be similar to that of using afl-clang-fast. +You can use xargs to do this most efficiently, like so: + + $ ls queue/id* | xargs $path/afl-harness-cov + NOTE: Please also note that the coverage instrumentation hard-codes the absolute path for the instrumentation read and write files in the binary; so coverage data will always show up in the build directory no diff --git a/tools/fuzz/x86_instruction_emulator/afl-harness.c b/tools/fuzz/x86_instruction_emulator/afl-harness.c index 31ae1daef1..e0c56aadf7 100644 --- a/tools/fuzz/x86_instruction_emulator/afl-harness.c +++ b/tools/fuzz/x86_instruction_emulator/afl-harness.c @@ -16,6 +16,7 @@ int main(int argc, char **argv) { size_t size; FILE *fp = NULL; + int max, count; setbuf(stdin, NULL); setbuf(stdout, NULL); @@ -42,8 +43,7 @@ int main(int argc, char **argv) break; case '?': - usage: - printf("Usage: %s $FILE | [--min-input-size]\n", argv[0]); + printf("Usage: %s $FILE [$FILE...] | [--min-input-size]\n", argv[0]); exit(-1); break; @@ -54,10 +54,13 @@ int main(int argc, char **argv) } } - if ( optind == argc ) /* No positional parameters. Use stdin. */ + max = argc - optind; + + if ( !max ) /* No positional parameters. Use stdin. */ + { + max = 1; fp = stdin; - else if ( optind != (argc - 1) ) - goto usage; + } if ( LLVMFuzzerInitialize(&argc, &argv) ) exit(-1); @@ -65,12 +68,15 @@ int main(int argc, char **argv) #ifdef __AFL_HAVE_MANUAL_CONTROL __AFL_INIT(); - while ( __AFL_LOOP(1000) ) + for( count = 0; __AFL_LOOP(1000); ) +#else + for( count = 0; count < max; count++ ) #endif { if ( fp != stdin ) /* If not using stdin, open the provided file. */ { - fp = fopen(argv[optind], "rb"); + printf("Opening file %s\n", argv[optind + count]); + fp = fopen(argv[optind + count], "rb"); if ( fp == NULL ) { perror("fopen"); @@ -100,7 +106,10 @@ int main(int argc, char **argv) if ( !feof(fp) ) { printf("Input too large\n"); - exit(-1); + /* Don't exit if we're doing batch processing */ + if ( max == 1 ) + exit(-1); + continue; } if ( fp != stdin )