From patchwork Fri Aug 25 16:43:38 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: George Dunlap X-Patchwork-Id: 9922499 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork.web.codeaurora.org (Postfix) with ESMTP id 556A06022E for ; Fri, 25 Aug 2017 16:46:26 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 45742281D2 for ; Fri, 25 Aug 2017 16:46:26 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 3A5E6283CF; Fri, 25 Aug 2017 16:46:26 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-4.2 required=2.0 tests=BAYES_00, RCVD_IN_DNSWL_MED autolearn=ham version=3.3.1 Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by mail.wl.linuxfoundation.org (Postfix) with ESMTPS id D054F281D2 for ; Fri, 25 Aug 2017 16:46:25 +0000 (UTC) Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.84_2) (envelope-from ) id 1dlHi6-0004xn-Dh; Fri, 25 Aug 2017 16:43:54 +0000 Received: from mail6.bemta6.messagelabs.com ([193.109.254.103]) by lists.xenproject.org with esmtp (Exim 4.84_2) (envelope-from ) id 1dlHi4-0004wh-Qd for xen-devel@lists.xenproject.org; Fri, 25 Aug 2017 16:43:52 +0000 Received: from [85.158.143.35] by server-6.bemta-6.messagelabs.com id 4B/21-03937-8C350A95; Fri, 25 Aug 2017 16:43:52 +0000 X-Brightmail-Tracker: H4sIAAAAAAAAA+NgFupjkeJIrShJLcpLzFFi42JxWrohUvd48IJ Ig4mXVSy+b5nM5MDocfjDFZYAxijWzLyk/IoE1oz/q+azFFyUr7jdsJ21gfGxeBcjJ4eEgL9E y4tGRhCbTUBPYt7xryxdjBwcIgIqErf3GnQxcnEwC+xnlOj9+oMNpEZYwEdi4+krYDUsAqoSq x8zgYR5BWwltrYsZoMYKS9x7sFtZpASTgE7iV0nuUBMIaCSF2f0QCqEgBoXPzjKDtEpKHFy5h MWEJtZQELi4IsXzBMYeWchSc1CklrAyLSKUaM4tagstUjX0EAvqSgzPaMkNzEzB8gz08tNLS5 OTE/NSUwq1kvOz93ECAwbBiDYwXj8fdwhRkkOJiVRXuuX8yOF+JLyUyozEosz4otKc1KLDzHK cHAoSfBOCFoQKSRYlJqeWpGWmQMMYJi0BAePkghvG0iat7ggMbc4Mx0idYpRl2PO7x1fmIRY8 vLzUqXEeUtBigRAijJK8+BGwKLpEqOslDAvI9BRQjwFqUW5mSWo8q8YxTkYlYR5/wcCTeHJzC uB2/QK6AgmoCMmnZgDckRJIkJKqoGRJ2iLVuLS3fqn4hqf2j6bzWX542DSJ+5tGaviHdobPmf VKRY9Wj6974nngzVxO72zvpUenC9sdnHmOoX4uq5r39Y3aTIdOvEspOZwY0RNZI5oHv/KqBtC iVW8b4vX87WcCOb7vya//12Z+uuCh1MnJ1l8OVD0fzrzlx0/TIuD3Ew6zh697/tJiaU4I9FQi 7moOBEASCkpw6ECAAA= X-Env-Sender: prvs=4030414b0=George.Dunlap@citrix.com X-Msg-Ref: server-11.tower-21.messagelabs.com!1503679427!76491514!4 X-Originating-IP: [66.165.176.89] X-SpamReason: No, hits=0.0 required=7.0 tests=sa_preprocessor: VHJ1c3RlZCBJUDogNjYuMTY1LjE3Ni44OSA9PiAyMDMwMDc=\n, received_headers: No Received headers X-StarScan-Received: X-StarScan-Version: 9.4.45; banners=-,-,- X-VirusChecked: Checked Received: (qmail 7188 invoked from network); 25 Aug 2017 16:43:51 -0000 Received: from smtp.citrix.com (HELO SMTP.CITRIX.COM) (66.165.176.89) by server-11.tower-21.messagelabs.com with RC4-SHA encrypted SMTP; 25 Aug 2017 16:43:51 -0000 X-IronPort-AV: E=Sophos;i="5.41,426,1498521600"; d="scan'208";a="436909236" From: George Dunlap To: Date: Fri, 25 Aug 2017 17:43:38 +0100 Message-ID: <20170825164343.29015-9-george.dunlap@citrix.com> X-Mailer: git-send-email 2.14.1 In-Reply-To: <20170825164343.29015-1-george.dunlap@citrix.com> References: <20170825164343.29015-1-george.dunlap@citrix.com> MIME-Version: 1.0 Cc: Ian Jackson , Wei Liu , George Dunlap , Jan Beulich , Andrew Cooper Subject: [Xen-devel] [PATCH 09/14] fuzz/x86_emulate: Take multiple test files for inputs X-BeenThere: xen-devel@lists.xen.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xen.org Sender: "Xen-devel" X-Virus-Scanned: ClamAV using ClamSMTP Finding aggregate coverage for a set of test files means running each afl-generated test case through the harness. At the moment, this is done by re-executing afl-harness-cov with each input file. When a large number of test cases have been generated, this can take a significant amonut of time; a recent test with 30k total files generated by 4 parallel fuzzers took over 7 minutes. The vast majority of this time is taken up with 'exec', however. Since the harness is already designed to loop over multiple inputs for llvm "persistent mode", just allow it to take a large number of inputs on the same when *not* running in llvm "persistent mode".. Then the command can be efficiently executed like this: ls */queue/id* | xargs $path/afl-harness-cov For the above-mentioned test on 30k files, the time to generate coverage data was reduced from 7 minutes to under 30 seconds. Signed-off-by: George Dunlap --- CC: Ian Jackson CC: Wei Liu CC: Andrew Cooper CC: Jan Beulich --- tools/fuzz/README.afl | 7 +++++++ tools/fuzz/x86_instruction_emulator/afl-harness.c | 23 ++++++++++++++++------- 2 files changed, 23 insertions(+), 7 deletions(-) diff --git a/tools/fuzz/README.afl b/tools/fuzz/README.afl index 0d955b2687..e8c23d734c 100644 --- a/tools/fuzz/README.afl +++ b/tools/fuzz/README.afl @@ -49,6 +49,13 @@ generate coverage data. To do this, use the target `afl-cov`: $ make afl-cov #produces afl-harness-cov +In order to speed up the process of checking total coverage, +`afl-harness-cov` can take several test inputs on its command-line; +the speed-up effect should be similar to that of using afl-clang-fast. +You can use xargs to do this most efficiently, like so: + + $ ls queue/id* | xargs $path/afl-harness-cov + NOTE: Please also note that the coverage instrumentation hard-codes the absolute path for the instrumentation read and write files in the binary; so coverage data will always show up in the build directory no diff --git a/tools/fuzz/x86_instruction_emulator/afl-harness.c b/tools/fuzz/x86_instruction_emulator/afl-harness.c index 51e0183356..79f8aec653 100644 --- a/tools/fuzz/x86_instruction_emulator/afl-harness.c +++ b/tools/fuzz/x86_instruction_emulator/afl-harness.c @@ -16,6 +16,8 @@ int main(int argc, char **argv) { size_t size; FILE *fp = NULL; + int count = 0; + int max; setbuf(stdin, NULL); setbuf(stdout, NULL); @@ -42,8 +44,7 @@ int main(int argc, char **argv) break; case '?': - usage: - printf("Usage: %s $FILE | [--min-input-size]\n", argv[0]); + printf("Usage: %s $FILE [$FILE...] | [--min-input-size]\n", argv[0]); exit(-1); break; @@ -54,21 +55,27 @@ int main(int argc, char **argv) } } - if ( optind == argc ) /* No positional parameters. Use stdin. */ + max = argc - optind; + + if ( !max ) /* No positional parameters. Use stdin. */ + { + max = 1; fp = stdin; - else if ( optind != (argc - 1) ) - goto usage; + } if ( LLVMFuzzerInitialize(&argc, &argv) ) exit(-1); #ifdef __AFL_HAVE_MANUAL_CONTROL while ( __AFL_LOOP(1000) ) +#else + for( count = 0; count < max; count++ ) #endif { if ( fp != stdin ) /* If not using stdin, open the provided file. */ { - fp = fopen(argv[optind], "rb"); + printf("Opening file %s\n", argv[optind]); + fp = fopen(argv[optind + count], "rb"); if ( fp == NULL ) { perror("fopen"); @@ -87,7 +94,9 @@ int main(int argc, char **argv) if ( !feof(fp) || size > INPUT_SIZE ) { printf("Input too large\n"); - exit(-1); + if ( optind + 1 == argc ) + exit(-1); + continue; } if ( fp != stdin )