From patchwork Tue Oct 10 16:20:11 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: George Dunlap X-Patchwork-Id: 9997937 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork.web.codeaurora.org (Postfix) with ESMTP id 5AFCD601AE for ; Tue, 10 Oct 2017 21:35:41 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 4BAC728705 for ; Tue, 10 Oct 2017 21:35:41 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 3ECD5287BE; Tue, 10 Oct 2017 21:35:41 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-2.6 required=2.0 tests=BAYES_00, DATE_IN_PAST_03_06, RCVD_IN_DNSWL_MED autolearn=ham version=3.3.1 Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by mail.wl.linuxfoundation.org (Postfix) with ESMTPS id 7E7EC287BB for ; Tue, 10 Oct 2017 21:35:40 +0000 (UTC) Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.84_2) (envelope-from ) id 1e229T-0000xV-RG; Tue, 10 Oct 2017 21:33:23 +0000 Received: from mail6.bemta6.messagelabs.com ([193.109.254.103]) by lists.xenproject.org with esmtp (Exim 4.84_2) (envelope-from ) id 1e229S-0000xA-Ov for xen-devel@lists.xenproject.org; Tue, 10 Oct 2017 21:33:22 +0000 Received: from [193.109.254.147] by server-4.bemta-6.messagelabs.com id 58/CD-31244-2AC3DD95; Tue, 10 Oct 2017 21:33:22 +0000 X-Brightmail-Tracker: H4sIAAAAAAAAA+NgFprOIsWRWlGSWpSXmKPExsXitHSDve5Cm7u RBituyVl83zKZyYHR4/CHKywBjFGsmXlJ+RUJrBkndzWxF1zRrNj9eBtbA+NxxS5GTg4JAX+J vfc2soDYbAJ6EvOOfwWyOThEBFQkbu816GLk4mAW2M8o0fv1BxtIjbBAmsT/7Z3sIDaLgKrEr tsTmEBsXgFbiXf31rJAzJSXeL/gPiOIzQkU332uF6xeSMBGYlv7ZCYIW1Vi8YOj7BC9ghInZz 4B62UWkJA4+OIF8wRG3llIUrOQpBYwMq1i1ChOLSpLLdI1NNVLKspMzyjJTczM0TU0MNPLTS0 uTkxPzUlMKtZLzs/dxAgMHgYg2MH4bVnAIUZJDiYlUd7X6ncjhfiS8lMqMxKLM+KLSnNSiw8x ynBwKEnwPrMGygkWpaanVqRl5gDDGCYtwcGjJMI7CyTNW1yQmFucmQ6ROsVozHHoxe0/TBwdN +/+YRJiycvPS5US560FKRUAKc0ozYMbBIuvS4yyUsK8jECnCfEUpBblZpagyr9iFOdgVBLmnQ cyhSczrwRu3yugU5iAThFNuwNySkkiQkqqgZGvwjj5pqJpT51c7GnJ3YrtNpmVa2Y+1Z0kuXq 67VKP28XCTwPnJ0sVHBXdtdan2+WT3jelbnbTnw7MtW1bDvG08dfVPlj2nLP3Du+apidJDdtm zvNasOtYZLb5js/d2vlX2bxuFe9usVA7c7G6M2b17IN2+xTl1nv0367JZwrky7kcrpa/SImlO CPRUIu5qDgRALpXx6aqAgAA X-Env-Sender: prvs=4495d6a26=George.Dunlap@citrix.com X-Msg-Ref: server-4.tower-27.messagelabs.com!1507671198!110258899!2 X-Originating-IP: [66.165.176.63] X-SpamReason: No, hits=0.0 required=7.0 tests=sa_preprocessor: VHJ1c3RlZCBJUDogNjYuMTY1LjE3Ni42MyA9PiAzMDYwNDg=\n, received_headers: No Received headers X-StarScan-Received: X-StarScan-Version: 9.4.45; banners=-,-,- X-VirusChecked: Checked Received: (qmail 8283 invoked from network); 10 Oct 2017 21:33:21 -0000 Received: from smtp02.citrix.com (HELO SMTP02.CITRIX.COM) (66.165.176.63) by server-4.tower-27.messagelabs.com with RC4-SHA encrypted SMTP; 10 Oct 2017 21:33:21 -0000 X-IronPort-AV: E=Sophos;i="5.43,359,1503360000"; d="scan'208";a="452872782" From: George Dunlap To: Date: Tue, 10 Oct 2017 17:20:11 +0100 Message-ID: <20171010162011.9629-12-george.dunlap@citrix.com> X-Mailer: git-send-email 2.14.2 In-Reply-To: <20171010162011.9629-1-george.dunlap@citrix.com> References: <20171010162011.9629-1-george.dunlap@citrix.com> MIME-Version: 1.0 Cc: Ian Jackson , Wei Liu , George Dunlap , Jan Beulich , Andrew Cooper Subject: [Xen-devel] [PATCH v3 12/12] fuzz/x86_emulate: Add an option to limit the number of instructions executed X-BeenThere: xen-devel@lists.xen.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xen.org Sender: "Xen-devel" X-Virus-Scanned: ClamAV using ClamSMTP AFL considers a testcase to be a useful addition not only if there are tuples exercised by that testcase which were not exercised otherwise, but also if the *number* of times an individual tuple is exercised changes significantly; in particular, if the number of the highest non-zero bit changes (i.e., if it is run 1, 2-3, 4-7, 8-15, &c). One simple way to increase these stats it to execute the same (or similar) instructions multiple times: If executing a given instruction once hits a particular tuple 2 times, executing it twice will hit the tuple 4 times, four times will hit the tuple 8 times, and so on. All of these will look different to AFL, and so it is likely that many of the "unique test cases" will simply be the same instruction repeated powers of 2 times until the tuple counts max out (at 128). It is unlikely that executing a single instruction more than a handful of times will generate any state we actually care about; but such long testcases take exponentially longer to fuzz: the fuzzer spends more time flipping bits looking for meaningful changes, and each execution takes longer because it is doing more things. So long paths which add nothing to the actual code coverage but effectively "distract" the fuzzer, making it less effective. Experiments have shown that not allowing infinite number of instruction retries for the old (non-compact) format does indeed speed up and increase code coverage. However, it has also shown that on the new, more compact format, having no instruction limit causes the highest throughput in code coverage. So leave the option in, but have it default to 0 (no limit). Signed-off-by: George Dunlap Acked-by: Jan Beulich --- v3: - Change opt_instruction_limit to unsigned, default to UINT_MAX - Simplify limit checking (now that the actual variable itself will never be 0) - Change counter to unsigned - Update changelog to try to be a bit more clear CC: Ian Jackson CC: Wei Liu CC: Andrew Cooper CC: Jan Beulich --- tools/fuzz/x86_instruction_emulator/afl-harness.c | 11 ++++++++++- tools/fuzz/x86_instruction_emulator/fuzz-emul.c | 5 ++++- tools/fuzz/x86_instruction_emulator/fuzz-emul.h | 1 + 3 files changed, 15 insertions(+), 2 deletions(-) diff --git a/tools/fuzz/x86_instruction_emulator/afl-harness.c b/tools/fuzz/x86_instruction_emulator/afl-harness.c index 4a55ac3c3f..7d09cc29c6 100644 --- a/tools/fuzz/x86_instruction_emulator/afl-harness.c +++ b/tools/fuzz/x86_instruction_emulator/afl-harness.c @@ -1,4 +1,5 @@ #include +#include #include #include #include @@ -24,11 +25,13 @@ int main(int argc, char **argv) OPT_MIN_SIZE, OPT_COMPACT, OPT_RERUN, + OPT_INSTRUCTION_LIMIT, }; static const struct option lopts[] = { { "min-input-size", no_argument, NULL, OPT_MIN_SIZE }, { "compact", required_argument, NULL, OPT_COMPACT }, { "rerun", no_argument, NULL, OPT_RERUN }, + { "instruction-limit", required_argument, NULL, OPT_INSTRUCTION_LIMIT }, { 0, 0, 0, 0 } }; int c = getopt_long_only(argc, argv, "", lopts, NULL); @@ -51,8 +54,14 @@ int main(int argc, char **argv) opt_rerun = true; break; + case OPT_INSTRUCTION_LIMIT: + opt_instruction_limit = atoi(optarg); + if ( !opt_instruction_limit ) + opt_instruction_limit = UINT_MAX; + break; + case '?': - printf("Usage: %s [--compact=0|1] [--rerun] $FILE [$FILE...] | [--min-input-size]\n", argv[0]); + printf("Usage: %s [--compact=0|1] [--rerun] [--instruction-limit=N] $FILE [$FILE...] | [--min-input-size]\n", argv[0]); exit(-1); break; diff --git a/tools/fuzz/x86_instruction_emulator/fuzz-emul.c b/tools/fuzz/x86_instruction_emulator/fuzz-emul.c index 79dd36ec30..8aaec93973 100644 --- a/tools/fuzz/x86_instruction_emulator/fuzz-emul.c +++ b/tools/fuzz/x86_instruction_emulator/fuzz-emul.c @@ -969,10 +969,13 @@ static void setup_fuzz_state(struct fuzz_state *state, const void *data_p, size_ state->data_num = size; } +unsigned int opt_instruction_limit = UINT_MAX; + static int runtest(struct fuzz_state *state) { int rc; struct x86_emulate_ctxt *ctxt = &state->ctxt; + unsigned int icount = 0; state->ops = all_fuzzer_ops; @@ -996,7 +999,7 @@ static int runtest(struct fuzz_state *state) { rc = x86_emulate(ctxt, &state->ops); printf("Emulation result: %d\n", rc); - } while ( rc == X86EMUL_OKAY ); + } while ( rc == X86EMUL_OKAY && ++icount < opt_instruction_limit ); save_fpu_state(state->fxsave); diff --git a/tools/fuzz/x86_instruction_emulator/fuzz-emul.h b/tools/fuzz/x86_instruction_emulator/fuzz-emul.h index 4863bf2166..746e1b542d 100644 --- a/tools/fuzz/x86_instruction_emulator/fuzz-emul.h +++ b/tools/fuzz/x86_instruction_emulator/fuzz-emul.h @@ -7,6 +7,7 @@ extern unsigned int fuzz_minimal_input_size(void); extern bool opt_compact; extern bool opt_rerun; +extern unsigned int opt_instruction_limit; #define INPUT_SIZE 4096