From patchwork Wed Feb 9 22:18:46 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Luis Chamberlain X-Patchwork-Id: 12740980 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 35FB7C433F5 for ; Wed, 9 Feb 2022 22:18:56 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232268AbiBIWSv (ORCPT ); Wed, 9 Feb 2022 17:18:51 -0500 Received: from gmail-smtp-in.l.google.com ([23.128.96.19]:44750 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S235224AbiBIWSv (ORCPT ); Wed, 9 Feb 2022 17:18:51 -0500 Received: from bombadil.infradead.org (bombadil.infradead.org [IPv6:2607:7c80:54:e::133]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id B7006C1DC5D1 for ; Wed, 9 Feb 2022 14:18:53 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=bombadil.20210309; h=Sender:Content-Transfer-Encoding: MIME-Version:References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From: Reply-To:Content-Type:Content-ID:Content-Description; bh=exJ/KGoqZkK8mhMXqxR7kAJ/ddgoBl0DHSmx/sevlYg=; b=Xe06hZP4GFMBK+IJRgXnjJHJil CUvmtytR+5igahuJRtf+2BK37z7+M9On5oVtgAjOlp7cpRyLDH0D6r+9Xzk3nCWyQz20HmoiZINmP 4/JYCbQX0CX4+804AbcXOEMz3Z7V0itePuZGuOMor0LRj0c+JZj0K7Br40SgKpla8pHdY9KXfpRFy E+M2aks5KPjKjSz1q1F44UGbFnxRYmlkONfn2XZ9zuOCR05sUepj9GeKteXvc7KgXWjZt/AYlxhKZ 4k6wA9boqhmYhAmfuL3KLQsmbE456OMLw9TcstBmc9/r9IdHa0PpfZ7p64OBZp6aaAOFMwjLWx5Cw hePLQihQ==; Received: from mcgrof by bombadil.infradead.org with local (Exim 4.94.2 #2 (Red Hat Linux)) id 1nHvIY-001p4h-Mh; Wed, 09 Feb 2022 22:18:52 +0000 From: Luis Chamberlain To: raymond.barbiero.dev@gmail.com Cc: fstests@vger.kernel.org, jack@suse.cz, mgorman@techsingularity.net, dave@stgolabs.net, Luis Chamberlain Subject: [PATCH 1/4] Allow reporting of workload execution times Date: Wed, 9 Feb 2022 14:18:46 -0800 Message-Id: <20220209221849.434616-2-mcgrof@kernel.org> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20220209221849.434616-1-mcgrof@kernel.org> References: <20220209221849.434616-1-mcgrof@kernel.org> MIME-Version: 1.0 Sender: Luis Chamberlain Precedence: bulk List-ID: X-Mailing-List: fstests@vger.kernel.org From: Jan Kara Add option --show-execution-time which reports time it took to each client to execute the given workload. This allows for better statistics to be done with dbench results. Signed-off-by: Jan Kara Signed-off-by: Luis Chamberlain --- child.c | 18 ++++++++++++++++++ dbench.c | 3 +++ dbench.h | 1 + 3 files changed, 22 insertions(+) diff --git a/child.c b/child.c index aea8106..e4ae230 100644 --- a/child.c +++ b/child.c @@ -326,6 +326,7 @@ void child_run(struct child_struct *child0, const char *loadfile) int have_random = 0; unsigned loop_count = 0; z_off_t loop_start = 0; + struct timeval start; gzf = gzopen(loadfile, "r"); if (gzf == NULL) { @@ -349,6 +350,8 @@ again: nb_time_reset(child); } + gettimeofday(&start, NULL); + while (gzgets(gzf, line, sizeof(line)-1)) { unsigned repeat_count = 1; @@ -529,6 +532,21 @@ loop_again: } } + if (options.show_execute_time) { + struct timeval end; + unsigned int duration; + + gettimeofday(&end, NULL); + duration = (end.tv_sec - start.tv_sec) * 1000 + + (end.tv_usec - start.tv_usec) / 1000; + if (options.machine_readable) + printf("@E@%d@%u\n", child0->id, duration); + else { + printf("%4d completed in %u ms\n", child0->id, + duration); + } + } + if (options.run_once) { goto done; } diff --git a/dbench.c b/dbench.c index 1369a38..178a175 100644 --- a/dbench.c +++ b/dbench.c @@ -50,6 +50,7 @@ struct options options = { .trunc_io = 0, .iscsi_initiatorname = "iqn.2011-09.org.samba.dbench:client", .machine_readable = 0, + .show_execute_time = 0, }; static struct timeval tv_start; @@ -433,6 +434,8 @@ static void process_opts(int argc, const char **argv) "How many seconds of warmup to run", NULL }, { "machine-readable", 0, POPT_ARG_NONE, &options.machine_readable, 0, "Print data in more machine-readable friendly format", NULL}, + { "show-execute-time", 0, POPT_ARG_NONE, &options.show_execute_time, 0, + "Print time to execute passed workload", NULL}, #ifdef HAVE_LIBSMBCLIENT { "smb-share", 0, POPT_ARG_STRING, &options.smb_share, 0, "//SERVER/SHARE to use", NULL }, diff --git a/dbench.h b/dbench.h index 14a5a70..465cf3b 100644 --- a/dbench.h +++ b/dbench.h @@ -159,6 +159,7 @@ struct options { const char *iscsi_device; const char *iscsi_initiatorname; int machine_readable; + int show_execute_time; const char *smb_share; const char *smb_user; }; From patchwork Wed Feb 9 22:18:47 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Luis Chamberlain X-Patchwork-Id: 12740982 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 0619AC43217 for ; Wed, 9 Feb 2022 22:18:58 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S235284AbiBIWSw (ORCPT ); Wed, 9 Feb 2022 17:18:52 -0500 Received: from gmail-smtp-in.l.google.com ([23.128.96.19]:44762 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S235265AbiBIWSv (ORCPT ); Wed, 9 Feb 2022 17:18:51 -0500 Received: from bombadil.infradead.org (bombadil.infradead.org [IPv6:2607:7c80:54:e::133]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id B6624C03BFF4 for ; Wed, 9 Feb 2022 14:18:53 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=bombadil.20210309; h=Sender:Content-Transfer-Encoding: MIME-Version:References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From: Reply-To:Content-Type:Content-ID:Content-Description; bh=d4z7KruYGznXYVz9GGxnpEiYpVxIrkvo22KUrQiAN/k=; b=pnmjkd3H5wre10SLX3z3uuksuX cHsXBTbD17tkW35f/LVsVt+xdiWByRFDIYOoIEMhbDiPSl9QJuJKBhhxeKHyjhva/BpB5aIARKxfL 5SkeJsntAy4amC1pmSZ2x4qwFtkrdkms/MAlRBriEgWES3olPjM38lRuHgAh7fjVJ+OtHKMnSidQq P1sz2QI0DRrGkAZ67kkNYUSTAObAyICWenCYASBZ0Xna1axMeVojjmTrfP1LyX7bOiBue+1DU2EdH KxTUVA88a8fPKSw4KIc+/pQthUnqRlbr66dGYD04cAccryEsTKqnURdJ+6PSdyJQ8ezs3DS0N3ukv 45BhNlTw==; Received: from mcgrof by bombadil.infradead.org with local (Exim 4.94.2 #2 (Red Hat Linux)) id 1nHvIa-001p4l-Ef; Wed, 09 Feb 2022 22:18:52 +0000 From: Luis Chamberlain To: raymond.barbiero.dev@gmail.com Cc: fstests@vger.kernel.org, jack@suse.cz, mgorman@techsingularity.net, dave@stgolabs.net, Luis Chamberlain Subject: [PATCH 2/4] Defer reporting of execution times Date: Wed, 9 Feb 2022 14:18:47 -0800 Message-Id: <20220209221849.434616-3-mcgrof@kernel.org> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20220209221849.434616-1-mcgrof@kernel.org> References: <20220209221849.434616-1-mcgrof@kernel.org> MIME-Version: 1.0 Sender: Luis Chamberlain Precedence: bulk List-ID: X-Mailing-List: fstests@vger.kernel.org From: Mel Gorman If loadfiles are completed rapidly, there is a large amount of data sent to stddout and then recorded which generates IO in itself. This patch buffers durations and runtimes for a time. In some cases, it'll be buffered until the end of the benchmark. Signed-off-by: Mel Gorman Signed-off-by: Luis Chamberlain --- child.c | 38 ++++++++++++++++++++++++++++++++++---- 1 file changed, 34 insertions(+), 4 deletions(-) diff --git a/child.c b/child.c index e4ae230..828295d 100644 --- a/child.c +++ b/child.c @@ -309,9 +309,19 @@ finished: return 0; } +void dump_samples(int id, unsigned int *duration, unsigned int *runtime, unsigned int nr_samples) +{ + unsigned int i; + + for (i = 0; i < nr_samples; i++) { + printf("%4d completed in %u ms at time %u ms\n", id, + duration[i], runtime[i]); + } +} /* run a test that simulates an approximate netbench client load */ #define MAX_PARM_LEN 1024 +#define MAX_SAMPLES 130672 void child_run(struct child_struct *child0, const char *loadfile) { int i; @@ -326,7 +336,16 @@ void child_run(struct child_struct *child0, const char *loadfile) int have_random = 0; unsigned loop_count = 0; z_off_t loop_start = 0; - struct timeval start; + struct timeval start, begin; + unsigned nr_samples = 0; + + unsigned int *sample_duration = malloc(sizeof(unsigned int) * (MAX_SAMPLES + 1)); + unsigned int *sample_runtime = malloc(sizeof(unsigned int) * (MAX_SAMPLES + 1)); + + if (!sample_duration || !sample_runtime) { + printf("ENOMEM for samples\n"); + exit(1); + } gzf = gzopen(loadfile, "r"); if (gzf == NULL) { @@ -345,6 +364,8 @@ void child_run(struct child_struct *child0, const char *loadfile) memset(sparams[i], 0, MAX_PARM_LEN); } + gettimeofday(&begin, NULL); + again: for (child=child0;childid, duration); else { - printf("%4d completed in %u ms\n", child0->id, - duration); + sample_duration[nr_samples] = duration; + sample_runtime[nr_samples] = runtime; + nr_samples++; + if (nr_samples == MAX_SAMPLES) { + dump_samples(child0->id, sample_duration, sample_runtime, nr_samples); + nr_samples = 0; + } } } @@ -556,6 +584,8 @@ loop_again: done: gzclose(gzf); + usleep(child0->id * 5000); + dump_samples(child0->id, sample_duration, sample_runtime, nr_samples); for (child=child0;childcleanup = 1; fflush(stdout); From patchwork Wed Feb 9 22:18:48 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Luis Chamberlain X-Patchwork-Id: 12740981 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id C054DC4332F for ; Wed, 9 Feb 2022 22:18:56 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S235291AbiBIWSw (ORCPT ); Wed, 9 Feb 2022 17:18:52 -0500 Received: from gmail-smtp-in.l.google.com ([23.128.96.19]:44764 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S235284AbiBIWSv (ORCPT ); Wed, 9 Feb 2022 17:18:51 -0500 Received: from bombadil.infradead.org (bombadil.infradead.org [IPv6:2607:7c80:54:e::133]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id B6AF1C102FD8 for ; Wed, 9 Feb 2022 14:18:53 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=bombadil.20210309; h=Sender:Content-Transfer-Encoding: MIME-Version:References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From: Reply-To:Content-Type:Content-ID:Content-Description; bh=nZ4rv4B2BEaTEpMULbqseHpbts1amhVOc2IxOGa6Xcc=; b=WTcn7FueIB2f9WrMWLA8VBtgKR YR8cP1crx0QX1wUj46MBiT+Ez6zGn39AHxWIxv+m6f5SkodLiOTclS1m9JjVbahrDhB2dDAcU8WO8 XNQYPbjw2m76xwLljXS36jPJHRjUzF7In/iHgvxABBKWUTvrNCvRJ2fKa9o14F8GvGyzXaRdFNLLc LBTeRH4tyK1Xcruyk1DLYLmHEyYEEX/8lxhe4MGF+iCFEzop3TUd2NHKyccD/M6kI4n3Ku0OfN0eQ 1GOTXvAj3hsEnGt/dQBMKVKeg7H5t6zzKHb6R/rWcjdRQ1c3U2swB8N7hGZDP6xhcbeCBwda9b2JF NzJWXgfg==; Received: from mcgrof by bombadil.infradead.org with local (Exim 4.94.2 #2 (Red Hat Linux)) id 1nHvIa-001p4p-Fq; Wed, 09 Feb 2022 22:18:52 +0000 From: Luis Chamberlain To: raymond.barbiero.dev@gmail.com Cc: fstests@vger.kernel.org, jack@suse.cz, mgorman@techsingularity.net, dave@stgolabs.net, Luis Chamberlain Subject: [PATCH 3/4] libnfs: Include stdint.h Date: Wed, 9 Feb 2022 14:18:48 -0800 Message-Id: <20220209221849.434616-4-mcgrof@kernel.org> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20220209221849.434616-1-mcgrof@kernel.org> References: <20220209221849.434616-1-mcgrof@kernel.org> MIME-Version: 1.0 Sender: Luis Chamberlain Precedence: bulk List-ID: X-Mailing-List: fstests@vger.kernel.org From: Mel Gorman Changes in kernel headers break build unless this is included. Signed-off-by: Mel Gorman Signed-off-by: Luis Chamberlain --- libnfs.c | 1 + 1 file changed, 1 insertion(+) diff --git a/libnfs.c b/libnfs.c index edbcd61..e853e47 100644 --- a/libnfs.c +++ b/libnfs.c @@ -21,6 +21,7 @@ #include "mount.h" #include "nfs.h" #include "libnfs.h" +#include #include #include #include From patchwork Wed Feb 9 22:18:49 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Luis Chamberlain X-Patchwork-Id: 12740983 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 4AEF7C433FE for ; Wed, 9 Feb 2022 22:18:58 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S235224AbiBIWSx (ORCPT ); Wed, 9 Feb 2022 17:18:53 -0500 Received: from gmail-smtp-in.l.google.com ([23.128.96.19]:44766 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S235269AbiBIWSv (ORCPT ); Wed, 9 Feb 2022 17:18:51 -0500 Received: from bombadil.infradead.org (bombadil.infradead.org [IPv6:2607:7c80:54:e::133]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id B6EDAC1DC5C6 for ; Wed, 9 Feb 2022 14:18:53 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=bombadil.20210309; h=Sender:Content-Transfer-Encoding: MIME-Version:References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From: Reply-To:Content-Type:Content-ID:Content-Description; bh=rUFlLx/XoXG7etYfjYGzpR75JZOkIMQuhIW5+RifPGc=; b=mglxgUQ1blPtzFhFlKkbKfZwFK L0As875Bxm49HQOELstAAGBJCVwEHhhAYwyw9y4y957DV0UAM6DW4DlmEgRD3ideIwz9KXIhx7FOu KlCWAg2Qnfh5GBNCgceSqwN2pipb7FRrFP5yNuBEVtiDuqz+2xQZH5SuI4TyEkvVvszviU3Ev230y A7UxrA9gwH6KHAyZD+DBwr4eoF78Zd4aHIjzmZ1TUGz3kQFHHsXQGgephvAGF0VO1b5op3y5faw93 f1n1apTBWBkgEvj5exqKSCdB2J7aJedg1VopAARHbNUJ9A3Jx9aQ+UWsdi59vSJvfPj1zJMTBunTE 46Truhvg==; Received: from mcgrof by bombadil.infradead.org with local (Exim 4.94.2 #2 (Red Hat Linux)) id 1nHvIa-001p4t-Gn; Wed, 09 Feb 2022 22:18:52 +0000 From: Luis Chamberlain To: raymond.barbiero.dev@gmail.com Cc: fstests@vger.kernel.org, jack@suse.cz, mgorman@techsingularity.net, dave@stgolabs.net, Luis Chamberlain Subject: [PATCH 4/4] Check if parent is alive once per loadfile processed Date: Wed, 9 Feb 2022 14:18:49 -0800 Message-Id: <20220209221849.434616-5-mcgrof@kernel.org> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20220209221849.434616-1-mcgrof@kernel.org> References: <20220209221849.434616-1-mcgrof@kernel.org> MIME-Version: 1.0 Sender: Luis Chamberlain Precedence: bulk List-ID: X-Mailing-List: fstests@vger.kernel.org From: Mel Gorman strace reports that a high percentage of time is spent calling kill() with 12,000,000 calls in 3 minutes. Check if the parent is alive once per load file processed. With later versions of apparmor, kill() is permission checked which is very expensive in itself and unnecessary. Instead use the ligher getppid() call and check against the cached value. Signed-off-by: Mel Gorman Signed-off-by: Luis Chamberlain --- child.c | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/child.c b/child.c index 828295d..7abb238 100644 --- a/child.c +++ b/child.c @@ -371,6 +371,10 @@ again: nb_time_reset(child); } + if (getppid() != parent) { + exit(1); + } + gettimeofday(&start, NULL); while (gzgets(gzf, line, sizeof(line)-1)) { @@ -384,10 +388,6 @@ again: params = sparams; - if (kill(parent, 0) == -1) { - exit(1); - } - loop_again: /* if this is a "LOOP " line, * remember the current file position and move to the next line