From patchwork Thu Aug 29 22:52:27 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Reinette Chatre X-Patchwork-Id: 13784004 Received: from mgamail.intel.com (mgamail.intel.com [192.198.163.12]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 56F991BB6A0; Thu, 29 Aug 2024 22:52:40 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=192.198.163.12 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1724971962; cv=none; b=aVDhZ95L3+HuqsJv2st5FEJYdVj93pD90ojApBx5pPhvGImqwfkAqLkdHv8le8zTHjHp3Gafy3klTfWg8jWIpQ0hkVZ0gR1+OAuxOArW6k7OFZXF6AJ42ajvgubLkeSJdZKxoXkKhao2j+CvhAq6yoX9Lq/9RSJybH0jkR5m5wg= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1724971962; c=relaxed/simple; bh=iKp4xCXqNgDjqiNx2zTCptY03/NMsct47JdU65TyF24=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=lNAJ1txMH9CjOkyEliVj/xp+Z3Fg0FURUNACLW2UsnUXM4vAda4g3E6XyRC29gCE5Dw3ezN29cubKjlUOsD606uTUhBvTv8OVIQLMfBsGER5tlSxAm0dP3mZ84HmKo5xcuVJPhZH6Lge9foH8yinKFsYbPBl+idahih3bhEpcdk= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com; spf=pass smtp.mailfrom=intel.com; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b=gOR4ZBzv; arc=none smtp.client-ip=192.198.163.12 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=intel.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b="gOR4ZBzv" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1724971960; x=1756507960; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=iKp4xCXqNgDjqiNx2zTCptY03/NMsct47JdU65TyF24=; b=gOR4ZBzvuSehOIFgsY8dOY/QhCHaLwru209Juqiukk0pPzKyNYIbBx1b R4M5mwa6X4/gr1SF4aD+K2ew7UKVS7aZREWKuCHqSD/1iuwIIZIo2zwCE DKlo/ahqN0zrPzdGs9DddGxd+ZH8kiouz483nCP53drilxVnRikpXKojc C4APm1DrirGdl5uy4AiLcCmILsefT0u0GUvDOKzcEJQRiGeyLw7mGMd/f EjNJ6oZOjN999xKl9Dz0lvQ33cwqkXhjaDLYgHPPDaEiYktWTGRRJXJio a2X79aapX6rXpj+vHoukyAHbNWnFCnLd/pCgu5XPPOSpQBCcxABcq0gAe w==; X-CSE-ConnectionGUID: PItHW7abSXClFDzCe7sOOQ== X-CSE-MsgGUID: Aec3CK/PR2+D38xkOABU1Q== X-IronPort-AV: E=McAfee;i="6700,10204,11179"; a="27479155" X-IronPort-AV: E=Sophos;i="6.10,186,1719903600"; d="scan'208";a="27479155" Received: from fmviesa007.fm.intel.com ([10.60.135.147]) by fmvoesa106.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 29 Aug 2024 15:52:39 -0700 X-CSE-ConnectionGUID: SKskw/HRQqmZyejLlvWAyw== X-CSE-MsgGUID: r5HhghX/T0a1Sa9ay8iHUg== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.10,186,1719903600"; d="scan'208";a="63415122" Received: from rchatre-mobl4.amr.corp.intel.com (HELO rchatre-mobl4.intel.com) ([10.125.111.220]) by fmviesa007-auth.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 29 Aug 2024 15:52:38 -0700 From: Reinette Chatre To: fenghua.yu@intel.com, shuah@kernel.org, tony.luck@intel.com, peternewman@google.com, babu.moger@amd.com, ilpo.jarvinen@linux.intel.com Cc: maciej.wieczor-retman@intel.com, reinette.chatre@intel.com, linux-kselftest@vger.kernel.org, linux-kernel@vger.kernel.org Subject: [PATCH 1/6] selftests/resctrl: Fix sparse warnings Date: Thu, 29 Aug 2024 15:52:27 -0700 Message-ID: X-Mailer: git-send-email 2.46.0 In-Reply-To: References: Precedence: bulk X-Mailing-List: linux-kselftest@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Fix following sparse warnings: tools/testing/selftests/resctrl/resctrl_val.c:47:6: warning: symbol 'membw_initialize_perf_event_attr' was not declared. Should it be static? tools/testing/selftests/resctrl/resctrl_val.c:64:6: warning: symbol 'membw_ioctl_perf_event_ioc_reset_enable' was not declared. Should it be static? tools/testing/selftests/resctrl/resctrl_val.c:70:6: warning: symbol 'membw_ioctl_perf_event_ioc_disable' was not declared. Should it be static? tools/testing/selftests/resctrl/resctrl_val.c:81:6: warning: symbol 'get_event_and_umask' was not declared. Should it be static? Signed-off-by: Reinette Chatre Reviewed-by: Ilpo Järvinen --- tools/testing/selftests/resctrl/resctrl_val.c | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/tools/testing/selftests/resctrl/resctrl_val.c b/tools/testing/selftests/resctrl/resctrl_val.c index 8c275f6b4dd7..70e8e31f5d1a 100644 --- a/tools/testing/selftests/resctrl/resctrl_val.c +++ b/tools/testing/selftests/resctrl/resctrl_val.c @@ -44,7 +44,7 @@ static int imcs; static struct imc_counter_config imc_counters_config[MAX_IMCS][2]; static const struct resctrl_test *current_test; -void membw_initialize_perf_event_attr(int i, int j) +static void membw_initialize_perf_event_attr(int i, int j) { memset(&imc_counters_config[i][j].pe, 0, sizeof(struct perf_event_attr)); @@ -61,13 +61,13 @@ void membw_initialize_perf_event_attr(int i, int j) PERF_FORMAT_TOTAL_TIME_ENABLED | PERF_FORMAT_TOTAL_TIME_RUNNING; } -void membw_ioctl_perf_event_ioc_reset_enable(int i, int j) +static void membw_ioctl_perf_event_ioc_reset_enable(int i, int j) { ioctl(imc_counters_config[i][j].fd, PERF_EVENT_IOC_RESET, 0); ioctl(imc_counters_config[i][j].fd, PERF_EVENT_IOC_ENABLE, 0); } -void membw_ioctl_perf_event_ioc_disable(int i, int j) +static void membw_ioctl_perf_event_ioc_disable(int i, int j) { ioctl(imc_counters_config[i][j].fd, PERF_EVENT_IOC_DISABLE, 0); } @@ -78,7 +78,7 @@ void membw_ioctl_perf_event_ioc_disable(int i, int j) * @count: iMC number * @op: Operation (read/write) */ -void get_event_and_umask(char *cas_count_cfg, int count, bool op) +static void get_event_and_umask(char *cas_count_cfg, int count, bool op) { char *token[MAX_TOKENS]; int i = 0; From patchwork Thu Aug 29 22:52:28 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Reinette Chatre X-Patchwork-Id: 13784005 Received: from mgamail.intel.com (mgamail.intel.com [192.198.163.12]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id BB1EC18B47B; Thu, 29 Aug 2024 22:52:40 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=192.198.163.12 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1724971962; cv=none; b=Fo5akK2pS4p/lS/i8ETiDHlACsePauJO4EAxJg0JJVFEpUpnodA4USapwbXTSj0f8rqDxcDexl6/UuqRtkwXYk7ghkR5W524oEFDYRgGX5QbvL0PfXjcS5QhP6ZVL3608cZkTMVMeUTzv+ZIKWxiGKqmbFeFIHnucZGFgTjrGCo= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1724971962; c=relaxed/simple; bh=0OjFF+To1zRUAdydXuWXh/Hwmgo2V5m4CUUVuEHN6gk=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=Vmpia2oWhJPrSZJWqX1wqGKsLXY5A1BoWuYgwjooFIuxHKWA6f5eQYT9B+Iyk1y8p4tSJRVEQK4m8ruf7u5B2dUz0VaJ1NQjUz1C7Bmk2TLJjgu+IQgcr0L3B7NgfJXYukt+rkbD0iHVC5rYmW+qi8KF2AP5I3eVG4Das1s9Nw8= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com; spf=pass smtp.mailfrom=intel.com; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b=el9KEMn0; arc=none smtp.client-ip=192.198.163.12 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=intel.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b="el9KEMn0" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1724971961; x=1756507961; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=0OjFF+To1zRUAdydXuWXh/Hwmgo2V5m4CUUVuEHN6gk=; b=el9KEMn0a9ld6nmNCHCdCmVfKrqSqnVtWDle6bSeq7X46VUztFGNIPgW SKRZ2Jnw3iOK154aHXwvMYq5DqekG6Ry43rpVXp3RoEFTnSgq3f/DEuHI WGXPfo02woVgr3BsMtOgrvdhWBN1G3NBGw9GbsUR+ZTXAf5kM9hOh438g +pP1OTed0WFedDPgEMR3+FCW6nKLk5zRajpUcCRCU3FV/3wTR1BP7RanT GuLBymCbIZb8ARtf0hn/X1UrGX494DPzj0DMUgJkNzeL3nR44PhfWpheM y5CkTyE4auy2IaZb+00vX7MC/jbh0pNGfTzDwSqz7leseHmPWUejornFb w==; X-CSE-ConnectionGUID: 0kLUHk83S7uvt8iypF0z2g== X-CSE-MsgGUID: 70qk6Zr8SwaMsTPkAnOyrg== X-IronPort-AV: E=McAfee;i="6700,10204,11179"; a="27479159" X-IronPort-AV: E=Sophos;i="6.10,186,1719903600"; d="scan'208";a="27479159" Received: from fmviesa007.fm.intel.com ([10.60.135.147]) by fmvoesa106.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 29 Aug 2024 15:52:39 -0700 X-CSE-ConnectionGUID: KpkrqGBLS5Gf5tEg7/evUQ== X-CSE-MsgGUID: 6axeBCsNTT2OYRYLcluLjA== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.10,186,1719903600"; d="scan'208";a="63415128" Received: from rchatre-mobl4.amr.corp.intel.com (HELO rchatre-mobl4.intel.com) ([10.125.111.220]) by fmviesa007-auth.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 29 Aug 2024 15:52:39 -0700 From: Reinette Chatre To: fenghua.yu@intel.com, shuah@kernel.org, tony.luck@intel.com, peternewman@google.com, babu.moger@amd.com, ilpo.jarvinen@linux.intel.com Cc: maciej.wieczor-retman@intel.com, reinette.chatre@intel.com, linux-kselftest@vger.kernel.org, linux-kernel@vger.kernel.org Subject: [PATCH 2/6] selftests/resctrl: Ensure measurements skip initialization of default benchmark Date: Thu, 29 Aug 2024 15:52:28 -0700 Message-ID: X-Mailer: git-send-email 2.46.0 In-Reply-To: References: Precedence: bulk X-Mailing-List: linux-kselftest@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 The CMT, MBA, and MBM tests rely on the resctrl_val() wrapper to start and run a benchmark while providing test specific flows via callbacks to do test specific configuration and measurements. At a high level, the resctrl_val() flow is: a) Start by fork()ing a child process that installs a signal handler for SIGUSR1 that, on receipt of SIGUSR1, will start running a benchmark. b) Assign the child process created in (a) to the resctrl control and monitoring group that dictates the memory and cache allocations with which the process can run and will contain all resctrl monitoring data of that process. c) Once parent and child are considered "ready" (determined via a message over a pipe) the parent signals the child (via SIGUSR1) to start the benchmark, waits one second for the benchmark to run, and then starts collecting monitoring data for the tests, potentially also changing allocation configuration depending on the various test callbacks. A problem with the above flow is the "black box" view of the benchmark that is combined with an arbitrarily chosen "wait one second" before measurements start. No matter what the benchmark does, it is given one second to initialize before measurements start. The default benchmark "fill_buf" consists of two parts, first it prepares a buffer (allocate, initialize, then flush), then it reads from the buffer (in unpredictable ways) until terminated. Depending on the system and the size of the buffer, the first "prepare" part may not be complete by the time the one second delay expires. Test measurements may thus start before the work needing to be measured runs. Split the default benchmark into its "prepare" and "runtime" parts and simplify the resctrl_val() wrapper while doing so. This same split cannot be done for the user provided benchmark (without a user interface change), so the current behavior is maintained for user provided benchmark. Assign the test itself to the control and monitoring group and run the "prepare" part of the benchmark in this context, ensuring it runs with required cache and memory bandwidth allocations. With the benchmark preparation complete it is only needed to fork() the "runtime" part of the benchmark (or entire user provided benchmark). Keep the "wait one second" delay before measurements start. For the default "fill_buf" benchmark this time now covers only the "runtime" portion that needs to be measured. For the user provided benchmark this delay maintains current behavior. Signed-off-by: Reinette Chatre --- tools/testing/selftests/resctrl/fill_buf.c | 19 +- tools/testing/selftests/resctrl/resctrl.h | 2 +- tools/testing/selftests/resctrl/resctrl_val.c | 225 ++++++------------ 3 files changed, 70 insertions(+), 176 deletions(-) diff --git a/tools/testing/selftests/resctrl/fill_buf.c b/tools/testing/selftests/resctrl/fill_buf.c index ae120f1735c0..12c71bb44cb6 100644 --- a/tools/testing/selftests/resctrl/fill_buf.c +++ b/tools/testing/selftests/resctrl/fill_buf.c @@ -114,7 +114,7 @@ void fill_cache_read(unsigned char *buf, size_t buf_size, bool once) *value_sink = ret; } -static void fill_cache_write(unsigned char *buf, size_t buf_size, bool once) +void fill_cache_write(unsigned char *buf, size_t buf_size, bool once) { while (1) { fill_one_span_write(buf, buf_size); @@ -150,20 +150,3 @@ unsigned char *alloc_buffer(size_t buf_size, int memflush) return buf; } - -int run_fill_buf(size_t buf_size, int memflush, int op, bool once) -{ - unsigned char *buf; - - buf = alloc_buffer(buf_size, memflush); - if (!buf) - return -1; - - if (op == 0) - fill_cache_read(buf, buf_size, once); - else - fill_cache_write(buf, buf_size, once); - free(buf); - - return 0; -} diff --git a/tools/testing/selftests/resctrl/resctrl.h b/tools/testing/selftests/resctrl/resctrl.h index 2dda56084588..0afbc4dd18e4 100644 --- a/tools/testing/selftests/resctrl/resctrl.h +++ b/tools/testing/selftests/resctrl/resctrl.h @@ -142,7 +142,7 @@ int perf_event_open(struct perf_event_attr *hw_event, pid_t pid, int cpu, unsigned char *alloc_buffer(size_t buf_size, int memflush); void mem_flush(unsigned char *buf, size_t buf_size); void fill_cache_read(unsigned char *buf, size_t buf_size, bool once); -int run_fill_buf(size_t buf_size, int memflush, int op, bool once); +void fill_cache_write(unsigned char *buf, size_t buf_size, bool once); int initialize_mem_bw_imc(void); int measure_mem_bw(const struct user_params *uparams, struct resctrl_val_param *param, pid_t bm_pid, diff --git a/tools/testing/selftests/resctrl/resctrl_val.c b/tools/testing/selftests/resctrl/resctrl_val.c index 70e8e31f5d1a..574b72604f95 100644 --- a/tools/testing/selftests/resctrl/resctrl_val.c +++ b/tools/testing/selftests/resctrl/resctrl_val.c @@ -448,7 +448,7 @@ static int get_mem_bw_resctrl(FILE *fp, unsigned long *mbm_total) return 0; } -static pid_t bm_pid, ppid; +static pid_t bm_pid; void ctrlc_handler(int signum, siginfo_t *info, void *ptr) { @@ -506,13 +506,6 @@ void signal_handler_unregister(void) } } -static void parent_exit(pid_t ppid) -{ - kill(ppid, SIGKILL); - umount_resctrlfs(); - exit(EXIT_FAILURE); -} - /* * print_results_bw: the memory bandwidth results are stored in a file * @filename: file that stores the results @@ -614,61 +607,6 @@ int measure_mem_bw(const struct user_params *uparams, return ret; } -/* - * run_benchmark - Run a specified benchmark or fill_buf (default benchmark) - * in specified signal. Direct benchmark stdio to /dev/null. - * @signum: signal number - * @info: signal info - * @ucontext: user context in signal handling - */ -static void run_benchmark(int signum, siginfo_t *info, void *ucontext) -{ - int operation, ret, memflush; - char **benchmark_cmd; - size_t span; - bool once; - FILE *fp; - - benchmark_cmd = info->si_ptr; - - /* - * Direct stdio of child to /dev/null, so that only parent writes to - * stdio (console) - */ - fp = freopen("/dev/null", "w", stdout); - if (!fp) { - ksft_perror("Unable to direct benchmark status to /dev/null"); - parent_exit(ppid); - } - - if (strcmp(benchmark_cmd[0], "fill_buf") == 0) { - /* Execute default fill_buf benchmark */ - span = strtoul(benchmark_cmd[1], NULL, 10); - memflush = atoi(benchmark_cmd[2]); - operation = atoi(benchmark_cmd[3]); - if (!strcmp(benchmark_cmd[4], "true")) { - once = true; - } else if (!strcmp(benchmark_cmd[4], "false")) { - once = false; - } else { - ksft_print_msg("Invalid once parameter\n"); - parent_exit(ppid); - } - - if (run_fill_buf(span, memflush, operation, once)) - fprintf(stderr, "Error in running fill buffer\n"); - } else { - /* Execute specified benchmark */ - ret = execvp(benchmark_cmd[0], benchmark_cmd); - if (ret) - ksft_perror("execvp"); - } - - fclose(stdout); - ksft_print_msg("Unable to run specified benchmark\n"); - parent_exit(ppid); -} - /* * resctrl_val: execute benchmark and measure memory bandwidth on * the benchmark @@ -684,11 +622,13 @@ int resctrl_val(const struct resctrl_test *test, const char * const *benchmark_cmd, struct resctrl_val_param *param) { - struct sigaction sigact; - int ret = 0, pipefd[2]; - char pipe_message = 0; - union sigval value; - int domain_id; + int domain_id, operation = 0, memflush = 1; + size_t span = DEFAULT_SPAN; + unsigned char *buf = NULL; + cpu_set_t old_affinity; + bool once = false; + int ret = 0; + pid_t ppid; if (strcmp(param->filename, "") == 0) sprintf(param->filename, "stdio"); @@ -699,111 +639,80 @@ int resctrl_val(const struct resctrl_test *test, return ret; } - /* - * If benchmark wasn't successfully started by child, then child should - * kill parent, so save parent's pid - */ ppid = getpid(); - if (pipe(pipefd)) { - ksft_perror("Unable to create pipe"); + /* Taskset test to specified CPU. */ + ret = taskset_benchmark(ppid, uparams->cpu, &old_affinity); + if (ret) + return ret; - return -1; + /* Write test to specified control & monitoring group in resctrl FS. */ + ret = write_bm_pid_to_resctrl(ppid, param->ctrlgrp, param->mongrp); + if (ret) + goto reset_affinity; + + if (param->init) { + ret = param->init(param, domain_id); + if (ret) + goto reset_affinity; } /* - * Fork to start benchmark, save child's pid so that it can be killed - * when needed + * If not running user provided benchmark, run the default + * "fill_buf". First phase of "fill_buf" is to prepare the + * buffer that the benchmark will operate on. No measurements + * are needed during this phase and prepared memory will be + * passed to next part of benchmark via copy-on-write. TBD + * how this impacts "write" benchmark, but no test currently + * uses this. */ - fflush(stdout); + if (strcmp(benchmark_cmd[0], "fill_buf") == 0) { + span = strtoul(benchmark_cmd[1], NULL, 10); + memflush = atoi(benchmark_cmd[2]); + operation = atoi(benchmark_cmd[3]); + if (!strcmp(benchmark_cmd[4], "true")) { + once = true; + } else if (!strcmp(benchmark_cmd[4], "false")) { + once = false; + } else { + ksft_print_msg("Invalid once parameter\n"); + ret = -EINVAL; + goto reset_affinity; + } + + buf = alloc_buffer(span, memflush); + if (!buf) { + ret = -ENOMEM; + goto reset_affinity; + } + } + bm_pid = fork(); if (bm_pid == -1) { + ret = -errno; ksft_perror("Unable to fork"); - - return -1; + goto free_buf; } + /* + * What needs to be measured runs in separate process until + * terminated. + */ if (bm_pid == 0) { - /* - * Mask all signals except SIGUSR1, parent uses SIGUSR1 to - * start benchmark - */ - sigfillset(&sigact.sa_mask); - sigdelset(&sigact.sa_mask, SIGUSR1); - - sigact.sa_sigaction = run_benchmark; - sigact.sa_flags = SA_SIGINFO; - - /* Register for "SIGUSR1" signal from parent */ - if (sigaction(SIGUSR1, &sigact, NULL)) { - ksft_perror("Can't register child for signal"); - parent_exit(ppid); + if (strcmp(benchmark_cmd[0], "fill_buf") == 0) { + if (operation == 0) + fill_cache_read(buf, span, once); + else + fill_cache_write(buf, span, once); + } else { + execvp(benchmark_cmd[0], (char **)benchmark_cmd); } - - /* Tell parent that child is ready */ - close(pipefd[0]); - pipe_message = 1; - if (write(pipefd[1], &pipe_message, sizeof(pipe_message)) < - sizeof(pipe_message)) { - ksft_perror("Failed signaling parent process"); - close(pipefd[1]); - return -1; - } - close(pipefd[1]); - - /* Suspend child until delivery of "SIGUSR1" from parent */ - sigsuspend(&sigact.sa_mask); - - ksft_perror("Child is done"); - parent_exit(ppid); + exit(EXIT_SUCCESS); } ksft_print_msg("Benchmark PID: %d\n", (int)bm_pid); - /* - * The cast removes constness but nothing mutates benchmark_cmd within - * the context of this process. At the receiving process, it becomes - * argv, which is mutable, on exec() but that's after fork() so it - * doesn't matter for the process running the tests. - */ - value.sival_ptr = (void *)benchmark_cmd; - - /* Taskset benchmark to specified cpu */ - ret = taskset_benchmark(bm_pid, uparams->cpu, NULL); - if (ret) - goto out; - - /* Write benchmark to specified control&monitoring grp in resctrl FS */ - ret = write_bm_pid_to_resctrl(bm_pid, param->ctrlgrp, param->mongrp); - if (ret) - goto out; - - if (param->init) { - ret = param->init(param, domain_id); - if (ret) - goto out; - } - - /* Parent waits for child to be ready. */ - close(pipefd[1]); - while (pipe_message != 1) { - if (read(pipefd[0], &pipe_message, sizeof(pipe_message)) < - sizeof(pipe_message)) { - ksft_perror("Failed reading message from child process"); - close(pipefd[0]); - goto out; - } - } - close(pipefd[0]); - - /* Signal child to start benchmark */ - if (sigqueue(bm_pid, SIGUSR1, value) == -1) { - ksft_perror("sigqueue SIGUSR1 to child"); - ret = -1; - goto out; - } - - /* Give benchmark enough time to fully run */ + /* Give benchmark enough time to fully run. */ sleep(1); /* Test runs until the callback setup() tells the test to stop. */ @@ -821,8 +730,10 @@ int resctrl_val(const struct resctrl_test *test, break; } -out: kill(bm_pid, SIGKILL); - +free_buf: + free(buf); +reset_affinity: + taskset_restore(ppid, &old_affinity); return ret; } From patchwork Thu Aug 29 22:52:29 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Reinette Chatre X-Patchwork-Id: 13784008 Received: from mgamail.intel.com (mgamail.intel.com [192.198.163.12]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id B5DAF1BE23C; Thu, 29 Aug 2024 22:52:41 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=192.198.163.12 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1724971964; cv=none; b=d+Q2x3iv7FQoOEo9n8uBA6JXRgo9dRJxwkasaqyw68pz2ZGzLo+UK6JpE2RuH25pKvP6QhraeBbrz1HvdxzcBKI2W1cmgM/gEkcxIr5ZKR7g9aM7lrrk65R19NiS769ZAej4Jgy5kkpxN7ruL9VpACXMTAfCVXfwX614tAT3Ovk= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1724971964; c=relaxed/simple; bh=aqlSuxz4AFaQXCpMSDXNV0hpsqTQBg2lF8eW+Et3apw=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=NJ1Qphum2bBUt8F6bDGqEENZeVL0CKUyVjD6YyzDeA4pSCF8sFasAlohebbfKJnbWxAai34PJesw0bsjDAPpjQjg8noWpL5WlsEqYtWXoqxmLF5+vltqyWnJerc1J3MMHH2hpx0tptyW/TgVJTmpaTEP7ouREAi0SXJF+cqRD0A= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com; spf=pass smtp.mailfrom=intel.com; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b=S8elLQgz; arc=none smtp.client-ip=192.198.163.12 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=intel.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b="S8elLQgz" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1724971963; x=1756507963; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=aqlSuxz4AFaQXCpMSDXNV0hpsqTQBg2lF8eW+Et3apw=; b=S8elLQgzisJYV6G7Hm0Cr9/tCytOtaMaC6uw9FS6qY/s0VlKcet6D+WN rxQEXp93groEgXFzZQgnpaapkwjqDhd/psFVCXB4ahIdDFruYNZKtVcK3 OyekL4E7ukbGSK/KkAUvFsLUlXziFfnsZQEVMo/cMS7ErchIC2gIb/5lQ BaCX6M9RYAeGg+XAPWLrCvoyjVsnzaSmgiscHZ9guOjX2V0IA0BFEzeDz 1PaPY29+Cb2YmesBgz7bEQUUviVIAVG8XHo09T+unULk4DQlk3f7Fg77W WKZ3YT+xzijEe/RxE2rsSM+cEMIAtwrtZiqvTqll0pmzAXQMgG1DDWKXJ w==; X-CSE-ConnectionGUID: 7ZENxrSHTU6XR+VUnDwlWQ== X-CSE-MsgGUID: 7EfpcUh4SMmuq9TluBvUrg== X-IronPort-AV: E=McAfee;i="6700,10204,11179"; a="27479164" X-IronPort-AV: E=Sophos;i="6.10,186,1719903600"; d="scan'208";a="27479164" Received: from fmviesa007.fm.intel.com ([10.60.135.147]) by fmvoesa106.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 29 Aug 2024 15:52:40 -0700 X-CSE-ConnectionGUID: PZB3zBKERJeCE8hBoWxD2Q== X-CSE-MsgGUID: oI2Ya5xYS52NSPtxITApjg== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.10,186,1719903600"; d="scan'208";a="63415135" Received: from rchatre-mobl4.amr.corp.intel.com (HELO rchatre-mobl4.intel.com) ([10.125.111.220]) by fmviesa007-auth.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 29 Aug 2024 15:52:39 -0700 From: Reinette Chatre To: fenghua.yu@intel.com, shuah@kernel.org, tony.luck@intel.com, peternewman@google.com, babu.moger@amd.com, ilpo.jarvinen@linux.intel.com Cc: maciej.wieczor-retman@intel.com, reinette.chatre@intel.com, linux-kselftest@vger.kernel.org, linux-kernel@vger.kernel.org Subject: [PATCH 3/6] selftests/resctrl: Simplify benchmark parameter passing Date: Thu, 29 Aug 2024 15:52:29 -0700 Message-ID: X-Mailer: git-send-email 2.46.0 In-Reply-To: References: Precedence: bulk X-Mailing-List: linux-kselftest@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 The benchmark used during the CMT, MBM, and MBA tests can be provided by the user via (-b) parameter to the tests, if not provided the default "fill_buf" benchmark is used. The "fill_buf" benchmark requires parameters and these are managed as an array of strings. Using an array of strings to manage the "fill_buf" parameters is complex because it requires transformations to/from strings at every producer and consumer. This is made worse for the individual tests where the default benchmark parameters values may not be appropriate and additional data wrangling is required. For example, the CMT test duplicates the entire array of strings in order to replace one of the parameters. Replace the "array of strings" parameters used for "fill_buf" with a struct that contains the "fill_buf" parameters that can be used directly without transformations to/from strings. Make these parameters part of the parameters associated with each test so that each test can set benchmark parameters that are appropriate for it. Signed-off-by: Reinette Chatre --- tools/testing/selftests/resctrl/cmt_test.c | 28 +++-------- tools/testing/selftests/resctrl/mba_test.c | 7 ++- tools/testing/selftests/resctrl/mbm_test.c | 9 +++- tools/testing/selftests/resctrl/resctrl.h | 49 +++++++++++++------ .../testing/selftests/resctrl/resctrl_tests.c | 15 +----- tools/testing/selftests/resctrl/resctrl_val.c | 38 +++++--------- 6 files changed, 69 insertions(+), 77 deletions(-) diff --git a/tools/testing/selftests/resctrl/cmt_test.c b/tools/testing/selftests/resctrl/cmt_test.c index 0c045080d808..f09d5dfab38c 100644 --- a/tools/testing/selftests/resctrl/cmt_test.c +++ b/tools/testing/selftests/resctrl/cmt_test.c @@ -116,15 +116,12 @@ static void cmt_test_cleanup(void) static int cmt_run_test(const struct resctrl_test *test, const struct user_params *uparams) { - const char * const *cmd = uparams->benchmark_cmd; - const char *new_cmd[BENCHMARK_ARGS]; unsigned long cache_total_size = 0; int n = uparams->bits ? : 5; unsigned long long_mask; - char *span_str = NULL; int count_of_bits; size_t span; - int ret, i; + int ret; ret = get_full_cbm("L3", &long_mask); if (ret) @@ -155,32 +152,21 @@ static int cmt_run_test(const struct resctrl_test *test, const struct user_param span = cache_portion_size(cache_total_size, param.mask, long_mask); - if (strcmp(cmd[0], "fill_buf") == 0) { - /* Duplicate the command to be able to replace span in it */ - for (i = 0; uparams->benchmark_cmd[i]; i++) - new_cmd[i] = uparams->benchmark_cmd[i]; - new_cmd[i] = NULL; - - ret = asprintf(&span_str, "%zu", span); - if (ret < 0) - return -1; - new_cmd[1] = span_str; - cmd = new_cmd; - } + param.fill_buf.buf_size = span; + param.fill_buf.memflush = 1; + param.fill_buf.operation = 0; + param.fill_buf.once = false; remove(RESULT_FILE_NAME); - ret = resctrl_val(test, uparams, cmd, ¶m); + ret = resctrl_val(test, uparams, ¶m); if (ret) - goto out; + return ret; ret = check_results(¶m, span, n); if (ret && (get_vendor() == ARCH_INTEL)) ksft_print_msg("Intel CMT may be inaccurate when Sub-NUMA Clustering is enabled. Check BIOS configuration.\n"); -out: - free(span_str); - return ret; } diff --git a/tools/testing/selftests/resctrl/mba_test.c b/tools/testing/selftests/resctrl/mba_test.c index ab8496a4925b..8ad433495f61 100644 --- a/tools/testing/selftests/resctrl/mba_test.c +++ b/tools/testing/selftests/resctrl/mba_test.c @@ -174,7 +174,12 @@ static int mba_run_test(const struct resctrl_test *test, const struct user_param remove(RESULT_FILE_NAME); - ret = resctrl_val(test, uparams, uparams->benchmark_cmd, ¶m); + param.fill_buf.buf_size = DEFAULT_SPAN; + param.fill_buf.memflush = 1; + param.fill_buf.operation = 0; + param.fill_buf.once = false; + + ret = resctrl_val(test, uparams, ¶m); if (ret) return ret; diff --git a/tools/testing/selftests/resctrl/mbm_test.c b/tools/testing/selftests/resctrl/mbm_test.c index 6b5a3b52d861..b6883f274c74 100644 --- a/tools/testing/selftests/resctrl/mbm_test.c +++ b/tools/testing/selftests/resctrl/mbm_test.c @@ -142,11 +142,16 @@ static int mbm_run_test(const struct resctrl_test *test, const struct user_param remove(RESULT_FILE_NAME); - ret = resctrl_val(test, uparams, uparams->benchmark_cmd, ¶m); + param.fill_buf.buf_size = DEFAULT_SPAN; + param.fill_buf.memflush = 1; + param.fill_buf.operation = 0; + param.fill_buf.once = false; + + ret = resctrl_val(test, uparams, ¶m); if (ret) return ret; - ret = check_results(DEFAULT_SPAN); + ret = check_results(param.fill_buf.buf_size); if (ret && (get_vendor() == ARCH_INTEL)) ksft_print_msg("Intel MBM may be inaccurate when Sub-NUMA Clustering is enabled. Check BIOS configuration.\n"); diff --git a/tools/testing/selftests/resctrl/resctrl.h b/tools/testing/selftests/resctrl/resctrl.h index 0afbc4dd18e4..0e5456165a6a 100644 --- a/tools/testing/selftests/resctrl/resctrl.h +++ b/tools/testing/selftests/resctrl/resctrl.h @@ -79,6 +79,26 @@ struct resctrl_test { void (*cleanup)(void); }; +/* + * fill_buf_param: "fill_buf" benchmark parameters + * @buf_size: Size (in bytes) of buffer used in benchmark. + * "fill_buf" allocates and initializes buffer of + * @buf_size. + * @operation: If 0, then only read operations are performed on + * the buffer, if 1 then only write operations are + * performed on the buffer. + * @memflush: 1 if buffer should be flushed after + * allocation and initialization. + * @once: Benchmark will perform @operation once if true, + * infinitely (until terminated) if false. + */ +struct fill_buf_param { + size_t buf_size; + int operation; + int memflush; + int once; +}; + /* * resctrl_val_param: resctrl test parameters * @ctrlgrp: Name of the control monitor group (con_mon grp) @@ -87,21 +107,23 @@ struct resctrl_test { * @init: Callback function to initialize test environment * @setup: Callback function to setup per test run environment * @measure: Callback that performs the measurement (a single test) + * @fill_buf: Parameters for default "fill_buf" benchmark */ struct resctrl_val_param { - const char *ctrlgrp; - const char *mongrp; - char filename[64]; - unsigned long mask; - int num_of_runs; - int (*init)(const struct resctrl_val_param *param, - int domain_id); - int (*setup)(const struct resctrl_test *test, - const struct user_params *uparams, - struct resctrl_val_param *param); - int (*measure)(const struct user_params *uparams, - struct resctrl_val_param *param, - pid_t bm_pid); + const char *ctrlgrp; + const char *mongrp; + char filename[64]; + unsigned long mask; + int num_of_runs; + int (*init)(const struct resctrl_val_param *param, + int domain_id); + int (*setup)(const struct resctrl_test *test, + const struct user_params *uparams, + struct resctrl_val_param *param); + int (*measure)(const struct user_params *uparams, + struct resctrl_val_param *param, + pid_t bm_pid); + struct fill_buf_param fill_buf; }; struct perf_event_read { @@ -151,7 +173,6 @@ void initialize_mem_bw_resctrl(const struct resctrl_val_param *param, int domain_id); int resctrl_val(const struct resctrl_test *test, const struct user_params *uparams, - const char * const *benchmark_cmd, struct resctrl_val_param *param); unsigned long create_bit_mask(unsigned int start, unsigned int len); unsigned int count_contiguous_bits(unsigned long val, unsigned int *start); diff --git a/tools/testing/selftests/resctrl/resctrl_tests.c b/tools/testing/selftests/resctrl/resctrl_tests.c index ecbb7605a981..ce8fcc769d57 100644 --- a/tools/testing/selftests/resctrl/resctrl_tests.c +++ b/tools/testing/selftests/resctrl/resctrl_tests.c @@ -162,7 +162,7 @@ int main(int argc, char **argv) bool test_param_seen = false; struct user_params uparams; char *span_str = NULL; - int ret, c, i; + int c, i; init_user_params(&uparams); @@ -257,19 +257,6 @@ int main(int argc, char **argv) filter_dmesg(); - if (!uparams.benchmark_cmd[0]) { - /* If no benchmark is given by "-b" argument, use fill_buf. */ - uparams.benchmark_cmd[0] = "fill_buf"; - ret = asprintf(&span_str, "%u", DEFAULT_SPAN); - if (ret < 0) - ksft_exit_fail_msg("Out of memory!\n"); - uparams.benchmark_cmd[1] = span_str; - uparams.benchmark_cmd[2] = "1"; - uparams.benchmark_cmd[3] = "0"; - uparams.benchmark_cmd[4] = "false"; - uparams.benchmark_cmd[5] = NULL; - } - ksft_set_plan(tests); for (i = 0; i < ARRAY_SIZE(resctrl_tests); i++) diff --git a/tools/testing/selftests/resctrl/resctrl_val.c b/tools/testing/selftests/resctrl/resctrl_val.c index 574b72604f95..9a5a9a67e059 100644 --- a/tools/testing/selftests/resctrl/resctrl_val.c +++ b/tools/testing/selftests/resctrl/resctrl_val.c @@ -612,21 +612,17 @@ int measure_mem_bw(const struct user_params *uparams, * the benchmark * @test: test information structure * @uparams: user supplied parameters - * @benchmark_cmd: benchmark command and its arguments * @param: parameters passed to resctrl_val() * * Return: 0 when the test was run, < 0 on error. */ int resctrl_val(const struct resctrl_test *test, const struct user_params *uparams, - const char * const *benchmark_cmd, struct resctrl_val_param *param) { - int domain_id, operation = 0, memflush = 1; - size_t span = DEFAULT_SPAN; unsigned char *buf = NULL; cpu_set_t old_affinity; - bool once = false; + int domain_id; int ret = 0; pid_t ppid; @@ -666,21 +662,9 @@ int resctrl_val(const struct resctrl_test *test, * how this impacts "write" benchmark, but no test currently * uses this. */ - if (strcmp(benchmark_cmd[0], "fill_buf") == 0) { - span = strtoul(benchmark_cmd[1], NULL, 10); - memflush = atoi(benchmark_cmd[2]); - operation = atoi(benchmark_cmd[3]); - if (!strcmp(benchmark_cmd[4], "true")) { - once = true; - } else if (!strcmp(benchmark_cmd[4], "false")) { - once = false; - } else { - ksft_print_msg("Invalid once parameter\n"); - ret = -EINVAL; - goto reset_affinity; - } - - buf = alloc_buffer(span, memflush); + if (!uparams->benchmark_cmd[0]) { + buf = alloc_buffer(param->fill_buf.buf_size, + param->fill_buf.memflush); if (!buf) { ret = -ENOMEM; goto reset_affinity; @@ -699,13 +683,17 @@ int resctrl_val(const struct resctrl_test *test, * terminated. */ if (bm_pid == 0) { - if (strcmp(benchmark_cmd[0], "fill_buf") == 0) { - if (operation == 0) - fill_cache_read(buf, span, once); + if (!uparams->benchmark_cmd[0]) { + if (param->fill_buf.operation == 0) + fill_cache_read(buf, + param->fill_buf.buf_size, + param->fill_buf.once); else - fill_cache_write(buf, span, once); + fill_cache_write(buf, + param->fill_buf.buf_size, + param->fill_buf.once); } else { - execvp(benchmark_cmd[0], (char **)benchmark_cmd); + execvp(uparams->benchmark_cmd[0], (char **)uparams->benchmark_cmd); } exit(EXIT_SUCCESS); } From patchwork Thu Aug 29 22:52:30 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Reinette Chatre X-Patchwork-Id: 13784006 Received: from mgamail.intel.com (mgamail.intel.com [192.198.163.12]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 2965C1BE22B; Thu, 29 Aug 2024 22:52:42 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=192.198.163.12 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1724971964; cv=none; b=UYp8U3VlWpG1N746q/P301H5F+y8nw+IuJeyifiY0/X3ZberroOE4P2UN2JgUYxQwRjH9fH1WL9WJbdIpcWAFfIQ+Y0nwC0PK085Xyyv/XuG+0fi0o2LLhIt/EG7vDFL+UJYoeVmTl1QD7P8Mw/t5aartGhW1wSVF+FWbVAT/WI= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1724971964; c=relaxed/simple; bh=idF4CTKFsmnnUxvbJlS1PZZ3Ktwhc6P2TlPzn2oeFTA=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=T0gMIm+lpPAgbSfM+cRXx3aIloBqpdqTnOhh99NmeCSAVJwXYf5eGZQk6k2nwidi93i+m2KoCi1j1aBjUR8biqlcYhPBEbuJlbZhlJySVNxKm3XgboXNJHOxPQA8ptCRCVl4tz5Da43goA/usdCGpd93FA7bKgQcEALUrfW+fek= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com; spf=pass smtp.mailfrom=intel.com; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b=g1kndo4B; arc=none smtp.client-ip=192.198.163.12 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=intel.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b="g1kndo4B" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1724971962; x=1756507962; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=idF4CTKFsmnnUxvbJlS1PZZ3Ktwhc6P2TlPzn2oeFTA=; b=g1kndo4BDTdCekQ6Dg1nhdtJ+1322BrhwAWCueCnY2MOSFsSp/+ncZkh 3rtCil33+yP0hKjaNw+vNy1GjE4xWJndHzCYoYk99iF6g7LtEXdYz47jQ HhnmoNXxGDxsGJy6b1tPdkmH5jTO4S1VB0DDwXAQHSsP2geqP7jVGfOQ1 umKC7sKVZXIgWYmC1qoW5yH7FbYD+qkpoobdA9AWG56ttDEThfmrL5vDH sYHM7duSj/oN+wkUaIIxfwF5oZ1/4OaohAHUGDh0tiIE8M2Wwb5Zs3KW/ AArL/S8aHjlOYsyNgNFgDruR97I1gjjZqBYdFZtHx3U4qRQ1Aj4gwOYNd Q==; X-CSE-ConnectionGUID: dr2ht3/OSKKsLof7QhJVCw== X-CSE-MsgGUID: VGhaikf9T8uE5Ehxecjgbg== X-IronPort-AV: E=McAfee;i="6700,10204,11179"; a="27479168" X-IronPort-AV: E=Sophos;i="6.10,186,1719903600"; d="scan'208";a="27479168" Received: from fmviesa007.fm.intel.com ([10.60.135.147]) by fmvoesa106.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 29 Aug 2024 15:52:40 -0700 X-CSE-ConnectionGUID: cnOQrdNmTaW584l+E2YsLw== X-CSE-MsgGUID: WvIqZecIRTSjorz+rLtmaQ== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.10,186,1719903600"; d="scan'208";a="63415144" Received: from rchatre-mobl4.amr.corp.intel.com (HELO rchatre-mobl4.intel.com) ([10.125.111.220]) by fmviesa007-auth.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 29 Aug 2024 15:52:40 -0700 From: Reinette Chatre To: fenghua.yu@intel.com, shuah@kernel.org, tony.luck@intel.com, peternewman@google.com, babu.moger@amd.com, ilpo.jarvinen@linux.intel.com Cc: maciej.wieczor-retman@intel.com, reinette.chatre@intel.com, linux-kselftest@vger.kernel.org, linux-kernel@vger.kernel.org Subject: [PATCH 4/6] selftests/resctrl: Use cache size to determine "fill_buf" buffer size Date: Thu, 29 Aug 2024 15:52:30 -0700 Message-ID: X-Mailer: git-send-email 2.46.0 In-Reply-To: References: Precedence: bulk X-Mailing-List: linux-kselftest@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 By default the MBM and MBA tests use the "fill_buf" benchmark to read from a buffer with the goal to measure the memory bandwidth generated by this buffer access. Care should be taken when sizing the buffer used by the "fill_buf" benchmark. If the buffer is small enough to fit in the cache then it cannot be expected that the benchmark will generate much memory bandwidth. For example, on a system with 320MB L3 cache the existing hardcoded default of 250MB is insufficient. Use the measured cache size to determine a buffer size that can be expected to trigger memory access while keeping the existing default as minimum that has been appropriate for testing so far. Signed-off-by: Reinette Chatre --- tools/testing/selftests/resctrl/mba_test.c | 8 +++++++- tools/testing/selftests/resctrl/mbm_test.c | 8 +++++++- 2 files changed, 14 insertions(+), 2 deletions(-) diff --git a/tools/testing/selftests/resctrl/mba_test.c b/tools/testing/selftests/resctrl/mba_test.c index 8ad433495f61..cad473b81a64 100644 --- a/tools/testing/selftests/resctrl/mba_test.c +++ b/tools/testing/selftests/resctrl/mba_test.c @@ -170,11 +170,17 @@ static int mba_run_test(const struct resctrl_test *test, const struct user_param .setup = mba_setup, .measure = mba_measure, }; + unsigned long cache_total_size = 0; int ret; remove(RESULT_FILE_NAME); - param.fill_buf.buf_size = DEFAULT_SPAN; + ret = get_cache_size(uparams->cpu, "L3", &cache_total_size); + if (ret) + return ret; + + param.fill_buf.buf_size = cache_total_size > DEFAULT_SPAN ? + cache_total_size * 2 : DEFAULT_SPAN; param.fill_buf.memflush = 1; param.fill_buf.operation = 0; param.fill_buf.once = false; diff --git a/tools/testing/selftests/resctrl/mbm_test.c b/tools/testing/selftests/resctrl/mbm_test.c index b6883f274c74..734bfa4f42b3 100644 --- a/tools/testing/selftests/resctrl/mbm_test.c +++ b/tools/testing/selftests/resctrl/mbm_test.c @@ -138,11 +138,17 @@ static int mbm_run_test(const struct resctrl_test *test, const struct user_param .setup = mbm_setup, .measure = mbm_measure, }; + unsigned long cache_total_size = 0; int ret; remove(RESULT_FILE_NAME); - param.fill_buf.buf_size = DEFAULT_SPAN; + ret = get_cache_size(uparams->cpu, "L3", &cache_total_size); + if (ret) + return ret; + + param.fill_buf.buf_size = cache_total_size > DEFAULT_SPAN ? + cache_total_size * 2 : DEFAULT_SPAN; param.fill_buf.memflush = 1; param.fill_buf.operation = 0; param.fill_buf.once = false; From patchwork Thu Aug 29 22:52:31 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Reinette Chatre X-Patchwork-Id: 13784007 Received: from mgamail.intel.com (mgamail.intel.com [192.198.163.12]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id E3E491BE242; Thu, 29 Aug 2024 22:52:42 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=192.198.163.12 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1724971964; cv=none; b=c+vOr7tC4471Ck6j4eOTE3SSvLOqTBTWh1oY12Pf446jFwC7kUT4XthQtuap3/CgNuF7bOq7TOqP55fgfq3pBJhUIPRFefWAf2HxztpbA9HNU54mJvtwblqMsLc9EOWZYT8FLUDKqSpiVCRQ2gmjns6QEIw1oWsfUxh/QENROF4= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1724971964; c=relaxed/simple; bh=Pk/7UL5060saBr2FSXLTHwKgFHKGTMQhLToZZ8b+6FQ=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=HtPd8UsYICBz3PfEmL1Uhx1W3De8cqQUoBIANkBQeWPGFX4u0toGSPzfFJRPVBJ7de2c5SXybPABN6jaelrGXaVi6Lqq4IDjOaBTERPq5ecEbgnVQZqmjwHW2NxzIAEVztuJQsu3CrlTiAEkKtUCFHG/eWqkZKnlIv/DjF5EYbs= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com; spf=pass smtp.mailfrom=intel.com; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b=LM9c1eoN; arc=none smtp.client-ip=192.198.163.12 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=intel.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b="LM9c1eoN" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1724971963; x=1756507963; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=Pk/7UL5060saBr2FSXLTHwKgFHKGTMQhLToZZ8b+6FQ=; b=LM9c1eoNcMAKroJJdKNvgS/FNqsCdEf4LfeMfK3F9dK4B6TizsvhNqYn Xmc57r2XFx4Dxa2YhN46SVl45ED9FHm7saQP5A80F74AuhU09kGbu2Pxf HRIPN7G4zR12ydw1wqjAqlkgYBqfz4oQbHNPI+RB3e9GYFYxdkJXuBA/E V3V1MQzMgFi9422wlKLDF/TUwapz/WWFV0XadBhS3HHuunF9MYnbzdxkp mBIYs/gHwd/bsRVsiz6Y+PPW8u2sohP1KjLkmyjH3BcJFFpOdwNsWtCI2 /M6OXlhvu2jnacQIkKORAijJp53ovpdT93/KbCTzCbwPQPoqJMsM+ilGx Q==; X-CSE-ConnectionGUID: 3i6e+gC4S/KVIygEFltd4g== X-CSE-MsgGUID: HAWwvBrdROeJUHJfx4MONw== X-IronPort-AV: E=McAfee;i="6700,10204,11179"; a="27479172" X-IronPort-AV: E=Sophos;i="6.10,186,1719903600"; d="scan'208";a="27479172" Received: from fmviesa007.fm.intel.com ([10.60.135.147]) by fmvoesa106.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 29 Aug 2024 15:52:41 -0700 X-CSE-ConnectionGUID: TziuriAlR9uvrAzC7mI39g== X-CSE-MsgGUID: /KpWHAr1Q7GJI3FEZl25pw== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.10,186,1719903600"; d="scan'208";a="63415149" Received: from rchatre-mobl4.amr.corp.intel.com (HELO rchatre-mobl4.intel.com) ([10.125.111.220]) by fmviesa007-auth.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 29 Aug 2024 15:52:40 -0700 From: Reinette Chatre To: fenghua.yu@intel.com, shuah@kernel.org, tony.luck@intel.com, peternewman@google.com, babu.moger@amd.com, ilpo.jarvinen@linux.intel.com Cc: maciej.wieczor-retman@intel.com, reinette.chatre@intel.com, linux-kselftest@vger.kernel.org, linux-kernel@vger.kernel.org Subject: [PATCH 5/6] selftests/resctrl: Do not compare performance counters and resctrl at low bandwidth Date: Thu, 29 Aug 2024 15:52:31 -0700 Message-ID: <9bbefa3b9a62319698907d10e8b78f1b999c311b.1724970211.git.reinette.chatre@intel.com> X-Mailer: git-send-email 2.46.0 In-Reply-To: References: Precedence: bulk X-Mailing-List: linux-kselftest@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 The MBA test incrementally throttles memory bandwidth, each time followed by a comparison between the memory bandwidth observed by the performance counters and resctrl respectively. While a comparison between performance counters and resctrl is generally appropriate, they do not have an identical view of memory bandwidth. For example RAS features or memory performance features that generate memory traffic may drive accesses that are counted differently by performance counters and MBM respectively, for instance generating "overhead" traffic which is not counted against any specific RMID. As a ratio, this different view of memory bandwidth becomes more apparent at low memory bandwidths. It is not practical to enable/disable the various features that may generate memory bandwidth to give performance counters and resctrl an identical view. Instead, do not compare performance counters and resctrl view of memory bandwidth when the memory bandwidth is low. Bandwidth throttling behaves differently across platforms so it is not appropriate to drop measurement data simply based on the throttling level. Instead, use a threshold of 750MiB that has been observed to support adequate comparison between performance counters and resctrl. Signed-off-by: Reinette Chatre Reported-by: Ilpo Järvinen Signed-off-by: Reinette Chatre Reviewed-by: Ilpo Järvinen --- tools/testing/selftests/resctrl/mba_test.c | 7 +++++++ tools/testing/selftests/resctrl/resctrl.h | 6 ++++++ 2 files changed, 13 insertions(+) diff --git a/tools/testing/selftests/resctrl/mba_test.c b/tools/testing/selftests/resctrl/mba_test.c index cad473b81a64..204b9ac4b108 100644 --- a/tools/testing/selftests/resctrl/mba_test.c +++ b/tools/testing/selftests/resctrl/mba_test.c @@ -96,6 +96,13 @@ static bool show_mba_info(unsigned long *bw_imc, unsigned long *bw_resc) avg_bw_imc = sum_bw_imc / (NUM_OF_RUNS - 1); avg_bw_resc = sum_bw_resc / (NUM_OF_RUNS - 1); + if (avg_bw_imc < THROTTLE_THRESHOLD || avg_bw_resc < THROTTLE_THRESHOLD) { + ksft_print_msg("Bandwidth below threshold (%d MiB). Dropping results from MBA schemata %u.\n", + THROTTLE_THRESHOLD, + ALLOCATION_MAX - ALLOCATION_STEP * allocation); + break; + } + avg_diff = (float)labs(avg_bw_resc - avg_bw_imc) / avg_bw_imc; avg_diff_per = (int)(avg_diff * 100); diff --git a/tools/testing/selftests/resctrl/resctrl.h b/tools/testing/selftests/resctrl/resctrl.h index 0e5456165a6a..e65c5fb76b17 100644 --- a/tools/testing/selftests/resctrl/resctrl.h +++ b/tools/testing/selftests/resctrl/resctrl.h @@ -43,6 +43,12 @@ #define DEFAULT_SPAN (250 * MB) +/* + * Memory bandwidth (in MiB) below which the bandwidth comparisons + * between iMC and resctrl are considered unreliable. + */ +#define THROTTLE_THRESHOLD 750 + /* * user_params: User supplied parameters * @cpu: CPU number to which the benchmark will be bound to From patchwork Thu Aug 29 22:52:32 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Reinette Chatre X-Patchwork-Id: 13784009 Received: from mgamail.intel.com (mgamail.intel.com [192.198.163.12]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id C4A611BE854; Thu, 29 Aug 2024 22:52:44 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=192.198.163.12 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1724971966; cv=none; b=q+Ec9tWP/YzgLAKshcaYZf/Ydxb7cD+veBl0A7rXtdJ/8m8Dq4d25dR1cMEIjNF4fYqLMw4IK9te3UCjWSb2EfPUaDAnsf9jZ9aTxIRIaZw6YmH6wV8l37Ja+Jxd2vSY7L3lER1lI6BSvgIlvBvBObnvDLNbaY61Sut2xP8AZVU= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1724971966; c=relaxed/simple; bh=I0V/mXYSyqAaEqRosYfRH9ZnCyLxtliSkutcjvFE7bE=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=bQyxzsGRWfXz6SRkczCi+ut9HZvGgKZkqk26cWSW+NMB6UX6n/A0gFiH0Czy92ez83TJ1yh++1ex5HY9iRQP0/IlNCbq+3DGHfY4hyAGoZ6Wubrs/HBVF/+hu9D+Qq2OQMCVDU/HGKh64uhONezrzholeyEAzhCE5wlb22enOH0= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com; spf=pass smtp.mailfrom=intel.com; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b=LvedgY2D; arc=none smtp.client-ip=192.198.163.12 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=intel.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b="LvedgY2D" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1724971965; x=1756507965; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=I0V/mXYSyqAaEqRosYfRH9ZnCyLxtliSkutcjvFE7bE=; b=LvedgY2DMUOso6YKj/gO0zaSx0wTFMekO5Oqm4/nvU5/DSQ6kdBwBsCW WtV9nBTaXPCMEzLinFq15ba8crhOhzOd9QWgvoVBiLncVwSiWm7oRVEyk nk7dWfTMk1h4DULUbxMi28Pu1DR1AzcksMnh/mI8m8+KIpv81ZKNC6RCb iSCgGqNvgJMG8/Cq4E9/11uB19bNwFzXe/Fx+rMwHinxVJGKWR217dtVJ ddLUYd9+WDVd+AMJrcWLbMgk07mVazGAzHfhkxKlWakrpBxH501WoLmzJ dfUD0CPJD53N21LDVoh0BDN4CyLyIRp15CEgM+QU3OKTzYqTIvZ410aoW A==; X-CSE-ConnectionGUID: LoawZaj6RkCzdZ+LmRkm9A== X-CSE-MsgGUID: Ie6v+o++RvWhy7tl2PHsQQ== X-IronPort-AV: E=McAfee;i="6700,10204,11179"; a="27479176" X-IronPort-AV: E=Sophos;i="6.10,186,1719903600"; d="scan'208";a="27479176" Received: from fmviesa007.fm.intel.com ([10.60.135.147]) by fmvoesa106.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 29 Aug 2024 15:52:41 -0700 X-CSE-ConnectionGUID: BcpweYAKS1Kin8ptnRCQRA== X-CSE-MsgGUID: vHblY/NITC28FsoqGqp+dw== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.10,186,1719903600"; d="scan'208";a="63415152" Received: from rchatre-mobl4.amr.corp.intel.com (HELO rchatre-mobl4.intel.com) ([10.125.111.220]) by fmviesa007-auth.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 29 Aug 2024 15:52:41 -0700 From: Reinette Chatre To: fenghua.yu@intel.com, shuah@kernel.org, tony.luck@intel.com, peternewman@google.com, babu.moger@amd.com, ilpo.jarvinen@linux.intel.com Cc: maciej.wieczor-retman@intel.com, reinette.chatre@intel.com, linux-kselftest@vger.kernel.org, linux-kernel@vger.kernel.org Subject: [PATCH 6/6] selftests/resctrl: Keep results from first test run Date: Thu, 29 Aug 2024 15:52:32 -0700 Message-ID: <772813207becd105d309de43d1ab03e52bbd3091.1724970211.git.reinette.chatre@intel.com> X-Mailer: git-send-email 2.46.0 In-Reply-To: References: Precedence: bulk X-Mailing-List: linux-kselftest@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 The resctrl selftests drop the results from every first test run to avoid (per comment) "inaccurate due to monitoring setup transition phase" data. Previously inaccurate data resulted from workloads needing some time to "settle" and also the measurements themselves to account for earlier measurements to measure across needed timeframe. commit da50de0a92f3 ("selftests/resctrl: Calculate resctrl FS derived mem bw over sleep(1) only") ensured that measurements accurately measure just the time frame of interest. The default "fill_buf" benchmark since separated the buffer prepare phase from the benchmark run phase reducing the need for the tests themselves to accommodate the benchmark's "settle" time. With these enhancements there are no remaining portions needing to "settle" and the first test run can contribute to measurements. Signed-off-by: Reinette Chatre --- tools/testing/selftests/resctrl/cmt_test.c | 5 ++--- tools/testing/selftests/resctrl/mba_test.c | 6 +++--- tools/testing/selftests/resctrl/mbm_test.c | 10 +++------- 3 files changed, 8 insertions(+), 13 deletions(-) diff --git a/tools/testing/selftests/resctrl/cmt_test.c b/tools/testing/selftests/resctrl/cmt_test.c index f09d5dfab38c..85cb93d525b8 100644 --- a/tools/testing/selftests/resctrl/cmt_test.c +++ b/tools/testing/selftests/resctrl/cmt_test.c @@ -99,14 +99,13 @@ static int check_results(struct resctrl_val_param *param, size_t span, int no_of } /* Field 3 is llc occ resc value */ - if (runs > 0) - sum_llc_occu_resc += strtoul(token_array[3], NULL, 0); + sum_llc_occu_resc += strtoul(token_array[3], NULL, 0); runs++; } fclose(fp); return show_results_info(sum_llc_occu_resc, no_of_bits, span, - MAX_DIFF, MAX_DIFF_PERCENT, runs - 1, true); + MAX_DIFF, MAX_DIFF_PERCENT, runs, true); } static void cmt_test_cleanup(void) diff --git a/tools/testing/selftests/resctrl/mba_test.c b/tools/testing/selftests/resctrl/mba_test.c index 204b9ac4b108..dddf9bc04cfa 100644 --- a/tools/testing/selftests/resctrl/mba_test.c +++ b/tools/testing/selftests/resctrl/mba_test.c @@ -88,14 +88,14 @@ static bool show_mba_info(unsigned long *bw_imc, unsigned long *bw_resc) * The first run is discarded due to inaccurate value from * phase transition. */ - for (runs = NUM_OF_RUNS * allocation + 1; + for (runs = NUM_OF_RUNS * allocation; runs < NUM_OF_RUNS * allocation + NUM_OF_RUNS ; runs++) { sum_bw_imc += bw_imc[runs]; sum_bw_resc += bw_resc[runs]; } - avg_bw_imc = sum_bw_imc / (NUM_OF_RUNS - 1); - avg_bw_resc = sum_bw_resc / (NUM_OF_RUNS - 1); + avg_bw_imc = sum_bw_imc / NUM_OF_RUNS; + avg_bw_resc = sum_bw_resc / NUM_OF_RUNS; if (avg_bw_imc < THROTTLE_THRESHOLD || avg_bw_resc < THROTTLE_THRESHOLD) { ksft_print_msg("Bandwidth below threshold (%d MiB). Dropping results from MBA schemata %u.\n", THROTTLE_THRESHOLD, diff --git a/tools/testing/selftests/resctrl/mbm_test.c b/tools/testing/selftests/resctrl/mbm_test.c index 734bfa4f42b3..bbacba4ec195 100644 --- a/tools/testing/selftests/resctrl/mbm_test.c +++ b/tools/testing/selftests/resctrl/mbm_test.c @@ -22,17 +22,13 @@ show_bw_info(unsigned long *bw_imc, unsigned long *bw_resc, size_t span) int runs, ret, avg_diff_per; float avg_diff = 0; - /* - * Discard the first value which is inaccurate due to monitoring setup - * transition phase. - */ - for (runs = 1; runs < NUM_OF_RUNS ; runs++) { + for (runs = 0; runs < NUM_OF_RUNS ; runs++) { sum_bw_imc += bw_imc[runs]; sum_bw_resc += bw_resc[runs]; } - avg_bw_imc = sum_bw_imc / 4; - avg_bw_resc = sum_bw_resc / 4; + avg_bw_imc = sum_bw_imc / NUM_OF_RUNS; + avg_bw_resc = sum_bw_resc / NUM_OF_RUNS; avg_diff = (float)labs(avg_bw_resc - avg_bw_imc) / avg_bw_imc; avg_diff_per = (int)(avg_diff * 100);