From patchwork Mon Sep 13 19:39:06 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yucong Sun X-Patchwork-Id: 12490691 X-Patchwork-Delegate: bpf@iogearbox.net Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-19.1 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id EE82FC433F5 for ; Mon, 13 Sep 2021 19:39:24 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id D2A43610FE for ; Mon, 13 Sep 2021 19:39:24 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S242627AbhIMTkj (ORCPT ); Mon, 13 Sep 2021 15:40:39 -0400 Received: from mx0a-00082601.pphosted.com ([67.231.145.42]:14950 "EHLO mx0a-00082601.pphosted.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S242192AbhIMTkc (ORCPT ); Mon, 13 Sep 2021 15:40:32 -0400 Received: from pps.filterd (m0109333.ppops.net [127.0.0.1]) by mx0a-00082601.pphosted.com (8.16.1.2/8.16.0.43) with SMTP id 18DH3k70027733 for ; Mon, 13 Sep 2021 12:39:16 -0700 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=fb.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding : content-type; s=facebook; bh=LUpdezofHAcX6ebpNw1fDf59yQYEvBNFN7zZfxYjuCM=; b=FEkXsiYNdtfFhFdX4exT1yvyj1/bUe4zaXiHIzMt+lGQxOlnUW2ATPqgl++1tBWKgXug QxGoxMLmC5MWk26dO6OAGGz/HIIJ5Ipgk0Clv5lZ3fj84pKc+tTpcPVOuWHoJCmJvWK8 DXUoOgrTRlL5lcYlldGNfzHwu4ZwhmvRNIU= Received: from mail.thefacebook.com ([163.114.132.120]) by mx0a-00082601.pphosted.com with ESMTP id 3b1vwnwfhv-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128 verify=NOT) for ; Mon, 13 Sep 2021 12:39:16 -0700 Received: from intmgw002.46.prn1.facebook.com (2620:10d:c085:208::f) by mail.thefacebook.com (2620:10d:c085:11d::7) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2308.14; Mon, 13 Sep 2021 12:39:15 -0700 Received: by devvm2661.vll0.facebook.com (Postfix, from userid 200310) id 76DB537AD02C; Mon, 13 Sep 2021 12:39:07 -0700 (PDT) From: Yucong Sun To: CC: , Yucong Sun Subject: [PATCH v4 bpf-next 3/3] selftests/bpf: pin some tests to worker 0 Date: Mon, 13 Sep 2021 12:39:06 -0700 Message-ID: <20210913193906.2813357-3-fallentree@fb.com> X-Mailer: git-send-email 2.30.2 In-Reply-To: <20210913193906.2813357-1-fallentree@fb.com> References: <20210913193906.2813357-1-fallentree@fb.com> MIME-Version: 1.0 X-FB-Internal: Safe X-FB-Source: Intern X-Proofpoint-ORIG-GUID: SrEU75M_VCPawzUlE5qMRU5_Fdf8VHsO X-Proofpoint-GUID: SrEU75M_VCPawzUlE5qMRU5_Fdf8VHsO X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.182.1,Aquarius:18.0.790,Hydra:6.0.391,FMLib:17.0.607.475 definitions=2021-09-13_09,2021-09-09_01,2020-04-07_01 X-Proofpoint-Spam-Details: rule=fb_default_notspam policy=fb_default score=0 clxscore=1015 impostorscore=0 mlxscore=0 malwarescore=0 adultscore=0 spamscore=0 bulkscore=0 mlxlogscore=999 suspectscore=0 priorityscore=1501 phishscore=0 lowpriorityscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2109030001 definitions=main-2109130117 X-FB-Internal: deliver Precedence: bulk List-ID: X-Mailing-List: bpf@vger.kernel.org X-Patchwork-Delegate: bpf@iogearbox.net From: Yucong Sun This patch adds a simple name list to pin some tests that fail to run in parallel to worker 0. On encountering these tests, all other threads will wait on a conditional variable, which worker 0 will signal once the tests has finished running. Additionally, before running the test, thread 0 also check and wait until all other threads has finished their work, to make sure the pinned test really are the only test running in the system. After this change, all tests should pass in '-j' mode. Signed-off-by: Yucong Sun --- tools/testing/selftests/bpf/test_progs.c | 109 ++++++++++++++++++++--- 1 file changed, 97 insertions(+), 12 deletions(-) diff --git a/tools/testing/selftests/bpf/test_progs.c b/tools/testing/selftests/bpf/test_progs.c index f0eeb17c348d..dc72b3f526a6 100644 --- a/tools/testing/selftests/bpf/test_progs.c +++ b/tools/testing/selftests/bpf/test_progs.c @@ -18,6 +18,16 @@ #include #include +char *TESTS_MUST_SERIALIZE[] = { + "netcnt", + "select_reuseport", + "sockmap_listen", + "tc_redirect", + "xdp_bonding", + "xdp_info", + NULL, +}; + /* Adapted from perf/util/string.c */ static bool glob_match(const char *str, const char *pat) { @@ -821,6 +831,7 @@ void crash_handler(int signum) int current_test_idx = 0; pthread_mutex_t current_test_lock; +pthread_cond_t wait_for_worker0 = PTHREAD_COND_INITIALIZER; struct test_result { int error_cnt; @@ -887,6 +898,29 @@ struct dispatch_data { int sock; }; +static const char *get_test_name(int idx) +{ + struct prog_test_def *test; + + test = &prog_test_defs[idx]; + return test->test_name; +} + +bool is_test_must_serialize(int idx) +{ + struct prog_test_def *test; + char **p; + + test = &prog_test_defs[idx]; + p = &TESTS_MUST_SERIALIZE[0]; + while (*p != NULL) { + if (strcmp(*p, test->test_name) == 0) + return true; + p++; + } + return false; +} + void *dispatch_thread(void *_data) { struct dispatch_data *data; @@ -901,6 +935,8 @@ void *dispatch_thread(void *_data) struct prog_test_def *test; struct test_result *result; + env.worker_current_test[data->idx] = -1; + /* grab a test */ { pthread_mutex_lock(¤t_test_lock); @@ -909,8 +945,31 @@ void *dispatch_thread(void *_data) pthread_mutex_unlock(¤t_test_lock); goto done; } + test_to_run = current_test_idx; - current_test_idx++; + + if (is_test_must_serialize(current_test_idx)) { + if (data->idx != 0) { + fprintf(stderr, "[%d]: Waiting for thread 0 to finish serialized test: %d.\n", + data->idx, current_test_idx + 1); + /* wait for worker 0 to pick this job up and finish */ + pthread_cond_wait(&wait_for_worker0, ¤t_test_lock); + pthread_mutex_unlock(¤t_test_lock); + goto next; + } else { + /* wait until all other worker has parked */ + for (int i = 1; i < env.workers; i++) { + if (env.worker_current_test[i] != -1) { + fprintf(stderr, "[%d]: Waiting for other threads to finish current test...\n", data->idx); + pthread_mutex_unlock(¤t_test_lock); + usleep(1 * 1000 * 1000); + goto next; + } + } + } + } else { + current_test_idx++; + } pthread_mutex_unlock(¤t_test_lock); } @@ -975,7 +1034,15 @@ void *dispatch_thread(void *_data) fclose(log_fd); log_fd = NULL; } + } /* wait for test done */ + + /* unblock all other dispatcher threads */ + if (is_test_must_serialize(test_to_run) && data->idx == 0) { + current_test_idx++; + pthread_cond_broadcast(&wait_for_worker0); } +next: + continue; } /* while (true) */ error: fprintf(stderr, "[%d]: Protocol/IO error: %s", data->idx, strerror(errno)); @@ -997,16 +1064,19 @@ int server_main(void) { pthread_t *dispatcher_threads; struct dispatch_data *data; + int all_finished = false; dispatcher_threads = calloc(sizeof(pthread_t), env.workers); data = calloc(sizeof(struct dispatch_data), env.workers); env.worker_current_test = calloc(sizeof(int), env.workers); + for (int i = 0; i < env.workers; i++) { int rc; data[i].idx = i; data[i].sock = env.worker_socks[i]; + env.worker_current_test[i] = -1; rc = pthread_create(&dispatcher_threads[i], NULL, dispatch_thread, &data[i]); if (rc < 0) { perror("Failed to launch dispatcher thread"); @@ -1015,17 +1085,27 @@ int server_main(void) } /* wait for all dispatcher to finish */ - for (int i = 0; i < env.workers; i++) { - while (true) { - struct timespec timeout = { - .tv_sec = time(NULL) + 5, - .tv_nsec = 0 - }; - if (pthread_timedjoin_np(dispatcher_threads[i], NULL, &timeout) != ETIMEDOUT) - break; - fprintf(stderr, "Still waiting for thread %d (test %d).\n", i, env.worker_current_test[i] + 1); + while (!all_finished) { + all_finished = true; + for (int i = 0; i < env.workers; i++) { + if (!dispatcher_threads[i]) + continue; + + if (pthread_tryjoin_np(dispatcher_threads[i], NULL) == EBUSY) { + all_finished = false; + if (env.worker_current_test[i] == -1) + fprintf(stderr, "Still waiting for thread %d (blocked by thread 0).\n", i); + else + fprintf(stderr, "Still waiting for thread %d (test #%d:%s).\n", + i, env.worker_current_test[i] + 1, + get_test_name(env.worker_current_test[i])); + } else { + dispatcher_threads[i] = 0; + } } + usleep(10 * 1000 * 1000); } + free(dispatcher_threads); free(env.worker_current_test); free(data); @@ -1100,8 +1180,11 @@ int worker_main(int sock) test_to_run = msg.u.message_do_test.num; - fprintf(stderr, "[%d]: Running test %d.\n", - env.worker_index, test_to_run + 1); + if (env.verbosity > VERBOSE_NONE) + fprintf(stderr, "[%d]: #%d:%s running.\n", + env.worker_index, + test_to_run + 1, + get_test_name(test_to_run)); test = &prog_test_defs[test_to_run]; @@ -1172,6 +1255,8 @@ int worker_main(int sock) env.log_buf = NULL; env.log_cnt = 0; } + fprintf(stderr, "[%d]: #%d:%s done.\n", + env.worker_index, test_to_run + 1, get_test_name(test_to_run)); break; } /* case MSG_DO_TEST */ default: