From patchwork Tue Aug 10 00:16:21 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yucong Sun X-Patchwork-Id: 12427653 X-Patchwork-Delegate: bpf@iogearbox.net Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-19.5 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, URIBL_BLOCKED,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id B7250C4320A for ; Tue, 10 Aug 2021 00:17:37 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 9A82060F38 for ; Tue, 10 Aug 2021 00:17:37 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S236124AbhHJAR5 (ORCPT ); Mon, 9 Aug 2021 20:17:57 -0400 Received: from mx0b-00082601.pphosted.com ([67.231.153.30]:18098 "EHLO mx0a-00082601.pphosted.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S231439AbhHJAR5 (ORCPT ); Mon, 9 Aug 2021 20:17:57 -0400 Received: from pps.filterd (m0001303.ppops.net [127.0.0.1]) by m0001303.ppops.net (8.16.0.43/8.16.0.43) with SMTP id 17A0D1pV012610 for ; Mon, 9 Aug 2021 17:17:36 -0700 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=fb.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding : content-type; s=facebook; bh=O8sHeNDWTZBptedNdbvKRS0XhkRECy7xRVDIHKw+9ZQ=; b=f76rD91bbelwXSVCY5AH+yZXYwRlCsg70V9+wK++4vcpfHCaz4HOyWDQufL7iT6wMPPk zcYYb9M0AaK61iAkozJsKEyPKrGEOCy3gYingCyeA08gVOgeeKUgTdig8+51dKcGaEx8 rvdeZ5XeA/YULmQx15K1OKwnsk52qtRImUs= Received: from mail.thefacebook.com ([163.114.132.120]) by m0001303.ppops.net with ESMTP id 3abbsqry8g-2 (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128 verify=NOT) for ; Mon, 09 Aug 2021 17:17:35 -0700 Received: from intmgw002.46.prn1.facebook.com (2620:10d:c085:208::f) by mail.thefacebook.com (2620:10d:c085:21d::5) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2176.2; Mon, 9 Aug 2021 17:17:34 -0700 Received: by devvm2661.vll0.facebook.com (Postfix, from userid 200310) id 2FF4C1E2CB68; Mon, 9 Aug 2021 17:17:25 -0700 (PDT) From: Yucong Sun To: CC: , , Yucong Sun Subject: [PATCH v2 bpf-next 1/5] Skip loading bpf_testmod when using -l to list tests. Date: Mon, 9 Aug 2021 17:16:21 -0700 Message-ID: <20210810001625.1140255-2-fallentree@fb.com> X-Mailer: git-send-email 2.30.2 In-Reply-To: <20210810001625.1140255-1-fallentree@fb.com> References: <20210810001625.1140255-1-fallentree@fb.com> MIME-Version: 1.0 X-FB-Internal: Safe X-FB-Source: Intern X-Proofpoint-GUID: BCxzVSPK7QJMs8r26RDVnyfyTdrDorlM X-Proofpoint-ORIG-GUID: BCxzVSPK7QJMs8r26RDVnyfyTdrDorlM X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:6.0.391,18.0.790 definitions=2021-08-09_09:2021-08-06,2021-08-09 signatures=0 X-Proofpoint-Spam-Details: rule=fb_default_notspam policy=fb_default score=0 bulkscore=0 suspectscore=0 adultscore=0 lowpriorityscore=0 clxscore=1015 mlxscore=0 priorityscore=1501 spamscore=0 malwarescore=0 mlxlogscore=833 phishscore=0 impostorscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2107140000 definitions=main-2108100000 X-FB-Internal: deliver Precedence: bulk List-ID: X-Mailing-List: bpf@vger.kernel.org X-Patchwork-Delegate: bpf@iogearbox.net This patch remove bpf_testmod load test when using "-l", making output cleaner. Signed-off-by: Yucong Sun --- tools/testing/selftests/bpf/test_progs.c | 10 ++++++---- 1 file changed, 6 insertions(+), 4 deletions(-) diff --git a/tools/testing/selftests/bpf/test_progs.c b/tools/testing/selftests/bpf/test_progs.c index 6f103106a39b..74dde0af1592 100644 --- a/tools/testing/selftests/bpf/test_progs.c +++ b/tools/testing/selftests/bpf/test_progs.c @@ -754,10 +754,12 @@ int main(int argc, char **argv) save_netns(); stdio_hijack(); - env.has_testmod = true; - if (load_bpf_testmod()) { - fprintf(env.stderr, "WARNING! Selftests relying on bpf_testmod.ko will be skipped.\n"); - env.has_testmod = false; + if (!env.list_test_names) { + env.has_testmod = true; + if (load_bpf_testmod()) { + fprintf(env.stderr, "WARNING! Selftests relying on bpf_testmod.ko will be skipped.\n"); + env.has_testmod = false; + } } for (i = 0; i < prog_test_cnt; i++) { struct prog_test_def *test = &prog_test_defs[i]; From patchwork Tue Aug 10 00:16:22 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yucong Sun X-Patchwork-Id: 12427643 X-Patchwork-Delegate: bpf@iogearbox.net Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-19.5 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, URIBL_BLOCKED,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 65866C4338F for ; Tue, 10 Aug 2021 00:17:34 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 3F69060F6F for ; Tue, 10 Aug 2021 00:17:34 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S236033AbhHJARy (ORCPT ); Mon, 9 Aug 2021 20:17:54 -0400 Received: from mx0b-00082601.pphosted.com ([67.231.153.30]:40980 "EHLO mx0a-00082601.pphosted.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S231439AbhHJARy (ORCPT ); Mon, 9 Aug 2021 20:17:54 -0400 Received: from pps.filterd (m0001303.ppops.net [127.0.0.1]) by m0001303.ppops.net (8.16.0.43/8.16.0.43) with SMTP id 17A0D9kM012854 for ; Mon, 9 Aug 2021 17:17:32 -0700 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=fb.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding : content-type; s=facebook; bh=ke8fywZq1kzC8TbePblyntQOf4q/GzDba0UZDrhrecw=; b=NvOPFQDt9304ybyFFSzYtxcLyu6bbchazxQwsZ6iQlBqjE3Cz5gdTKVV1Hk9KYlZ8oG7 paSq9uBmKpPgI3PBAYa2oGmwagtAyHqLahwdD1G5Asxkvco8UZdaP+sZIE1HNu8QhhqK A2WRqUkqfeDa5FZFfFrMmuChdVNr3nBKWj0= Received: from maileast.thefacebook.com ([163.114.130.16]) by m0001303.ppops.net with ESMTP id 3abbsqry8a-2 (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128 verify=NOT) for ; Mon, 09 Aug 2021 17:17:32 -0700 Received: from intmgw001.05.prn6.facebook.com (2620:10d:c0a8:1b::d) by mail.thefacebook.com (2620:10d:c0a8:83::5) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2176.2; Mon, 9 Aug 2021 17:17:31 -0700 Received: by devvm2661.vll0.facebook.com (Postfix, from userid 200310) id 348411E2CB6A; Mon, 9 Aug 2021 17:17:25 -0700 (PDT) From: Yucong Sun To: CC: , , Yucong Sun Subject: [PATCH v2 bpf-next 2/5] Add glob matching for test selector in test_progs. Date: Mon, 9 Aug 2021 17:16:22 -0700 Message-ID: <20210810001625.1140255-3-fallentree@fb.com> X-Mailer: git-send-email 2.30.2 In-Reply-To: <20210810001625.1140255-1-fallentree@fb.com> References: <20210810001625.1140255-1-fallentree@fb.com> MIME-Version: 1.0 X-FB-Internal: Safe X-FB-Source: Intern X-Proofpoint-GUID: aUlIBX1ZsoA01TCmORAMVvib7vB-YZ1_ X-Proofpoint-ORIG-GUID: aUlIBX1ZsoA01TCmORAMVvib7vB-YZ1_ X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:6.0.391,18.0.790 definitions=2021-08-09_09:2021-08-06,2021-08-09 signatures=0 X-Proofpoint-Spam-Details: rule=fb_default_notspam policy=fb_default score=0 bulkscore=0 suspectscore=0 adultscore=0 lowpriorityscore=0 clxscore=1015 mlxscore=0 priorityscore=1501 spamscore=0 malwarescore=0 mlxlogscore=999 phishscore=0 impostorscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2107140000 definitions=main-2108100000 X-FB-Internal: deliver Precedence: bulk List-ID: X-Mailing-List: bpf@vger.kernel.org X-Patchwork-Delegate: bpf@iogearbox.net This patch adds glob matching to test selector, it accepts simple glob pattern "*", "?", "[]" to match the test names to run. The glob matching function is copied from perf/util/string.c Signed-off-by: Yucong Sun --- tools/testing/selftests/bpf/test_progs.c | 94 +++++++++++++++++++++++- 1 file changed, 92 insertions(+), 2 deletions(-) diff --git a/tools/testing/selftests/bpf/test_progs.c b/tools/testing/selftests/bpf/test_progs.c index 74dde0af1592..c5bffd2e78ae 100644 --- a/tools/testing/selftests/bpf/test_progs.c +++ b/tools/testing/selftests/bpf/test_progs.c @@ -13,6 +13,96 @@ #include /* backtrace */ #include +// Copied from perf/util/string.c + +/* Character class matching */ +static bool __match_charclass(const char *pat, char c, const char **npat) +{ + bool complement = false, ret = true; + + if (*pat == '!') { + complement = true; + pat++; + } + if (*pat++ == c) /* First character is special */ + goto end; + + while (*pat && *pat != ']') { /* Matching */ + if (*pat == '-' && *(pat + 1) != ']') { /* Range */ + if (*(pat - 1) <= c && c <= *(pat + 1)) + goto end; + if (*(pat - 1) > *(pat + 1)) + goto error; + pat += 2; + } else if (*pat++ == c) + goto end; + } + if (!*pat) + goto error; + ret = false; + +end: + while (*pat && *pat != ']') /* Searching closing */ + pat++; + if (!*pat) + goto error; + *npat = pat + 1; + return complement ? !ret : ret; + +error: + return false; +} + +// Copied from perf/util/string.c +/* Glob/lazy pattern matching */ +static bool __match_glob(const char *str, const char *pat, bool ignore_space, + bool case_ins) +{ + while (*str && *pat && *pat != '*') { + if (ignore_space) { + /* Ignore spaces for lazy matching */ + if (isspace(*str)) { + str++; + continue; + } + if (isspace(*pat)) { + pat++; + continue; + } + } + if (*pat == '?') { /* Matches any single character */ + str++; + pat++; + continue; + } else if (*pat == '[') /* Character classes/Ranges */ + if (__match_charclass(pat + 1, *str, &pat)) { + str++; + continue; + } else + return false; + else if (*pat == '\\') /* Escaped char match as normal char */ + pat++; + if (case_ins) { + if (tolower(*str) != tolower(*pat)) + return false; + } else if (*str != *pat) + return false; + str++; + pat++; + } + /* Check wild card */ + if (*pat == '*') { + while (*pat == '*') + pat++; + if (!*pat) /* Tail wild card matches all */ + return true; + while (*str) + if (__match_glob(str++, pat, ignore_space, case_ins)) + return true; + } + return !*str && !*pat; +} + #define EXIT_NO_TEST 2 #define EXIT_ERR_SETUP_INFRA 3 @@ -55,12 +145,12 @@ static bool should_run(struct test_selector *sel, int num, const char *name) int i; for (i = 0; i < sel->blacklist.cnt; i++) { - if (strstr(name, sel->blacklist.strs[i])) + if (__match_glob(name, sel->blacklist.strs[i], false, false)) return false; } for (i = 0; i < sel->whitelist.cnt; i++) { - if (strstr(name, sel->whitelist.strs[i])) + if (__match_glob(name, sel->whitelist.strs[i], false, false)) return true; } From patchwork Tue Aug 10 00:16:23 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yucong Sun X-Patchwork-Id: 12427647 X-Patchwork-Delegate: bpf@iogearbox.net Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-19.5 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, URIBL_BLOCKED,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 2254BC4320E for ; Tue, 10 Aug 2021 00:17:36 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 05A836101D for ; Tue, 10 Aug 2021 00:17:35 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S236088AbhHJAR4 (ORCPT ); Mon, 9 Aug 2021 20:17:56 -0400 Received: from mx0a-00082601.pphosted.com ([67.231.145.42]:62626 "EHLO mx0a-00082601.pphosted.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S236124AbhHJARz (ORCPT ); Mon, 9 Aug 2021 20:17:55 -0400 Received: from pps.filterd (m0109334.ppops.net [127.0.0.1]) by mx0a-00082601.pphosted.com (8.16.0.43/8.16.0.43) with SMTP id 17A0DrWd027181 for ; Mon, 9 Aug 2021 17:17:34 -0700 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=fb.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding : content-type; s=facebook; bh=WZ3VN32/Y5TVisYde9Xm0GuNFBpxMSph6z5teIZnSfk=; b=m94RmmzAju04sYakkPgq5aRzkWcBz9ZgcEWV91MRqRD8XcI7vg8ieDeT1oDg7fLEjihI 3g8WYtj0Rr1d+Pk/6hyF5qfVW0y1qfM2rAcTRX7BfQepMttqZlBrk2PBt67k1vadOBvk BS3UXRhcBaO/FzGT2yZnK3Ox2OzdrV6b/+A= Received: from mail.thefacebook.com ([163.114.132.120]) by mx0a-00082601.pphosted.com with ESMTP id 3abdp30awf-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128 verify=NOT) for ; Mon, 09 Aug 2021 17:17:34 -0700 Received: from intmgw002.46.prn1.facebook.com (2620:10d:c085:108::4) by mail.thefacebook.com (2620:10d:c085:11d::6) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2176.2; Mon, 9 Aug 2021 17:17:34 -0700 Received: by devvm2661.vll0.facebook.com (Postfix, from userid 200310) id 395261E2CB6C; Mon, 9 Aug 2021 17:17:25 -0700 (PDT) From: Yucong Sun To: CC: , , Yucong Sun Subject: [PATCH v2 bpf-next 3/5] Correctly display subtest skip status Date: Mon, 9 Aug 2021 17:16:23 -0700 Message-ID: <20210810001625.1140255-4-fallentree@fb.com> X-Mailer: git-send-email 2.30.2 In-Reply-To: <20210810001625.1140255-1-fallentree@fb.com> References: <20210810001625.1140255-1-fallentree@fb.com> MIME-Version: 1.0 X-FB-Internal: Safe X-FB-Source: Intern X-Proofpoint-GUID: ceP_gLwLh-jA7QuoZj72P8bPstnFzFzC X-Proofpoint-ORIG-GUID: ceP_gLwLh-jA7QuoZj72P8bPstnFzFzC X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:6.0.391,18.0.790 definitions=2021-08-09_09:2021-08-06,2021-08-09 signatures=0 X-Proofpoint-Spam-Details: rule=fb_default_notspam policy=fb_default score=0 mlxscore=0 bulkscore=0 mlxlogscore=850 phishscore=0 priorityscore=1501 suspectscore=0 clxscore=1015 impostorscore=0 adultscore=0 malwarescore=0 lowpriorityscore=0 spamscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2107140000 definitions=main-2108100000 X-FB-Internal: deliver Precedence: bulk List-ID: X-Mailing-List: bpf@vger.kernel.org X-Patchwork-Delegate: bpf@iogearbox.net In skip_account(), test->skip_cnt is set to 0 at the end, this makes next print statement never display SKIP status for the subtest. This patch moves the accounting logic after the print statement, fixing the issue. Signed-off-by: Yucong Sun --- tools/testing/selftests/bpf/test_progs.c | 12 ++++++------ 1 file changed, 6 insertions(+), 6 deletions(-) diff --git a/tools/testing/selftests/bpf/test_progs.c b/tools/testing/selftests/bpf/test_progs.c index c5bffd2e78ae..82d012671552 100644 --- a/tools/testing/selftests/bpf/test_progs.c +++ b/tools/testing/selftests/bpf/test_progs.c @@ -238,18 +238,18 @@ void test__end_subtest() struct prog_test_def *test = env.test; int sub_error_cnt = test->error_cnt - test->old_error_cnt; - if (sub_error_cnt) - env.fail_cnt++; - else if (test->skip_cnt == 0) - env.sub_succ_cnt++; - skip_account(); - dump_test_log(test, sub_error_cnt); fprintf(env.stdout, "#%d/%d %s:%s\n", test->test_num, test->subtest_num, test->subtest_name, sub_error_cnt ? "FAIL" : (test->skip_cnt ? "SKIP" : "OK")); + if (sub_error_cnt) + env.fail_cnt++; + else if (test->skip_cnt == 0) + env.sub_succ_cnt++; + skip_account(); + free(test->subtest_name); test->subtest_name = NULL; } From patchwork Tue Aug 10 00:16:24 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yucong Sun X-Patchwork-Id: 12427649 X-Patchwork-Delegate: bpf@iogearbox.net Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-19.5 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, URIBL_BLOCKED,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 6D4FDC4338F for ; Tue, 10 Aug 2021 00:17:36 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 53DFD60F6F for ; Tue, 10 Aug 2021 00:17:36 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S236316AbhHJAR4 (ORCPT ); Mon, 9 Aug 2021 20:17:56 -0400 Received: from mx0a-00082601.pphosted.com ([67.231.145.42]:49760 "EHLO mx0a-00082601.pphosted.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231439AbhHJAR4 (ORCPT ); Mon, 9 Aug 2021 20:17:56 -0400 Received: from pps.filterd (m0044010.ppops.net [127.0.0.1]) by mx0a-00082601.pphosted.com (8.16.0.43/8.16.0.43) with SMTP id 17A0Fq5j028729 for ; Mon, 9 Aug 2021 17:17:35 -0700 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=fb.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding : content-type; s=facebook; bh=+heyorf5B1DnnuxN8u9u35O3pP4+xe948WdFEXa0Vbg=; b=EMJL5d6nw+PQPlOEbceT8tdTo9HgaCQmPSG2uuoJcGKJeXYgfLVTpwGCw4SrOps2R5gI yxaqgZVFqXYI5Lb2DpDtrcVS0K77JcijV6b/Uphdkb3x5bDc0zp8GeeHh5XAgsKqzoMJ CITe2UKAO5ICBXeBh2lrD9mPC1nf7koUeEM= Received: from maileast.thefacebook.com ([163.114.130.16]) by mx0a-00082601.pphosted.com with ESMTP id 3ab71pjx6b-3 (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128 verify=NOT) for ; Mon, 09 Aug 2021 17:17:35 -0700 Received: from intmgw001.27.prn2.facebook.com (2620:10d:c0a8:1b::d) by mail.thefacebook.com (2620:10d:c0a8:83::5) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2176.2; Mon, 9 Aug 2021 17:17:32 -0700 Received: by devvm2661.vll0.facebook.com (Postfix, from userid 200310) id 3DE1A1E2CB6E; Mon, 9 Aug 2021 17:17:25 -0700 (PDT) From: Yucong Sun To: CC: , , Yucong Sun Subject: [PATCH v2 bpf-next 4/5] Display test number when listing test names Date: Mon, 9 Aug 2021 17:16:24 -0700 Message-ID: <20210810001625.1140255-5-fallentree@fb.com> X-Mailer: git-send-email 2.30.2 In-Reply-To: <20210810001625.1140255-1-fallentree@fb.com> References: <20210810001625.1140255-1-fallentree@fb.com> MIME-Version: 1.0 X-FB-Internal: Safe X-FB-Source: Intern X-Proofpoint-GUID: 9oAalS4Pwo0nDOSTy_0sP8AtbAZWr8V1 X-Proofpoint-ORIG-GUID: 9oAalS4Pwo0nDOSTy_0sP8AtbAZWr8V1 X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:6.0.391,18.0.790 definitions=2021-08-09_09:2021-08-06,2021-08-09 signatures=0 X-Proofpoint-Spam-Details: rule=fb_default_notspam policy=fb_default score=0 suspectscore=0 priorityscore=1501 lowpriorityscore=0 mlxlogscore=750 mlxscore=0 bulkscore=0 phishscore=0 spamscore=0 impostorscore=0 clxscore=1015 malwarescore=0 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2107140000 definitions=main-2108100000 X-FB-Internal: deliver Precedence: bulk List-ID: X-Mailing-List: bpf@vger.kernel.org X-Patchwork-Delegate: bpf@iogearbox.net This patch adds tests number to the output when using "test_prog -l". Signed-off-by: Yucong Sun --- tools/testing/selftests/bpf/test_progs.c | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/tools/testing/selftests/bpf/test_progs.c b/tools/testing/selftests/bpf/test_progs.c index 82d012671552..5cc808992b00 100644 --- a/tools/testing/selftests/bpf/test_progs.c +++ b/tools/testing/selftests/bpf/test_progs.c @@ -867,7 +867,8 @@ int main(int argc, char **argv) } if (env.list_test_names) { - fprintf(env.stdout, "%s\n", test->test_name); + fprintf(env.stdout, "# %d %s\n", + test->test_num, test->test_name); env.succ_cnt++; continue; } From patchwork Tue Aug 10 00:16:25 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yucong Sun X-Patchwork-Id: 12427651 X-Patchwork-Delegate: bpf@iogearbox.net Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-19.5 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, URIBL_BLOCKED,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 5B042C432BE for ; Tue, 10 Aug 2021 00:17:37 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 36A9660F6F for ; Tue, 10 Aug 2021 00:17:37 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S236315AbhHJAR5 (ORCPT ); Mon, 9 Aug 2021 20:17:57 -0400 Received: from mx0b-00082601.pphosted.com ([67.231.153.30]:57940 "EHLO mx0a-00082601.pphosted.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S236124AbhHJAR4 (ORCPT ); Mon, 9 Aug 2021 20:17:56 -0400 Received: from pps.filterd (m0001303.ppops.net [127.0.0.1]) by m0001303.ppops.net (8.16.0.43/8.16.0.43) with SMTP id 17A0D1pU012610 for ; Mon, 9 Aug 2021 17:17:35 -0700 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=fb.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding : content-type; s=facebook; bh=WtyLkZn7i/eLfXS8/FLa7u0rPdNbli2nCleZ2TUQUN4=; b=WNuxIPSYQkiQC2c0LCgoCXnDbNasTog8MXAu6CVw5I65dFBbr7hqp0q5uEd2UipkeBzT ktl62I1xSgQnTxH62aXAYr29T43+M9GbtRH7LHY9udU+z7n33WQ2dfaQjGI0ghicq0CP 9/YyA+y/5q+XeintA/4p6B/6ZRtVVCa+8to= Received: from mail.thefacebook.com ([163.114.132.120]) by m0001303.ppops.net with ESMTP id 3abbsqry8g-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128 verify=NOT) for ; Mon, 09 Aug 2021 17:17:35 -0700 Received: from intmgw002.46.prn1.facebook.com (2620:10d:c085:108::4) by mail.thefacebook.com (2620:10d:c085:21d::5) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2176.2; Mon, 9 Aug 2021 17:17:34 -0700 Received: by devvm2661.vll0.facebook.com (Postfix, from userid 200310) id 42DC21E2CB70; Mon, 9 Aug 2021 17:17:25 -0700 (PDT) From: Yucong Sun To: CC: , , Yucong Sun Subject: [PATCH v2 bpf-next 5/5] Record all failed tests and output after the summary line. Date: Mon, 9 Aug 2021 17:16:25 -0700 Message-ID: <20210810001625.1140255-6-fallentree@fb.com> X-Mailer: git-send-email 2.30.2 In-Reply-To: <20210810001625.1140255-1-fallentree@fb.com> References: <20210810001625.1140255-1-fallentree@fb.com> MIME-Version: 1.0 X-FB-Internal: Safe X-FB-Source: Intern X-Proofpoint-GUID: h4YYVPn-r1MdOV_pUzokWpASRnCyAa0V X-Proofpoint-ORIG-GUID: h4YYVPn-r1MdOV_pUzokWpASRnCyAa0V X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:6.0.391,18.0.790 definitions=2021-08-09_09:2021-08-06,2021-08-09 signatures=0 X-Proofpoint-Spam-Details: rule=fb_default_notspam policy=fb_default score=0 bulkscore=0 suspectscore=0 adultscore=0 lowpriorityscore=0 clxscore=1015 mlxscore=0 priorityscore=1501 spamscore=0 malwarescore=0 mlxlogscore=885 phishscore=0 impostorscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2107140000 definitions=main-2108100000 X-FB-Internal: deliver Precedence: bulk List-ID: X-Mailing-List: bpf@vger.kernel.org X-Patchwork-Delegate: bpf@iogearbox.net This patch records all failed tests and subtests during the run, output them after the summary line, making it easier to identify failed tests in the long output. Signed-off-by: Yucong Sun --- tools/testing/selftests/bpf/test_progs.c | 25 +++++++++++++++++++++++- tools/testing/selftests/bpf/test_progs.h | 2 ++ 2 files changed, 26 insertions(+), 1 deletion(-) diff --git a/tools/testing/selftests/bpf/test_progs.c b/tools/testing/selftests/bpf/test_progs.c index 5cc808992b00..51a70031f07e 100644 --- a/tools/testing/selftests/bpf/test_progs.c +++ b/tools/testing/selftests/bpf/test_progs.c @@ -244,6 +244,11 @@ void test__end_subtest() test->test_num, test->subtest_num, test->subtest_name, sub_error_cnt ? "FAIL" : (test->skip_cnt ? "SKIP" : "OK")); + if (sub_error_cnt) { + fprintf(env.summary_errors, "#%d/%d %s: FAIL\n", + test->test_num, test->subtest_num, test->subtest_name); + } + if (sub_error_cnt) env.fail_cnt++; else if (test->skip_cnt == 0) @@ -816,6 +821,10 @@ int main(int argc, char **argv) .sa_flags = SA_RESETHAND, }; int err, i; + /* record errors to print after summary line */ + char *summary_errors_buf; + size_t summary_errors_cnt; + sigaction(SIGSEGV, &sigact, NULL); @@ -823,6 +832,9 @@ int main(int argc, char **argv) if (err) return err; + env.summary_errors = open_memstream( + &summary_errors_buf, &summary_errors_cnt); + err = cd_flavor_subdir(argv[0]); if (err) return err; @@ -891,6 +903,11 @@ int main(int argc, char **argv) test->test_num, test->test_name, test->error_cnt ? "FAIL" : "OK"); + if(test->error_cnt) { + fprintf(env.summary_errors, "#%d %s: FAIL\n", + test->test_num, test->test_name); + } + reset_affinity(); restore_netns(); if (test->need_cgroup_cleanup) @@ -908,9 +925,14 @@ int main(int argc, char **argv) if (env.list_test_names) goto out; - fprintf(stdout, "Summary: %d/%d PASSED, %d SKIPPED, %d FAILED\n", + fprintf(stdout, "\nSummary: %d/%d PASSED, %d SKIPPED, %d FAILED\n\n", env.succ_cnt, env.sub_succ_cnt, env.skip_cnt, env.fail_cnt); + fclose(env.summary_errors); + if(env.fail_cnt) { + fprintf(stdout, "%s", summary_errors_buf); + } + out: free_str_set(&env.test_selector.blacklist); free_str_set(&env.test_selector.whitelist); @@ -919,6 +941,7 @@ int main(int argc, char **argv) free_str_set(&env.subtest_selector.whitelist); free(env.subtest_selector.num_set); close(env.saved_netns_fd); + free(summary_errors_buf); if (env.succ_cnt + env.fail_cnt + env.skip_cnt == 0) return EXIT_NO_TEST; diff --git a/tools/testing/selftests/bpf/test_progs.h b/tools/testing/selftests/bpf/test_progs.h index c8c2bf878f67..63f4e534c6e5 100644 --- a/tools/testing/selftests/bpf/test_progs.h +++ b/tools/testing/selftests/bpf/test_progs.h @@ -82,6 +82,8 @@ struct test_env { int skip_cnt; /* skipped tests */ int saved_netns_fd; + + FILE* summary_errors; }; extern struct test_env env;