From patchwork Wed May 11 22:02:43 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yonghong Song X-Patchwork-Id: 12846656 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 48BE4C433EF for ; Wed, 11 May 2022 22:02:58 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S236481AbiEKWC5 (ORCPT ); Wed, 11 May 2022 18:02:57 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:46478 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S234242AbiEKWC4 (ORCPT ); Wed, 11 May 2022 18:02:56 -0400 Received: from mx0a-00082601.pphosted.com (mx0a-00082601.pphosted.com [67.231.145.42]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 768F313F5B for ; Wed, 11 May 2022 15:02:55 -0700 (PDT) Received: from pps.filterd (m0109334.ppops.net [127.0.0.1]) by mx0a-00082601.pphosted.com (8.17.1.5/8.17.1.5) with ESMTP id 24BJfw3F007788 for ; Wed, 11 May 2022 15:02:54 -0700 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=fb.com; h=from : to : cc : subject : date : message-id : mime-version : content-transfer-encoding : content-type; s=facebook; bh=BpSYOXyg6wnT7RS6o6SrXwZrXLpUQf9GZTa0R/0RJu4=; b=FpVDJBxjffDE1AKMe7I0CIntf6qDOAVSVC2PEWPQh3vJ6TGEA1Ty7Wrt4WTe07EAqlop 7+si63cvXFCNU+h4fYrsTfrWc+clN8Blzo6yteK3OPRTzqwRqUsNlccs1yURUzSAbbKr YJbDN0/1mEUeaq8MAl1CEdlJz0425EhhCr0= Received: from maileast.thefacebook.com ([163.114.130.16]) by mx0a-00082601.pphosted.com (PPS) with ESMTPS id 3g0gat2qnr-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128 verify=NOT) for ; Wed, 11 May 2022 15:02:54 -0700 Received: from twshared8307.18.frc3.facebook.com (2620:10d:c0a8:1b::d) by mail.thefacebook.com (2620:10d:c0a8:82::c) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2375.24; Wed, 11 May 2022 15:02:53 -0700 Received: by devbig309.ftw3.facebook.com (Postfix, from userid 128203) id C9078A2C7776; Wed, 11 May 2022 15:02:43 -0700 (PDT) From: Yonghong Song To: Arnaldo Carvalho de Melo , CC: Alexei Starovoitov , Andrii Nakryiko , , Daniel Borkmann , Subject: [PATCH dwarves 1/2] libbpf: Sync with latest libbpf repo Date: Wed, 11 May 2022 15:02:43 -0700 Message-ID: <20220511220243.525215-1-yhs@fb.com> X-Mailer: git-send-email 2.30.2 MIME-Version: 1.0 X-FB-Internal: Safe X-Proofpoint-ORIG-GUID: BJfCiPD00-1_VZaFPeFP9volUXs0g_7T X-Proofpoint-GUID: BJfCiPD00-1_VZaFPeFP9volUXs0g_7T X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.205,Aquarius:18.0.858,Hydra:6.0.486,FMLib:17.11.64.514 definitions=2022-05-11_07,2022-05-11_01,2022-02-23_01 Precedence: bulk List-ID: X-Mailing-List: bpf@vger.kernel.org Sync up to commit 87dff0a2c775 (vmtest: allow building foreign debian rootfs). Signed-off-by: Yonghong Song --- lib/bpf | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/lib/bpf b/lib/bpf index 393a058..87dff0a 160000 --- a/lib/bpf +++ b/lib/bpf @@ -1 +1 @@ -Subproject commit 393a058d061d49d5c3055fa9eefafb4c0c31ccc3 +Subproject commit 87dff0a2c775c5943ca9233e69c81a25f2ed1a77 From patchwork Wed May 11 22:02:49 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yonghong Song X-Patchwork-Id: 12846657 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id A98F7C4332F for ; Wed, 11 May 2022 22:02:59 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S240084AbiEKWC6 (ORCPT ); Wed, 11 May 2022 18:02:58 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:46564 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S234242AbiEKWC5 (ORCPT ); Wed, 11 May 2022 18:02:57 -0400 Received: from mx0a-00082601.pphosted.com (mx0b-00082601.pphosted.com [67.231.153.30]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 21D773B7 for ; Wed, 11 May 2022 15:02:56 -0700 (PDT) Received: from pps.filterd (m0089730.ppops.net [127.0.0.1]) by m0089730.ppops.net (8.17.1.5/8.17.1.5) with ESMTP id 24BJg6ZX009576 for ; Wed, 11 May 2022 15:02:55 -0700 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=fb.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding : content-type; s=facebook; bh=wG+TyC5sQQzFrOs2Hg2vznQCUWMiDbTiNRH/a5wVJDM=; b=PWU6vbILFORfTkR5Zzic7RGnC6T6NHjggJz6jdF3fMlN3h7ZOJAcCcYgrgj5hLqbmzDH aNQil8IiRC6T0Y/Kp2BZbynjTYn5WBQJ1G0uodmIqezQr2NTKFCw5+1e4jQXTPh6VzrS lMiw0u2YSTAQ5+Z7DkQw6Y4xE7uKUZruIuE= Received: from maileast.thefacebook.com ([163.114.130.16]) by m0089730.ppops.net (PPS) with ESMTPS id 3fyx1h91jq-4 (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128 verify=NOT) for ; Wed, 11 May 2022 15:02:55 -0700 Received: from twshared8307.18.frc3.facebook.com (2620:10d:c0a8:1b::d) by mail.thefacebook.com (2620:10d:c0a8:83::5) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2375.24; Wed, 11 May 2022 15:02:53 -0700 Received: by devbig309.ftw3.facebook.com (Postfix, from userid 128203) id 0DA40A2C7786; Wed, 11 May 2022 15:02:49 -0700 (PDT) From: Yonghong Song To: Arnaldo Carvalho de Melo , CC: Alexei Starovoitov , Andrii Nakryiko , , Daniel Borkmann , Subject: [PATCH dwarves 2/2] btf_encoder: Normalize array index type for parallel dwarf loading case Date: Wed, 11 May 2022 15:02:49 -0700 Message-ID: <20220511220249.525908-1-yhs@fb.com> X-Mailer: git-send-email 2.30.2 In-Reply-To: <20220511220243.525215-1-yhs@fb.com> References: <20220511220243.525215-1-yhs@fb.com> MIME-Version: 1.0 X-FB-Internal: Safe X-Proofpoint-GUID: 0GytbrvbTT9uNqaJcEYsAoIPAJh96Aor X-Proofpoint-ORIG-GUID: 0GytbrvbTT9uNqaJcEYsAoIPAJh96Aor X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.205,Aquarius:18.0.858,Hydra:6.0.486,FMLib:17.11.64.514 definitions=2022-05-11_07,2022-05-11_01,2022-02-23_01 Precedence: bulk List-ID: X-Mailing-List: bpf@vger.kernel.org With latest llvm15 built kernel (make -j LLVM=1), I hit the following error when build selftests (make -C tools/testing/selftests/bpf -j LLVM=1): In file included from skeleton/pid_iter.bpf.c:3: .../selftests/bpf/tools/build/bpftool/vmlinux.h:84050:9: error: unknown type name '__builtin_va_list___2'; did you mean '__builtin_va_list'? typedef __builtin_va_list___2 va_list___2; ^~~~~~~~~~~~~~~~~~~~~ __builtin_va_list note: '__builtin_va_list' declared here In file included from skeleton/profiler.bpf.c:3: .../selftests/bpf/tools/build/bpftool/vmlinux.h:84050:9: error: unknown type name '__builtin_va_list__ _2'; did you mean '__builtin_va_list'? typedef __builtin_va_list___2 va_list___2; ^~~~~~~~~~~~~~~~~~~~~ __builtin_va_list note: '__builtin_va_list' declared here The error can be easily explained with after-dedup vmlinux btf: [21] INT 'int' size=4 bits_offset=0 nr_bits=32 encoding=SIGNED [2300] STRUCT '__va_list_tag' size=24 vlen=4 'gp_offset' type_id=2 bits_offset=0 'fp_offset' type_id=2 bits_offset=32 'overflow_arg_area' type_id=32 bits_offset=64 'reg_save_area' type_id=32 bits_offset=128 [2308] TYPEDEF 'va_list' type_id=2309 [2309] TYPEDEF '__builtin_va_list' type_id=2310 [2310] ARRAY '(anon)' type_id=2300 index_type_id=21 nr_elems=1 [5289] PTR '(anon)' type_id=2308 [158520] STRUCT 'warn_args' size=32 vlen=2 'fmt' type_id=14 bits_offset=0 'args' type_id=2308 bits_offset=64 [27299] INT '__ARRAY_SIZE_TYPE__' size=4 bits_offset=0 nr_bits=32 encoding=(none) [34590] TYPEDEF '__builtin_va_list' type_id=34591 [34591] ARRAY '(anon)' type_id=2300 index_type_id=27299 nr_elems=1 The typedef __builtin_va_list is a builtin type for the compiler. In the above case, two typedef __builtin_va_list are generated. The reason is due to different array index_type_id. This happened when pahole is running with more than one jobs when parsing dwarf and generating btfs. Function btf_encoder__encode_cu() is used to do btf encoding for each cu. The function will try to find an "int" type for the cu if it is available, otherwise, it will create a special type with name __ARRAY_SIZE_TYPE__. For example, file1: yes 'int' type file2: no 'int' type In serial mode, file1 is processed first, followed by file2. both will have 'int' type as the array index type since file2 will inherit the index type from file1. In parallel mode though, arrays in file1 will have index type 'int', and arrays in file2 wil have index type '__ARRAY_SIZE_TYPE__'. This will prevent some legitimate dedup and may have generated vmlinux.h having compilation error. This patch fixed the issue by normalizing all array_index types to be the first array_index type in the whole btf. Signed-off-by: Yonghong Song --- btf_encoder.c | 24 +++++++++++++++++++++--- btf_encoder.h | 2 +- pahole.c | 2 +- 3 files changed, 23 insertions(+), 5 deletions(-) diff --git a/btf_encoder.c b/btf_encoder.c index 1a42094..6164a3d 100644 --- a/btf_encoder.c +++ b/btf_encoder.c @@ -1056,17 +1056,35 @@ out: return err; } -int btf_encoder__encode(struct btf_encoder *encoder) +int btf_encoder__encode(struct btf_encoder *encoder, bool normalize_array_index_tid) { - int err; + int i, err, nr_types, index_type_id = 0; if (gobuffer__size(&encoder->percpu_secinfo) != 0) btf_encoder__add_datasec(encoder, PERCPU_SECTION); /* Empty file, nothing to do, so... done! */ - if (btf__type_cnt(encoder->btf) == 1) + nr_types = btf__type_cnt(encoder->btf); + if (nr_types == 1) return 0; + if (normalize_array_index_tid) { + for (i = 1; i < nr_types; i++) { + /* remove the 'const' qualifier so the index_type can be changed. */ + struct btf_type *t = (struct btf_type *)btf__type_by_id(encoder->btf, i); + struct btf_array *arr_info; + + if (!btf_is_array(t)) + continue; + + arr_info = btf_array(t); + if (index_type_id == 0) + index_type_id = arr_info->index_type; + else + arr_info->index_type = index_type_id; + } + } + if (btf__dedup(encoder->btf, NULL)) { fprintf(stderr, "%s: btf__dedup failed!\n", __func__); return -1; diff --git a/btf_encoder.h b/btf_encoder.h index 339fae2..9a4c79e 100644 --- a/btf_encoder.h +++ b/btf_encoder.h @@ -19,7 +19,7 @@ struct list_head; struct btf_encoder *btf_encoder__new(struct cu *cu, const char *detached_filename, struct btf *base_btf, bool skip_encoding_vars, bool force, bool gen_floats, bool verbose); void btf_encoder__delete(struct btf_encoder *encoder); -int btf_encoder__encode(struct btf_encoder *encoder); +int btf_encoder__encode(struct btf_encoder *encoder, bool normalize_array_index_tid); int btf_encoder__encode_cu(struct btf_encoder *encoder, struct cu *cu); diff --git a/pahole.c b/pahole.c index 78caa08..2c3b2ac 100644 --- a/pahole.c +++ b/pahole.c @@ -3530,7 +3530,7 @@ try_sole_arg_as_class_names: header = NULL; if (btf_encode && btf_encoder) { // maybe all CUs were filtered out and thus we don't have an encoder? - err = btf_encoder__encode(btf_encoder); + err = btf_encoder__encode(btf_encoder, conf_load.nr_jobs > 1); if (err) { fputs("Failed to encode BTF\n", stderr); goto out_cus_delete;