From patchwork Thu Jul 20 20:14:42 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alan Maguire X-Patchwork-Id: 13321021 Received: from lindbergh.monkeyblade.net (lindbergh.monkeyblade.net [23.128.96.19]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 248161BE75 for ; Thu, 20 Jul 2023 20:15:14 +0000 (UTC) Received: from mx0b-00069f02.pphosted.com (mx0b-00069f02.pphosted.com [205.220.177.32]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 5773C270D for ; Thu, 20 Jul 2023 13:15:13 -0700 (PDT) Received: from pps.filterd (m0246632.ppops.net [127.0.0.1]) by mx0b-00069f02.pphosted.com (8.17.1.19/8.17.1.19) with ESMTP id 36KFEGvZ021595; Thu, 20 Jul 2023 20:14:53 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding; s=corp-2023-03-30; bh=LyUB1F0in10jA2XieHb2yZ7pRxLRRuvJP4TqAChCLmw=; b=1dgIEGY6MF58bBUA55wc39rm/gPZZ5d/IUsQOt/VJV6rAbeGI7pht19Tj23CKW3/wNlz uVa6ZORI00z0IEzMnvJK2IHIqoKqX+OhKdrIwwDD4br0fi47qFrUc/K5Q758h7jZEuOz qrxROb7iLFRjAufmKZAc9J2+rsaclh/PwKv7Y3vdnKLI+/CL2elLBtqbR2j0OLZyJ8hj HewQIN5draGlcYpMsfVaJXkFBPR69pgOqnyq9SQLDwv+IkuFip7lU2Q+O5yi2NZ7LZTs GoAZcqEciAr0NmRAWgkCSXjsR1OL9MKW494dHxd9Zs7zgUMcb6UCbyqbO5Pdw6qb18nN 4Q== Received: from iadpaimrmta01.imrmtpd1.prodappiadaev1.oraclevcn.com (iadpaimrmta01.appoci.oracle.com [130.35.100.223]) by mx0b-00069f02.pphosted.com (PPS) with ESMTPS id 3ry1m4hcjy-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Thu, 20 Jul 2023 20:14:53 +0000 Received: from pps.filterd (iadpaimrmta01.imrmtpd1.prodappiadaev1.oraclevcn.com [127.0.0.1]) by iadpaimrmta01.imrmtpd1.prodappiadaev1.oraclevcn.com (8.17.1.19/8.17.1.19) with ESMTP id 36KJqns6000827; Thu, 20 Jul 2023 20:14:52 GMT Received: from pps.reinject (localhost [127.0.0.1]) by iadpaimrmta01.imrmtpd1.prodappiadaev1.oraclevcn.com (PPS) with ESMTPS id 3ruhw95tvp-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Thu, 20 Jul 2023 20:14:52 +0000 Received: from iadpaimrmta01.imrmtpd1.prodappiadaev1.oraclevcn.com (iadpaimrmta01.imrmtpd1.prodappiadaev1.oraclevcn.com [127.0.0.1]) by pps.reinject (8.17.1.5/8.17.1.5) with ESMTP id 36KKEmZY036089; Thu, 20 Jul 2023 20:14:51 GMT Received: from bpf.uk.oracle.com (dhcp-10-175-199-137.vpn.oracle.com [10.175.199.137]) by iadpaimrmta01.imrmtpd1.prodappiadaev1.oraclevcn.com (PPS) with ESMTP id 3ruhw95tr1-2; Thu, 20 Jul 2023 20:14:51 +0000 From: Alan Maguire To: acme@kernel.org Cc: andrii.nakryiko@gmail.com, ast@kernel.org, daniel@iogearbox.net, jolsa@kernel.org, martin.lau@linux.dev, song@kernel.org, yhs@fb.com, john.fastabend@gmail.com, kpsingh@kernel.org, sdf@google.com, haoluo@google.com, mykolal@fb.com, bpf@vger.kernel.org, Alan Maguire Subject: [RFC dwarves 1/2] dwarves: auto-detect maximum kind supported by vmlinux Date: Thu, 20 Jul 2023 21:14:42 +0100 Message-Id: <20230720201443.224040-2-alan.maguire@oracle.com> X-Mailer: git-send-email 2.39.3 In-Reply-To: <20230720201443.224040-1-alan.maguire@oracle.com> References: <20230720201443.224040-1-alan.maguire@oracle.com> Precedence: bulk X-Mailing-List: bpf@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.254,Aquarius:18.0.957,Hydra:6.0.591,FMLib:17.11.176.26 definitions=2023-07-20_10,2023-07-20_01,2023-05-22_02 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 spamscore=0 mlxlogscore=999 suspectscore=0 phishscore=0 bulkscore=0 mlxscore=0 malwarescore=0 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2306200000 definitions=main-2307200172 X-Proofpoint-GUID: 9nr5v5HwkIr-VpcrD-SaPqMqXQyF9JDY X-Proofpoint-ORIG-GUID: 9nr5v5HwkIr-VpcrD-SaPqMqXQyF9JDY X-Spam-Status: No, score=-2.1 required=5.0 tests=BAYES_00,DKIMWL_WL_MED, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF, RCVD_IN_DNSWL_BLOCKED,RCVD_IN_MSPIKE_H5,RCVD_IN_MSPIKE_WL, SPF_HELO_NONE,SPF_NONE,T_SCC_BODY_TEXT_LINE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net X-Patchwork-State: RFC When a newer pahole is run on an older kernel, it often knows about BTF kinds that the kernel does not support. This is a problem because the BTF generated is then embedded in the kernel image and read, and if unknown kinds are found, BTF handling fails and core BPF functionality is unavailable. The scripts/pahole-flags.sh script enumerates the various pahole options available associated with various versions of pahole, but the problem is what matters in the case of an older kernel is the set of kinds the kernel understands. Because recent features such as BTF_KIND_ENUM64 are added by default (and only skipped if --skip_encoding_btf_* is set), BTF will be created with these newer kinds that the older kernel cannot read. This can be fixed by stable-backporting --skip options, but this is cumbersome and would have to be done every time a new BTF kind is introduced. Here instead we pre-process the DWARF information associated with the target for BTF generation; if we find an enum with a BTF_KIND_MAX value in the DWARF associated with the object, we use that to determine the maximum BTF kind supported. Note that the enum representation of BTF kinds starts for the 5.16 kernel; prior to this The benefit of auto-detection is that no work is required for older kernels when new kinds are added, and --skip_encoding options are less needed. [1] https://github.com/oracle-samples/bpftune/issues/35 Signed-off-by: Alan Maguire --- btf_encoder.c | 12 ++++++++++++ dwarf_loader.c | 52 ++++++++++++++++++++++++++++++++++++++++++++++++++ dwarves.h | 2 ++ 3 files changed, 66 insertions(+) diff --git a/btf_encoder.c b/btf_encoder.c index 65f6e71..98c7529 100644 --- a/btf_encoder.c +++ b/btf_encoder.c @@ -1889,3 +1889,15 @@ struct btf *btf_encoder__btf(struct btf_encoder *encoder) { return encoder->btf; } + +void dwarves__set_btf_kind_max(struct conf_load *conf_load, int btf_kind_max) +{ + if (btf_kind_max < 0 || btf_kind_max >= BTF_KIND_MAX) + return; + if (btf_kind_max < BTF_KIND_DECL_TAG) + conf_load->skip_encoding_btf_decl_tag = true; + if (btf_kind_max < BTF_KIND_TYPE_TAG) + conf_load->skip_encoding_btf_type_tag = true; + if (btf_kind_max < BTF_KIND_ENUM64) + conf_load->skip_encoding_btf_enum64 = true; +} diff --git a/dwarf_loader.c b/dwarf_loader.c index ccf3194..8984043 100644 --- a/dwarf_loader.c +++ b/dwarf_loader.c @@ -3358,8 +3358,60 @@ static int __dwarf_cus__process_cus(struct dwarf_cus *dcus) return 0; } +/* Find enumeration value for BTF_KIND_MAX; replace conf_load->btf_kind_max with + * this value if found since it indicates that the target object does not know + * about kinds > its BTF_KIND_MAX value. This is valuable for kernel/module + * BTF where a newer pahole/libbpf operate on an older kernel which cannot + * parse some of the newer kinds pahole can generate. + */ +static void dwarf__find_btf_kind_max(struct dwarf_cus *dcus) +{ + struct conf_load *conf = dcus->conf; + uint8_t pointer_size, offset_size; + Dwarf_Off off = 0, noff; + size_t cuhl; + + while (dwarf_nextcu(dcus->dw, off, &noff, &cuhl, NULL, &pointer_size, &offset_size) == 0) { + Dwarf_Die die_mem; + Dwarf_Die *cu_die = dwarf_offdie(dcus->dw, off + cuhl, &die_mem); + Dwarf_Die child; + + if (cu_die == NULL) + break; + if (dwarf_child(cu_die, &child) == 0) { + Dwarf_Die *die = &child; + + do { + Dwarf_Die echild, *edie; + + if (dwarf_tag(die) != DW_TAG_enumeration_type || + !dwarf_haschildren(die) || + dwarf_child(die, &echild) != 0) + continue; + edie = &echild; + do { + const char *ename; + int btf_kind_max; + + if (dwarf_tag(edie) != DW_TAG_enumerator) + continue; + ename = attr_string(edie, DW_AT_name, conf); + if (!ename || strcmp(ename, "BTF_KIND_MAX") != 0) + continue; + btf_kind_max = attr_numeric(edie, DW_AT_const_value); + dwarves__set_btf_kind_max(conf, btf_kind_max); + return; + } while (dwarf_siblingof(edie, edie) == 0); + } while (dwarf_siblingof(die, die) == 0); + } + off = noff; + } +} + static int dwarf_cus__process_cus(struct dwarf_cus *dcus) { + dwarf__find_btf_kind_max(dcus); + if (dcus->conf->nr_jobs > 1) return dwarf_cus__threaded_process_cus(dcus); diff --git a/dwarves.h b/dwarves.h index eb1a6df..f4d9347 100644 --- a/dwarves.h +++ b/dwarves.h @@ -1480,4 +1480,6 @@ extern const char tabs[]; #define DW_TAG_skeleton_unit 0x4a #endif +void dwarves__set_btf_kind_max(struct conf_load *conf_load, int btf_kind_max); + #endif /* _DWARVES_H_ */ From patchwork Thu Jul 20 20:14:43 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alan Maguire X-Patchwork-Id: 13321022 Received: from lindbergh.monkeyblade.net (lindbergh.monkeyblade.net [23.128.96.19]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 23E511BE75 for ; Thu, 20 Jul 2023 20:15:17 +0000 (UTC) Received: from mx0b-00069f02.pphosted.com (mx0b-00069f02.pphosted.com [205.220.177.32]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 0BAAC270B for ; Thu, 20 Jul 2023 13:15:15 -0700 (PDT) Received: from pps.filterd (m0246630.ppops.net [127.0.0.1]) by mx0b-00069f02.pphosted.com (8.17.1.19/8.17.1.19) with ESMTP id 36KFEHH0025059; Thu, 20 Jul 2023 20:14:57 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding; s=corp-2023-03-30; bh=FIMKp7Mtt9rvWtMqIX4x8QIFlnWg42yPRAgam+DmULk=; b=r2b9clCS2etApIsfoDtYzWDclzZt62NsRopgox6FPtArfTjq4q8gwTsWrXGVbv2FwSTp /C+RwlUdCoasF3P5/dUEs42eeFEdHFTfw7QjAvDLaeC8dfsD4qZcC+hYnpW+hb2l3yW+ GeP4OWfFcuDirjXwGftZEZPZVzzGkl3579CclgCHErh8+qBCwYfU2vAZTFcO46SK7fzb jRLUlSw0p9YN4lkJJRC65BORTMeTVzg6p+auS4EQQ2qO0BRJ3a1/Ud6hq01dAyV1oL7X iGl3IEwFUJm7vmU7KiAUmfGGcrdKbkb0VWjqkORfIZ/8/Gns++pnV/3GX2DlToyKKEhF jw== Received: from iadpaimrmta01.imrmtpd1.prodappiadaev1.oraclevcn.com (iadpaimrmta01.appoci.oracle.com [130.35.100.223]) by mx0b-00069f02.pphosted.com (PPS) with ESMTPS id 3run78agds-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Thu, 20 Jul 2023 20:14:57 +0000 Received: from pps.filterd (iadpaimrmta01.imrmtpd1.prodappiadaev1.oraclevcn.com [127.0.0.1]) by iadpaimrmta01.imrmtpd1.prodappiadaev1.oraclevcn.com (8.17.1.19/8.17.1.19) with ESMTP id 36KJIuWB000815; Thu, 20 Jul 2023 20:14:56 GMT Received: from pps.reinject (localhost [127.0.0.1]) by iadpaimrmta01.imrmtpd1.prodappiadaev1.oraclevcn.com (PPS) with ESMTPS id 3ruhw95tyg-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Thu, 20 Jul 2023 20:14:56 +0000 Received: from iadpaimrmta01.imrmtpd1.prodappiadaev1.oraclevcn.com (iadpaimrmta01.imrmtpd1.prodappiadaev1.oraclevcn.com [127.0.0.1]) by pps.reinject (8.17.1.5/8.17.1.5) with ESMTP id 36KKEmZa036089; Thu, 20 Jul 2023 20:14:55 GMT Received: from bpf.uk.oracle.com (dhcp-10-175-199-137.vpn.oracle.com [10.175.199.137]) by iadpaimrmta01.imrmtpd1.prodappiadaev1.oraclevcn.com (PPS) with ESMTP id 3ruhw95tr1-3; Thu, 20 Jul 2023 20:14:55 +0000 From: Alan Maguire To: acme@kernel.org Cc: andrii.nakryiko@gmail.com, ast@kernel.org, daniel@iogearbox.net, jolsa@kernel.org, martin.lau@linux.dev, song@kernel.org, yhs@fb.com, john.fastabend@gmail.com, kpsingh@kernel.org, sdf@google.com, haoluo@google.com, mykolal@fb.com, bpf@vger.kernel.org, Alan Maguire Subject: [RFC dwarves 2/2] btf_encoder: learn BTF_KIND_MAX value from base BTF when generating split BTF Date: Thu, 20 Jul 2023 21:14:43 +0100 Message-Id: <20230720201443.224040-3-alan.maguire@oracle.com> X-Mailer: git-send-email 2.39.3 In-Reply-To: <20230720201443.224040-1-alan.maguire@oracle.com> References: <20230720201443.224040-1-alan.maguire@oracle.com> Precedence: bulk X-Mailing-List: bpf@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.254,Aquarius:18.0.957,Hydra:6.0.591,FMLib:17.11.176.26 definitions=2023-07-20_10,2023-07-20_01,2023-05-22_02 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 spamscore=0 mlxlogscore=999 suspectscore=0 phishscore=0 bulkscore=0 mlxscore=0 malwarescore=0 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2306200000 definitions=main-2307200172 X-Proofpoint-ORIG-GUID: QpnBZZAQggMC_0miTFxR3Oln9c1skDlI X-Proofpoint-GUID: QpnBZZAQggMC_0miTFxR3Oln9c1skDlI X-Spam-Status: No, score=-2.1 required=5.0 tests=BAYES_00,DKIMWL_WL_MED, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF, RCVD_IN_DNSWL_BLOCKED,RCVD_IN_MSPIKE_H5,RCVD_IN_MSPIKE_WL, SPF_HELO_NONE,SPF_NONE,T_SCC_BODY_TEXT_LINE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net X-Patchwork-State: RFC When creating module BTF, the module likely will not have a DWARF specificiation of BTF_KIND_MAX, so look for it in the base BTF. For vmlinux base BTF, the enumeration value is present, so the base BTF can be checked to limit BTF kind representation. Signed-off-by: Alan Maguire --- btf_encoder.c | 24 ++++++++++++++++++++++++ btf_encoder.h | 2 ++ pahole.c | 2 ++ 3 files changed, 28 insertions(+) diff --git a/btf_encoder.c b/btf_encoder.c index 98c7529..269ed48 100644 --- a/btf_encoder.c +++ b/btf_encoder.c @@ -1901,3 +1901,27 @@ void dwarves__set_btf_kind_max(struct conf_load *conf_load, int btf_kind_max) if (btf_kind_max < BTF_KIND_ENUM64) conf_load->skip_encoding_btf_enum64 = true; } + +void btf__set_btf_kind_max(struct conf_load *conf_load, struct btf *btf) +{ + __u32 id, type_cnt = btf__type_cnt(btf); + + for (id = 1; id < type_cnt; id++) { + const struct btf_type *t = btf__type_by_id(btf, id); + const struct btf_enum *e; + __u16 vlen, i; + + if (!t || !btf_is_enum(t)) + continue; + vlen = btf_vlen(t); + e = btf_enum(t); + for (i = 0; i < vlen; e++, i++) { + const char *name = btf__name_by_offset(btf, e->name_off); + + if (!name || strcmp(name, "BTF_KIND_MAX")) + continue; + dwarves__set_btf_kind_max(conf_load, e->val); + return; + } + } +} diff --git a/btf_encoder.h b/btf_encoder.h index 34516bb..e5e12ef 100644 --- a/btf_encoder.h +++ b/btf_encoder.h @@ -27,4 +27,6 @@ struct btf *btf_encoder__btf(struct btf_encoder *encoder); int btf_encoder__add_encoder(struct btf_encoder *encoder, struct btf_encoder *other); +void btf__set_btf_kind_max(struct conf_load *conf_load, struct btf *btf); + #endif /* _BTF_ENCODER_H_ */ diff --git a/pahole.c b/pahole.c index 6fc4ed6..542921a 100644 --- a/pahole.c +++ b/pahole.c @@ -3462,6 +3462,7 @@ int main(int argc, char *argv[]) base_btf_file, libbpf_get_error(conf_load.base_btf)); goto out; } + btf__set_btf_kind_max(&conf_load, conf_load.base_btf); if (!btf_encode && !ctf_encode) { // Force "btf" since a btf_base is being informed conf_load.format_path = "btf"; @@ -3505,6 +3506,7 @@ try_sole_arg_as_class_names: base_btf_file, libbpf_get_error(conf_load.base_btf)); goto out; } + btf__set_btf_kind_max(&conf_load, conf_load.base_btf); } }