From patchwork Wed Apr 24 15:47:54 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alan Maguire X-Patchwork-Id: 13642122 X-Patchwork-Delegate: bpf@iogearbox.net Received: from mx0b-00069f02.pphosted.com (mx0b-00069f02.pphosted.com [205.220.177.32]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id B240915EFAD for ; Wed, 24 Apr 2024 15:48:41 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=205.220.177.32 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1713973723; cv=none; b=cr6IqIXVrjYjK2oF1eFKZQ2YpuD1kRNnBeF/oJUNl738WD8X8u0BduM7hpOlQkLHb1oeVTwTtKldusnsK0VnlD+QWvLKPcnbYp9GsBlkGopZFlGdqEevPLO7Wkl/GuaYFwbOi9K9ei6oSdyw+0S/IY4qwcSRzOGdtDfHxIY56fI= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1713973723; c=relaxed/simple; bh=naBE7idfmkEzeDp0s3d/+P02p9pu6zewWCgBdysiGCQ=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=RC6Zo6QKUlV0rxm6Zi4dJA1tqVAQP6Q7s4FmsCEuBgEHq4mSFbPyBHUBrJQtlSDc/4ehPSNmmFUyen3bSooMkxBsh635KIpehtu+4R8tRxvCP8muu//z3yCec02xjdC847btDVnGLO/ihTePSnjfHHIbp0WhwIi7PeSoubIe4xE= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=oracle.com; spf=pass smtp.mailfrom=oracle.com; dkim=pass (2048-bit key) header.d=oracle.com header.i=@oracle.com header.b=ilKUJj+l; arc=none smtp.client-ip=205.220.177.32 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=oracle.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=oracle.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=oracle.com header.i=@oracle.com header.b="ilKUJj+l" Received: from pps.filterd (m0246631.ppops.net [127.0.0.1]) by mx0b-00069f02.pphosted.com (8.17.1.19/8.17.1.19) with ESMTP id 43OAAcAK014716; Wed, 24 Apr 2024 15:48:18 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding; s=corp-2023-11-20; bh=hUOrKSuivqj9lUqm8qadsBBlQvYCtgkCuDgzNx/LZ8Y=; b=ilKUJj+lqDlyLnjfoKoWHwSUki/c8HBvwn36TKmNGMV7wy68iaQukB4TjXbuwvAoWIhP 1zdHDE7ACY0QnzGXMxBf6oNzqPj2tAke23/diyz0EXR4XdgB7cozhKKwBOfm0xvoQxq3 xefT2amZbo6c0JBCCzlZ/sjf/jFKy3IGcLVMIQI1OT8lZRsLzjXrA0eVsQD6+0fnQQjM zrHUH0rQUG46aq+wDMZTBBSfYvltfcCMp+u6pvH1OhY5raNvvPA2y0gYRMV3dVpHxi9W rZLZGCKyx/m87t8oBiWz0PR8dkO6gdVNE4RzGfroPpDpwQ6X3CBt6IKRw2X4SFWQhHMN pA== Received: from iadpaimrmta02.imrmtpd1.prodappiadaev1.oraclevcn.com (iadpaimrmta02.appoci.oracle.com [147.154.18.20]) by mx0b-00069f02.pphosted.com (PPS) with ESMTPS id 3xm4g4gjcw-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Wed, 24 Apr 2024 15:48:18 +0000 Received: from pps.filterd (iadpaimrmta02.imrmtpd1.prodappiadaev1.oraclevcn.com [127.0.0.1]) by iadpaimrmta02.imrmtpd1.prodappiadaev1.oraclevcn.com (8.17.1.19/8.17.1.19) with ESMTP id 43OFK5jn025252; Wed, 24 Apr 2024 15:48:18 GMT Received: from pps.reinject (localhost [127.0.0.1]) by iadpaimrmta02.imrmtpd1.prodappiadaev1.oraclevcn.com (PPS) with ESMTPS id 3xm45fay1g-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Wed, 24 Apr 2024 15:48:17 +0000 Received: from iadpaimrmta02.imrmtpd1.prodappiadaev1.oraclevcn.com (iadpaimrmta02.imrmtpd1.prodappiadaev1.oraclevcn.com [127.0.0.1]) by pps.reinject (8.17.1.5/8.17.1.5) with ESMTP id 43OFmCoQ008769; Wed, 24 Apr 2024 15:48:17 GMT Received: from bpf.uk.oracle.com (dhcp-10-175-216-158.vpn.oracle.com [10.175.216.158]) by iadpaimrmta02.imrmtpd1.prodappiadaev1.oraclevcn.com (PPS) with ESMTP id 3xm45faxuq-2; Wed, 24 Apr 2024 15:48:17 +0000 From: Alan Maguire To: andrii@kernel.org, ast@kernel.org Cc: jolsa@kernel.org, acme@redhat.com, quentin@isovalent.com, eddyz87@gmail.com, mykolal@fb.com, daniel@iogearbox.net, martin.lau@linux.dev, song@kernel.org, yonghong.song@linux.dev, john.fastabend@gmail.com, kpsingh@kernel.org, sdf@google.com, haoluo@google.com, houtao1@huawei.com, bpf@vger.kernel.org, masahiroy@kernel.org, mcgrof@kernel.org, nathan@kernel.org, Alan Maguire Subject: [PATCH v2 bpf-next 01/13] libbpf: add support to btf__add_fwd() for ENUM64 Date: Wed, 24 Apr 2024 16:47:54 +0100 Message-Id: <20240424154806.3417662-2-alan.maguire@oracle.com> X-Mailer: git-send-email 2.39.3 In-Reply-To: <20240424154806.3417662-1-alan.maguire@oracle.com> References: <20240424154806.3417662-1-alan.maguire@oracle.com> Precedence: bulk X-Mailing-List: bpf@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293,Aquarius:18.0.1011,Hydra:6.0.650,FMLib:17.11.176.26 definitions=2024-04-24_13,2024-04-24_01,2023-05-22_02 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 suspectscore=0 mlxlogscore=999 mlxscore=0 phishscore=0 spamscore=0 malwarescore=0 adultscore=0 bulkscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2404010000 definitions=main-2404240063 X-Proofpoint-GUID: 0-iyE_rhLeIlfqO8fs2S3LQk2lrDcq1d X-Proofpoint-ORIG-GUID: 0-iyE_rhLeIlfqO8fs2S3LQk2lrDcq1d X-Patchwork-Delegate: bpf@iogearbox.net Forward declaration of BTF_KIND_ENUM64 is added by supporting BTF_FWD_ENUM64 as an enumerated value for btf_fwd_kind; an ENUM64 forward is an 8-byte signed enum64 with no values. Signed-off-by: Alan Maguire --- tools/lib/bpf/btf.c | 7 ++++++- tools/lib/bpf/btf.h | 1 + 2 files changed, 7 insertions(+), 1 deletion(-) diff --git a/tools/lib/bpf/btf.c b/tools/lib/bpf/btf.c index 2d0840ef599a..44afae098369 100644 --- a/tools/lib/bpf/btf.c +++ b/tools/lib/bpf/btf.c @@ -2418,7 +2418,7 @@ int btf__add_enum64_value(struct btf *btf, const char *name, __u64 value) * Append new BTF_KIND_FWD type with: * - *name*, non-empty/non-NULL name; * - *fwd_kind*, kind of forward declaration, one of BTF_FWD_STRUCT, - * BTF_FWD_UNION, or BTF_FWD_ENUM; + * BTF_FWD_UNION, BTF_FWD_ENUM or BTF_FWD_ENUM64; * Returns: * - >0, type ID of newly added BTF type; * - <0, on error. @@ -2446,6 +2446,11 @@ int btf__add_fwd(struct btf *btf, const char *name, enum btf_fwd_kind fwd_kind) * values; we also assume a standard 4-byte size for it */ return btf__add_enum(btf, name, sizeof(int)); + case BTF_FWD_ENUM64: + /* enum64 forward is similarly just an enum64 with no enum + * values; assume 8 byte size, signed. + */ + return btf__add_enum64(btf, name, sizeof(__u64), true); default: return libbpf_err(-EINVAL); } diff --git a/tools/lib/bpf/btf.h b/tools/lib/bpf/btf.h index 8e6880d91c84..47d3e00b25c7 100644 --- a/tools/lib/bpf/btf.h +++ b/tools/lib/bpf/btf.h @@ -194,6 +194,7 @@ enum btf_fwd_kind { BTF_FWD_STRUCT = 0, BTF_FWD_UNION = 1, BTF_FWD_ENUM = 2, + BTF_FWD_ENUM64 = 3, }; LIBBPF_API int btf__add_fwd(struct btf *btf, const char *name, enum btf_fwd_kind fwd_kind); From patchwork Wed Apr 24 15:47:55 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alan Maguire X-Patchwork-Id: 13642124 X-Patchwork-Delegate: bpf@iogearbox.net Received: from mx0b-00069f02.pphosted.com (mx0b-00069f02.pphosted.com [205.220.177.32]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 6D43115FD07 for ; Wed, 24 Apr 2024 15:48:47 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=205.220.177.32 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1713973729; cv=none; b=TqHYbMzU5ObQpVKyxRQHtc48dcPHJgj06U+VQalqlPCibhd+g+lsIUN2/RlFvhwkrpecJ4S0s103HBJt0KTKEV6uwMF6KzRiUTX3EdxWboGgl/JnxL0GRBZ4ZP/K4qHLhudCbVODFDWVM8aORV5jRCxOlUVgEaXtNjcJHxxwg5Q= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1713973729; c=relaxed/simple; bh=8zJxa+4QpvDS8yI/O0OKbkk4m3YhafEaMoKP+ULJI/k=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=WEvDv7YtT3g7BeehsYc2iQ7UMjhdAbOT2bSACMIO/rR7wxldDWcQdaW0CMw0jF1g61cxk3diva11S9J4bVtJa2lMJUdT5nbsmkxePYIZy93YyxL/kaGBxi1pWF80artvYL5DXlU43zgMgRx7yb5Wl0quXAktkBCXhplPLtpA9/0= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=oracle.com; spf=pass smtp.mailfrom=oracle.com; dkim=pass (2048-bit key) header.d=oracle.com header.i=@oracle.com header.b=DfmJBUpF; arc=none smtp.client-ip=205.220.177.32 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=oracle.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=oracle.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=oracle.com header.i=@oracle.com header.b="DfmJBUpF" Received: from pps.filterd (m0246632.ppops.net [127.0.0.1]) by mx0b-00069f02.pphosted.com (8.17.1.19/8.17.1.19) with ESMTP id 43OA7WEx023291; Wed, 24 Apr 2024 15:48:24 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding; s=corp-2023-11-20; bh=6KpLJCYkImANnhICCx+Igp525bhmqr5Aq1hg5Xq8pfk=; b=DfmJBUpFKXIgsX/7hOs4TJxCX1xXdF+bTOVaSFdl2HDWiNYqFNjw9H3JrDmgEgQOCB4u Eb91IJuNHnsgF4s2DTRnE1BHCHYTL5Xj7RoUaJd36A+2y8Oq3iDCvUdFxq7rmxjoSR3N oYieTzHqBFHu3x22/YlO8pnuIQnUvXVNF1KcOC9T9btzgmluG3YmpAQ2lDGBT4kKsQcI MXi0UzQ2hkLGCsWgIvK+W6NspaIhGVbIrMMFUgYHcHZvU7hpbLm/42wGfFsVxVdIw4ym DewM36t1GTSiUOshfsvyjDF49ofE/ARydLieYr67LjEnLc8ung6sjRgFIqH2O/duzCoO XQ== Received: from iadpaimrmta02.imrmtpd1.prodappiadaev1.oraclevcn.com (iadpaimrmta02.appoci.oracle.com [147.154.18.20]) by mx0b-00069f02.pphosted.com (PPS) with ESMTPS id 3xm5aurgtm-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Wed, 24 Apr 2024 15:48:23 +0000 Received: from pps.filterd (iadpaimrmta02.imrmtpd1.prodappiadaev1.oraclevcn.com [127.0.0.1]) by iadpaimrmta02.imrmtpd1.prodappiadaev1.oraclevcn.com (8.17.1.19/8.17.1.19) with ESMTP id 43OEs6xk025222; Wed, 24 Apr 2024 15:48:23 GMT Received: from pps.reinject (localhost [127.0.0.1]) by iadpaimrmta02.imrmtpd1.prodappiadaev1.oraclevcn.com (PPS) with ESMTPS id 3xm45fay6d-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Wed, 24 Apr 2024 15:48:23 +0000 Received: from iadpaimrmta02.imrmtpd1.prodappiadaev1.oraclevcn.com (iadpaimrmta02.imrmtpd1.prodappiadaev1.oraclevcn.com [127.0.0.1]) by pps.reinject (8.17.1.5/8.17.1.5) with ESMTP id 43OFmCoS008769; Wed, 24 Apr 2024 15:48:22 GMT Received: from bpf.uk.oracle.com (dhcp-10-175-216-158.vpn.oracle.com [10.175.216.158]) by iadpaimrmta02.imrmtpd1.prodappiadaev1.oraclevcn.com (PPS) with ESMTP id 3xm45faxuq-3; Wed, 24 Apr 2024 15:48:22 +0000 From: Alan Maguire To: andrii@kernel.org, ast@kernel.org Cc: jolsa@kernel.org, acme@redhat.com, quentin@isovalent.com, eddyz87@gmail.com, mykolal@fb.com, daniel@iogearbox.net, martin.lau@linux.dev, song@kernel.org, yonghong.song@linux.dev, john.fastabend@gmail.com, kpsingh@kernel.org, sdf@google.com, haoluo@google.com, houtao1@huawei.com, bpf@vger.kernel.org, masahiroy@kernel.org, mcgrof@kernel.org, nathan@kernel.org, Alan Maguire Subject: [PATCH v2 bpf-next 02/13] libbpf: add btf__distill_base() creating split BTF with distilled base BTF Date: Wed, 24 Apr 2024 16:47:55 +0100 Message-Id: <20240424154806.3417662-3-alan.maguire@oracle.com> X-Mailer: git-send-email 2.39.3 In-Reply-To: <20240424154806.3417662-1-alan.maguire@oracle.com> References: <20240424154806.3417662-1-alan.maguire@oracle.com> Precedence: bulk X-Mailing-List: bpf@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293,Aquarius:18.0.1011,Hydra:6.0.650,FMLib:17.11.176.26 definitions=2024-04-24_13,2024-04-24_01,2023-05-22_02 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 suspectscore=0 mlxlogscore=999 mlxscore=0 phishscore=0 spamscore=0 malwarescore=0 adultscore=0 bulkscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2404010000 definitions=main-2404240063 X-Proofpoint-ORIG-GUID: FBNQm422YcssWjNfK0022YXzg6qPk-du X-Proofpoint-GUID: FBNQm422YcssWjNfK0022YXzg6qPk-du X-Patchwork-Delegate: bpf@iogearbox.net To support more robust split BTF, adding supplemental context for the base BTF type ids that split BTF refers to is required. Without such references, a simple shuffling of base BTF type ids (without any other significant change) invalidates the split BTF. Here the attempt is made to store additional context to make split BTF more robust. This context comes in the form of distilled base BTF - this base BTF constitutes the minimal BTF representation needed to disambiguate split BTF references to base BTF. The rules are as follows: - INT, FLOAT are recorded in full. - if a named base BTF STRUCT or UNION is referred to from split BTF, it will be encoded either as a zero-member sized STRUCT/UNION (preserving size for later relocation checks) or as a named FWD. Only base BTF STRUCT/UNIONs that are embedded in split BTF STRUCT/UNIONs need to preserve size information, so a FWD representation will be used in most cases. - if an ENUM[64] is named, a ENUM[64] forward representation (an ENUM[64] with no values) is used. - if a STRUCT, UNION, ENUM or ENUM64 is not named, it is recorded in full. - base BTF reference types like CONST, RESTRICT, TYPEDEF, PTR are recorded as-is. Avoiding struct/union/enum/enum64 expansion is important to keep the distilled base BTF representation to a minimum size; however anonymous struct, union and enum[64] types are represented in full since type details are needed to disambiguate the reference - the name is not enough in those cases since there is no name. In practice these are rare; in sample cases where reference base BTF was generated for in-tree kernel modules, only a few were needed in distilled base BTF. These represent the anonymous struct/unions that are used by the module but were de-duplicated to use base vmlinux BTF ids instead. When successful, new representations of the distilled base BTF and new split BTF that refers to it are returned. Both need to be freed by the caller. So to take a simple example, with split BTF with a type referring to "struct sk_buff", we will generate base reference BTF with a FWD struct sk_buff, and the split BTF will refer to it instead. Tools like pahole can utilize such split BTF to popuate the .BTF section (split BTF) and an additional .BTF.base section. Then when the split BTF is loaded, the distilled base BTF can be used to relocate split BTF to reference the current - and possibly changed - base BTF. So for example if "struct sk_buff" was id 502 when the split BTF was originally generated, we can use the distilled base BTF to see that id 502 refers to a "struct sk_buff" and replace instances of id 502 with the current (relocated) base BTF sk_buff type id. Distilled base BTF is small; when building a kernel with all modules using distilled base BTF as a test, the average size for module distilled base BTF is 1555 bytes (standard deviation 1563). The maximum distilled base BTF size across ~2700 modules was 37895 bytes. Signed-off-by: Alan Maguire --- tools/lib/bpf/btf.c | 316 ++++++++++++++++++++++++++++++++++++++- tools/lib/bpf/btf.h | 20 +++ tools/lib/bpf/libbpf.map | 1 + 3 files changed, 331 insertions(+), 6 deletions(-) diff --git a/tools/lib/bpf/btf.c b/tools/lib/bpf/btf.c index 44afae098369..419cc4fa2e86 100644 --- a/tools/lib/bpf/btf.c +++ b/tools/lib/bpf/btf.c @@ -1771,9 +1771,8 @@ static int btf_rewrite_str(__u32 *str_off, void *ctx) return 0; } -int btf__add_type(struct btf *btf, const struct btf *src_btf, const struct btf_type *src_type) +static int btf_add_type(struct btf_pipe *p, const struct btf_type *src_type) { - struct btf_pipe p = { .src = src_btf, .dst = btf }; struct btf_type *t; int sz, err; @@ -1782,20 +1781,27 @@ int btf__add_type(struct btf *btf, const struct btf *src_btf, const struct btf_t return libbpf_err(sz); /* deconstruct BTF, if necessary, and invalidate raw_data */ - if (btf_ensure_modifiable(btf)) + if (btf_ensure_modifiable(p->dst)) return libbpf_err(-ENOMEM); - t = btf_add_type_mem(btf, sz); + t = btf_add_type_mem(p->dst, sz); if (!t) return libbpf_err(-ENOMEM); memcpy(t, src_type, sz); - err = btf_type_visit_str_offs(t, btf_rewrite_str, &p); + err = btf_type_visit_str_offs(t, btf_rewrite_str, p); if (err) return libbpf_err(err); - return btf_commit_type(btf, sz); + return btf_commit_type(p->dst, sz); +} + +int btf__add_type(struct btf *btf, const struct btf *src_btf, const struct btf_type *src_type) +{ + struct btf_pipe p = { .src = src_btf, .dst = btf }; + + return btf_add_type(&p, src_type); } static int btf_rewrite_type_ids(__u32 *type_id, void *ctx) @@ -5217,3 +5223,301 @@ int btf_ext_visit_str_offs(struct btf_ext *btf_ext, str_off_visit_fn visit, void return 0; } + +struct btf_distill_id { + int id; + bool embedded; /* true if id refers to a struct/union in base BTF + * that is embedded in a split BTF struct/union. + */ +}; + +struct btf_distill { + struct btf_pipe pipe; + struct btf_distill_id *ids; + __u32 query_id; + unsigned int nr_base_types; + unsigned int diff_id; +}; + +/* Check if a member of a split BTF struct/union refers to a base BTF + * struct/union. Members can be const/restrict/volatile/typedef + * reference types, but if a pointer is encountered, type is no longer + * considered embedded. + */ +static int btf_find_embedded_composite_type_ids(__u32 *id, void *ctx) +{ + struct btf_distill *dist = ctx; + const struct btf_type *t; + __u32 next_id = *id; + + do { + if (next_id == 0) + return 0; + t = btf_type_by_id(dist->pipe.src, next_id); + switch (btf_kind(t)) { + case BTF_KIND_CONST: + case BTF_KIND_RESTRICT: + case BTF_KIND_VOLATILE: + case BTF_KIND_TYPEDEF: + next_id = t->type; + break; + case BTF_KIND_ARRAY: { + struct btf_array *a = btf_array(t); + + next_id = a->type; + break; + } + case BTF_KIND_STRUCT: + case BTF_KIND_UNION: + dist->ids[next_id].embedded = next_id > 0 && + next_id <= dist->nr_base_types; + return 0; + default: + return 0; + } + + } while (next_id != 0); + + return 0; +} + +static bool btf_is_eligible_named_fwd(const struct btf_type *t) +{ + return (btf_is_composite(t) || btf_is_any_enum(t)) && t->name_off != 0; +} + +static int btf_add_distilled_type_ids(__u32 *id, void *ctx) +{ + struct btf_distill *dist = ctx; + struct btf_type *t = btf_type_by_id(dist->pipe.src, *id); + int ret; + + /* split BTF id, not needed */ + if (*id > dist->nr_base_types) + return 0; + /* already added ? */ + if (dist->ids[*id].id >= 0) + return 0; + dist->ids[*id].id = *id; + + /* only a subset of base BTF types should be referenced from split + * BTF; ensure nothing unexpected is referenced. + */ + switch (btf_kind(t)) { + case BTF_KIND_INT: + case BTF_KIND_FLOAT: + case BTF_KIND_ARRAY: + case BTF_KIND_STRUCT: + case BTF_KIND_UNION: + case BTF_KIND_TYPEDEF: + case BTF_KIND_ENUM: + case BTF_KIND_ENUM64: + case BTF_KIND_PTR: + case BTF_KIND_CONST: + case BTF_KIND_RESTRICT: + case BTF_KIND_VOLATILE: + case BTF_KIND_FUNC_PROTO: + break; + default: + pr_warn("unexpected reference to base type[%u] of kind [%u] when creating distilled base BTF.\n", + *id, btf_kind(t)); + return -EINVAL; + } + + /* struct/union members not needed, except for anonymous structs + * and unions, which we need since name won't help us determine + * matches; so if a named struct/union, no need to recurse + * into members. + */ + if (btf_is_eligible_named_fwd(t)) + return 0; + + /* ensure references in type are added also. */ + ret = btf_type_visit_type_ids(t, btf_add_distilled_type_ids, ctx); + if (ret < 0) + return ret; + return 0; +} + +/* All split BTF ids will be shifted downwards since there are less base BTF + * in distilled base BTF, and for those that refer to base BTF, we use the + * reference map to map from original base BTF to distilled base BTF id. + */ +static int btf_update_distilled_type_ids(__u32 *id, void *ctx) +{ + struct btf_distill *dist = ctx; + + if (*id >= dist->nr_base_types) + *id -= dist->diff_id; + else + *id = dist->ids[*id].id; + return 0; +} + +/* Create updated /split BTF with distilled base BTF; distilled base BTF + * consists of BTF information required to clarify the types that split + * BTF refers to, omitting unneeded details. Specifically it will contain + * base types and forward declarations of structs, unions and enumerated + * types, along with associated reference types like pointers, arrays etc. + * + * The only case where structs, unions or enumerated types are fully represented + * is when they are anonymous; in such cases, info about type content is needed + * to clarify type references. + * + * We return newly-created split BTF where the split BTf refers to a newly-created + * distilled base BTF. Both must be freed separately by the caller. + * + * When creating the BTF representation for a module and provided with the + * distilled_base option, pahole will create split BTF using this API, and store + * the distilled base BTF in the .BTF.base.distilled section. + */ +int btf__distill_base(const struct btf *src_btf, struct btf **new_base_btf, + struct btf **new_split_btf) +{ + struct btf *new_base = NULL, *new_split = NULL; + unsigned int n = btf__type_cnt(src_btf); + struct btf_distill dist = {}; + struct btf_type *t; + __u32 i, id = 0; + int ret = 0; + + /* src BTF must be split BTF. */ + if (!new_base_btf || !new_split_btf || !btf__base_btf(src_btf)) { + errno = EINVAL; + return -EINVAL; + } + new_base = btf__new_empty(); + if (!new_base) + return -ENOMEM; + dist.ids = calloc(n, sizeof(*dist.ids)); + if (!dist.ids) { + ret = -ENOMEM; + goto err_out; + } + for (i = 1; i < n; i++) + dist.ids[i].id = -1; + dist.pipe.src = src_btf; + dist.pipe.dst = new_base; + dist.pipe.str_off_map = hashmap__new(btf_dedup_identity_hash_fn, btf_dedup_equal_fn, NULL); + if (IS_ERR(dist.pipe.str_off_map)) { + ret = -ENOMEM; + goto err_out; + } + dist.nr_base_types = btf__type_cnt(btf__base_btf(src_btf)); + + /* Pass over src split BTF; generate the list of base BTF + * type ids it references; these will constitute our distilled + * base BTF set. + */ + for (i = src_btf->start_id; i < n; i++) { + t = (struct btf_type *)btf__type_by_id(src_btf, i); + + /* check if members of struct/union in split BTF refer to base BTF + * struct/union; if so, we will use an empty sized struct to represent + * it rather than a FWD because its size must match on later BTF + * relocation. + */ + if (btf_is_composite(t)) { + ret = btf_type_visit_type_ids(t, btf_find_embedded_composite_type_ids, + &dist); + if (ret < 0) + goto err_out; + } + ret = btf_type_visit_type_ids(t, btf_add_distilled_type_ids, &dist); + if (ret < 0) + goto err_out; + } + /* Next add types for each of the required references. */ + for (i = 1; i < src_btf->start_id; i++) { + if (dist.ids[i].id < 0) + continue; + t = btf_type_by_id(src_btf, i); + + if (dist.ids[i].embedded) { + /* If a named struct/union in base BTF is referenced as a type + * in split BTF without use of a pointer - i.e. as an embedded + * struct/union - add an empty struct/union preserving size + * since size must be consistent when relocating split and + * possibly changed base BTF. + */ + ret = btf_add_composite(new_base, btf_kind(t), + btf__name_by_offset(src_btf, t->name_off), + t->size); + } else if (btf_is_eligible_named_fwd(t)) { + enum btf_fwd_kind fwd_kind; + + /* If not embedded, use a fwd for named struct/unions since we + * can match via name without any other details. + */ + switch (btf_kind(t)) { + case BTF_KIND_STRUCT: + fwd_kind = BTF_FWD_STRUCT; + break; + case BTF_KIND_UNION: + fwd_kind = BTF_FWD_UNION; + break; + case BTF_KIND_ENUM: + fwd_kind = BTF_FWD_ENUM; + break; + case BTF_KIND_ENUM64: + fwd_kind = BTF_FWD_ENUM64; + break; + default: + pr_warn("unexpected kind [%u] when creating distilled base BTF.\n", + btf_kind(t)); + goto err_out; + } + ret = btf__add_fwd(new_base, btf__name_by_offset(src_btf, t->name_off), + fwd_kind); + } else { + ret = btf_add_type(&dist.pipe, t); + } + if (ret < 0) + goto err_out; + dist.ids[i].id = ++id; + } + /* now create new split BTF with distilled base BTF as its base; we end up with + * split BTF that has base BTF that represents enough about its base references + * to allow it to be relocated with the base BTF available. + */ + new_split = btf__new_empty_split(new_base); + if (!new_split_btf) { + ret = libbpf_get_error(new_split); + goto err_out; + } + + dist.pipe.dst = new_split; + /* all split BTF ids will be shifted downwards since there are less base BTF ids + * in distilled base BTF. + */ + dist.diff_id = dist.nr_base_types - btf__type_cnt(new_base); + + /* First add all split types */ + for (i = src_btf->start_id; i < n; i++) { + t = btf_type_by_id(src_btf, i); + ret = btf_add_type(&dist.pipe, t); + if (ret < 0) + goto err_out; + } + n = btf__type_cnt(new_split); + /* Now update base/split BTF ids. */ + for (i = 1; i < n; i++) { + t = btf_type_by_id(new_split, i); + + ret = btf_type_visit_type_ids(t, btf_update_distilled_type_ids, &dist); + if (ret < 0) + goto err_out; + } + free(dist.ids); + hashmap__free(dist.pipe.str_off_map); + *new_base_btf = new_base; + *new_split_btf = new_split; + return 0; +err_out: + free(dist.ids); + hashmap__free(dist.pipe.str_off_map); + btf__free(new_split); + btf__free(new_base); + errno = -ret; + return ret; +} diff --git a/tools/lib/bpf/btf.h b/tools/lib/bpf/btf.h index 47d3e00b25c7..025ed28b7fe8 100644 --- a/tools/lib/bpf/btf.h +++ b/tools/lib/bpf/btf.h @@ -107,6 +107,26 @@ LIBBPF_API struct btf *btf__new_empty(void); */ LIBBPF_API struct btf *btf__new_empty_split(struct btf *base_btf); +/** + * @brief **btf__distill_base()** creates new versions of the split BTF + * *src_btf* and its base BTF. The new base BTF will only contain the types + * needed to improve robustness of the split BTF to small changes in base BTF. + * When that split BTF is loaded against a (possibly changed) base, this + * distilled base BTF will help update references to that (possibly changed) + * base BTF. + * + * Both the new split and its associated new base BTF must be freed by + * the caller. + * + * If successful, 0 is returned and **new_base_btf** and **new_split_btf** + * will point at new base/split BTF. Both the new split and its associated + * new base BTF must be freed by the caller. + * + * A negative value is returned on error. + */ +LIBBPF_API int btf__distill_base(const struct btf *src_btf, struct btf **new_base_btf, + struct btf **new_split_btf); + LIBBPF_API struct btf *btf__parse(const char *path, struct btf_ext **btf_ext); LIBBPF_API struct btf *btf__parse_split(const char *path, struct btf *base_btf); LIBBPF_API struct btf *btf__parse_elf(const char *path, struct btf_ext **btf_ext); diff --git a/tools/lib/bpf/libbpf.map b/tools/lib/bpf/libbpf.map index c1ce8aa3520b..c4d9bd7d3220 100644 --- a/tools/lib/bpf/libbpf.map +++ b/tools/lib/bpf/libbpf.map @@ -420,6 +420,7 @@ LIBBPF_1.4.0 { LIBBPF_1.5.0 { global: bpf_program__attach_sockmap; + btf__distill_base; ring__consume_n; ring_buffer__consume_n; } LIBBPF_1.4.0; From patchwork Wed Apr 24 15:47:56 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alan Maguire X-Patchwork-Id: 13642126 X-Patchwork-Delegate: bpf@iogearbox.net Received: from mx0a-00069f02.pphosted.com (mx0a-00069f02.pphosted.com [205.220.165.32]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id BB4ED1607B8 for ; Wed, 24 Apr 2024 15:48:59 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=205.220.165.32 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1713973741; cv=none; b=RmiZcWGfoWJD1bSsP1urZFHg2wdbg6LTNHAn2raj6Adii7d7umQJw+vIPeebuxgsGruwY4hgSbHaoFpXi1MGTWDv1jdu0w648NjptnTUQtMvGKeAoaMREIom+70hVxkUZy0CJJlIPM4w2n+xTa2+LKizn6+R8RgCLF8ZCEkJ07k= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1713973741; c=relaxed/simple; bh=1Tm64U2VXDNxRKAQFFzWe7+YlLEjWBfe7UGqNk3HrFE=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=iJnlRKFwlpazyA2IDx5Vffw06Pyi9038+hfVPGpgZmh6e4tWFkM75u7jtejoFq8C7W3kjn/IYxKyJZZUKsleqgswOtEEOYmsYtMqhHQYLIQyXPzNpYX+KT/O0RGDP/NP5cFDKWWry6/aeYq2bmQPqHnSOqhomFf8/b0/6oi9j54= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=oracle.com; spf=pass smtp.mailfrom=oracle.com; dkim=pass (2048-bit key) header.d=oracle.com header.i=@oracle.com header.b=hIeMOx6S; arc=none smtp.client-ip=205.220.165.32 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=oracle.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=oracle.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=oracle.com header.i=@oracle.com header.b="hIeMOx6S" Received: from pps.filterd (m0246629.ppops.net [127.0.0.1]) by mx0b-00069f02.pphosted.com (8.17.1.19/8.17.1.19) with ESMTP id 43OFa4Bi009710; Wed, 24 Apr 2024 15:48:30 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding; s=corp-2023-11-20; bh=UAjEGmcXA0+jAPQBfwk/h4UgTHuvNnI0HhOIA5YT7tY=; b=hIeMOx6SRBT0ex4QUahICSB+CNBXYHO+YgP1lV83T9zgkpLp3xjyEmlNvnvLXzhzCa+r M2919Z+9OUCXgjQVXBsFMWr/siQ5JAhbmO/ZMi6JamSSqbOW5R7RDSToUco6/awFHB+c 2drt6vjqB0acZ5DdUyuifXzTRO9EDqb2LBl3znbQY+q4Z1SX06A+dhMmbst4M8fNDsWN 2pqPjsOpWHhnK3mvzEEHCzRg6c78Oys19WGG/ODZFk6uXsAXVJxRKfQiN/oYRDgqfpzK 1sReyCUH0RTlru6R4UUk13dd6hPbiDjLnaGZbkZNPUVQRQvB+TZmZfpt7TwplnoWvcGx Sw== Received: from iadpaimrmta02.imrmtpd1.prodappiadaev1.oraclevcn.com (iadpaimrmta02.appoci.oracle.com [147.154.18.20]) by mx0b-00069f02.pphosted.com (PPS) with ESMTPS id 3xm5kbs3tq-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Wed, 24 Apr 2024 15:48:30 +0000 Received: from pps.filterd (iadpaimrmta02.imrmtpd1.prodappiadaev1.oraclevcn.com [127.0.0.1]) by iadpaimrmta02.imrmtpd1.prodappiadaev1.oraclevcn.com (8.17.1.19/8.17.1.19) with ESMTP id 43OEguDx025273; Wed, 24 Apr 2024 15:48:28 GMT Received: from pps.reinject (localhost [127.0.0.1]) by iadpaimrmta02.imrmtpd1.prodappiadaev1.oraclevcn.com (PPS) with ESMTPS id 3xm45faybm-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Wed, 24 Apr 2024 15:48:28 +0000 Received: from iadpaimrmta02.imrmtpd1.prodappiadaev1.oraclevcn.com (iadpaimrmta02.imrmtpd1.prodappiadaev1.oraclevcn.com [127.0.0.1]) by pps.reinject (8.17.1.5/8.17.1.5) with ESMTP id 43OFmCoU008769; Wed, 24 Apr 2024 15:48:27 GMT Received: from bpf.uk.oracle.com (dhcp-10-175-216-158.vpn.oracle.com [10.175.216.158]) by iadpaimrmta02.imrmtpd1.prodappiadaev1.oraclevcn.com (PPS) with ESMTP id 3xm45faxuq-4; Wed, 24 Apr 2024 15:48:27 +0000 From: Alan Maguire To: andrii@kernel.org, ast@kernel.org Cc: jolsa@kernel.org, acme@redhat.com, quentin@isovalent.com, eddyz87@gmail.com, mykolal@fb.com, daniel@iogearbox.net, martin.lau@linux.dev, song@kernel.org, yonghong.song@linux.dev, john.fastabend@gmail.com, kpsingh@kernel.org, sdf@google.com, haoluo@google.com, houtao1@huawei.com, bpf@vger.kernel.org, masahiroy@kernel.org, mcgrof@kernel.org, nathan@kernel.org, Alan Maguire Subject: [PATCH v2 bpf-next 03/13] selftests/bpf: test distilled base, split BTF generation Date: Wed, 24 Apr 2024 16:47:56 +0100 Message-Id: <20240424154806.3417662-4-alan.maguire@oracle.com> X-Mailer: git-send-email 2.39.3 In-Reply-To: <20240424154806.3417662-1-alan.maguire@oracle.com> References: <20240424154806.3417662-1-alan.maguire@oracle.com> Precedence: bulk X-Mailing-List: bpf@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293,Aquarius:18.0.1011,Hydra:6.0.650,FMLib:17.11.176.26 definitions=2024-04-24_13,2024-04-24_01,2023-05-22_02 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 suspectscore=0 mlxlogscore=999 mlxscore=0 phishscore=0 spamscore=0 malwarescore=0 adultscore=0 bulkscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2404010000 definitions=main-2404240063 X-Proofpoint-GUID: hW0Xf4b7er45WY4soSZNvZWA6MrCiICt X-Proofpoint-ORIG-GUID: hW0Xf4b7er45WY4soSZNvZWA6MrCiICt X-Patchwork-Delegate: bpf@iogearbox.net Test generation of split+distilled base BTF, ensuring that - base BTF STRUCTs which are embedded in split BTF structs are represented as 0-member sized structs, allowing size checking - FWDs are used in place of full named struct/union declarations - FWDs are used in place of full named enum declarations - anonymous struct/unions are represented in full - anonymous enums are represented in full - types unreferenced from split BTF are not present in distilled base BTF Also test that with vmlinux BTF and split BTF based upon it, we only represent needed base types referenced from split BTF in distilled base. Signed-off-by: Alan Maguire --- .../selftests/bpf/prog_tests/btf_distill.c | 253 ++++++++++++++++++ 1 file changed, 253 insertions(+) create mode 100644 tools/testing/selftests/bpf/prog_tests/btf_distill.c diff --git a/tools/testing/selftests/bpf/prog_tests/btf_distill.c b/tools/testing/selftests/bpf/prog_tests/btf_distill.c new file mode 100644 index 000000000000..aae9aef68bd6 --- /dev/null +++ b/tools/testing/selftests/bpf/prog_tests/btf_distill.c @@ -0,0 +1,253 @@ +// SPDX-License-Identifier: GPL-2.0 +/* Copyright (c) 2024, Oracle and/or its affiliates. */ + +#include +#include +#include "btf_helpers.h" + +/* Fabricate base, split BTF with references to base types needed; then create + * split BTF with distilled base BTF and ensure expectations are met: + * - only referenced base types from split BTF are present + * - struct/union/enum are represented as FWDs unless anonymous, when they + * are represented in full, or if embedded in a split BTF struct, in which + * case they are represented by a STRUCT with specified size and vlen=0. + */ +static void test_distilled_base(void) +{ + struct btf *btf1 = NULL, *btf2 = NULL, *btf3 = NULL, *btf4 = NULL; + + btf1 = btf__new_empty(); + if (!ASSERT_OK_PTR(btf1, "empty_main_btf")) + return; + + btf__add_int(btf1, "int", 4, BTF_INT_SIGNED); /* [1] int */ + btf__add_ptr(btf1, 1); /* [2] ptr to int */ + btf__add_struct(btf1, "s1", 8); /* [3] struct s1 { */ + btf__add_field(btf1, "f1", 2, 0, 0); /* int *f1; */ + /* } */ + btf__add_struct(btf1, "", 12); /* [4] struct { */ + btf__add_field(btf1, "f1", 1, 0, 0); /* int f1; */ + btf__add_field(btf1, "f2", 3, 32, 0); /* struct s1 f2; */ + /* } */ + btf__add_int(btf1, "unsigned int", 4, 0); /* [5] unsigned int */ + btf__add_union(btf1, "u1", 12); /* [6] union u1 { */ + btf__add_field(btf1, "f1", 1, 0, 0); /* int f1; */ + btf__add_field(btf1, "f2", 2, 0, 0); /* int *f2; */ + /* } */ + btf__add_union(btf1, "", 4); /* [7] union { */ + btf__add_field(btf1, "f1", 1, 0, 0); /* int f1; */ + /* } */ + btf__add_enum(btf1, "e1", 4); /* [8] enum e1 { */ + btf__add_enum_value(btf1, "v1", 1); /* v1 = 1; */ + /* } */ + btf__add_enum(btf1, "", 4); /* [9] enum { */ + btf__add_enum_value(btf1, "av1", 2); /* av1 = 2; */ + /* } */ + btf__add_enum64(btf1, "e641", 8, true); /* [10] enum64 { */ + btf__add_enum64_value(btf1, "v1", 1024); /* v1 = 1024; */ + /* } */ + btf__add_enum64(btf1, "", 8, true); /* [11] enum64 { */ + btf__add_enum64_value(btf1, "v1", 1025); /* v1 = 1025; */ + /* } */ + btf__add_struct(btf1, "unneeded", 4); /* [12] struct unneeded { */ + btf__add_field(btf1, "f1", 1, 0, 0); /* int f1; */ + /* } */ + btf__add_struct(btf1, "embedded", 4); /* [13] struct embedded { */ + btf__add_field(btf1, "f1", 1, 0, 0); /* int f1; */ + /* } */ + btf__add_func_proto(btf1, 1); /* [14] int (*)(int *p1); */ + btf__add_func_param(btf1, "p1", 1); + + btf__add_array(btf1, 1, 1, 3); /* [15] int [3]; */ + + VALIDATE_RAW_BTF( + btf1, + "[1] INT 'int' size=4 bits_offset=0 nr_bits=32 encoding=SIGNED", + "[2] PTR '(anon)' type_id=1", + "[3] STRUCT 's1' size=8 vlen=1\n" + "\t'f1' type_id=2 bits_offset=0", + "[4] STRUCT '(anon)' size=12 vlen=2\n" + "\t'f1' type_id=1 bits_offset=0\n" + "\t'f2' type_id=3 bits_offset=32", + "[5] INT 'unsigned int' size=4 bits_offset=0 nr_bits=32 encoding=(none)", + "[6] UNION 'u1' size=12 vlen=2\n" + "\t'f1' type_id=1 bits_offset=0\n" + "\t'f2' type_id=2 bits_offset=0", + "[7] UNION '(anon)' size=4 vlen=1\n" + "\t'f1' type_id=1 bits_offset=0", + "[8] ENUM 'e1' encoding=UNSIGNED size=4 vlen=1\n" + "\t'v1' val=1", + "[9] ENUM '(anon)' encoding=UNSIGNED size=4 vlen=1\n" + "\t'av1' val=2", + "[10] ENUM64 'e641' encoding=SIGNED size=8 vlen=1\n" + "\t'v1' val=1024", + "[11] ENUM64 '(anon)' encoding=SIGNED size=8 vlen=1\n" + "\t'v1' val=1025", + "[12] STRUCT 'unneeded' size=4 vlen=1\n" + "\t'f1' type_id=1 bits_offset=0", + "[13] STRUCT 'embedded' size=4 vlen=1\n" + "\t'f1' type_id=1 bits_offset=0", + "[14] FUNC_PROTO '(anon)' ret_type_id=1 vlen=1\n" + "\t'p1' type_id=1", + "[15] ARRAY '(anon)' type_id=1 index_type_id=1 nr_elems=3"); + + btf2 = btf__new_empty_split(btf1); + if (!ASSERT_OK_PTR(btf2, "empty_split_btf")) + goto cleanup; + + btf__add_ptr(btf2, 3); /* [16] ptr to struct s1 */ + /* add ptr to struct anon */ + btf__add_ptr(btf2, 4); /* [17] ptr to struct (anon) */ + btf__add_const(btf2, 6); /* [18] const union u1 */ + btf__add_restrict(btf2, 7); /* [19] restrict union (anon) */ + btf__add_volatile(btf2, 8); /* [20] volatile enum e1 */ + btf__add_typedef(btf2, "et", 9); /* [21] typedef enum (anon) */ + btf__add_const(btf2, 10); /* [22] const enum64 e641 */ + btf__add_ptr(btf2, 11); /* [23] restrict enum64 (anon) */ + btf__add_struct(btf2, "with_embedded", 4); /* [24] struct with_embedded { */ + btf__add_field(btf2, "f1", 13, 0, 0); /* struct embedded f1; */ + /* } */ + btf__add_func(btf2, "fn", BTF_FUNC_STATIC, 14); /* [25] int fn(int p1); */ + btf__add_typedef(btf2, "arraytype", 15); /* [26] typedef int[3] foo; */ + + VALIDATE_RAW_BTF( + btf2, + "[1] INT 'int' size=4 bits_offset=0 nr_bits=32 encoding=SIGNED", + "[2] PTR '(anon)' type_id=1", + "[3] STRUCT 's1' size=8 vlen=1\n" + "\t'f1' type_id=2 bits_offset=0", + "[4] STRUCT '(anon)' size=12 vlen=2\n" + "\t'f1' type_id=1 bits_offset=0\n" + "\t'f2' type_id=3 bits_offset=32", + "[5] INT 'unsigned int' size=4 bits_offset=0 nr_bits=32 encoding=(none)", + "[6] UNION 'u1' size=12 vlen=2\n" + "\t'f1' type_id=1 bits_offset=0\n" + "\t'f2' type_id=2 bits_offset=0", + "[7] UNION '(anon)' size=4 vlen=1\n" + "\t'f1' type_id=1 bits_offset=0", + "[8] ENUM 'e1' encoding=UNSIGNED size=4 vlen=1\n" + "\t'v1' val=1", + "[9] ENUM '(anon)' encoding=UNSIGNED size=4 vlen=1\n" + "\t'av1' val=2", + "[10] ENUM64 'e641' encoding=SIGNED size=8 vlen=1\n" + "\t'v1' val=1024", + "[11] ENUM64 '(anon)' encoding=SIGNED size=8 vlen=1\n" + "\t'v1' val=1025", + "[12] STRUCT 'unneeded' size=4 vlen=1\n" + "\t'f1' type_id=1 bits_offset=0", + "[13] STRUCT 'embedded' size=4 vlen=1\n" + "\t'f1' type_id=1 bits_offset=0", + "[14] FUNC_PROTO '(anon)' ret_type_id=1 vlen=1\n" + "\t'p1' type_id=1", + "[15] ARRAY '(anon)' type_id=1 index_type_id=1 nr_elems=3", + "[16] PTR '(anon)' type_id=3", + "[17] PTR '(anon)' type_id=4", + "[18] CONST '(anon)' type_id=6", + "[19] RESTRICT '(anon)' type_id=7", + "[20] VOLATILE '(anon)' type_id=8", + "[21] TYPEDEF 'et' type_id=9", + "[22] CONST '(anon)' type_id=10", + "[23] PTR '(anon)' type_id=11", + "[24] STRUCT 'with_embedded' size=4 vlen=1\n" + "\t'f1' type_id=13 bits_offset=0", + "[25] FUNC 'fn' type_id=14 linkage=static", + "[26] TYPEDEF 'arraytype' type_id=15"); + + if (!ASSERT_EQ(0, btf__distill_base(btf2, &btf3, &btf4), + "distilled_base") || + !ASSERT_OK_PTR(btf3, "distilled_base") || + !ASSERT_OK_PTR(btf4, "distilled_split")) + goto cleanup; + + VALIDATE_RAW_BTF( + btf4, + "[1] INT 'int' size=4 bits_offset=0 nr_bits=32 encoding=SIGNED", + "[2] FWD 's1' fwd_kind=struct", + "[3] STRUCT '(anon)' size=12 vlen=2\n" + "\t'f1' type_id=1 bits_offset=0\n" + "\t'f2' type_id=2 bits_offset=32", + "[4] FWD 'u1' fwd_kind=union", + "[5] UNION '(anon)' size=4 vlen=1\n" + "\t'f1' type_id=1 bits_offset=0", + "[6] ENUM 'e1' encoding=UNSIGNED size=4 vlen=0", + "[7] ENUM '(anon)' encoding=UNSIGNED size=4 vlen=1\n" + "\t'av1' val=2", + "[8] ENUM64 'e641' encoding=SIGNED size=8 vlen=0", + "[9] ENUM64 '(anon)' encoding=SIGNED size=8 vlen=1\n" + "\t'v1' val=1025", + "[10] STRUCT 'embedded' size=4 vlen=0", + "[11] FUNC_PROTO '(anon)' ret_type_id=1 vlen=1\n" + "\t'p1' type_id=1", + "[12] ARRAY '(anon)' type_id=1 index_type_id=1 nr_elems=3", + "[13] PTR '(anon)' type_id=2", + "[14] PTR '(anon)' type_id=3", + "[15] CONST '(anon)' type_id=4", + "[16] RESTRICT '(anon)' type_id=5", + "[17] VOLATILE '(anon)' type_id=6", + "[18] TYPEDEF 'et' type_id=7", + "[19] CONST '(anon)' type_id=8", + "[20] PTR '(anon)' type_id=9", + "[21] STRUCT 'with_embedded' size=4 vlen=1\n" + "\t'f1' type_id=10 bits_offset=0", + "[22] FUNC 'fn' type_id=11 linkage=static", + "[23] TYPEDEF 'arraytype' type_id=12"); + +cleanup: + btf__free(btf4); + btf__free(btf3); + btf__free(btf2); + btf__free(btf1); +} + +/* create split reference BTF from vmlinux + split BTF with a few type references; + * ensure the resultant split reference BTF is as expected, containing only types + * needed to disambiguate references from split BTF. + */ +static void test_distilled_base_vmlinux(void) +{ + struct btf *split_btf = NULL, *vmlinux_btf = btf__load_vmlinux_btf(); + struct btf *split_dist = NULL, *base_dist = NULL; + __s32 int_id, sk_buff_id; + + if (!ASSERT_OK_PTR(vmlinux_btf, "load_vmlinux")) + return; + int_id = btf__find_by_name_kind(vmlinux_btf, "int", BTF_KIND_INT); + if (!ASSERT_GT(int_id, 0, "find_int")) + goto cleanup; + sk_buff_id = btf__find_by_name_kind(vmlinux_btf, "sk_buff", BTF_KIND_STRUCT); + if (!ASSERT_GT(sk_buff_id, 0, "find_sk_buff_id")) + goto cleanup; + split_btf = btf__new_empty_split(vmlinux_btf); + if (!ASSERT_OK_PTR(split_btf, "new_split")) + goto cleanup; + btf__add_typedef(split_btf, "myint", int_id); + btf__add_ptr(split_btf, sk_buff_id); + + if (!ASSERT_EQ(btf__distill_base(split_btf, &base_dist, &split_dist), 0, + "distill_vmlinux_base")) + goto cleanup; + + if (!ASSERT_OK_PTR(split_dist, "split_distilled") || + !ASSERT_OK_PTR(base_dist, "base_dist")) + goto cleanup; + VALIDATE_RAW_BTF( + split_dist, + "[1] INT 'int' size=4 bits_offset=0 nr_bits=32 encoding=SIGNED", + "[2] FWD 'sk_buff' fwd_kind=struct", + "[3] TYPEDEF 'myint' type_id=1", + "[4] PTR '(anon)' type_id=2"); + +cleanup: + btf__free(split_dist); + btf__free(base_dist); + btf__free(split_btf); + btf__free(vmlinux_btf); +} + +void test_btf_distill(void) +{ + if (test__start_subtest("distilled_base")) + test_distilled_base(); + if (test__start_subtest("distilled_base_vmlinux")) + test_distilled_base_vmlinux(); +} From patchwork Wed Apr 24 15:47:57 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alan Maguire X-Patchwork-Id: 13642125 X-Patchwork-Delegate: bpf@iogearbox.net Received: from mx0b-00069f02.pphosted.com (mx0b-00069f02.pphosted.com [205.220.177.32]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 7F30615FCE1 for ; Wed, 24 Apr 2024 15:48:59 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=205.220.177.32 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1713973741; cv=none; b=Y3jBfs/vGvL4h2USkfnYfC1ZocS6Bh/TEGSYrMVU9J2sZpl13VykCo+NSmnZXtzOmxrMytLpDR8iMormG8hD1whGoAGRRocoQWSQivipkFIpI4+O5xtOEdWFYzSkZ5hRN8eTBK0wwXP4txVz065eWjVDecGB0tLas/45AoSxdVc= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1713973741; c=relaxed/simple; bh=44Cc1/AIc/2DCzwvGnPSYOOrFK7dLVPG6mFMUjrAuPM=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=nnldcHfPg4YN4++Uv4O7mjxdSxwt8zJSbVUld2y3c3MqK58xHMytmDA/o61uvBba7ZvDZ61U3CC4m+FyQmfdfNyboaYOjZO5F969SBWeq/RBsD5AnvYfqRSlukFarW3672e0n2zsIIkn3Avv+/TR0uy/tAdLkojEmN2zX1SOt1g= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=oracle.com; spf=pass smtp.mailfrom=oracle.com; dkim=pass (2048-bit key) header.d=oracle.com header.i=@oracle.com header.b=GIA0aGN7; arc=none smtp.client-ip=205.220.177.32 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=oracle.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=oracle.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=oracle.com header.i=@oracle.com header.b="GIA0aGN7" Received: from pps.filterd (m0246631.ppops.net [127.0.0.1]) by mx0b-00069f02.pphosted.com (8.17.1.19/8.17.1.19) with ESMTP id 43OFeBA7014728; Wed, 24 Apr 2024 15:48:34 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding; s=corp-2023-11-20; bh=ZiA+gCy0CmRzmrhIzCQ9x+uW8vKTKOdiALEBrbBj0Qg=; b=GIA0aGN7T+Qe5L9vmZBW3qasZm+UOyjV5CvfFpMU5PLADsrJfG+PheJkN2mPV92lP+DC 2rDXw0xhsr89FmkYY2u4x1peHriYCyCPtQhug4h6UmWTEPIo80fJsfWr1m9/xLWV4Sue zUOu5gaozzfIscrwNu83b00LiETuzp9yY8aNW0XMX4EsfNht8+UqKvhRHFJxDQqbF17w vh5ZnABnmiVvoFoJbUksmXmXJ54rMWKeqgrZoQrHfwNrll8tHDzEQ2ULMxqaRatKWkj6 rH2gqqOEsu7dHO8oNXHbOA6R5OzRjdbSY+/N24BWppEOxYP709EHSb/rFjNdgtiLD/1T EQ== Received: from iadpaimrmta02.imrmtpd1.prodappiadaev1.oraclevcn.com (iadpaimrmta02.appoci.oracle.com [147.154.18.20]) by mx0b-00069f02.pphosted.com (PPS) with ESMTPS id 3xm4g4gjet-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Wed, 24 Apr 2024 15:48:34 +0000 Received: from pps.filterd (iadpaimrmta02.imrmtpd1.prodappiadaev1.oraclevcn.com [127.0.0.1]) by iadpaimrmta02.imrmtpd1.prodappiadaev1.oraclevcn.com (8.17.1.19/8.17.1.19) with ESMTP id 43OEhdd5025265; Wed, 24 Apr 2024 15:48:34 GMT Received: from pps.reinject (localhost [127.0.0.1]) by iadpaimrmta02.imrmtpd1.prodappiadaev1.oraclevcn.com (PPS) with ESMTPS id 3xm45fayhp-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Wed, 24 Apr 2024 15:48:33 +0000 Received: from iadpaimrmta02.imrmtpd1.prodappiadaev1.oraclevcn.com (iadpaimrmta02.imrmtpd1.prodappiadaev1.oraclevcn.com [127.0.0.1]) by pps.reinject (8.17.1.5/8.17.1.5) with ESMTP id 43OFmCoW008769; Wed, 24 Apr 2024 15:48:33 GMT Received: from bpf.uk.oracle.com (dhcp-10-175-216-158.vpn.oracle.com [10.175.216.158]) by iadpaimrmta02.imrmtpd1.prodappiadaev1.oraclevcn.com (PPS) with ESMTP id 3xm45faxuq-5; Wed, 24 Apr 2024 15:48:32 +0000 From: Alan Maguire To: andrii@kernel.org, ast@kernel.org Cc: jolsa@kernel.org, acme@redhat.com, quentin@isovalent.com, eddyz87@gmail.com, mykolal@fb.com, daniel@iogearbox.net, martin.lau@linux.dev, song@kernel.org, yonghong.song@linux.dev, john.fastabend@gmail.com, kpsingh@kernel.org, sdf@google.com, haoluo@google.com, houtao1@huawei.com, bpf@vger.kernel.org, masahiroy@kernel.org, mcgrof@kernel.org, nathan@kernel.org, Alan Maguire Subject: [PATCH v2 bpf-next 04/13] libbpf: add btf__parse_opts() API for flexible BTF parsing Date: Wed, 24 Apr 2024 16:47:57 +0100 Message-Id: <20240424154806.3417662-5-alan.maguire@oracle.com> X-Mailer: git-send-email 2.39.3 In-Reply-To: <20240424154806.3417662-1-alan.maguire@oracle.com> References: <20240424154806.3417662-1-alan.maguire@oracle.com> Precedence: bulk X-Mailing-List: bpf@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293,Aquarius:18.0.1011,Hydra:6.0.650,FMLib:17.11.176.26 definitions=2024-04-24_13,2024-04-24_01,2023-05-22_02 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 suspectscore=0 mlxlogscore=999 mlxscore=0 phishscore=0 spamscore=0 malwarescore=0 adultscore=0 bulkscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2404010000 definitions=main-2404240063 X-Proofpoint-GUID: WlGof1GaCRFcyretH1o51jEknOGPhZPt X-Proofpoint-ORIG-GUID: WlGof1GaCRFcyretH1o51jEknOGPhZPt X-Patchwork-Delegate: bpf@iogearbox.net Options cover existing parsing scenarios (ELF, raw, retrieving .BTF.ext) and also allow specification of the ELF section name containing BTF. This will allow consumers to retrieve BTF from .BTF.base sections (BTF_BASE_ELF_SEC) also. Signed-off-by: Alan Maguire --- tools/lib/bpf/btf.c | 50 ++++++++++++++++++++++++++++------------ tools/lib/bpf/btf.h | 32 +++++++++++++++++++++++++ tools/lib/bpf/libbpf.map | 1 + 3 files changed, 68 insertions(+), 15 deletions(-) diff --git a/tools/lib/bpf/btf.c b/tools/lib/bpf/btf.c index 419cc4fa2e86..9036c1dc45d0 100644 --- a/tools/lib/bpf/btf.c +++ b/tools/lib/bpf/btf.c @@ -1084,7 +1084,7 @@ struct btf *btf__new_split(const void *data, __u32 size, struct btf *base_btf) return libbpf_ptr(btf_new(data, size, base_btf)); } -static struct btf *btf_parse_elf(const char *path, struct btf *base_btf, +static struct btf *btf_parse_elf(const char *path, const char *btf_sec, struct btf *base_btf, struct btf_ext **btf_ext) { Elf_Data *btf_data = NULL, *btf_ext_data = NULL; @@ -1146,7 +1146,7 @@ static struct btf *btf_parse_elf(const char *path, struct btf *base_btf, idx, path); goto done; } - if (strcmp(name, BTF_ELF_SEC) == 0) { + if (strcmp(name, btf_sec) == 0) { btf_data = elf_getdata(scn, 0); if (!btf_data) { pr_warn("failed to get section(%d, %s) data from %s\n", @@ -1166,7 +1166,7 @@ static struct btf *btf_parse_elf(const char *path, struct btf *base_btf, } if (!btf_data) { - pr_warn("failed to find '%s' ELF section in %s\n", BTF_ELF_SEC, path); + pr_warn("failed to find '%s' ELF section in %s\n", btf_sec, path); err = -ENODATA; goto done; } @@ -1212,12 +1212,12 @@ static struct btf *btf_parse_elf(const char *path, struct btf *base_btf, struct btf *btf__parse_elf(const char *path, struct btf_ext **btf_ext) { - return libbpf_ptr(btf_parse_elf(path, NULL, btf_ext)); + return libbpf_ptr(btf_parse_elf(path, BTF_ELF_SEC, NULL, btf_ext)); } struct btf *btf__parse_elf_split(const char *path, struct btf *base_btf) { - return libbpf_ptr(btf_parse_elf(path, base_btf, NULL)); + return libbpf_ptr(btf_parse_elf(path, BTF_ELF_SEC, base_btf, NULL)); } static struct btf *btf_parse_raw(const char *path, struct btf *base_btf) @@ -1293,7 +1293,8 @@ struct btf *btf__parse_raw_split(const char *path, struct btf *base_btf) return libbpf_ptr(btf_parse_raw(path, base_btf)); } -static struct btf *btf_parse(const char *path, struct btf *base_btf, struct btf_ext **btf_ext) +static struct btf *btf_parse(const char *path, const char *btf_elf_sec, struct btf *base_btf, + struct btf_ext **btf_ext) { struct btf *btf; int err; @@ -1301,23 +1302,42 @@ static struct btf *btf_parse(const char *path, struct btf *base_btf, struct btf_ if (btf_ext) *btf_ext = NULL; - btf = btf_parse_raw(path, base_btf); - err = libbpf_get_error(btf); - if (!err) - return btf; - if (err != -EPROTO) - return ERR_PTR(err); - return btf_parse_elf(path, base_btf, btf_ext); + if (!btf_elf_sec) { + btf = btf_parse_raw(path, base_btf); + err = libbpf_get_error(btf); + if (!err) + return btf; + if (err != -EPROTO) + return ERR_PTR(err); + } + if (!btf_elf_sec) + btf_elf_sec = BTF_ELF_SEC; + + return btf_parse_elf(path, btf_elf_sec, base_btf, btf_ext); +} + +struct btf *btf__parse_opts(const char *path, struct btf_parse_opts *opts) +{ + struct btf *base_btf; + const char *btf_sec; + struct btf_ext **btf_ext; + + if (!OPTS_VALID(opts, btf_parse_opts)) + return libbpf_err_ptr(-EINVAL); + base_btf = OPTS_GET(opts, base_btf, NULL); + btf_sec = OPTS_GET(opts, btf_sec, NULL); + btf_ext = OPTS_GET(opts, btf_ext, NULL); + return libbpf_ptr(btf_parse(path, btf_sec, base_btf, btf_ext)); } struct btf *btf__parse(const char *path, struct btf_ext **btf_ext) { - return libbpf_ptr(btf_parse(path, NULL, btf_ext)); + return libbpf_ptr(btf_parse(path, NULL, NULL, btf_ext)); } struct btf *btf__parse_split(const char *path, struct btf *base_btf) { - return libbpf_ptr(btf_parse(path, base_btf, NULL)); + return libbpf_ptr(btf_parse(path, NULL, base_btf, NULL)); } static void *btf_get_raw_data(const struct btf *btf, __u32 *size, bool swap_endian); diff --git a/tools/lib/bpf/btf.h b/tools/lib/bpf/btf.h index 025ed28b7fe8..94dfdfdef617 100644 --- a/tools/lib/bpf/btf.h +++ b/tools/lib/bpf/btf.h @@ -18,6 +18,7 @@ extern "C" { #define BTF_ELF_SEC ".BTF" #define BTF_EXT_ELF_SEC ".BTF.ext" +#define BTF_BASE_ELF_SEC ".BTF.base" #define MAPS_ELF_SEC ".maps" struct btf; @@ -134,6 +135,37 @@ LIBBPF_API struct btf *btf__parse_elf_split(const char *path, struct btf *base_b LIBBPF_API struct btf *btf__parse_raw(const char *path); LIBBPF_API struct btf *btf__parse_raw_split(const char *path, struct btf *base_btf); +struct btf_parse_opts { + size_t sz; + /* use base BTF to parse split BTF */ + struct btf *base_btf; + /* retrieve optional .BTF.ext info */ + struct btf_ext **btf_ext; + /* BTF section name */ + const char *btf_sec; + size_t:0; +}; + +#define btf_parse_opts__last_field btf_sec + +/* @brief **btf__parse_opts()** parses BTF information from either a + * raw BTF file (*btf_sec* is NULL) or from the specified BTF section, + * also retrieving .BTF.ext info if *btf_ext* is non-NULL. If + * *base_btf* is specified, use it to parse split BTF from the + * specified location. + * + * @return new BTF object instance which has to be eventually freed with + * **btf__free()** + * + * On error, error-code-encoded-as-pointer is returned, not a NULL. To extract + * error code from such a pointer `libbpf_get_error()` should be used. If + * `libbpf_set_strict_mode(LIBBPF_STRICT_CLEAN_PTRS)` is enabled, NULL is + * returned on error instead. In both cases thread-local `errno` variable is + * always set to error code as well. + */ + +LIBBPF_API struct btf *btf__parse_opts(const char *path, struct btf_parse_opts *opts); + LIBBPF_API struct btf *btf__load_vmlinux_btf(void); LIBBPF_API struct btf *btf__load_module_btf(const char *module_name, struct btf *vmlinux_btf); diff --git a/tools/lib/bpf/libbpf.map b/tools/lib/bpf/libbpf.map index c4d9bd7d3220..a9151e31dfa9 100644 --- a/tools/lib/bpf/libbpf.map +++ b/tools/lib/bpf/libbpf.map @@ -421,6 +421,7 @@ LIBBPF_1.5.0 { global: bpf_program__attach_sockmap; btf__distill_base; + btf__parse_opts; ring__consume_n; ring_buffer__consume_n; } LIBBPF_1.4.0; From patchwork Wed Apr 24 15:47:58 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alan Maguire X-Patchwork-Id: 13642127 X-Patchwork-Delegate: bpf@iogearbox.net Received: from mx0a-00069f02.pphosted.com (mx0a-00069f02.pphosted.com [205.220.165.32]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id E95641422AF for ; Wed, 24 Apr 2024 15:49:08 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=205.220.165.32 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1713973750; cv=none; b=h1NZR+iUfqPK5Rq8oTZ/J57oGC9SynIKuen5oseXAlWp8iBUJm9Mz7/hKJO570L7iGjAmd6N0brgqB2k/snuADm89kBYBSGdVy97giXmwo0vHTza8jK8EDhST01J07zQ0KgPQhFAqJs17IU6jen9Vu7EJ8Js7WD2rRk5RUXIK70= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1713973750; c=relaxed/simple; bh=tnXTHe3xU4E7y/5Yhk4AbMHTlu41IBQUux2Fa7q+eb8=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=uTLiOyUvdzS6SMALcwzFl8E+c2IMR61S3FCTQYLoyfqnFp/enb6kKLgkovV3DehtuHUu+un+EAOI2rlL76fkj+4kt/XcjETI0ZTf8frFhC/GMqpHi6kNJaBnZ37NKl2JqTSPJA//DcOHEuhWilZFzGxBIYasNESOOTNCPtutwK4= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=oracle.com; spf=pass smtp.mailfrom=oracle.com; dkim=pass (2048-bit key) header.d=oracle.com header.i=@oracle.com header.b=Pdm7qB+V; arc=none smtp.client-ip=205.220.165.32 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=oracle.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=oracle.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=oracle.com header.i=@oracle.com header.b="Pdm7qB+V" Received: from pps.filterd (m0246617.ppops.net [127.0.0.1]) by mx0b-00069f02.pphosted.com (8.17.1.19/8.17.1.19) with ESMTP id 43OFmTnT010764; Wed, 24 Apr 2024 15:48:41 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding; s=corp-2023-11-20; bh=NbIzoDctJO5F58EQSIWbFu7MzKLE3C1mK+jaeDc3h5w=; b=Pdm7qB+VLhx5lRf0BNf0dXu6FOFr8xWv65319WJ0ZbjA0wUIkA7Y5ZiLgHKKK5HMwlRZ SQeo9M6GbN2xGQE7Jh/M1JKbtuZOkRKXduBDfhxdFLOwc78c1s/fdwnxyfnz28C/q7t9 I07YuOxgAeIkIpLe++kcmF47ZqKNJjnWF9dlwrV1lRmTHUtOHtgHl1jdbDtXmz6hNMvL kGQECu/z2M1VZ8YvsDyiZAqjaY5EYgLMcMjR5CTy3Zu6DcgWq4rBqA7TLJZDCQACBNR2 YHBORUigJA7tDFt/1xX9ask5A9KPCr3O/wSUSKy8NHtAwiPU3GeN0aDzCC1NtVMfRJJJ 4Q== Received: from iadpaimrmta02.imrmtpd1.prodappiadaev1.oraclevcn.com (iadpaimrmta02.appoci.oracle.com [147.154.18.20]) by mx0b-00069f02.pphosted.com (PPS) with ESMTPS id 3xm68vh4s5-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Wed, 24 Apr 2024 15:48:40 +0000 Received: from pps.filterd (iadpaimrmta02.imrmtpd1.prodappiadaev1.oraclevcn.com [127.0.0.1]) by iadpaimrmta02.imrmtpd1.prodappiadaev1.oraclevcn.com (8.17.1.19/8.17.1.19) with ESMTP id 43OFHTri025242; Wed, 24 Apr 2024 15:48:39 GMT Received: from pps.reinject (localhost [127.0.0.1]) by iadpaimrmta02.imrmtpd1.prodappiadaev1.oraclevcn.com (PPS) with ESMTPS id 3xm45fayq8-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Wed, 24 Apr 2024 15:48:39 +0000 Received: from iadpaimrmta02.imrmtpd1.prodappiadaev1.oraclevcn.com (iadpaimrmta02.imrmtpd1.prodappiadaev1.oraclevcn.com [127.0.0.1]) by pps.reinject (8.17.1.5/8.17.1.5) with ESMTP id 43OFmCoY008769; Wed, 24 Apr 2024 15:48:38 GMT Received: from bpf.uk.oracle.com (dhcp-10-175-216-158.vpn.oracle.com [10.175.216.158]) by iadpaimrmta02.imrmtpd1.prodappiadaev1.oraclevcn.com (PPS) with ESMTP id 3xm45faxuq-6; Wed, 24 Apr 2024 15:48:38 +0000 From: Alan Maguire To: andrii@kernel.org, ast@kernel.org Cc: jolsa@kernel.org, acme@redhat.com, quentin@isovalent.com, eddyz87@gmail.com, mykolal@fb.com, daniel@iogearbox.net, martin.lau@linux.dev, song@kernel.org, yonghong.song@linux.dev, john.fastabend@gmail.com, kpsingh@kernel.org, sdf@google.com, haoluo@google.com, houtao1@huawei.com, bpf@vger.kernel.org, masahiroy@kernel.org, mcgrof@kernel.org, nathan@kernel.org, Alan Maguire Subject: [PATCH v2 bpf-next 05/13] bpftool: support displaying raw split BTF using base BTF section as base Date: Wed, 24 Apr 2024 16:47:58 +0100 Message-Id: <20240424154806.3417662-6-alan.maguire@oracle.com> X-Mailer: git-send-email 2.39.3 In-Reply-To: <20240424154806.3417662-1-alan.maguire@oracle.com> References: <20240424154806.3417662-1-alan.maguire@oracle.com> Precedence: bulk X-Mailing-List: bpf@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293,Aquarius:18.0.1011,Hydra:6.0.650,FMLib:17.11.176.26 definitions=2024-04-24_13,2024-04-24_01,2023-05-22_02 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 suspectscore=0 mlxlogscore=999 mlxscore=0 phishscore=0 spamscore=0 malwarescore=0 adultscore=0 bulkscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2404010000 definitions=main-2404240063 X-Proofpoint-GUID: ks4thJGurJkld7_9ZgyzpQSSui0luenO X-Proofpoint-ORIG-GUID: ks4thJGurJkld7_9ZgyzpQSSui0luenO X-Patchwork-Delegate: bpf@iogearbox.net If no base BTF can be found, fall back to checking for the .BTF.base section and use it to display split BTF. Signed-off-by: Alan Maguire --- tools/bpf/bpftool/btf.c | 9 +++++++++ 1 file changed, 9 insertions(+) diff --git a/tools/bpf/bpftool/btf.c b/tools/bpf/bpftool/btf.c index 91fcb75babe3..2e8bd2c9f0a3 100644 --- a/tools/bpf/bpftool/btf.c +++ b/tools/bpf/bpftool/btf.c @@ -631,6 +631,15 @@ static int do_dump(int argc, char **argv) base = get_vmlinux_btf_from_sysfs(); btf = btf__parse_split(*argv, base ?: base_btf); + /* Finally check for presence of base BTF section */ + if (!btf && !base && !base_btf) { + LIBBPF_OPTS(btf_parse_opts, optp); + + optp.btf_sec = BTF_BASE_ELF_SEC; + base_btf = btf__parse_opts(*argv, &optp); + if (base_btf) + btf = btf__parse_split(*argv, base_btf); + } if (!btf) { err = -errno; p_err("failed to load BTF from %s: %s", From patchwork Wed Apr 24 15:47:59 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alan Maguire X-Patchwork-Id: 13642128 X-Patchwork-Delegate: bpf@iogearbox.net Received: from mx0a-00069f02.pphosted.com (mx0a-00069f02.pphosted.com [205.220.165.32]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 142001422AF for ; Wed, 24 Apr 2024 15:49:13 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=205.220.165.32 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1713973754; cv=none; b=eXjO1ysWuPPp/XlaPtpc75UVCPa6oYnEv4vHHeb694z2nE7TbYCGO67pbdCYhJsB6HBdnMRO6rr1qAA6J1xU9U9Ym8mb06rj3VrJS+WrYdW+wg1alJLuVd5NfygZKNRRruFD7785Pn5ro9MrX1j5BHjM3a0BvgU8duZc5KZXQos= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1713973754; c=relaxed/simple; bh=TO14thCzwXY6q/gOV/R7Vt8NAwvkuhfkNyg1N71U4Bg=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=Pi0HdesufsqfiQlVZsicofmJeUyFdxxI0d92FLGAOZSICvNe89zJqSiAKC/kwyQ6U/fw5dWSUXcY7pAn17URBupW2ccjgaMg29oC5LzG6FesLAWGXuWHZ8UvzADMvAOpkuo/8TXuxIRFOQHWXaZZGTt4f+lvEtKnIdv690ukP5Q= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=oracle.com; spf=pass smtp.mailfrom=oracle.com; dkim=pass (2048-bit key) header.d=oracle.com header.i=@oracle.com header.b=K698+oNR; arc=none smtp.client-ip=205.220.165.32 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=oracle.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=oracle.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=oracle.com header.i=@oracle.com header.b="K698+oNR" Received: from pps.filterd (m0246629.ppops.net [127.0.0.1]) by mx0b-00069f02.pphosted.com (8.17.1.19/8.17.1.19) with ESMTP id 43OFa4Bn009710; Wed, 24 Apr 2024 15:48:47 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding; s=corp-2023-11-20; bh=IrXZwDNwTkREGmLvYeavfzoV9wcq5Yozo3kQ5GRGPKs=; b=K698+oNRg4DC8shJ/NVqurIvV3QPQLeNP55EWvM/wSwK/RweS3Lky0pdbYan/7V0I6h3 V84nMJsmAmnbIiHEryyP+RvEoWkC+2oNVw3BlslyRwDPDFezH4DtzBkqOZjhbujAAZmn SYjepg9ILkCsY/pToTlTWkqc9szuvtJTV+RdtF8epkVPS6ul0cPV/ztAf8F76bbqUU/z A33UMupYS6dXxg8lyteDU9whgph+EhBzZWLbrxA0YS4r5YW0S673mIWAS8pXq9Du+f8K 9XbNxwpzI/vXdI9Q2IwrkyLQbY1JWIWfIfT8KLrVdgmBE8NY/4guDdxUtJjZI/TXPUCm XA== Received: from iadpaimrmta02.imrmtpd1.prodappiadaev1.oraclevcn.com (iadpaimrmta02.appoci.oracle.com [147.154.18.20]) by mx0b-00069f02.pphosted.com (PPS) with ESMTPS id 3xm5kbs3u9-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Wed, 24 Apr 2024 15:48:46 +0000 Received: from pps.filterd (iadpaimrmta02.imrmtpd1.prodappiadaev1.oraclevcn.com [127.0.0.1]) by iadpaimrmta02.imrmtpd1.prodappiadaev1.oraclevcn.com (8.17.1.19/8.17.1.19) with ESMTP id 43OFblxn025235; Wed, 24 Apr 2024 15:48:44 GMT Received: from pps.reinject (localhost [127.0.0.1]) by iadpaimrmta02.imrmtpd1.prodappiadaev1.oraclevcn.com (PPS) with ESMTPS id 3xm45fayx1-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Wed, 24 Apr 2024 15:48:44 +0000 Received: from iadpaimrmta02.imrmtpd1.prodappiadaev1.oraclevcn.com (iadpaimrmta02.imrmtpd1.prodappiadaev1.oraclevcn.com [127.0.0.1]) by pps.reinject (8.17.1.5/8.17.1.5) with ESMTP id 43OFmCoa008769; Wed, 24 Apr 2024 15:48:43 GMT Received: from bpf.uk.oracle.com (dhcp-10-175-216-158.vpn.oracle.com [10.175.216.158]) by iadpaimrmta02.imrmtpd1.prodappiadaev1.oraclevcn.com (PPS) with ESMTP id 3xm45faxuq-7; Wed, 24 Apr 2024 15:48:43 +0000 From: Alan Maguire To: andrii@kernel.org, ast@kernel.org Cc: jolsa@kernel.org, acme@redhat.com, quentin@isovalent.com, eddyz87@gmail.com, mykolal@fb.com, daniel@iogearbox.net, martin.lau@linux.dev, song@kernel.org, yonghong.song@linux.dev, john.fastabend@gmail.com, kpsingh@kernel.org, sdf@google.com, haoluo@google.com, houtao1@huawei.com, bpf@vger.kernel.org, masahiroy@kernel.org, mcgrof@kernel.org, nathan@kernel.org, Alan Maguire Subject: [PATCH v2 bpf-next 06/13] kbuild,bpf: switch to using --btf_features for pahole v1.26 and later Date: Wed, 24 Apr 2024 16:47:59 +0100 Message-Id: <20240424154806.3417662-7-alan.maguire@oracle.com> X-Mailer: git-send-email 2.39.3 In-Reply-To: <20240424154806.3417662-1-alan.maguire@oracle.com> References: <20240424154806.3417662-1-alan.maguire@oracle.com> Precedence: bulk X-Mailing-List: bpf@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293,Aquarius:18.0.1011,Hydra:6.0.650,FMLib:17.11.176.26 definitions=2024-04-24_13,2024-04-24_01,2023-05-22_02 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 suspectscore=0 mlxlogscore=999 mlxscore=0 phishscore=0 spamscore=0 malwarescore=0 adultscore=0 bulkscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2404010000 definitions=main-2404240063 X-Proofpoint-GUID: 8oGCX-aI0byMvPQkObqRWN0SBLzRuGJ5 X-Proofpoint-ORIG-GUID: 8oGCX-aI0byMvPQkObqRWN0SBLzRuGJ5 X-Patchwork-Delegate: bpf@iogearbox.net The btf_features list can be used for pahole v1.26 and later - it is useful because if a feature is not yet implemented it will not exit with a failure message. This will allow us to add feature requests to the pahole options without having to check pahole versions in future; if the version of pahole supports the feature it will be added. Signed-off-by: Alan Maguire --- scripts/Makefile.btf | 7 +++++-- 1 file changed, 5 insertions(+), 2 deletions(-) diff --git a/scripts/Makefile.btf b/scripts/Makefile.btf index 82377e470aed..8e6a9d4b492e 100644 --- a/scripts/Makefile.btf +++ b/scripts/Makefile.btf @@ -12,8 +12,11 @@ pahole-flags-$(call test-ge, $(pahole-ver), 121) += --btf_gen_floats pahole-flags-$(call test-ge, $(pahole-ver), 122) += -j -pahole-flags-$(CONFIG_PAHOLE_HAS_LANG_EXCLUDE) += --lang_exclude=rust - pahole-flags-$(call test-ge, $(pahole-ver), 125) += --skip_encoding_btf_inconsistent_proto --btf_gen_optimized +# Switch to using --btf_features for v1.26 and later. +pahole-flags-$(call test-ge, $(pahole-ver), 126) = -j --btf_features=encode_force,var,float,enum64,decl_tag,type_tag,optimized_func,consistent_func + +pahole-flags-$(CONFIG_PAHOLE_HAS_LANG_EXCLUDE) += --lang_exclude=rust + export PAHOLE_FLAGS := $(pahole-flags-y) From patchwork Wed Apr 24 15:48:00 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alan Maguire X-Patchwork-Id: 13642129 X-Patchwork-Delegate: bpf@iogearbox.net Received: from mx0a-00069f02.pphosted.com (mx0a-00069f02.pphosted.com [205.220.165.32]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 19E2913DBB2 for ; Wed, 24 Apr 2024 15:49:15 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=205.220.165.32 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1713973756; cv=none; b=KE/KK/vZzI/puKS5kAwR0svmSITwnucfVvoitDAmhg99Q6c18NrA5w1pyWEKDXraToqyUnoa4gRNHv5A78d6cwJxo6UN7W1S0wJ1nAf/bnikqWKFMsPTmFh2mANHdv3ZoUCUtDx06TqasyTbQi8VRlKKMbzLiDcQEj9Bj48DDuw= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1713973756; c=relaxed/simple; bh=LrtakGVwnNjl3RUAG7TSbYfqN76f4eEpCagAxQacSV8=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=Qol9hL1Whn0VfTsfuYTa8oS4It2yCNATsgLXanlK5XQDKTunLkYWX3z7YYyYTN8kuhoCYRNynTrNZ84KbERgnW+NuYrTmWn5HdPIpnQcWx8IF4/+W4RMbeJ2xLxXIAuC2KZS1Tzk7Hu2TwqGJSSJbp/CGWvK8EVfS+Ez0yku0OQ= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=oracle.com; spf=pass smtp.mailfrom=oracle.com; dkim=pass (2048-bit key) header.d=oracle.com header.i=@oracle.com header.b=iZmCjJvR; arc=none smtp.client-ip=205.220.165.32 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=oracle.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=oracle.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=oracle.com header.i=@oracle.com header.b="iZmCjJvR" Received: from pps.filterd (m0246629.ppops.net [127.0.0.1]) by mx0b-00069f02.pphosted.com (8.17.1.19/8.17.1.19) with ESMTP id 43OF4FcV009721; Wed, 24 Apr 2024 15:48:50 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding; s=corp-2023-11-20; bh=9vVumfilR+beseT9VynUB6rIgLLZavAEYPMF8/CttSk=; b=iZmCjJvRf4raAi10ftVpAuGMPF79Gg/xocaqgY3/bE9fEykddM7iJrKDw5e9ofc3LB1n C+xft+sj61vgo/31Cc5OWVXFGHv+5bs0EzFdoG06l8Mhwat8ue1oD6mUbJ1YSxrZguGy RzEa1VP3wpmUH3SgiUdH/fpDQ68Zapjidq6x5kAt3IKFTy9PTMw2Kj1dBPzLHpdrBErw fztU7wQgmia5Kws+zU1PVOJ2rI5uYJ1vbOUGXhCt3P0bniuP86faaZIA/+drztIRtniK NApsBw9k9eEoj66NYlFgvK1vbZG++SLxZgqbR5+TmVGDN2mW5+cMhiYDMTehyf1PLUIZ zg== Received: from iadpaimrmta02.imrmtpd1.prodappiadaev1.oraclevcn.com (iadpaimrmta02.appoci.oracle.com [147.154.18.20]) by mx0b-00069f02.pphosted.com (PPS) with ESMTPS id 3xm5kbs3up-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Wed, 24 Apr 2024 15:48:50 +0000 Received: from pps.filterd (iadpaimrmta02.imrmtpd1.prodappiadaev1.oraclevcn.com [127.0.0.1]) by iadpaimrmta02.imrmtpd1.prodappiadaev1.oraclevcn.com (8.17.1.19/8.17.1.19) with ESMTP id 43OEmeYx025298; Wed, 24 Apr 2024 15:48:49 GMT Received: from pps.reinject (localhost [127.0.0.1]) by iadpaimrmta02.imrmtpd1.prodappiadaev1.oraclevcn.com (PPS) with ESMTPS id 3xm45fb028-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Wed, 24 Apr 2024 15:48:49 +0000 Received: from iadpaimrmta02.imrmtpd1.prodappiadaev1.oraclevcn.com (iadpaimrmta02.imrmtpd1.prodappiadaev1.oraclevcn.com [127.0.0.1]) by pps.reinject (8.17.1.5/8.17.1.5) with ESMTP id 43OFmCoc008769; Wed, 24 Apr 2024 15:48:48 GMT Received: from bpf.uk.oracle.com (dhcp-10-175-216-158.vpn.oracle.com [10.175.216.158]) by iadpaimrmta02.imrmtpd1.prodappiadaev1.oraclevcn.com (PPS) with ESMTP id 3xm45faxuq-8; Wed, 24 Apr 2024 15:48:48 +0000 From: Alan Maguire To: andrii@kernel.org, ast@kernel.org Cc: jolsa@kernel.org, acme@redhat.com, quentin@isovalent.com, eddyz87@gmail.com, mykolal@fb.com, daniel@iogearbox.net, martin.lau@linux.dev, song@kernel.org, yonghong.song@linux.dev, john.fastabend@gmail.com, kpsingh@kernel.org, sdf@google.com, haoluo@google.com, houtao1@huawei.com, bpf@vger.kernel.org, masahiroy@kernel.org, mcgrof@kernel.org, nathan@kernel.org, Alan Maguire Subject: [PATCH v2 bpf-next 07/13] resolve_btfids: use .BTF.base ELF section as base BTF if -B option is used Date: Wed, 24 Apr 2024 16:48:00 +0100 Message-Id: <20240424154806.3417662-8-alan.maguire@oracle.com> X-Mailer: git-send-email 2.39.3 In-Reply-To: <20240424154806.3417662-1-alan.maguire@oracle.com> References: <20240424154806.3417662-1-alan.maguire@oracle.com> Precedence: bulk X-Mailing-List: bpf@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293,Aquarius:18.0.1011,Hydra:6.0.650,FMLib:17.11.176.26 definitions=2024-04-24_13,2024-04-24_01,2023-05-22_02 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 suspectscore=0 mlxlogscore=999 mlxscore=0 phishscore=0 spamscore=0 malwarescore=0 adultscore=0 bulkscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2404010000 definitions=main-2404240063 X-Proofpoint-GUID: gOcq5k3l6GKL5ZfjK_eVrFS_wJfPnzfz X-Proofpoint-ORIG-GUID: gOcq5k3l6GKL5ZfjK_eVrFS_wJfPnzfz X-Patchwork-Delegate: bpf@iogearbox.net When resolving BTF ids, use the BTF in the module .BTF.base section when passed the -B option. Both references to base BTF from split BTF and BTF ids will be relocated for base vmlinux on module load. Signed-off-by: Alan Maguire --- tools/bpf/resolve_btfids/main.c | 22 ++++++++++++++++++++-- 1 file changed, 20 insertions(+), 2 deletions(-) diff --git a/tools/bpf/resolve_btfids/main.c b/tools/bpf/resolve_btfids/main.c index d9520cb826b3..c5b622a31f18 100644 --- a/tools/bpf/resolve_btfids/main.c +++ b/tools/bpf/resolve_btfids/main.c @@ -115,6 +115,7 @@ struct object { const char *path; const char *btf; const char *base_btf_path; + int base; struct { int fd; @@ -532,11 +533,26 @@ static int symbols_resolve(struct object *obj) __u32 nr_types; if (obj->base_btf_path) { - base_btf = btf__parse(obj->base_btf_path, NULL); + LIBBPF_OPTS(btf_parse_opts, optp); + const char *path; + + if (obj->base) { + optp.btf_sec = BTF_BASE_ELF_SEC; + path = obj->path; + base_btf = btf__parse_opts(path, &optp); + /* fall back to normal base parsing if no BTF_BASE_ELF_SEC */ + if (libbpf_get_error(base_btf)) + base_btf = NULL; + } + if (!base_btf) { + optp.btf_sec = BTF_ELF_SEC; + path = obj->base_btf_path; + base_btf = btf__parse_opts(path, &optp); + } err = libbpf_get_error(base_btf); if (err) { pr_err("FAILED: load base BTF from %s: %s\n", - obj->base_btf_path, strerror(-err)); + path, strerror(-err)); return -1; } } @@ -781,6 +797,8 @@ int main(int argc, const char **argv) "BTF data"), OPT_STRING('b', "btf_base", &obj.base_btf_path, "file", "path of file providing base BTF"), + OPT_INCR('B', "base", &obj.base, + "use " BTF_BASE_ELF_SEC " ELF section BTF as base"), OPT_END() }; int err = -1; From patchwork Wed Apr 24 15:48:01 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alan Maguire X-Patchwork-Id: 13642130 X-Patchwork-Delegate: bpf@iogearbox.net Received: from mx0a-00069f02.pphosted.com (mx0a-00069f02.pphosted.com [205.220.165.32]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 67531165FBB for ; Wed, 24 Apr 2024 15:49:20 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=205.220.165.32 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1713973761; cv=none; b=e09tsGMNgKoJLm4jATaj56tNRLWJw/8qM3uzyqrkVm2qxk0NtzLhpYN2pdLJqlt/2xQP7Dwya2cnqiolLVR7q8F5VG/aZi+JiC7me6JxbkaAt3wJXvDFrWR+ZA12XSipnD6I3ahUW4N0LmX1h27B4REWOJoUAh96tcu53MX0b+g= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1713973761; c=relaxed/simple; bh=vOYxFhf67kt58MDrxG+/MOZJGIDjFYB+MMJJWQg87L4=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=IzfQiB0tTsE6coHLLH7/nUN3U0aoaQoFq14aCnUsZFOiUw4KI+JWi9HXMPoanxZb0rGzDSPj++p5/bZHiCAKXeGheEeG5e6jHLmOwd0L0Hd/ndR6VRzE2FwoH6JEj7UEt3dSdyI00aTdqwEkCqv74BKarM9alEGksNepUeewa3Y= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=oracle.com; spf=pass smtp.mailfrom=oracle.com; dkim=pass (2048-bit key) header.d=oracle.com header.i=@oracle.com header.b=h37RH13P; arc=none smtp.client-ip=205.220.165.32 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=oracle.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=oracle.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=oracle.com header.i=@oracle.com header.b="h37RH13P" Received: from pps.filterd (m0246629.ppops.net [127.0.0.1]) by mx0b-00069f02.pphosted.com (8.17.1.19/8.17.1.19) with ESMTP id 43OFm8Df009699; Wed, 24 Apr 2024 15:48:55 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding; s=corp-2023-11-20; bh=EWbe9pbeW+PODY1pnDYbMW+P2CTEuuHTSy0RVT6ghww=; b=h37RH13PUkeKGnBR+CbTGCvVwqyEZXz8WTCwrhLy6VkFM6qRIuId0NCIMPLVU/QaU8n9 5rv3q7dyEsQ+DC1aHunEWh7p8+IxUQXiaEOy4ID63IUvVzCDnZ6EC6JuqU4Jai5sSPGd c37mGrp0ONFlAqyz2YM1NQBMIh8WMTtqc9ryHo3OKdLNBw9EZSpiUSOOT/Pld71nipde O7hwxC+rD+tKmkzshDPYahqMoEESqM9Wys2zH+JcFc4QjjiY32BljENx6Rdvc43XcbSA xQBQF/7xL2dTTWtwfjpPfCePStf9Q0kEheNet8FmfM2cFdvfeyvWHCPgCaH8XvmcqX5+ Bw== Received: from iadpaimrmta02.imrmtpd1.prodappiadaev1.oraclevcn.com (iadpaimrmta02.appoci.oracle.com [147.154.18.20]) by mx0b-00069f02.pphosted.com (PPS) with ESMTPS id 3xm5kbs3uy-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Wed, 24 Apr 2024 15:48:55 +0000 Received: from pps.filterd (iadpaimrmta02.imrmtpd1.prodappiadaev1.oraclevcn.com [127.0.0.1]) by iadpaimrmta02.imrmtpd1.prodappiadaev1.oraclevcn.com (8.17.1.19/8.17.1.19) with ESMTP id 43OFHTs0025242; Wed, 24 Apr 2024 15:48:53 GMT Received: from pps.reinject (localhost [127.0.0.1]) by iadpaimrmta02.imrmtpd1.prodappiadaev1.oraclevcn.com (PPS) with ESMTPS id 3xm45fb06d-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Wed, 24 Apr 2024 15:48:53 +0000 Received: from iadpaimrmta02.imrmtpd1.prodappiadaev1.oraclevcn.com (iadpaimrmta02.imrmtpd1.prodappiadaev1.oraclevcn.com [127.0.0.1]) by pps.reinject (8.17.1.5/8.17.1.5) with ESMTP id 43OFmCoe008769; Wed, 24 Apr 2024 15:48:52 GMT Received: from bpf.uk.oracle.com (dhcp-10-175-216-158.vpn.oracle.com [10.175.216.158]) by iadpaimrmta02.imrmtpd1.prodappiadaev1.oraclevcn.com (PPS) with ESMTP id 3xm45faxuq-9; Wed, 24 Apr 2024 15:48:52 +0000 From: Alan Maguire To: andrii@kernel.org, ast@kernel.org Cc: jolsa@kernel.org, acme@redhat.com, quentin@isovalent.com, eddyz87@gmail.com, mykolal@fb.com, daniel@iogearbox.net, martin.lau@linux.dev, song@kernel.org, yonghong.song@linux.dev, john.fastabend@gmail.com, kpsingh@kernel.org, sdf@google.com, haoluo@google.com, houtao1@huawei.com, bpf@vger.kernel.org, masahiroy@kernel.org, mcgrof@kernel.org, nathan@kernel.org, Alan Maguire Subject: [PATCH v2 bpf-next 08/13] kbuild, bpf: add module-specific pahole/resolve_btfids flags for distilled base BTF Date: Wed, 24 Apr 2024 16:48:01 +0100 Message-Id: <20240424154806.3417662-9-alan.maguire@oracle.com> X-Mailer: git-send-email 2.39.3 In-Reply-To: <20240424154806.3417662-1-alan.maguire@oracle.com> References: <20240424154806.3417662-1-alan.maguire@oracle.com> Precedence: bulk X-Mailing-List: bpf@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293,Aquarius:18.0.1011,Hydra:6.0.650,FMLib:17.11.176.26 definitions=2024-04-24_13,2024-04-24_01,2023-05-22_02 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 suspectscore=0 mlxlogscore=999 mlxscore=0 phishscore=0 spamscore=0 malwarescore=0 adultscore=0 bulkscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2404010000 definitions=main-2404240063 X-Proofpoint-GUID: ydMQ_kC6TxlMDai5pzqqhN6gSCRVu25p X-Proofpoint-ORIG-GUID: ydMQ_kC6TxlMDai5pzqqhN6gSCRVu25p X-Patchwork-Delegate: bpf@iogearbox.net Support creation of module BTF along with distilled base BTF; the latter is stored in a .BTF.base ELF section and supplements split BTF references to base BTF with information about base types, allowing for later relocation of split BTF with a (possibly changed) base. resolve_btfids uses the "-B" option to specify that the BTF.ids section should be populated with split BTF relative to the added .BTF.base section rather than relative to the vmlinux base. Modules will be built with a distilled .BTF.base section for external module build, i.e. make -C. -M=path2/module ...while in-tree module build as part of a normal kernel build will not generate distilled base BTF; this is because in-tree modules change with the kernel and do not require BTF relocation for the running vmlinux. Signed-off-by: Alan Maguire --- scripts/Makefile.btf | 7 +++++++ scripts/Makefile.modfinal | 4 ++-- 2 files changed, 9 insertions(+), 2 deletions(-) diff --git a/scripts/Makefile.btf b/scripts/Makefile.btf index 8e6a9d4b492e..8a3f45813c1e 100644 --- a/scripts/Makefile.btf +++ b/scripts/Makefile.btf @@ -19,4 +19,11 @@ pahole-flags-$(call test-ge, $(pahole-ver), 126) = -j --btf_features=encode_forc pahole-flags-$(CONFIG_PAHOLE_HAS_LANG_EXCLUDE) += --lang_exclude=rust +ifneq ($(KBUILD_EXTMOD),) +module-pahole-flags-$(call test-ge, $(pahole-ver), 126) += --btf_features=distilled_base +module-resolve-btfids-flags-$(call test-ge, $(pahole-ver), 126) = -B +endif + export PAHOLE_FLAGS := $(pahole-flags-y) +export MODULE_PAHOLE_FLAGS := $(module-pahole-flags-y) +export MODULE_RESOLVE_BTFIDS_FLAGS := $(module-resolve-btfids-flags-y) diff --git a/scripts/Makefile.modfinal b/scripts/Makefile.modfinal index 8568d256d6fb..22f5bb0a60a6 100644 --- a/scripts/Makefile.modfinal +++ b/scripts/Makefile.modfinal @@ -39,8 +39,8 @@ quiet_cmd_btf_ko = BTF [M] $@ if [ ! -f vmlinux ]; then \ printf "Skipping BTF generation for %s due to unavailability of vmlinux\n" $@ 1>&2; \ else \ - LLVM_OBJCOPY="$(OBJCOPY)" $(PAHOLE) -J $(PAHOLE_FLAGS) --btf_base vmlinux $@; \ - $(RESOLVE_BTFIDS) -b vmlinux $@; \ + LLVM_OBJCOPY="$(OBJCOPY)" $(PAHOLE) -J $(PAHOLE_FLAGS) $(MODULE_PAHOLE_FLAGS) --btf_base vmlinux $@; \ + $(RESOLVE_BTFIDS) $(MODULE_RESOLVE_BTFIDS_FLAGS) -b vmlinux $@; \ fi; # Same as newer-prereqs, but allows to exclude specified extra dependencies From patchwork Wed Apr 24 15:48:02 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alan Maguire X-Patchwork-Id: 13642132 X-Patchwork-Delegate: bpf@iogearbox.net Received: from mx0a-00069f02.pphosted.com (mx0a-00069f02.pphosted.com [205.220.165.32]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 20CBF16D331 for ; Wed, 24 Apr 2024 15:49:27 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=205.220.165.32 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1713973770; cv=none; b=B2w2R2e3k1q9P6ydhQEzmP7ZNb/HKsw9iKY3X9g0oGuMYOHQAU3tCM9EJeAq8fM9kp26v5+degbx6tZDLzCIWRaHurp+8U47x4HhKfr86Jbsn8p5WDSRu2KEZCVJKs5EU0m2qzRtWojohKcMucG/eZ2x9RZ6ncnegA9eSFAL3y8= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1713973770; c=relaxed/simple; bh=pZztmVu8zS5kervyHu37bsZxkSjnkz1FIaFrGhz0SSg=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=DDNFRyh/QCYqcmDOqNsUD6d/Au/s6bA2XwEc3bBS4uXyR2ycpp2TH3+Z8WiU7kYdK6nI//7x1JSRtdA9ew1cYCq9oB91gXjwi/aKQOcDZjnJp9/tVqgdY7+U0reV5W2dIdCzLKwwcjzM/gPe+P7zyM+qIaF1+CWH9CAricUHPFc= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=oracle.com; spf=pass smtp.mailfrom=oracle.com; dkim=pass (2048-bit key) header.d=oracle.com header.i=@oracle.com header.b=QrjTMqfq; arc=none smtp.client-ip=205.220.165.32 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=oracle.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=oracle.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=oracle.com header.i=@oracle.com header.b="QrjTMqfq" Received: from pps.filterd (m0333521.ppops.net [127.0.0.1]) by mx0b-00069f02.pphosted.com (8.17.1.19/8.17.1.19) with ESMTP id 43OA1QhY019778; Wed, 24 Apr 2024 15:49:00 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding; s=corp-2023-11-20; bh=aJmH1aYmvtMWnal5SxOHpzObLhEb3vLt5zmQ92QRkgo=; b=QrjTMqfqCkhauu2ZHX59QIKJoBfdNxU+Z10KDjmlDwthoR6k1wN0pGbB6+D6vgA1fx0K WhLWAdkwU51JBB9foMKMS2+raemKYFnPUourjoDUXE1MuvXtwAdogYHpqCD+U2cXgv6y XMysd9T4nzFV8Diwk9LXjvzGkFXkyJcMnnnEBBuPdKwJg6b66Zjh1IxZCmnXfmmbjOSR QbNYmeYsF2WoZ0Kp+zMFvpAMvQpo+nA9+EC5KgzNmuApZJLf3q1JZV6nYWqM7cetibqg pVj/rUeyDhk9VWiFjh8PRKv5UzIjDoHXfLem8NNIcGgaHRYuG9smzHfVBdqMId7oKmh4 Rg== Received: from iadpaimrmta02.imrmtpd1.prodappiadaev1.oraclevcn.com (iadpaimrmta02.appoci.oracle.com [147.154.18.20]) by mx0b-00069f02.pphosted.com (PPS) with ESMTPS id 3xm4md97p9-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Wed, 24 Apr 2024 15:48:59 +0000 Received: from pps.filterd (iadpaimrmta02.imrmtpd1.prodappiadaev1.oraclevcn.com [127.0.0.1]) by iadpaimrmta02.imrmtpd1.prodappiadaev1.oraclevcn.com (8.17.1.19/8.17.1.19) with ESMTP id 43OEguER025273; Wed, 24 Apr 2024 15:48:58 GMT Received: from pps.reinject (localhost [127.0.0.1]) by iadpaimrmta02.imrmtpd1.prodappiadaev1.oraclevcn.com (PPS) with ESMTPS id 3xm45fb0ac-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Wed, 24 Apr 2024 15:48:58 +0000 Received: from iadpaimrmta02.imrmtpd1.prodappiadaev1.oraclevcn.com (iadpaimrmta02.imrmtpd1.prodappiadaev1.oraclevcn.com [127.0.0.1]) by pps.reinject (8.17.1.5/8.17.1.5) with ESMTP id 43OFmCog008769; Wed, 24 Apr 2024 15:48:57 GMT Received: from bpf.uk.oracle.com (dhcp-10-175-216-158.vpn.oracle.com [10.175.216.158]) by iadpaimrmta02.imrmtpd1.prodappiadaev1.oraclevcn.com (PPS) with ESMTP id 3xm45faxuq-10; Wed, 24 Apr 2024 15:48:57 +0000 From: Alan Maguire To: andrii@kernel.org, ast@kernel.org Cc: jolsa@kernel.org, acme@redhat.com, quentin@isovalent.com, eddyz87@gmail.com, mykolal@fb.com, daniel@iogearbox.net, martin.lau@linux.dev, song@kernel.org, yonghong.song@linux.dev, john.fastabend@gmail.com, kpsingh@kernel.org, sdf@google.com, haoluo@google.com, houtao1@huawei.com, bpf@vger.kernel.org, masahiroy@kernel.org, mcgrof@kernel.org, nathan@kernel.org, Alan Maguire Subject: [PATCH v2 bpf-next 09/13] libbpf: split BTF relocation Date: Wed, 24 Apr 2024 16:48:02 +0100 Message-Id: <20240424154806.3417662-10-alan.maguire@oracle.com> X-Mailer: git-send-email 2.39.3 In-Reply-To: <20240424154806.3417662-1-alan.maguire@oracle.com> References: <20240424154806.3417662-1-alan.maguire@oracle.com> Precedence: bulk X-Mailing-List: bpf@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293,Aquarius:18.0.1011,Hydra:6.0.650,FMLib:17.11.176.26 definitions=2024-04-24_13,2024-04-24_01,2023-05-22_02 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 suspectscore=0 mlxlogscore=999 mlxscore=0 phishscore=0 spamscore=0 malwarescore=0 adultscore=0 bulkscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2404010000 definitions=main-2404240063 X-Proofpoint-ORIG-GUID: 8m0wzbshmhVps650oDzO_n-cE0SrR52D X-Proofpoint-GUID: 8m0wzbshmhVps650oDzO_n-cE0SrR52D X-Patchwork-Delegate: bpf@iogearbox.net Map distilled base BTF type ids referenced in split BTF and their references to the base BTF passed in, and if the mapping succeeds, reparent the split BTF to the base BTF. Relocation rules are - base types must match exactly - enum[64] types should match all value name/value pairs, but the to-be-relocated enum[64] can also define additional name/value pairs - an enum64 can match an enum and vice versa provided the values match as described above - named fwds match to the correspondingly-named struct/union/enum/enum64 - structs with no members match to the correspondingly-named struct/union provided their sizes match - anon struct/unions must have field names/offsets specified in base reference BTF matched by those in base BTF we are matching with Relocation can not recurse, since it will be used in-kernel also and we do not want to blow up the kernel stack when carrying out type compatibility checks. Hence we use a stack for reference type relocation rather then recursive function calls. The approach however is the same; we use a depth-first search to match the referents associated with reference types, and work back from there to match the reference type itself. Signed-off-by: Alan Maguire --- tools/lib/bpf/Build | 2 +- tools/lib/bpf/btf.c | 58 +++ tools/lib/bpf/btf.h | 8 + tools/lib/bpf/btf_relocate.c | 601 ++++++++++++++++++++++++++++++++ tools/lib/bpf/libbpf.map | 1 + tools/lib/bpf/libbpf_internal.h | 2 + 6 files changed, 671 insertions(+), 1 deletion(-) create mode 100644 tools/lib/bpf/btf_relocate.c diff --git a/tools/lib/bpf/Build b/tools/lib/bpf/Build index b6619199a706..336da6844d42 100644 --- a/tools/lib/bpf/Build +++ b/tools/lib/bpf/Build @@ -1,4 +1,4 @@ libbpf-y := libbpf.o bpf.o nlattr.o btf.o libbpf_errno.o str_error.o \ netlink.o bpf_prog_linfo.o libbpf_probes.o hashmap.o \ btf_dump.o ringbuf.o strset.o linker.o gen_loader.o relo_core.o \ - usdt.o zip.o elf.o features.o + usdt.o zip.o elf.o features.o btf_relocate.o diff --git a/tools/lib/bpf/btf.c b/tools/lib/bpf/btf.c index 9036c1dc45d0..f00a84fea9b5 100644 --- a/tools/lib/bpf/btf.c +++ b/tools/lib/bpf/btf.c @@ -5541,3 +5541,61 @@ int btf__distill_base(const struct btf *src_btf, struct btf **new_base_btf, errno = -ret; return ret; } + +struct btf_rewrite_strs { + struct btf *btf; + const struct btf *old_base_btf; + int str_start; + int str_diff; +}; + +static int btf_rewrite_strs(__u32 *str_off, void *ctx) +{ + struct btf_rewrite_strs *r = ctx; + const char *s; + int off; + + if (!*str_off) + return 0; + if (*str_off >= r->str_start) { + *str_off += r->str_diff; + } else { + s = btf__str_by_offset(r->old_base_btf, *str_off); + if (!s) + return -ENOENT; + off = btf__add_str(r->btf, s); + if (off < 0) + return off; + *str_off = off; + } + return 0; +} + +int btf_set_base_btf(struct btf *btf, struct btf *base_btf) +{ + struct btf_rewrite_strs r = {}; + struct btf_type *t; + int i, err; + + r.old_base_btf = btf__base_btf(btf); + if (!r.old_base_btf) + return -EINVAL; + r.btf = btf; + r.str_start = r.old_base_btf->hdr->str_len; + r.str_diff = base_btf->hdr->str_len - r.old_base_btf->hdr->str_len; + btf->base_btf = base_btf; + btf->start_id = btf__type_cnt(base_btf); + btf->start_str_off = base_btf->hdr->str_len; + for (i = 0; i < btf->nr_types; i++) { + t = (struct btf_type *)btf__type_by_id(btf, i + btf->start_id); + err = btf_type_visit_str_offs(t, btf_rewrite_strs, &r); + if (err) + break; + } + return err; +} + +int btf__relocate(struct btf *btf, const struct btf *base_btf) +{ + return btf_relocate(btf, base_btf, NULL); +} diff --git a/tools/lib/bpf/btf.h b/tools/lib/bpf/btf.h index 94dfdfdef617..00e885998ba1 100644 --- a/tools/lib/bpf/btf.h +++ b/tools/lib/bpf/btf.h @@ -284,6 +284,14 @@ struct btf_dedup_opts { LIBBPF_API int btf__dedup(struct btf *btf, const struct btf_dedup_opts *opts); +/** + * @brief **btf__relocate()** will check the split BTF *btf* for references + * to base BTF kinds, and verify those references are compatible with + * *base_btf*; if they are, *btf* is adjusted such that is re-parented to + * *base_btf* and type ids and strings are adjusted to accommodate this. + */ +LIBBPF_API int btf__relocate(struct btf *btf, const struct btf *base_btf); + struct btf_dump; struct btf_dump_opts { diff --git a/tools/lib/bpf/btf_relocate.c b/tools/lib/bpf/btf_relocate.c new file mode 100644 index 000000000000..d9340375f4a3 --- /dev/null +++ b/tools/lib/bpf/btf_relocate.c @@ -0,0 +1,601 @@ +// SPDX-License-Identifier: GPL-2.0 +/* Copyright (c) 2024, Oracle and/or its affiliates. */ + +#include "btf.h" +#include "bpf.h" +#include "libbpf.h" +#include "libbpf_internal.h" + +struct btf; + +#define BTF_MAX_NR_TYPES 0x7fffffffU +#define BTF_UNPROCESSED_ID ((__u32)-1) + +struct btf_relocate { + struct btf *btf; + const struct btf *base_btf; + const struct btf *dist_base_btf; + unsigned int nr_base_types; + __u32 *map; + __u32 *stack; + unsigned int stack_size; + unsigned int stack_limit; +}; + +/* Find next type after *id in base BTF that matches kind of type t passed in + * and name (if it is specified). Match fwd kinds to appropriate kind also. + */ +static int btf_relocate_find_next(struct btf_relocate *r, const struct btf_type *t, + __u32 *id, const struct btf_type **tp) +{ + const struct btf_type *nt; + int kind, tkind = btf_kind(t); + int tkflag = btf_kflag(t); + __u32 i; + + for (i = *id + 1; i < r->nr_base_types; i++) { + nt = btf__type_by_id(r->base_btf, i); + kind = btf_kind(nt); + /* enum[64] can match either enum or enum64; + * a fwd can match a struct/union of the appropriate + * type; otherwise kinds must match. + */ + switch (tkind) { + case BTF_KIND_ENUM: + case BTF_KIND_ENUM64: + switch (kind) { + case BTF_KIND_ENUM64: + case BTF_KIND_ENUM: + break; + default: + continue; + } + break; + case BTF_KIND_FWD: + switch (kind) { + case BTF_KIND_FWD: + continue; + case BTF_KIND_STRUCT: + if (tkflag) + continue; + break; + case BTF_KIND_UNION: + if (!tkflag) + continue; + break; + default: + break; + } + break; + default: + if (kind != tkind) + continue; + break; + } + /* either names must match or both be anon. */ + if (t->name_off && nt->name_off) { + if (strcmp(btf__name_by_offset(r->btf, t->name_off), + btf__name_by_offset(r->base_btf, nt->name_off))) + continue; + } else if (t->name_off != nt->name_off) { + continue; + } + *tp = nt; + *id = i; + return 0; + } + return -ENOENT; +} + +static int btf_relocate_int(struct btf_relocate *r, const char *name, + const struct btf_type *t, const struct btf_type *bt) +{ + __u8 encoding, bencoding, bits, bbits; + + if (t->size != bt->size) { + pr_warn("INT types '%s' disagree on size; distilled base BTF says %d; base BTF says %d\n", + name, t->size, bt->size); + return -EINVAL; + } + encoding = btf_int_encoding(t); + bencoding = btf_int_encoding(bt); + if (encoding != bencoding) { + pr_warn("INT types '%s' disagree on encoding; distilled base BTF says '(%s/%s/%s); base BTF says '(%s/%s/%s)'\n", + name, + encoding & BTF_INT_SIGNED ? "signed" : "unsigned", + encoding & BTF_INT_CHAR ? "char" : "nonchar", + encoding & BTF_INT_BOOL ? "bool" : "nonbool", + bencoding & BTF_INT_SIGNED ? "signed" : "unsigned", + bencoding & BTF_INT_CHAR ? "char" : "nonchar", + bencoding & BTF_INT_BOOL ? "bool" : "nonbool"); + return -EINVAL; + } + bits = btf_int_bits(t); + bbits = btf_int_bits(bt); + if (bits != bbits) { + pr_warn("INT types '%s' disagree on bit size; distilled base BTF says %d; base BTF says %d\n", + name, bits, bbits); + return -EINVAL; + } + return 0; +} + +static int btf_relocate_float(struct btf_relocate *r, const char *name, + const struct btf_type *t, const struct btf_type *bt) +{ + + if (t->size != bt->size) { + pr_warn("float types '%s' disagree on size; distilled base BTF says %d; base BTF says %d\n", + name, t->size, bt->size); + return -EINVAL; + } + return 0; +} + +/* ensure each enum[64] value in type t has equivalent in base BTF and that + * values match; we must support matching enum64 to enum and vice versa + * as well as enum to enum and enum64 to enum64. + */ +static int btf_relocate_enum(struct btf_relocate *r, const char *name, + const struct btf_type *t, const struct btf_type *bt) +{ + struct btf_enum *v = btf_enum(t); + struct btf_enum *bv = btf_enum(bt); + struct btf_enum64 *v64 = btf_enum64(t); + struct btf_enum64 *bv64 = btf_enum64(bt); + bool found, match, bisenum, isenum; + const char *vname, *bvname; + __u32 name_off, bname_off; + __u64 val = 0, bval = 0; + int i, j; + + isenum = btf_kind(t) == BTF_KIND_ENUM; + for (i = 0; i < btf_vlen(t); i++, v++, v64++) { + found = match = false; + + if (isenum) { + name_off = v->name_off; + val = v->val; + } else { + name_off = v64->name_off; + val = btf_enum64_value(v64); + } + if (!name_off) + continue; + vname = btf__name_by_offset(r->dist_base_btf, name_off); + + bisenum = btf_kind(bt) == BTF_KIND_ENUM; + for (j = 0; j < btf_vlen(bt); j++, bv++, bv64++) { + if (bisenum) { + bname_off = bv->name_off; + bval = bv->val; + } else { + bname_off = bv64->name_off; + bval = btf_enum64_value(bv64); + } + if (!bname_off) + continue; + bvname = btf__name_by_offset(r->base_btf, bname_off); + if (strcmp(vname, bvname) != 0) + continue; + found = true; + match = val == bval; + break; + } + if (!found) { + if (t->name_off) + pr_warn("ENUM[64] types '%s' disagree; distilled base BTF has enum[64] value '%s' (%lld), base BTF does not have that value.\n", + name, vname, val); + return -EINVAL; + } + if (!match) { + if (t->name_off) + pr_warn("ENUM[64] types '%s' disagree on enum value '%s'; distilled base BTF specifies value %lld; base BTF specifies value %lld\n", + name, vname, val, bval); + return -EINVAL; + } + } + return 0; +} + +/* relocate base types (int, float, enum, enum64 and fwd) */ +static int btf_relocate_base_type(struct btf_relocate *r, __u32 id) +{ + const struct btf_type *t = btf_type_by_id(r->dist_base_btf, id); + const char *name = btf__name_by_offset(r->dist_base_btf, t->name_off); + const struct btf_type *bt = NULL; + __u32 base_id = 0; + int err = 0; + + switch (btf_kind(t)) { + case BTF_KIND_INT: + case BTF_KIND_ENUM: + case BTF_KIND_FLOAT: + case BTF_KIND_ENUM64: + case BTF_KIND_FWD: + break; + default: + return 0; + } + + if (r->map[id] <= BTF_MAX_NR_TYPES) + return 0; + + while ((err = btf_relocate_find_next(r, t, &base_id, &bt)) != -ENOENT) { + bt = btf_type_by_id(r->base_btf, base_id); + switch (btf_kind(t)) { + case BTF_KIND_INT: + err = btf_relocate_int(r, name, t, bt); + break; + case BTF_KIND_ENUM: + case BTF_KIND_ENUM64: + err = btf_relocate_enum(r, name, t, bt); + break; + case BTF_KIND_FLOAT: + err = btf_relocate_float(r, name, t, bt); + break; + case BTF_KIND_FWD: + err = 0; + break; + default: + return 0; + } + if (!err) { + r->map[id] = base_id; + return 0; + } + } + return err; +} + +/* all distilled base BTF members must be in base BTF equivalent. */ +static int btf_relocate_check_member(struct btf_relocate *r, const char *name, + struct btf_member *m, const struct btf_type *bt, + bool verbose) +{ + struct btf_member *bm = (struct btf_member *)(bt + 1); + const char *kindstr = btf_kind(bt) == BTF_KIND_STRUCT ? "STRUCT" : "UNION"; + const char *mname, *bmname; + int i, bvlen = btf_vlen(bt); + + mname = btf__name_by_offset(r->dist_base_btf, m->name_off); + for (i = 0; i < bvlen; i++, bm++) { + bmname = btf__name_by_offset(r->base_btf, bm->name_off); + + if (!m->name_off || !bm->name_off) { + if (m->name_off != bm->name_off) + continue; + if (bm->offset != m->offset) + continue; + } else { + if (strcmp(mname, bmname) != 0) + continue; + if (bm->offset != m->offset) { + if (verbose) { + pr_warn("%s '%s' member '%s' disagrees about offset; %d in distilled base BTF versus %d in base BTF\n", + kindstr, name, mname, bm->offset, m->offset); + return -EINVAL; + } + } + } + return 0; + } + if (verbose) + pr_warn("%s '%s' missing member '%s' found in distilled base BTF\n", + kindstr, name, mname); + return -EINVAL; +} + +static int btf_relocate_struct_type(struct btf_relocate *r, __u32 id) +{ + const struct btf_type *t = btf_type_by_id(r->dist_base_btf, id); + const char *name = btf__name_by_offset(r->dist_base_btf, t->name_off); + const struct btf_type *bt = NULL; + struct btf_member *m; + const char *kindstr; + int i, vlen, err = 0; + __u32 base_id = 0; + + switch (btf_kind(t)) { + case BTF_KIND_STRUCT: + kindstr = "STRUCT"; + break; + case BTF_KIND_UNION: + kindstr = "UNION"; + break; + default: + return 0; + } + + if (r->map[id] <= BTF_MAX_NR_TYPES) + return 0; + + vlen = btf_vlen(t); + + while ((err = btf_relocate_find_next(r, t, &base_id, &bt)) != -ENOENT) { + /* vlen 0 named types (signalling type is embedded in + * a split BTF struct/union) must match size exactly + */ + if (t->name_off && vlen == 0) { + if (bt->size != t->size) { + pr_warn("%s '%s' disagrees about size; is size (%d) in distilled base BTF; in base BTF it is size (%d)\n", + kindstr, name, t->size, bt->size); + return -EINVAL; + } + } + /* otherwise must be at least as big */ + if (bt->size < t->size) { + if (t->name_off) { + pr_warn("%s '%s' disagrees about size with distilled base BTF (%d); base BTF is smaller (%d)\n", + kindstr, name, t->size, bt->size); + return -EINVAL; + } + continue; + } + /* must have at least as many elements */ + if (btf_vlen(bt) < vlen) { + if (t->name_off) { + pr_warn("%s '%s' disagrees about number of members with distilled base BTF (%d); base BTF has less (%d)\n", + kindstr, name, vlen, btf_vlen(bt)); + return -EINVAL; + } + continue; + } + m = (struct btf_member *)(t + 1); + for (i = 0; i < vlen; i++, m++) { + if (btf_relocate_check_member(r, name, m, bt, t->name_off != 0)) { + if (t->name_off) + return -EINVAL; + err = -EINVAL; + break; + } + } + if (!err) { + r->map[id] = base_id; + return 0; + } + } + return err; +} + +/* Use a stack rather than recursion to manage dependent reference types. + * When a reference type with dependents is encountered, the approach we + * take depends on whether the dependents have been resolved to base + * BTF references via the map[]. If they all have, we can simply search + * for the base BTF type that has those references. If the references + * are not resolved, we need to push the type and its dependents onto + * the stack for later resolution. We first pop the dependents, and + * once these have been resolved we pop the reference type with dependents + * now resolved. + */ +static int btf_relocate_push(struct btf_relocate *r, __u32 id) +{ + if (r->stack_size >= r->stack_limit) + return -ENOSPC; + r->stack[r->stack_size++] = id; + return 0; +} + +static __u32 btf_relocate_pop(struct btf_relocate *r) +{ + if (r->stack_size > 0) + return r->stack[--r->stack_size]; + return BTF_UNPROCESSED_ID; +} + +static int btf_relocate_ref_type(struct btf_relocate *r, __u32 id) +{ + const struct btf_type *t; + const struct btf_type *bt; + __u32 base_id; + int err = 0; + + do { + if (r->map[id] <= BTF_MAX_NR_TYPES) + continue; + t = btf_type_by_id(r->dist_base_btf, id); + switch (btf_kind(t)) { + case BTF_KIND_CONST: + case BTF_KIND_VOLATILE: + case BTF_KIND_RESTRICT: + case BTF_KIND_PTR: + case BTF_KIND_TYPEDEF: + case BTF_KIND_FUNC: + case BTF_KIND_TYPE_TAG: + case BTF_KIND_DECL_TAG: + if (r->map[t->type] <= BTF_MAX_NR_TYPES) { + bt = NULL; + base_id = 0; + while ((err = btf_relocate_find_next(r, t, &base_id, &bt)) + != -ENOENT) { + if (btf_kind(t) == BTF_KIND_DECL_TAG) { + if (btf_decl_tag(t) != btf_decl_tag(bt)) + continue; + } + if (bt->type != r->map[t->type]) + continue; + r->map[id] = base_id; + break; + } + if (err) { + pr_warn("could not find base BTF type for distilled base BTF type[%u]\n", + id); + return err; + } + } else { + if (btf_relocate_push(r, id) < 0 || + btf_relocate_push(r, t->type) < 0) + return -ENOSPC; + } + break; + case BTF_KIND_ARRAY: { + struct btf_array *ba, *a = btf_array(t); + + if (r->map[a->type] <= BTF_MAX_NR_TYPES && + r->map[a->index_type] <= BTF_MAX_NR_TYPES) { + bt = NULL; + base_id = 0; + while ((err = btf_relocate_find_next(r, t, &base_id, &bt)) + != -ENOENT) { + ba = btf_array(bt); + if (a->nelems != ba->nelems || + r->map[a->type] != ba->type || + r->map[a->index_type] != ba->index_type) + continue; + r->map[id] = base_id; + break; + } + if (err) { + pr_warn("could not matching find base BTF ARRAY for distilled base BTF ARRAY[%u]\n", + id); + return err; + } + } else { + if (btf_relocate_push(r, id) < 0 || + btf_relocate_push(r, a->type) < 0 || + btf_relocate_push(r, a->index_type) < 0) + return -ENOSPC; + } + break; + } + case BTF_KIND_FUNC_PROTO: { + struct btf_param *p = btf_params(t); + int i, vlen = btf_vlen(t); + + for (i = 0; i < vlen; i++, p++) { + if (r->map[p->type] > BTF_MAX_NR_TYPES) + break; + } + if (i == vlen && r->map[t->type] <= BTF_MAX_NR_TYPES) { + bt = NULL; + base_id = 0; + while ((err = btf_relocate_find_next(r, t, &base_id, &bt)) + != -ENOENT) { + struct btf_param *bp = btf_params(bt); + int bvlen = btf_vlen(bt); + int j; + + if (bvlen != vlen) + continue; + if (r->map[t->type] != bt->type) + continue; + for (j = 0, p = btf_params(t); j < bvlen; j++, bp++, p++) { + if (r->map[p->type] != bp->type) + break; + } + if (j < bvlen) + continue; + r->map[id] = base_id; + break; + } + if (err) { + pr_warn("could not find matching base BTF FUNC_PROTO for distilled base BTF FUNC_PROTO[%u]\n", + id); + return err; + } + } else { + if (btf_relocate_push(r, id) < 0 || + btf_relocate_push(r, t->type) < 0) + return -ENOSPC; + for (i = 0, p = btf_params(t); i < btf_vlen(t); i++, p++) { + if (btf_relocate_push(r, p->type) < 0) + return -ENOSPC; + } + } + break; + } + default: + return -EINVAL; + } + } while ((id = btf_relocate_pop(r)) <= BTF_MAX_NR_TYPES); + + return 0; +} + +static int btf_relocate_rewrite_type_id(__u32 *id, void *ctx) +{ + struct btf_relocate *r = ctx; + + *id = r->map[*id]; + return 0; +} + +/* If successful, output of relocation is updated BTF with base BTF pointing + * at base_btf, and type ids, strings adjusted accordingly + */ +int btf_relocate(struct btf *btf, const struct btf *base_btf, __u32 **map_ids) +{ + const struct btf *dist_base_btf = btf__base_btf(btf); + unsigned int nr_split_types, nr_dist_base_types; + unsigned int nr_types = btf__type_cnt(btf); + struct btf_relocate r = {}; + const struct btf_type *t; + int diff_id, err = 0; + __u32 id, i; + + if (!base_btf || dist_base_btf == base_btf) + return 0; + + nr_dist_base_types = btf__type_cnt(dist_base_btf); + r.nr_base_types = btf__type_cnt(base_btf); + nr_split_types = nr_types - nr_dist_base_types; + r.map = calloc(nr_types, sizeof(*r.map)); + r.stack_limit = nr_dist_base_types; + r.stack = calloc(r.stack_limit, sizeof(*r.stack)); + if (!r.map || !r.stack) { + err = -ENOMEM; + goto err_out; + } + diff_id = r.nr_base_types - nr_dist_base_types; + for (id = 1; id < nr_dist_base_types; id++) + r.map[id] = BTF_UNPROCESSED_ID; + for (id = nr_dist_base_types; id < nr_types; id++) + r.map[id] = id + diff_id; + + r.btf = btf; + r.dist_base_btf = dist_base_btf; + r.base_btf = base_btf; + + /* Build a map from base references to actual base BTF ids; it is used + * to track the state of comparisons. First map base types and fwds, + * next structs/unions, and finally reference types (const, restrict, + * ptr, array, func, func_proto etc). + */ + for (id = 1; id < nr_dist_base_types; id++) { + err = btf_relocate_base_type(&r, id); + if (err) + goto err_out; + } + for (id = 1; id < nr_dist_base_types; id++) { + err = btf_relocate_struct_type(&r, id); + if (err) + goto err_out; + } + for (id = 1; id < nr_dist_base_types; id++) { + err = btf_relocate_ref_type(&r, id); + if (err) + goto err_out; + } + /* Next, rewrite type ids in split BTF, replacing split ids with updated + * ids based on number of types in base BTF, and base ids with + * relocated ids from base_btf. + */ + for (i = 0, id = nr_dist_base_types; i < nr_split_types; i++, id++) { + t = btf__type_by_id(btf, id); + err = btf_type_visit_type_ids((struct btf_type *)t, + btf_relocate_rewrite_type_id, &r); + if (err) + goto err_out; + } + /* Finally reset base BTF to base_btf; as part of this operation, string + * offsets are also updated, and we are done. + */ + err = btf_set_base_btf(r.btf, (struct btf *)r.base_btf); +err_out: + if (!err && map_ids) + *map_ids = r.map; + else + free(r.map); + free(r.stack); + return err; +} diff --git a/tools/lib/bpf/libbpf.map b/tools/lib/bpf/libbpf.map index a9151e31dfa9..b245350f456c 100644 --- a/tools/lib/bpf/libbpf.map +++ b/tools/lib/bpf/libbpf.map @@ -422,6 +422,7 @@ LIBBPF_1.5.0 { bpf_program__attach_sockmap; btf__distill_base; btf__parse_opts; + btf__relocate; ring__consume_n; ring_buffer__consume_n; } LIBBPF_1.4.0; diff --git a/tools/lib/bpf/libbpf_internal.h b/tools/lib/bpf/libbpf_internal.h index a0dcfb82e455..e38e1b01e86e 100644 --- a/tools/lib/bpf/libbpf_internal.h +++ b/tools/lib/bpf/libbpf_internal.h @@ -234,6 +234,8 @@ struct btf_type; struct btf_type *btf_type_by_id(const struct btf *btf, __u32 type_id); const char *btf_kind_str(const struct btf_type *t); const struct btf_type *skip_mods_and_typedefs(const struct btf *btf, __u32 id, __u32 *res_id); +int btf_set_base_btf(struct btf *btf, struct btf *base_btf); +int btf_relocate(struct btf *btf, const struct btf *base_btf, __u32 **map_ids); static inline enum btf_func_linkage btf_func_linkage(const struct btf_type *t) { From patchwork Wed Apr 24 15:48:03 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alan Maguire X-Patchwork-Id: 13642131 X-Patchwork-Delegate: bpf@iogearbox.net Received: from mx0b-00069f02.pphosted.com (mx0b-00069f02.pphosted.com [205.220.177.32]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 804AA16C859 for ; Wed, 24 Apr 2024 15:49:25 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=205.220.177.32 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1713973767; cv=none; b=OMMFN7E8rn2j/qByPTuVQ8r1+1zdt3wDKBNfiAhgwUqcbfqEtM2PovFmcq2ylmaVyIzMjloUTLzbHKuMPMAYQZ+LDyJAz4Wt6Yh67jMuU4MTJJpYUTOt8flxvSFJEBmvsSEOnWGQxn7AMCLhujdNb2JXQo0RNlvDGorYoC7I3tk= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1713973767; c=relaxed/simple; bh=cBcfmcVBrgf6azkPS8GIkEODkINDbJ+MQWMXCGDO06I=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=KkROzZ/bAsOn+yYnpvaCoxvxEKVnaEFMm/4D1UfbxbqaUx95ea8KpnYrUs7XTpOyu9n/p5yINnYjy7d/xjRtB0ZpskMJ9VDHTPfvinyS0sDzR2oapwqexo66dydtTnj4h4Sjgo2DdmBcNQmJi22bwxM3F7K5NlZjdypp8anCQyQ= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=oracle.com; spf=pass smtp.mailfrom=oracle.com; dkim=pass (2048-bit key) header.d=oracle.com header.i=@oracle.com header.b=hEnCD7It; arc=none smtp.client-ip=205.220.177.32 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=oracle.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=oracle.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=oracle.com header.i=@oracle.com header.b="hEnCD7It" Received: from pps.filterd (m0333520.ppops.net [127.0.0.1]) by mx0b-00069f02.pphosted.com (8.17.1.19/8.17.1.19) with ESMTP id 43OF13jn018870; Wed, 24 Apr 2024 15:49:03 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding; s=corp-2023-11-20; bh=NLTtqzFIHAlwjSGVFNQNXoDScRZ3Sw+F0BFfEuNRB7Q=; b=hEnCD7ItShHUehbtI/BtEKslzJGtcBqTLqkaBDvvJ12mN5Uyw/AiNdzuNNffwN8xuXfB AaLmgFXztHUeWD9IZmzyLA/tiCqU6CSbFVfOhMK39sevKRVTZvWyY7SpyXa3FS+os4S8 17aCbHSrMnIQGGMcfm4stM5gujttHt9IbtZyRru4gYjIObLV3soQME/e1tfqJTF6yObo 8GVXj73ydJ9DfyhLaI6uCU+jL+D0wLEBwQgAVahKQA8ZQe67B1MzlgWt3lMk/zX+BGOX lkYghd9la94nIdLzoMejR7NoS32A0k68lkm3vBfvGpK5gm+qbyQFwHBAcaihsdOjp5Mg cA== Received: from iadpaimrmta02.imrmtpd1.prodappiadaev1.oraclevcn.com (iadpaimrmta02.appoci.oracle.com [147.154.18.20]) by mx0b-00069f02.pphosted.com (PPS) with ESMTPS id 3xm5re0kd4-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Wed, 24 Apr 2024 15:49:03 +0000 Received: from pps.filterd (iadpaimrmta02.imrmtpd1.prodappiadaev1.oraclevcn.com [127.0.0.1]) by iadpaimrmta02.imrmtpd1.prodappiadaev1.oraclevcn.com (8.17.1.19/8.17.1.19) with ESMTP id 43OEs60R025222; Wed, 24 Apr 2024 15:49:02 GMT Received: from pps.reinject (localhost [127.0.0.1]) by iadpaimrmta02.imrmtpd1.prodappiadaev1.oraclevcn.com (PPS) with ESMTPS id 3xm45fb0d9-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Wed, 24 Apr 2024 15:49:02 +0000 Received: from iadpaimrmta02.imrmtpd1.prodappiadaev1.oraclevcn.com (iadpaimrmta02.imrmtpd1.prodappiadaev1.oraclevcn.com [127.0.0.1]) by pps.reinject (8.17.1.5/8.17.1.5) with ESMTP id 43OFmCoi008769; Wed, 24 Apr 2024 15:49:01 GMT Received: from bpf.uk.oracle.com (dhcp-10-175-216-158.vpn.oracle.com [10.175.216.158]) by iadpaimrmta02.imrmtpd1.prodappiadaev1.oraclevcn.com (PPS) with ESMTP id 3xm45faxuq-11; Wed, 24 Apr 2024 15:49:01 +0000 From: Alan Maguire To: andrii@kernel.org, ast@kernel.org Cc: jolsa@kernel.org, acme@redhat.com, quentin@isovalent.com, eddyz87@gmail.com, mykolal@fb.com, daniel@iogearbox.net, martin.lau@linux.dev, song@kernel.org, yonghong.song@linux.dev, john.fastabend@gmail.com, kpsingh@kernel.org, sdf@google.com, haoluo@google.com, houtao1@huawei.com, bpf@vger.kernel.org, masahiroy@kernel.org, mcgrof@kernel.org, nathan@kernel.org, Alan Maguire Subject: [PATCH v2 bpf-next 10/13] module, bpf: store BTF base pointer in struct module Date: Wed, 24 Apr 2024 16:48:03 +0100 Message-Id: <20240424154806.3417662-11-alan.maguire@oracle.com> X-Mailer: git-send-email 2.39.3 In-Reply-To: <20240424154806.3417662-1-alan.maguire@oracle.com> References: <20240424154806.3417662-1-alan.maguire@oracle.com> Precedence: bulk X-Mailing-List: bpf@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293,Aquarius:18.0.1011,Hydra:6.0.650,FMLib:17.11.176.26 definitions=2024-04-24_13,2024-04-24_01,2023-05-22_02 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 suspectscore=0 mlxlogscore=999 mlxscore=0 phishscore=0 spamscore=0 malwarescore=0 adultscore=0 bulkscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2404010000 definitions=main-2404240063 X-Proofpoint-ORIG-GUID: _PiVMRJtAuDmBvGQn0jB8tzKElY4hTue X-Proofpoint-GUID: _PiVMRJtAuDmBvGQn0jB8tzKElY4hTue X-Patchwork-Delegate: bpf@iogearbox.net ...as this will allow split BTF modules with a base BTF representation (rather than the full vmlinux BTF at time of BTF encoding) to resolve their references to kernel types in a way that is more resilient to small changes in kernel types. This will allow modules that are not built every time the kernel is to provide more resilient BTF, rather than have it invalidated every time BTF ids for core kernel types change. Signed-off-by: Alan Maguire --- include/linux/module.h | 2 ++ kernel/module/main.c | 5 ++++- 2 files changed, 6 insertions(+), 1 deletion(-) diff --git a/include/linux/module.h b/include/linux/module.h index 1153b0d99a80..f127a79a95d9 100644 --- a/include/linux/module.h +++ b/include/linux/module.h @@ -510,6 +510,8 @@ struct module { #ifdef CONFIG_DEBUG_INFO_BTF_MODULES unsigned int btf_data_size; void *btf_data; + unsigned int btf_base_data_size; + void *btf_base_data; #endif #ifdef CONFIG_JUMP_LABEL struct jump_entry *jump_entries; diff --git a/kernel/module/main.c b/kernel/module/main.c index e1e8a7a9d6c1..e18683abec07 100644 --- a/kernel/module/main.c +++ b/kernel/module/main.c @@ -2148,6 +2148,8 @@ static int find_module_sections(struct module *mod, struct load_info *info) #endif #ifdef CONFIG_DEBUG_INFO_BTF_MODULES mod->btf_data = any_section_objs(info, ".BTF", 1, &mod->btf_data_size); + mod->btf_base_data = any_section_objs(info, ".BTF.base", 1, + &mod->btf_base_data_size); #endif #ifdef CONFIG_JUMP_LABEL mod->jump_entries = section_objs(info, "__jump_table", @@ -2587,8 +2589,9 @@ static noinline int do_init_module(struct module *mod) } #ifdef CONFIG_DEBUG_INFO_BTF_MODULES - /* .BTF is not SHF_ALLOC and will get removed, so sanitize pointer */ + /* .BTF is not SHF_ALLOC and will get removed, so sanitize pointers */ mod->btf_data = NULL; + mod->btf_base_data = NULL; #endif /* * We want to free module_init, but be aware that kallsyms may be From patchwork Wed Apr 24 15:48:04 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alan Maguire X-Patchwork-Id: 13642134 X-Patchwork-Delegate: bpf@iogearbox.net Received: from mx0a-00069f02.pphosted.com (mx0a-00069f02.pphosted.com [205.220.165.32]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 6C8A616D4E5 for ; Wed, 24 Apr 2024 15:49:35 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=205.220.165.32 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1713973777; cv=none; b=aem72Q9MxNUvslOWizBRrcCkMOixRhAEog7OM3s/FFtEIMD7ziBg0f+6ESR7CbDqMgun/e8hX1FwxkR6AFMu7g4S/QIpyclqWMeQEwArIwZlabjk3RrPaveowYGxmgg0VwJMSIxT0HpijkPXEa/TT6xxWZz5fh69bdVLOgUF1Dc= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1713973777; c=relaxed/simple; bh=sqYw1gf30HYbeH6v0/pi81thPgpckdN8mMbX9WBWPsM=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=NYaE5bDKBTlGr4fkD73UZOUgkV8KTxZB7BqIZs7ueRjV3La3lUDGOv9VNgMMjFGccv8SFSqu51drrHgVzoOuAweSjpwam279AS/WfBy3HaE9Wl38cyI+nLuTngORW8IqsDzJ2Pw+PIE/9Ar0EGZIv6NyOQMkL70lu3F0zQQN9Xg= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=oracle.com; spf=pass smtp.mailfrom=oracle.com; dkim=pass (2048-bit key) header.d=oracle.com header.i=@oracle.com header.b=L47y7f5e; arc=none smtp.client-ip=205.220.165.32 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=oracle.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=oracle.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=oracle.com header.i=@oracle.com header.b="L47y7f5e" Received: from pps.filterd (m0246629.ppops.net [127.0.0.1]) by mx0b-00069f02.pphosted.com (8.17.1.19/8.17.1.19) with ESMTP id 43OFm8Dl009699; Wed, 24 Apr 2024 15:49:09 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding; s=corp-2023-11-20; bh=EgUGl9Rz7betOOk0UaDJwL3d2atUaBAz19C2JavFdjI=; b=L47y7f5eg5Go21naD+kkUXpz6oe3bAJjuwWelMCRVei2UXFvg2MegSxHt0zrCK3eHafF xdSDd9ovcS1dKo6qn0fuyu4+mFj7tf9LR5LhwtBr6gEoiED0bAcpRNToackARFOJmNmB 9x1DBrqh47a/osQ7pS+8DPqaSyh7hGRgrvr1NKs+VTfAjOJ5anmT7PzlAeQEWpCNYOVH RutyOD2HRFv8+hpOEY8gxWW/z40DNnqpklagzH5DxWPTUkFLiwOlNp100puQ6G2YCN5g TrE0V2C5GRrunNQotjOkH5dks7S47CkZs68oPYAA1DUXuNVSc0jp4w3AUdKqq4ZFnntw mw== Received: from iadpaimrmta02.imrmtpd1.prodappiadaev1.oraclevcn.com (iadpaimrmta02.appoci.oracle.com [147.154.18.20]) by mx0b-00069f02.pphosted.com (PPS) with ESMTPS id 3xm5kbs3vt-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Wed, 24 Apr 2024 15:49:08 +0000 Received: from pps.filterd (iadpaimrmta02.imrmtpd1.prodappiadaev1.oraclevcn.com [127.0.0.1]) by iadpaimrmta02.imrmtpd1.prodappiadaev1.oraclevcn.com (8.17.1.19/8.17.1.19) with ESMTP id 43OFmTC9025314; Wed, 24 Apr 2024 15:49:07 GMT Received: from pps.reinject (localhost [127.0.0.1]) by iadpaimrmta02.imrmtpd1.prodappiadaev1.oraclevcn.com (PPS) with ESMTPS id 3xm45fb0g3-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Wed, 24 Apr 2024 15:49:07 +0000 Received: from iadpaimrmta02.imrmtpd1.prodappiadaev1.oraclevcn.com (iadpaimrmta02.imrmtpd1.prodappiadaev1.oraclevcn.com [127.0.0.1]) by pps.reinject (8.17.1.5/8.17.1.5) with ESMTP id 43OFmCok008769; Wed, 24 Apr 2024 15:49:06 GMT Received: from bpf.uk.oracle.com (dhcp-10-175-216-158.vpn.oracle.com [10.175.216.158]) by iadpaimrmta02.imrmtpd1.prodappiadaev1.oraclevcn.com (PPS) with ESMTP id 3xm45faxuq-12; Wed, 24 Apr 2024 15:49:05 +0000 From: Alan Maguire To: andrii@kernel.org, ast@kernel.org Cc: jolsa@kernel.org, acme@redhat.com, quentin@isovalent.com, eddyz87@gmail.com, mykolal@fb.com, daniel@iogearbox.net, martin.lau@linux.dev, song@kernel.org, yonghong.song@linux.dev, john.fastabend@gmail.com, kpsingh@kernel.org, sdf@google.com, haoluo@google.com, houtao1@huawei.com, bpf@vger.kernel.org, masahiroy@kernel.org, mcgrof@kernel.org, nathan@kernel.org, Alan Maguire Subject: [PATCH v2 bpf-next 11/13] libbpf,bpf: share BTF relocate-related code with kernel Date: Wed, 24 Apr 2024 16:48:04 +0100 Message-Id: <20240424154806.3417662-12-alan.maguire@oracle.com> X-Mailer: git-send-email 2.39.3 In-Reply-To: <20240424154806.3417662-1-alan.maguire@oracle.com> References: <20240424154806.3417662-1-alan.maguire@oracle.com> Precedence: bulk X-Mailing-List: bpf@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293,Aquarius:18.0.1011,Hydra:6.0.650,FMLib:17.11.176.26 definitions=2024-04-24_13,2024-04-24_01,2023-05-22_02 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 suspectscore=0 mlxlogscore=999 mlxscore=0 phishscore=0 spamscore=0 malwarescore=0 adultscore=0 bulkscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2404010000 definitions=main-2404240063 X-Proofpoint-GUID: 7BU1Ylnl5L3JEY1OD6WHDphG0_xud7iA X-Proofpoint-ORIG-GUID: 7BU1Ylnl5L3JEY1OD6WHDphG0_xud7iA X-Patchwork-Delegate: bpf@iogearbox.net Share relocation implementation with the kernel. As part of this, we also need the type/string visitation functions so add them to a btf_common.c file that also gets shared with the kernel. Relocation code in kernel and userspace is identical save for the impementation of the reparenting of split BTF to the relocated base BTF; this depends on struct btf internals so is different in kernel and userspace. One other wrinkle on the kernel side is we have to map .BTF.ids in modules as they were generated with the type ids used at BTF encoding time. btf_relocate() optionally returns an array mapping from old BTF ids to relocated ids, so we use that to fix up these references where needed for kfuncs. Signed-off-by: Alan Maguire --- include/linux/btf.h | 32 +++++ kernel/bpf/Makefile | 8 ++ kernel/bpf/btf.c | 227 ++++++++++++++++++++++++++++------- tools/lib/bpf/Build | 2 +- tools/lib/bpf/btf.c | 130 -------------------- tools/lib/bpf/btf_common.c | 146 ++++++++++++++++++++++ tools/lib/bpf/btf_relocate.c | 29 +++++ 7 files changed, 399 insertions(+), 175 deletions(-) create mode 100644 tools/lib/bpf/btf_common.c diff --git a/include/linux/btf.h b/include/linux/btf.h index f9e56fd12a9f..1cc20844f163 100644 --- a/include/linux/btf.h +++ b/include/linux/btf.h @@ -214,6 +214,7 @@ bool btf_is_kernel(const struct btf *btf); bool btf_is_module(const struct btf *btf); struct module *btf_try_get_module(const struct btf *btf); u32 btf_nr_types(const struct btf *btf); +struct btf *btf_base_btf(const struct btf *btf); bool btf_member_is_reg_int(const struct btf *btf, const struct btf_type *s, const struct btf_member *m, u32 expected_offset, u32 expected_size); @@ -515,8 +516,15 @@ static inline const struct bpf_struct_ops_desc *bpf_struct_ops_find(struct btf * } #endif +typedef int (*type_id_visit_fn)(__u32 *type_id, void *ctx); +typedef int (*str_off_visit_fn)(__u32 *str_off, void *ctx); + #ifdef CONFIG_BPF_SYSCALL const struct btf_type *btf_type_by_id(const struct btf *btf, u32 type_id); +int btf_set_base_btf(struct btf *btf, struct btf *base_btf); +int btf_relocate(struct btf *btf, const struct btf *base_btf, __u32 **map_ids); +int btf_type_visit_type_ids(struct btf_type *t, type_id_visit_fn visit, void *ctx); +int btf_type_visit_str_offs(struct btf_type *t, str_off_visit_fn visit, void *ctx); const char *btf_name_by_offset(const struct btf *btf, u32 offset); struct btf *btf_parse_vmlinux(void); struct btf *bpf_prog_get_target_btf(const struct bpf_prog *prog); @@ -543,6 +551,30 @@ static inline const struct btf_type *btf_type_by_id(const struct btf *btf, { return NULL; } + +static inline int btf_set_base_btf(struct btf *btf, struct btf *base_btf) +{ + return 0; +} + +static inline int btf_relocate(void *log, struct btf *btf, const struct btf *base_btf, + __u32 **map_ids) +{ + return 0; +} + +static inline int btf_type_visit_type_ids(struct btf_type *t, type_id_visit_fn visit, + void *ctx) +{ + return 0; +} + +static inline int btf_type_visit_str_offs(struct btf_type *t, str_off_visit_fn visit, + void *ctx) +{ + return 0; +} + static inline const char *btf_name_by_offset(const struct btf *btf, u32 offset) { diff --git a/kernel/bpf/Makefile b/kernel/bpf/Makefile index 368c5d86b5b7..d705dbe2b226 100644 --- a/kernel/bpf/Makefile +++ b/kernel/bpf/Makefile @@ -49,3 +49,11 @@ obj-$(CONFIG_BPF_PRELOAD) += preload/ obj-$(CONFIG_BPF_SYSCALL) += relo_core.o $(obj)/relo_core.o: $(srctree)/tools/lib/bpf/relo_core.c FORCE $(call if_changed_rule,cc_o_c) + +obj-$(CONFIG_BPF_SYSCALL) += btf_common.o +$(obj)/btf_common.o: $(srctree)/tools/lib/bpf/btf_common.c FORCE + $(call if_changed_rule,cc_o_c) + +obj-$(CONFIG_BPF_SYSCALL) += btf_relocate.o +$(obj)/btf_relocate.o: $(srctree)/tools/lib/bpf/btf_relocate.c FORCE + $(call if_changed_rule,cc_o_c) diff --git a/kernel/bpf/btf.c b/kernel/bpf/btf.c index 8291fbfd27b1..2f304b77bab4 100644 --- a/kernel/bpf/btf.c +++ b/kernel/bpf/btf.c @@ -273,6 +273,7 @@ struct btf { u32 start_str_off; /* first string offset (0 for base BTF) */ char name[MODULE_NAME_LEN]; bool kernel_btf; + __u32 *base_map; /* map from distilled base BTF -> vmlinux BTF ids */ }; enum verifier_phase { @@ -1734,7 +1735,13 @@ static void btf_free(struct btf *btf) kvfree(btf->types); kvfree(btf->resolved_sizes); kvfree(btf->resolved_ids); - kvfree(btf->data); + /* only split BTF allocates data, but btf->data is non-NULL for + * vmlinux BTF too. + */ + if (btf->base_btf) + kvfree(btf->data); + if (btf->kernel_btf) + kvfree(btf->base_map); kfree(btf); } @@ -1763,6 +1770,90 @@ void btf_put(struct btf *btf) } } +struct btf *btf_base_btf(const struct btf *btf) +{ + return btf->base_btf; +} + +struct btf_rewrite_strs { + struct btf *btf; + const struct btf *old_base_btf; + int str_start; + int str_diff; + __u32 *str_map; +}; + +static __u32 btf_find_str(struct btf *btf, const char *s) +{ + __u32 offset = 0; + + while (offset < btf->hdr.str_len) { + while (!btf->strings[offset]) + offset++; + if (strcmp(s, &btf->strings[offset]) == 0) + return offset; + while (btf->strings[offset]) + offset++; + } + return -ENOENT; +} + +static int btf_rewrite_strs(__u32 *str_off, void *ctx) +{ + struct btf_rewrite_strs *r = ctx; + const char *s; + int off; + + if (!*str_off) + return 0; + if (*str_off >= r->str_start) { + *str_off += r->str_diff; + } else { + s = btf_str_by_offset(r->old_base_btf, *str_off); + if (!s) + return -ENOENT; + if (r->str_map[*str_off]) { + off = r->str_map[*str_off]; + } else { + off = btf_find_str(r->btf->base_btf, s); + if (off < 0) + return off; + r->str_map[*str_off] = off; + } + *str_off = off; + } + return 0; +} + +int btf_set_base_btf(struct btf *btf, struct btf *base_btf) +{ + struct btf_rewrite_strs r = {}; + struct btf_type *t; + int i, err; + + r.old_base_btf = btf_base_btf(btf); + if (!r.old_base_btf) + return -EINVAL; + r.btf = btf; + r.str_start = r.old_base_btf->hdr.str_len; + r.str_diff = base_btf->hdr.str_len - r.old_base_btf->hdr.str_len; + r.str_map = kvcalloc(r.old_base_btf->hdr.str_len, sizeof(*r.str_map), + GFP_KERNEL | __GFP_NOWARN); + if (!r.str_map) + return -ENOMEM; + btf->base_btf = base_btf; + btf->start_id = btf_nr_types(base_btf); + btf->start_str_off = base_btf->hdr.str_len; + for (i = 0; i < btf->nr_types; i++) { + t = (struct btf_type *)btf_type_by_id(btf, i + btf->start_id); + err = btf_type_visit_str_offs((struct btf_type *)t, btf_rewrite_strs, &r); + if (err) + break; + } + kvfree(r.str_map); + return err; +} + static int env_resolve_init(struct btf_verifier_env *env) { struct btf *btf = env->btf; @@ -5981,23 +6072,15 @@ int get_kern_ctx_btf_id(struct bpf_verifier_log *log, enum bpf_prog_type prog_ty BTF_ID_LIST(bpf_ctx_convert_btf_id) BTF_ID(struct, bpf_ctx_convert) -struct btf *btf_parse_vmlinux(void) +static struct btf *btf_parse_base(struct btf_verifier_env *env, const char *name, + void *data, unsigned int data_size) { - struct btf_verifier_env *env = NULL; - struct bpf_verifier_log *log; struct btf *btf = NULL; int err; if (!IS_ENABLED(CONFIG_DEBUG_INFO_BTF)) return ERR_PTR(-ENOENT); - env = kzalloc(sizeof(*env), GFP_KERNEL | __GFP_NOWARN); - if (!env) - return ERR_PTR(-ENOMEM); - - log = &env->log; - log->level = BPF_LOG_KERNEL; - btf = kzalloc(sizeof(*btf), GFP_KERNEL | __GFP_NOWARN); if (!btf) { err = -ENOMEM; @@ -6005,10 +6088,10 @@ struct btf *btf_parse_vmlinux(void) } env->btf = btf; - btf->data = __start_BTF; - btf->data_size = __stop_BTF - __start_BTF; + btf->data = data; + btf->data_size = data_size; btf->kernel_btf = true; - snprintf(btf->name, sizeof(btf->name), "vmlinux"); + snprintf(btf->name, sizeof(btf->name), "%s", name); err = btf_parse_hdr(env); if (err) @@ -6028,20 +6111,11 @@ struct btf *btf_parse_vmlinux(void) if (err) goto errout; - /* btf_parse_vmlinux() runs under bpf_verifier_lock */ - bpf_ctx_convert.t = btf_type_by_id(btf, bpf_ctx_convert_btf_id[0]); - refcount_set(&btf->refcnt, 1); - err = btf_alloc_id(btf); - if (err) - goto errout; - - btf_verifier_env_free(env); return btf; errout: - btf_verifier_env_free(env); if (btf) { kvfree(btf->types); kfree(btf); @@ -6049,19 +6123,59 @@ struct btf *btf_parse_vmlinux(void) return ERR_PTR(err); } +struct btf *btf_parse_vmlinux(void) +{ + struct btf_verifier_env *env = NULL; + struct bpf_verifier_log *log; + struct btf *btf; + int err; + + env = kzalloc(sizeof(*env), GFP_KERNEL | __GFP_NOWARN); + if (!env) + return ERR_PTR(-ENOMEM); + + log = &env->log; + log->level = BPF_LOG_KERNEL; + btf = btf_parse_base(env, "vmlinux", __start_BTF, __stop_BTF - __start_BTF); + if (!IS_ERR(btf)) { + /* btf_parse_vmlinux() runs under bpf_verifier_lock */ + bpf_ctx_convert.t = btf_type_by_id(btf, bpf_ctx_convert_btf_id[0]); + err = btf_alloc_id(btf); + if (err) { + btf_free(btf); + btf = ERR_PTR(err); + } + } + btf_verifier_env_free(env); + return btf; +} + #ifdef CONFIG_DEBUG_INFO_BTF_MODULES -static struct btf *btf_parse_module(const char *module_name, const void *data, unsigned int data_size) +/* If .BTF_ids section was created with distilled base BTF, both base and + * split BTF ids will need to be mapped to actual base/split ids for + * BTF now that it has been relocated. + */ +static __u32 btf_id_map(const struct btf *btf, __u32 id) +{ + if (!btf->base_btf || !btf->base_map) + return id; + return btf->base_map[id]; +} + +static struct btf *btf_parse_module(const char *module_name, const void *data, + unsigned int data_size, void *base_data, + unsigned int base_data_size) { + struct btf *btf = NULL, *vmlinux_btf, *base_btf = NULL; struct btf_verifier_env *env = NULL; struct bpf_verifier_log *log; - struct btf *btf = NULL, *base_btf; - int err; + int err = 0; - base_btf = bpf_get_btf_vmlinux(); - if (IS_ERR(base_btf)) - return base_btf; - if (!base_btf) + vmlinux_btf = bpf_get_btf_vmlinux(); + if (IS_ERR(vmlinux_btf)) + return vmlinux_btf; + if (!vmlinux_btf) return ERR_PTR(-EINVAL); env = kzalloc(sizeof(*env), GFP_KERNEL | __GFP_NOWARN); @@ -6071,6 +6185,16 @@ static struct btf *btf_parse_module(const char *module_name, const void *data, u log = &env->log; log->level = BPF_LOG_KERNEL; + if (base_data) { + base_btf = btf_parse_base(env, ".BTF.base", base_data, base_data_size); + if (IS_ERR(base_btf)) { + err = PTR_ERR(base_btf); + goto errout; + } + } else { + base_btf = vmlinux_btf; + } + btf = kzalloc(sizeof(*btf), GFP_KERNEL | __GFP_NOWARN); if (!btf) { err = -ENOMEM; @@ -6110,12 +6234,22 @@ static struct btf *btf_parse_module(const char *module_name, const void *data, u if (err) goto errout; + if (base_btf != vmlinux_btf) { + err = btf_relocate(btf, vmlinux_btf, &btf->base_map); + if (err) + goto errout; + btf_free(base_btf); + base_btf = vmlinux_btf; + } + btf_verifier_env_free(env); refcount_set(&btf->refcnt, 1); return btf; errout: btf_verifier_env_free(env); + if (base_btf != vmlinux_btf) + btf_free(base_btf); if (btf) { kvfree(btf->data); kvfree(btf->types); @@ -7668,7 +7802,8 @@ static int btf_module_notify(struct notifier_block *nb, unsigned long op, err = -ENOMEM; goto out; } - btf = btf_parse_module(mod->name, mod->btf_data, mod->btf_data_size); + btf = btf_parse_module(mod->name, mod->btf_data, mod->btf_data_size, + mod->btf_base_data, mod->btf_base_data_size); if (IS_ERR(btf)) { kfree(btf_mod); if (!IS_ENABLED(CONFIG_MODULE_ALLOW_BTF_MISMATCH)) { @@ -7992,7 +8127,7 @@ static int btf_populate_kfunc_set(struct btf *btf, enum btf_kfunc_hook hook, bool add_filter = !!kset->filter; struct btf_kfunc_set_tab *tab; struct btf_id_set8 *set; - u32 set_cnt; + u32 set_cnt, i; int ret; if (hook >= BTF_KFUNC_HOOK_MAX) { @@ -8038,21 +8173,15 @@ static int btf_populate_kfunc_set(struct btf *btf, enum btf_kfunc_hook hook, goto end; } - /* We don't need to allocate, concatenate, and sort module sets, because - * only one is allowed per hook. Hence, we can directly assign the - * pointer and return. - */ - if (!vmlinux_set) { - tab->sets[hook] = add_set; - goto do_add_filter; - } - /* In case of vmlinux sets, there may be more than one set being * registered per hook. To create a unified set, we allocate a new set * and concatenate all individual sets being registered. While each set * is individually sorted, they may become unsorted when concatenated, * hence re-sorting the final set again is required to make binary * searching the set using btf_id_set8_contains function work. + * + * For module sets, we need to allocate as we may need to relocate + * BTF ids. */ set_cnt = set ? set->cnt : 0; @@ -8082,11 +8211,14 @@ static int btf_populate_kfunc_set(struct btf *btf, enum btf_kfunc_hook hook, /* Concatenate the two sets */ memcpy(set->pairs + set->cnt, add_set->pairs, add_set->cnt * sizeof(set->pairs[0])); + /* Now that the set is copied, update with relocated BTF ids */ + for (i = set->cnt; i < set->cnt + add_set->cnt; i++) + set->pairs[i].id = btf_id_map(btf, set->pairs[i].id); + set->cnt += add_set->cnt; sort(set->pairs, set->cnt, sizeof(set->pairs[0]), btf_id_cmp_func, NULL); -do_add_filter: if (add_filter) { hook_filter = &tab->hook_filters[hook]; hook_filter->filters[hook_filter->nr_filters++] = kset->filter; @@ -8204,7 +8336,7 @@ static int __register_btf_kfunc_id_set(enum btf_kfunc_hook hook, return PTR_ERR(btf); for (i = 0; i < kset->set->cnt; i++) { - ret = btf_check_kfunc_protos(btf, kset->set->pairs[i].id, + ret = btf_check_kfunc_protos(btf, btf_id_map(btf, kset->set->pairs[i].id), kset->set->pairs[i].flags); if (ret) goto err_out; @@ -8303,7 +8435,7 @@ int register_btf_id_dtor_kfuncs(const struct btf_id_dtor_kfunc *dtors, u32 add_c { struct btf_id_dtor_kfunc_tab *tab; struct btf *btf; - u32 tab_cnt; + u32 tab_cnt, i; int ret; btf = btf_get_module_btf(owner); @@ -8354,6 +8486,13 @@ int register_btf_id_dtor_kfuncs(const struct btf_id_dtor_kfunc *dtors, u32 add_c btf->dtor_kfunc_tab = tab; memcpy(tab->dtors + tab->cnt, dtors, add_cnt * sizeof(tab->dtors[0])); + + /* remap BTF ids based on BTF relocation (if any) */ + for (i = tab_cnt; i < tab_cnt + add_cnt; i++) { + tab->dtors[i].btf_id = btf_id_map(btf, tab->dtors[i].btf_id); + tab->dtors[i].kfunc_btf_id = btf_id_map(btf, tab->dtors[i].kfunc_btf_id); + } + tab->cnt += add_cnt; sort(tab->dtors, tab->cnt, sizeof(tab->dtors[0]), btf_id_cmp_func, NULL); diff --git a/tools/lib/bpf/Build b/tools/lib/bpf/Build index 336da6844d42..567abaa52131 100644 --- a/tools/lib/bpf/Build +++ b/tools/lib/bpf/Build @@ -1,4 +1,4 @@ libbpf-y := libbpf.o bpf.o nlattr.o btf.o libbpf_errno.o str_error.o \ netlink.o bpf_prog_linfo.o libbpf_probes.o hashmap.o \ btf_dump.o ringbuf.o strset.o linker.o gen_loader.o relo_core.o \ - usdt.o zip.o elf.o features.o btf_relocate.o + usdt.o zip.o elf.o features.o btf_common.o btf_relocate.o diff --git a/tools/lib/bpf/btf.c b/tools/lib/bpf/btf.c index f00a84fea9b5..7be33560bd94 100644 --- a/tools/lib/bpf/btf.c +++ b/tools/lib/bpf/btf.c @@ -5034,136 +5034,6 @@ struct btf *btf__load_module_btf(const char *module_name, struct btf *vmlinux_bt return btf__parse_split(path, vmlinux_btf); } -int btf_type_visit_type_ids(struct btf_type *t, type_id_visit_fn visit, void *ctx) -{ - int i, n, err; - - switch (btf_kind(t)) { - case BTF_KIND_INT: - case BTF_KIND_FLOAT: - case BTF_KIND_ENUM: - case BTF_KIND_ENUM64: - return 0; - - case BTF_KIND_FWD: - case BTF_KIND_CONST: - case BTF_KIND_VOLATILE: - case BTF_KIND_RESTRICT: - case BTF_KIND_PTR: - case BTF_KIND_TYPEDEF: - case BTF_KIND_FUNC: - case BTF_KIND_VAR: - case BTF_KIND_DECL_TAG: - case BTF_KIND_TYPE_TAG: - return visit(&t->type, ctx); - - case BTF_KIND_ARRAY: { - struct btf_array *a = btf_array(t); - - err = visit(&a->type, ctx); - err = err ?: visit(&a->index_type, ctx); - return err; - } - - case BTF_KIND_STRUCT: - case BTF_KIND_UNION: { - struct btf_member *m = btf_members(t); - - for (i = 0, n = btf_vlen(t); i < n; i++, m++) { - err = visit(&m->type, ctx); - if (err) - return err; - } - return 0; - } - - case BTF_KIND_FUNC_PROTO: { - struct btf_param *m = btf_params(t); - - err = visit(&t->type, ctx); - if (err) - return err; - for (i = 0, n = btf_vlen(t); i < n; i++, m++) { - err = visit(&m->type, ctx); - if (err) - return err; - } - return 0; - } - - case BTF_KIND_DATASEC: { - struct btf_var_secinfo *m = btf_var_secinfos(t); - - for (i = 0, n = btf_vlen(t); i < n; i++, m++) { - err = visit(&m->type, ctx); - if (err) - return err; - } - return 0; - } - - default: - return -EINVAL; - } -} - -int btf_type_visit_str_offs(struct btf_type *t, str_off_visit_fn visit, void *ctx) -{ - int i, n, err; - - err = visit(&t->name_off, ctx); - if (err) - return err; - - switch (btf_kind(t)) { - case BTF_KIND_STRUCT: - case BTF_KIND_UNION: { - struct btf_member *m = btf_members(t); - - for (i = 0, n = btf_vlen(t); i < n; i++, m++) { - err = visit(&m->name_off, ctx); - if (err) - return err; - } - break; - } - case BTF_KIND_ENUM: { - struct btf_enum *m = btf_enum(t); - - for (i = 0, n = btf_vlen(t); i < n; i++, m++) { - err = visit(&m->name_off, ctx); - if (err) - return err; - } - break; - } - case BTF_KIND_ENUM64: { - struct btf_enum64 *m = btf_enum64(t); - - for (i = 0, n = btf_vlen(t); i < n; i++, m++) { - err = visit(&m->name_off, ctx); - if (err) - return err; - } - break; - } - case BTF_KIND_FUNC_PROTO: { - struct btf_param *m = btf_params(t); - - for (i = 0, n = btf_vlen(t); i < n; i++, m++) { - err = visit(&m->name_off, ctx); - if (err) - return err; - } - break; - } - default: - break; - } - - return 0; -} - int btf_ext_visit_type_ids(struct btf_ext *btf_ext, type_id_visit_fn visit, void *ctx) { const struct btf_ext_info *seg; diff --git a/tools/lib/bpf/btf_common.c b/tools/lib/bpf/btf_common.c new file mode 100644 index 000000000000..ddec3f3ac423 --- /dev/null +++ b/tools/lib/bpf/btf_common.c @@ -0,0 +1,146 @@ +// SPDX-License-Identifier: (LGPL-2.1 OR BSD-2-Clause) +/* Copyright (c) 2021 Facebook */ +/* Copyright (c) 2024, Oracle and/or its affiliates. */ + +#ifdef __KERNEL__ +#include +#include + +static inline struct btf_var_secinfo *btf_var_secinfos(const struct btf_type *t) +{ + return (struct btf_var_secinfo *)(t + 1); +} + +#else +#include "btf.h" +#include "libbpf_internal.h" +#endif + +int btf_type_visit_type_ids(struct btf_type *t, type_id_visit_fn visit, void *ctx) +{ + int i, n, err; + + switch (btf_kind(t)) { + case BTF_KIND_INT: + case BTF_KIND_FLOAT: + case BTF_KIND_ENUM: + case BTF_KIND_ENUM64: + return 0; + + case BTF_KIND_FWD: + case BTF_KIND_CONST: + case BTF_KIND_VOLATILE: + case BTF_KIND_RESTRICT: + case BTF_KIND_PTR: + case BTF_KIND_TYPEDEF: + case BTF_KIND_FUNC: + case BTF_KIND_VAR: + case BTF_KIND_DECL_TAG: + case BTF_KIND_TYPE_TAG: + return visit(&t->type, ctx); + + case BTF_KIND_ARRAY: { + struct btf_array *a = btf_array(t); + + err = visit(&a->type, ctx); + err = err ?: visit(&a->index_type, ctx); + return err; + } + + case BTF_KIND_STRUCT: + case BTF_KIND_UNION: { + struct btf_member *m = btf_members(t); + + for (i = 0, n = btf_vlen(t); i < n; i++, m++) { + err = visit(&m->type, ctx); + if (err) + return err; + } + return 0; + } + case BTF_KIND_FUNC_PROTO: { + struct btf_param *m = btf_params(t); + + err = visit(&t->type, ctx); + if (err) + return err; + for (i = 0, n = btf_vlen(t); i < n; i++, m++) { + err = visit(&m->type, ctx); + if (err) + return err; + } + return 0; + } + + case BTF_KIND_DATASEC: { + struct btf_var_secinfo *m = btf_var_secinfos(t); + + for (i = 0, n = btf_vlen(t); i < n; i++, m++) { + err = visit(&m->type, ctx); + if (err) + return err; + } + return 0; + } + + default: + return -EINVAL; + } +} + +int btf_type_visit_str_offs(struct btf_type *t, str_off_visit_fn visit, void *ctx) +{ + int i, n, err; + + err = visit(&t->name_off, ctx); + if (err) + return err; + + switch (btf_kind(t)) { + case BTF_KIND_STRUCT: + case BTF_KIND_UNION: { + struct btf_member *m = btf_members(t); + + for (i = 0, n = btf_vlen(t); i < n; i++, m++) { + err = visit(&m->name_off, ctx); + if (err) + return err; + } + break; + } + case BTF_KIND_ENUM: { + struct btf_enum *m = btf_enum(t); + + for (i = 0, n = btf_vlen(t); i < n; i++, m++) { + err = visit(&m->name_off, ctx); + if (err) + return err; + } + break; + } + case BTF_KIND_ENUM64: { + struct btf_enum64 *m = btf_enum64(t); + + for (i = 0, n = btf_vlen(t); i < n; i++, m++) { + err = visit(&m->name_off, ctx); + if (err) + return err; + } + break; + } + case BTF_KIND_FUNC_PROTO: { + struct btf_param *m = btf_params(t); + + for (i = 0, n = btf_vlen(t); i < n; i++, m++) { + err = visit(&m->name_off, ctx); + if (err) + return err; + } + break; + } + default: + break; + } + + return 0; +} diff --git a/tools/lib/bpf/btf_relocate.c b/tools/lib/bpf/btf_relocate.c index d9340375f4a3..8d3865cf193b 100644 --- a/tools/lib/bpf/btf_relocate.c +++ b/tools/lib/bpf/btf_relocate.c @@ -1,11 +1,40 @@ // SPDX-License-Identifier: GPL-2.0 /* Copyright (c) 2024, Oracle and/or its affiliates. */ +#ifdef __KERNEL__ +#include +#include +#include +#include + +#define btf__type_by_id btf_type_by_id +#define btf__type_cnt btf_nr_types +#define btf__base_btf btf_base_btf +#define btf__name_by_offset btf_name_by_offset +#define btf_kflag btf_type_kflag + +#define calloc(nmemb, size) kvcalloc(nmemb, size, GFP_KERNEL | __GFP_NOWARN) +#define free(ptr) kvfree(ptr) + +static inline __u8 btf_int_bits(const struct btf_type *t) +{ + return BTF_INT_BITS(*(__u32 *)(t + 1)); +} + +static inline struct btf_decl_tag *btf_decl_tag(const struct btf_type *t) +{ + return (struct btf_decl_tag *)(t + 1); +} + +#else + #include "btf.h" #include "bpf.h" #include "libbpf.h" #include "libbpf_internal.h" +#endif /* __KERNEL__ */ + struct btf; #define BTF_MAX_NR_TYPES 0x7fffffffU From patchwork Wed Apr 24 15:48:05 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alan Maguire X-Patchwork-Id: 13642133 X-Patchwork-Delegate: bpf@iogearbox.net Received: from mx0b-00069f02.pphosted.com (mx0b-00069f02.pphosted.com [205.220.177.32]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id EEAF416D9A5 for ; Wed, 24 Apr 2024 15:49:34 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=205.220.177.32 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1713973776; cv=none; b=uNE8O0F6m1/rdVztoRpO/7MQhKblX0q7xeTwKYZB3HrCESpFrRglWrXCe4OvPSRj1r5a3B6TsBgrz4pt1p3qY1lvYp2qbyZjZxVjLxNpzzaVhTPzpUIogKrSIbJLep+goX0s8GmuZsdCH0rfKf0FPwXTRzQTOGnmoJ7sbjXH55s= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1713973776; c=relaxed/simple; bh=I7oSYWdT+ZdkD7DRzoExIWzxFKLReJOFDQOPawyLKaA=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=IZO7XMBXTR3KpXMHv/uHe9ErSV/fdWX8fjRNbCAKp4qUM7ywtZEJGVTq9RcBY+t4sDdwYdwhKBzMvHWEQKrjx2GzgPnolwy31BMsjzO/wEFRrt7VdXQoW6nE5DfM5nzT3UG2gyYRJpmQSNj4g5B93alKj6MKESBgRn4SsNmhkhs= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=oracle.com; spf=pass smtp.mailfrom=oracle.com; dkim=pass (2048-bit key) header.d=oracle.com header.i=@oracle.com header.b=P4EYKmQ5; arc=none smtp.client-ip=205.220.177.32 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=oracle.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=oracle.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=oracle.com header.i=@oracle.com header.b="P4EYKmQ5" Received: from pps.filterd (m0246632.ppops.net [127.0.0.1]) by mx0b-00069f02.pphosted.com (8.17.1.19/8.17.1.19) with ESMTP id 43OAHGeW023141; Wed, 24 Apr 2024 15:49:13 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding; s=corp-2023-11-20; bh=hLaDzzRRHGr4jqCWf5vwHZXmrbE0YLSBXG9nIJS8igM=; b=P4EYKmQ5tTZN45rUHJyBook+gij/0NzmA9JyyMdmFnyoblzpbrpPbx61ev4KafiR8hmZ D71wmemu5yUhdYKRTngfKwtoAWnTfOmFkK51IfbHykxemmhvKlnYctB763R0ybUYWBfn ezwGhwiNPNqkvBaPgBmtYY79P6X23Q7fVD9RSPADOzO6KAPPM9rMw21OP56xJS8lWe4d tSSbBiIJxiW+U5sW1+/aEgdqKd8PmD+wFhhbMWX9Wb+zWpWbEYY6lNQSjUSY9wZFarHn 5quHdxqCU0heYHD8xrXpDlPYMylb0+pApGRbU3c4bdFkLpl2AMJprhCUXPvzX0p15MHs GQ== Received: from iadpaimrmta02.imrmtpd1.prodappiadaev1.oraclevcn.com (iadpaimrmta02.appoci.oracle.com [147.154.18.20]) by mx0b-00069f02.pphosted.com (PPS) with ESMTPS id 3xm5aurgv4-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Wed, 24 Apr 2024 15:49:12 +0000 Received: from pps.filterd (iadpaimrmta02.imrmtpd1.prodappiadaev1.oraclevcn.com [127.0.0.1]) by iadpaimrmta02.imrmtpd1.prodappiadaev1.oraclevcn.com (8.17.1.19/8.17.1.19) with ESMTP id 43OEapci025241; Wed, 24 Apr 2024 15:49:12 GMT Received: from pps.reinject (localhost [127.0.0.1]) by iadpaimrmta02.imrmtpd1.prodappiadaev1.oraclevcn.com (PPS) with ESMTPS id 3xm45fb0j7-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Wed, 24 Apr 2024 15:49:12 +0000 Received: from iadpaimrmta02.imrmtpd1.prodappiadaev1.oraclevcn.com (iadpaimrmta02.imrmtpd1.prodappiadaev1.oraclevcn.com [127.0.0.1]) by pps.reinject (8.17.1.5/8.17.1.5) with ESMTP id 43OFmCom008769; Wed, 24 Apr 2024 15:49:11 GMT Received: from bpf.uk.oracle.com (dhcp-10-175-216-158.vpn.oracle.com [10.175.216.158]) by iadpaimrmta02.imrmtpd1.prodappiadaev1.oraclevcn.com (PPS) with ESMTP id 3xm45faxuq-13; Wed, 24 Apr 2024 15:49:11 +0000 From: Alan Maguire To: andrii@kernel.org, ast@kernel.org Cc: jolsa@kernel.org, acme@redhat.com, quentin@isovalent.com, eddyz87@gmail.com, mykolal@fb.com, daniel@iogearbox.net, martin.lau@linux.dev, song@kernel.org, yonghong.song@linux.dev, john.fastabend@gmail.com, kpsingh@kernel.org, sdf@google.com, haoluo@google.com, houtao1@huawei.com, bpf@vger.kernel.org, masahiroy@kernel.org, mcgrof@kernel.org, nathan@kernel.org, Alan Maguire Subject: [PATCH v2 bpf-next 12/13] selftests/bpf: extend distilled BTF tests to cover BTF relocation Date: Wed, 24 Apr 2024 16:48:05 +0100 Message-Id: <20240424154806.3417662-13-alan.maguire@oracle.com> X-Mailer: git-send-email 2.39.3 In-Reply-To: <20240424154806.3417662-1-alan.maguire@oracle.com> References: <20240424154806.3417662-1-alan.maguire@oracle.com> Precedence: bulk X-Mailing-List: bpf@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293,Aquarius:18.0.1011,Hydra:6.0.650,FMLib:17.11.176.26 definitions=2024-04-24_13,2024-04-24_01,2023-05-22_02 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 suspectscore=0 mlxlogscore=999 mlxscore=0 phishscore=0 spamscore=0 malwarescore=0 adultscore=0 bulkscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2404010000 definitions=main-2404240063 X-Proofpoint-ORIG-GUID: ciDw1GBAOzoDyj8dJ8RbU_p_C1gWmuEw X-Proofpoint-GUID: ciDw1GBAOzoDyj8dJ8RbU_p_C1gWmuEw X-Patchwork-Delegate: bpf@iogearbox.net Ensure relocated BTF looks as expected; in this case identical to original split BTF. Signed-off-by: Alan Maguire --- .../selftests/bpf/prog_tests/btf_distill.c | 45 +++++++++++++++++++ 1 file changed, 45 insertions(+) diff --git a/tools/testing/selftests/bpf/prog_tests/btf_distill.c b/tools/testing/selftests/bpf/prog_tests/btf_distill.c index aae9aef68bd6..67cc98227c12 100644 --- a/tools/testing/selftests/bpf/prog_tests/btf_distill.c +++ b/tools/testing/selftests/bpf/prog_tests/btf_distill.c @@ -192,6 +192,51 @@ static void test_distilled_base(void) "[22] FUNC 'fn' type_id=11 linkage=static", "[23] TYPEDEF 'arraytype' type_id=12"); + if (!ASSERT_EQ(btf__relocate(btf4, btf1), 0, "relocate_split")) + goto cleanup; + VALIDATE_RAW_BTF( + btf4, + "[1] INT 'int' size=4 bits_offset=0 nr_bits=32 encoding=SIGNED", + "[2] PTR '(anon)' type_id=1", + "[3] STRUCT 's1' size=8 vlen=1\n" + "\t'f1' type_id=2 bits_offset=0", + "[4] STRUCT '(anon)' size=12 vlen=2\n" + "\t'f1' type_id=1 bits_offset=0\n" + "\t'f2' type_id=3 bits_offset=32", + "[5] INT 'unsigned int' size=4 bits_offset=0 nr_bits=32 encoding=(none)", + "[6] UNION 'u1' size=12 vlen=2\n" + "\t'f1' type_id=1 bits_offset=0\n" + "\t'f2' type_id=2 bits_offset=0", + "[7] UNION '(anon)' size=4 vlen=1\n" + "\t'f1' type_id=1 bits_offset=0", + "[8] ENUM 'e1' encoding=UNSIGNED size=4 vlen=1\n" + "\t'v1' val=1", + "[9] ENUM '(anon)' encoding=UNSIGNED size=4 vlen=1\n" + "\t'av1' val=2", + "[10] ENUM64 'e641' encoding=SIGNED size=8 vlen=1\n" + "\t'v1' val=1024", + "[11] ENUM64 '(anon)' encoding=SIGNED size=8 vlen=1\n" + "\t'v1' val=1025", + "[12] STRUCT 'unneeded' size=4 vlen=1\n" + "\t'f1' type_id=1 bits_offset=0", + "[13] STRUCT 'embedded' size=4 vlen=1\n" + "\t'f1' type_id=1 bits_offset=0", + "[14] FUNC_PROTO '(anon)' ret_type_id=1 vlen=1\n" + "\t'p1' type_id=1", + "[15] ARRAY '(anon)' type_id=1 index_type_id=1 nr_elems=3", + "[16] PTR '(anon)' type_id=3", + "[17] PTR '(anon)' type_id=4", + "[18] CONST '(anon)' type_id=6", + "[19] RESTRICT '(anon)' type_id=7", + "[20] VOLATILE '(anon)' type_id=8", + "[21] TYPEDEF 'et' type_id=9", + "[22] CONST '(anon)' type_id=10", + "[23] PTR '(anon)' type_id=11", + "[24] STRUCT 'with_embedded' size=4 vlen=1\n" + "\t'f1' type_id=13 bits_offset=0", + "[25] FUNC 'fn' type_id=14 linkage=static", + "[26] TYPEDEF 'arraytype' type_id=15"); + cleanup: btf__free(btf4); btf__free(btf3); From patchwork Wed Apr 24 15:48:06 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alan Maguire X-Patchwork-Id: 13642135 X-Patchwork-Delegate: bpf@iogearbox.net Received: from mx0b-00069f02.pphosted.com (mx0b-00069f02.pphosted.com [205.220.177.32]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 8740116193E for ; Wed, 24 Apr 2024 15:49:42 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=205.220.177.32 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1713973784; cv=none; b=AIxcr8utvlD1ee12XoLuj8iUduKZrgUN1py5VQTlpQg2GErDNMtlI18qGH69MSVTqzXz5Ybo3q+rDa4TJfTuzzHcpvxeEIYY8C557I+qtcLeFmhVyBX74y8ysUp8PKkii37bRUCZkKEphet3u6QrGLWUpYNm6C24vkWI+t54nOo= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1713973784; c=relaxed/simple; bh=+HZXhraZgxOkG2ZhCmB2sEfbIoA6LJa7cp7h9ZQwenc=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=PB+GSrAZYc4u/kQpaXhq53dzm1sGcs9is5bKjHdYBeNdnM7yndgfBfmQGyEz4478VfycFOq/LDjBYersogWZKfO3mXbq3vmWs3kdnYz7bGxGgZ3BzETXVnbExzLQ7t5DV8uE3U2gCEAjcfpVJ/dyKvNdsnGapFgmPb3rpBOoYYo= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=oracle.com; spf=pass smtp.mailfrom=oracle.com; dkim=pass (2048-bit key) header.d=oracle.com header.i=@oracle.com header.b=Jl+mtTZ0; arc=none smtp.client-ip=205.220.177.32 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=oracle.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=oracle.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=oracle.com header.i=@oracle.com header.b="Jl+mtTZ0" Received: from pps.filterd (m0246632.ppops.net [127.0.0.1]) by mx0b-00069f02.pphosted.com (8.17.1.19/8.17.1.19) with ESMTP id 43OFQBKE023161; Wed, 24 Apr 2024 15:49:17 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding; s=corp-2023-11-20; bh=fWkruEM9KuqgXoWx/b4UIshz413t7thTfPdnBi4uYuY=; b=Jl+mtTZ0T9aKaYHmALpy2FxSDuqeoohY8qVxFYxggjgdlzslUpnvAp4XR+BeVJ1GbPK2 YqsHAPPK7T9H3uFzhz5WzwOFihW+FO/sB1sHucfuLJG4jMXuNuzlr5BQdxelDLci/mJ0 CFC3/tuvDv2d1GUjPbdh4s+0vrr7bhbJkVOUQ9NHepmYwhGlH8fUqJqQDPAO8ZmNkjUB uxQZPuSH+U4CLQ0q80JGa5/A3uWxGrXPOBvYHFErWEupfCgzb7rLaA/G5ffYFDqbUFuK wuRWshxILgnbcF4ZkubjTQWjB7wMamqLiiikcHA0xN8CD3djHzdnf4Na44NvBudNgu3C +Q== Received: from iadpaimrmta02.imrmtpd1.prodappiadaev1.oraclevcn.com (iadpaimrmta02.appoci.oracle.com [147.154.18.20]) by mx0b-00069f02.pphosted.com (PPS) with ESMTPS id 3xm5aurgvc-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Wed, 24 Apr 2024 15:49:17 +0000 Received: from pps.filterd (iadpaimrmta02.imrmtpd1.prodappiadaev1.oraclevcn.com [127.0.0.1]) by iadpaimrmta02.imrmtpd1.prodappiadaev1.oraclevcn.com (8.17.1.19/8.17.1.19) with ESMTP id 43OFbl0H025235; Wed, 24 Apr 2024 15:49:17 GMT Received: from pps.reinject (localhost [127.0.0.1]) by iadpaimrmta02.imrmtpd1.prodappiadaev1.oraclevcn.com (PPS) with ESMTPS id 3xm45fb0n2-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Wed, 24 Apr 2024 15:49:16 +0000 Received: from iadpaimrmta02.imrmtpd1.prodappiadaev1.oraclevcn.com (iadpaimrmta02.imrmtpd1.prodappiadaev1.oraclevcn.com [127.0.0.1]) by pps.reinject (8.17.1.5/8.17.1.5) with ESMTP id 43OFmCoo008769; Wed, 24 Apr 2024 15:49:16 GMT Received: from bpf.uk.oracle.com (dhcp-10-175-216-158.vpn.oracle.com [10.175.216.158]) by iadpaimrmta02.imrmtpd1.prodappiadaev1.oraclevcn.com (PPS) with ESMTP id 3xm45faxuq-14; Wed, 24 Apr 2024 15:49:16 +0000 From: Alan Maguire To: andrii@kernel.org, ast@kernel.org Cc: jolsa@kernel.org, acme@redhat.com, quentin@isovalent.com, eddyz87@gmail.com, mykolal@fb.com, daniel@iogearbox.net, martin.lau@linux.dev, song@kernel.org, yonghong.song@linux.dev, john.fastabend@gmail.com, kpsingh@kernel.org, sdf@google.com, haoluo@google.com, houtao1@huawei.com, bpf@vger.kernel.org, masahiroy@kernel.org, mcgrof@kernel.org, nathan@kernel.org, Alan Maguire Subject: [PATCH v2 bpf-next 13/13] bpftool: support displaying relocated-with-base split BTF Date: Wed, 24 Apr 2024 16:48:06 +0100 Message-Id: <20240424154806.3417662-14-alan.maguire@oracle.com> X-Mailer: git-send-email 2.39.3 In-Reply-To: <20240424154806.3417662-1-alan.maguire@oracle.com> References: <20240424154806.3417662-1-alan.maguire@oracle.com> Precedence: bulk X-Mailing-List: bpf@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293,Aquarius:18.0.1011,Hydra:6.0.650,FMLib:17.11.176.26 definitions=2024-04-24_13,2024-04-24_01,2023-05-22_02 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 suspectscore=0 mlxlogscore=999 mlxscore=0 phishscore=0 spamscore=0 malwarescore=0 adultscore=0 bulkscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2404010000 definitions=main-2404240063 X-Proofpoint-ORIG-GUID: 0xtKpJNIRuM-eqgfaWGbEN01gmd4XL9W X-Proofpoint-GUID: 0xtKpJNIRuM-eqgfaWGbEN01gmd4XL9W X-Patchwork-Delegate: bpf@iogearbox.net If the -R option is used, we can display BTF that has been generated with distilled base BTF in its relocated form. For example for bpf_testmod.ko (which is built as an out-of-tree module, so has a distilled .BTF.base section: bpftool btf dump file bpf_testmod.ko Alternatively, we can display content relocated with (a possibly changed) base BTF via bpftool btf dump -R /sys/kernel/btf/vmlinux bpf_testmod.ko The latter mirrors how the kernel will handle such split BTF; it relocates its representation with the running kernel, and if successful, renumbers BTF ids to reference the current vmlinux BTF. Signed-off-by: Alan Maguire --- tools/bpf/bpftool/Documentation/bpftool-btf.rst | 15 ++++++++++++++- tools/bpf/bpftool/bash-completion/bpftool | 7 ++++--- tools/bpf/bpftool/btf.c | 11 ++++++++++- tools/bpf/bpftool/main.c | 14 +++++++++++++- tools/bpf/bpftool/main.h | 2 ++ 5 files changed, 43 insertions(+), 6 deletions(-) diff --git a/tools/bpf/bpftool/Documentation/bpftool-btf.rst b/tools/bpf/bpftool/Documentation/bpftool-btf.rst index eaba24320fb2..fd6bb1280e7b 100644 --- a/tools/bpf/bpftool/Documentation/bpftool-btf.rst +++ b/tools/bpf/bpftool/Documentation/bpftool-btf.rst @@ -16,7 +16,7 @@ SYNOPSIS **bpftool** [*OPTIONS*] **btf** *COMMAND* -*OPTIONS* := { |COMMON_OPTIONS| | { **-B** | **--base-btf** } } +*OPTIONS* := { |COMMON_OPTIONS| | { **-B** | **--base-btf** } { **-R** | **relocate-base-btf** } } *COMMANDS* := { **dump** | **help** } @@ -85,6 +85,19 @@ OPTIONS BTF object is passed through other handles, this option becomes necessary. +-R, --relocate-base-btf *FILE* + When split BTF is generated with distilled base BTF for relocation, + the latter is stored in a .BTF.base section and allows us to later + relocate split BTF and a potentially-changed base BTF by using + information in the .BTF.base section about the base types referenced + from split BTF. Relocation is carried out against the split BTF + supplied via this parameter and the split BTF will then refer to + the base types supplied in *FILE*. + + If this option is not used, split BTF is shown relative to the + .BTF.base, which contains just enough information to support later + relocation. + EXAMPLES ======== **# bpftool btf dump id 1226** diff --git a/tools/bpf/bpftool/bash-completion/bpftool b/tools/bpf/bpftool/bash-completion/bpftool index 04afe2ac2228..878cf3d49a76 100644 --- a/tools/bpf/bpftool/bash-completion/bpftool +++ b/tools/bpf/bpftool/bash-completion/bpftool @@ -262,7 +262,7 @@ _bpftool() # Deal with options if [[ ${words[cword]} == -* ]]; then local c='--version --json --pretty --bpffs --mapcompat --debug \ - --use-loader --base-btf' + --use-loader --base-btf --relocate-base-btf' COMPREPLY=( $( compgen -W "$c" -- "$cur" ) ) return 0 fi @@ -283,7 +283,7 @@ _bpftool() _sysfs_get_netdevs return 0 ;; - file|pinned|-B|--base-btf) + file|pinned|-B|-R|--base-btf|--relocate-base-btf) _filedir return 0 ;; @@ -297,7 +297,8 @@ _bpftool() local i pprev for (( i=1; i < ${#words[@]}; )); do if [[ ${words[i]::1} == - ]] && - [[ ${words[i]} != "-B" ]] && [[ ${words[i]} != "--base-btf" ]]; then + [[ ${words[i]} != "-B" ]] && [[ ${words[i]} != "--base-btf" ]] && + [[ ${words[i]} != "-R" ]] && [[ ${words[i]} != "--relocate-base-btf" ]]; then words=( "${words[@]:0:i}" "${words[@]:i+1}" ) [[ $i -le $cword ]] && cword=$(( cword - 1 )) else diff --git a/tools/bpf/bpftool/btf.c b/tools/bpf/bpftool/btf.c index 2e8bd2c9f0a3..7df8a686fef7 100644 --- a/tools/bpf/bpftool/btf.c +++ b/tools/bpf/bpftool/btf.c @@ -639,6 +639,14 @@ static int do_dump(int argc, char **argv) base_btf = btf__parse_opts(*argv, &optp); if (base_btf) btf = btf__parse_split(*argv, base_btf); + if (btf && relocate_base_btf) { + err = btf__relocate(btf, relocate_base_btf); + if (err) { + p_err("could not relocate BTF from '%s' with base BTF '%s': %s\n", + *argv, relocate_base_btf_path, strerror(-err)); + goto done; + } + } } if (!btf) { err = -errno; @@ -1076,7 +1084,8 @@ static int do_help(int argc, char **argv) " " HELP_SPEC_MAP "\n" " " HELP_SPEC_PROGRAM "\n" " " HELP_SPEC_OPTIONS " |\n" - " {-B|--base-btf} }\n" + " {-B|--base-btf} |\n" + " {-R|--relocate-base-btf} }\n" "", bin_name, "btf"); diff --git a/tools/bpf/bpftool/main.c b/tools/bpf/bpftool/main.c index 08d0ac543c67..69d4906bec5c 100644 --- a/tools/bpf/bpftool/main.c +++ b/tools/bpf/bpftool/main.c @@ -32,6 +32,8 @@ bool verifier_logs; bool relaxed_maps; bool use_loader; struct btf *base_btf; +struct btf *relocate_base_btf; +const char *relocate_base_btf_path; struct hashmap *refs_table; static void __noreturn clean_and_exit(int i) @@ -448,6 +450,7 @@ int main(int argc, char **argv) { "debug", no_argument, NULL, 'd' }, { "use-loader", no_argument, NULL, 'L' }, { "base-btf", required_argument, NULL, 'B' }, + { "relocate-base-btf", required_argument, NULL, 'R' }, { 0 } }; bool version_requested = false; @@ -473,7 +476,7 @@ int main(int argc, char **argv) bin_name = "bpftool"; opterr = 0; - while ((opt = getopt_long(argc, argv, "VhpjfLmndB:l", + while ((opt = getopt_long(argc, argv, "VhpjfLmndB:lR:", options, NULL)) >= 0) { switch (opt) { case 'V': @@ -519,6 +522,15 @@ int main(int argc, char **argv) case 'L': use_loader = true; break; + case 'R': + relocate_base_btf_path = optarg; + relocate_base_btf = btf__parse(optarg, NULL); + if (!relocate_base_btf) { + p_err("failed to parse base BTF for relocation at '%s': %d\n", + optarg, -errno); + return -1; + } + break; default: p_err("unrecognized option '%s'", argv[optind - 1]); if (json_output) diff --git a/tools/bpf/bpftool/main.h b/tools/bpf/bpftool/main.h index 9eb764fe4cc8..bbf8194a2d76 100644 --- a/tools/bpf/bpftool/main.h +++ b/tools/bpf/bpftool/main.h @@ -83,6 +83,8 @@ extern bool verifier_logs; extern bool relaxed_maps; extern bool use_loader; extern struct btf *base_btf; +extern struct btf *relocate_base_btf; +extern const char *relocate_base_btf_path; extern struct hashmap *refs_table; void __printf(1, 2) p_err(const char *fmt, ...);