From patchwork Wed May 31 20:19:28 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alan Maguire X-Patchwork-Id: 13262581 X-Patchwork-Delegate: bpf@iogearbox.net Received: from lindbergh.monkeyblade.net (lindbergh.monkeyblade.net [23.128.96.19]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 1367834CE2 for ; Wed, 31 May 2023 20:21:24 +0000 (UTC) Received: from mx0b-00069f02.pphosted.com (mx0b-00069f02.pphosted.com [205.220.177.32]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 03B66121 for ; Wed, 31 May 2023 13:21:21 -0700 (PDT) Received: from pps.filterd (m0246631.ppops.net [127.0.0.1]) by mx0b-00069f02.pphosted.com (8.17.1.19/8.17.1.19) with ESMTP id 34VK7HN8031553; Wed, 31 May 2023 20:20:43 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding; s=corp-2023-03-30; bh=edpUMvx9nc746n340mpDJsAQcNBqSlvkm5A3SNEBO2o=; b=x0Axjs4ReFukVQqA4xBDO6tR83YkadLe6Vwxj/JSyWrmkwpEuaxLbVSfU2UhVYWw3ZzF hlV1ooFRfRsIcJjx5JEGXsc60v/OOhhnimDu1/kujGLj2By9e1OfSm9WyQfTIM/Sq/Am 7zxqHGRmKihYbKHp1UndXJ3COKxCrfNnND1ZjTUobqQ+EQsZVC5w230aZuHT2Lrf948F AL9WeOjsDRDuqtDyJGYO/zgDf5NW5JG1MWwcWmiumJoo5ClVWg+IYtafNuI7AmBiA1qc lM4fHMITMKsjfZQFFaLRwHnkHimxYwBSPx3MPDwQY996Yejx45ZhEZ3T18KgOilZ3W1Y iw== Received: from phxpaimrmta02.imrmtpd1.prodappphxaev1.oraclevcn.com (phxpaimrmta02.appoci.oracle.com [147.154.114.232]) by mx0b-00069f02.pphosted.com (PPS) with ESMTPS id 3qvhj4xww8-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Wed, 31 May 2023 20:20:42 +0000 Received: from pps.filterd (phxpaimrmta02.imrmtpd1.prodappphxaev1.oraclevcn.com [127.0.0.1]) by phxpaimrmta02.imrmtpd1.prodappphxaev1.oraclevcn.com (8.17.1.19/8.17.1.19) with ESMTP id 34VJkL0q019748; Wed, 31 May 2023 20:20:41 GMT Received: from pps.reinject (localhost [127.0.0.1]) by phxpaimrmta02.imrmtpd1.prodappphxaev1.oraclevcn.com (PPS) with ESMTPS id 3qu8a6djh2-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Wed, 31 May 2023 20:20:41 +0000 Received: from phxpaimrmta02.imrmtpd1.prodappphxaev1.oraclevcn.com (phxpaimrmta02.imrmtpd1.prodappphxaev1.oraclevcn.com [127.0.0.1]) by pps.reinject (8.17.1.5/8.17.1.5) with ESMTP id 34VKKaEN000653; Wed, 31 May 2023 20:20:40 GMT Received: from bpf.uk.oracle.com (dhcp-10-175-201-40.vpn.oracle.com [10.175.201.40]) by phxpaimrmta02.imrmtpd1.prodappphxaev1.oraclevcn.com (PPS) with ESMTP id 3qu8a6djab-2; Wed, 31 May 2023 20:20:40 +0000 From: Alan Maguire To: ast@kernel.org, daniel@iogearbox.net, andrii@kernel.org, acme@kernel.org Cc: martin.lau@linux.dev, song@kernel.org, yhs@fb.com, john.fastabend@gmail.com, kpsingh@kernel.org, sdf@google.com, haoluo@google.com, jolsa@kernel.org, quentin@isovalent.com, mykolal@fb.com, bpf@vger.kernel.org, Alan Maguire Subject: [RFC bpf-next 1/8] btf: add kind metadata encoding to UAPI Date: Wed, 31 May 2023 21:19:28 +0100 Message-Id: <20230531201936.1992188-2-alan.maguire@oracle.com> X-Mailer: git-send-email 2.39.3 In-Reply-To: <20230531201936.1992188-1-alan.maguire@oracle.com> References: <20230531201936.1992188-1-alan.maguire@oracle.com> Precedence: bulk X-Mailing-List: bpf@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.254,Aquarius:18.0.957,Hydra:6.0.573,FMLib:17.11.176.26 definitions=2023-05-31_14,2023-05-31_03,2023-05-22_02 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 spamscore=0 malwarescore=0 phishscore=0 bulkscore=0 adultscore=0 mlxscore=0 suspectscore=0 mlxlogscore=999 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2304280000 definitions=main-2305310172 X-Proofpoint-GUID: z4KnLF4RDRZ6s2A9h2PzI33ZCAL-J0Z- X-Proofpoint-ORIG-GUID: z4KnLF4RDRZ6s2A9h2PzI33ZCAL-J0Z- X-Spam-Status: No, score=-2.8 required=5.0 tests=BAYES_00,DKIMWL_WL_MED, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_LOW, RCVD_IN_MSPIKE_H5,RCVD_IN_MSPIKE_WL,SPF_HELO_NONE,SPF_NONE, T_SCC_BODY_TEXT_LINE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net X-Patchwork-Delegate: bpf@iogearbox.net X-Patchwork-State: RFC BTF kind metadata provides information to parse BTF kinds. By separating parsing BTF from using all the information it provides, we allow BTF to encode new features even if they cannot be used. This is helpful in particular for cases where newer tools for BTF generation run on an older kernel; BTF kinds may be present that the kernel cannot yet use, but at least it can parse the BTF provided. Meanwhile userspace tools with newer libbpf may be able to use the newer information. The intent is to support encoding of kind metadata optionally so that tools like pahole can add this information. So for each kind we record - a kind name string - kind-related flags - length of singular element following struct btf_type - length of each of the btf_vlen() elements following In addition we make space in the metadata for CRC32s computed over the BTF along with a CRC for the base BTF; this allows split BTF to identify a mismatch explicitly. Finally we provide an offset for an optional description string. The ideas here were discussed at [1] hence Suggested-by: Andrii Nakryiko Signed-off-by: Alan Maguire [1] https://lore.kernel.org/bpf/CAEf4BzYjWHRdNNw4B=eOXOs_ONrDwrgX4bn=Nuc1g8JPFC34MA@mail.gmail.com/ --- include/uapi/linux/btf.h | 29 +++++++++++++++++++++++++++++ tools/include/uapi/linux/btf.h | 29 +++++++++++++++++++++++++++++ 2 files changed, 58 insertions(+) diff --git a/include/uapi/linux/btf.h b/include/uapi/linux/btf.h index ec1798b6d3ff..94c1f4518249 100644 --- a/include/uapi/linux/btf.h +++ b/include/uapi/linux/btf.h @@ -8,6 +8,34 @@ #define BTF_MAGIC 0xeB9F #define BTF_VERSION 1 +/* is this information required? If so it cannot be sanitized safely. */ +#define BTF_KIND_META_OPTIONAL (1 << 0) + +struct btf_kind_meta { + __u32 name_off; /* kind name string offset */ + __u16 flags; /* see BTF_KIND_META_* values above */ + __u8 info_sz; /* size of singular element after btf_type */ + __u8 elem_sz; /* size of each of btf_vlen(t) elements */ +}; + +/* for CRCs for BTF, base BTF to be considered usable, flags must be set. */ +#define BTF_META_CRC_SET (1 << 0) +#define BTF_META_BASE_CRC_SET (1 << 1) + +struct btf_metadata { + __u8 kind_meta_cnt; /* number of struct btf_kind_meta */ + __u32 flags; + __u32 description_off; /* optional description string */ + __u32 crc; /* crc32 of BTF */ + __u32 base_crc; /* crc32 of base BTF */ + struct btf_kind_meta kind_meta[]; +}; + +struct btf_meta_header { + __u32 meta_off; /* offset of metadata section */ + __u32 meta_len; /* length of metadata section */ +}; + struct btf_header { __u16 magic; __u8 version; @@ -19,6 +47,7 @@ struct btf_header { __u32 type_len; /* length of type section */ __u32 str_off; /* offset of string section */ __u32 str_len; /* length of string section */ + struct btf_meta_header meta_header; }; /* Max # of type identifier */ diff --git a/tools/include/uapi/linux/btf.h b/tools/include/uapi/linux/btf.h index ec1798b6d3ff..94c1f4518249 100644 --- a/tools/include/uapi/linux/btf.h +++ b/tools/include/uapi/linux/btf.h @@ -8,6 +8,34 @@ #define BTF_MAGIC 0xeB9F #define BTF_VERSION 1 +/* is this information required? If so it cannot be sanitized safely. */ +#define BTF_KIND_META_OPTIONAL (1 << 0) + +struct btf_kind_meta { + __u32 name_off; /* kind name string offset */ + __u16 flags; /* see BTF_KIND_META_* values above */ + __u8 info_sz; /* size of singular element after btf_type */ + __u8 elem_sz; /* size of each of btf_vlen(t) elements */ +}; + +/* for CRCs for BTF, base BTF to be considered usable, flags must be set. */ +#define BTF_META_CRC_SET (1 << 0) +#define BTF_META_BASE_CRC_SET (1 << 1) + +struct btf_metadata { + __u8 kind_meta_cnt; /* number of struct btf_kind_meta */ + __u32 flags; + __u32 description_off; /* optional description string */ + __u32 crc; /* crc32 of BTF */ + __u32 base_crc; /* crc32 of base BTF */ + struct btf_kind_meta kind_meta[]; +}; + +struct btf_meta_header { + __u32 meta_off; /* offset of metadata section */ + __u32 meta_len; /* length of metadata section */ +}; + struct btf_header { __u16 magic; __u8 version; @@ -19,6 +47,7 @@ struct btf_header { __u32 type_len; /* length of type section */ __u32 str_off; /* offset of string section */ __u32 str_len; /* length of string section */ + struct btf_meta_header meta_header; }; /* Max # of type identifier */ From patchwork Wed May 31 20:19:29 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alan Maguire X-Patchwork-Id: 13262582 X-Patchwork-Delegate: bpf@iogearbox.net Received: from lindbergh.monkeyblade.net (lindbergh.monkeyblade.net [23.128.96.19]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 5DEB334CE2 for ; Wed, 31 May 2023 20:21:33 +0000 (UTC) Received: from mx0b-00069f02.pphosted.com (mx0b-00069f02.pphosted.com [205.220.177.32]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 595B210F for ; Wed, 31 May 2023 13:21:31 -0700 (PDT) Received: from pps.filterd (m0333520.ppops.net [127.0.0.1]) by mx0b-00069f02.pphosted.com (8.17.1.19/8.17.1.19) with ESMTP id 34VJdLCB023290; Wed, 31 May 2023 20:20:47 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding; s=corp-2023-03-30; bh=FCr74+Ee35aH+w4HAX8moS1a9f6ti5Ewo8PizQLylvg=; b=DmYBsyWlQmsl22+dcjEHxB5QyOlE85V4k3kRY64Q45Ju+FCOWTB9ZOQJyKvvUK74u0BP Joi4au6KRecJ2sxbKqiaksIHuHdivw2PgC3fZXHYh0VCo0M8/2lgpq8apJYF1xwSRIMB t7MD15vf52R1rsKhzA7qdzU++uESpku4EdqUuwPE47QfsBDVn6E8gYdeM5XEadgRY9gO gjXNL9QBbm3rzPYnD1UVfagufRlW2R9urle0mVs9gBL5CbkW4lm9r9Gl98um2Q7j7PNl aTuLjd+8qNufDsh5xH8rsvqv2tYuffUBi9vH1U4Z8EMeFVA1+ioBk72Jggsx1XZF0N52 gw== Received: from phxpaimrmta02.imrmtpd1.prodappphxaev1.oraclevcn.com (phxpaimrmta02.appoci.oracle.com [147.154.114.232]) by mx0b-00069f02.pphosted.com (PPS) with ESMTPS id 3qvhd9y2ck-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Wed, 31 May 2023 20:20:46 +0000 Received: from pps.filterd (phxpaimrmta02.imrmtpd1.prodappphxaev1.oraclevcn.com [127.0.0.1]) by phxpaimrmta02.imrmtpd1.prodappphxaev1.oraclevcn.com (8.17.1.19/8.17.1.19) with ESMTP id 34VJkL0s019748; Wed, 31 May 2023 20:20:45 GMT Received: from pps.reinject (localhost [127.0.0.1]) by phxpaimrmta02.imrmtpd1.prodappphxaev1.oraclevcn.com (PPS) with ESMTPS id 3qu8a6djkv-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Wed, 31 May 2023 20:20:45 +0000 Received: from phxpaimrmta02.imrmtpd1.prodappphxaev1.oraclevcn.com (phxpaimrmta02.imrmtpd1.prodappphxaev1.oraclevcn.com [127.0.0.1]) by pps.reinject (8.17.1.5/8.17.1.5) with ESMTP id 34VKKaEP000653; Wed, 31 May 2023 20:20:44 GMT Received: from bpf.uk.oracle.com (dhcp-10-175-201-40.vpn.oracle.com [10.175.201.40]) by phxpaimrmta02.imrmtpd1.prodappphxaev1.oraclevcn.com (PPS) with ESMTP id 3qu8a6djab-3; Wed, 31 May 2023 20:20:44 +0000 From: Alan Maguire To: ast@kernel.org, daniel@iogearbox.net, andrii@kernel.org, acme@kernel.org Cc: martin.lau@linux.dev, song@kernel.org, yhs@fb.com, john.fastabend@gmail.com, kpsingh@kernel.org, sdf@google.com, haoluo@google.com, jolsa@kernel.org, quentin@isovalent.com, mykolal@fb.com, bpf@vger.kernel.org, Alan Maguire Subject: [RFC bpf-next 2/8] libbpf: support handling of metadata section in BTF Date: Wed, 31 May 2023 21:19:29 +0100 Message-Id: <20230531201936.1992188-3-alan.maguire@oracle.com> X-Mailer: git-send-email 2.39.3 In-Reply-To: <20230531201936.1992188-1-alan.maguire@oracle.com> References: <20230531201936.1992188-1-alan.maguire@oracle.com> Precedence: bulk X-Mailing-List: bpf@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.254,Aquarius:18.0.957,Hydra:6.0.573,FMLib:17.11.176.26 definitions=2023-05-31_14,2023-05-31_03,2023-05-22_02 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 spamscore=0 malwarescore=0 phishscore=0 bulkscore=0 adultscore=0 mlxscore=0 suspectscore=0 mlxlogscore=999 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2304280000 definitions=main-2305310172 X-Proofpoint-GUID: 1AxV1jeHJeo9lpZrL4A4tMqwD0UDamAH X-Proofpoint-ORIG-GUID: 1AxV1jeHJeo9lpZrL4A4tMqwD0UDamAH X-Spam-Status: No, score=-2.8 required=5.0 tests=BAYES_00,DKIMWL_WL_MED, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_LOW, RCVD_IN_MSPIKE_H5,RCVD_IN_MSPIKE_WL,SPF_HELO_NONE,SPF_NONE, T_SCC_BODY_TEXT_LINE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net X-Patchwork-Delegate: bpf@iogearbox.net X-Patchwork-State: RFC support reading in metadata, fixing endian issues on reading; also support writing metadata section to raw BTF object. There is not yet an API to populate the metadata with meaningful information. Signed-off-by: Alan Maguire --- tools/lib/bpf/btf.c | 141 ++++++++++++++++++++++++++++++++++---------- 1 file changed, 111 insertions(+), 30 deletions(-) diff --git a/tools/lib/bpf/btf.c b/tools/lib/bpf/btf.c index 8484b563b53d..036dc1505969 100644 --- a/tools/lib/bpf/btf.c +++ b/tools/lib/bpf/btf.c @@ -16,6 +16,7 @@ #include #include #include +#include #include "btf.h" #include "bpf.h" #include "libbpf.h" @@ -39,36 +40,40 @@ struct btf { /* * When BTF is loaded from an ELF or raw memory it is stored - * in a contiguous memory block. The hdr, type_data, and, strs_data - * point inside that memory region to their respective parts of BTF - * representation: + * in a contiguous memory block. The hdr, type_data, strs_data, + * and optional meta_data point inside that memory region to their + * respective parts of BTF representation: * - * +--------------------------------+ - * | Header | Types | Strings | - * +--------------------------------+ - * ^ ^ ^ - * | | | - * hdr | | - * types_data-+ | - * strs_data------------+ + * +--------------------------------+----------+ + * | Header | Types | Strings | Metadata | + * +--------------------------------+----------+ + * ^ ^ ^ ^ + * | | | | + * hdr | | | + * types_data-+ | | + * strs_data------------+ | + * meta_data------------------------+ + * + * meta_data is optional. * * If BTF data is later modified, e.g., due to types added or * removed, BTF deduplication performed, etc, this contiguous - * representation is broken up into three independently allocated - * memory regions to be able to modify them independently. + * representation is broken up into three or four independently + * allocated memory regions to be able to modify them independently. * raw_data is nulled out at that point, but can be later allocated * and cached again if user calls btf__raw_data(), at which point - * raw_data will contain a contiguous copy of header, types, and - * strings: + * raw_data will contain a contiguous copy of header, types, strings + * and (again optionally) metadata: * - * +----------+ +---------+ +-----------+ - * | Header | | Types | | Strings | - * +----------+ +---------+ +-----------+ - * ^ ^ ^ - * | | | - * hdr | | - * types_data----+ | - * strset__data(strs_set)-----+ + * +----------+ +---------+ +-----------+ +----------+ + * | Header | | Types | | Strings | | Metadata | + * +----------+ +---------+ +-----------+ +---------_+ + * ^ ^ ^ ^ + * | | | | + * hdr | | | + * types_data----+ | | + * strset__data(strs_set)-----+ | + * meta_data---------------------------------+ * * +----------+---------+-----------+ * | Header | Types | Strings | @@ -116,6 +121,8 @@ struct btf { /* whether strings are already deduplicated */ bool strs_deduped; + void *meta_data; + /* BTF object FD, if loaded into kernel */ int fd; @@ -215,6 +222,11 @@ static void btf_bswap_hdr(struct btf_header *h) h->type_len = bswap_32(h->type_len); h->str_off = bswap_32(h->str_off); h->str_len = bswap_32(h->str_len); + if (h->hdr_len >= sizeof(struct btf_header)) { + h->meta_header.meta_off = bswap_32(h->meta_header.meta_off); + h->meta_header.meta_len = bswap_32(h->meta_header.meta_len); + } + } static int btf_parse_hdr(struct btf *btf) @@ -222,14 +234,17 @@ static int btf_parse_hdr(struct btf *btf) struct btf_header *hdr = btf->hdr; __u32 meta_left; - if (btf->raw_size < sizeof(struct btf_header)) { + if (btf->raw_size < sizeof(struct btf_header) - sizeof(struct btf_meta_header)) { pr_debug("BTF header not found\n"); return -EINVAL; } if (hdr->magic == bswap_16(BTF_MAGIC)) { + int swapped_len = bswap_32(hdr->hdr_len); + btf->swapped_endian = true; - if (bswap_32(hdr->hdr_len) != sizeof(struct btf_header)) { + if (swapped_len != sizeof(struct btf_header) && + swapped_len != sizeof(struct btf_header) - sizeof(struct btf_meta_header)) { pr_warn("Can't load BTF with non-native endianness due to unsupported header length %u\n", bswap_32(hdr->hdr_len)); return -ENOTSUP; @@ -285,6 +300,42 @@ static int btf_parse_str_sec(struct btf *btf) return 0; } +static void btf_bswap_meta(struct btf_metadata *meta, int len) +{ + struct btf_kind_meta *m = &meta->kind_meta[0]; + struct btf_kind_meta *end = (void *)meta + len; + + meta->flags = bswap_32(meta->flags); + meta->crc = bswap_32(meta->crc); + meta->base_crc = bswap_32(meta->base_crc); + meta->description_off = bswap_32(meta->description_off); + + while (m < end) { + m->name_off = bswap_32(m->name_off); + m->flags = bswap_16(m->flags); + m++; + } +} + +static int btf_parse_meta_sec(struct btf *btf) +{ + const struct btf_header *hdr = btf->hdr; + + if (hdr->hdr_len < sizeof(struct btf_header) || + !hdr->meta_header.meta_off || !hdr->meta_header.meta_len) + return 0; + if (hdr->meta_header.meta_len < sizeof(struct btf_metadata)) { + pr_debug("Invalid BTF metadata section\n"); + return -EINVAL; + } + btf->meta_data = btf->raw_data + btf->hdr->hdr_len + btf->hdr->meta_header.meta_off; + + if (btf->swapped_endian) + btf_bswap_meta(btf->meta_data, hdr->meta_header.meta_len); + + return 0; +} + static int btf_type_size(const struct btf_type *t) { const int base_size = sizeof(struct btf_type); @@ -904,6 +955,7 @@ static struct btf *btf_new(const void *data, __u32 size, struct btf *base_btf) err = err ?: btf_parse_type_sec(btf); if (err) goto done; + err = btf_parse_meta_sec(btf); done: if (err) { @@ -1267,6 +1319,11 @@ static void *btf_get_raw_data(const struct btf *btf, __u32 *size, bool swap_endi } data_sz = hdr->hdr_len + hdr->type_len + hdr->str_len; + if (btf->meta_data) { + data_sz = roundup(data_sz, 8); + data_sz += hdr->meta_header.meta_len; + hdr->meta_header.meta_off = roundup(hdr->type_len + hdr->str_len, 8); + } data = calloc(1, data_sz); if (!data) return NULL; @@ -1293,8 +1350,21 @@ static void *btf_get_raw_data(const struct btf *btf, __u32 *size, bool swap_endi p += hdr->type_len; memcpy(p, btf_strs_data(btf), hdr->str_len); - p += hdr->str_len; + /* round up to 8 byte alignment to match offset above */ + p = data + hdr->hdr_len + roundup(hdr->type_len + hdr->str_len, 8); + + if (btf->meta_data) { + struct btf_metadata *meta = p; + memcpy(p, btf->meta_data, hdr->meta_header.meta_len); + if (!swap_endian) { + meta->crc = crc32(0L, (const Bytef *)&data, sizeof(data)); + meta->flags |= BTF_META_CRC_SET; + } + if (swap_endian) + btf_bswap_meta(p, hdr->meta_header.meta_len); + p += hdr->meta_header.meta_len; + } *size = data_sz; return data; err_out: @@ -1425,13 +1495,13 @@ static void btf_invalidate_raw_data(struct btf *btf) } } -/* Ensure BTF is ready to be modified (by splitting into a three memory - * regions for header, types, and strings). Also invalidate cached - * raw_data, if any. +/* Ensure BTF is ready to be modified (by splitting into a three or four memory + * regions for header, types, strings and optional metadata). Also invalidate + * cached raw_data, if any. */ static int btf_ensure_modifiable(struct btf *btf) { - void *hdr, *types; + void *hdr, *types, *meta = NULL; struct strset *set = NULL; int err = -ENOMEM; @@ -1446,9 +1516,17 @@ static int btf_ensure_modifiable(struct btf *btf) types = malloc(btf->hdr->type_len); if (!hdr || !types) goto err_out; + if (btf->hdr->hdr_len >= sizeof(struct btf_header) && + btf->hdr->meta_header.meta_off && btf->hdr->meta_header.meta_len) { + meta = calloc(1, btf->hdr->meta_header.meta_len); + if (!meta) + goto err_out; + } memcpy(hdr, btf->hdr, btf->hdr->hdr_len); memcpy(types, btf->types_data, btf->hdr->type_len); + if (meta) + memcpy(meta, btf->meta_data, btf->hdr->meta_header.meta_len); /* build lookup index for all strings */ set = strset__new(BTF_MAX_STR_OFFSET, btf->strs_data, btf->hdr->str_len); @@ -1463,6 +1541,8 @@ static int btf_ensure_modifiable(struct btf *btf) btf->types_data_cap = btf->hdr->type_len; btf->strs_data = NULL; btf->strs_set = set; + btf->meta_data = meta; + /* if BTF was created from scratch, all strings are guaranteed to be * unique and deduplicated */ @@ -1480,6 +1560,7 @@ static int btf_ensure_modifiable(struct btf *btf) strset__free(set); free(hdr); free(types); + free(meta); return err; } From patchwork Wed May 31 20:19:30 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alan Maguire X-Patchwork-Id: 13262585 X-Patchwork-Delegate: bpf@iogearbox.net Received: from lindbergh.monkeyblade.net (lindbergh.monkeyblade.net [23.128.96.19]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id A830B34CE2 for ; Wed, 31 May 2023 20:21:50 +0000 (UTC) Received: from mx0b-00069f02.pphosted.com (mx0b-00069f02.pphosted.com [205.220.177.32]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 1F43F12C for ; Wed, 31 May 2023 13:21:49 -0700 (PDT) Received: from pps.filterd (m0246630.ppops.net [127.0.0.1]) by mx0b-00069f02.pphosted.com (8.17.1.19/8.17.1.19) with ESMTP id 34VK8wPU027627; Wed, 31 May 2023 20:20:50 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding; s=corp-2023-03-30; bh=s+8PTZY3Mo6Krv2mU6KA99y+vThw6jAJj9SdT1xBUoA=; b=VGVQPw0INUiIJh2ciXhFt33NHqVBPbC/Wsx0uY2qlTuzeDlmbS0GLvQb1io2SvJBT62v Eco6nhlZySdauMGhkmmBcFQ+NgxiZeucek+GQLBd3dqDSn0D7xpLDrm+WnNrcMAiyEnx GHtjpKaoXgZ2Xrr+MR2//RohJfPoR6CbUMzXsO3RZpqA2vxKQ1V7soCF1865UVN/MFYW gtRxdPFEZnr/tnd7SYv7PS6EEHE4/ZSxI/3ic2n6PtAAIT2HeaAfMI3aVjzKA3CSY0VJ WTRrOVaHSc6ek3KXsuZgm6S2LOcOHbB0p9/anePGcK1vc84oqzJs4mxC/5idBoF0JMvr jA== Received: from phxpaimrmta02.imrmtpd1.prodappphxaev1.oraclevcn.com (phxpaimrmta02.appoci.oracle.com [147.154.114.232]) by mx0b-00069f02.pphosted.com (PPS) with ESMTPS id 3qvhwweuc0-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Wed, 31 May 2023 20:20:50 +0000 Received: from pps.filterd (phxpaimrmta02.imrmtpd1.prodappphxaev1.oraclevcn.com [127.0.0.1]) by phxpaimrmta02.imrmtpd1.prodappphxaev1.oraclevcn.com (8.17.1.19/8.17.1.19) with ESMTP id 34VKHcc1019873; Wed, 31 May 2023 20:20:49 GMT Received: from pps.reinject (localhost [127.0.0.1]) by phxpaimrmta02.imrmtpd1.prodappphxaev1.oraclevcn.com (PPS) with ESMTPS id 3qu8a6djp5-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Wed, 31 May 2023 20:20:49 +0000 Received: from phxpaimrmta02.imrmtpd1.prodappphxaev1.oraclevcn.com (phxpaimrmta02.imrmtpd1.prodappphxaev1.oraclevcn.com [127.0.0.1]) by pps.reinject (8.17.1.5/8.17.1.5) with ESMTP id 34VKKaER000653; Wed, 31 May 2023 20:20:49 GMT Received: from bpf.uk.oracle.com (dhcp-10-175-201-40.vpn.oracle.com [10.175.201.40]) by phxpaimrmta02.imrmtpd1.prodappphxaev1.oraclevcn.com (PPS) with ESMTP id 3qu8a6djab-4; Wed, 31 May 2023 20:20:48 +0000 From: Alan Maguire To: ast@kernel.org, daniel@iogearbox.net, andrii@kernel.org, acme@kernel.org Cc: martin.lau@linux.dev, song@kernel.org, yhs@fb.com, john.fastabend@gmail.com, kpsingh@kernel.org, sdf@google.com, haoluo@google.com, jolsa@kernel.org, quentin@isovalent.com, mykolal@fb.com, bpf@vger.kernel.org, Alan Maguire Subject: [RFC bpf-next 3/8] libbpf: use metadata to compute an unknown kind size Date: Wed, 31 May 2023 21:19:30 +0100 Message-Id: <20230531201936.1992188-4-alan.maguire@oracle.com> X-Mailer: git-send-email 2.39.3 In-Reply-To: <20230531201936.1992188-1-alan.maguire@oracle.com> References: <20230531201936.1992188-1-alan.maguire@oracle.com> Precedence: bulk X-Mailing-List: bpf@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.254,Aquarius:18.0.957,Hydra:6.0.573,FMLib:17.11.176.26 definitions=2023-05-31_14,2023-05-31_03,2023-05-22_02 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 spamscore=0 malwarescore=0 phishscore=0 bulkscore=0 adultscore=0 mlxscore=0 suspectscore=0 mlxlogscore=933 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2304280000 definitions=main-2305310172 X-Proofpoint-ORIG-GUID: pYY-R5k0kc1JIxB0xJnWzP720Xlc6D9t X-Proofpoint-GUID: pYY-R5k0kc1JIxB0xJnWzP720Xlc6D9t X-Spam-Status: No, score=-2.8 required=5.0 tests=BAYES_00,DKIMWL_WL_MED, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_LOW, RCVD_IN_MSPIKE_H5,RCVD_IN_MSPIKE_WL,SPF_HELO_NONE,SPF_NONE, T_SCC_BODY_TEXT_LINE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net X-Patchwork-Delegate: bpf@iogearbox.net X-Patchwork-State: RFC This allows BTF parsing to proceed even if we do not know the kind. Signed-off-by: Alan Maguire --- tools/lib/bpf/btf.c | 50 ++++++++++++++++++++++++++++++++++++--------- 1 file changed, 40 insertions(+), 10 deletions(-) diff --git a/tools/lib/bpf/btf.c b/tools/lib/bpf/btf.c index 036dc1505969..77a072716d58 100644 --- a/tools/lib/bpf/btf.c +++ b/tools/lib/bpf/btf.c @@ -336,7 +336,40 @@ static int btf_parse_meta_sec(struct btf *btf) return 0; } -static int btf_type_size(const struct btf_type *t) +/* for unknown kinds, consult kind metadata. */ +static int btf_type_size_unknown(const struct btf *btf, const struct btf_type *t) +{ + int size = sizeof(struct btf_type); + struct btf_kind_meta *m = NULL; + __u16 vlen = btf_vlen(t); + __u8 kind = btf_kind(t); + + if (btf->meta_data) { + struct btf_metadata *meta = btf->meta_data; + + if (kind < meta->kind_meta_cnt) + m = &meta->kind_meta[kind]; + } + + if (!m || (void *)m > (btf->meta_data + btf->hdr->meta_header.meta_len)) { + pr_debug("Unsupported BTF_KIND: %u\n", btf_kind(t)); + return -EINVAL; + } + + if (!(m->flags & BTF_KIND_META_OPTIONAL)) { + /* a required kind, and we do not know about it.. */ + pr_debug("unknown but required kind: %s(%u)\n", + btf__name_by_offset(btf, m->name_off), kind); + return -EINVAL; + } + + size += m->info_sz; + size += vlen * m->elem_sz; + + return size; +} + +static int btf_type_size(const struct btf *btf, const struct btf_type *t) { const int base_size = sizeof(struct btf_type); __u16 vlen = btf_vlen(t); @@ -372,8 +405,7 @@ static int btf_type_size(const struct btf_type *t) case BTF_KIND_DECL_TAG: return base_size + sizeof(struct btf_decl_tag); default: - pr_debug("Unsupported BTF_KIND:%u\n", btf_kind(t)); - return -EINVAL; + return btf_type_size_unknown(btf, t); } } @@ -472,7 +504,7 @@ static int btf_parse_type_sec(struct btf *btf) if (btf->swapped_endian) btf_bswap_type_base(next_type); - type_size = btf_type_size(next_type); + type_size = btf_type_size(btf, next_type); if (type_size < 0) return type_size; if (next_type + type_size > end_type) { @@ -952,10 +984,8 @@ static struct btf *btf_new(const void *data, __u32 size, struct btf *base_btf) btf->types_data = btf->raw_data + btf->hdr->hdr_len + btf->hdr->type_off; err = btf_parse_str_sec(btf); + err = err ?: btf_parse_meta_sec(btf); err = err ?: btf_parse_type_sec(btf); - if (err) - goto done; - err = btf_parse_meta_sec(btf); done: if (err) { @@ -1687,7 +1717,7 @@ int btf__add_type(struct btf *btf, const struct btf *src_btf, const struct btf_t struct btf_type *t; int sz, err; - sz = btf_type_size(src_type); + sz = btf_type_size(src_btf, src_type); if (sz < 0) return libbpf_err(sz); @@ -1768,7 +1798,7 @@ int btf__add_btf(struct btf *btf, const struct btf *src_btf) memcpy(t, src_btf->types_data, data_sz); for (i = 0; i < cnt; i++) { - sz = btf_type_size(t); + sz = btf_type_size(src_btf, t); if (sz < 0) { /* unlikely, has to be corrupted src_btf */ err = sz; @@ -4764,7 +4794,7 @@ static int btf_dedup_compact_types(struct btf_dedup *d) continue; t = btf__type_by_id(d->btf, id); - len = btf_type_size(t); + len = btf_type_size(d->btf, t); if (len < 0) return len; From patchwork Wed May 31 20:19:31 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alan Maguire X-Patchwork-Id: 13262583 X-Patchwork-Delegate: bpf@iogearbox.net Received: from lindbergh.monkeyblade.net (lindbergh.monkeyblade.net [23.128.96.19]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id A478034CE2 for ; Wed, 31 May 2023 20:21:34 +0000 (UTC) Received: from mx0b-00069f02.pphosted.com (mx0b-00069f02.pphosted.com [205.220.177.32]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 234FFA3 for ; Wed, 31 May 2023 13:21:33 -0700 (PDT) Received: from pps.filterd (m0246630.ppops.net [127.0.0.1]) by mx0b-00069f02.pphosted.com (8.17.1.19/8.17.1.19) with ESMTP id 34VK2Omf027743; Wed, 31 May 2023 20:20:55 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding; s=corp-2023-03-30; bh=btrRVJb/IgDmWROlAX1sswZVqFVvDVNxCF0mqehH0X0=; b=3gUGWncVu4b8QEvSsYvAKLadnm99zjJbOUOgUdWfRG6b2PE26CULcAo8OTq++o1bj+L0 eWJXg40nyGndmSMprt/JGali7GN1EjqVTJsHqAoiSxhihUedPqfIMafHJP2YJEs0gi/i qge0PkY3qVeBgMpSQoCFoNaNkw9sSFQU9j6p9f1x37xg40DBGdEWIHkYUlY56sNYldVx 8Shd+auGf+FXgQtg34uCNuYvrzdWRv3pqRhJ8nhL7RXkhZkO8z3nuwF0HWxfNJtNRqL3 sjTBREe3YlYcm2HkzRuzSQejKqtQLKZ/UUwfKZUe+rl58B1bSpMujm8oK0grYKPB6tyf hQ== Received: from phxpaimrmta02.imrmtpd1.prodappphxaev1.oraclevcn.com (phxpaimrmta02.appoci.oracle.com [147.154.114.232]) by mx0b-00069f02.pphosted.com (PPS) with ESMTPS id 3qvhwweuc8-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Wed, 31 May 2023 20:20:54 +0000 Received: from pps.filterd (phxpaimrmta02.imrmtpd1.prodappphxaev1.oraclevcn.com [127.0.0.1]) by phxpaimrmta02.imrmtpd1.prodappphxaev1.oraclevcn.com (8.17.1.19/8.17.1.19) with ESMTP id 34VIXc7j019823; Wed, 31 May 2023 20:20:53 GMT Received: from pps.reinject (localhost [127.0.0.1]) by phxpaimrmta02.imrmtpd1.prodappphxaev1.oraclevcn.com (PPS) with ESMTPS id 3qu8a6djrx-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Wed, 31 May 2023 20:20:53 +0000 Received: from phxpaimrmta02.imrmtpd1.prodappphxaev1.oraclevcn.com (phxpaimrmta02.imrmtpd1.prodappphxaev1.oraclevcn.com [127.0.0.1]) by pps.reinject (8.17.1.5/8.17.1.5) with ESMTP id 34VKKaET000653; Wed, 31 May 2023 20:20:53 GMT Received: from bpf.uk.oracle.com (dhcp-10-175-201-40.vpn.oracle.com [10.175.201.40]) by phxpaimrmta02.imrmtpd1.prodappphxaev1.oraclevcn.com (PPS) with ESMTP id 3qu8a6djab-5; Wed, 31 May 2023 20:20:52 +0000 From: Alan Maguire To: ast@kernel.org, daniel@iogearbox.net, andrii@kernel.org, acme@kernel.org Cc: martin.lau@linux.dev, song@kernel.org, yhs@fb.com, john.fastabend@gmail.com, kpsingh@kernel.org, sdf@google.com, haoluo@google.com, jolsa@kernel.org, quentin@isovalent.com, mykolal@fb.com, bpf@vger.kernel.org, Alan Maguire Subject: [RFC bpf-next 4/8] btf: support kernel parsing of BTF with metadata, use it to parse BTF with unknown kinds Date: Wed, 31 May 2023 21:19:31 +0100 Message-Id: <20230531201936.1992188-5-alan.maguire@oracle.com> X-Mailer: git-send-email 2.39.3 In-Reply-To: <20230531201936.1992188-1-alan.maguire@oracle.com> References: <20230531201936.1992188-1-alan.maguire@oracle.com> Precedence: bulk X-Mailing-List: bpf@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.254,Aquarius:18.0.957,Hydra:6.0.573,FMLib:17.11.176.26 definitions=2023-05-31_14,2023-05-31_03,2023-05-22_02 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 spamscore=0 malwarescore=0 phishscore=0 bulkscore=0 adultscore=0 mlxscore=0 suspectscore=0 mlxlogscore=999 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2304280000 definitions=main-2305310172 X-Proofpoint-ORIG-GUID: rSvjfsm8XQSn011aIdS-GubazLEhGNma X-Proofpoint-GUID: rSvjfsm8XQSn011aIdS-GubazLEhGNma X-Spam-Status: No, score=-2.8 required=5.0 tests=BAYES_00,DKIMWL_WL_MED, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_LOW, RCVD_IN_MSPIKE_H5,RCVD_IN_MSPIKE_WL,SPF_HELO_NONE,SPF_NONE, T_SCC_BODY_TEXT_LINE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net X-Patchwork-Delegate: bpf@iogearbox.net X-Patchwork-State: RFC Validate metadata if present, and use it to parse BTF with unrecognized kinds. Reject BTF that contains a type of a kind that is not optional. Signed-off-by: Alan Maguire --- kernel/bpf/btf.c | 102 +++++++++++++++++++++++++++++++++++++++-------- 1 file changed, 85 insertions(+), 17 deletions(-) diff --git a/kernel/bpf/btf.c b/kernel/bpf/btf.c index bd2cac057928..67f42d9ce099 100644 --- a/kernel/bpf/btf.c +++ b/kernel/bpf/btf.c @@ -257,6 +257,7 @@ struct btf { struct btf_kfunc_set_tab *kfunc_set_tab; struct btf_id_dtor_kfunc_tab *dtor_kfunc_tab; struct btf_struct_metas *struct_meta_tab; + struct btf_metadata *meta_data; /* split BTF support */ struct btf *base_btf; @@ -4965,22 +4966,41 @@ static s32 btf_check_meta(struct btf_verifier_env *env, return -EINVAL; } - if (BTF_INFO_KIND(t->info) > BTF_KIND_MAX || - BTF_INFO_KIND(t->info) == BTF_KIND_UNKN) { - btf_verifier_log(env, "[%u] Invalid kind:%u", - env->log_type_id, BTF_INFO_KIND(t->info)); - return -EINVAL; - } - if (!btf_name_offset_valid(env->btf, t->name_off)) { btf_verifier_log(env, "[%u] Invalid name_offset:%u", env->log_type_id, t->name_off); return -EINVAL; } - var_meta_size = btf_type_ops(t)->check_meta(env, t, meta_left); - if (var_meta_size < 0) - return var_meta_size; + if (BTF_INFO_KIND(t->info) == BTF_KIND_UNKN) { + btf_verifier_log(env, "[%u] Invalid kind:%u", + env->log_type_id, BTF_INFO_KIND(t->info)); + return -EINVAL; + } + + if (BTF_INFO_KIND(t->info) > BTF_KIND_MAX && env->btf->meta_data && + BTF_INFO_KIND(t->info) < env->btf->meta_data->kind_meta_cnt) { + struct btf_kind_meta *m = &env->btf->meta_data->kind_meta[BTF_INFO_KIND(t->info)]; + + if (!(m->flags & BTF_KIND_META_OPTIONAL)) { + btf_verifier_log(env, "[%u] unknown but required kind '%s'(%u)", + env->log_type_id, + btf_name_by_offset(env->btf, m->name_off), + BTF_INFO_KIND(t->info)); + return -EINVAL; + } + var_meta_size = sizeof(struct btf_type); + var_meta_size += m->info_sz + (btf_type_vlen(t) * m->elem_sz); + } else { + if (BTF_INFO_KIND(t->info) > BTF_KIND_MAX) { + btf_verifier_log(env, "[%u] Invalid kind:%u", + env->log_type_id, BTF_INFO_KIND(t->info)); + return -EINVAL; + } + var_meta_size = btf_type_ops(t)->check_meta(env, t, meta_left); + if (var_meta_size < 0) + return var_meta_size; + } meta_left -= var_meta_size; @@ -5155,7 +5175,8 @@ static int btf_parse_str_sec(struct btf_verifier_env *env) start = btf->nohdr_data + hdr->str_off; end = start + hdr->str_len; - if (end != btf->data + btf->data_size) { + if (hdr->hdr_len < sizeof(struct btf_header) && + end != btf->data + btf->data_size) { btf_verifier_log(env, "String section is not at the end"); return -EINVAL; } @@ -5176,9 +5197,47 @@ static int btf_parse_str_sec(struct btf_verifier_env *env) return 0; } +static int btf_parse_meta_sec(struct btf_verifier_env *env) +{ + const struct btf_header *hdr = &env->btf->hdr; + struct btf *btf = env->btf; + void *start, *end; + + if (hdr->hdr_len < sizeof(struct btf_header) || + hdr->meta_header.meta_len == 0) + return 0; + + /* Meta section must align to 8 bytes */ + if (hdr->meta_header.meta_off & (sizeof(u64) - 1)) { + btf_verifier_log(env, "Unaligned meta_off"); + return -EINVAL; + } + start = btf->nohdr_data + hdr->meta_header.meta_off; + end = start + hdr->meta_header.meta_len; + + if (hdr->meta_header.meta_len < sizeof(struct btf_meta_header)) { + btf_verifier_log(env, "Metadata section is too small"); + return -EINVAL; + } + if (end != btf->data + btf->data_size) { + btf_verifier_log(env, "Metadata section is not at the end"); + return -EINVAL; + } + btf->meta_data = start; + + if (hdr->meta_header.meta_len != sizeof(struct btf_metadata) + + (btf->meta_data->kind_meta_cnt * + sizeof(struct btf_kind_meta))) { + btf_verifier_log(env, "Metadata section size mismatch"); + return -EINVAL; + } + return 0; +} + static const size_t btf_sec_info_offset[] = { offsetof(struct btf_header, type_off), offsetof(struct btf_header, str_off), + offsetof(struct btf_header, meta_header.meta_off), }; static int btf_sec_info_cmp(const void *a, const void *b) @@ -5193,15 +5252,19 @@ static int btf_check_sec_info(struct btf_verifier_env *env, u32 btf_data_size) { struct btf_sec_info secs[ARRAY_SIZE(btf_sec_info_offset)]; - u32 total, expected_total, i; + u32 nr_secs = ARRAY_SIZE(btf_sec_info_offset); + u32 total, expected_total, gap, i; const struct btf_header *hdr; const struct btf *btf; btf = env->btf; hdr = &btf->hdr; + if (hdr->hdr_len < sizeof(struct btf_header)) + nr_secs--; + /* Populate the secs from hdr */ - for (i = 0; i < ARRAY_SIZE(btf_sec_info_offset); i++) + for (i = 0; i < nr_secs; i++) secs[i] = *(struct btf_sec_info *)((void *)hdr + btf_sec_info_offset[i]); @@ -5216,9 +5279,10 @@ static int btf_check_sec_info(struct btf_verifier_env *env, btf_verifier_log(env, "Invalid section offset"); return -EINVAL; } - if (total < secs[i].off) { - /* gap */ - btf_verifier_log(env, "Unsupported section found"); + gap = secs[i].off - total; + if (gap >= 8) { + /* gap larger than alignment gap */ + btf_verifier_log(env, "Unsupported section gap found"); return -EINVAL; } if (total > secs[i].off) { @@ -5230,7 +5294,7 @@ static int btf_check_sec_info(struct btf_verifier_env *env, "Total section length too long"); return -EINVAL; } - total += secs[i].len; + total += secs[i].len + gap; } /* There is data other than hdr and known sections */ @@ -5530,6 +5594,10 @@ static struct btf *btf_parse(const union bpf_attr *attr, bpfptr_t uattr, u32 uat if (err) goto errout; + err = btf_parse_meta_sec(env); + if (err) + goto errout; + err = btf_parse_type_sec(env); if (err) goto errout; From patchwork Wed May 31 20:19:32 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alan Maguire X-Patchwork-Id: 13262584 X-Patchwork-Delegate: bpf@iogearbox.net Received: from lindbergh.monkeyblade.net (lindbergh.monkeyblade.net [23.128.96.19]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 2823C34CE2 for ; Wed, 31 May 2023 20:21:45 +0000 (UTC) Received: from mx0b-00069f02.pphosted.com (mx0b-00069f02.pphosted.com [205.220.177.32]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id A111510F for ; Wed, 31 May 2023 13:21:43 -0700 (PDT) Received: from pps.filterd (m0333520.ppops.net [127.0.0.1]) by mx0b-00069f02.pphosted.com (8.17.1.19/8.17.1.19) with ESMTP id 34VERTaa023246; Wed, 31 May 2023 20:20:59 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding; s=corp-2023-03-30; bh=ft+QiSZvAuBxPiNnIUGCCTaNMByaHNoEjKUEDAw6ofo=; b=yENQwx+LVc+M6LIkxOll0jPnnYOzleFFpELonlyp7yS8+zsf7vzAiPbxqyk0PBYoxkJ5 hQfR/T+4siRRL3F0Jlybt37hjpzrYWA5S+vaQzjPkhw57hdv+a5XJNG7U/9BAg+w904b IGscxjHzdXubQEXE0qQLZSbavXE//maqXGwbWDo1/1rBbXmW5yCA0l0/FkHw+6gePYYB qf50ZJb2JzPU0th+VjyM6k3KQ3LOhhE2Bn4TAUANUTOvaGAtMhYDnV6TupBv3yJ1+8Kz jDBlM0sCKyEN7D7bzF99+aq1cZGUl3uY+2H07F1cW55mSXhtdZf/cJFdY8SLChm4xVuT 9Q== Received: from phxpaimrmta02.imrmtpd1.prodappphxaev1.oraclevcn.com (phxpaimrmta02.appoci.oracle.com [147.154.114.232]) by mx0b-00069f02.pphosted.com (PPS) with ESMTPS id 3qvhd9y2dg-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Wed, 31 May 2023 20:20:59 +0000 Received: from pps.filterd (phxpaimrmta02.imrmtpd1.prodappphxaev1.oraclevcn.com [127.0.0.1]) by phxpaimrmta02.imrmtpd1.prodappphxaev1.oraclevcn.com (8.17.1.19/8.17.1.19) with ESMTP id 34VJqsmu019729; Wed, 31 May 2023 20:20:58 GMT Received: from pps.reinject (localhost [127.0.0.1]) by phxpaimrmta02.imrmtpd1.prodappphxaev1.oraclevcn.com (PPS) with ESMTPS id 3qu8a6djuk-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Wed, 31 May 2023 20:20:58 +0000 Received: from phxpaimrmta02.imrmtpd1.prodappphxaev1.oraclevcn.com (phxpaimrmta02.imrmtpd1.prodappphxaev1.oraclevcn.com [127.0.0.1]) by pps.reinject (8.17.1.5/8.17.1.5) with ESMTP id 34VKKaEV000653; Wed, 31 May 2023 20:20:57 GMT Received: from bpf.uk.oracle.com (dhcp-10-175-201-40.vpn.oracle.com [10.175.201.40]) by phxpaimrmta02.imrmtpd1.prodappphxaev1.oraclevcn.com (PPS) with ESMTP id 3qu8a6djab-6; Wed, 31 May 2023 20:20:57 +0000 From: Alan Maguire To: ast@kernel.org, daniel@iogearbox.net, andrii@kernel.org, acme@kernel.org Cc: martin.lau@linux.dev, song@kernel.org, yhs@fb.com, john.fastabend@gmail.com, kpsingh@kernel.org, sdf@google.com, haoluo@google.com, jolsa@kernel.org, quentin@isovalent.com, mykolal@fb.com, bpf@vger.kernel.org, Alan Maguire Subject: [RFC bpf-next 5/8] libbpf: add metadata encoding support Date: Wed, 31 May 2023 21:19:32 +0100 Message-Id: <20230531201936.1992188-6-alan.maguire@oracle.com> X-Mailer: git-send-email 2.39.3 In-Reply-To: <20230531201936.1992188-1-alan.maguire@oracle.com> References: <20230531201936.1992188-1-alan.maguire@oracle.com> Precedence: bulk X-Mailing-List: bpf@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.254,Aquarius:18.0.957,Hydra:6.0.573,FMLib:17.11.176.26 definitions=2023-05-31_14,2023-05-31_03,2023-05-22_02 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 spamscore=0 malwarescore=0 phishscore=0 bulkscore=0 adultscore=0 mlxscore=0 suspectscore=0 mlxlogscore=999 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2304280000 definitions=main-2305310172 X-Proofpoint-GUID: -UXU5sjrBRxhtlonOxbofZRJriJYZslX X-Proofpoint-ORIG-GUID: -UXU5sjrBRxhtlonOxbofZRJriJYZslX X-Spam-Status: No, score=-2.8 required=5.0 tests=BAYES_00,DKIMWL_WL_MED, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_LOW, RCVD_IN_MSPIKE_H5,RCVD_IN_MSPIKE_WL,SPF_HELO_NONE,SPF_NONE, T_SCC_BODY_TEXT_LINE,URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net X-Patchwork-Delegate: bpf@iogearbox.net X-Patchwork-State: RFC Support encoding of BTF metadata via btf__new_empty_opts(). Current supported opts are base_btf, add_meta and description for metadata. Signed-off-by: Alan Maguire --- tools/lib/bpf/btf.c | 108 +++++++++++++++++++++++++++++++++++++-- tools/lib/bpf/btf.h | 11 ++++ tools/lib/bpf/libbpf.map | 1 + 3 files changed, 117 insertions(+), 3 deletions(-) diff --git a/tools/lib/bpf/btf.c b/tools/lib/bpf/btf.c index 77a072716d58..4b85325336b4 100644 --- a/tools/lib/bpf/btf.c +++ b/tools/lib/bpf/btf.c @@ -122,6 +122,7 @@ struct btf { bool strs_deduped; void *meta_data; + size_t meta_sz; /* BTF object FD, if loaded into kernel */ int fd; @@ -896,8 +897,84 @@ void btf__free(struct btf *btf) free(btf); } -static struct btf *btf_new_empty(struct btf *base_btf) +static void __btf_add_kind_meta(struct btf *btf, __u8 kind, const char *name, + __u16 flags, __u8 info_sz, __u8 elem_sz) { + struct btf_metadata *meta = btf->meta_data; + struct btf_kind_meta *kind_meta = &meta->kind_meta[kind]; + + kind_meta->name_off = btf__add_str(btf, name); + kind_meta->flags = flags; + kind_meta->info_sz = info_sz; + kind_meta->elem_sz = elem_sz; +} + +#define btf_add_kind_meta(btf, kind, flags, info_sz, elem_sz) \ + __btf_add_kind_meta(btf, kind, #kind, flags, info_sz, elem_sz) + +static int btf_ensure_modifiable(struct btf *btf); + +static int btf_add_meta(struct btf *btf, struct btf_new_opts *opts) +{ + const char *description = OPTS_GET(opts, description, NULL); + struct btf_metadata *meta; + + if (btf_ensure_modifiable(btf)) + return libbpf_err(-ENOMEM); + + btf->meta_sz = sizeof(struct btf_metadata) + + (NR_BTF_KINDS * sizeof(struct btf_kind_meta)); + btf->meta_data = calloc(1, btf->meta_sz); + + if (!btf->meta_data) { + btf->meta_sz = 0; + return -ENOMEM; + } + + meta = btf->meta_data; + + if (btf->base_btf) { + struct btf_metadata *base_meta = btf->base_btf->meta_data; + + if (base_meta && (base_meta->flags & BTF_META_CRC_SET)) { + meta->base_crc = base_meta->crc; + meta->flags |= BTF_META_BASE_CRC_SET; + } + } + + if (description) + meta->description_off = btf__add_str(btf, description); + + meta->kind_meta_cnt = NR_BTF_KINDS; + + /* all supported kinds should describe their format here. */ + btf_add_kind_meta(btf, BTF_KIND_UNKN, 0, 0, 0); + btf_add_kind_meta(btf, BTF_KIND_INT, 0, sizeof(__u32), 0); + btf_add_kind_meta(btf, BTF_KIND_PTR, 0, 0, 0); + btf_add_kind_meta(btf, BTF_KIND_ARRAY, 0, sizeof(struct btf_array), 0); + btf_add_kind_meta(btf, BTF_KIND_STRUCT, 0, 0, sizeof(struct btf_member)); + btf_add_kind_meta(btf, BTF_KIND_UNION, 0, 0, sizeof(struct btf_member)); + btf_add_kind_meta(btf, BTF_KIND_ENUM, 0, 0, sizeof(struct btf_enum)); + btf_add_kind_meta(btf, BTF_KIND_FWD, 0, 0, 0); + btf_add_kind_meta(btf, BTF_KIND_TYPEDEF, 0, 0, 0); + btf_add_kind_meta(btf, BTF_KIND_VOLATILE, 0, 0, 0); + btf_add_kind_meta(btf, BTF_KIND_CONST, 0, 0, 0); + btf_add_kind_meta(btf, BTF_KIND_RESTRICT, 0, 0, 0); + btf_add_kind_meta(btf, BTF_KIND_FUNC, 0, 0, 0); + btf_add_kind_meta(btf, BTF_KIND_FUNC_PROTO, 0, 0, sizeof(struct btf_param)); + btf_add_kind_meta(btf, BTF_KIND_VAR, 0, sizeof(struct btf_var), 0); + btf_add_kind_meta(btf, BTF_KIND_DATASEC, 0, 0, sizeof(struct btf_var_secinfo)); + btf_add_kind_meta(btf, BTF_KIND_FLOAT, 0, 0, 0); + btf_add_kind_meta(btf, BTF_KIND_DECL_TAG, BTF_KIND_META_OPTIONAL, + sizeof(struct btf_decl_tag), 0); + btf_add_kind_meta(btf, BTF_KIND_TYPE_TAG, BTF_KIND_META_OPTIONAL, 0, 0); + btf_add_kind_meta(btf, BTF_KIND_ENUM64, 0, 0, sizeof(struct btf_enum64)); + return 0; +} + +static struct btf *btf_new_empty(struct btf_new_opts *opts) +{ + struct btf *base_btf = OPTS_GET(opts, base_btf, NULL); struct btf *btf; btf = calloc(1, sizeof(*btf)); @@ -934,17 +1011,42 @@ static struct btf *btf_new_empty(struct btf *base_btf) btf->strs_data = btf->raw_data + btf->hdr->hdr_len; btf->hdr->str_len = base_btf ? 0 : 1; /* empty string at offset 0 */ + if (opts->add_meta) { + int err = btf_add_meta(btf, opts); + + if (err) { + free(btf->raw_data); + free(btf); + return ERR_PTR(err); + } + btf->hdr->meta_header.meta_len = btf->meta_sz; + } + return btf; } struct btf *btf__new_empty(void) { - return libbpf_ptr(btf_new_empty(NULL)); + LIBBPF_OPTS(btf_new_opts, opts); + + return libbpf_ptr(btf_new_empty(&opts)); } struct btf *btf__new_empty_split(struct btf *base_btf) { - return libbpf_ptr(btf_new_empty(base_btf)); + LIBBPF_OPTS(btf_new_opts, opts); + + opts.base_btf = base_btf; + + return libbpf_ptr(btf_new_empty(&opts)); +} + +struct btf *btf__new_empty_opts(struct btf_new_opts *opts) +{ + if (!OPTS_VALID(opts, btf_new_opts)) + return libbpf_err_ptr(-EINVAL); + + return libbpf_ptr(btf_new_empty(opts)); } static struct btf *btf_new(const void *data, __u32 size, struct btf *base_btf) diff --git a/tools/lib/bpf/btf.h b/tools/lib/bpf/btf.h index 8e6880d91c84..6de445225f02 100644 --- a/tools/lib/bpf/btf.h +++ b/tools/lib/bpf/btf.h @@ -107,6 +107,17 @@ LIBBPF_API struct btf *btf__new_empty(void); */ LIBBPF_API struct btf *btf__new_empty_split(struct btf *base_btf); +struct btf_new_opts { + size_t sz; + struct btf *base_btf; /* optional base BTF */ + bool add_meta; /* add BTF metadata about encoding */ + const char *description;/* add description to metadata */ + size_t :0; +}; +#define btf_new_opts__last_field description + +LIBBPF_API struct btf *btf__new_empty_opts(struct btf_new_opts *opts); + LIBBPF_API struct btf *btf__parse(const char *path, struct btf_ext **btf_ext); LIBBPF_API struct btf *btf__parse_split(const char *path, struct btf *base_btf); LIBBPF_API struct btf *btf__parse_elf(const char *path, struct btf_ext **btf_ext); diff --git a/tools/lib/bpf/libbpf.map b/tools/lib/bpf/libbpf.map index 7521a2fb7626..edd8be4b21d0 100644 --- a/tools/lib/bpf/libbpf.map +++ b/tools/lib/bpf/libbpf.map @@ -395,4 +395,5 @@ LIBBPF_1.2.0 { LIBBPF_1.3.0 { global: bpf_obj_pin_opts; + btf__new_empty_opts; } LIBBPF_1.2.0; From patchwork Wed May 31 20:19:33 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alan Maguire X-Patchwork-Id: 13262586 X-Patchwork-Delegate: bpf@iogearbox.net Received: from lindbergh.monkeyblade.net (lindbergh.monkeyblade.net [23.128.96.19]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id B19983C0B3 for ; Wed, 31 May 2023 20:21:50 +0000 (UTC) Received: from mx0b-00069f02.pphosted.com (mx0b-00069f02.pphosted.com [205.220.177.32]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 33D8912F for ; Wed, 31 May 2023 13:21:49 -0700 (PDT) Received: from pps.filterd (m0246630.ppops.net [127.0.0.1]) by mx0b-00069f02.pphosted.com (8.17.1.19/8.17.1.19) with ESMTP id 34VK8wPX027627; Wed, 31 May 2023 20:21:03 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding; s=corp-2023-03-30; bh=ChEoXaLy2bpe6uJDlcCe7o+5RqE+y02pBdTn7k/bXSU=; b=rZ2+ztJDlvPHTtrLSws6PCqb35VxEs/JkKh4tRaS8ysDQulf8aolYctCUttDMb7M0tY7 IlOy9uDZitUOgNbZyJ2DDuMoDWniCM1rwG35NwCSb/XwBhQh0uiPtURACe30V9VK7/Uv NfYKdrPRDIoScCNN1448rSK8XXge0dPJ1Xiac/A/ESr0VnAPxsVfobXHUz3W92l0oX/7 KMuKigstXwExnPWdJKwd7GXrPmzlPDvaVGKL8ddtRHELpcW5yEkfbmMgiB94SUIB5KxH iNBpCheSQ+UfCFiF0BLiESUA9CqiUYGiAgEVzaNPFHv6qwXubT9oaXGyOT6YySasKOEg uw== Received: from phxpaimrmta02.imrmtpd1.prodappphxaev1.oraclevcn.com (phxpaimrmta02.appoci.oracle.com [147.154.114.232]) by mx0b-00069f02.pphosted.com (PPS) with ESMTPS id 3qvhwweuct-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Wed, 31 May 2023 20:21:03 +0000 Received: from pps.filterd (phxpaimrmta02.imrmtpd1.prodappphxaev1.oraclevcn.com [127.0.0.1]) by phxpaimrmta02.imrmtpd1.prodappphxaev1.oraclevcn.com (8.17.1.19/8.17.1.19) with ESMTP id 34VKEfB3019715; Wed, 31 May 2023 20:21:02 GMT Received: from pps.reinject (localhost [127.0.0.1]) by phxpaimrmta02.imrmtpd1.prodappphxaev1.oraclevcn.com (PPS) with ESMTPS id 3qu8a6djxm-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Wed, 31 May 2023 20:21:02 +0000 Received: from phxpaimrmta02.imrmtpd1.prodappphxaev1.oraclevcn.com (phxpaimrmta02.imrmtpd1.prodappphxaev1.oraclevcn.com [127.0.0.1]) by pps.reinject (8.17.1.5/8.17.1.5) with ESMTP id 34VKKaEX000653; Wed, 31 May 2023 20:21:01 GMT Received: from bpf.uk.oracle.com (dhcp-10-175-201-40.vpn.oracle.com [10.175.201.40]) by phxpaimrmta02.imrmtpd1.prodappphxaev1.oraclevcn.com (PPS) with ESMTP id 3qu8a6djab-7; Wed, 31 May 2023 20:21:01 +0000 From: Alan Maguire To: ast@kernel.org, daniel@iogearbox.net, andrii@kernel.org, acme@kernel.org Cc: martin.lau@linux.dev, song@kernel.org, yhs@fb.com, john.fastabend@gmail.com, kpsingh@kernel.org, sdf@google.com, haoluo@google.com, jolsa@kernel.org, quentin@isovalent.com, mykolal@fb.com, bpf@vger.kernel.org, Alan Maguire Subject: [RFC bpf-next 6/8] btf: generate metadata for vmlinux/module BTF Date: Wed, 31 May 2023 21:19:33 +0100 Message-Id: <20230531201936.1992188-7-alan.maguire@oracle.com> X-Mailer: git-send-email 2.39.3 In-Reply-To: <20230531201936.1992188-1-alan.maguire@oracle.com> References: <20230531201936.1992188-1-alan.maguire@oracle.com> Precedence: bulk X-Mailing-List: bpf@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.254,Aquarius:18.0.957,Hydra:6.0.573,FMLib:17.11.176.26 definitions=2023-05-31_14,2023-05-31_03,2023-05-22_02 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 spamscore=0 malwarescore=0 phishscore=0 bulkscore=0 adultscore=0 mlxscore=0 suspectscore=0 mlxlogscore=999 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2304280000 definitions=main-2305310172 X-Proofpoint-ORIG-GUID: ShR4g_SQpXxrBDXASGKo4ou6_yTR-j8L X-Proofpoint-GUID: ShR4g_SQpXxrBDXASGKo4ou6_yTR-j8L X-Spam-Status: No, score=-2.8 required=5.0 tests=BAYES_00,DKIMWL_WL_MED, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_LOW, RCVD_IN_MSPIKE_H5,RCVD_IN_MSPIKE_WL,SPF_HELO_NONE,SPF_NONE, T_SCC_BODY_TEXT_LINE,URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net X-Patchwork-Delegate: bpf@iogearbox.net X-Patchwork-State: RFC Generate BTF metadata for kernel and module BTF. Signed-off-by: Alan Maguire --- scripts/pahole-flags.sh | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/scripts/pahole-flags.sh b/scripts/pahole-flags.sh index 728d55190d97..28fa9120a770 100755 --- a/scripts/pahole-flags.sh +++ b/scripts/pahole-flags.sh @@ -24,7 +24,7 @@ if [ "${pahole_ver}" -ge "124" ]; then extra_paholeopt="${extra_paholeopt} --lang_exclude=rust" fi if [ "${pahole_ver}" -ge "125" ]; then - extra_paholeopt="${extra_paholeopt} --skip_encoding_btf_inconsistent_proto --btf_gen_optimized" + extra_paholeopt="${extra_paholeopt} --skip_encoding_btf_inconsistent_proto --btf_gen_optimized --btf_gen_meta" fi echo ${extra_paholeopt} From patchwork Wed May 31 20:19:34 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alan Maguire X-Patchwork-Id: 13262588 X-Patchwork-Delegate: bpf@iogearbox.net Received: from lindbergh.monkeyblade.net (lindbergh.monkeyblade.net [23.128.96.19]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 2480D34CE2 for ; Wed, 31 May 2023 20:21:57 +0000 (UTC) Received: from mx0b-00069f02.pphosted.com (mx0b-00069f02.pphosted.com [205.220.177.32]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id E493D134 for ; Wed, 31 May 2023 13:21:52 -0700 (PDT) Received: from pps.filterd (m0246630.ppops.net [127.0.0.1]) by mx0b-00069f02.pphosted.com (8.17.1.19/8.17.1.19) with ESMTP id 34VK2Omj027743; Wed, 31 May 2023 20:21:08 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding; s=corp-2023-03-30; bh=Ol19eRQkoVVrpX2aGsZ286OEGEfhsIKyIEy7NBNLfw4=; b=zrkl/DY/EZ1xi7gYyp69IaISbSt+53ET8hfvUopt39mo5aRhOMBXsDmCsXe7IuZ5d5xz 8cSWHFL7miOLAUtHpHYSwoRQ+phyPQLDnNGK/v2eAzRQ64zgSh4mI1j7DdA52RPQhwn8 +v+nQlyr+ooCpz9fsUAGr91p/07DG/biokW1hejTyiTORHMjYO42UIHVBIqDWvykp2pK tSWCFTWa/j/kZ3Ch78YTnxzmcwJpKu8Fl+SuSLys1gPgUTw08B1IP6Tl8BxyUCcHcjs4 +vBv9E93cUP/4HXJooowSa2NgofepR47AqnzogEkTXmiKxsmZ1myrnTxZvC7cc+RdPSu Sw== Received: from phxpaimrmta02.imrmtpd1.prodappphxaev1.oraclevcn.com (phxpaimrmta02.appoci.oracle.com [147.154.114.232]) by mx0b-00069f02.pphosted.com (PPS) with ESMTPS id 3qvhwweucy-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Wed, 31 May 2023 20:21:07 +0000 Received: from pps.filterd (phxpaimrmta02.imrmtpd1.prodappphxaev1.oraclevcn.com [127.0.0.1]) by phxpaimrmta02.imrmtpd1.prodappphxaev1.oraclevcn.com (8.17.1.19/8.17.1.19) with ESMTP id 34VKHccB019873; Wed, 31 May 2023 20:21:06 GMT Received: from pps.reinject (localhost [127.0.0.1]) by phxpaimrmta02.imrmtpd1.prodappphxaev1.oraclevcn.com (PPS) with ESMTPS id 3qu8a6dk17-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Wed, 31 May 2023 20:21:06 +0000 Received: from phxpaimrmta02.imrmtpd1.prodappphxaev1.oraclevcn.com (phxpaimrmta02.imrmtpd1.prodappphxaev1.oraclevcn.com [127.0.0.1]) by pps.reinject (8.17.1.5/8.17.1.5) with ESMTP id 34VKKaEZ000653; Wed, 31 May 2023 20:21:06 GMT Received: from bpf.uk.oracle.com (dhcp-10-175-201-40.vpn.oracle.com [10.175.201.40]) by phxpaimrmta02.imrmtpd1.prodappphxaev1.oraclevcn.com (PPS) with ESMTP id 3qu8a6djab-8; Wed, 31 May 2023 20:21:05 +0000 From: Alan Maguire To: ast@kernel.org, daniel@iogearbox.net, andrii@kernel.org, acme@kernel.org Cc: martin.lau@linux.dev, song@kernel.org, yhs@fb.com, john.fastabend@gmail.com, kpsingh@kernel.org, sdf@google.com, haoluo@google.com, jolsa@kernel.org, quentin@isovalent.com, mykolal@fb.com, bpf@vger.kernel.org, Alan Maguire Subject: [RFC bpf-next 7/8] bpftool: add BTF dump "format meta" to dump header/metadata Date: Wed, 31 May 2023 21:19:34 +0100 Message-Id: <20230531201936.1992188-8-alan.maguire@oracle.com> X-Mailer: git-send-email 2.39.3 In-Reply-To: <20230531201936.1992188-1-alan.maguire@oracle.com> References: <20230531201936.1992188-1-alan.maguire@oracle.com> Precedence: bulk X-Mailing-List: bpf@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.254,Aquarius:18.0.957,Hydra:6.0.573,FMLib:17.11.176.26 definitions=2023-05-31_14,2023-05-31_03,2023-05-22_02 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 spamscore=0 malwarescore=0 phishscore=0 bulkscore=0 adultscore=0 mlxscore=0 suspectscore=0 mlxlogscore=999 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2304280000 definitions=main-2305310172 X-Proofpoint-ORIG-GUID: NNUG5CoxLXm4xruWdAmFYHv9_8a44jVQ X-Proofpoint-GUID: NNUG5CoxLXm4xruWdAmFYHv9_8a44jVQ X-Spam-Status: No, score=-2.8 required=5.0 tests=BAYES_00,DKIMWL_WL_MED, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_LOW, RCVD_IN_MSPIKE_H5,RCVD_IN_MSPIKE_WL,SPF_HELO_NONE,SPF_NONE, T_SCC_BODY_TEXT_LINE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net X-Patchwork-Delegate: bpf@iogearbox.net X-Patchwork-State: RFC Provide a way to dump BTF header and metadata info via bpftool; for example $ bpftool btf dump file vmliux format meta BTF: data size 4963656 Header: magic 0xeb9f, version 1, flags 0x0, hdr_len 32 Types: len 2927556, offset 0 Strings: len 2035881, offset 2927556 Metadata header found: len 184, offset 4963440, flags 0x1 Description: 'generated by dwarves v1.25' CRC 0x6da2a930 ; base CRC 0x0 Kind metadata for 20 kinds: BTF_KIND_UNKN[ 0] flags 0x0 info_sz 0 elem_sz 0 BTF_KIND_INT[ 1] flags 0x0 info_sz 4 elem_sz 0 BTF_KIND_PTR[ 2] flags 0x0 info_sz 0 elem_sz 0 BTF_KIND_ARRAY[ 3] flags 0x0 info_sz 12 elem_sz 0 BTF_KIND_STRUCT[ 4] flags 0x0 info_sz 0 elem_sz 12 BTF_KIND_UNION[ 5] flags 0x0 info_sz 0 elem_sz 12 BTF_KIND_ENUM[ 6] flags 0x0 info_sz 0 elem_sz 8 BTF_KIND_FWD[ 7] flags 0x0 info_sz 0 elem_sz 0 BTF_KIND_TYPEDEF[ 8] flags 0x0 info_sz 0 elem_sz 0 BTF_KIND_VOLATILE[ 9] flags 0x0 info_sz 0 elem_sz 0 BTF_KIND_CONST[10] flags 0x0 info_sz 0 elem_sz 0 BTF_KIND_RESTRICT[11] flags 0x0 info_sz 0 elem_sz 0 BTF_KIND_FUNC[12] flags 0x0 info_sz 0 elem_sz 0 BTF_KIND_FUNC_PROTO[13] flags 0x0 info_sz 0 elem_sz 8 BTF_KIND_VAR[14] flags 0x0 info_sz 4 elem_sz 0 BTF_KIND_DATASEC[15] flags 0x0 info_sz 0 elem_sz 12 BTF_KIND_FLOAT[16] flags 0x0 info_sz 0 elem_sz 0 BTF_KIND_DECL_TAG[17] flags 0x1 info_sz 4 elem_sz 0 BTF_KIND_TYPE_TAG[18] flags 0x1 info_sz 0 elem_sz 0 BTF_KIND_ENUM64[19] flags 0x0 info_sz 0 elem_sz 12 Signed-off-by: Alan Maguire --- tools/bpf/bpftool/btf.c | 46 +++++++++++++++++++++++++++++++++++++++++ 1 file changed, 46 insertions(+) diff --git a/tools/bpf/bpftool/btf.c b/tools/bpf/bpftool/btf.c index 91fcb75babe3..da4257e00ba8 100644 --- a/tools/bpf/bpftool/btf.c +++ b/tools/bpf/bpftool/btf.c @@ -504,6 +504,47 @@ static int dump_btf_c(const struct btf *btf, return err; } +static int dump_btf_meta(const struct btf *btf) +{ + const struct btf_header *hdr; + const struct btf_metadata *m; + const void *data; + __u32 data_sz; + __u8 i; + + data = btf__raw_data(btf, &data_sz); + if (!data) + return -ENOMEM; + hdr = data; + printf("BTF: data size %u\n", data_sz); + printf("Header: magic 0x%x, version %d, flags 0x%x, hdr_len %u\n", + hdr->magic, hdr->version, hdr->flags, hdr->hdr_len); + printf("Types: len %u, offset %u\n", hdr->type_len, hdr->type_off); + printf("Strings: len %u, offset %u\n", hdr->str_len, hdr->str_off); + + if (hdr->hdr_len < sizeof(struct btf_header) || + hdr->meta_header.meta_len == 0 || + hdr->meta_header.meta_off == 0) + return 0; + + m = (void *)hdr + hdr->hdr_len + hdr->meta_header.meta_off; + + printf("Metadata header found: len %u, offset %u, flags 0x%x\n", + hdr->meta_header.meta_len, hdr->meta_header.meta_off, m->flags); + if (m->description_off) + printf("Description: '%s'\n", btf__name_by_offset(btf, m->description_off)); + printf("CRC 0x%x ; base CRC 0x%x\n", m->crc, m->base_crc); + printf("Kind metadata for %d kinds:\n", m->kind_meta_cnt); + for (i = 0; i < m->kind_meta_cnt; i++) { + printf("%20s[%2d] flags 0x%-4x info_sz %2d elem_sz %2d\n", + btf__name_by_offset(btf, m->kind_meta[i].name_off), + i, m->kind_meta[i].flags, m->kind_meta[i].info_sz, + m->kind_meta[i].elem_sz); + } + + return 0; +} + static const char sysfs_vmlinux[] = "/sys/kernel/btf/vmlinux"; static struct btf *get_vmlinux_btf_from_sysfs(void) @@ -553,6 +594,7 @@ static int do_dump(int argc, char **argv) __u32 root_type_ids[2]; int root_type_cnt = 0; bool dump_c = false; + bool dump_meta = false; __u32 btf_id = -1; const char *src; int fd = -1; @@ -654,6 +696,8 @@ static int do_dump(int argc, char **argv) } if (strcmp(*argv, "c") == 0) { dump_c = true; + } else if (strcmp(*argv, "meta") == 0) { + dump_meta = true; } else if (strcmp(*argv, "raw") == 0) { dump_c = false; } else { @@ -692,6 +736,8 @@ static int do_dump(int argc, char **argv) goto done; } err = dump_btf_c(btf, root_type_ids, root_type_cnt); + } else if (dump_meta) { + err = dump_btf_meta(btf); } else { err = dump_btf_raw(btf, root_type_ids, root_type_cnt); } From patchwork Wed May 31 20:19:35 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alan Maguire X-Patchwork-Id: 13262589 X-Patchwork-Delegate: bpf@iogearbox.net Received: from lindbergh.monkeyblade.net (lindbergh.monkeyblade.net [23.128.96.19]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 2AC0034CE2 for ; Wed, 31 May 2023 20:22:03 +0000 (UTC) Received: from mx0b-00069f02.pphosted.com (mx0b-00069f02.pphosted.com [205.220.177.32]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 82FF412C for ; Wed, 31 May 2023 13:21:59 -0700 (PDT) Received: from pps.filterd (m0246632.ppops.net [127.0.0.1]) by mx0b-00069f02.pphosted.com (8.17.1.19/8.17.1.19) with ESMTP id 34VKD3IW016261; Wed, 31 May 2023 20:21:11 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding; s=corp-2023-03-30; bh=hwXmierehEslNjthUkN9zGuafnuA9WBpaZA6VilAp00=; b=kLs23nhquP8STz8GrCAzESgTopPYV246XPlXfcx87L42GiunEFJBEhRo2W2Xg9n6m5xz C+oDiCRyqezvarZd4/cUz/6FhgUkcw71hiQmlCnFzFsFvXIsxTudb2nlc6eTwiP/P6PR c8B600hTxnGcBPRA8D0LRIoRdcjnfvCzVNESCKVYMV4TC232CMP0EthkWBVj0C4nfOXS Q0MMGraM8svNgzMazOHise9RTCM6Dm49PLIz+JFphkGPG8x8K8Y1YSlumk/RWGLE9ZzC sbmI2lSa7WTwnU788Rdl6FTH0LmPu6Qh/P7J1emiN4BvAYGgZbmfe+anj5hxWXMdPwXy kg== Received: from phxpaimrmta02.imrmtpd1.prodappphxaev1.oraclevcn.com (phxpaimrmta02.appoci.oracle.com [147.154.114.232]) by mx0b-00069f02.pphosted.com (PPS) with ESMTPS id 3qvhmepwy6-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Wed, 31 May 2023 20:21:11 +0000 Received: from pps.filterd (phxpaimrmta02.imrmtpd1.prodappphxaev1.oraclevcn.com [127.0.0.1]) by phxpaimrmta02.imrmtpd1.prodappphxaev1.oraclevcn.com (8.17.1.19/8.17.1.19) with ESMTP id 34VJPuoM019721; Wed, 31 May 2023 20:21:10 GMT Received: from pps.reinject (localhost [127.0.0.1]) by phxpaimrmta02.imrmtpd1.prodappphxaev1.oraclevcn.com (PPS) with ESMTPS id 3qu8a6dk3v-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Wed, 31 May 2023 20:21:10 +0000 Received: from phxpaimrmta02.imrmtpd1.prodappphxaev1.oraclevcn.com (phxpaimrmta02.imrmtpd1.prodappphxaev1.oraclevcn.com [127.0.0.1]) by pps.reinject (8.17.1.5/8.17.1.5) with ESMTP id 34VKKaEb000653; Wed, 31 May 2023 20:21:10 GMT Received: from bpf.uk.oracle.com (dhcp-10-175-201-40.vpn.oracle.com [10.175.201.40]) by phxpaimrmta02.imrmtpd1.prodappphxaev1.oraclevcn.com (PPS) with ESMTP id 3qu8a6djab-9; Wed, 31 May 2023 20:21:09 +0000 From: Alan Maguire To: ast@kernel.org, daniel@iogearbox.net, andrii@kernel.org, acme@kernel.org Cc: martin.lau@linux.dev, song@kernel.org, yhs@fb.com, john.fastabend@gmail.com, kpsingh@kernel.org, sdf@google.com, haoluo@google.com, jolsa@kernel.org, quentin@isovalent.com, mykolal@fb.com, bpf@vger.kernel.org, Alan Maguire Subject: [RFC bpf-next 8/8] selftests/bpf: test kind encoding/decoding Date: Wed, 31 May 2023 21:19:35 +0100 Message-Id: <20230531201936.1992188-9-alan.maguire@oracle.com> X-Mailer: git-send-email 2.39.3 In-Reply-To: <20230531201936.1992188-1-alan.maguire@oracle.com> References: <20230531201936.1992188-1-alan.maguire@oracle.com> Precedence: bulk X-Mailing-List: bpf@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.254,Aquarius:18.0.957,Hydra:6.0.573,FMLib:17.11.176.26 definitions=2023-05-31_14,2023-05-31_03,2023-05-22_02 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 spamscore=0 malwarescore=0 phishscore=0 bulkscore=0 adultscore=0 mlxscore=0 suspectscore=0 mlxlogscore=999 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2304280000 definitions=main-2305310172 X-Proofpoint-ORIG-GUID: kjB7IP50Drzt7RP2rbJvYgxyYuYK-3Nn X-Proofpoint-GUID: kjB7IP50Drzt7RP2rbJvYgxyYuYK-3Nn X-Spam-Status: No, score=-2.8 required=5.0 tests=BAYES_00,DKIMWL_WL_MED, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_LOW, RCVD_IN_MSPIKE_H5,RCVD_IN_MSPIKE_WL,SPF_HELO_NONE,SPF_NONE, T_SCC_BODY_TEXT_LINE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net X-Patchwork-Delegate: bpf@iogearbox.net X-Patchwork-State: RFC verify btf__add_kinds() adds kind encodings for all kinds supported, and after adding kind-related types for an unknown kind, ensure that parsing uses this info when that kind is encountered rather than giving up. Signed-off-by: Alan Maguire --- .../selftests/bpf/prog_tests/btf_kind.c | 138 ++++++++++++++++++ 1 file changed, 138 insertions(+) create mode 100644 tools/testing/selftests/bpf/prog_tests/btf_kind.c diff --git a/tools/testing/selftests/bpf/prog_tests/btf_kind.c b/tools/testing/selftests/bpf/prog_tests/btf_kind.c new file mode 100644 index 000000000000..a928415c60ff --- /dev/null +++ b/tools/testing/selftests/bpf/prog_tests/btf_kind.c @@ -0,0 +1,138 @@ +// SPDX-License-Identifier: GPL-2.0 +/* Copyright (c) 2023, Oracle and/or its affiliates. */ + +#include +#include +#include + +/* verify kind encoding exists for each kind */ +void test_btf_kind_encoding(struct btf *btf, char *description) +{ + const struct btf_header *hdr; + const struct btf_metadata *meta; + const void *raw_btf; + __u32 raw_size; + __u16 i; + + raw_btf = btf__raw_data(btf, &raw_size); + if (!ASSERT_OK_PTR(raw_btf, "btf__raw_data")) + return; + + hdr = raw_btf; + meta = raw_btf + hdr->hdr_len + hdr->meta_header.meta_off; + + if (!ASSERT_EQ(meta->kind_meta_cnt, NR_BTF_KINDS, "unexpected kind_meta_cnt")) + return; + + if (!ASSERT_EQ(strcmp(description, btf__name_by_offset(btf, meta->description_off)), + 0, "check meta description")) + return; + + for (i = 0; i <= BTF_KIND_MAX; i++) { + const struct btf_kind_meta *k = &meta->kind_meta[i]; + + if (ASSERT_OK_PTR(btf__name_by_offset(btf, k->name_off), "kind_name_valid")) + return; + } +} + +/* fabricate an unrecognized kind at BTF_KIND_MAX + 1, and after adding + * the appropriate struct/typedefs to the BTF such that it recognizes + * this kind, ensure that parsing of BTF containing the unrecognized kind + * can succeed. + */ +void test_btf_kind_decoding(struct btf *btf) +{ + __s32 int_id, unrec_id, id; + struct btf_type *t; + char btf_path[64]; + const void *raw_btf; + void *new_raw_btf; + struct btf *new_btf; + struct btf_header *hdr; + struct btf_metadata *meta; + struct btf_kind_meta *k; + __u32 raw_size; + int fd; + + int_id = btf__add_int(btf, "test_char", 1, BTF_INT_CHAR); + if (!ASSERT_GT(int_id, 0, "add_int_id")) + return; + + /* now create our type with unrecognized kind by adding a typedef kind + * we will overwrite it with our unrecognized kind value. + */ + unrec_id = btf__add_typedef(btf, "unrec_kind", int_id); + if (!ASSERT_GT(unrec_id, 0, "add_unrec_id")) + return; + + /* add an id after it that we will look up to verify we can parse + * beyond unrecognized kinds. + */ + id = btf__add_typedef(btf, "test_lookup", int_id); + if (!ASSERT_GT(id, 0, "add_test_lookup_id")) + return; + + raw_btf = (void *)btf__raw_data(btf, &raw_size); + if (!ASSERT_OK_PTR(raw_btf, "btf__raw_data")) + return; + + new_raw_btf = calloc(1, raw_size + sizeof(*k)); + memcpy(new_raw_btf, raw_btf, raw_size); + + /* add new metadata description */ + hdr = new_raw_btf; + hdr->meta_header.meta_len += sizeof(*k); + meta = new_raw_btf + hdr->hdr_len + hdr->meta_header.meta_off; + meta->kind_meta_cnt += 1; + /* we will call our kinds UNKN, re-using the string offsets from BTF_KIND_UNKN */ + k = &meta->kind_meta[NR_BTF_KINDS]; + k->name_off = meta->kind_meta[0].name_off + strlen("BTF_KIND_"); + k->flags = BTF_KIND_META_OPTIONAL; + k->info_sz = 0; + k->elem_sz = 0; + + /* now modify our typedef added above to be an unrecognized kind. */ + t = (void *)hdr + hdr->hdr_len + hdr->type_off + sizeof(struct btf_type) + + sizeof(__u32); + t->info = (NR_BTF_KINDS << 24); + + /* now write our BTF to a raw file, ready for parsing. */ + snprintf(btf_path, sizeof(btf_path), "/tmp/btf_kind.%d", getpid()); + fd = open(btf_path, O_WRONLY | O_CREAT); + write(fd, new_raw_btf, raw_size + sizeof(*k)); + close(fd); + + /* verify parsing succeeds, and that we can read type info past + * the unrecognized kind. + */ + new_btf = btf__parse_raw(btf_path); + if (ASSERT_OK_PTR(new_btf, "btf__parse_raw")) { + ASSERT_EQ(btf__find_by_name_kind(new_btf, "test_lookup", + BTF_KIND_TYPEDEF), id, + "verify_id_lookup"); + /* verify the kernel can handle unrecognized kinds. */ + ASSERT_EQ(btf__load_into_kernel(new_btf), 0, "btf_load_into_kernel"); + } + unlink(btf_path); +} + +void test_btf_kind(void) +{ + LIBBPF_OPTS(btf_new_opts, opts); + char *description = "testing metadata!"; + + opts.add_meta = true; + opts.description = description; + + struct btf *btf = btf__new_empty_opts(&opts); + + if (!ASSERT_OK_PTR(btf, "btf_new")) + return; + + if (test__start_subtest("btf_kind_encoding")) + test_btf_kind_encoding(btf, description); + if (test__start_subtest("btf_kind_decoding")) + test_btf_kind_decoding(btf); + btf__free(btf); +}