From patchwork Fri Mar 19 20:59:07 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrii Nakryiko X-Patchwork-Id: 12151905 X-Patchwork-Delegate: bpf@iogearbox.net Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-17.0 required=3.0 tests=BAYES_00,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id D6BCBC433E0 for ; Fri, 19 Mar 2021 20:59:54 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id ABFA76197E for ; Fri, 19 Mar 2021 20:59:54 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229912AbhCSU7W convert rfc822-to-8bit (ORCPT ); Fri, 19 Mar 2021 16:59:22 -0400 Received: from mx0a-00082601.pphosted.com ([67.231.145.42]:22168 "EHLO mx0a-00082601.pphosted.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230239AbhCSU7U (ORCPT ); Fri, 19 Mar 2021 16:59:20 -0400 Received: from pps.filterd (m0044012.ppops.net [127.0.0.1]) by mx0a-00082601.pphosted.com (8.16.0.43/8.16.0.43) with SMTP id 12JKsw8v000300 for ; Fri, 19 Mar 2021 13:59:19 -0700 Received: from mail.thefacebook.com ([163.114.132.120]) by mx0a-00082601.pphosted.com with ESMTP id 37c4vnhv7h-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128 verify=NOT) for ; Fri, 19 Mar 2021 13:59:19 -0700 Received: from intmgw001.37.frc1.facebook.com (2620:10d:c085:108::8) by mail.thefacebook.com (2620:10d:c085:21d::4) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2176.2; Fri, 19 Mar 2021 13:59:18 -0700 Received: by devbig012.ftw2.facebook.com (Postfix, from userid 137359) id 1639A2ED268B; Fri, 19 Mar 2021 13:59:13 -0700 (PDT) From: Andrii Nakryiko To: , , , CC: , Subject: [PATCH bpf-next 1/3] bpftool: improve skeleton generation for objects without BTF Date: Fri, 19 Mar 2021 13:59:07 -0700 Message-ID: <20210319205909.1748642-2-andrii@kernel.org> X-Mailer: git-send-email 2.30.2 In-Reply-To: <20210319205909.1748642-1-andrii@kernel.org> References: <20210319205909.1748642-1-andrii@kernel.org> MIME-Version: 1.0 X-FB-Internal: Safe X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:6.0.369,18.0.761 definitions=2021-03-19_12:2021-03-19,2021-03-19 signatures=0 X-Proofpoint-Spam-Details: rule=fb_default_notspam policy=fb_default score=0 priorityscore=1501 phishscore=0 suspectscore=0 adultscore=0 mlxscore=0 malwarescore=0 impostorscore=0 lowpriorityscore=0 bulkscore=0 spamscore=0 mlxlogscore=999 clxscore=1015 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2009150000 definitions=main-2103190143 X-FB-Internal: deliver Precedence: bulk List-ID: X-Mailing-List: bpf@vger.kernel.org X-Patchwork-Delegate: bpf@iogearbox.net If BPF object file is using global variables, but is compiled without BTF or ends up having only some of DATASEC types due to static linking, generated skeleton won't compile, as some parts of skeleton would assume memory-mapped struct definitions for each special data section. Fix this by generating empty struct definition for such data sections. The benefit of that is that they still will be memory-mapped by skeleton handling code in libbpf, and user-space parts of user application would be able to access data in them (though pointer casting), assuming they know the memory layout they need. Fixes: 985ead416df3 ("bpftool: Add skeleton codegen command") Signed-off-by: Andrii Nakryiko --- tools/bpf/bpftool/gen.c | 81 ++++++++++++++++++++++++++++++++--------- 1 file changed, 64 insertions(+), 17 deletions(-) diff --git a/tools/bpf/bpftool/gen.c b/tools/bpf/bpftool/gen.c index 31ade77f5ef8..6ec7c2e90a01 100644 --- a/tools/bpf/bpftool/gen.c +++ b/tools/bpf/bpftool/gen.c @@ -85,6 +85,18 @@ static const char *get_map_ident(const struct bpf_map *map) else return NULL; } +static const char *datasec_ident(const char *sec_name) +{ + if (strcmp(sec_name, ".data") == 0) + return "data"; + if (strcmp(sec_name, ".bss") == 0) + return "bss"; + if (strcmp(sec_name, ".rodata") == 0) + return "rodata"; + if (strcmp(sec_name, ".kconfig") == 0) + return "kconfig"; + return NULL; +} static void codegen_btf_dump_printf(void *ctx, const char *fmt, va_list args) { @@ -104,18 +116,12 @@ static int codegen_datasec_def(struct bpf_object *obj, char var_ident[256]; bool strip_mods = false; - if (strcmp(sec_name, ".data") == 0) { - sec_ident = "data"; - } else if (strcmp(sec_name, ".bss") == 0) { - sec_ident = "bss"; - } else if (strcmp(sec_name, ".rodata") == 0) { - sec_ident = "rodata"; - strip_mods = true; - } else if (strcmp(sec_name, ".kconfig") == 0) { - sec_ident = "kconfig"; - } else { + sec_ident = datasec_ident(sec_name); + if (!sec_ident) return 0; - } + + if (strcmp(sec_name, ".rodata") == 0) + strip_mods = true; printf(" struct %s__%s {\n", obj_name, sec_ident); for (i = 0; i < vlen; i++, sec_var++) { @@ -188,22 +194,63 @@ static int codegen_datasecs(struct bpf_object *obj, const char *obj_name) struct btf *btf = bpf_object__btf(obj); int n = btf__get_nr_types(btf); struct btf_dump *d; + struct bpf_map *map; + const struct btf_type *sec; + const char *sec_ident, *map_ident; int i, err = 0; d = btf_dump__new(btf, NULL, NULL, codegen_btf_dump_printf); if (IS_ERR(d)) return PTR_ERR(d); - for (i = 1; i <= n; i++) { - const struct btf_type *t = btf__type_by_id(btf, i); + bpf_object__for_each_map(map, obj) { + /* only generate definitions for memory-mapped internal maps */ + if (!bpf_map__is_internal(map)) + continue; + if (!(bpf_map__def(map)->map_flags & BPF_F_MMAPABLE)) + continue; - if (!btf_is_datasec(t)) + map_ident = get_map_ident(map); + if (!map_ident) continue; - err = codegen_datasec_def(obj, btf, d, t, obj_name); - if (err) - goto out; + sec = NULL; + for (i = 1; i <= n; i++) { + const struct btf_type *t = btf__type_by_id(btf, i); + const char *name; + + if (!btf_is_datasec(t)) + continue; + + name = btf__str_by_offset(btf, t->name_off); + sec_ident = datasec_ident(name); + if (!sec_ident) + continue; + + if (strcmp(sec_ident, map_ident) == 0) { + sec = t; + break; + } + } + + /* In rare cases when BPF object file is using global + * variables, but is compiled without BTF, we will have + * special internal map, but no corresponding DATASEC BTF + * type. In such case, generate empty structs for each such + * map. It will still be memory-mapped as a convenience for + * applications that know exact memory layout to expect. + */ + if (!sec) { + printf(" struct %s__%s {\n", obj_name, map_ident); + printf(" } *%s;\n", map_ident); + } else { + err = codegen_datasec_def(obj, btf, d, sec, obj_name); + if (err) + goto out; + } } + + out: btf_dump__free(d); return err; From patchwork Fri Mar 19 20:59:08 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrii Nakryiko X-Patchwork-Id: 12151907 X-Patchwork-Delegate: bpf@iogearbox.net Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-17.0 required=3.0 tests=BAYES_00,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id E0489C433E1 for ; Fri, 19 Mar 2021 21:00:26 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id BBBAC6197B for ; Fri, 19 Mar 2021 21:00:26 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230351AbhCSU7y convert rfc822-to-8bit (ORCPT ); Fri, 19 Mar 2021 16:59:54 -0400 Received: from mx0a-00082601.pphosted.com ([67.231.145.42]:21756 "EHLO mx0a-00082601.pphosted.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230177AbhCSU7W (ORCPT ); Fri, 19 Mar 2021 16:59:22 -0400 Received: from pps.filterd (m0044010.ppops.net [127.0.0.1]) by mx0a-00082601.pphosted.com (8.16.0.43/8.16.0.43) with SMTP id 12JKuMRZ026758 for ; Fri, 19 Mar 2021 13:59:22 -0700 Received: from mail.thefacebook.com ([163.114.132.120]) by mx0a-00082601.pphosted.com with ESMTP id 37ca5v838e-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128 verify=NOT) for ; Fri, 19 Mar 2021 13:59:22 -0700 Received: from intmgw001.06.ash9.facebook.com (2620:10d:c085:108::8) by mail.thefacebook.com (2620:10d:c085:11d::7) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2176.2; Fri, 19 Mar 2021 13:59:21 -0700 Received: by devbig012.ftw2.facebook.com (Postfix, from userid 137359) id 41FA52ED268B; Fri, 19 Mar 2021 13:59:15 -0700 (PDT) From: Andrii Nakryiko To: , , , CC: , , Jiri Olsa Subject: [PATCH bpf-next 2/3] libbpf: skip BTF fixup if object file has no BTF Date: Fri, 19 Mar 2021 13:59:08 -0700 Message-ID: <20210319205909.1748642-3-andrii@kernel.org> X-Mailer: git-send-email 2.30.2 In-Reply-To: <20210319205909.1748642-1-andrii@kernel.org> References: <20210319205909.1748642-1-andrii@kernel.org> MIME-Version: 1.0 X-FB-Internal: Safe X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:6.0.369,18.0.761 definitions=2021-03-19_12:2021-03-19,2021-03-19 signatures=0 X-Proofpoint-Spam-Details: rule=fb_default_notspam policy=fb_default score=0 clxscore=1015 phishscore=0 spamscore=0 mlxscore=0 lowpriorityscore=0 adultscore=0 impostorscore=0 mlxlogscore=999 priorityscore=1501 malwarescore=0 bulkscore=0 suspectscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2009150000 definitions=main-2103190143 X-FB-Internal: deliver Precedence: bulk List-ID: X-Mailing-List: bpf@vger.kernel.org X-Patchwork-Delegate: bpf@iogearbox.net Skip BTF fixup step when input object file is missing BTF altogether. Reported-by: Jiri Olsa Fixes: 8fd27bf69b86 ("libbpf: Add BPF static linker BTF and BTF.ext support") Signed-off-by: Andrii Nakryiko Tested-by: Jiri Olsa --- tools/lib/bpf/linker.c | 3 +++ 1 file changed, 3 insertions(+) diff --git a/tools/lib/bpf/linker.c b/tools/lib/bpf/linker.c index b4fff912dce2..5e0aa2f2c0ca 100644 --- a/tools/lib/bpf/linker.c +++ b/tools/lib/bpf/linker.c @@ -1313,6 +1313,9 @@ static int linker_fixup_btf(struct src_obj *obj) struct src_sec *sec; int i, j, n, m; + if (!obj->btf) + return 0; + n = btf__get_nr_types(obj->btf); for (i = 1; i <= n; i++) { struct btf_var_secinfo *vi; From patchwork Fri Mar 19 20:59:09 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrii Nakryiko X-Patchwork-Id: 12151901 X-Patchwork-Delegate: bpf@iogearbox.net Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-17.0 required=3.0 tests=BAYES_00,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 0EA14C433DB for ; Fri, 19 Mar 2021 20:59:54 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id D3A6D6197E for ; Fri, 19 Mar 2021 20:59:53 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230203AbhCSU7W convert rfc822-to-8bit (ORCPT ); Fri, 19 Mar 2021 16:59:22 -0400 Received: from mx0a-00082601.pphosted.com ([67.231.145.42]:34144 "EHLO mx0a-00082601.pphosted.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230240AbhCSU7V (ORCPT ); Fri, 19 Mar 2021 16:59:21 -0400 Received: from pps.filterd (m0148461.ppops.net [127.0.0.1]) by mx0a-00082601.pphosted.com (8.16.0.43/8.16.0.43) with SMTP id 12JKwkXc004259 for ; Fri, 19 Mar 2021 13:59:21 -0700 Received: from maileast.thefacebook.com ([163.114.130.16]) by mx0a-00082601.pphosted.com with ESMTP id 37bs1ew1rs-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128 verify=NOT) for ; Fri, 19 Mar 2021 13:59:21 -0700 Received: from intmgw006.03.ash8.facebook.com (2620:10d:c0a8:1b::d) by mail.thefacebook.com (2620:10d:c0a8:83::7) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2176.2; Fri, 19 Mar 2021 13:59:20 -0700 Received: by devbig012.ftw2.facebook.com (Postfix, from userid 137359) id 6C6AA2ED268B; Fri, 19 Mar 2021 13:59:17 -0700 (PDT) From: Andrii Nakryiko To: , , , CC: , Subject: [PATCH bpf-next 3/3] selftests/bpf: allow compiling BPF objects without BTF Date: Fri, 19 Mar 2021 13:59:09 -0700 Message-ID: <20210319205909.1748642-4-andrii@kernel.org> X-Mailer: git-send-email 2.30.2 In-Reply-To: <20210319205909.1748642-1-andrii@kernel.org> References: <20210319205909.1748642-1-andrii@kernel.org> MIME-Version: 1.0 X-FB-Internal: Safe X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:6.0.369,18.0.761 definitions=2021-03-19_12:2021-03-19,2021-03-19 signatures=0 X-Proofpoint-Spam-Details: rule=fb_default_notspam policy=fb_default score=0 suspectscore=0 phishscore=0 bulkscore=0 spamscore=0 clxscore=1015 lowpriorityscore=0 priorityscore=1501 mlxscore=0 adultscore=0 mlxlogscore=999 malwarescore=0 impostorscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2009150000 definitions=main-2103190144 X-FB-Internal: deliver Precedence: bulk List-ID: X-Mailing-List: bpf@vger.kernel.org X-Patchwork-Delegate: bpf@iogearbox.net Add ability to skip BTF generation for some BPF object files. This is done through using a convention of .nobtf.c file name suffix. Also add third statically linked file to static_linked selftest. This file has no BTF, causing resulting object file to have only some of DATASEC BTF types. It also is using (from BPF code) global variables. This tests both libbpf's static linking logic and bpftool's skeleton generation logic. Signed-off-by: Andrii Nakryiko --- tools/testing/selftests/bpf/Makefile | 21 +++++++---- .../selftests/bpf/prog_tests/static_linked.c | 6 +++- .../bpf/progs/test_static_linked3.nobtf.c | 36 +++++++++++++++++++ 3 files changed, 56 insertions(+), 7 deletions(-) create mode 100644 tools/testing/selftests/bpf/progs/test_static_linked3.nobtf.c diff --git a/tools/testing/selftests/bpf/Makefile b/tools/testing/selftests/bpf/Makefile index 6448c626498f..0a481a75a416 100644 --- a/tools/testing/selftests/bpf/Makefile +++ b/tools/testing/selftests/bpf/Makefile @@ -270,7 +270,7 @@ IS_LITTLE_ENDIAN = $(shell $(CC) -dM -E - bss->var1, 1 * 2 + 2 + 3, "var1"); ASSERT_EQ(skel->bss->var2, 4 * 3 + 5 + 6, "var2"); + err = bpf_map_lookup_elem(bpf_map__fd(skel->maps.legacy_map), &key, &value); + ASSERT_OK(err, "legacy_map_lookup"); + ASSERT_EQ(value, 1 * 3 + 3, "legacy_map_value"); + cleanup: test_static_linked__destroy(skel); } diff --git a/tools/testing/selftests/bpf/progs/test_static_linked3.nobtf.c b/tools/testing/selftests/bpf/progs/test_static_linked3.nobtf.c new file mode 100644 index 000000000000..e5fbde21381c --- /dev/null +++ b/tools/testing/selftests/bpf/progs/test_static_linked3.nobtf.c @@ -0,0 +1,36 @@ +// SPDX-License-Identifier: GPL-2.0 +/* Copyright (c) 2021 Facebook */ + +#include +#include + +/* global variables don't need BTF to be used, but are extremely unconvenient + * to be consumed from user-space without BPF skeleton, that uses BTF + */ + +static volatile int mul3 = 3; +static volatile int add3 = 3; + +/* same "subprog" name in all files */ +static __noinline int subprog(int x) +{ + /* but different formula */ + return x * mul3 + add3; +} + +struct bpf_map_def SEC("maps") legacy_map = { + .type = BPF_MAP_TYPE_ARRAY, + .key_size = sizeof(int), + .value_size = sizeof(int), + .max_entries = 1, +}; + +SEC("raw_tp/sys_enter") +int handler3(const void *ctx) +{ + int key = 0, value = subprog(1); + + bpf_map_update_elem(&legacy_map, &key, &value, BPF_ANY); + + return 0; +}