From patchwork Sun Feb 20 13:47:59 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kumar Kartikeya Dwivedi X-Patchwork-Id: 12752703 X-Patchwork-Delegate: bpf@iogearbox.net Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id D84D7C433FE for ; Sun, 20 Feb 2022 13:48:26 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S241184AbiBTNsp (ORCPT ); Sun, 20 Feb 2022 08:48:45 -0500 Received: from mxb-00190b01.gslb.pphosted.com ([23.128.96.19]:48384 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229884AbiBTNsl (ORCPT ); Sun, 20 Feb 2022 08:48:41 -0500 Received: from mail-pl1-x644.google.com (mail-pl1-x644.google.com [IPv6:2607:f8b0:4864:20::644]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 4523921813; Sun, 20 Feb 2022 05:48:20 -0800 (PST) Received: by mail-pl1-x644.google.com with SMTP id l8so10858885pls.7; Sun, 20 Feb 2022 05:48:20 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=PzcNo+2Cl6iYUCYcvpUBH3gr0tfx0tpOlkynbpKJJzs=; b=gnkR8SFne9AxfmrY6l1U3pCSAtxEEohSpRVSFpn4eHh0ml4Jme0rYQTjuVFoDLakMh N4eQNTxJkoRY+/myDHc96XRvuVXV5Jujoy7Cq2Ruk5e00DC5/VQEpl9oXD/Grlf7RbrR cPy3ZAFPgyA0l09vRq5MECIRYXovWC3s9sGWkTrBiKElJCKDZjD0nesWncdFN8m39Kar uvPvX3XiCwpsduDJ0IineI1MF62YDqT3fAUKwJPUGv9n9Lq3JjlLKCcRFmr0jje4kX69 UpvDtq0CX6UjbnQxen/FOQYStpjDzyUnbx5iwZughSYpJGRhhpmOncefnA8rXhsmrRHT qm3Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=PzcNo+2Cl6iYUCYcvpUBH3gr0tfx0tpOlkynbpKJJzs=; b=EgMqZUllYqhhlHhK2l8oP03ADEhScGhM09lo6A6qdAjzPDCn0drRTfHM3HKr9491/n x+yx05BCNTANSWR3lqneL7PnKnwQwYUrJLfff/w+bXCJuTZtc/WF0187I9jiW28U7VQd kaaq0LnaZS3WwKCsgGG75g+nDE7JoImla/Inl82HYN7LVmIrF1OJL8JngzNa9evpRxjk atKNM+3+j0Ut5Djg360CtEPrieAfWgCyVVWqFoZqXdGj825lk6VdD8R6iIBvIfZVp5qG iVGDqRzyx8s2vw5IL7qyW7fILI0PpKtmnMA3jXnRb8o5x1ARrocXlSdI9SyqidGbIV80 6acg== X-Gm-Message-State: AOAM530NXBrzMJLgLyvWv9FcSlY7FAhiGOI183DW7NtZqSv4tryVEfJ2 bj+g1ZL7gNs30oDzoeU2Km4Ek0kpfeo= X-Google-Smtp-Source: ABdhPJxqFb7mKuV/HBOE8jJordIDkenzpU+FOrtk0NuYyxSbBgqwHYFumCCvdIjy4oBUBuMqkzlYeA== X-Received: by 2002:a17:903:2283:b0:14d:b86b:165b with SMTP id b3-20020a170903228300b0014db86b165bmr15171985plh.41.1645364899656; Sun, 20 Feb 2022 05:48:19 -0800 (PST) Received: from localhost ([2405:201:6014:d0c0:6243:316e:a9e1:adda]) by smtp.gmail.com with ESMTPSA id g63sm9060558pfb.65.2022.02.20.05.48.18 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 20 Feb 2022 05:48:19 -0800 (PST) From: Kumar Kartikeya Dwivedi To: bpf@vger.kernel.org Cc: Alexei Starovoitov , Andrii Nakryiko , Daniel Borkmann , =?utf-8?q?Toke_H=C3=B8iland-J=C3=B8?= =?utf-8?q?rgensen?= , Jesper Dangaard Brouer , netfilter-devel@vger.kernel.org, netdev@vger.kernel.org Subject: [PATCH bpf-next v1 01/15] bpf: Factor out fd returning from bpf_btf_find_by_name_kind Date: Sun, 20 Feb 2022 19:17:59 +0530 Message-Id: <20220220134813.3411982-2-memxor@gmail.com> X-Mailer: git-send-email 2.35.1 In-Reply-To: <20220220134813.3411982-1-memxor@gmail.com> References: <20220220134813.3411982-1-memxor@gmail.com> MIME-Version: 1.0 X-Developer-Signature: v=1; a=openpgp-sha256; l=2938; h=from:subject; bh=/DfuqiVvvM+Atumhzb4HyEeRwNtGOpOsgZS5ZW8b5tQ=; b=owEBbQKS/ZANAwAIAUzgyIZIvxHKAcsmYgBiEkZXkMyYe/Lt+F66RQ4bL9Z/mevwOrvjMnzoNuHb UewkETyJAjMEAAEIAB0WIQRLvip+Buz51YI8YRFM4MiGSL8RygUCYhJGVwAKCRBM4MiGSL8RyhZEEA C65AeyeTfBXOEkpFn9Eq6xuZtpdc9omJPpWReu529cRJRFrjUYDvFK0KN+F9rNDwJsC2O0Hxicpy6B CX7BOpIrOKrrlbiGluICkOiEB+GiCyndtyJ+UC71Po2tbAaEm2Yz1evJL41M2ovtdUCLHPG/q7QA3O Zql7ooRamhF9UD86tNobw5esP7xJXDa8BCkX+M/8bnIloKXZZvMd4sFeVdhZTPHPUws6jTA88hvNYL AF/S5SVAf2Sa0vz5+yWB5tAQUvKhlVJba4zuaYngd8eq68u36hFhMZLI0j8Us0AUsSIN1aCrzPAgXQ 9s2Qf0nTw3HzBDNwwi6o3ybUvfq7QXTp8Dyp/01x8+EJQSCNt94fCIKbVSxoE+S7j5wrLcx+XJp5O1 A7mQT+eA/GCBFU32NP93PIcqsi0izNtyfNuGPjY2Xj55QxF9/mJMxesFgpvlVU1I89rKvUNgQANaKs MS7ImcUoiz+nA46oQqueVemKYom36FY+Yt8rY60N+yiMuoDuq0dp9wbCRu7TMiEEHtPCwT8CONw2Bq 419k3FJ7RN9oJJ3MQftXr43AZcO7VCnu0V/lV3dCsbD7M8TcQLaTqxk4c2z+Vay+CuKcazISe9Z51a EO0pTAYniMtdqUdRaYX/2WU55kj+Qh4R60p+ubbAx0E1YXDQxkWql+7AnAHA== X-Developer-Key: i=memxor@gmail.com; a=openpgp; fpr=4BBE2A7E06ECF9D5823C61114CE0C88648BF11CA Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org X-Patchwork-Delegate: bpf@iogearbox.net In next few patches, we need a helper that searches all kernel BTFs (vmlinux and module BTFs), and finds the type denoted by 'name' and 'kind'. Turns out bpf_btf_find_by_name_kind already does the same thing, but it instead returns a BTF ID and optionally fd (if module BTF). This is used for relocating ksyms in BPF loader code (bpftool gen skel -L). We extract the core code out into a new helper btf_find_by_name_kind_all, which returns the BTF ID and BTF pointer in an out parameter. The reference for the returned BTF pointer is only bumped if it is a module BTF, this needs to be kept in mind when using this helper. Hence, the user must release the BTF reference iff btf_is_module is true, otherwise transfer the ownership to e.g. an fd. In case of the helper, the fd is only allocated for module BTFs, so no extra handling for btf_vmlinux case is required. Signed-off-by: Kumar Kartikeya Dwivedi --- kernel/bpf/btf.c | 47 +++++++++++++++++++++++++++++++---------------- 1 file changed, 31 insertions(+), 16 deletions(-) diff --git a/kernel/bpf/btf.c b/kernel/bpf/btf.c index 2c4c5dbe2abe..3645d8c14a18 100644 --- a/kernel/bpf/btf.c +++ b/kernel/bpf/btf.c @@ -6545,16 +6545,10 @@ static struct btf *btf_get_module_btf(const struct module *module) return btf; } -BPF_CALL_4(bpf_btf_find_by_name_kind, char *, name, int, name_sz, u32, kind, int, flags) +static s32 btf_find_by_name_kind_all(const char *name, u32 kind, struct btf **btfp) { struct btf *btf; - long ret; - - if (flags) - return -EINVAL; - - if (name_sz <= 1 || name[name_sz - 1]) - return -EINVAL; + s32 ret; btf = bpf_get_btf_vmlinux(); if (IS_ERR(btf)) @@ -6580,19 +6574,40 @@ BPF_CALL_4(bpf_btf_find_by_name_kind, char *, name, int, name_sz, u32, kind, int spin_unlock_bh(&btf_idr_lock); ret = btf_find_by_name_kind(mod_btf, name, kind); if (ret > 0) { - int btf_obj_fd; - - btf_obj_fd = __btf_new_fd(mod_btf); - if (btf_obj_fd < 0) { - btf_put(mod_btf); - return btf_obj_fd; - } - return ret | (((u64)btf_obj_fd) << 32); + *btfp = mod_btf; + return ret; } spin_lock_bh(&btf_idr_lock); btf_put(mod_btf); } spin_unlock_bh(&btf_idr_lock); + } else { + *btfp = btf; + } + return ret; +} + +BPF_CALL_4(bpf_btf_find_by_name_kind, char *, name, int, name_sz, u32, kind, int, flags) +{ + struct btf *btf = NULL; + int btf_obj_fd = 0; + long ret; + + if (flags) + return -EINVAL; + + if (name_sz <= 1 || name[name_sz - 1]) + return -EINVAL; + + ret = btf_find_by_name_kind_all(name, kind, &btf); + if (ret > 0 && btf_is_module(btf)) { + /* reference for btf is only raised if module BTF */ + btf_obj_fd = __btf_new_fd(btf); + if (btf_obj_fd < 0) { + btf_put(btf); + return btf_obj_fd; + } + return ret | (((u64)btf_obj_fd) << 32); } return ret; } From patchwork Sun Feb 20 13:48:00 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kumar Kartikeya Dwivedi X-Patchwork-Id: 12752704 X-Patchwork-Delegate: bpf@iogearbox.net Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 68729C4321E for ; Sun, 20 Feb 2022 13:48:28 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S243208AbiBTNsq (ORCPT ); Sun, 20 Feb 2022 08:48:46 -0500 Received: from mxb-00190b01.gslb.pphosted.com ([23.128.96.19]:48544 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S238277AbiBTNso (ORCPT ); Sun, 20 Feb 2022 08:48:44 -0500 Received: from mail-pj1-x1041.google.com (mail-pj1-x1041.google.com [IPv6:2607:f8b0:4864:20::1041]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 62CA62D1DD; Sun, 20 Feb 2022 05:48:23 -0800 (PST) Received: by mail-pj1-x1041.google.com with SMTP id q8-20020a17090a178800b001bc299b8de1so1167396pja.1; Sun, 20 Feb 2022 05:48:23 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=tScrAsZNPZ4z7UHmft7Hv1gjRNBu/0AWO2o1jJpasDE=; b=SDbmL2q4GpAxfF5DdIY7SiWIrev11XaxkZ2hhzmM44JQrnAspiKVZI1rqtfWyJLk3B kny6ghEkJPYQo7c2iiVvlNj9HdHsFwsOTmeCZ0+i7WlryJCn4M4nUq6tb5UxziMmY/3y 4OlRMyGVV9li9DS2lIjiM26Okf3FrBgbFTVjkdNC+x/5sLxCHvVZwCJLHsvS8pVeqyO4 UY47TOYrUGm8rACpEirj4pFW8Zl+2mcD/hEeIquJGfZo1a6Nl1jizGEI3Ot+glF6C2v0 l1SXSlnjiPllANtinj6Gsf0EX2PqY6GP13GzAPOocroK+sPjgvAMZX2LBN+JUzqu4wCd eFqA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=tScrAsZNPZ4z7UHmft7Hv1gjRNBu/0AWO2o1jJpasDE=; b=h7mmwttjPv64ltlpH9WVphtSY4PurK/lgMy0hPzQ70hMvvBV8+Y6GMJDYvPtB0OCpG FnxhdCoDUO3xLm7/pS/6jPoNiCEJJfphwSRBkq2gS0WQ7bkeO724L+XlqDdBYaB3e1W2 zpagA5niz6kPBh8oWjPRjHpWp8jMdXc/kyl3w6jrUNYgZ8867hOzlk3gZXbm3B987su9 +V8l4R2Bd4Z+ra5PzREhcyaY0IdQmjWMF6bnD1o/CfghFxz7fHdnlqu1Sv83st82YQxk 6uILUPvK2CYJZvtU5DLtOkcekiorlAXYCGUSYyPKsn2JxyZGPJTyARdDV4VZ2gThqQ+H 7gIQ== X-Gm-Message-State: AOAM5316t/M+Gc3vlKXPDBRw72sbrhD3zQMbY2T6e7MpUGXd45zMVKfK vTY4/3NcUpEeNmWn8mwfjYH2os8Ja0o= X-Google-Smtp-Source: ABdhPJyWrN+KgyPgY+jZViH70EO8u5MlWSZ2EFYQ6PgYwobuq+3rVGdHHO9BHuoSzZHIC+2qKr/sjQ== X-Received: by 2002:a17:902:db04:b0:14c:f43b:e9df with SMTP id m4-20020a170902db0400b0014cf43be9dfmr15305668plx.76.1645364902676; Sun, 20 Feb 2022 05:48:22 -0800 (PST) Received: from localhost ([2405:201:6014:d0c0:6243:316e:a9e1:adda]) by smtp.gmail.com with ESMTPSA id m19sm5989212pfk.15.2022.02.20.05.48.21 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 20 Feb 2022 05:48:22 -0800 (PST) From: Kumar Kartikeya Dwivedi To: bpf@vger.kernel.org Cc: Alexei Starovoitov , Andrii Nakryiko , Daniel Borkmann , =?utf-8?q?Toke_H=C3=B8iland-J=C3=B8?= =?utf-8?q?rgensen?= , Jesper Dangaard Brouer , netfilter-devel@vger.kernel.org, netdev@vger.kernel.org Subject: [PATCH bpf-next v1 02/15] bpf: Make btf_find_field more generic Date: Sun, 20 Feb 2022 19:18:00 +0530 Message-Id: <20220220134813.3411982-3-memxor@gmail.com> X-Mailer: git-send-email 2.35.1 In-Reply-To: <20220220134813.3411982-1-memxor@gmail.com> References: <20220220134813.3411982-1-memxor@gmail.com> MIME-Version: 1.0 X-Developer-Signature: v=1; a=openpgp-sha256; l=6216; h=from:subject; bh=0qXHd1TkFCU1UpyfBjDxPH7u7vulr4FC18+1Eom8vyo=; b=owEBbQKS/ZANAwAIAUzgyIZIvxHKAcsmYgBiEkZXmmZlmlAH5SDX0rZCSizYI+2cvUqtVhJkOtm5 RJe0VeOJAjMEAAEIAB0WIQRLvip+Buz51YI8YRFM4MiGSL8RygUCYhJGVwAKCRBM4MiGSL8RyvF9D/ 0cU6szyBxLtBnTLArn5FKK5r1zQGJ2pWsJpfFiCtz52oSl+3mn1TWRs5wjqrFfuhUOCClT4OZ9nNr4 RzMkaC69RiTgU4X7noxj11dWpmzmmrl56hb2IlTwX49nBFpyVQje4Ms0F8XRsSVGwRjOdalk9jWYbu S1un+SOCf0xPj3MlOcOoMncP52waHh7OC2gdWv7mK7vNYjj827VvS0Cr8npPoSVvizTA02IcBNbFBY so6VtrxEGo+Mw6JNCpH1h+w/PPZgDEkzEMcoiQ5R4IZYkxO7FsDhXgh8G7tFRy21SUHaqLfIduL2aK aw/OF7pdWrpl8EHrVi97E5bHefdZP3XUHYWlJD5RnQtkAz06uB2cpLJpu9l0RzcWcUGxOTHd73bBjl F9+6xQTNPsiqEKdLu9lOPbr6IjruggtI8+9HwS6Fl/rkV6ZJ1Prm4rNyrbbuGqUnG8irARx2vNcATX yiyOv+GBvt7G6kHfRtvu/0d4HM89VkACWaZq9QbYf7T1ZmMKkuas+fFLHqsD0Dggr8nlRcnKTMT55E v1TrEpwbXOFgMeYrjkJJDub975c1A2RL0QFNK+1Hk0RkY2vvmIyVWbj86cDtLNo44CtWMgdNnPmdjL sk3qq7A3aZq30R9Ww16TXxHuH8Byqv6OLA8CDC0LztdRMP9St6HrRGAAvJBg== X-Developer-Key: i=memxor@gmail.com; a=openpgp; fpr=4BBE2A7E06ECF9D5823C61114CE0C88648BF11CA Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org X-Patchwork-Delegate: bpf@iogearbox.net Next commit's field type will not be struct, but pointer, and it will not be limited to one offset, but multiple ones. Make existing btf_find_struct_field and btf_find_datasec_var functions amenable to use for finding BTF ID pointers in map value, by taking a moving spin_lock and timer specific checks into their own function. The alignment, and name are checked before the function is called, so it is the last point where we can skip field or return an error before the next loop iteration happens. This is important, because we'll be potentially reallocating memory inside this function in next commit, so being able to do that when everything else is in order is going to be more convenient. The name parameter is now optional, and only checked if it is not NULL. The size must be checked in the function, because in case of PTR it will instead point to the underlying BTF ID it is pointing to (or modifiers), so the check becomes wrong to do outside of function, and the base type has to be obtained by removing modifiers. Signed-off-by: Kumar Kartikeya Dwivedi --- kernel/bpf/btf.c | 119 +++++++++++++++++++++++++++++++++-------------- 1 file changed, 85 insertions(+), 34 deletions(-) diff --git a/kernel/bpf/btf.c b/kernel/bpf/btf.c index 3645d8c14a18..55f6ccac3388 100644 --- a/kernel/bpf/btf.c +++ b/kernel/bpf/btf.c @@ -3119,71 +3119,108 @@ static void btf_struct_log(struct btf_verifier_env *env, btf_verifier_log(env, "size=%u vlen=%u", t->size, btf_type_vlen(t)); } +enum { + BTF_FIELD_SPIN_LOCK, + BTF_FIELD_TIMER, +}; + +static int btf_find_field_struct(const struct btf *btf, const struct btf_type *t, + u32 off, int sz, void *data) +{ + u32 *offp = data; + + if (!__btf_type_is_struct(t)) + return 0; + if (t->size != sz) + return 0; + if (*offp != -ENOENT) + /* only one such field is allowed */ + return -E2BIG; + *offp = off; + return 0; +} + static int btf_find_struct_field(const struct btf *btf, const struct btf_type *t, - const char *name, int sz, int align) + const char *name, int sz, int align, int field_type, + void *data) { const struct btf_member *member; - u32 i, off = -ENOENT; + u32 i, off; + int ret; for_each_member(i, t, member) { const struct btf_type *member_type = btf_type_by_id(btf, member->type); - if (!__btf_type_is_struct(member_type)) - continue; - if (member_type->size != sz) - continue; - if (strcmp(__btf_name_by_offset(btf, member_type->name_off), name)) - continue; - if (off != -ENOENT) - /* only one such field is allowed */ - return -E2BIG; + off = __btf_member_bit_offset(t, member); + + if (name && strcmp(__btf_name_by_offset(btf, member_type->name_off), name)) + continue; if (off % 8) /* valid C code cannot generate such BTF */ return -EINVAL; off /= 8; if (off % align) return -EINVAL; + + switch (field_type) { + case BTF_FIELD_SPIN_LOCK: + case BTF_FIELD_TIMER: + ret = btf_find_field_struct(btf, member_type, off, sz, data); + if (ret < 0) + return ret; + break; + default: + pr_err("verifier bug: unknown field type requested\n"); + return -EFAULT; + } } - return off; + return 0; } static int btf_find_datasec_var(const struct btf *btf, const struct btf_type *t, - const char *name, int sz, int align) + const char *name, int sz, int align, int field_type, + void *data) { const struct btf_var_secinfo *vsi; - u32 i, off = -ENOENT; + u32 i, off; + int ret; for_each_vsi(i, t, vsi) { const struct btf_type *var = btf_type_by_id(btf, vsi->type); const struct btf_type *var_type = btf_type_by_id(btf, var->type); - if (!__btf_type_is_struct(var_type)) - continue; - if (var_type->size != sz) + off = vsi->offset; + + if (name && strcmp(__btf_name_by_offset(btf, var_type->name_off), name)) continue; if (vsi->size != sz) continue; - if (strcmp(__btf_name_by_offset(btf, var_type->name_off), name)) - continue; - if (off != -ENOENT) - /* only one such field is allowed */ - return -E2BIG; - off = vsi->offset; if (off % align) return -EINVAL; + + switch (field_type) { + case BTF_FIELD_SPIN_LOCK: + case BTF_FIELD_TIMER: + ret = btf_find_field_struct(btf, var_type, off, sz, data); + if (ret < 0) + return ret; + break; + default: + return -EFAULT; + } } - return off; + return 0; } static int btf_find_field(const struct btf *btf, const struct btf_type *t, - const char *name, int sz, int align) + const char *name, int sz, int align, int field_type, + void *data) { - if (__btf_type_is_struct(t)) - return btf_find_struct_field(btf, t, name, sz, align); + return btf_find_struct_field(btf, t, name, sz, align, field_type, data); else if (btf_type_is_datasec(t)) - return btf_find_datasec_var(btf, t, name, sz, align); + return btf_find_datasec_var(btf, t, name, sz, align, field_type, data); return -EINVAL; } @@ -3193,16 +3230,30 @@ static int btf_find_field(const struct btf *btf, const struct btf_type *t, */ int btf_find_spin_lock(const struct btf *btf, const struct btf_type *t) { - return btf_find_field(btf, t, "bpf_spin_lock", - sizeof(struct bpf_spin_lock), - __alignof__(struct bpf_spin_lock)); + u32 off = -ENOENT; + int ret; + + ret = btf_find_field(btf, t, "bpf_spin_lock", + sizeof(struct bpf_spin_lock), + __alignof__(struct bpf_spin_lock), + BTF_FIELD_SPIN_LOCK, &off); + if (ret < 0) + return ret; + return off; } int btf_find_timer(const struct btf *btf, const struct btf_type *t) { - return btf_find_field(btf, t, "bpf_timer", - sizeof(struct bpf_timer), - __alignof__(struct bpf_timer)); + u32 off = -ENOENT; + int ret; + + ret = btf_find_field(btf, t, "bpf_timer", + sizeof(struct bpf_timer), + __alignof__(struct bpf_timer), + BTF_FIELD_TIMER, &off); + if (ret < 0) + return ret; + return off; } static void __btf_struct_show(const struct btf *btf, const struct btf_type *t, From patchwork Sun Feb 20 13:48:01 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kumar Kartikeya Dwivedi X-Patchwork-Id: 12752705 X-Patchwork-Delegate: bpf@iogearbox.net Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 6AAA7C433FE for ; Sun, 20 Feb 2022 13:48:34 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S243869AbiBTNsx (ORCPT ); Sun, 20 Feb 2022 08:48:53 -0500 Received: from mxb-00190b01.gslb.pphosted.com ([23.128.96.19]:48828 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S238277AbiBTNst (ORCPT ); Sun, 20 Feb 2022 08:48:49 -0500 Received: from mail-pl1-x641.google.com (mail-pl1-x641.google.com [IPv6:2607:f8b0:4864:20::641]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id AD1A721813; Sun, 20 Feb 2022 05:48:26 -0800 (PST) Received: by mail-pl1-x641.google.com with SMTP id 4so2246438pll.6; Sun, 20 Feb 2022 05:48:26 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=wlBIsnH0cG6LQh9cPjAUVPWs/qbSZZzGZWdz+JWcGgs=; b=R5HVDARgtEwOYoUsLzS0LJJgYPBdFFz50uE2VhghE7ZnNGkQN7+VdP58Jh2irNM6q1 rdgdQWR5okhIOnp6lZboesGRWVVzaNhBpFombMTzuRc9Z7FUHe53oafNWCTxCvtUyeWC B1OZjHS6ds0noBUUYTzY8G2rNkzf6FDh3JXQ5IIrYBXT1PIscrgW2jzVU/EYisK05/eK RXQ+jBIAnDFPrtRovmDx5KRhwQwsUL5ZRqzHL04vggp2CMxKh7UiwBi6ZKNfe468b6AH NXco8h2ILRMTLpXwnc44TcyqPAVAYj/d1AqAiJ4SLf4R3L8YkyUJ1PFnTI4EtjEKMu6a p+cQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=wlBIsnH0cG6LQh9cPjAUVPWs/qbSZZzGZWdz+JWcGgs=; b=SPtNeK7u0nlXsGB8pzvOdttZRJV3HZYTSQnTRsyfvHWCY2nMWDNd/ALzvnav0YYzHP dwk/fK6g89CGZdHjLTnUQLjEBo9TZX/MoCaiB+uWtCE6oyncOYaY9bReLq447HQ+STT1 AoEe0xZM/fR4TXs5YL4hLOQatAldEuLj8IOzWyWTqOYCOM9ks+l9Y+m8rW2cKOHmahHr FHKS1MbtIqeWFuIKSI+FL9pwSRTUsfy5teSDG0G3UZFLmKP2LV6YSaMgHV7MEpx4P2DA GUT4PqpfSIQU6jry5L/dG/azXGkB3wTUHvL7cMIFnUwxK8ud5wOhcDr5W+j3OzYVghh6 oeww== X-Gm-Message-State: AOAM532l7pubDYjbs5FX+jXvGFF3U51VJuAbCVLVGuwZNuklpycvvWwj /fePmOl4sworYx1OVXZV70w+9mDyOJQ= X-Google-Smtp-Source: ABdhPJweIPU5lrBHjH2fVoa9iVnlb0wr7l5BlsktMwnVLLYOth0b1H03rowUEBlxB+Pie4poW5H6IQ== X-Received: by 2002:a17:902:f550:b0:14f:acf1:c729 with SMTP id h16-20020a170902f55000b0014facf1c729mr2709329plf.82.1645364905776; Sun, 20 Feb 2022 05:48:25 -0800 (PST) Received: from localhost ([2405:201:6014:d0c0:6243:316e:a9e1:adda]) by smtp.gmail.com with ESMTPSA id s40sm9912412pfg.145.2022.02.20.05.48.25 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 20 Feb 2022 05:48:25 -0800 (PST) From: Kumar Kartikeya Dwivedi To: bpf@vger.kernel.org Cc: Alexei Starovoitov , Andrii Nakryiko , Daniel Borkmann , =?utf-8?q?Toke_H=C3=B8iland-J=C3=B8?= =?utf-8?q?rgensen?= , Jesper Dangaard Brouer , netfilter-devel@vger.kernel.org, netdev@vger.kernel.org Subject: [PATCH bpf-next v1 03/15] bpf: Allow storing PTR_TO_BTF_ID in map Date: Sun, 20 Feb 2022 19:18:01 +0530 Message-Id: <20220220134813.3411982-4-memxor@gmail.com> X-Mailer: git-send-email 2.35.1 In-Reply-To: <20220220134813.3411982-1-memxor@gmail.com> References: <20220220134813.3411982-1-memxor@gmail.com> MIME-Version: 1.0 X-Developer-Signature: v=1; a=openpgp-sha256; l=23122; h=from:subject; bh=Nb8zX0dVk97u11szAu1GlZRnuc4N89+pLNYcp4j10s4=; b=owEBbQKS/ZANAwAIAUzgyIZIvxHKAcsmYgBiEkZXQMiMs7iuzB4JYSESy0NQTkyUd6JOnePf0E0E W3g6Im+JAjMEAAEIAB0WIQRLvip+Buz51YI8YRFM4MiGSL8RygUCYhJGVwAKCRBM4MiGSL8RyiHUD/ 9n6kpwLhIHGbeTy8TFGprPpkGWJpHhirh87uKHU1HoinLOQspJ7yuw8xTM5kYISyVVK+ZiOpLvjrnF S/rVBTo+rDzwdH1yR2134bNAeZGaFLmNz0Ya0iY19ZN74UDkEXNKbMLButkeaJUMnEZe6i1JO+vClm E1wcPFMN2rhbrMFcrNY+5PpKfr2+hDBP6h4uipI5pC64uuccnbK5lOE25Kr9a+/ygLuiUjkM95akXg jzrlLRNH8tverRbt/YApGurALEEAqOLhHKs8UZGZW2uxtdy3nJ08B++Yf5V4GaOwuURGPLlXINQpJE FAbGunsJKZ1tW5CVFOUDjHiJdgGcNXRG/vDGIUXFXVFWaeDfySHfSiYaGNrZ/evnTRxCgRoEu3268x vRyQchrgTepdGCQTnXHaWrX4G6NSeZzxWgD8CkxZA22E1pd9uL/dgA6vmFrWFrTx+0XC9EYEQvIE+t sqCLe+/ntURIx0+wK1r2sA8NE6U4Pmbem64w8zCB2Fmb3iRbQW7QMNEmLe5f2BzY/NZh2D0u6Q+OqU ruc/I4uC/x68EzHrugLzER0fYxRnNDrI+JCr3TUZDK1bmN/KyNAROjzhi7LIpc2RGg37zyoJqS9ff8 K/3ohuV9+JtSjjqQ/xY1cIFo4DSDytPQY1i71GUnC/hZGZ8rOeHDVeTHiLaw== X-Developer-Key: i=memxor@gmail.com; a=openpgp; fpr=4BBE2A7E06ECF9D5823C61114CE0C88648BF11CA Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org X-Patchwork-Delegate: bpf@iogearbox.net This patch allows user to embed PTR_TO_BTF_ID in map value, such that loading it marks the destination register as having the appropriate register type and such a pointer can be dereferenced like usual PTR_TO_BTF_ID and be passed to various BPF helpers. This feature can be useful to store an object in a map for a long time, and then inspect it later. Since PTR_TO_BTF_ID is safe against invalid access, verifier doesn't need to perform any complex lifetime checks. It can be useful in cases where user already knows pointer will remain valid, so any dereference at a later time (possibly in entirely different BPF program invocation) will yield correct results as far the data read from kernel memory is concerned. Note that it is quite possible such BTF ID pointer is invalid, in this case the verifier's built-in exception handling mechanism where it converts loads into PTR_TO_BTF_ID into PROBE_MEM loads, would handle the invalid case. Next patch which adds referenced PTR_TO_BTF_ID would need to take more care in ensuring a correct value is stored in the BPF map. The user indicates that a certain pointer must be treated as PTR_TO_BTF_ID by using a BTF type tag 'btf_id' on the pointed to type of the pointer. Then, this information is recorded in the object BTF which will be passed into the kernel by way of map's BTF information. The kernel then records the type, and offset of all such pointers, and finds their corresponding built-in kernel type by the name and BTF kind. Later, during verification this information is used that access to such pointers is sized correctly, and done at a proper offset into the map value. Only BPF_LDX, BPF_STX, and BPF_ST with 0 (to denote NULL) are allowed instructions that can access such a pointer. On BPF_LDX, the destination register is updated to be a PTR_TO_BTF_ID, and on BPF_STX, it is checked whether the source register type is same PTR_TO_BTF_ID, and whether the BTF ID (reg->btf and reg->btf_id) matches the type specified in the map value's definition. Hence, the verifier allows flexible access to kernel data across program invocations in a type safe manner, without compromising on the runtime safety of the kernel. Next patch will extend this support to referenced PTR_TO_BTF_ID. Signed-off-by: Kumar Kartikeya Dwivedi --- include/linux/bpf.h | 30 +++++++- include/linux/btf.h | 3 + kernel/bpf/btf.c | 127 ++++++++++++++++++++++++++++++++++ kernel/bpf/map_in_map.c | 5 +- kernel/bpf/syscall.c | 137 ++++++++++++++++++++++++++++++++++++- kernel/bpf/verifier.c | 148 ++++++++++++++++++++++++++++++++++++++++ 6 files changed, 446 insertions(+), 4 deletions(-) diff --git a/include/linux/bpf.h b/include/linux/bpf.h index f19abc59b6cd..ce45ffb79f82 100644 --- a/include/linux/bpf.h +++ b/include/linux/bpf.h @@ -155,6 +155,23 @@ struct bpf_map_ops { const struct bpf_iter_seq_info *iter_seq_info; }; +enum { + /* Support at most 8 pointers in a BPF map value */ + BPF_MAP_VALUE_OFF_MAX = 8, +}; + +struct bpf_map_value_off_desc { + u32 offset; + u32 btf_id; + struct btf *btf; + struct module *module; +}; + +struct bpf_map_value_off { + u32 nr_off; + struct bpf_map_value_off_desc off[]; +}; + struct bpf_map { /* The first two cachelines with read-mostly members of which some * are also accessed in fast-path (e.g. ops, max_entries). @@ -171,6 +188,7 @@ struct bpf_map { u64 map_extra; /* any per-map-type extra fields */ u32 map_flags; int spin_lock_off; /* >=0 valid offset, <0 error */ + struct bpf_map_value_off *ptr_off_tab; int timer_off; /* >=0 valid offset, <0 error */ u32 id; int numa_node; @@ -184,7 +202,7 @@ struct bpf_map { char name[BPF_OBJ_NAME_LEN]; bool bypass_spec_v1; bool frozen; /* write-once; write-protected by freeze_mutex */ - /* 14 bytes hole */ + /* 6 bytes hole */ /* The 3rd and 4th cacheline with misc members to avoid false sharing * particularly with refcounting. @@ -217,6 +235,11 @@ static inline bool map_value_has_timer(const struct bpf_map *map) return map->timer_off >= 0; } +static inline bool map_value_has_ptr_to_btf_id(const struct bpf_map *map) +{ + return !IS_ERR_OR_NULL(map->ptr_off_tab); +} + static inline void check_and_init_map_value(struct bpf_map *map, void *dst) { if (unlikely(map_value_has_spin_lock(map))) @@ -1490,6 +1513,11 @@ void bpf_prog_put(struct bpf_prog *prog); void bpf_prog_free_id(struct bpf_prog *prog, bool do_idr_lock); void bpf_map_free_id(struct bpf_map *map, bool do_idr_lock); +struct bpf_map_value_off_desc *bpf_map_ptr_off_contains(struct bpf_map *map, u32 offset); +void bpf_map_free_ptr_off_tab(struct bpf_map *map); +struct bpf_map_value_off *bpf_map_copy_ptr_off_tab(const struct bpf_map *map); +bool bpf_map_equal_ptr_off_tab(const struct bpf_map *map_a, const struct bpf_map *map_b); + struct bpf_map *bpf_map_get(u32 ufd); struct bpf_map *bpf_map_get_with_uref(u32 ufd); struct bpf_map *__bpf_map_get(struct fd f); diff --git a/include/linux/btf.h b/include/linux/btf.h index 36bc09b8e890..6592183aeb23 100644 --- a/include/linux/btf.h +++ b/include/linux/btf.h @@ -26,6 +26,7 @@ struct btf_type; union bpf_attr; struct btf_show; struct btf_id_set; +struct bpf_map; struct btf_kfunc_id_set { struct module *owner; @@ -123,6 +124,8 @@ bool btf_member_is_reg_int(const struct btf *btf, const struct btf_type *s, u32 expected_offset, u32 expected_size); int btf_find_spin_lock(const struct btf *btf, const struct btf_type *t); int btf_find_timer(const struct btf *btf, const struct btf_type *t); +int btf_find_ptr_to_btf_id(const struct btf *btf, const struct btf_type *t, + struct bpf_map *map); bool btf_type_is_void(const struct btf_type *t); s32 btf_find_by_name_kind(const struct btf *btf, const char *name, u8 kind); const struct btf_type *btf_type_skip_modifiers(const struct btf *btf, diff --git a/kernel/bpf/btf.c b/kernel/bpf/btf.c index 55f6ccac3388..1edb5710e155 100644 --- a/kernel/bpf/btf.c +++ b/kernel/bpf/btf.c @@ -3122,6 +3122,7 @@ static void btf_struct_log(struct btf_verifier_env *env, enum { BTF_FIELD_SPIN_LOCK, BTF_FIELD_TIMER, + BTF_FIELD_KPTR, }; static int btf_find_field_struct(const struct btf *btf, const struct btf_type *t, @@ -3140,6 +3141,106 @@ static int btf_find_field_struct(const struct btf *btf, const struct btf_type *t return 0; } +static s32 btf_find_by_name_kind_all(const char *name, u32 kind, struct btf **btfp); + +static int btf_find_field_kptr(const struct btf *btf, const struct btf_type *t, + u32 off, int sz, void *data) +{ + struct bpf_map_value_off *tab; + struct bpf_map *map = data; + struct module *mod = NULL; + bool btf_id_tag = false; + struct btf *kernel_btf; + int nr_off, ret; + s32 id; + + /* For PTR, sz is always == 8 */ + if (!btf_type_is_ptr(t)) + return 0; + t = btf_type_by_id(btf, t->type); + + while (btf_type_is_type_tag(t)) { + if (!strcmp("kernel.bpf.btf_id", __btf_name_by_offset(btf, t->name_off))) { + /* repeated tag */ + if (btf_id_tag) { + ret = -EINVAL; + goto end; + } + btf_id_tag = true; + } else if (!strncmp("kernel.", __btf_name_by_offset(btf, t->name_off), + sizeof("kernel.") - 1)) { + /* TODO: Should we reject these when loading BTF? */ + /* Unavailable tag in reserved tag namespace */ + ret = -EACCES; + goto end; + } + /* Look for next tag */ + t = btf_type_by_id(btf, t->type); + } + if (!btf_id_tag) + return 0; + + /* Get the base type */ + if (btf_type_is_modifier(t)) + t = btf_type_skip_modifiers(btf, t->type, NULL); + /* Only pointer to struct is allowed */ + if (!__btf_type_is_struct(t)) { + ret = -EINVAL; + goto end; + } + + id = btf_find_by_name_kind_all(__btf_name_by_offset(btf, t->name_off), + BTF_INFO_KIND(t->info), &kernel_btf); + if (id < 0) { + ret = id; + goto end; + } + + nr_off = map->ptr_off_tab ? map->ptr_off_tab->nr_off : 0; + if (nr_off == BPF_MAP_VALUE_OFF_MAX) { + ret = -E2BIG; + goto end_btf; + } + + tab = krealloc(map->ptr_off_tab, offsetof(struct bpf_map_value_off, off[nr_off + 1]), + GFP_KERNEL | __GFP_NOWARN); + if (!tab) { + ret = -ENOMEM; + goto end_btf; + } + /* Initialize nr_off for newly allocated ptr_off_tab */ + if (!map->ptr_off_tab) + tab->nr_off = 0; + map->ptr_off_tab = tab; + + /* We take reference to make sure valid pointers into module data don't + * become invalid across program invocation. + */ + if (btf_is_module(kernel_btf)) { + mod = btf_try_get_module(kernel_btf); + if (!mod) { + ret = -ENXIO; + goto end_btf; + } + } + + tab->off[nr_off].offset = off; + tab->off[nr_off].btf_id = id; + tab->off[nr_off].btf = kernel_btf; + tab->off[nr_off].module = mod; + tab->nr_off++; + + return 0; +end_btf: + /* Reference is only raised for module BTF */ + if (btf_is_module(kernel_btf)) + btf_put(kernel_btf); +end: + bpf_map_free_ptr_off_tab(map); + map->ptr_off_tab = ERR_PTR(ret); + return ret; +} + static int btf_find_struct_field(const struct btf *btf, const struct btf_type *t, const char *name, int sz, int align, int field_type, void *data) @@ -3170,6 +3271,11 @@ static int btf_find_struct_field(const struct btf *btf, const struct btf_type *t if (ret < 0) return ret; break; + case BTF_FIELD_KPTR: + ret = btf_find_field_kptr(btf, member_type, off, sz, data); + if (ret < 0) + return ret; + break; default: pr_err("verifier bug: unknown field type requested\n"); return -EFAULT; @@ -3206,6 +3312,11 @@ static int btf_find_datasec_var(const struct btf *btf, const struct btf_type *t, if (ret < 0) return ret; break; + case BTF_FIELD_KPTR: + ret = btf_find_field_kptr(btf, var_type, off, sz, data); + if (ret < 0) + return ret; + break; default: return -EFAULT; } @@ -3256,6 +3367,22 @@ int btf_find_timer(const struct btf *btf, const struct btf_type *t) return off; } +int btf_find_ptr_to_btf_id(const struct btf *btf, const struct btf_type *t, + struct bpf_map *map) +{ + int ret; + + ret = btf_find_field(btf, t, NULL, sizeof(u64), __alignof__(u64), + BTF_FIELD_KPTR, map); + /* While btf_find_field_kptr cleans up after itself, later iterations + * can still return error without calling it, so call free function + * again. + */ + if (ret < 0) + bpf_map_free_ptr_off_tab(map); + return ret; +} + static void __btf_struct_show(const struct btf *btf, const struct btf_type *t, u32 type_id, void *data, u8 bits_offset, struct btf_show *show) diff --git a/kernel/bpf/map_in_map.c b/kernel/bpf/map_in_map.c index 5cd8f5277279..293e41a4f0b3 100644 --- a/kernel/bpf/map_in_map.c +++ b/kernel/bpf/map_in_map.c @@ -52,6 +52,7 @@ struct bpf_map *bpf_map_meta_alloc(int inner_map_ufd) inner_map_meta->max_entries = inner_map->max_entries; inner_map_meta->spin_lock_off = inner_map->spin_lock_off; inner_map_meta->timer_off = inner_map->timer_off; + inner_map_meta->ptr_off_tab = bpf_map_copy_ptr_off_tab(inner_map); if (inner_map->btf) { btf_get(inner_map->btf); inner_map_meta->btf = inner_map->btf; @@ -71,6 +72,7 @@ struct bpf_map *bpf_map_meta_alloc(int inner_map_ufd) void bpf_map_meta_free(struct bpf_map *map_meta) { + bpf_map_free_ptr_off_tab(map_meta); btf_put(map_meta->btf); kfree(map_meta); } @@ -83,7 +85,8 @@ bool bpf_map_meta_equal(const struct bpf_map *meta0, meta0->key_size == meta1->key_size && meta0->value_size == meta1->value_size && meta0->timer_off == meta1->timer_off && - meta0->map_flags == meta1->map_flags; + meta0->map_flags == meta1->map_flags && + bpf_map_equal_ptr_off_tab(meta0, meta1); } void *bpf_map_fd_get_ptr(struct bpf_map *map, diff --git a/kernel/bpf/syscall.c b/kernel/bpf/syscall.c index 9c7a72b65eee..beb96866f34d 100644 --- a/kernel/bpf/syscall.c +++ b/kernel/bpf/syscall.c @@ -6,6 +6,7 @@ #include #include #include +#include #include #include #include @@ -472,12 +473,123 @@ static void bpf_map_release_memcg(struct bpf_map *map) } #endif +static int bpf_map_ptr_off_cmp(const void *a, const void *b) +{ + const struct bpf_map_value_off_desc *off_desc1 = a, *off_desc2 = b; + + if (off_desc1->offset < off_desc2->offset) + return -1; + else if (off_desc1->offset > off_desc2->offset) + return 1; + return 0; +} + +struct bpf_map_value_off_desc *bpf_map_ptr_off_contains(struct bpf_map *map, u32 offset) +{ + /* Since members are iterated in btf_find_field in increasing order, + * offsets appended to ptr_off_tab are in increasing order, so we can + * do bsearch to find exact match. + */ + struct bpf_map_value_off *tab; + + if (!map_value_has_ptr_to_btf_id(map)) + return NULL; + tab = map->ptr_off_tab; + return bsearch(&offset, tab->off, tab->nr_off, sizeof(tab->off[0]), bpf_map_ptr_off_cmp); +} + +void bpf_map_free_ptr_off_tab(struct bpf_map *map) +{ + struct bpf_map_value_off *tab = map->ptr_off_tab; + int i; + + if (IS_ERR_OR_NULL(tab)) + return; + for (i = 0; i < tab->nr_off; i++) { + struct module *mod = tab->off[i].module; + struct btf *btf = tab->off[i].btf; + + /* off[i].btf is obtained from bpf_btf_find_by_name_kind_all, + * which only takes reference for module BTF, not vmlinux BTF. + */ + if (btf_is_module(btf)) { + module_put(mod); + btf_put(btf); + } + } + kfree(tab); + map->ptr_off_tab = NULL; +} + +struct bpf_map_value_off *bpf_map_copy_ptr_off_tab(const struct bpf_map *map) +{ + struct bpf_map_value_off *tab = map->ptr_off_tab, *new_tab; + int size, i, ret; + + if (IS_ERR_OR_NULL(tab)) + return tab; + /* Increment references that we have to transfer into the new + * ptr_off_tab. + */ + for (i = 0; i < tab->nr_off; i++) { + struct btf *btf = tab->off[i].btf; + + if (btf_is_module(btf)) { + if (!btf_try_get_module(btf)) { + ret = -ENXIO; + /* No references for off_desc at index 'i' have + * been taken at this point, so the cleanup loop + * at 'end' will start releasing from previous + * index. + */ + goto end; + } + btf_get(btf); + } + } + + size = offsetof(struct bpf_map_value_off, off[tab->nr_off]); + new_tab = kmalloc(size, GFP_KERNEL | __GFP_NOWARN); + if (!new_tab) { + ret = -ENOMEM; + goto end; + } + memcpy(new_tab, tab, size); + return new_tab; +end: + while (i--) { + if (btf_is_module(tab->off[i].btf)) { + module_put(tab->off[i].module); + btf_put(tab->off[i].btf); + } + } + return ERR_PTR(ret); +} + +bool bpf_map_equal_ptr_off_tab(const struct bpf_map *map_a, const struct bpf_map *map_b) +{ + struct bpf_map_value_off *tab_a = map_a->ptr_off_tab, *tab_b = map_b->ptr_off_tab; + int size; + + if (IS_ERR(tab_a) || IS_ERR(tab_b)) + return false; + if (!tab_a && !tab_b) + return true; + if ((!tab_a && tab_b) || (tab_a && !tab_b)) + return false; + if (tab_a->nr_off != tab_b->nr_off) + return false; + size = offsetof(struct bpf_map_value_off, off[tab_a->nr_off]); + return !memcmp(tab_a, tab_b, size); +} + /* called from workqueue */ static void bpf_map_free_deferred(struct work_struct *work) { struct bpf_map *map = container_of(work, struct bpf_map, work); security_bpf_map_free(map); + bpf_map_free_ptr_off_tab(map); bpf_map_release_memcg(map); /* implementation dependent freeing */ map->ops->map_free(map); @@ -639,7 +751,7 @@ static int bpf_map_mmap(struct file *filp, struct vm_area_struct *vma) int err; if (!map->ops->map_mmap || map_value_has_spin_lock(map) || - map_value_has_timer(map)) + map_value_has_timer(map) || map_value_has_ptr_to_btf_id(map)) return -ENOTSUPP; if (!(vma->vm_flags & VM_SHARED)) @@ -819,9 +931,30 @@ static int map_check_btf(struct bpf_map *map, const struct btf *btf, return -EOPNOTSUPP; } - if (map->ops->map_check_btf) + /* We can ignore the return value */ + btf_find_ptr_to_btf_id(btf, value_type, map); + if (map_value_has_ptr_to_btf_id(map)) { + if (map->map_flags & BPF_F_RDONLY_PROG) { + ret = -EACCES; + goto free_map_tab; + } + if (map->map_type != BPF_MAP_TYPE_HASH && + map->map_type != BPF_MAP_TYPE_LRU_HASH && + map->map_type != BPF_MAP_TYPE_ARRAY) { + ret = -EOPNOTSUPP; + goto free_map_tab; + } + } + + if (map->ops->map_check_btf) { ret = map->ops->map_check_btf(map, btf, key_type, value_type); + if (ret < 0) + goto free_map_tab; + } + return ret; +free_map_tab: + bpf_map_free_ptr_off_tab(map); return ret; } diff --git a/kernel/bpf/verifier.c b/kernel/bpf/verifier.c index d7473fee247c..1ffefddebaea 100644 --- a/kernel/bpf/verifier.c +++ b/kernel/bpf/verifier.c @@ -3465,6 +3465,118 @@ static int check_mem_region_access(struct bpf_verifier_env *env, u32 regno, return 0; } +static int map_ptr_to_btf_id_match_type(struct bpf_verifier_env *env, + struct bpf_map_value_off_desc *off_desc, + struct bpf_reg_state *reg, u32 regno) +{ + const char *targ_name = kernel_type_name(off_desc->btf, off_desc->btf_id); + const char *reg_name = ""; + + if (reg->type != PTR_TO_BTF_ID && reg->type != PTR_TO_BTF_ID_OR_NULL) + goto end; + + if (!btf_is_kernel(reg->btf)) { + verbose(env, "R%d must point to kernel BTF\n", regno); + return -EINVAL; + } + /* We need to verify reg->type and reg->btf, before accessing reg->btf */ + reg_name = kernel_type_name(reg->btf, reg->btf_id); + + if (reg->off < 0) { + verbose(env, + "R%d is ptr_%s invalid negative access: off=%d\n", + regno, reg_name, reg->off); + return -EINVAL; + } + + if (!tnum_is_const(reg->var_off) || reg->var_off.value) { + char tn_buf[48]; + + tnum_strn(tn_buf, sizeof(tn_buf), reg->var_off); + verbose(env, + "R%d is ptr_%s invalid variable offset: off=%d, var_off=%s\n", + regno, reg_name, reg->off, tn_buf); + return -EINVAL; + } + + if (!btf_struct_ids_match(&env->log, reg->btf, reg->btf_id, reg->off, + off_desc->btf, off_desc->btf_id)) + goto end; + return 0; +end: + verbose(env, "invalid btf_id pointer access, R%d type=%s%s ", regno, + reg_type_str(env, reg->type), reg_name); + verbose(env, "expected=%s%s\n", reg_type_str(env, PTR_TO_BTF_ID), targ_name); + return -EINVAL; +} + +/* Returns an error, or 0 if ignoring the access, or 1 if register state was + * updated, in which case later updates must be skipped. + */ +static int check_map_ptr_to_btf_id(struct bpf_verifier_env *env, u32 regno, int off, int size, + int value_regno, enum bpf_access_type t, int insn_idx) +{ + struct bpf_reg_state *reg = reg_state(env, regno), *val_reg; + struct bpf_insn *insn = &env->prog->insnsi[insn_idx]; + struct bpf_map_value_off_desc *off_desc; + int insn_class = BPF_CLASS(insn->code); + struct bpf_map *map = reg->map_ptr; + + /* Things we already checked for in check_map_access: + * - Reject cases where variable offset may touch BTF ID pointer + * - size of access (must be BPF_DW) + * - off_desc->offset == off + reg->var_off.value + */ + if (!tnum_is_const(reg->var_off)) + return 0; + + off_desc = bpf_map_ptr_off_contains(map, off + reg->var_off.value); + if (!off_desc) + return 0; + + if (WARN_ON_ONCE(size != bpf_size_to_bytes(BPF_DW))) + return -EACCES; + + if (BPF_MODE(insn->code) != BPF_MEM) + goto end; + + if (!env->bpf_capable) { + verbose(env, "btf_id pointer in map only allowed for CAP_BPF and CAP_SYS_ADMIN\n"); + return -EPERM; + } + + if (insn_class == BPF_LDX) { + if (WARN_ON_ONCE(value_regno < 0)) + return -EFAULT; + val_reg = reg_state(env, value_regno); + /* We can simply mark the value_regno receiving the pointer + * value from map as PTR_TO_BTF_ID, with the correct type. + */ + mark_btf_ld_reg(env, cur_regs(env), value_regno, PTR_TO_BTF_ID, off_desc->btf, + off_desc->btf_id, PTR_MAYBE_NULL); + val_reg->id = ++env->id_gen; + } else if (insn_class == BPF_STX) { + if (WARN_ON_ONCE(value_regno < 0)) + return -EFAULT; + val_reg = reg_state(env, value_regno); + if (!register_is_null(val_reg) && + map_ptr_to_btf_id_match_type(env, off_desc, val_reg, value_regno)) + return -EACCES; + } else if (insn_class == BPF_ST) { + if (insn->imm) { + verbose(env, "BPF_ST imm must be 0 when writing to btf_id pointer at off=%u\n", + off_desc->offset); + return -EACCES; + } + } else { + goto end; + } + return 1; +end: + verbose(env, "btf_id pointer in map can only be accessed using BPF_LDX/BPF_STX/BPF_ST\n"); + return -EACCES; +} + /* check read/write into a map element with possible variable offset */ static int check_map_access(struct bpf_verifier_env *env, u32 regno, int off, int size, bool zero_size_allowed) @@ -3503,6 +3615,36 @@ static int check_map_access(struct bpf_verifier_env *env, u32 regno, return -EACCES; } } + if (map_value_has_ptr_to_btf_id(map)) { + struct bpf_map_value_off *tab = map->ptr_off_tab; + bool known_off = tnum_is_const(reg->var_off); + int i; + + for (i = 0; i < tab->nr_off; i++) { + u32 p = tab->off[i].offset; + + if (reg->smin_value + off < p + sizeof(u64) && + p < reg->umax_value + off + size) { + if (!known_off) { + verbose(env, "btf_id pointer cannot be accessed by variable offset load/store\n"); + return -EACCES; + } + if (p != off + reg->var_off.value) { + verbose(env, "btf_id pointer offset incorrect\n"); + return -EACCES; + } + if (size != sizeof(u64)) { + verbose(env, "btf_id pointer load/store size must be 8\n"); + return -EACCES; + } + break; + } + } + } else if (IS_ERR(map->ptr_off_tab)) { + /* Reject program using map with incorrectly tagged btf_id pointer */ + verbose(env, "invalid btf_id pointer tagging in map value\n"); + return PTR_ERR(map->ptr_off_tab); + } return err; } @@ -4404,6 +4546,10 @@ static int check_mem_access(struct bpf_verifier_env *env, int insn_idx, u32 regn if (err) return err; err = check_map_access(env, regno, off, size, false); + if (!err) + err = check_map_ptr_to_btf_id(env, regno, off, size, value_regno, + t, insn_idx); + /* if err == 0, check_map_ptr_to_btf_id ignored the access */ if (!err && t == BPF_READ && value_regno >= 0) { struct bpf_map *map = reg->map_ptr; @@ -4425,6 +4571,8 @@ static int check_mem_access(struct bpf_verifier_env *env, int insn_idx, u32 regn mark_reg_unknown(env, regs, value_regno); } } + if (err == 1) + err = 0; } else if (base_type(reg->type) == PTR_TO_MEM) { bool rdonly_mem = type_is_rdonly_mem(reg->type); From patchwork Sun Feb 20 13:48:02 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kumar Kartikeya Dwivedi X-Patchwork-Id: 12752707 X-Patchwork-Delegate: bpf@iogearbox.net Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id B3751C433FE for ; Sun, 20 Feb 2022 13:48:40 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S243881AbiBTNs7 (ORCPT ); Sun, 20 Feb 2022 08:48:59 -0500 Received: from mxb-00190b01.gslb.pphosted.com ([23.128.96.19]:49010 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S243855AbiBTNsv (ORCPT ); Sun, 20 Feb 2022 08:48:51 -0500 Received: from mail-pj1-x1041.google.com (mail-pj1-x1041.google.com [IPv6:2607:f8b0:4864:20::1041]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id BD71A2D1E8; Sun, 20 Feb 2022 05:48:29 -0800 (PST) Received: by mail-pj1-x1041.google.com with SMTP id v13-20020a17090ac90d00b001b87bc106bdso16509817pjt.4; Sun, 20 Feb 2022 05:48:29 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=u553XL18yly1/cCm3ekwYOyq08NaxGCEKxwUe2SELgg=; b=Kqmh7FesAY7zCexo1xKseWmIO7Umn7hcZPO9WhTV/zokXk3BXMhUOnYKZ3aFkJARF+ MUaJcmkJAUZdklrZ61bBLg5kgr6x+lsJzzJJqa/sSzkFPjPXEcj4xQ1dxk6S4zzev9TA vGsje+A4Rh6iWk4+JB1o1lKLhKZDnj2/fFm4n5JFFwBxajwSyNwqu97dUmU1nmJ3QnMj Grfn+sKqEIht8uRsWC/c4P2A0utNKVeuywslZynWP84Vsx6g/o7zfioFeLPv8sbsnm17 mYyk4WnOWoqCexr8YZZlwFyVAVPJGXawdc4TFW9I8UWljus6R3DCNGdiwvWobfzupym4 emlA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=u553XL18yly1/cCm3ekwYOyq08NaxGCEKxwUe2SELgg=; b=25yY5y0tQjYtOC9d53515HtfKa+rMbssntnrKW2QrPxEawZoRuMNoqxZ2K2xkyiYtZ UMY1MYE7rqZrcNLsnlkFMi6skgQdf2ftBAx3Amuvvd/3UjirVetmMp70cuxOevPUNQNk IJOfe/R/atEDIWwxdwJ0J3LOLq4bjIzWb0QHnmAosO0voSZJ2lDookpUn3mVEFVPcCRb o5EdK7NOHB5gwVhVx1DVOMQgyhnAMJ6ZwqpRXsjUdy9SoO3Y0PMRR6+bNZtS++ZvApb+ mPYroyjZBb3J47BH0rfCkRQw9S/8OpjrFmXVC8v8xo/LikpHl/Wej8bDQRUEbTMYjF+a tC+w== X-Gm-Message-State: AOAM531+qWtdyfQTBRANw1d5fx/hldvBgDYhE6EOxMDWGT/9PP+w3ZUO 33Z4WoXhhJucJIIWnDxsWfcuTaMOl18= X-Google-Smtp-Source: ABdhPJyKVuhjcQR/fifASHAAq8Ew9Nt/fQs2IXtaHvLobSmnSU7rgQ9sDSkwpOjNlZGorzyP0JXnUQ== X-Received: by 2002:a17:903:228a:b0:14d:aa04:2278 with SMTP id b10-20020a170903228a00b0014daa042278mr15397892plh.58.1645364908891; Sun, 20 Feb 2022 05:48:28 -0800 (PST) Received: from localhost ([2405:201:6014:d0c0:6243:316e:a9e1:adda]) by smtp.gmail.com with ESMTPSA id k4sm9620741pff.39.2022.02.20.05.48.28 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 20 Feb 2022 05:48:28 -0800 (PST) From: Kumar Kartikeya Dwivedi To: bpf@vger.kernel.org Cc: Alexei Starovoitov , Andrii Nakryiko , Daniel Borkmann , =?utf-8?q?Toke_H=C3=B8iland-J=C3=B8?= =?utf-8?q?rgensen?= , Jesper Dangaard Brouer , netfilter-devel@vger.kernel.org, netdev@vger.kernel.org Subject: [PATCH bpf-next v1 04/15] bpf: Allow storing referenced PTR_TO_BTF_ID in map Date: Sun, 20 Feb 2022 19:18:02 +0530 Message-Id: <20220220134813.3411982-5-memxor@gmail.com> X-Mailer: git-send-email 2.35.1 In-Reply-To: <20220220134813.3411982-1-memxor@gmail.com> References: <20220220134813.3411982-1-memxor@gmail.com> MIME-Version: 1.0 X-Developer-Signature: v=1; a=openpgp-sha256; l=17467; h=from:subject; bh=eyWdpQxuVOys9KdWZkDYB65kBnBE5WEaZrsN5VgWjY4=; b=owEBbQKS/ZANAwAIAUzgyIZIvxHKAcsmYgBiEkZXYKi7Qa/9mPx6fXKfGyg53rEu0MeITQLXkD3T a9ftsTCJAjMEAAEIAB0WIQRLvip+Buz51YI8YRFM4MiGSL8RygUCYhJGVwAKCRBM4MiGSL8RymzpEA CIGRKmAVSYEhzVr1RzollinmCl08tjkSdzv9zrqgh4G8hC+SRCMHRSDnLp5bgeB3uGo44VphuIAD5j CuHr1ylrGAWr+jHelGr2Gk8Dy5eR99EGaXaHGDQxsTA2oG88g/+bl6fCN9pjRjD/b3n/lMmuzoy4Bk /54xCDkObSFi288JZTBpyfesRz5MK2FNeLYjsu3wbO20LZoRNWxLyozMpnOqF0LpwQHmruBNKUVQKf dQlBKyJP8n6o9kEFL/WAFaW+aZ6KiJyojiiDykV1P7ZoMbW/gogRlBXkyIMlGY7RDfDfDcuATnFilF 9ArwFoKB2M+PQ6x1bsEwXolWI5th1FxHawUT8U0sWZwWuDItcQyYv/CdUaLcdqVSMf1mSeAhYqRjAx C68KkWvnupKFFLC3cUett5Rym5nWrUqfZ2pukPVDUpXWihe0wkXMByd+9el+dx9ygaAUcwigiOKJa4 eXmSx/1UrdBVJmUmklnX2QCVJ/wYzsiBCGk8PugwrRs9/e9cMbH9FqrrBnIfJ7Vy+FfLyVwOe7UF30 sUXF92Sd9KVK9LhUeVglHWaCBr79Uc2OfG0k7Wk/XzgfYtuEFDnr0HZNkb0ignEWESNsDyJhCl3u/f UMEZLnv+wQBLlt8Tw8N9T3BsHtxmMmR1syY9LkOYzutljP5TY3OH86RyIyxQ== X-Developer-Key: i=memxor@gmail.com; a=openpgp; fpr=4BBE2A7E06ECF9D5823C61114CE0C88648BF11CA Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org X-Patchwork-Delegate: bpf@iogearbox.net This commit enables storing referenced PTR_TO_BTF_ID pointers in maps, with some restrictions to ensure the value of the pointer remains consistent, as it needs to be eventuall freed using a release function, either by the BPF program itself in a later invocation, or by the map's free path. Such a pointer must be tagged using both 'btf_id' and 'ref' type tags on the type being pointed to by the pointer, in the map value's BTF. The verifier will only permit updating such pointers using BPF_XCHG instruction. There are valid reasons to enforce this restriction: - The pointer value must be consistent in face of concurrent modification, and any prior values contained in the map must also be released before a new one is moved into the map. To ensure proper transfer of this ownership, BPF_XCHG returns the old value, which the verifier would require the user to either free or move into another map, and released the reference held for the pointer being moved in. Hence, after a BPF_XCHG on a map value, the user releases the reference for the pointer they are moving in, and acquires a reference for the old pointer returned by the exchange instruction (in src_reg). In case of unreferenced PTR_TO_BTF_ID in maps, losing the old value by a store had no adverse effect. The only thing we need to ensure is that the actual word sized pointer is written and read concurrently using word sized instructions on the actual architecture, even if BPF ABI has 64-bit pointers, the underlying pointer value on 32-bit systems will be 32-bit, so emitting a load and store as two 32-bit sized loads and stores would still be valid, however doing the same on a 64-bit system would be wrong, as the pointer value being read can be inconsistent. This is because while pointer dereference inside a BPF program is concerned, the verifier patches loads to PROBE_MEM loads, which support exception handling of faulting loads, but PTR_TO_BTF_ID can also be passed into BPF helpers and kernel functions, which do not have the same liberty. - This also ensures that BPF programs executing concurrently never end up in a state where a certain pointer value was lost due to manipulations of the map value, thus leaking the reference that was moved in. There is always an entity which releases the reference eventually, it will either be the map's free path (which will detect and release live pointers in the map), or the BPF program itself, which can exchange a referenced pointer with NULL and free the old reference. - In case of multiple such pointers, doing many BPF_XCHG can be a bit costly, especially if those pointers are already protected by a BPF spin lock against concurrent modification. In the future, this support can be extended so that a single spin lock protects multiple such pointers and this move operation can be enforced using a helper, while also ensuring linearizability of the pointer move operations. This will amortize the cost of each individual BPF_XCHG that would be needed otherwise using a spin lock. However, this is work that has been left as an exercise for a future patch. This mechanism would also require the user to indicate to the verifier which members of the map value are protected by the BPF spin lock, by using annotation in map's BTF. Signed-off-by: Kumar Kartikeya Dwivedi --- include/linux/bpf.h | 5 ++ kernel/bpf/btf.c | 22 +++++++- kernel/bpf/verifier.c | 117 ++++++++++++++++++++++++++++++++++++------ 3 files changed, 126 insertions(+), 18 deletions(-) diff --git a/include/linux/bpf.h b/include/linux/bpf.h index ce45ffb79f82..923b9f36c275 100644 --- a/include/linux/bpf.h +++ b/include/linux/bpf.h @@ -160,11 +160,16 @@ enum { BPF_MAP_VALUE_OFF_MAX = 8, }; +enum { + BPF_MAP_VALUE_OFF_F_REF = (1U << 0), +}; + struct bpf_map_value_off_desc { u32 offset; u32 btf_id; struct btf *btf; struct module *module; + int flags; }; struct bpf_map_value_off { diff --git a/kernel/bpf/btf.c b/kernel/bpf/btf.c index 1edb5710e155..20124f4a421c 100644 --- a/kernel/bpf/btf.c +++ b/kernel/bpf/btf.c @@ -3146,10 +3146,10 @@ static s32 btf_find_by_name_kind_all(const char *name, u32 kind, struct btf **bt static int btf_find_field_kptr(const struct btf *btf, const struct btf_type *t, u32 off, int sz, void *data) { + bool btf_id_tag = false, ref_tag = false; struct bpf_map_value_off *tab; struct bpf_map *map = data; struct module *mod = NULL; - bool btf_id_tag = false; struct btf *kernel_btf; int nr_off, ret; s32 id; @@ -3167,6 +3167,13 @@ static int btf_find_field_kptr(const struct btf *btf, const struct btf_type *t, goto end; } btf_id_tag = true; + } else if (!strcmp("kernel.bpf.ref", __btf_name_by_offset(btf, t->name_off))) { + /* repeated tag */ + if (ref_tag) { + ret = -EINVAL; + goto end; + } + ref_tag = true; } else if (!strncmp("kernel.", __btf_name_by_offset(btf, t->name_off), sizeof("kernel.") - 1)) { /* TODO: Should we reject these when loading BTF? */ @@ -3177,8 +3184,14 @@ static int btf_find_field_kptr(const struct btf *btf, const struct btf_type *t, /* Look for next tag */ t = btf_type_by_id(btf, t->type); } - if (!btf_id_tag) + if (!btf_id_tag) { + /* 'ref' tag must be specified together with 'btf_id' tag */ + if (ref_tag) { + ret = -EINVAL; + goto end; + } return 0; + } /* Get the base type */ if (btf_type_is_modifier(t)) @@ -3215,6 +3228,10 @@ static int btf_find_field_kptr(const struct btf *btf, const struct btf_type *t, /* We take reference to make sure valid pointers into module data don't * become invalid across program invocation. + * + * We also need to hold a reference to the module, which corresponds to + * the referenced type, as it has the destructor function we need to + * call when map goes away and a live pointer exists at offset. */ if (btf_is_module(kernel_btf)) { mod = btf_try_get_module(kernel_btf); @@ -3228,6 +3245,7 @@ static int btf_find_field_kptr(const struct btf *btf, const struct btf_type *t, tab->off[nr_off].btf_id = id; tab->off[nr_off].btf = kernel_btf; tab->off[nr_off].module = mod; + tab->off[nr_off].flags = ref_tag ? BPF_MAP_VALUE_OFF_F_REF : 0; tab->nr_off++; return 0; diff --git a/kernel/bpf/verifier.c b/kernel/bpf/verifier.c index 1ffefddebaea..a9d8c0d3c919 100644 --- a/kernel/bpf/verifier.c +++ b/kernel/bpf/verifier.c @@ -521,6 +521,13 @@ static bool is_ptr_cast_function(enum bpf_func_id func_id) func_id == BPF_FUNC_skc_to_tcp_request_sock; } +static bool is_xchg_insn(const struct bpf_insn *insn) +{ + return BPF_CLASS(insn->code) == BPF_STX && + BPF_MODE(insn->code) == BPF_ATOMIC && + insn->imm == BPF_XCHG; +} + static bool is_cmpxchg_insn(const struct bpf_insn *insn) { return BPF_CLASS(insn->code) == BPF_STX && @@ -3467,7 +3474,8 @@ static int check_mem_region_access(struct bpf_verifier_env *env, u32 regno, static int map_ptr_to_btf_id_match_type(struct bpf_verifier_env *env, struct bpf_map_value_off_desc *off_desc, - struct bpf_reg_state *reg, u32 regno) + struct bpf_reg_state *reg, u32 regno, + bool ref_ptr) { const char *targ_name = kernel_type_name(off_desc->btf, off_desc->btf_id); const char *reg_name = ""; @@ -3498,6 +3506,20 @@ static int map_ptr_to_btf_id_match_type(struct bpf_verifier_env *env, regno, reg_name, reg->off, tn_buf); return -EINVAL; } + /* reg->off can be used to store pointer to a certain type formed by + * incrementing pointer of a parent structure the object is embedded in, + * e.g. map may expect unreferenced struct path *, and user should be + * allowed a store using &file->f_path. However, in the case of + * referenced pointer, we cannot do this, because the reference is only + * for the parent structure, not its embedded object(s), and because + * the transfer of ownership happens for the original pointer to and + * from the map (before its eventual release). + */ + if (reg->off && ref_ptr) { + verbose(env, "R%d stored to referenced btf_id pointer cannot have non-zero offset\n", + regno); + return -EINVAL; + } if (!btf_struct_ids_match(&env->log, reg->btf, reg->btf_id, reg->off, off_desc->btf, off_desc->btf_id)) @@ -3510,17 +3532,23 @@ static int map_ptr_to_btf_id_match_type(struct bpf_verifier_env *env, return -EINVAL; } +static int release_reference(struct bpf_verifier_env *env, int ref_obj_id); + /* Returns an error, or 0 if ignoring the access, or 1 if register state was * updated, in which case later updates must be skipped. */ static int check_map_ptr_to_btf_id(struct bpf_verifier_env *env, u32 regno, int off, int size, - int value_regno, enum bpf_access_type t, int insn_idx) + int value_regno, enum bpf_access_type t, int insn_idx, + struct bpf_reg_state *atomic_load_reg) { struct bpf_reg_state *reg = reg_state(env, regno), *val_reg; struct bpf_insn *insn = &env->prog->insnsi[insn_idx]; struct bpf_map_value_off_desc *off_desc; int insn_class = BPF_CLASS(insn->code); struct bpf_map *map = reg->map_ptr; + bool ref_ptr = false; + u32 ref_obj_id = 0; + int ret; /* Things we already checked for in check_map_access: * - Reject cases where variable offset may touch BTF ID pointer @@ -3533,11 +3561,12 @@ static int check_map_ptr_to_btf_id(struct bpf_verifier_env *env, u32 regno, int off_desc = bpf_map_ptr_off_contains(map, off + reg->var_off.value); if (!off_desc) return 0; + ref_ptr = off_desc->flags & BPF_MAP_VALUE_OFF_F_REF; if (WARN_ON_ONCE(size != bpf_size_to_bytes(BPF_DW))) return -EACCES; - if (BPF_MODE(insn->code) != BPF_MEM) + if (BPF_MODE(insn->code) != BPF_MEM && BPF_MODE(insn->code) != BPF_ATOMIC) goto end; if (!env->bpf_capable) { @@ -3545,10 +3574,50 @@ static int check_map_ptr_to_btf_id(struct bpf_verifier_env *env, u32 regno, int return -EPERM; } - if (insn_class == BPF_LDX) { + if (is_xchg_insn(insn)) { + /* We do checks and updates during register fill call for fetch case */ + if (t != BPF_READ || value_regno < 0) + return 1; + val_reg = reg_state(env, value_regno); + if (!register_is_null(atomic_load_reg) && + map_ptr_to_btf_id_match_type(env, off_desc, atomic_load_reg, value_regno, ref_ptr)) + return -EACCES; + /* Acquire new reference state for old pointer, and release + * current reference state for exchanged pointer. + */ + if (ref_ptr) { + if (!register_is_null(atomic_load_reg)) { + if (!atomic_load_reg->ref_obj_id) { + verbose(env, "R%d type=%s%s must be referenced\n", + value_regno, reg_type_str(env, atomic_load_reg->type), + kernel_type_name(reg->btf, reg->btf_id)); + return -EACCES; + } + ret = release_reference(env, atomic_load_reg->ref_obj_id); + if (ret < 0) + return ret; + } + ret = acquire_reference_state(env, insn_idx); + if (ret < 0) + return ret; + ref_obj_id = ret; + } + /* val_reg might be NULL at this point */ + mark_btf_ld_reg(env, cur_regs(env), value_regno, PTR_TO_BTF_ID, off_desc->btf, + off_desc->btf_id, PTR_MAYBE_NULL); + /* __mark_ptr_or_null_regs needs ref_obj_id == id to clear + * reference state for ptr == NULL branch. + */ + val_reg->id = ref_obj_id ?: ++env->id_gen; + val_reg->ref_obj_id = ref_obj_id; + } else if (insn_class == BPF_LDX) { if (WARN_ON_ONCE(value_regno < 0)) return -EFAULT; val_reg = reg_state(env, value_regno); + if (ref_ptr) { + verbose(env, "referenced btf_id pointer can only be accessed using BPF_XCHG\n"); + return -EACCES; + } /* We can simply mark the value_regno receiving the pointer * value from map as PTR_TO_BTF_ID, with the correct type. */ @@ -3559,10 +3628,18 @@ static int check_map_ptr_to_btf_id(struct bpf_verifier_env *env, u32 regno, int if (WARN_ON_ONCE(value_regno < 0)) return -EFAULT; val_reg = reg_state(env, value_regno); + if (ref_ptr) { + verbose(env, "referenced btf_id pointer can only be accessed using BPF_XCHG\n"); + return -EACCES; + } if (!register_is_null(val_reg) && - map_ptr_to_btf_id_match_type(env, off_desc, val_reg, value_regno)) + map_ptr_to_btf_id_match_type(env, off_desc, val_reg, value_regno, false)) return -EACCES; } else if (insn_class == BPF_ST) { + if (ref_ptr) { + verbose(env, "referenced btf_id pointer can only be accessed using BPF_XCHG\n"); + return -EACCES; + } if (insn->imm) { verbose(env, "BPF_ST imm must be 0 when writing to btf_id pointer at off=%u\n", off_desc->offset); @@ -3573,7 +3650,7 @@ static int check_map_ptr_to_btf_id(struct bpf_verifier_env *env, u32 regno, int } return 1; end: - verbose(env, "btf_id pointer in map can only be accessed using BPF_LDX/BPF_STX/BPF_ST\n"); + verbose(env, "btf_id pointer in map can only be accessed using BPF_LDX, BPF_STX, BPF_ST, BPF_XCHG\n"); return -EACCES; } @@ -4505,7 +4582,8 @@ static int check_stack_access_within_bounds( */ static int check_mem_access(struct bpf_verifier_env *env, int insn_idx, u32 regno, int off, int bpf_size, enum bpf_access_type t, - int value_regno, bool strict_alignment_once) + int value_regno, bool strict_alignment_once, + struct bpf_reg_state *atomic_load_reg) { struct bpf_reg_state *regs = cur_regs(env); struct bpf_reg_state *reg = regs + regno; @@ -4548,7 +4626,7 @@ static int check_mem_access(struct bpf_verifier_env *env, int insn_idx, u32 regn err = check_map_access(env, regno, off, size, false); if (!err) err = check_map_ptr_to_btf_id(env, regno, off, size, value_regno, - t, insn_idx); + t, insn_idx, atomic_load_reg); /* if err == 0, check_map_ptr_to_btf_id ignored the access */ if (!err && t == BPF_READ && value_regno >= 0) { struct bpf_map *map = reg->map_ptr; @@ -4743,9 +4821,12 @@ static int check_mem_access(struct bpf_verifier_env *env, int insn_idx, u32 regn static int check_atomic(struct bpf_verifier_env *env, int insn_idx, struct bpf_insn *insn) { + struct bpf_reg_state atomic_load_reg; int load_reg; int err; + __mark_reg_unknown(env, &atomic_load_reg); + switch (insn->imm) { case BPF_ADD: case BPF_ADD | BPF_FETCH: @@ -4813,6 +4894,7 @@ static int check_atomic(struct bpf_verifier_env *env, int insn_idx, struct bpf_i else load_reg = insn->src_reg; + atomic_load_reg = *reg_state(env, load_reg); /* check and record load of old value */ err = check_reg_arg(env, load_reg, DST_OP); if (err) @@ -4825,20 +4907,21 @@ static int check_atomic(struct bpf_verifier_env *env, int insn_idx, struct bpf_i } /* Check whether we can read the memory, with second call for fetch - * case to simulate the register fill. + * case to simulate the register fill, which also triggers checks + * for manipulation of BTF ID pointers embedded in BPF maps. */ err = check_mem_access(env, insn_idx, insn->dst_reg, insn->off, - BPF_SIZE(insn->code), BPF_READ, -1, true); + BPF_SIZE(insn->code), BPF_READ, -1, true, NULL); if (!err && load_reg >= 0) err = check_mem_access(env, insn_idx, insn->dst_reg, insn->off, BPF_SIZE(insn->code), BPF_READ, load_reg, - true); + true, load_reg >= 0 ? &atomic_load_reg : NULL); if (err) return err; /* Check whether we can write into the same memory. */ err = check_mem_access(env, insn_idx, insn->dst_reg, insn->off, - BPF_SIZE(insn->code), BPF_WRITE, -1, true); + BPF_SIZE(insn->code), BPF_WRITE, -1, true, NULL); if (err) return err; @@ -6797,7 +6880,7 @@ static int check_helper_call(struct bpf_verifier_env *env, struct bpf_insn *insn */ for (i = 0; i < meta.access_size; i++) { err = check_mem_access(env, insn_idx, meta.regno, i, BPF_B, - BPF_WRITE, -1, false); + BPF_WRITE, -1, false, NULL); if (err) return err; } @@ -11662,7 +11745,8 @@ static int do_check(struct bpf_verifier_env *env) */ err = check_mem_access(env, env->insn_idx, insn->src_reg, insn->off, BPF_SIZE(insn->code), - BPF_READ, insn->dst_reg, false); + BPF_READ, insn->dst_reg, false, + NULL); if (err) return err; @@ -11717,7 +11801,8 @@ static int do_check(struct bpf_verifier_env *env) /* check that memory (dst_reg + off) is writeable */ err = check_mem_access(env, env->insn_idx, insn->dst_reg, insn->off, BPF_SIZE(insn->code), - BPF_WRITE, insn->src_reg, false); + BPF_WRITE, insn->src_reg, false, + NULL); if (err) return err; @@ -11751,7 +11836,7 @@ static int do_check(struct bpf_verifier_env *env) /* check that memory (dst_reg + off) is writeable */ err = check_mem_access(env, env->insn_idx, insn->dst_reg, insn->off, BPF_SIZE(insn->code), - BPF_WRITE, -1, false); + BPF_WRITE, -1, false, NULL); if (err) return err; From patchwork Sun Feb 20 13:48:03 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kumar Kartikeya Dwivedi X-Patchwork-Id: 12752708 X-Patchwork-Delegate: bpf@iogearbox.net Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id B562EC433EF for ; Sun, 20 Feb 2022 13:48:43 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S243888AbiBTNtB (ORCPT ); Sun, 20 Feb 2022 08:49:01 -0500 Received: from mxb-00190b01.gslb.pphosted.com ([23.128.96.19]:49216 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S238277AbiBTNsy (ORCPT ); Sun, 20 Feb 2022 08:48:54 -0500 Received: from mail-pj1-x1043.google.com (mail-pj1-x1043.google.com [IPv6:2607:f8b0:4864:20::1043]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 9E14E2D1EA; Sun, 20 Feb 2022 05:48:32 -0800 (PST) Received: by mail-pj1-x1043.google.com with SMTP id m1-20020a17090a668100b001bc023c6f34so2660062pjj.3; Sun, 20 Feb 2022 05:48:32 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=le4LIxEm3r82lu6hJpPIFi0Z4KJGRcY5KDu3riF/uWI=; b=FLTk51NkcubhThZoBeGCRC/yBW+JAJHZF3/lQvXmhPNHeMdJDeNuteSVjtR50cG9Ve BQjh6n4reZ1gKjr4gzkYzX6dbc/x3yfJNITspAFXWnk4mTGfW9qUjmXmmPPr9+T4q+dx 0YyA8dkem3PIGsCxdNzQwka9jq+XhY8/mlrMWddBZYQaONzO9mNIlURgB0H/lcJSLg97 6n/VSCN85SJOeKEEIllx75ubueDbKvhGYFcPzFca2w5+j1PBWy6S2iLTKibsgTKrjU7y R4ivFiQlNdqg7nS7UErOi2SyInBlR8xLkWJkfGaWVnGD92FhWsFKDv/APv8NNdoV98Vc 0jQQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=le4LIxEm3r82lu6hJpPIFi0Z4KJGRcY5KDu3riF/uWI=; b=4+eQlpagsnuKCs8N8elTX3fPZbqvCkUQXI8ByA60jYVjrwShjO6QQ8WAPtOO4ENaaF c/1p5AF09NMc6WFn/aYGjpvDhP5gS4y/rt1aX1BVCL/VyEHKsc06toEOVjAreUTMjlGW ZMvdRyjasAp/bpvpjxskCcOVyLVFnOFbbhrRejTqRfRlAMvIT6ibcgD7bWLE3FBhKc8k 8oqIWEL6xzfznmanVxY6j24lJPQjVjiqZM32u1L4YLW2xwPmraCvMpMz8aeP3DD6Zcqk TBlPWihi4ySsU1MXYJqtN4VcrgQFC0O312+meRx4HSiRlD5ocIsQ7xhEJuWl59OA3XeP 6MuA== X-Gm-Message-State: AOAM532guvcLSTgKMiKI13xOYgcP/yHKKzwOfSImT3J/3HC1JU1oyvrG l0ejuzwmsFq4Q17fDdXMq491kayNSWc= X-Google-Smtp-Source: ABdhPJz6wfKXAihBFvMHy5nDZ3ylshSNW228U4y465R/Mp0DZs9vIJD8z2IAzL7emyi5HNXRCWw1nQ== X-Received: by 2002:a17:902:b701:b0:14d:b457:e506 with SMTP id d1-20020a170902b70100b0014db457e506mr15085464pls.32.1645364911953; Sun, 20 Feb 2022 05:48:31 -0800 (PST) Received: from localhost ([2405:201:6014:d0c0:6243:316e:a9e1:adda]) by smtp.gmail.com with ESMTPSA id c20sm1137425pfl.131.2022.02.20.05.48.31 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 20 Feb 2022 05:48:31 -0800 (PST) From: Kumar Kartikeya Dwivedi To: bpf@vger.kernel.org Cc: Hao Luo , Alexei Starovoitov , Andrii Nakryiko , Daniel Borkmann , =?utf-8?q?Toke_H=C3=B8iland-J=C3=B8rgensen?= , Jesper Dangaard Brouer , netfilter-devel@vger.kernel.org, netdev@vger.kernel.org Subject: [PATCH bpf-next v1 05/15] bpf: Allow storing PTR_TO_PERCPU_BTF_ID in map Date: Sun, 20 Feb 2022 19:18:03 +0530 Message-Id: <20220220134813.3411982-6-memxor@gmail.com> X-Mailer: git-send-email 2.35.1 In-Reply-To: <20220220134813.3411982-1-memxor@gmail.com> References: <20220220134813.3411982-1-memxor@gmail.com> MIME-Version: 1.0 X-Developer-Signature: v=1; a=openpgp-sha256; l=7909; h=from:subject; bh=mX65qkyAv1vHKbst+MgKHGZ+AuS+wXKig4npj53xi00=; b=owEBbQKS/ZANAwAIAUzgyIZIvxHKAcsmYgBiEkZXBVGDFD789R/fe7Q/K3EyQLh7xNIUGlE0moqg m7v1KbqJAjMEAAEIAB0WIQRLvip+Buz51YI8YRFM4MiGSL8RygUCYhJGVwAKCRBM4MiGSL8RyppqD/ 9bSYN2FPBZDa4dBU4dx5B6GFJaaKRC9/g0ovDxrOSLZbO+5a7ocJypUyCARDgHZlxgtgqrTuNAYDl7 GAN9PQx6E3DzTLMalQk5+kBwUSXJWgaZuci2OaTh/EmFS9lvNZRd5zAFeSoyx8Ff4Or1oIVxYFHa29 2QotkqzVdAsF6f0gNR04kXr5/sjxbDs+PWAy1X21b6llUvF0zUCArE1eB7WyqipBaFHtNE9bjw0aua ADwnfqkfdJIQR/R2YZmBUUjTDnvXuzexN7Ab2ONrhh5QzdND2k3SVgJDsMyLinYVJr5m6kabWtHgN6 es96bAxaJsa+RzO8YgYo1tA48GPXhG8z9UDoBFZsZPb7SHO62iuYMKWUFmcQkF2gesN3mhULa1G7tL xOzisaZNRND29ewYTakKnQLVk6GVlVQkZpWVPKbGg1S7/aoKgJ85oKr7qB59rfdMH0jKlyjiNSin59 mbalug/0Bf8iPyV94ZSdYE2w+cI99IPCJWStpoxtQbciiHSE8745BD9zwo0iCsIPuUZ4FYKA5dNu6P FiBEblXE5yaLCnjs4IuIs2HHq/I8f0m8pDoSZg59db565ZTOGa9fAPL1QZZus4eB9UCjd2vncfA3aD CtkSsfNdi5Nk5B2Wi5TeBIbtOOi1x5VF01MOfhhj5o8nCj3UFvZCwnAHP8Eg== X-Developer-Key: i=memxor@gmail.com; a=openpgp; fpr=4BBE2A7E06ECF9D5823C61114CE0C88648BF11CA Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org X-Patchwork-Delegate: bpf@iogearbox.net Make adjustments to the code to allow storing PTR_TO_PERCPU_BTF_ID in a map. Note that these are not yet supported as referenced pointers, so that is explicitly disallowed during BTF tag parsing. Similar to 'ref' tag, a new 'percpu' tag composes with 'btf_id' tag on the pointed to type to hint that it is a percpu btf_id pointer. Cc: Hao Luo Signed-off-by: Kumar Kartikeya Dwivedi Reported-by: kernel test robot --- include/linux/bpf.h | 3 ++- kernel/bpf/btf.c | 27 ++++++++++++++++++++++----- kernel/bpf/verifier.c | 37 ++++++++++++++++++++++++++++--------- 3 files changed, 52 insertions(+), 15 deletions(-) diff --git a/include/linux/bpf.h b/include/linux/bpf.h index 923b9f36c275..843c8c01cf9d 100644 --- a/include/linux/bpf.h +++ b/include/linux/bpf.h @@ -161,7 +161,8 @@ enum { }; enum { - BPF_MAP_VALUE_OFF_F_REF = (1U << 0), + BPF_MAP_VALUE_OFF_F_REF = (1U << 0), + BPF_MAP_VALUE_OFF_F_PERCPU = (1U << 1), }; struct bpf_map_value_off_desc { diff --git a/kernel/bpf/btf.c b/kernel/bpf/btf.c index 20124f4a421c..eb57584ee0a8 100644 --- a/kernel/bpf/btf.c +++ b/kernel/bpf/btf.c @@ -3146,12 +3146,12 @@ static s32 btf_find_by_name_kind_all(const char *name, u32 kind, struct btf **bt static int btf_find_field_kptr(const struct btf *btf, const struct btf_type *t, u32 off, int sz, void *data) { - bool btf_id_tag = false, ref_tag = false; + bool btf_id_tag = false, ref_tag = false, percpu_tag = false; struct bpf_map_value_off *tab; struct bpf_map *map = data; + int nr_off, ret, flags = 0; struct module *mod = NULL; struct btf *kernel_btf; - int nr_off, ret; s32 id; /* For PTR, sz is always == 8 */ @@ -3174,6 +3174,13 @@ static int btf_find_field_kptr(const struct btf *btf, const struct btf_type *t, goto end; } ref_tag = true; + } else if (!strcmp("kernel.bpf.percpu", __btf_name_by_offset(btf, t->name_off))) { + /* repeated tag */ + if (percpu_tag) { + ret = -EINVAL; + goto end; + } + percpu_tag = true; } else if (!strncmp("kernel.", __btf_name_by_offset(btf, t->name_off), sizeof("kernel.") - 1)) { /* TODO: Should we reject these when loading BTF? */ @@ -3185,13 +3192,18 @@ static int btf_find_field_kptr(const struct btf *btf, const struct btf_type *t, t = btf_type_by_id(btf, t->type); } if (!btf_id_tag) { - /* 'ref' tag must be specified together with 'btf_id' tag */ - if (ref_tag) { + /* 'ref' or 'percpu' tag must be specified together with 'btf_id' tag */ + if (ref_tag || percpu_tag) { ret = -EINVAL; goto end; } return 0; } + /* referenced percpu btf_id pointer is not yet supported */ + if (ref_tag && percpu_tag) { + ret = -EINVAL; + goto end; + } /* Get the base type */ if (btf_type_is_modifier(t)) @@ -3241,11 +3253,16 @@ static int btf_find_field_kptr(const struct btf *btf, const struct btf_type *t, } } + if (ref_tag) + flags |= BPF_MAP_VALUE_OFF_F_REF; + else if (percpu_tag) + flags |= BPF_MAP_VALUE_OFF_F_PERCPU; + tab->off[nr_off].offset = off; tab->off[nr_off].btf_id = id; tab->off[nr_off].btf = kernel_btf; tab->off[nr_off].module = mod; - tab->off[nr_off].flags = ref_tag ? BPF_MAP_VALUE_OFF_F_REF : 0; + tab->off[nr_off].flags = flags; tab->nr_off++; return 0; diff --git a/kernel/bpf/verifier.c b/kernel/bpf/verifier.c index a9d8c0d3c919..00d6ab49033d 100644 --- a/kernel/bpf/verifier.c +++ b/kernel/bpf/verifier.c @@ -1559,12 +1559,13 @@ static void mark_btf_ld_reg(struct bpf_verifier_env *env, struct btf *btf, u32 btf_id, enum bpf_type_flag flag) { - if (reg_type == SCALAR_VALUE) { + if (reg_type == SCALAR_VALUE || + WARN_ON_ONCE(reg_type != PTR_TO_BTF_ID && reg_type != PTR_TO_PERCPU_BTF_ID)) { mark_reg_unknown(env, regs, regno); return; } mark_reg_known_zero(env, regs, regno); - regs[regno].type = PTR_TO_BTF_ID | flag; + regs[regno].type = reg_type | flag; regs[regno].btf = btf; regs[regno].btf_id = btf_id; } @@ -3478,10 +3479,18 @@ static int map_ptr_to_btf_id_match_type(struct bpf_verifier_env *env, bool ref_ptr) { const char *targ_name = kernel_type_name(off_desc->btf, off_desc->btf_id); + enum bpf_reg_type reg_type; const char *reg_name = ""; - if (reg->type != PTR_TO_BTF_ID && reg->type != PTR_TO_BTF_ID_OR_NULL) - goto end; + if (off_desc->flags & BPF_MAP_VALUE_OFF_F_PERCPU) { + if (reg->type != PTR_TO_PERCPU_BTF_ID && + reg->type != (PTR_TO_PERCPU_BTF_ID | PTR_MAYBE_NULL)) + goto end; + } else { /* referenced and unreferenced case */ + if (reg->type != PTR_TO_BTF_ID && + reg->type != (PTR_TO_BTF_ID | PTR_MAYBE_NULL)) + goto end; + } if (!btf_is_kernel(reg->btf)) { verbose(env, "R%d must point to kernel BTF\n", regno); @@ -3524,11 +3533,16 @@ static int map_ptr_to_btf_id_match_type(struct bpf_verifier_env *env, if (!btf_struct_ids_match(&env->log, reg->btf, reg->btf_id, reg->off, off_desc->btf, off_desc->btf_id)) goto end; + return 0; end: + if (off_desc->flags & BPF_MAP_VALUE_OFF_F_PERCPU) + reg_type = PTR_TO_PERCPU_BTF_ID | PTR_MAYBE_NULL; + else + reg_type = PTR_TO_BTF_ID | PTR_MAYBE_NULL; verbose(env, "invalid btf_id pointer access, R%d type=%s%s ", regno, reg_type_str(env, reg->type), reg_name); - verbose(env, "expected=%s%s\n", reg_type_str(env, PTR_TO_BTF_ID), targ_name); + verbose(env, "expected=%s%s\n", reg_type_str(env, reg_type), targ_name); return -EINVAL; } @@ -3543,10 +3557,11 @@ static int check_map_ptr_to_btf_id(struct bpf_verifier_env *env, u32 regno, int { struct bpf_reg_state *reg = reg_state(env, regno), *val_reg; struct bpf_insn *insn = &env->prog->insnsi[insn_idx]; + enum bpf_reg_type reg_type = PTR_TO_BTF_ID; + bool ref_ptr = false, percpu_ptr = false; struct bpf_map_value_off_desc *off_desc; int insn_class = BPF_CLASS(insn->code); struct bpf_map *map = reg->map_ptr; - bool ref_ptr = false; u32 ref_obj_id = 0; int ret; @@ -3561,7 +3576,6 @@ static int check_map_ptr_to_btf_id(struct bpf_verifier_env *env, u32 regno, int off_desc = bpf_map_ptr_off_contains(map, off + reg->var_off.value); if (!off_desc) return 0; - ref_ptr = off_desc->flags & BPF_MAP_VALUE_OFF_F_REF; if (WARN_ON_ONCE(size != bpf_size_to_bytes(BPF_DW))) return -EACCES; @@ -3574,6 +3588,11 @@ static int check_map_ptr_to_btf_id(struct bpf_verifier_env *env, u32 regno, int return -EPERM; } + ref_ptr = off_desc->flags & BPF_MAP_VALUE_OFF_F_REF; + percpu_ptr = off_desc->flags & BPF_MAP_VALUE_OFF_F_PERCPU; + if (percpu_ptr) + reg_type = PTR_TO_PERCPU_BTF_ID; + if (is_xchg_insn(insn)) { /* We do checks and updates during register fill call for fetch case */ if (t != BPF_READ || value_regno < 0) @@ -3603,7 +3622,7 @@ static int check_map_ptr_to_btf_id(struct bpf_verifier_env *env, u32 regno, int ref_obj_id = ret; } /* val_reg might be NULL at this point */ - mark_btf_ld_reg(env, cur_regs(env), value_regno, PTR_TO_BTF_ID, off_desc->btf, + mark_btf_ld_reg(env, cur_regs(env), value_regno, reg_type, off_desc->btf, off_desc->btf_id, PTR_MAYBE_NULL); /* __mark_ptr_or_null_regs needs ref_obj_id == id to clear * reference state for ptr == NULL branch. @@ -3621,7 +3640,7 @@ static int check_map_ptr_to_btf_id(struct bpf_verifier_env *env, u32 regno, int /* We can simply mark the value_regno receiving the pointer * value from map as PTR_TO_BTF_ID, with the correct type. */ - mark_btf_ld_reg(env, cur_regs(env), value_regno, PTR_TO_BTF_ID, off_desc->btf, + mark_btf_ld_reg(env, cur_regs(env), value_regno, reg_type, off_desc->btf, off_desc->btf_id, PTR_MAYBE_NULL); val_reg->id = ++env->id_gen; } else if (insn_class == BPF_STX) { From patchwork Sun Feb 20 13:48:04 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kumar Kartikeya Dwivedi X-Patchwork-Id: 12752706 X-Patchwork-Delegate: bpf@iogearbox.net Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id B9D9CC4321E for ; Sun, 20 Feb 2022 13:48:41 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S243880AbiBTNtA (ORCPT ); Sun, 20 Feb 2022 08:49:00 -0500 Received: from mxb-00190b01.gslb.pphosted.com ([23.128.96.19]:49418 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S243842AbiBTNs4 (ORCPT ); Sun, 20 Feb 2022 08:48:56 -0500 Received: from mail-pj1-x1043.google.com (mail-pj1-x1043.google.com [IPv6:2607:f8b0:4864:20::1043]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id AB4632DA86; Sun, 20 Feb 2022 05:48:35 -0800 (PST) Received: by mail-pj1-x1043.google.com with SMTP id v13-20020a17090ac90d00b001b87bc106bdso16509985pjt.4; Sun, 20 Feb 2022 05:48:35 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=R9d2PpgqfauFJJjkRlCvvbMhZKsS/YVJK4MsuC6I8N0=; b=ZCI/j3azJWDM65HsBnnX5qjBvmN0fLz4r3rWoOOvlhaZlxSiIa4d6LL18ZyZ9BUvWJ sNuDIyu/FQ2KFfNoqMD3COslhcDYLuseRUGe29OnmnQDHRb+Lv1MCupgsVlS1y7tyroX SdZw/Ic+NVb25iquYB9hWE5sWwq6TqRPmRNAE9ET1mYB2CobJ/k4wZlcmxhd0KwiHcc4 bxMmKt+DCWjpGgJ0QiLEehiUY3MIGRptYFH13N19hh3exIt/IdyNTT/mk1a69E5KCFGp JQf0rHLbJeUa5xw8E8VGxMRRcSnkruhu5IsChufUteRQz4v58cYKhMbtGj+MQrUbSV8k o65A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=R9d2PpgqfauFJJjkRlCvvbMhZKsS/YVJK4MsuC6I8N0=; b=iu/WO2H/VdhBjtzmvTx2PfxRO+N6ma7FrkZ2PZTxtPOCnOWAqP0kEYJcAXxQ11wNEC xo9yrXQCCLWdBO1FCYDP497PN9V30YTGND/OSMeKfFKEjVNN/1uSTiEbC5CZt6AV9DwD iaSK7yPaGX5MjNHKq0KU0ZwtgwWqrjBbUkrxJPlqfPX7uKyEDYmnJJyeuYNi7s5z3wwN 7kgICabZTUixymgkThfIuvthiR9ztOyEuy0FfuuMVqSOxtp2SRabzBjHi4PcfvZaRFJo 5ZGLTOWpMSlq/0rBHTjtXNu8B5HDn2hzG230Yv2tqsm4i/ndYtlA6OFUgJ6hd3Hf0n/6 D8gA== X-Gm-Message-State: AOAM531SJdOyDKcfVEMydHjx9Eamr5eJbNSyh2djY1fjFMYAWU91Ta0T zzjPeKSK7kKgq+Etxt6AzaDGCfQAg44= X-Google-Smtp-Source: ABdhPJxOUHncv9ivzRcctx7PwuFM8hfyviiF0JS2hla6Xnku9b8LAM4SGV7HqV9S2tKeJ/vAHBYa5A== X-Received: by 2002:a17:90b:1d91:b0:1b9:d5fd:3c8a with SMTP id pf17-20020a17090b1d9100b001b9d5fd3c8amr21440556pjb.213.1645364915014; Sun, 20 Feb 2022 05:48:35 -0800 (PST) Received: from localhost ([2405:201:6014:d0c0:6243:316e:a9e1:adda]) by smtp.gmail.com with ESMTPSA id a14sm9759980pfv.51.2022.02.20.05.48.34 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 20 Feb 2022 05:48:34 -0800 (PST) From: Kumar Kartikeya Dwivedi To: bpf@vger.kernel.org Cc: Alexei Starovoitov , Andrii Nakryiko , Daniel Borkmann , =?utf-8?q?Toke_H=C3=B8iland-J=C3=B8?= =?utf-8?q?rgensen?= , Jesper Dangaard Brouer , netfilter-devel@vger.kernel.org, netdev@vger.kernel.org Subject: [PATCH bpf-next v1 06/15] bpf: Allow storing __user PTR_TO_BTF_ID in map Date: Sun, 20 Feb 2022 19:18:04 +0530 Message-Id: <20220220134813.3411982-7-memxor@gmail.com> X-Mailer: git-send-email 2.35.1 In-Reply-To: <20220220134813.3411982-1-memxor@gmail.com> References: <20220220134813.3411982-1-memxor@gmail.com> MIME-Version: 1.0 X-Developer-Signature: v=1; a=openpgp-sha256; l=6532; h=from:subject; bh=d2wogHsh3SI7Hcu5QYrntWdvoEgeiQvZgIP9xtiBwrU=; b=owEBbQKS/ZANAwAIAUzgyIZIvxHKAcsmYgBiEkZX4+2EeGW6HzIGTAy16zwZeMmxnmOUgj8rPFSB XCIy18+JAjMEAAEIAB0WIQRLvip+Buz51YI8YRFM4MiGSL8RygUCYhJGVwAKCRBM4MiGSL8RynZZD/ 9DamE0brQ9+dzNmjcWrKPFl74LTgMQmypmdL5a5h0Glp5IlpIdZzWAYRMLzu32bV+uZINvn2y/fP90 ow2oOGbdH1spM9LBxoe4nFsI3BXEkbR0tPkktKgO0QI+OWWB0/ERfP0FI2B/xyQxZ8shaFHuU1JrKK thXBIPHatBhwgcnDm2OfrqFbsou/QtyfaOyJDEUwb53dLlf4INVAsXqpoGumZ2F/8GH2C/LAcN/YYq GW/RU8lj6R6LOk1yvctU/WRxysocKtWV3lAXlfr2vUDEiA0xZwkz2bvI/PjBXMah488Jzoz63zDY5g MwDLIFvmJZmP5hZ74+TL0+/0Un8sg0lud4Rge2N96lIauHxZZ9j6+f86sV92F51LSOVEjfe36svkFb L5omNfGG7oco5defkgtwLq/pxzoFW7OyQQc6xyQdMt2hdoQEkrnOp4GTWNI5li8mpulIAhHe/Pajrx VLEoB6LSRlCuIDa+WA9hqZeyldK8r07szvaz5EpAMKrwtI8cD46YmhZOkHxMBOB5Fp4egNA/Tc/waF NuqddaD9Zpel39uelWch8LaCcZ/p8ErbWUcL+euZPWhgmyvIv0U4bKgBF3vLpwfs49GJAyIhES8QBB dpmcAc2hwdlbeXIRcYQ2qNZ+cJBv8uVmUNa2nQcLUedl0F4XJIKSlL7Xrunw== X-Developer-Key: i=memxor@gmail.com; a=openpgp; fpr=4BBE2A7E06ECF9D5823C61114CE0C88648BF11CA Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org X-Patchwork-Delegate: bpf@iogearbox.net Recently, verifier gained __user annotation support [0] where it prevents BPF program from normally derefering user memory pointer in the kernel, and instead requires use of bpf_probe_read_user. We can allow the user to also store these pointers in BPF maps, with the logic that whenever user loads it from the BPF map, it gets marked as MEM_USER. [0]: https://lore.kernel.org/bpf/20220127154555.650886-1-yhs@fb.com Signed-off-by: Kumar Kartikeya Dwivedi --- include/linux/bpf.h | 1 + kernel/bpf/btf.c | 20 +++++++++++++++----- kernel/bpf/verifier.c | 21 +++++++++++++++------ 3 files changed, 31 insertions(+), 11 deletions(-) diff --git a/include/linux/bpf.h b/include/linux/bpf.h index 843c8c01cf9d..37ca92f4c7b7 100644 --- a/include/linux/bpf.h +++ b/include/linux/bpf.h @@ -163,6 +163,7 @@ enum { enum { BPF_MAP_VALUE_OFF_F_REF = (1U << 0), BPF_MAP_VALUE_OFF_F_PERCPU = (1U << 1), + BPF_MAP_VALUE_OFF_F_USER = (1U << 2), }; struct bpf_map_value_off_desc { diff --git a/kernel/bpf/btf.c b/kernel/bpf/btf.c index eb57584ee0a8..bafceae90c32 100644 --- a/kernel/bpf/btf.c +++ b/kernel/bpf/btf.c @@ -3146,7 +3146,7 @@ static s32 btf_find_by_name_kind_all(const char *name, u32 kind, struct btf **bt static int btf_find_field_kptr(const struct btf *btf, const struct btf_type *t, u32 off, int sz, void *data) { - bool btf_id_tag = false, ref_tag = false, percpu_tag = false; + bool btf_id_tag = false, ref_tag = false, percpu_tag = false, user_tag = false; struct bpf_map_value_off *tab; struct bpf_map *map = data; int nr_off, ret, flags = 0; @@ -3181,6 +3181,13 @@ static int btf_find_field_kptr(const struct btf *btf, const struct btf_type *t, goto end; } percpu_tag = true; + } else if (!strcmp("kernel.bpf.user", __btf_name_by_offset(btf, t->name_off))) { + /* repeated tag */ + if (user_tag) { + ret = -EINVAL; + goto end; + } + user_tag = true; } else if (!strncmp("kernel.", __btf_name_by_offset(btf, t->name_off), sizeof("kernel.") - 1)) { /* TODO: Should we reject these when loading BTF? */ @@ -3192,15 +3199,16 @@ static int btf_find_field_kptr(const struct btf *btf, const struct btf_type *t, t = btf_type_by_id(btf, t->type); } if (!btf_id_tag) { - /* 'ref' or 'percpu' tag must be specified together with 'btf_id' tag */ - if (ref_tag || percpu_tag) { + /* 'ref', 'percpu', 'user' tag must be specified together with 'btf_id' tag */ + if (ref_tag || percpu_tag || user_tag) { ret = -EINVAL; goto end; } return 0; } - /* referenced percpu btf_id pointer is not yet supported */ - if (ref_tag && percpu_tag) { + /* All three are mutually exclusive */ + ret = ref_tag + percpu_tag + user_tag; + if (ret > 1) { ret = -EINVAL; goto end; } @@ -3257,6 +3265,8 @@ static int btf_find_field_kptr(const struct btf *btf, const struct btf_type *t, flags |= BPF_MAP_VALUE_OFF_F_REF; else if (percpu_tag) flags |= BPF_MAP_VALUE_OFF_F_PERCPU; + else if (user_tag) + flags |= BPF_MAP_VALUE_OFF_F_USER; tab->off[nr_off].offset = off; tab->off[nr_off].btf_id = id; diff --git a/kernel/bpf/verifier.c b/kernel/bpf/verifier.c index 00d6ab49033d..28da858bb921 100644 --- a/kernel/bpf/verifier.c +++ b/kernel/bpf/verifier.c @@ -3482,7 +3482,11 @@ static int map_ptr_to_btf_id_match_type(struct bpf_verifier_env *env, enum bpf_reg_type reg_type; const char *reg_name = ""; - if (off_desc->flags & BPF_MAP_VALUE_OFF_F_PERCPU) { + if (off_desc->flags & BPF_MAP_VALUE_OFF_F_USER) { + if (reg->type != (PTR_TO_BTF_ID | MEM_USER) && + reg->type != (PTR_TO_BTF_ID | PTR_MAYBE_NULL | MEM_USER)) + goto end; + } else if (off_desc->flags & BPF_MAP_VALUE_OFF_F_PERCPU) { if (reg->type != PTR_TO_PERCPU_BTF_ID && reg->type != (PTR_TO_PERCPU_BTF_ID | PTR_MAYBE_NULL)) goto end; @@ -3536,7 +3540,9 @@ static int map_ptr_to_btf_id_match_type(struct bpf_verifier_env *env, return 0; end: - if (off_desc->flags & BPF_MAP_VALUE_OFF_F_PERCPU) + if (off_desc->flags & BPF_MAP_VALUE_OFF_F_USER) + reg_type = PTR_TO_BTF_ID | PTR_MAYBE_NULL | MEM_USER; + else if (off_desc->flags & BPF_MAP_VALUE_OFF_F_PERCPU) reg_type = PTR_TO_PERCPU_BTF_ID | PTR_MAYBE_NULL; else reg_type = PTR_TO_BTF_ID | PTR_MAYBE_NULL; @@ -3556,14 +3562,14 @@ static int check_map_ptr_to_btf_id(struct bpf_verifier_env *env, u32 regno, int struct bpf_reg_state *atomic_load_reg) { struct bpf_reg_state *reg = reg_state(env, regno), *val_reg; + bool ref_ptr = false, percpu_ptr = false, user_ptr = false; struct bpf_insn *insn = &env->prog->insnsi[insn_idx]; enum bpf_reg_type reg_type = PTR_TO_BTF_ID; - bool ref_ptr = false, percpu_ptr = false; struct bpf_map_value_off_desc *off_desc; int insn_class = BPF_CLASS(insn->code); + int ret, reg_flags = PTR_MAYBE_NULL; struct bpf_map *map = reg->map_ptr; u32 ref_obj_id = 0; - int ret; /* Things we already checked for in check_map_access: * - Reject cases where variable offset may touch BTF ID pointer @@ -3590,8 +3596,11 @@ static int check_map_ptr_to_btf_id(struct bpf_verifier_env *env, u32 regno, int ref_ptr = off_desc->flags & BPF_MAP_VALUE_OFF_F_REF; percpu_ptr = off_desc->flags & BPF_MAP_VALUE_OFF_F_PERCPU; + user_ptr = off_desc->flags & BPF_MAP_VALUE_OFF_F_USER; if (percpu_ptr) reg_type = PTR_TO_PERCPU_BTF_ID; + else if (user_ptr) + reg_flags |= MEM_USER; if (is_xchg_insn(insn)) { /* We do checks and updates during register fill call for fetch case */ @@ -3623,7 +3632,7 @@ static int check_map_ptr_to_btf_id(struct bpf_verifier_env *env, u32 regno, int } /* val_reg might be NULL at this point */ mark_btf_ld_reg(env, cur_regs(env), value_regno, reg_type, off_desc->btf, - off_desc->btf_id, PTR_MAYBE_NULL); + off_desc->btf_id, reg_flags); /* __mark_ptr_or_null_regs needs ref_obj_id == id to clear * reference state for ptr == NULL branch. */ @@ -3641,7 +3650,7 @@ static int check_map_ptr_to_btf_id(struct bpf_verifier_env *env, u32 regno, int * value from map as PTR_TO_BTF_ID, with the correct type. */ mark_btf_ld_reg(env, cur_regs(env), value_regno, reg_type, off_desc->btf, - off_desc->btf_id, PTR_MAYBE_NULL); + off_desc->btf_id, reg_flags); val_reg->id = ++env->id_gen; } else if (insn_class == BPF_STX) { if (WARN_ON_ONCE(value_regno < 0)) From patchwork Sun Feb 20 13:48:05 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kumar Kartikeya Dwivedi X-Patchwork-Id: 12752709 X-Patchwork-Delegate: bpf@iogearbox.net Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id A11E4C433EF for ; Sun, 20 Feb 2022 13:48:46 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S243890AbiBTNtD (ORCPT ); Sun, 20 Feb 2022 08:49:03 -0500 Received: from mxb-00190b01.gslb.pphosted.com ([23.128.96.19]:49652 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S243864AbiBTNs7 (ORCPT ); Sun, 20 Feb 2022 08:48:59 -0500 Received: from mail-pj1-x1041.google.com (mail-pj1-x1041.google.com [IPv6:2607:f8b0:4864:20::1041]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id A63E62DAA5; Sun, 20 Feb 2022 05:48:38 -0800 (PST) Received: by mail-pj1-x1041.google.com with SMTP id q8-20020a17090a178800b001bc299b8de1so1167699pja.1; Sun, 20 Feb 2022 05:48:38 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=RnTzyrPN1gycMBmbewX188jeez+a/yeyOEr6gJG3f+Y=; b=bFbj9bV6K8a3Rlr4t/RR1IlNRc69Z5gko3EBHgmng6WZoLVW/zBl14o7UOoBXJIvYU rGMBDinhff+Z18RVo308Dk2GghtibKLfob/t5a6wvpAWN9ewShhQN+utBMcA3gDzQWSQ ARfnZYitF9Z1eXCn1waaRUn0y0OXLhXFVlmQCwrAeKewsQiKEXax9guzWrY4NarbP74A Np8IAjEHJXIz3emHLgouLvGSQYK3r3PiGpLc82+DI+6uF98X+Ev76vRpb+w9nrp75reA nvmRM7FZ+W2iYGlF20//8V+P2BnPOGzUwgfNzA7HRtqVSgs9KR4KXkIHEJNAPpRNVW4B pMrQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=RnTzyrPN1gycMBmbewX188jeez+a/yeyOEr6gJG3f+Y=; b=NhLDP+jkQdXXBqjQEG2FcpC95AHKnBT9/GY7JrHsXiRWBE3anYzRrK5BWwRLDMZ3+g wwg6qtgSetPLZwuYKe6FEcKpFKMkOKpLtF9zpvCy1eM422vCSusl7PUvQdJJfY3LXHND GvAVxfRklicXc6oM6DgZBQQuUhJZdcI1EgHaCNrkmzlXjA1Na63Cg+ZtA/A2D6TCkxNQ DJ7CmZgmTdg8DMdbmAtSmm7eybkHqIKHHBmnGX26EgsrjoWR3QWc3bPS1xZZDDYy/T9N IIbGrC9YfDOb5H/gb8TeGVQZrHfodqbtwsk25Xe5k8+KChdZDdME/9MX52NRpTu8mFCj I8gg== X-Gm-Message-State: AOAM530L6lK87Sf6FL/6yfYJiDaSpv1azJCMfWFqFULr+LF7NZB18pLv u0sHFM6+PP2xQA1KLpiH6JTW6cdfV9o= X-Google-Smtp-Source: ABdhPJxAGalx6+eGvybddHpkcMjL1+QsclhW0mshnpqG7PU2+O5rEmAANH14JVVQV04DwqybtEWXmw== X-Received: by 2002:a17:90b:28f:b0:1bc:299d:39a8 with SMTP id az15-20020a17090b028f00b001bc299d39a8mr2433231pjb.37.1645364918031; Sun, 20 Feb 2022 05:48:38 -0800 (PST) Received: from localhost ([2405:201:6014:d0c0:6243:316e:a9e1:adda]) by smtp.gmail.com with ESMTPSA id b14sm9348428pfm.17.2022.02.20.05.48.37 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 20 Feb 2022 05:48:37 -0800 (PST) From: Kumar Kartikeya Dwivedi To: bpf@vger.kernel.org Cc: Alexei Starovoitov , Andrii Nakryiko , Daniel Borkmann , =?utf-8?q?Toke_H=C3=B8iland-J=C3=B8?= =?utf-8?q?rgensen?= , Jesper Dangaard Brouer , netfilter-devel@vger.kernel.org, netdev@vger.kernel.org Subject: [PATCH bpf-next v1 07/15] bpf: Prevent escaping of pointers loaded from maps Date: Sun, 20 Feb 2022 19:18:05 +0530 Message-Id: <20220220134813.3411982-8-memxor@gmail.com> X-Mailer: git-send-email 2.35.1 In-Reply-To: <20220220134813.3411982-1-memxor@gmail.com> References: <20220220134813.3411982-1-memxor@gmail.com> MIME-Version: 1.0 X-Developer-Signature: v=1; a=openpgp-sha256; l=5602; h=from:subject; bh=7UyEom+I59hJQUO2S4Ny604SxmBWVuMgkWYVqDd2Kiw=; b=owEBbQKS/ZANAwAIAUzgyIZIvxHKAcsmYgBiEkZXytxd+LuA78DGayPrTa0kjZ8Qf3jQquOITi3n MT7H7vKJAjMEAAEIAB0WIQRLvip+Buz51YI8YRFM4MiGSL8RygUCYhJGVwAKCRBM4MiGSL8RymT4EA C+prnpoetmZ72YD4fVgYRw7vM/H8mHwPPhQEzi/SXtSc7BqYZGy4EX1qxtJSYdoi66PtuAMIUszagu 2MFxKzHd1h+XypPkNiE6CzsIaPSspbIsTZR6WqJbHxeLxpPveQm64E6k7lGmA3Muiu4pQDSG9ffKlr Li9Y2IzzIyY01fcj/9xudgSl2zpwm2+tYvpDRhzrdTykWPGvMtE2JMHIP1ygAlzy3+Dv2ZXKzeCcFf Wz2tPizM3t6NXieOcEyMGoBcwa7WFFH8sNPlHviHGWz9V3S23BUw8qAVHdXAr9Uy14QktWpcc4jPE2 u72ShDG7MUY8Tp3cExw12TDqtvY8iNF+8y6t5VMTe8zeA3WjLm5yHu5sxoL8lcZLbPNNwLRS0NjFOY 90Z+1HY0+PiF23tgEccoaF1mOHXGCxAUByPjO6xgZtW8UleikkQ+LQSTj4Zy4SFZDnsuQCLkuDnJdK bKb7aYdXSiQWKiKT0mSxwKGgxYpZ4nRYmkbFVst2j7VquPAjFHkZwfMTQewaBhMRJ39MN/Yw6OIAYx +R4lhuYJA6mHP06v87CuqVcGJwuTW0oVjxo2ACfWUejrqeOq1fb3dmFX+3BKAcAfTSK/DFK3NB6PWQ BqSUl8BrjVTsZiAR6cP1bYt+Z/qWMs1VwyfpP1hgkIC3qnUYn9ARP4SpfEww== X-Developer-Key: i=memxor@gmail.com; a=openpgp; fpr=4BBE2A7E06ECF9D5823C61114CE0C88648BF11CA Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org X-Patchwork-Delegate: bpf@iogearbox.net While we can guarantee that even for unreferenced pointer, the object pointer points to being freed etc. can be handled by the verifier's exception handling (normal load patching to PROBE_MEM loads), we still cannot allow the user to pass these pointers to BPF helpers and kfunc, because the same exception handling won't be done for accesses inside the kernel. Hence introduce a new type flag, PTR_UNTRUSTED, which is used to mark all registers loading unreferenced PTR_TO_BTF_ID from BPF maps, and ensure they can never escape the BPF program and into the kernel by way of calling stable/unstable helpers. Adjust the check in check_mem_access so that we allow calling check_ptr_to_btf_access only when no or PTR_UNTRUSTED type flag is set. Signed-off-by: Kumar Kartikeya Dwivedi --- include/linux/bpf.h | 7 +++++++ kernel/bpf/verifier.c | 29 ++++++++++++++++++++++++++--- 2 files changed, 33 insertions(+), 3 deletions(-) diff --git a/include/linux/bpf.h b/include/linux/bpf.h index 37ca92f4c7b7..ae599aaf8d4c 100644 --- a/include/linux/bpf.h +++ b/include/linux/bpf.h @@ -364,6 +364,13 @@ enum bpf_type_flag { /* MEM is in user address space. */ MEM_USER = BIT(3 + BPF_BASE_TYPE_BITS), + /* PTR is not trusted. This is only used with PTR_TO_BTF_ID, to mark + * unrefcounted pointers loaded from map value, so that they can only + * be dereferenced but not escape the BPF program into the kernel (i.e. + * cannot be passed as arguments to kfunc or bpf helpers). + */ + PTR_UNTRUSTED = BIT(4 + BPF_BASE_TYPE_BITS), + __BPF_TYPE_LAST_FLAG = MEM_USER, }; diff --git a/kernel/bpf/verifier.c b/kernel/bpf/verifier.c index 28da858bb921..0a2cd21d9ec1 100644 --- a/kernel/bpf/verifier.c +++ b/kernel/bpf/verifier.c @@ -582,6 +582,8 @@ static const char *reg_type_str(struct bpf_verifier_env *env, strncpy(prefix, "alloc_", 32); if (type & MEM_USER) strncpy(prefix, "user_", 32); + if (type & PTR_UNTRUSTED) + strncpy(prefix, "untrusted_", 32); snprintf(env->type_str_buf, TYPE_STR_BUF_LEN, "%s%s%s", prefix, str[base_type(type)], postfix); @@ -3490,10 +3492,17 @@ static int map_ptr_to_btf_id_match_type(struct bpf_verifier_env *env, if (reg->type != PTR_TO_PERCPU_BTF_ID && reg->type != (PTR_TO_PERCPU_BTF_ID | PTR_MAYBE_NULL)) goto end; - } else { /* referenced and unreferenced case */ + } else if (off_desc->flags & BPF_MAP_VALUE_OFF_F_REF) { + /* register state (ref_obj_id) must be checked by caller */ if (reg->type != PTR_TO_BTF_ID && reg->type != (PTR_TO_BTF_ID | PTR_MAYBE_NULL)) goto end; + } else { /* only unreferenced case accepts untrusted pointers */ + if (reg->type != PTR_TO_BTF_ID && + reg->type != (PTR_TO_BTF_ID | PTR_MAYBE_NULL) && + reg->type != (PTR_TO_BTF_ID | PTR_UNTRUSTED) && + reg->type != (PTR_TO_BTF_ID | PTR_MAYBE_NULL | PTR_UNTRUSTED)) + goto end; } if (!btf_is_kernel(reg->btf)) { @@ -3597,10 +3606,13 @@ static int check_map_ptr_to_btf_id(struct bpf_verifier_env *env, u32 regno, int ref_ptr = off_desc->flags & BPF_MAP_VALUE_OFF_F_REF; percpu_ptr = off_desc->flags & BPF_MAP_VALUE_OFF_F_PERCPU; user_ptr = off_desc->flags & BPF_MAP_VALUE_OFF_F_USER; + if (percpu_ptr) reg_type = PTR_TO_PERCPU_BTF_ID; else if (user_ptr) reg_flags |= MEM_USER; + else + reg_flags |= PTR_UNTRUSTED; if (is_xchg_insn(insn)) { /* We do checks and updates during register fill call for fetch case */ @@ -3629,6 +3641,10 @@ static int check_map_ptr_to_btf_id(struct bpf_verifier_env *env, u32 regno, int if (ret < 0) return ret; ref_obj_id = ret; + /* Unset PTR_UNTRUSTED, so that it can be passed to bpf + * helpers or kfunc. + */ + reg_flags &= ~PTR_UNTRUSTED; } /* val_reg might be NULL at this point */ mark_btf_ld_reg(env, cur_regs(env), value_regno, reg_type, off_desc->btf, @@ -4454,6 +4470,12 @@ static int check_ptr_to_btf_access(struct bpf_verifier_env *env, if (ret < 0) return ret; + /* If this is an untrusted pointer, all btf_id pointers formed by + * walking it also inherit the untrusted flag. + */ + if (type_flag(reg->type) & PTR_UNTRUSTED) + flag |= PTR_UNTRUSTED; + if (atype == BPF_READ && value_regno >= 0) mark_btf_ld_reg(env, regs, value_regno, ret, reg->btf, btf_id, flag); @@ -4804,7 +4826,7 @@ static int check_mem_access(struct bpf_verifier_env *env, int insn_idx, u32 regn err = check_tp_buffer_access(env, reg, regno, off, size); if (!err && t == BPF_READ && value_regno >= 0) mark_reg_unknown(env, regs, value_regno); - } else if (reg->type == PTR_TO_BTF_ID) { + } else if (base_type(reg->type) == PTR_TO_BTF_ID && !(type_flag(reg->type) & ~PTR_UNTRUSTED)) { err = check_ptr_to_btf_access(env, regs, regno, off, size, t, value_regno); } else if (reg->type == CONST_PTR_TO_MAP) { @@ -13041,7 +13063,7 @@ static int convert_ctx_accesses(struct bpf_verifier_env *env) if (!ctx_access) continue; - switch (env->insn_aux_data[i + delta].ptr_type) { + switch ((int)env->insn_aux_data[i + delta].ptr_type) { case PTR_TO_CTX: if (!ops->convert_ctx_access) continue; @@ -13058,6 +13080,7 @@ static int convert_ctx_accesses(struct bpf_verifier_env *env) convert_ctx_access = bpf_xdp_sock_convert_ctx_access; break; case PTR_TO_BTF_ID: + case PTR_TO_BTF_ID | PTR_UNTRUSTED: if (type == BPF_READ) { insn->code = BPF_LDX | BPF_PROBE_MEM | BPF_SIZE((insn)->code); From patchwork Sun Feb 20 13:48:06 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kumar Kartikeya Dwivedi X-Patchwork-Id: 12752710 X-Patchwork-Delegate: bpf@iogearbox.net Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id E5F67C433EF for ; Sun, 20 Feb 2022 13:48:55 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S243903AbiBTNtO (ORCPT ); Sun, 20 Feb 2022 08:49:14 -0500 Received: from mxb-00190b01.gslb.pphosted.com ([23.128.96.19]:49780 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S243889AbiBTNtD (ORCPT ); Sun, 20 Feb 2022 08:49:03 -0500 Received: from mail-pf1-x442.google.com (mail-pf1-x442.google.com [IPv6:2607:f8b0:4864:20::442]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id C8B0921813; Sun, 20 Feb 2022 05:48:41 -0800 (PST) Received: by mail-pf1-x442.google.com with SMTP id i21so6403317pfd.13; Sun, 20 Feb 2022 05:48:41 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=pIC5+WXrEeyIxvYSwThdX0RKAIalXL8BJkc2S7tcfKw=; b=U09A3aHEWLcSZtIQs1bZGdvaq6QvKB6EcaoJk8Il0kaUMuCnQgG3EeKbHAL0uZhhgu 81fImhEsABSb++rzCwIsZUrGFdrl/w3WkH58rxYcxTHxdOOLneZg3YAr++e7fjJA0tJH 0u3yuMB6QmBL/UxZ7jqUzaBaZeCpv618APzpMfkreTuY7D4BJvmDTDqu/s3axFgKRQkh yzUoV8tDWTs0KGisddQlB24bP/RFJuxCjoX6XN+Kl3e28cuyIKbMa2V+kAdId/AF/VaL ylyltQqaGBF4vjOHiYi28Q3tidValepLrB0eN5fOv2Kdj2r25D0NkHEwRxVsmMIg8N7e U8og== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=pIC5+WXrEeyIxvYSwThdX0RKAIalXL8BJkc2S7tcfKw=; b=hsdMdzQ9S2zSW6h23c/hF7VQDVnSgVNJweKUkZwVOqFKDRKF9IIwpSByYLf0vM6PjL r6+0KSNHrjIPucJ3j+5qmlDjCcPMCSAtMQFDJK1ZwkjymWNcUvm2PaEekKsOAWZ8vVgZ fUVZri7AJTSDV22/o4mwN8pcnfPIkPUjaJg/q1t9rJvB2kWQOrZkwRrFhPA47n3J4f5z 49Vb7S/Mypdq4wPn3GI04z3G5mwqctPmOmWaUihuUxg92XfLDemxRlVFdye21EnOotBm 3326xMa3f3OGqHhxZo1Xikp1/SDO/NBMwgusUYVuylpatv8GNAiCbM7ynJCfBOxZj1jv kOqQ== X-Gm-Message-State: AOAM5312p5QxXCOmfQ1I2eelFG9kQSuLL7YMylf60qoHjjExU5t277DS lypIjTJrW4BaSiaYnh4DJSQTneCmsTk= X-Google-Smtp-Source: ABdhPJzVaiY8SWSJqGrRIImmIAebrxfjydSNtUMRtokoWlWFgeHPxDXgj6g2I+oEAcAkY8wnUxAEGw== X-Received: by 2002:a65:4c0f:0:b0:373:f389:b7e0 with SMTP id u15-20020a654c0f000000b00373f389b7e0mr7104280pgq.411.1645364921086; Sun, 20 Feb 2022 05:48:41 -0800 (PST) Received: from localhost ([2405:201:6014:d0c0:6243:316e:a9e1:adda]) by smtp.gmail.com with ESMTPSA id g20sm9526597pfj.64.2022.02.20.05.48.40 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 20 Feb 2022 05:48:40 -0800 (PST) From: Kumar Kartikeya Dwivedi To: bpf@vger.kernel.org Cc: Alexei Starovoitov , Andrii Nakryiko , Daniel Borkmann , =?utf-8?q?Toke_H=C3=B8iland-J=C3=B8?= =?utf-8?q?rgensen?= , Jesper Dangaard Brouer , netfilter-devel@vger.kernel.org, netdev@vger.kernel.org Subject: [PATCH bpf-next v1 08/15] bpf: Adapt copy_map_value for multiple offset case Date: Sun, 20 Feb 2022 19:18:06 +0530 Message-Id: <20220220134813.3411982-9-memxor@gmail.com> X-Mailer: git-send-email 2.35.1 In-Reply-To: <20220220134813.3411982-1-memxor@gmail.com> References: <20220220134813.3411982-1-memxor@gmail.com> MIME-Version: 1.0 X-Developer-Signature: v=1; a=openpgp-sha256; l=5506; h=from:subject; bh=QC3eOPhRZ70Vfv0ZVgnLYlIXFjYgOARRsEoQm0vCP0c=; b=owEBbQKS/ZANAwAIAUzgyIZIvxHKAcsmYgBiEkZX8enrcy5zjuM51gXuZUW/extU3HrWA0LHh0RX sZCmGKKJAjMEAAEIAB0WIQRLvip+Buz51YI8YRFM4MiGSL8RygUCYhJGVwAKCRBM4MiGSL8RyudYD/ 4+0UiSISVfkfKRAa6IDsSX+iz6hWQgpXqBpxv2NMjJbQrxDmhk8/89wG/4gfFXl0WH8uLHalZsgbzs ttYc3Pi+/Hve9coTHxnli32ex5dQzRwpl7YbZJDKcwgr2zL/dSxLp4wl5RNrCpRZC8LNI4FMBbNLhB CwOz4Xv5HRvqTidT9lEFP4qAs2bI+FdC/U1ZabHtOXOMiLly77NXPIZkUI/sCdJ/L2fWUMk6XuWGDJ 4Pc7z/wCXTaIwsAvS2RlnhCT9aroZU5PXTolVL8ok/DgJLlVEwsdAud8Jlp0tZnSZotni2V1CqYQhH i3Q6zrQJHOhJ3b0FNEs6LluKjqmxjjzhxsns3yy81t2LW2w+xhrvOEVVf3KhDsVQ4vdRIEzgNoPP2c sDhKYVMCks9ViyZ8fMzFFXx5xGqPhFVmHe8Otuvc7UbcBNrBuyozNsKzi9VxSZxOx4TAVUb7ddeUKd cGZTtGQdleE4gYPr/SbkOMB+rtcfWB3ZhxrepvVwRHpoinTZTqcb5YRC53sgOGfKsYxcP98ePqEarD SWY3wTAwyvg93CrMgTfSL5m2GVUC1yhtmixv62g777r+13GHA8DIlQbImXTlIBc1Oxd2jY9zJJ1TZv wN264PZEcjPRZ0CTmqdbUTazUWarPGIKuyvvERrGyIdo9c9d4m2jMHOCqsNQ== X-Developer-Key: i=memxor@gmail.com; a=openpgp; fpr=4BBE2A7E06ECF9D5823C61114CE0C88648BF11CA Precedence: bulk List-ID: X-Mailing-List: bpf@vger.kernel.org X-Patchwork-Delegate: bpf@iogearbox.net The changes in this patch deserve closer look, so it has been split into its own independent patch. While earlier we just had to skip two objects at most while copying in and out of map, now we have potentially many objects (at most 8 + 2 = 10, due to the BPF_MAP_VALUE_OFF_MAX limit). Hence, divide the copy_map_value function into an inlined fast path and function call to slowpath. The slowpath handles the case of > 3 offsets, while we handle the most common cases (0, 1, 2, or 3 offsets) in the inline function itself. In copy_map_value_slow, we use 11 offsets, just to make the for loop that copies the value free of edge cases for the last offset, by using map->value_size as final offset to subtract remaining area to copy from. Signed-off-by: Kumar Kartikeya Dwivedi --- include/linux/bpf.h | 43 +++++++++++++++++++++++++++++++--- kernel/bpf/syscall.c | 55 ++++++++++++++++++++++++++++++++++++++++++++ 2 files changed, 95 insertions(+), 3 deletions(-) diff --git a/include/linux/bpf.h b/include/linux/bpf.h index ae599aaf8d4c..5d845ca02eba 100644 --- a/include/linux/bpf.h +++ b/include/linux/bpf.h @@ -253,12 +253,22 @@ static inline void check_and_init_map_value(struct bpf_map *map, void *dst) memset(dst + map->spin_lock_off, 0, sizeof(struct bpf_spin_lock)); if (unlikely(map_value_has_timer(map))) memset(dst + map->timer_off, 0, sizeof(struct bpf_timer)); + if (unlikely(map_value_has_ptr_to_btf_id(map))) { + struct bpf_map_value_off *tab = map->ptr_off_tab; + int i; + + for (i = 0; i < tab->nr_off; i++) + *(u64 *)(dst + tab->off[i].offset) = 0; + } } +void copy_map_value_slow(struct bpf_map *map, void *dst, void *src, u32 s_off, + u32 s_sz, u32 t_off, u32 t_sz); + /* copy everything but bpf_spin_lock and bpf_timer. There could be one of each. */ static inline void copy_map_value(struct bpf_map *map, void *dst, void *src) { - u32 s_off = 0, s_sz = 0, t_off = 0, t_sz = 0; + u32 s_off = 0, s_sz = 0, t_off = 0, t_sz = 0, p_off = 0, p_sz = 0; if (unlikely(map_value_has_spin_lock(map))) { s_off = map->spin_lock_off; @@ -268,13 +278,40 @@ static inline void copy_map_value(struct bpf_map *map, void *dst, void *src) t_off = map->timer_off; t_sz = sizeof(struct bpf_timer); } + /* Multiple offset case is slow, offload to function */ + if (unlikely(map_value_has_ptr_to_btf_id(map))) { + struct bpf_map_value_off *tab = map->ptr_off_tab; + + /* Inline the likely common case */ + if (likely(tab->nr_off == 1)) { + p_off = tab->off[0].offset; + p_sz = sizeof(u64); + } else { + copy_map_value_slow(map, dst, src, s_off, s_sz, t_off, t_sz); + return; + } + } + + if (unlikely(s_sz || t_sz || p_sz)) { + /* The order is p_off, t_off, s_off, use insertion sort */ - if (unlikely(s_sz || t_sz)) { + if (t_off < p_off || !t_sz) { + swap(t_off, p_off); + swap(t_sz, p_sz); + } if (s_off < t_off || !s_sz) { swap(s_off, t_off); swap(s_sz, t_sz); + if (t_off < p_off || !t_sz) { + swap(t_off, p_off); + swap(t_sz, p_sz); + } } - memcpy(dst, src, t_off); + + memcpy(dst, src, p_off); + memcpy(dst + p_off + p_sz, + src + p_off + p_sz, + t_off - p_off - p_sz); memcpy(dst + t_off + t_sz, src + t_off + t_sz, s_off - t_off - t_sz); diff --git a/kernel/bpf/syscall.c b/kernel/bpf/syscall.c index beb96866f34d..83d71d6912f5 100644 --- a/kernel/bpf/syscall.c +++ b/kernel/bpf/syscall.c @@ -30,6 +30,7 @@ #include #include #include +#include #include #include #include @@ -230,6 +231,60 @@ static int bpf_map_update_value(struct bpf_map *map, struct fd f, void *key, return err; } +static int copy_map_value_cmp(const void *_a, const void *_b) +{ + const u32 a = *(const u32 *)_a; + const u32 b = *(const u32 *)_b; + + /* We only need to sort based on offset */ + if (a < b) + return -1; + else if (a > b) + return 1; + return 0; +} + +void copy_map_value_slow(struct bpf_map *map, void *dst, void *src, u32 s_off, + u32 s_sz, u32 t_off, u32 t_sz) +{ + struct bpf_map_value_off *tab = map->ptr_off_tab; /* already set to non-NULL */ + /* 3 = 2 for bpf_timer, bpf_spin_lock, 1 for map->value_size sentinel */ + struct { + u32 off; + u32 sz; + } off_arr[BPF_MAP_VALUE_OFF_MAX + 3]; + int i, cnt = 0; + + /* Reconsider stack usage when bumping BPF_MAP_VALUE_OFF_MAX */ + BUILD_BUG_ON(sizeof(off_arr) != 88); + + for (i = 0; i < tab->nr_off; i++) { + off_arr[cnt].off = tab->off[i].offset; + off_arr[cnt++].sz = sizeof(u64); + } + if (s_sz) { + off_arr[cnt].off = s_off; + off_arr[cnt++].sz = s_sz; + } + if (t_sz) { + off_arr[cnt].off = t_off; + off_arr[cnt++].sz = t_sz; + } + off_arr[cnt].off = map->value_size; + + sort(off_arr, cnt, sizeof(off_arr[0]), copy_map_value_cmp, NULL); + + /* There is always at least one element */ + memcpy(dst, src, off_arr[0].off); + /* Copy the rest, while skipping other regions */ + for (i = 1; i < cnt; i++) { + u32 curr_off = off_arr[i - 1].off + off_arr[i - 1].sz; + u32 next_off = off_arr[i].off; + + memcpy(dst + curr_off, src + curr_off, next_off - curr_off); + } +} + static int bpf_map_copy_value(struct bpf_map *map, void *key, void *value, __u64 flags) { From patchwork Sun Feb 20 13:48:07 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kumar Kartikeya Dwivedi X-Patchwork-Id: 12752711 X-Patchwork-Delegate: bpf@iogearbox.net Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id B5080C433F5 for ; Sun, 20 Feb 2022 13:48:58 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S243897AbiBTNtQ (ORCPT ); Sun, 20 Feb 2022 08:49:16 -0500 Received: from mxb-00190b01.gslb.pphosted.com ([23.128.96.19]:49894 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S243893AbiBTNtF (ORCPT ); Sun, 20 Feb 2022 08:49:05 -0500 Received: from mail-pj1-x1043.google.com (mail-pj1-x1043.google.com [IPv6:2607:f8b0:4864:20::1043]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id B0BEA2D1EA; Sun, 20 Feb 2022 05:48:44 -0800 (PST) Received: by mail-pj1-x1043.google.com with SMTP id v8-20020a17090a634800b001bb78857ccdso14208793pjs.1; Sun, 20 Feb 2022 05:48:44 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=tRgersc7+YiHsFoqjkDw4Ohre3w5vPK2wp9oy5KJBzI=; b=gmMjHzzthS+Pc8+pZwqt44EjB3kst7Y1OcDKz4vCjRs/4jNcP0oHae8quxqX+cEQOx Fw1T9wpe9qXc6KuK8FmuKfFy5nbzy0p1uQ4MMspodbiz6o9hmEBtGDaDAx7BiOHEWsz0 HgAF14kep+RfALD8EewfXDs5InzNL7j1ewIaVm3isMo3tJkNx+CarcakDIeOouyq5/94 bHYZh8ADNFJ2F/JP0gK5ZZVyAUachPhf4c+9W5J8jIRdfhusoHh8ikH0I/p2eRp1imkP tMW9OYTR/8tY6D+YFTb3h3Vr94lCbiboBDquHRtf5DPTjugsaZ2CnYkiOMFDsBMo4kII XFjw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=tRgersc7+YiHsFoqjkDw4Ohre3w5vPK2wp9oy5KJBzI=; b=HeQNSwoN96QobCzp+/hfx3G8PKg9zIyi78HLRwtDXjXw8Adp4nXyQPlM2gV8a/hJMF DwiCGbfLcKO5S4QorquqocUHFMnnMs5nFY+qFuA33vzd5aKWq7cwyRv4EBrzV6E/dSkq C4tlTY0a3ebBaCD2eNl5vSTnzYkp89XusQ7mGPkPNmqBQVqz91YRiKqhzjefUQ+arPyo Hw/hHZxKN3G5MCI1UjKqS/5XnKwgUjXuNY5zcM6eEMkCOG+Zv+ewX1Ix2aLRt0IrJ3Kf jC9ibPgavP+GOljIctG0WDBeUC9kLur/scB9g2P87L3axlVwqnKIN0RdEavSqReX43RN DZ0g== X-Gm-Message-State: AOAM5335nwMfa49zv2a1B4oTMdU9ikS3b9jasydCFFSTMhz+4l8hF0/R 4cnbFCKiMXHWjfp6c5unvJQJ3YC4iJo= X-Google-Smtp-Source: ABdhPJyOETrW6jI5qRGlR7NC506DjIBk1zKstL63YLIIcRI89gS37xcbOIJ4hYWpCBzWjCoGkNp0hw== X-Received: by 2002:a17:902:8497:b0:14f:919:9fdd with SMTP id c23-20020a170902849700b0014f09199fddmr15431878plo.52.1645364924085; Sun, 20 Feb 2022 05:48:44 -0800 (PST) Received: from localhost ([2405:201:6014:d0c0:6243:316e:a9e1:adda]) by smtp.gmail.com with ESMTPSA id q9sm9921393pfk.31.2022.02.20.05.48.43 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 20 Feb 2022 05:48:43 -0800 (PST) From: Kumar Kartikeya Dwivedi To: bpf@vger.kernel.org Cc: Alexei Starovoitov , Andrii Nakryiko , Daniel Borkmann , =?utf-8?q?Toke_H=C3=B8iland-J=C3=B8?= =?utf-8?q?rgensen?= , Jesper Dangaard Brouer , netfilter-devel@vger.kernel.org, netdev@vger.kernel.org Subject: [PATCH bpf-next v1 09/15] bpf: Populate pairs of btf_id and destructor kfunc in btf Date: Sun, 20 Feb 2022 19:18:07 +0530 Message-Id: <20220220134813.3411982-10-memxor@gmail.com> X-Mailer: git-send-email 2.35.1 In-Reply-To: <20220220134813.3411982-1-memxor@gmail.com> References: <20220220134813.3411982-1-memxor@gmail.com> MIME-Version: 1.0 X-Developer-Signature: v=1; a=openpgp-sha256; l=6298; h=from:subject; bh=bEtsOEx3UKJJcW3J592imsq8QQaOzR84HSnwrs6Zyzo=; b=owEBbQKS/ZANAwAIAUzgyIZIvxHKAcsmYgBiEkZY0debdctwSyfadfikEkSTZaceVwjMv9F6SfUe R4rLvMKJAjMEAAEIAB0WIQRLvip+Buz51YI8YRFM4MiGSL8RygUCYhJGWAAKCRBM4MiGSL8RymWLD/ 4kKQgF/G8Wx42sO4kS/V3jpxqvYE5QJIbwOrmxPXwqD5WuFUn1/Eq1cmaBdfwZ3bhxhTMxzuD923a9 ZLWRotwtCxo5BpyvVH0la4PckOrGvKqPISQbLNGzjt2ZQ7dHpi7QOcX4yrUggm+YN6G6NUBwi0Fj6W 5Eq9Fg6QdfVHep/V8Y6HdR7QfZ7ARw0isopS/3yGqvMrQjxciTinJveVqlXoe1dDAxcOG4kJ/B/qcG AI5KEt5hseY0ESGJU72iX2sEQouljLIOIowIyQkrGBBTq+qmXOHAlGc3L3eYFOmJ403zwE5nyMXQR4 OEvNqNqf9dCu/Hczet7DWBs6roQxPJ4UxUlJ1gsisUc2HHLc2PeIJOIQql7fLOUlEFBbFVGVx+wupt NIYzPruSd3DVj0SmSg8Jy0nVQte4Bt+aj68xlisWrGoRE1z2dzWI/kzqqjGD2m+kCAyx42wConwiV7 /c6rUZSOAZ4gxteVnkvoBuEmzOZlxcRdk/3UB7dqWtsWOg4eu1V3QrPJlnkxLfw2f9w76mIKFdTOIi s0mhmSJwKJe5uLq6eoDCnJePvktVbynmfJa+AyJlZEOJW/bIOmF9YQFedNJJHnM4wuAr7UOPbdX18m wdzwv2r6r1s29Bw5lrxdQxrqdRTqhkWWyicnNlzBAA6eDShpZXPbqcORQwbA== X-Developer-Key: i=memxor@gmail.com; a=openpgp; fpr=4BBE2A7E06ECF9D5823C61114CE0C88648BF11CA Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org X-Patchwork-Delegate: bpf@iogearbox.net To support storing referenced PTR_TO_BTF_ID in maps, we require associating a specific BTF ID with a 'destructor' kfunc. This is because we need to release a live referenced pointer at a certain offset in map value from the map destruction path, otherwise we end up leaking resources. Hence, introduce support for passing an array of btf_id, kfunc_btf_id pairs that denote a BTF ID and its associated release function. Then, add an accessor 'btf_find_dtor_kfunc' which can be used to look up the destructor kfunc of a certain BTF ID. If found, we can use it to free the object from the map free path. The registration of these pairs also serve as a whitelist of structures which are allowed as referenced PTR_TO_BTF_ID in a BPF map, because without finding the destructor kfunc, we will bail and return an error. Signed-off-by: Kumar Kartikeya Dwivedi --- include/linux/btf.h | 17 +++++++ kernel/bpf/btf.c | 109 ++++++++++++++++++++++++++++++++++++++++++++ 2 files changed, 126 insertions(+) diff --git a/include/linux/btf.h b/include/linux/btf.h index 6592183aeb23..a304a1ea39d9 100644 --- a/include/linux/btf.h +++ b/include/linux/btf.h @@ -41,6 +41,11 @@ struct btf_kfunc_id_set { }; }; +struct btf_id_dtor_kfunc { + u32 btf_id; + u32 kfunc_btf_id; +}; + extern const struct file_operations btf_fops; void btf_get(struct btf *btf); @@ -347,6 +352,9 @@ bool btf_kfunc_id_set_contains(const struct btf *btf, enum btf_kfunc_type type, u32 kfunc_btf_id); int register_btf_kfunc_id_set(enum bpf_prog_type prog_type, const struct btf_kfunc_id_set *s); +s32 btf_find_dtor_kfunc(struct btf *btf, u32 btf_id); +int register_btf_id_dtor_kfuncs(const struct btf_id_dtor_kfunc *dtors, u32 add_cnt, + struct module *owner); #else static inline const struct btf_type *btf_type_by_id(const struct btf *btf, u32 type_id) @@ -370,6 +378,15 @@ static inline int register_btf_kfunc_id_set(enum bpf_prog_type prog_type, { return 0; } +static inline s32 btf_find_dtor_kfunc(struct btf *btf, u32 btf_id) +{ + return -ENOENT; +} +static inline int register_btf_id_dtor_kfuncs(const struct btf_id_dtor_kfunc *dtors, + u32 add_cnt, struct module *owner) +{ + return 0; +} #endif #endif diff --git a/kernel/bpf/btf.c b/kernel/bpf/btf.c index bafceae90c32..8a6ec1847f17 100644 --- a/kernel/bpf/btf.c +++ b/kernel/bpf/btf.c @@ -207,12 +207,18 @@ enum btf_kfunc_hook { enum { BTF_KFUNC_SET_MAX_CNT = 32, + BTF_DTOR_KFUNC_MAX_CNT = 256, }; struct btf_kfunc_set_tab { struct btf_id_set *sets[BTF_KFUNC_HOOK_MAX][BTF_KFUNC_TYPE_MAX]; }; +struct btf_id_dtor_kfunc_tab { + u32 cnt; + struct btf_id_dtor_kfunc dtors[]; +}; + struct btf { void *data; struct btf_type **types; @@ -228,6 +234,7 @@ struct btf { u32 id; struct rcu_head rcu; struct btf_kfunc_set_tab *kfunc_set_tab; + struct btf_id_dtor_kfunc_tab *dtor_kfunc_tab; /* split BTF support */ struct btf *base_btf; @@ -1572,8 +1579,19 @@ static void btf_free_kfunc_set_tab(struct btf *btf) btf->kfunc_set_tab = NULL; } +static void btf_free_dtor_kfunc_tab(struct btf *btf) +{ + struct btf_id_dtor_kfunc_tab *tab = btf->dtor_kfunc_tab; + + if (!tab) + return; + kfree(tab); + btf->dtor_kfunc_tab = NULL; +} + static void btf_free(struct btf *btf) { + btf_free_dtor_kfunc_tab(btf); btf_free_kfunc_set_tab(btf); kvfree(btf->types); kvfree(btf->resolved_sizes); @@ -7037,6 +7055,97 @@ int register_btf_kfunc_id_set(enum bpf_prog_type prog_type, } EXPORT_SYMBOL_GPL(register_btf_kfunc_id_set); +s32 btf_find_dtor_kfunc(struct btf *btf, u32 btf_id) +{ + struct btf_id_dtor_kfunc_tab *tab = btf->dtor_kfunc_tab; + struct btf_id_dtor_kfunc *dtor; + + if (!tab) + return -ENOENT; + /* Even though the size of tab->dtors[0] is > sizeof(u32), we only need + * to compare the first u32 with btf_id, so we can reuse btf_id_cmp_func. + */ + BUILD_BUG_ON(offsetof(struct btf_id_dtor_kfunc, btf_id) != 0); + dtor = bsearch(&btf_id, tab->dtors, tab->cnt, sizeof(tab->dtors[0]), btf_id_cmp_func); + if (!dtor) + return -ENOENT; + return dtor->kfunc_btf_id; +} + +/* This function must be invoked only from initcalls/module init functions */ +int register_btf_id_dtor_kfuncs(const struct btf_id_dtor_kfunc *dtors, u32 add_cnt, + struct module *owner) +{ + struct btf_id_dtor_kfunc_tab *tab; + struct btf *btf; + u32 tab_cnt; + int ret; + + btf = btf_get_module_btf(owner); + if (!btf) { + if (!owner && IS_ENABLED(CONFIG_DEBUG_INFO_BTF)) { + pr_err("missing vmlinux BTF, cannot register dtor kfuncs\n"); + return -ENOENT; + } + if (owner && IS_ENABLED(CONFIG_DEBUG_INFO_BTF_MODULES)) { + pr_err("missing module BTF, cannot register dtor kfuncs\n"); + return -ENOENT; + } + return 0; + } + if (IS_ERR(btf)) + return PTR_ERR(btf); + + if (add_cnt >= BTF_DTOR_KFUNC_MAX_CNT) { + pr_err("cannot register more than %d kfunc destructors\n", BTF_DTOR_KFUNC_MAX_CNT); + ret = -E2BIG; + goto end; + } + + tab = btf->dtor_kfunc_tab; + /* Only one call allowed for modules */ + if (WARN_ON_ONCE(tab && btf_is_module(btf))) { + ret = -EINVAL; + goto end; + } + + tab_cnt = tab ? tab->cnt : 0; + if (tab_cnt > U32_MAX - add_cnt) { + ret = -EOVERFLOW; + goto end; + } + if (tab_cnt + add_cnt >= BTF_DTOR_KFUNC_MAX_CNT) { + pr_err("cannot register more than %d kfunc destructors\n", BTF_DTOR_KFUNC_MAX_CNT); + ret = -E2BIG; + goto end; + } + + tab = krealloc(btf->dtor_kfunc_tab, + offsetof(struct btf_id_dtor_kfunc_tab, dtors[tab_cnt + add_cnt]), + GFP_KERNEL | __GFP_NOWARN); + if (!tab) { + ret = -ENOMEM; + goto end; + } + + if (!btf->dtor_kfunc_tab) + tab->cnt = 0; + btf->dtor_kfunc_tab = tab; + + memcpy(tab->dtors + tab->cnt, dtors, add_cnt * sizeof(tab->dtors[0])); + tab->cnt += add_cnt; + + sort(tab->dtors, tab->cnt, sizeof(tab->dtors[0]), btf_id_cmp_func, NULL); + + return 0; +end: + btf_free_dtor_kfunc_tab(btf); + if (btf_is_module(btf)) + btf_put(btf); + return ret; +} +EXPORT_SYMBOL_GPL(register_btf_id_dtor_kfuncs); + #define MAX_TYPES_ARE_COMPAT_DEPTH 2 static From patchwork Sun Feb 20 13:48:08 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kumar Kartikeya Dwivedi X-Patchwork-Id: 12752714 X-Patchwork-Delegate: bpf@iogearbox.net Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id CECCAC433EF for ; Sun, 20 Feb 2022 13:49:02 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S243917AbiBTNtV (ORCPT ); Sun, 20 Feb 2022 08:49:21 -0500 Received: from mxb-00190b01.gslb.pphosted.com ([23.128.96.19]:50478 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S243902AbiBTNtO (ORCPT ); Sun, 20 Feb 2022 08:49:14 -0500 Received: from mail-pf1-x442.google.com (mail-pf1-x442.google.com [IPv6:2607:f8b0:4864:20::442]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 10AD42DD47; Sun, 20 Feb 2022 05:48:47 -0800 (PST) Received: by mail-pf1-x442.google.com with SMTP id g1so6436125pfv.1; Sun, 20 Feb 2022 05:48:47 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=a/IXub8HL5Vl5EcFk4Zp2k+suEKjCBlYWofCgvll/y0=; b=niqeh3XsoWw9ifbDiRtbVAkZRkx2ueEvJ73ov7B8x+np+2bkxY0PAnDu8EJ+EWswTg 6tRnVcoVFGflaQixQ3a90h+FWXXXLMHG+F1FR6ym8k7bqmHCG8NS2lIE0KJqktL+3kZJ MgUlKjt0Z1sAhdbH4RcnMnh23RpUoHAXP/i2JmN0ha5ZSBZRk9cHTtIBTPNG+IFddiO9 wqt+KfB3rPzDGYzppznoGt5gdGLLHzJygNltBDBFAJGXTqSN5wGDUXKJfIp6hoFUGXRJ 65X4YB7I0w0Cube5dcMEu4bdJ5+JkMXR484qUftjpOIhx3R6eB27uxqGYxyuv0jcNTFE /h4Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=a/IXub8HL5Vl5EcFk4Zp2k+suEKjCBlYWofCgvll/y0=; b=ymMXW5SUj4RECQJLuWQElS0HO1YyLHn6XANtzYTJRd6pJZ+hq7c3g7Gw326qei+aZw 308E9yPtZS4U34rSSx73lSlBBLkczrxO0efUpaACe5Vwtx5+4MZWrCroW7rn/A7D4kst ag+0RBA0ifAE9o61+la3x21a7tgGWDHvNQA/9IHbwLkbiI6pgt9PApEp6M10tdvDuLNS 8jZ4cJfF3k11BGDxq5m7A9vCJUmg8Icc3pjv2XI+zHMFA1ikVKtG2VpSj5opPaQUvo03 NPlVGAPYTuE/482fx0ZBGrygFhk2SwBvgpgD22G8T19wM/TO1H4yeZvnPdlnMQIN76Qq mDWQ== X-Gm-Message-State: AOAM532lRfnmtzeOhTjkwA559GzZxWmsXkUbM38yUITvax6WX7uroxGn ZpwgL18ksbimTgWZ+w6JxMlm3ulvDNo= X-Google-Smtp-Source: ABdhPJxoHVpP4Y64pz05vEJMo5+WmR8xS0Z16EuIWREKBvX+U+nma9p9FrSCV7EU7Nd44a/uTeGIig== X-Received: by 2002:a63:571d:0:b0:372:8da2:10e1 with SMTP id l29-20020a63571d000000b003728da210e1mr12880439pgb.271.1645364927086; Sun, 20 Feb 2022 05:48:47 -0800 (PST) Received: from localhost ([2405:201:6014:d0c0:6243:316e:a9e1:adda]) by smtp.gmail.com with ESMTPSA id f11sm2471208pfc.189.2022.02.20.05.48.46 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 20 Feb 2022 05:48:46 -0800 (PST) From: Kumar Kartikeya Dwivedi To: bpf@vger.kernel.org Cc: Alexei Starovoitov , Andrii Nakryiko , Daniel Borkmann , =?utf-8?q?Toke_H=C3=B8iland-J=C3=B8?= =?utf-8?q?rgensen?= , Jesper Dangaard Brouer , netfilter-devel@vger.kernel.org, netdev@vger.kernel.org Subject: [PATCH bpf-next v1 10/15] bpf: Wire up freeing of referenced PTR_TO_BTF_ID in map Date: Sun, 20 Feb 2022 19:18:08 +0530 Message-Id: <20220220134813.3411982-11-memxor@gmail.com> X-Mailer: git-send-email 2.35.1 In-Reply-To: <20220220134813.3411982-1-memxor@gmail.com> References: <20220220134813.3411982-1-memxor@gmail.com> MIME-Version: 1.0 X-Developer-Signature: v=1; a=openpgp-sha256; l=12831; h=from:subject; bh=QBc8MuC6tBr+XK//lUnaSSR8TuHlQJnqJ8GNMM2b2LA=; b=owEBbQKS/ZANAwAIAUzgyIZIvxHKAcsmYgBiEkZYBW7WwJp3BM2RCoair/Htzp08YXfnAmB5SiCP 9wPmTiWJAjMEAAEIAB0WIQRLvip+Buz51YI8YRFM4MiGSL8RygUCYhJGWAAKCRBM4MiGSL8RyoBPEA CzNv2utzMs/YR1DnI2pVTQgu5rmaiAMdNbkN2oGFMgKRXiLvnQ1lmNiw5kSZe9htcTAmvgf1AGMAyI L5Xwwx1SRYYLHQij7tzkeKAxOF8mkfgDgxoXjvMTgVvkjWDQpKggg1xfrtEjAWEWNNtoFfWNY0HmlJ PDd1AkNFIauIDYaxUmLwkFwbCAbiNjLR68oof38MOGgpyN9/vG6OgnXz/RYVKLPD60wbdlY6+5TDRl t5asBB4MPszBsSENfz3QYhteBMeR/0kKMIghC4+Xvy9uSYQw9CT4r0/JLREDviuhBgrAIMcZWpYDcM 1gYdxhrdtMJpJzPk853wlqvw7cXBwzrpRZ565VW6xk14AIMKAM1VGg/JkuTC2wFrkZUsUghxj5rRrg KE6jiIARf/gNHXzRTnJFzzlWQ8jhZo5vny2lCV9a2uV+V8+UhIPQyaOVfje1rWSURMfTLw6i+ub7d6 jf3mhzJnVPpZyX8Bov2Y/oLhBjB13hJQ7Xt9g+MsTLSbddDkeCcSlJ+UvNhnXGaPo3zQQphOcyg2na 2XBqg9exDLoolg8lcs96+xgaPe0ApslGfNLvY5qp9W75ZxswCyIKm2OEph0THVkUuwlpHQ2IeKB8Un C/HVKOwS5ZPoFjyFk8DEbiijfA787+ZRUFGtlORjsvDU0pMW04uqNWUKgsNQ== X-Developer-Key: i=memxor@gmail.com; a=openpgp; fpr=4BBE2A7E06ECF9D5823C61114CE0C88648BF11CA Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org X-Patchwork-Delegate: bpf@iogearbox.net A destructor kfunc can be defined as void func(type *), where type may be void or any other pointer type as per convenience. The kfunc doesn't have to take care about the map side pointer width, as it will be passed a pointer after converting the u64 address embedded in the map. In this patch, we ensure that the type is sane and capture the function pointer into off_desc of ptr_off_tab for the specific pointer offset, with the invariant that the dtor pointer is always set when 'ref' tag is applied to the pointer's pointee type, which is indicated by the flag BPF_MAP_VALUE_OFF_F_REF. Note that only BTF IDs whose destructor kfunc is registered, thus become the allowed BTF IDs for embedding as referenced PTR_TO_BTF_ID. Hence btf_find_dtor_kfunc serves the purpose of finding dtor kfunc BTF ID, as well acting as a check against the whitelist of allowed BTF IDs for this purpose. Finally, wire up the actual freeing of the referenced pointer if any at all available offsets, so that no references are leaked after the BPF map goes away and the BPF program previously moved the ownership a referenced pointer into it. The behavior is similar to BPF timers, where bpf_map_{update,delete}_elem will free any existing referenced PTR_TO_BTF_ID. The same case is with LRU map's bpf_lru_push_free/htab_lru_push_free functions, which are extended to reset and free referenced pointers. Signed-off-by: Kumar Kartikeya Dwivedi Reported-by: kernel test robot Reported-by: kernel test robot Reported-by: kernel test robot --- include/linux/bpf.h | 3 ++ include/linux/btf.h | 2 ++ kernel/bpf/arraymap.c | 13 ++++++-- kernel/bpf/btf.c | 72 ++++++++++++++++++++++++++++++++++++++++++- kernel/bpf/hashtab.c | 27 ++++++++++------ kernel/bpf/syscall.c | 37 ++++++++++++++++++++-- 6 files changed, 139 insertions(+), 15 deletions(-) diff --git a/include/linux/bpf.h b/include/linux/bpf.h index 5d845ca02eba..744f1886cf91 100644 --- a/include/linux/bpf.h +++ b/include/linux/bpf.h @@ -23,6 +23,7 @@ #include #include #include +#include struct bpf_verifier_env; struct bpf_verifier_log; @@ -171,6 +172,7 @@ struct bpf_map_value_off_desc { u32 btf_id; struct btf *btf; struct module *module; + btf_dtor_kfunc_t dtor; /* only set when flags & BPF_MAP_VALUE_OFF_F_REF is true */ int flags; }; @@ -1568,6 +1570,7 @@ struct bpf_map_value_off_desc *bpf_map_ptr_off_contains(struct bpf_map *map, u32 void bpf_map_free_ptr_off_tab(struct bpf_map *map); struct bpf_map_value_off *bpf_map_copy_ptr_off_tab(const struct bpf_map *map); bool bpf_map_equal_ptr_off_tab(const struct bpf_map *map_a, const struct bpf_map *map_b); +void bpf_map_free_ptr_to_btf_id(struct bpf_map *map, void *map_value); struct bpf_map *bpf_map_get(u32 ufd); struct bpf_map *bpf_map_get_with_uref(u32 ufd); diff --git a/include/linux/btf.h b/include/linux/btf.h index a304a1ea39d9..c7e75be9637f 100644 --- a/include/linux/btf.h +++ b/include/linux/btf.h @@ -46,6 +46,8 @@ struct btf_id_dtor_kfunc { u32 kfunc_btf_id; }; +typedef void (*btf_dtor_kfunc_t)(void *); + extern const struct file_operations btf_fops; void btf_get(struct btf *btf); diff --git a/kernel/bpf/arraymap.c b/kernel/bpf/arraymap.c index 7f145aefbff8..de4baca3edd7 100644 --- a/kernel/bpf/arraymap.c +++ b/kernel/bpf/arraymap.c @@ -287,10 +287,12 @@ static int array_map_get_next_key(struct bpf_map *map, void *key, void *next_key return 0; } -static void check_and_free_timer_in_array(struct bpf_array *arr, void *val) +static void check_and_free_timer_and_ptr_in_array(struct bpf_array *arr, void *val) { if (unlikely(map_value_has_timer(&arr->map))) bpf_timer_cancel_and_free(val + arr->map.timer_off); + if (unlikely(map_value_has_ptr_to_btf_id(&arr->map))) + bpf_map_free_ptr_to_btf_id(&arr->map, val); } /* Called from syscall or from eBPF program */ @@ -327,7 +329,7 @@ static int array_map_update_elem(struct bpf_map *map, void *key, void *value, copy_map_value_locked(map, val, value, false); else copy_map_value(map, val, value); - check_and_free_timer_in_array(array, val); + check_and_free_timer_and_ptr_in_array(array, val); } return 0; } @@ -398,6 +400,13 @@ static void array_map_free_timers(struct bpf_map *map) static void array_map_free(struct bpf_map *map) { struct bpf_array *array = container_of(map, struct bpf_array, map); + int i; + + if (unlikely(map_value_has_ptr_to_btf_id(map))) { + for (i = 0; i < array->map.max_entries; i++) + bpf_map_free_ptr_to_btf_id(map, array->value + array->elem_size * i); + bpf_map_free_ptr_off_tab(map); + } if (array->map.map_type == BPF_MAP_TYPE_PERCPU_ARRAY) bpf_array_free_percpu(array); diff --git a/kernel/bpf/btf.c b/kernel/bpf/btf.c index 8a6ec1847f17..f322967da54b 100644 --- a/kernel/bpf/btf.c +++ b/kernel/bpf/btf.c @@ -3170,7 +3170,7 @@ static int btf_find_field_kptr(const struct btf *btf, const struct btf_type *t, int nr_off, ret, flags = 0; struct module *mod = NULL; struct btf *kernel_btf; - s32 id; + s32 id, dtor_btf_id; /* For PTR, sz is always == 8 */ if (!btf_type_is_ptr(t)) @@ -3291,9 +3291,79 @@ static int btf_find_field_kptr(const struct btf *btf, const struct btf_type *t, tab->off[nr_off].btf = kernel_btf; tab->off[nr_off].module = mod; tab->off[nr_off].flags = flags; + + /* Find and stash the function pointer for the destruction function that + * needs to be eventually invoked from the map free path. + * + * Note that we already took module reference, and the map free path + * always invoked the destructor for BTF ID before freeing ptr_off_tab, + * so calling the function should be safe in that context. + */ + if (ref_tag) { + const struct btf_type *dtor_func, *dtor_func_proto, *t; + const struct btf_param *args; + const char *dtor_func_name; + unsigned long addr; + u32 nr_args; + + /* This call also serves as a whitelist of allowed objects that + * can be used as a referenced pointer and be stored in a map at + * the same time. + */ + dtor_btf_id = btf_find_dtor_kfunc(kernel_btf, id); + if (dtor_btf_id < 0) { + ret = dtor_btf_id; + goto end_mod; + } + + dtor_func = btf_type_by_id(kernel_btf, dtor_btf_id); + if (!dtor_func || !btf_type_is_func(dtor_func)) { + ret = -EINVAL; + goto end_mod; + } + + dtor_func_proto = btf_type_by_id(kernel_btf, dtor_func->type); + if (!dtor_func_proto || !btf_type_is_func_proto(dtor_func_proto)) { + ret = -EINVAL; + goto end_mod; + } + + /* Make sure the prototype of the destructor kfunc is 'void func(type *)' */ + t = btf_type_by_id(kernel_btf, dtor_func_proto->type); + if (!t || !btf_type_is_void(t)) { + ret = -EINVAL; + goto end_mod; + } + + nr_args = btf_type_vlen(dtor_func_proto); + args = btf_params(dtor_func_proto); + + t = NULL; + if (nr_args) + t = btf_type_by_id(kernel_btf, args[0].type); + /* Allow any pointer type, as width on targets Linux supports + * will be same for all pointer types (i.e. sizeof(void *)) + */ + if (nr_args != 1 || !t || !btf_type_is_ptr(t)) { + ret = -EINVAL; + goto end_mod; + } + + dtor_func_name = __btf_name_by_offset(kernel_btf, dtor_func->name_off); + addr = kallsyms_lookup_name(dtor_func_name); + if (!addr) { + ret = -EINVAL; + goto end_mod; + } + tab->off[nr_off].dtor = (void *)addr; + } + tab->nr_off++; return 0; +end_mod: + if (mod) + module_put(mod); end_btf: /* Reference is only raised for module BTF */ if (btf_is_module(kernel_btf)) diff --git a/kernel/bpf/hashtab.c b/kernel/bpf/hashtab.c index d29af9988f37..3c33b58e8d3e 100644 --- a/kernel/bpf/hashtab.c +++ b/kernel/bpf/hashtab.c @@ -725,12 +725,15 @@ static int htab_lru_map_gen_lookup(struct bpf_map *map, return insn - insn_buf; } -static void check_and_free_timer(struct bpf_htab *htab, struct htab_elem *elem) +static void check_and_free_timer_and_ptr(struct bpf_htab *htab, + struct htab_elem *elem, bool free_ptr) { + void *map_value = elem->key + round_up(htab->map.key_size, 8); + if (unlikely(map_value_has_timer(&htab->map))) - bpf_timer_cancel_and_free(elem->key + - round_up(htab->map.key_size, 8) + - htab->map.timer_off); + bpf_timer_cancel_and_free(map_value + htab->map.timer_off); + if (unlikely(map_value_has_ptr_to_btf_id(&htab->map)) && free_ptr) + bpf_map_free_ptr_to_btf_id(&htab->map, map_value); } /* It is called from the bpf_lru_list when the LRU needs to delete @@ -757,7 +760,7 @@ static bool htab_lru_map_delete_node(void *arg, struct bpf_lru_node *node) hlist_nulls_for_each_entry_rcu(l, n, head, hash_node) if (l == tgt_l) { hlist_nulls_del_rcu(&l->hash_node); - check_and_free_timer(htab, l); + check_and_free_timer_and_ptr(htab, l, true); break; } @@ -829,7 +832,7 @@ static void htab_elem_free(struct bpf_htab *htab, struct htab_elem *l) { if (htab->map.map_type == BPF_MAP_TYPE_PERCPU_HASH) free_percpu(htab_elem_get_ptr(l, htab->map.key_size)); - check_and_free_timer(htab, l); + check_and_free_timer_and_ptr(htab, l, true); kfree(l); } @@ -857,7 +860,7 @@ static void free_htab_elem(struct bpf_htab *htab, struct htab_elem *l) htab_put_fd_value(htab, l); if (htab_is_prealloc(htab)) { - check_and_free_timer(htab, l); + check_and_free_timer_and_ptr(htab, l, true); __pcpu_freelist_push(&htab->freelist, &l->fnode); } else { atomic_dec(&htab->count); @@ -1104,7 +1107,7 @@ static int htab_map_update_elem(struct bpf_map *map, void *key, void *value, if (!htab_is_prealloc(htab)) free_htab_elem(htab, l_old); else - check_and_free_timer(htab, l_old); + check_and_free_timer_and_ptr(htab, l_old, true); } ret = 0; err: @@ -1114,7 +1117,7 @@ static int htab_map_update_elem(struct bpf_map *map, void *key, void *value, static void htab_lru_push_free(struct bpf_htab *htab, struct htab_elem *elem) { - check_and_free_timer(htab, elem); + check_and_free_timer_and_ptr(htab, elem, true); bpf_lru_push_free(&htab->lru, &elem->lru_node); } @@ -1420,7 +1423,10 @@ static void htab_free_malloced_timers(struct bpf_htab *htab) struct htab_elem *l; hlist_nulls_for_each_entry(l, n, head, hash_node) - check_and_free_timer(htab, l); + /* We are called from map_release_uref, so we don't free + * ref'd pointers. + */ + check_and_free_timer_and_ptr(htab, l, false); cond_resched_rcu(); } rcu_read_unlock(); @@ -1458,6 +1464,7 @@ static void htab_map_free(struct bpf_map *map) else prealloc_destroy(htab); + bpf_map_free_ptr_off_tab(map); free_percpu(htab->extra_elems); bpf_map_area_free(htab->buckets); for (i = 0; i < HASHTAB_MAP_LOCK_COUNT; i++) diff --git a/kernel/bpf/syscall.c b/kernel/bpf/syscall.c index 83d71d6912f5..925e8c615ad2 100644 --- a/kernel/bpf/syscall.c +++ b/kernel/bpf/syscall.c @@ -638,15 +638,48 @@ bool bpf_map_equal_ptr_off_tab(const struct bpf_map *map_a, const struct bpf_map return !memcmp(tab_a, tab_b, size); } +/* Caller must ensure map_value_has_ptr_to_btf_id is true. Note that this + * function can be called on a map value while the map_value is visible to BPF + * programs, as it ensures the correct synchronization, and we already enforce + * the same using the verifier on the BPF program side, esp. for referenced + * pointers. + */ +void bpf_map_free_ptr_to_btf_id(struct bpf_map *map, void *map_value) +{ + struct bpf_map_value_off *tab = map->ptr_off_tab; + u64 *btf_id_ptr; + int i; + + for (i = 0; i < tab->nr_off; i++) { + struct bpf_map_value_off_desc *off_desc = &tab->off[i]; + u64 old_ptr; + + btf_id_ptr = map_value + off_desc->offset; + if (!(off_desc->flags & BPF_MAP_VALUE_OFF_F_REF)) { + /* On 32-bit platforms, WRITE_ONCE 64-bit store tearing + * into two 32-bit stores is fine for us, as we only + * permit pointer values to be stored at this address, + * which are word sized, so the other half of 64-bit + * value will always be zeroed. + */ + WRITE_ONCE(*btf_id_ptr, 0); + continue; + } + old_ptr = xchg(btf_id_ptr, 0); + off_desc->dtor((void *)old_ptr); + } +} + /* called from workqueue */ static void bpf_map_free_deferred(struct work_struct *work) { struct bpf_map *map = container_of(work, struct bpf_map, work); security_bpf_map_free(map); - bpf_map_free_ptr_off_tab(map); bpf_map_release_memcg(map); - /* implementation dependent freeing */ + /* implementation dependent freeing, map_free callback also does + * bpf_map_free_ptr_off_tab, if needed. + */ map->ops->map_free(map); } From patchwork Sun Feb 20 13:48:09 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kumar Kartikeya Dwivedi X-Patchwork-Id: 12752713 X-Patchwork-Delegate: bpf@iogearbox.net Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 46878C433F5 for ; Sun, 20 Feb 2022 13:49:01 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S243916AbiBTNtT (ORCPT ); Sun, 20 Feb 2022 08:49:19 -0500 Received: from mxb-00190b01.gslb.pphosted.com ([23.128.96.19]:50436 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S233189AbiBTNtP (ORCPT ); Sun, 20 Feb 2022 08:49:15 -0500 Received: from mail-pl1-x642.google.com (mail-pl1-x642.google.com [IPv6:2607:f8b0:4864:20::642]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id BD45A5372F; Sun, 20 Feb 2022 05:48:50 -0800 (PST) Received: by mail-pl1-x642.google.com with SMTP id ay3so2025175plb.1; Sun, 20 Feb 2022 05:48:50 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=4Q14rygOKMclDll68ZrHDbgA4+wt3Mqwy9mZaIvZEmI=; b=M5z9ctX8K6E3UiaA41YI1MdPb2rdzWRXOnf8QI81KHSF5MzCDG8GNO4QwSyGSomDmN ajNjpwKARTL+3jMlL+KA+FihFUIHIsGqO4gUWTycUkpgA+sBvraenlHBPS45eIlnQlts BDmPUxwiXXzh+WDiLAfS6ReI3V3y+eqbgKXUKrZSAH4yGYE80IKsaaqMOb0LS2J4jDqB J8BUQN19VDlJT55G2l6u/PjcK2PBiv4Gou+vsh0Au6c/EXjbqKv4mhtxSFG8CWigdVth fCu3urU//cXmwvpHOyLPGTVRVx6fRp16bi+RjWvSqQc4aQaUid3r2/VRArYb2O0LFlx9 Z4JQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=4Q14rygOKMclDll68ZrHDbgA4+wt3Mqwy9mZaIvZEmI=; b=2OJ2foVVRrFObra9r8tZE3SBNH4H1Xcl5Ek2hsaQSFGXd7jeX89t4DspSCOhpJJMNp p4SR/BCByArwQzb9qo4MQe2773S8xsH50lZs5cznz/owHK2+uNgpbPK49dKj00jqt3iU AzLhcky4ntpNRdJkAQ89pZxraRoagfTH0in18Gl9zt2BtxcTMo/ZMEhBOJza5UlWYf6B hAp93uKWFVL3Cnje4YQ1szjLqeiu5K5/snhuupFtKakT7dWZv1ePVc8mZz+o6Uvgdd5O ktLn4OQemcMimfSM4gI7iscnwBEQzEK0W7ocj2XLtsfuiMjzCfSP79pdOf9IZPCVp92p M7aw== X-Gm-Message-State: AOAM532EmVP53zevxKfJmwcrKxsGZPjGOTu9R3Ix1Me9j07q7nRCAtbW JCRTgCx3+6VmZksCqE9BdwnDKcX/24s= X-Google-Smtp-Source: ABdhPJwdurat6QWsciAeBP7Ynt5zJUf36065WRGFBlph0iLcgl2YxnuGMSqANoWi6L4NUnWU3lZ3fA== X-Received: by 2002:a17:902:db0f:b0:14d:d55b:672b with SMTP id m15-20020a170902db0f00b0014dd55b672bmr14942500plx.133.1645364930094; Sun, 20 Feb 2022 05:48:50 -0800 (PST) Received: from localhost ([2405:201:6014:d0c0:6243:316e:a9e1:adda]) by smtp.gmail.com with ESMTPSA id g5sm9448829pfv.22.2022.02.20.05.48.49 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 20 Feb 2022 05:48:49 -0800 (PST) From: Kumar Kartikeya Dwivedi To: bpf@vger.kernel.org Cc: Alexei Starovoitov , Andrii Nakryiko , Daniel Borkmann , =?utf-8?q?Toke_H=C3=B8iland-J=C3=B8?= =?utf-8?q?rgensen?= , Jesper Dangaard Brouer , netfilter-devel@vger.kernel.org, netdev@vger.kernel.org Subject: [PATCH bpf-next v1 11/15] bpf: Teach verifier about kptr_get style kfunc helpers Date: Sun, 20 Feb 2022 19:18:09 +0530 Message-Id: <20220220134813.3411982-12-memxor@gmail.com> X-Mailer: git-send-email 2.35.1 In-Reply-To: <20220220134813.3411982-1-memxor@gmail.com> References: <20220220134813.3411982-1-memxor@gmail.com> MIME-Version: 1.0 X-Developer-Signature: v=1; a=openpgp-sha256; l=6273; h=from:subject; bh=C1Xbvo2hE9/7Vk7v/zS8Ww0BWodVX2dWBBWQaiWPd0s=; b=owEBbQKS/ZANAwAIAUzgyIZIvxHKAcsmYgBiEkZYPCub+KJBJZqjtTBGAAbbHFVkGbQhUubZG0Sa CHycgfGJAjMEAAEIAB0WIQRLvip+Buz51YI8YRFM4MiGSL8RygUCYhJGWAAKCRBM4MiGSL8Rym7pD/ 9S+gKDCU+FaLesksSvbW9eyhWrFDsLG57y1lykUkptQIJZRA9zhED5D2fQb4DwZPDb/SX2ecnDK86G lR4aZsxewx9OUGY6QhthsCN5BLX0pTMYzLREHSePmKATZ9OopwOjugxvolEuSMPxdKUnoZf2pZzbFg aFpxyvN93RHqKln7hDWDjGCc1+txFnfKur1HcTwbB+No5NCon2zIRB6MW6/IGj4wBD/YX3t5o0igZR 968qIlRzIUhDLG65LiTnVcEYyOFWkXFfM58LbP9YE1FxKcio/6zrlcjK2HGeCXQhUCj/SMXgYwshh5 etx1ZlLXKE2WO+qWslQkbgaxcp2A83a+wf/zJaOBCXwxWuPErRhlBK8Y2+ngtHFO3LSKqDpvMzfH7G YkVs78j1KPt1aTgXoY1mCTGk29jmGRK0+gsiyongP/+l0+gu+endv6ojwGg+Z649N7tCnxeZO0X8g9 QgkwANOYM11zfMKFtTNiq0Ai9ToatEy0hJD7tt/UZISg/1ectuBJRUvKmXpD8kiIz3VANld6hXuPeP XroB4SLFZqy2zXe4+ev8jV1F78T8MXKoizbyDD6ajd0NPWUB/VWD6fq86JX8EoCt+jUWiXAw1W0rPP 8Af9JhvjR4cwimy7+eNUsMdOapPcfEpKoNiM9cCNyo6T59PiU1/f+M3AX26Q== X-Developer-Key: i=memxor@gmail.com; a=openpgp; fpr=4BBE2A7E06ECF9D5823C61114CE0C88648BF11CA Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org X-Patchwork-Delegate: bpf@iogearbox.net We introduce a new style of kfunc helpers, namely *_kptr_get, where they take pointer to the map value which points to a referenced kernel pointer contained in the map. Since this is referenced, only BPF_XCHG from kernel and BPF side is allowed to change the current value, and each pointer that resides in that location would be referenced, and RCU protected (must be kept in mind while adding kernel types embeddable as reference kptr in BPF maps). This means that if do the load of the pointer value in an RCU read section, and find a live pointer, then as long as we hold RCU read lock, it won't be freed by a parallel xchg + release operation. This allows us to implement a safe refcount increment scheme. Hence, enforce that first argument of all such kfunc is a proper PTR_TO_MAP_VALUE pointing at the right offset to referenced pointer. For the rest of the arguments, they are subjected to typical kfunc argument checks, hence allowing some flexibility in passing more intent into how the reference should be taken. For instance, in case of struct nf_conn, it is not freed until all RCU grace period ends, but can still be reused for another tuple once refcount has dropped to zero. Hence, a bpf_ct_kptr_get helper not only needs to call refcount_inc_not_zero, but also do a tuple match after incrementing the reference, and when it fails to match it, put the reference again and return NULL. This can be implemented easily if we allow passing additional parameters to the bpf_ct_kptr_get kfunc, like a struct bpf_sock_tuple * and a tuple__sz pair. Signed-off-by: Kumar Kartikeya Dwivedi --- include/linux/btf.h | 2 ++ kernel/bpf/btf.c | 48 +++++++++++++++++++++++++++++++++++++++++-- kernel/bpf/verifier.c | 9 ++++---- 3 files changed, 53 insertions(+), 6 deletions(-) diff --git a/include/linux/btf.h b/include/linux/btf.h index c7e75be9637f..10918ac0e55f 100644 --- a/include/linux/btf.h +++ b/include/linux/btf.h @@ -17,6 +17,7 @@ enum btf_kfunc_type { BTF_KFUNC_TYPE_ACQUIRE, BTF_KFUNC_TYPE_RELEASE, BTF_KFUNC_TYPE_RET_NULL, + BTF_KFUNC_TYPE_KPTR_ACQUIRE, BTF_KFUNC_TYPE_MAX, }; @@ -36,6 +37,7 @@ struct btf_kfunc_id_set { struct btf_id_set *acquire_set; struct btf_id_set *release_set; struct btf_id_set *ret_null_set; + struct btf_id_set *kptr_acquire_set; }; struct btf_id_set *sets[BTF_KFUNC_TYPE_MAX]; }; diff --git a/kernel/bpf/btf.c b/kernel/bpf/btf.c index f322967da54b..1d112db4c124 100644 --- a/kernel/bpf/btf.c +++ b/kernel/bpf/btf.c @@ -6034,11 +6034,11 @@ static int btf_check_func_arg_match(struct bpf_verifier_env *env, struct bpf_verifier_log *log = &env->log; u32 i, nargs, ref_id, ref_obj_id = 0; bool is_kfunc = btf_is_kernel(btf); + bool rel = false, kptr_get = false; const char *func_name, *ref_tname; const struct btf_type *t, *ref_t; const struct btf_param *args; int ref_regno = 0; - bool rel = false; t = btf_type_by_id(btf, func_id); if (!t || !btf_type_is_func(t)) { @@ -6064,6 +6064,9 @@ static int btf_check_func_arg_match(struct bpf_verifier_env *env, return -EINVAL; } + if (is_kfunc) + kptr_get = btf_kfunc_id_set_contains(btf, resolve_prog_type(env->prog), + BTF_KFUNC_TYPE_KPTR_ACQUIRE, func_id); /* check that BTF function arguments match actual types that the * verifier sees. */ @@ -6087,7 +6090,48 @@ static int btf_check_func_arg_match(struct bpf_verifier_env *env, ref_t = btf_type_skip_modifiers(btf, t->type, &ref_id); ref_tname = btf_name_by_offset(btf, ref_t->name_off); - if (btf_get_prog_ctx_type(log, btf, t, + if (i == 0 && is_kfunc && kptr_get) { + struct bpf_map_value_off_desc *off_desc; + + if (reg->type != PTR_TO_MAP_VALUE) { + bpf_log(log, "arg#0 expected pointer to map value, but got %s\n", + btf_type_str(t)); + return -EINVAL; + } + + if (!tnum_is_const(reg->var_off)) { + bpf_log(log, "arg#0 cannot have variable offset\n"); + return -EINVAL; + } + + off_desc = bpf_map_ptr_off_contains(reg->map_ptr, reg->off + reg->var_off.value); + if (!off_desc || !(off_desc->flags & BPF_MAP_VALUE_OFF_F_REF)) { + bpf_log(log, "arg#0 no referenced pointer at map value offset=%llu\n", + reg->off + reg->var_off.value); + return -EINVAL; + } + + if (!btf_type_is_ptr(ref_t)) { + bpf_log(log, "arg#0 type must be a double pointer\n"); + return -EINVAL; + } + + ref_t = btf_type_skip_modifiers(btf, ref_t->type, &ref_id); + ref_tname = btf_name_by_offset(btf, ref_t->name_off); + + if (!btf_type_is_struct(ref_t)) { + bpf_log(log, "kernel function %s args#%d pointer type %s %s is not supported\n", + func_name, i, btf_type_str(ref_t), ref_tname); + return -EINVAL; + } + if (!btf_struct_ids_match(log, btf, ref_id, 0, off_desc->btf, off_desc->btf_id)) { + bpf_log(log, "kernel function %s args#%d expected pointer to %s %s\n", + func_name, i, btf_type_str(ref_t), ref_tname); + return -EINVAL; + } + + /* rest of the arguments can be anything, like normal kfunc */ + } else if (btf_get_prog_ctx_type(log, btf, t, env->prog->type, i)) { /* If function expects ctx type in BTF check that caller * is passing PTR_TO_CTX. diff --git a/kernel/bpf/verifier.c b/kernel/bpf/verifier.c index 0a2cd21d9ec1..a4ff951ea46f 100644 --- a/kernel/bpf/verifier.c +++ b/kernel/bpf/verifier.c @@ -3657,11 +3657,12 @@ static int check_map_ptr_to_btf_id(struct bpf_verifier_env *env, u32 regno, int } else if (insn_class == BPF_LDX) { if (WARN_ON_ONCE(value_regno < 0)) return -EFAULT; + /* We allow loading referenced pointer, but mark it as + * untrusted. User needs to use a kptr_get helper to obtain a + * trusted refcounted PTR_TO_BTF_ID by passing in the map + * value pointing to the referenced pointer. + */ val_reg = reg_state(env, value_regno); - if (ref_ptr) { - verbose(env, "referenced btf_id pointer can only be accessed using BPF_XCHG\n"); - return -EACCES; - } /* We can simply mark the value_regno receiving the pointer * value from map as PTR_TO_BTF_ID, with the correct type. */ From patchwork Sun Feb 20 13:48:10 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kumar Kartikeya Dwivedi X-Patchwork-Id: 12752712 X-Patchwork-Delegate: bpf@iogearbox.net Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 95DCAC433FE for ; Sun, 20 Feb 2022 13:48:59 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S243909AbiBTNtS (ORCPT ); Sun, 20 Feb 2022 08:49:18 -0500 Received: from mxb-00190b01.gslb.pphosted.com ([23.128.96.19]:50520 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S243916AbiBTNtP (ORCPT ); Sun, 20 Feb 2022 08:49:15 -0500 Received: from mail-pg1-x544.google.com (mail-pg1-x544.google.com [IPv6:2607:f8b0:4864:20::544]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id D511653732; Sun, 20 Feb 2022 05:48:53 -0800 (PST) Received: by mail-pg1-x544.google.com with SMTP id s16so12001635pgs.13; Sun, 20 Feb 2022 05:48:53 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=v7r7FHu3QAUhr3+7QG+/dRRC3q80hrKg8AEc1n+/igo=; b=kHV+QJWieq0jGqRzueAF4rk9YMSlkJUsWMJ3wTg9+D5mbEVNh9ZjZKV2OOgbZJrdSS 0ZMivsWI1P3+twe5PtEM56UQJE6a8tpF50RdHFXMvCYuL5bmLxmxYrkba9Z8s5ZJ9J0b jO0wrxgeYgiJzktjgdjzjWdXHlPpUMlOsK3vktXyFyuHitM1wq3rvJ2L1OgFOoOFJV21 /3WdnPWA4WS4WhyZksbJHHjmBSLZSNEw407pTHU++Z1dc8s5uC4510c4MZw0/LPTmCql C2MyQgHK3S+W7Gefgabgfvqn+6wTmV1/uiMmn1kCAet8cub2h8qEhXhuTuryw7dggfXt LWZA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=v7r7FHu3QAUhr3+7QG+/dRRC3q80hrKg8AEc1n+/igo=; b=ThBUZ51Ar44Py30x7qmynSpBSTILQ5VjW+Jia20sdZogjELv73f+5ul5cnRuuIHoZC +B+5nV9llOQqzluSi6X1XRnG/otviqtUCat51d/viQAZtteqGCALXsj+rRXizFUZ1xn4 LImzlLTmTI/cGYa6f+c0f8Yg405dQIeS2jywdSEykegWxeMQHT7Bbn44A/ce3gKQiCHF +5e22J6YF25cNMUbsxKOXOzss6Kv5Krp1q3l6NC2w0UlHz3vOKtIMZgwY8nwaY0Cnz32 SCf1IDjfq/oCrDLcw8Nx+C2m21bthPIlRFv4+0AIvLct7U5Duo0p5mc0lSBkmSMqv2zm 7fCA== X-Gm-Message-State: AOAM5313IfAF0QVsJJyH8MYZEc08UuBMuDCZrQvRHKeM7YnSHxb7xHl4 KsEMjt1zidzEfH8+3F+MfvGsUuZWdm8= X-Google-Smtp-Source: ABdhPJzK6kwqZpGGf6MnW/pEFOYgmd5WWEWPXLCg7HUqFdLFyn7nGenJJvSrroh8qdfi6ANthhc8vg== X-Received: by 2002:a05:6a00:124b:b0:4f0:eaa7:4677 with SMTP id u11-20020a056a00124b00b004f0eaa74677mr11208398pfi.85.1645364933191; Sun, 20 Feb 2022 05:48:53 -0800 (PST) Received: from localhost ([2405:201:6014:d0c0:6243:316e:a9e1:adda]) by smtp.gmail.com with ESMTPSA id v13sm10118464pfu.196.2022.02.20.05.48.52 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 20 Feb 2022 05:48:52 -0800 (PST) From: Kumar Kartikeya Dwivedi To: bpf@vger.kernel.org Cc: Alexei Starovoitov , Andrii Nakryiko , Daniel Borkmann , =?utf-8?q?Toke_H=C3=B8iland-J=C3=B8?= =?utf-8?q?rgensen?= , Jesper Dangaard Brouer , netfilter-devel@vger.kernel.org, netdev@vger.kernel.org Subject: [PATCH bpf-next v1 12/15] net/netfilter: Add bpf_ct_kptr_get helper Date: Sun, 20 Feb 2022 19:18:10 +0530 Message-Id: <20220220134813.3411982-13-memxor@gmail.com> X-Mailer: git-send-email 2.35.1 In-Reply-To: <20220220134813.3411982-1-memxor@gmail.com> References: <20220220134813.3411982-1-memxor@gmail.com> MIME-Version: 1.0 X-Developer-Signature: v=1; a=openpgp-sha256; l=9318; h=from:subject; bh=fN4g0NsdFmVTHRi7ezE3E/ZN+uig0L4IYC7S4pnjH60=; b=owEBbQKS/ZANAwAIAUzgyIZIvxHKAcsmYgBiEkZYTo+bmM8pp/rh3EIrMhbhLK9xV6cJ7gRXpSif /qkFJgeJAjMEAAEIAB0WIQRLvip+Buz51YI8YRFM4MiGSL8RygUCYhJGWAAKCRBM4MiGSL8Ryja/D/ 9QSz/4px2Q7U+6712nl2OxXmGQk+k6QbEWUab0LPdANl+E6B9qn/rybtXi7o013DzRB+X2wAye9dkc QGiIwqvlbdQSumKkzI4kk1t6lW2IIzLdCkA8wXOC3vw2xzQaaMDnVQJvM41WCyagf5JN4X/DciOjx8 pbjfaVJRR6hhndngIMVkI1bEJFbscK0Urd+KBPqUO1rXxMo/nvfv2/OOdD7G/azw/GDRGzAh0jVCxW tiZdjCUs9lWy1018/xSVNWZLOIBD3H1sym5sEXkjqAA8XIlGGeU+oPD4X0mVrsrJWupOwODKPEwkwY FvkKKE9PPI7mqj7/Lu3vh8SbxOpV/CegFg0W9VrNR7o/IIdG4ts/8Wy+o4hjZzFHFABTXW6ZWY965g k/IAJP9CvJHKpclaJjDv9OSEZfjpkApQdpwYcUeCi9AgmdaT1+rOfFYqHWrjio6v7V7dyM3TlqMWxT uolP00ZVzdZGf6MQKcJfXORxMa5JzQTl4SWxPUWJBl37YdPQ5/3MP/BvMOeQe39os/pTsM5WVfZi9Q Mcudx0rqjkEa6c/na54Bqh8/WeXvD/ytDpcvJSsZCHWBkZGv/VRzkJsAW3CqdT1qOg+4l70Gbd6ooH BclvVppaI4Tsu/P8G+Ht7A0gpDlQDJQN3TamXrc4jhdI+W2QeYVtq2wTcmKA== X-Developer-Key: i=memxor@gmail.com; a=openpgp; fpr=4BBE2A7E06ECF9D5823C61114CE0C88648BF11CA Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org X-Patchwork-Delegate: bpf@iogearbox.net Require some more feedback on whether this is OK, before refactoring netfilter functions to share code to increment reference and match the tuple. Also probably need to work on allowing taking reference to struct net * to save another lookup inside this function. Signed-off-by: Kumar Kartikeya Dwivedi Reported-by: kernel test robot --- include/net/netfilter/nf_conntrack_core.h | 17 +++ net/netfilter/nf_conntrack_bpf.c | 132 +++++++++++++++++----- net/netfilter/nf_conntrack_core.c | 17 --- 3 files changed, 119 insertions(+), 47 deletions(-) diff --git a/include/net/netfilter/nf_conntrack_core.h b/include/net/netfilter/nf_conntrack_core.h index 13807ea94cd2..09389769dce3 100644 --- a/include/net/netfilter/nf_conntrack_core.h +++ b/include/net/netfilter/nf_conntrack_core.h @@ -51,6 +51,23 @@ nf_conntrack_find_get(struct net *net, int __nf_conntrack_confirm(struct sk_buff *skb); +static inline bool +nf_ct_key_equal(struct nf_conntrack_tuple_hash *h, + const struct nf_conntrack_tuple *tuple, + const struct nf_conntrack_zone *zone, + const struct net *net) +{ + struct nf_conn *ct = nf_ct_tuplehash_to_ctrack(h); + + /* A conntrack can be recreated with the equal tuple, + * so we need to check that the conntrack is confirmed + */ + return nf_ct_tuple_equal(tuple, &h->tuple) && + nf_ct_zone_equal(ct, zone, NF_CT_DIRECTION(h)) && + nf_ct_is_confirmed(ct) && + net_eq(net, nf_ct_net(ct)); +} + /* Confirm a connection: returns NF_DROP if packet must be dropped. */ static inline int nf_conntrack_confirm(struct sk_buff *skb) { diff --git a/net/netfilter/nf_conntrack_bpf.c b/net/netfilter/nf_conntrack_bpf.c index 8ad3f52579f3..26211a5ec0c4 100644 --- a/net/netfilter/nf_conntrack_bpf.c +++ b/net/netfilter/nf_conntrack_bpf.c @@ -52,6 +52,30 @@ enum { NF_BPF_CT_OPTS_SZ = 12, }; +static int bpf_fill_nf_tuple(struct nf_conntrack_tuple *tuple, + struct bpf_sock_tuple *bpf_tuple, u32 tuple_len) +{ + switch (tuple_len) { + case sizeof(bpf_tuple->ipv4): + tuple->src.l3num = AF_INET; + tuple->src.u3.ip = bpf_tuple->ipv4.saddr; + tuple->src.u.tcp.port = bpf_tuple->ipv4.sport; + tuple->dst.u3.ip = bpf_tuple->ipv4.daddr; + tuple->dst.u.tcp.port = bpf_tuple->ipv4.dport; + break; + case sizeof(bpf_tuple->ipv6): + tuple->src.l3num = AF_INET6; + memcpy(tuple->src.u3.ip6, bpf_tuple->ipv6.saddr, sizeof(bpf_tuple->ipv6.saddr)); + tuple->src.u.tcp.port = bpf_tuple->ipv6.sport; + memcpy(tuple->dst.u3.ip6, bpf_tuple->ipv6.daddr, sizeof(bpf_tuple->ipv6.daddr)); + tuple->dst.u.tcp.port = bpf_tuple->ipv6.dport; + break; + default: + return -EAFNOSUPPORT; + } + return 0; +} + static struct nf_conn *__bpf_nf_ct_lookup(struct net *net, struct bpf_sock_tuple *bpf_tuple, u32 tuple_len, u8 protonum, @@ -59,6 +83,7 @@ static struct nf_conn *__bpf_nf_ct_lookup(struct net *net, { struct nf_conntrack_tuple_hash *hash; struct nf_conntrack_tuple tuple; + int ret; if (unlikely(protonum != IPPROTO_TCP && protonum != IPPROTO_UDP)) return ERR_PTR(-EPROTO); @@ -66,25 +91,9 @@ static struct nf_conn *__bpf_nf_ct_lookup(struct net *net, return ERR_PTR(-EINVAL); memset(&tuple, 0, sizeof(tuple)); - switch (tuple_len) { - case sizeof(bpf_tuple->ipv4): - tuple.src.l3num = AF_INET; - tuple.src.u3.ip = bpf_tuple->ipv4.saddr; - tuple.src.u.tcp.port = bpf_tuple->ipv4.sport; - tuple.dst.u3.ip = bpf_tuple->ipv4.daddr; - tuple.dst.u.tcp.port = bpf_tuple->ipv4.dport; - break; - case sizeof(bpf_tuple->ipv6): - tuple.src.l3num = AF_INET6; - memcpy(tuple.src.u3.ip6, bpf_tuple->ipv6.saddr, sizeof(bpf_tuple->ipv6.saddr)); - tuple.src.u.tcp.port = bpf_tuple->ipv6.sport; - memcpy(tuple.dst.u3.ip6, bpf_tuple->ipv6.daddr, sizeof(bpf_tuple->ipv6.daddr)); - tuple.dst.u.tcp.port = bpf_tuple->ipv6.dport; - break; - default: - return ERR_PTR(-EAFNOSUPPORT); - } - + ret = bpf_fill_nf_tuple(&tuple, bpf_tuple, tuple_len); + if (ret < 0) + return ERR_PTR(ret); tuple.dst.protonum = protonum; if (netns_id >= 0) { @@ -208,50 +217,113 @@ void bpf_ct_release(struct nf_conn *nfct) nf_ct_put(nfct); } +/* TODO: Just a PoC, need to reuse code in __nf_conntrack_find_get for this */ +struct nf_conn *bpf_ct_kptr_get(struct nf_conn **ptr, struct bpf_sock_tuple *bpf_tuple, + u32 tuple__sz, u8 protonum, u8 direction) +{ + struct nf_conntrack_tuple tuple; + struct nf_conn *nfct; + struct net *net; + u64 *nfct_p; + int ret; + + WARN_ON_ONCE(!rcu_read_lock_held()); + + if ((protonum != IPPROTO_TCP && protonum != IPPROTO_UDP) || + (direction != IP_CT_DIR_ORIGINAL && direction != IP_CT_DIR_REPLY)) + return NULL; + + /* ptr is actually pointer to u64 having address, hence recast u64 load + * to native pointer width. + */ + nfct_p = (u64 *)ptr; + nfct = (struct nf_conn *)READ_ONCE(*nfct_p); + if (!nfct || unlikely(!refcount_inc_not_zero(&nfct->ct_general.use))) + return NULL; + + memset(&tuple, 0, sizeof(tuple)); + ret = bpf_fill_nf_tuple(&tuple, bpf_tuple, tuple__sz); + if (ret < 0) + goto end; + tuple.dst.protonum = protonum; + + /* XXX: Need to allow passing in struct net *, or take netns_id, this is non-sense */ + net = nf_ct_net(nfct); + if (!nf_ct_key_equal(&nfct->tuplehash[direction], &tuple, + &nf_ct_zone_dflt, nf_ct_net(nfct))) + goto end; + return nfct; +end: + nf_ct_put(nfct); + return NULL; +} + __diag_pop() BTF_SET_START(nf_ct_xdp_check_kfunc_ids) BTF_ID(func, bpf_xdp_ct_lookup) +BTF_ID(func, bpf_ct_kptr_get) BTF_ID(func, bpf_ct_release) BTF_SET_END(nf_ct_xdp_check_kfunc_ids) BTF_SET_START(nf_ct_tc_check_kfunc_ids) BTF_ID(func, bpf_skb_ct_lookup) +BTF_ID(func, bpf_ct_kptr_get) BTF_ID(func, bpf_ct_release) BTF_SET_END(nf_ct_tc_check_kfunc_ids) BTF_SET_START(nf_ct_acquire_kfunc_ids) BTF_ID(func, bpf_xdp_ct_lookup) BTF_ID(func, bpf_skb_ct_lookup) +BTF_ID(func, bpf_ct_kptr_get) BTF_SET_END(nf_ct_acquire_kfunc_ids) BTF_SET_START(nf_ct_release_kfunc_ids) BTF_ID(func, bpf_ct_release) BTF_SET_END(nf_ct_release_kfunc_ids) +BTF_SET_START(nf_ct_kptr_acquire_kfunc_ids) +BTF_ID(func, bpf_ct_kptr_get) +BTF_SET_END(nf_ct_kptr_acquire_kfunc_ids) + /* Both sets are identical */ #define nf_ct_ret_null_kfunc_ids nf_ct_acquire_kfunc_ids static const struct btf_kfunc_id_set nf_conntrack_xdp_kfunc_set = { - .owner = THIS_MODULE, - .check_set = &nf_ct_xdp_check_kfunc_ids, - .acquire_set = &nf_ct_acquire_kfunc_ids, - .release_set = &nf_ct_release_kfunc_ids, - .ret_null_set = &nf_ct_ret_null_kfunc_ids, + .owner = THIS_MODULE, + .check_set = &nf_ct_xdp_check_kfunc_ids, + .acquire_set = &nf_ct_acquire_kfunc_ids, + .release_set = &nf_ct_release_kfunc_ids, + .ret_null_set = &nf_ct_ret_null_kfunc_ids, + .kptr_acquire_set = &nf_ct_kptr_acquire_kfunc_ids, }; static const struct btf_kfunc_id_set nf_conntrack_tc_kfunc_set = { - .owner = THIS_MODULE, - .check_set = &nf_ct_tc_check_kfunc_ids, - .acquire_set = &nf_ct_acquire_kfunc_ids, - .release_set = &nf_ct_release_kfunc_ids, - .ret_null_set = &nf_ct_ret_null_kfunc_ids, + .owner = THIS_MODULE, + .check_set = &nf_ct_tc_check_kfunc_ids, + .acquire_set = &nf_ct_acquire_kfunc_ids, + .release_set = &nf_ct_release_kfunc_ids, + .ret_null_set = &nf_ct_ret_null_kfunc_ids, + .kptr_acquire_set = &nf_ct_kptr_acquire_kfunc_ids, }; +BTF_ID_LIST(nf_conntrack_dtor_kfunc_ids) +BTF_ID(struct, nf_conn) +BTF_ID(func, bpf_ct_release) + int register_nf_conntrack_bpf(void) { + const struct btf_id_dtor_kfunc nf_conntrack_dtor_kfunc[] = { + { + .btf_id = nf_conntrack_dtor_kfunc_ids[0], + .kfunc_btf_id = nf_conntrack_dtor_kfunc_ids[1], + } + }; int ret; - ret = register_btf_kfunc_id_set(BPF_PROG_TYPE_XDP, &nf_conntrack_xdp_kfunc_set); + ret = register_btf_id_dtor_kfuncs(nf_conntrack_dtor_kfunc, + ARRAY_SIZE(nf_conntrack_dtor_kfunc), + THIS_MODULE); + ret = ret ?: register_btf_kfunc_id_set(BPF_PROG_TYPE_XDP, &nf_conntrack_xdp_kfunc_set); return ret ?: register_btf_kfunc_id_set(BPF_PROG_TYPE_SCHED_CLS, &nf_conntrack_tc_kfunc_set); } diff --git a/net/netfilter/nf_conntrack_core.c b/net/netfilter/nf_conntrack_core.c index 9b7f9c966f73..0aae98f60769 100644 --- a/net/netfilter/nf_conntrack_core.c +++ b/net/netfilter/nf_conntrack_core.c @@ -710,23 +710,6 @@ bool nf_ct_delete(struct nf_conn *ct, u32 portid, int report) } EXPORT_SYMBOL_GPL(nf_ct_delete); -static inline bool -nf_ct_key_equal(struct nf_conntrack_tuple_hash *h, - const struct nf_conntrack_tuple *tuple, - const struct nf_conntrack_zone *zone, - const struct net *net) -{ - struct nf_conn *ct = nf_ct_tuplehash_to_ctrack(h); - - /* A conntrack can be recreated with the equal tuple, - * so we need to check that the conntrack is confirmed - */ - return nf_ct_tuple_equal(tuple, &h->tuple) && - nf_ct_zone_equal(ct, zone, NF_CT_DIRECTION(h)) && - nf_ct_is_confirmed(ct) && - net_eq(net, nf_ct_net(ct)); -} - static inline bool nf_ct_match(const struct nf_conn *ct1, const struct nf_conn *ct2) { From patchwork Sun Feb 20 13:48:11 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kumar Kartikeya Dwivedi X-Patchwork-Id: 12752715 X-Patchwork-Delegate: bpf@iogearbox.net Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id EBB9EC433FE for ; Sun, 20 Feb 2022 13:49:04 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S243927AbiBTNtX (ORCPT ); Sun, 20 Feb 2022 08:49:23 -0500 Received: from mxb-00190b01.gslb.pphosted.com ([23.128.96.19]:50574 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S243893AbiBTNtR (ORCPT ); Sun, 20 Feb 2022 08:49:17 -0500 Received: from mail-pj1-x1042.google.com (mail-pj1-x1042.google.com [IPv6:2607:f8b0:4864:20::1042]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id EE86021813; Sun, 20 Feb 2022 05:48:56 -0800 (PST) Received: by mail-pj1-x1042.google.com with SMTP id y16so4473678pjt.0; Sun, 20 Feb 2022 05:48:56 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=zc9vhs9G+cyXNOZDjUOAkxyZIQRaNLzXarwKxHWmpLQ=; b=PW/q6Z9D7RxifY95IT2qq4If95So1cSsMDRmHAR+VFhmcnXuUMZ06+O6v9RxnvsAe2 hacO8nMildUQ9KdKVY8s8YSoSoBVRU8VMyGK6AsRIF27tvZXtASWohCbOcJ1Z+Qc9jJu uQI1vTwj1Z9+yVT4PbiQVNLNMqN4jA1wpOqUF2bLTyFIJ3hrNr93YzpUhGnpuyWCkF5Y BIrb108eEeIhljhT+sgsb2uKcpWjYazcxeN/g+nWbx+ZFUm0XmSOlnWYykSid08PUT/R 9NxcxvnQKEHRP88fg0Bma0BgJOo2/bOtjurZvptIo4rXm0DyQdxXV7h/2ACUbpE3UzeP jqbA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=zc9vhs9G+cyXNOZDjUOAkxyZIQRaNLzXarwKxHWmpLQ=; b=hlozwdi4YQEr0TDaELFgaqJOB2Pjnt4F1tOHCLsjjgCWekz06O8jaZEF7Tvn1WBJ2a kCf/IXZtQZHXvlYvGAxJfvPwQDuuQuYuGeg1idBwWkILR4dJHGQP2WW7F+QnfUyu7nRQ BtnIplJ7UFeXXuPGAV9nWRSzfYKIt4O/8INiz0v8U9/lPTvhrmRappE+GCHEFsII7Rrn qNukUzdnSR5bnlj/THwHBuAVcj2s6Q8xdSoy4OMOtT4IpaMMdAS2fl6SeCEtr60J8rxu rR7tzWtgU2KjIy59Mgs7GQixwAto6oSUk6cM3e2ESYmIkZdCkQ0bN8+tF+xCmYicRGyd Gdxw== X-Gm-Message-State: AOAM5326cW37OaHJk46mfR1lL95zea11ma5vOvNW0WCyIqOPL7ca8AQ6 lv+Os8DFkeqj8RpMdWvTEeIMA7NXFWU= X-Google-Smtp-Source: ABdhPJw07hfXuRWJYM+lPPKigrU9EJRsHaQA6Vt4BOdeWnHTLBfsgANKEhfzjB9Csf4pMPlmFTLeVw== X-Received: by 2002:a17:90a:6882:b0:1bc:495:45e0 with SMTP id a2-20020a17090a688200b001bc049545e0mr5842615pjd.208.1645364936307; Sun, 20 Feb 2022 05:48:56 -0800 (PST) Received: from localhost ([2405:201:6014:d0c0:6243:316e:a9e1:adda]) by smtp.gmail.com with ESMTPSA id q2sm9687854pfj.94.2022.02.20.05.48.55 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 20 Feb 2022 05:48:56 -0800 (PST) From: Kumar Kartikeya Dwivedi To: bpf@vger.kernel.org Cc: Alexei Starovoitov , Andrii Nakryiko , Daniel Borkmann , =?utf-8?q?Toke_H=C3=B8iland-J=C3=B8?= =?utf-8?q?rgensen?= , Jesper Dangaard Brouer , netfilter-devel@vger.kernel.org, netdev@vger.kernel.org Subject: [PATCH bpf-next v1 13/15] libbpf: Add __kptr* macros to bpf_helpers.h Date: Sun, 20 Feb 2022 19:18:11 +0530 Message-Id: <20220220134813.3411982-14-memxor@gmail.com> X-Mailer: git-send-email 2.35.1 In-Reply-To: <20220220134813.3411982-1-memxor@gmail.com> References: <20220220134813.3411982-1-memxor@gmail.com> MIME-Version: 1.0 X-Developer-Signature: v=1; a=openpgp-sha256; l=1322; h=from:subject; bh=NZD5OPd+5AhwTtB95pR1jIO53bA/xGYfngUoa0qcWuM=; b=owEBbQKS/ZANAwAIAUzgyIZIvxHKAcsmYgBiEkZYOKPlywQDBJ8xdF0F5WLgAw9iAlnqzark+xJY DRMqxniJAjMEAAEIAB0WIQRLvip+Buz51YI8YRFM4MiGSL8RygUCYhJGWAAKCRBM4MiGSL8RyiwmD/ 4h7mPEqLQl8kqvNbCenKbfasSveHCrPpUz5rACzMCHB+LOPcy8jLY9sib8y73E4PhBh87gFgJVvgXJ zihkInfyQWV96xbMnKSMQXea2MwMrfItAQgCpctnUnmHcb9zKRN+lwoKM5CbszzCbFfuvqrCAifKl1 M/dj6bpEWnT7XgUOfJk8ry5B51K1cq4K3+DUujMWttQE0gqPnK8BNr9+3JIaWZXeA6GlUM/4X6RpFa 5OvecyeTCCl+htOGcTuVkGMK/5ubLYoMO2P+4Xzc9QiPwzCDrT91eyjCT+5KY2tDdQPDzxd9V6EkPU qG5Soh9toFHZHLWtzivwjvroWrxdrM73S8rWJ6QNNZl9OM8UCfO2w6au3K1Z8z+fxexlWmksCSP/dG StdhoM36txuEa17Z4hBnd2HdQubd6UMydFY6AsXm3SwmgQno3naxgbaVfdqYMhmS36RCiS6Vgfkrsa pbhHeK0FxO/0SgUyN6oRU0UsqS3Frc7Wd5bwg0o1kuX4/184Wx3rWBsi1HwuhTwW9ZMoWwsL3BBuIc 0yDFlgMvY0N1iJ8g+sydKJPsJrKDRDANHS1Uw8B8dawHcN9rF84IYLGFNG9APoddAS7pos6fD/UMjQ 4wbPpykof1DxLrvrmOyKwQARcI6LC41/ir8BxEzFtjoIZy+4WVgVGiYhjR9w== X-Developer-Key: i=memxor@gmail.com; a=openpgp; fpr=4BBE2A7E06ECF9D5823C61114CE0C88648BF11CA Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org X-Patchwork-Delegate: bpf@iogearbox.net Include convenience definitions: __kptr: Unreferenced BTF ID pointer __kptr_ref: Referenced BTF ID pointer __kptr_percpu: per-CPU BTF ID pointer __kptr_user: Userspace BTF ID pointer Users can use them to tag the pointer type meant to be used with the new support directly in the map value definition. Note that these attributes require https://reviews.llvm.org/D119799 to be emitted in BPF object BTF correctly when applied to a non-builtin type. Signed-off-by: Kumar Kartikeya Dwivedi --- tools/lib/bpf/bpf_helpers.h | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/tools/lib/bpf/bpf_helpers.h b/tools/lib/bpf/bpf_helpers.h index 44df982d2a5c..feb311fe1c72 100644 --- a/tools/lib/bpf/bpf_helpers.h +++ b/tools/lib/bpf/bpf_helpers.h @@ -149,6 +149,10 @@ enum libbpf_tristate { #define __kconfig __attribute__((section(".kconfig"))) #define __ksym __attribute__((section(".ksyms"))) +#define __kptr __attribute__((btf_type_tag("kernel.bpf.btf_id"))) +#define __kptr_ref __kptr __attribute__((btf_type_tag("kernel.bpf.ref"))) +#define __kptr_percpu __kptr __attribute__((btf_type_tag("kernel.bpf.percpu"))) +#define __kptr_user __kptr __attribute__((btf_type_tag("kernel.bpf.user"))) #ifndef ___bpf_concat #define ___bpf_concat(a, b) a ## b From patchwork Sun Feb 20 13:48:12 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kumar Kartikeya Dwivedi X-Patchwork-Id: 12752716 X-Patchwork-Delegate: bpf@iogearbox.net Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id A6C7EC433F5 for ; Sun, 20 Feb 2022 13:49:24 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S243473AbiBTNtn (ORCPT ); Sun, 20 Feb 2022 08:49:43 -0500 Received: from mxb-00190b01.gslb.pphosted.com ([23.128.96.19]:50684 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S243933AbiBTNtk (ORCPT ); Sun, 20 Feb 2022 08:49:40 -0500 Received: from mail-pf1-x441.google.com (mail-pf1-x441.google.com [IPv6:2607:f8b0:4864:20::441]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 2275153731; Sun, 20 Feb 2022 05:49:00 -0800 (PST) Received: by mail-pf1-x441.google.com with SMTP id d17so6458773pfl.0; Sun, 20 Feb 2022 05:49:00 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=FQ8dyPiqPLmVFyu1015TusWyT0/TCJ+qkiFiAwTmZf8=; b=mLuRBE4/06WFBBkBAR/2xL6B4iIVkvAZAB3YXvtRSs1jkYcrMNRRtxnmb7ob/numhA +BySUCxf2CPAGnN3ASdpJ8xo2Y28ChSjgkIemZ6yagLssQDPkqQJQutaG6bjaVkVgUeZ sAhz6UIuCCCzOpS3Unq5Ekik1pSWPl1LUFu7Rt3d1agJNMC2sFXOD3uQzX2lb/jvvNjt YUnnUlH3MMGUZ+1Fqzf2T/xVsobc9H6FfmRk8g14b2BqRLVmivqOTII0h/G7mpSKMnmn 4ZINMZqfo7sQyW2WyDs2wMB/VtWmerGIhwIExkE6Kl8T37dbE6sQs9SI23jfjJGRwCNf wSMw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=FQ8dyPiqPLmVFyu1015TusWyT0/TCJ+qkiFiAwTmZf8=; b=oRapgkhqHrnTq6FlwikhOLb4d4Zo3biJIDzAl2+3trr81lzggb+jWS2m9XULMY0hIb uvDr7Oxd17vGQkdVU9SuKsvsAxnKu1DdywgPsDpLt+fu++hwZkCMuh9OV78K4MVCD41n +InasTZaxVCQTGe0SWwoXsgCKqX8+y2Lm4mwHF1jb1gEg9lNZ2jw709wOc8yDm45uu1k jvFnK/G3+emKpfLdBgtA8HjcEcN4HvUCfpttLCW2+rVZkbSYCwNZjIebyl2Y+DSEi5k3 hmr5NhCpYvGKlADoK4BenZBW/iN1nIaNP+umyI0vuzwu6egtGjUPfmg6GVeCjQF495uc RC4w== X-Gm-Message-State: AOAM5304MZemovvwi3zFBg6yWQULznSi3cu/y/XjbXE8YSgg6QhbWJ+M KnvwgPI3m5BiDGanaZkEzaG2R6DjxaM= X-Google-Smtp-Source: ABdhPJz9KW4uedO6mACM8AUtpi2JmrUkgqyYO7JeiSPQD8YSp2dj99ubnVuxXk11BWLgiYlQ4tIN7g== X-Received: by 2002:a63:e5c:0:b0:365:331e:8dcd with SMTP id 28-20020a630e5c000000b00365331e8dcdmr12983917pgo.431.1645364939411; Sun, 20 Feb 2022 05:48:59 -0800 (PST) Received: from localhost ([2405:201:6014:d0c0:6243:316e:a9e1:adda]) by smtp.gmail.com with ESMTPSA id h21sm9311207pfo.12.2022.02.20.05.48.58 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 20 Feb 2022 05:48:59 -0800 (PST) From: Kumar Kartikeya Dwivedi To: bpf@vger.kernel.org Cc: Alexei Starovoitov , Andrii Nakryiko , Daniel Borkmann , =?utf-8?q?Toke_H=C3=B8iland-J=C3=B8?= =?utf-8?q?rgensen?= , Jesper Dangaard Brouer , netfilter-devel@vger.kernel.org, netdev@vger.kernel.org Subject: [PATCH bpf-next v1 14/15] selftests/bpf: Add C tests for PTR_TO_BTF_ID in map Date: Sun, 20 Feb 2022 19:18:12 +0530 Message-Id: <20220220134813.3411982-15-memxor@gmail.com> X-Mailer: git-send-email 2.35.1 In-Reply-To: <20220220134813.3411982-1-memxor@gmail.com> References: <20220220134813.3411982-1-memxor@gmail.com> MIME-Version: 1.0 X-Developer-Signature: v=1; a=openpgp-sha256; l=5959; h=from:subject; bh=wl/5gCvad/dV1Vcz6TclnMJfJeC3nZQ5ne11tiVi4RY=; b=owEBbQKS/ZANAwAIAUzgyIZIvxHKAcsmYgBiEkZYYZ8ER8Flwj2NzkS2jdee14hb/4b6kPHnYMHD /umIAfyJAjMEAAEIAB0WIQRLvip+Buz51YI8YRFM4MiGSL8RygUCYhJGWAAKCRBM4MiGSL8RynnCD/ 0YuFM3ove6xY/uJJJ/5V06N0V5ClfXQB05Z8+JyuSb1BmDEoQRXAzogzVuQGbW8vzypEyt2PC4oP5L H8bQR2Xss6aZgdkM3uy0XKrV2Tdxoxx2FqdGQcHnTsqlIKJmb4FqVUtS8Dd9TzP6fB+U6Z++kZRyYk ghPGoYtivdTnLnPFqmWRPjATnAWTzc24bvNjO8HaUs8oK2EV1ftMgCf2UFdpCLI3MAEIbFvXBLUis1 cPID+5xqBeguQQ7M+5dkQxyzCTyeFCbdK6+Q2y9JuGyeNiRfhNeyvGcRMbFB4Jp+AtWJagUOKEyNjA XbDHmCgEv9GvUgmZAlIyR/zh2qYG4FywIKHhivwgacDLcHSu6B4qWbKkEpAoyLZMyf96bGMAvfKfGT Oxee3KvS0yUccLF2jGSEjNWZo8bK3vI9PZ213vvQHapPwQCR7q/DqzysN7Vd5LK5LN7WdwNdmDkRP7 qX1fo7wImxseXAlyXviP7xY9N8I+EySysZIRUjQRYoYaywvh3D/ra38UdjXzIdhYFWV5DaVFmRiBGJ /+W3+JjpEsPhaf+fs4xH38o2NjT6/xEukU96rOGvYK1QtllnUE76LCujBK/W+jbzonVMa+RFyRHzzB 1eqJKLxlktwYMuwaPHzw8vSDL/6YhIb3bHl1L6zvk2LA20fR33Rl5inAjfJA== X-Developer-Key: i=memxor@gmail.com; a=openpgp; fpr=4BBE2A7E06ECF9D5823C61114CE0C88648BF11CA Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org X-Patchwork-Delegate: bpf@iogearbox.net This uses the __kptr* macros as well, and tries to test the stuff that is supposed to work, since we have negative tests in test_verifier suite. Signed-off-by: Kumar Kartikeya Dwivedi --- .../selftests/bpf/prog_tests/map_btf_ptr.c | 13 +++ .../testing/selftests/bpf/progs/map_btf_ptr.c | 105 ++++++++++++++++++ .../testing/selftests/bpf/progs/test_bpf_nf.c | 31 ++++++ 3 files changed, 149 insertions(+) create mode 100644 tools/testing/selftests/bpf/prog_tests/map_btf_ptr.c create mode 100644 tools/testing/selftests/bpf/progs/map_btf_ptr.c diff --git a/tools/testing/selftests/bpf/prog_tests/map_btf_ptr.c b/tools/testing/selftests/bpf/prog_tests/map_btf_ptr.c new file mode 100644 index 000000000000..8fb6acf1b89d --- /dev/null +++ b/tools/testing/selftests/bpf/prog_tests/map_btf_ptr.c @@ -0,0 +1,13 @@ +#include + +#include "map_btf_ptr.skel.h" + +void test_map_btf_ptr(void) +{ + struct map_btf_ptr *skel; + + skel = map_btf_ptr__open_and_load(); + if (!ASSERT_OK_PTR(skel, "map_btf_ptr__open_and_load")) + return; + map_btf_ptr__destroy(skel); +} diff --git a/tools/testing/selftests/bpf/progs/map_btf_ptr.c b/tools/testing/selftests/bpf/progs/map_btf_ptr.c new file mode 100644 index 000000000000..b0c2ba595290 --- /dev/null +++ b/tools/testing/selftests/bpf/progs/map_btf_ptr.c @@ -0,0 +1,105 @@ +#include +#include +#include + +#define xchg(dst, src) __sync_lock_test_and_set(&(dst), (src)) + +struct map_value { + struct prog_test_ref_kfunc __kptr *unref_ptr; + /* Workarounds for https://lore.kernel.org/bpf/20220220071333.sltv4jrwniool2qy@apollo.legion */ + struct prog_test_ref_kfunc __kptr __attribute__((btf_type_tag("kernel.bpf.ref"))) *ref_ptr; + struct prog_test_ref_kfunc __kptr __attribute__((btf_type_tag("kernel.bpf.percpu"))) *percpu_ptr; + struct prog_test_ref_kfunc __kptr __attribute__((btf_type_tag("kernel.bpf.user"))) *user_ptr; +}; + +struct { + __uint(type, BPF_MAP_TYPE_ARRAY); + __type(key, int); + __type(value, struct map_value); + __uint(max_entries, 1); +} array_map SEC(".maps"); + +extern struct prog_test_ref_kfunc *bpf_kfunc_call_test_acquire(unsigned long *sp) __ksym; +extern void bpf_kfunc_call_test_release(struct prog_test_ref_kfunc *p) __ksym; + +SEC("tc") +int map_btf_ptr(struct __sk_buff *ctx) +{ + struct prog_test_ref_kfunc *p; + char buf[sizeof(*p)]; + struct map_value *v; + + v = bpf_map_lookup_elem(&array_map, &(int){0}); + if (!v) + return 0; + p = v->unref_ptr; + /* store untrusted_ptr_or_null_ */ + v->unref_ptr = p; + if (!p) + return 0; + if (p->a + p->b > 100) + return 1; + /* store untrusted_ptr_ */ + v->unref_ptr = p; + /* store NULL */ + v->unref_ptr = NULL; + + p = v->ref_ptr; + /* store ptr_or_null_ */ + v->unref_ptr = p; + if (!p) + return 0; + if (p->a + p->b > 100) + return 1; + /* store NULL */ + p = xchg(v->ref_ptr, NULL); + if (!p) + return 0; + if (p->a + p->b > 100) { + bpf_kfunc_call_test_release(p); + return 1; + } + /* store ptr_ */ + v->unref_ptr = p; + bpf_kfunc_call_test_release(p); + + p = bpf_kfunc_call_test_acquire(&(unsigned long){0}); + if (!p) + return 0; + /* store ptr_ */ + p = xchg(v->ref_ptr, p); + if (!p) + return 0; + if (p->a + p->b > 100) { + bpf_kfunc_call_test_release(p); + return 1; + } + bpf_kfunc_call_test_release(p); + + p = v->percpu_ptr; + /* store percpu_ptr_or_null_ */ + v->percpu_ptr = p; + if (!p) + return 0; + p = bpf_this_cpu_ptr(p); + if (p->a + p->b > 100) + return 1; + /* store percpu_ptr_ */ + v->percpu_ptr = p; + /* store NULL */ + v->percpu_ptr = NULL; + + p = v->user_ptr; + /* store user_ptr_or_null_ */ + v->user_ptr = p; + if (!p) + return 0; + bpf_probe_read_user(buf, sizeof(buf), p); + /* store user_ptr_ */ + v->user_ptr = p; + /* store NULL */ + v->user_ptr = NULL; + return 0; +} + +char _license[] SEC("license") = "GPL"; diff --git a/tools/testing/selftests/bpf/progs/test_bpf_nf.c b/tools/testing/selftests/bpf/progs/test_bpf_nf.c index f00a9731930e..74e3892be544 100644 --- a/tools/testing/selftests/bpf/progs/test_bpf_nf.c +++ b/tools/testing/selftests/bpf/progs/test_bpf_nf.c @@ -30,8 +30,21 @@ struct nf_conn *bpf_xdp_ct_lookup(struct xdp_md *, struct bpf_sock_tuple *, u32, struct bpf_ct_opts___local *, u32) __ksym; struct nf_conn *bpf_skb_ct_lookup(struct __sk_buff *, struct bpf_sock_tuple *, u32, struct bpf_ct_opts___local *, u32) __ksym; +struct nf_conn *bpf_ct_kptr_get(struct nf_conn **, struct bpf_sock_tuple *, u32, + u8, u8) __ksym; void bpf_ct_release(struct nf_conn *) __ksym; +struct nf_map_value { + struct nf_conn __kptr_ref *ct; +}; + +struct { + __uint(type, BPF_MAP_TYPE_ARRAY); + __type(key, int); + __type(value, struct nf_map_value); + __uint(max_entries, 1); +} array_map SEC(".maps"); + static __always_inline void nf_ct_test(struct nf_conn *(*func)(void *, struct bpf_sock_tuple *, u32, struct bpf_ct_opts___local *, u32), @@ -101,10 +114,27 @@ nf_ct_test(struct nf_conn *(*func)(void *, struct bpf_sock_tuple *, u32, test_eafnosupport = opts_def.error; } +static __always_inline void +nf_ct_test_kptr(void) +{ + struct bpf_sock_tuple tuple = {}; + struct nf_map_value *v; + struct nf_conn *ct; + + v = bpf_map_lookup_elem(&array_map, &(int){0}); + if (!v) + return; + ct = bpf_ct_kptr_get(&v->ct, &tuple, sizeof(tuple.ipv4), IPPROTO_TCP, IP_CT_DIR_ORIGINAL); + if (!ct) + return; + bpf_ct_release(ct); +} + SEC("xdp") int nf_xdp_ct_test(struct xdp_md *ctx) { nf_ct_test((void *)bpf_xdp_ct_lookup, ctx); + nf_ct_test_kptr(); return 0; } @@ -112,6 +142,7 @@ SEC("tc") int nf_skb_ct_test(struct __sk_buff *ctx) { nf_ct_test((void *)bpf_skb_ct_lookup, ctx); + nf_ct_test_kptr(); return 0; } From patchwork Sun Feb 20 13:48:13 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kumar Kartikeya Dwivedi X-Patchwork-Id: 12752717 X-Patchwork-Delegate: bpf@iogearbox.net Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 1B7C5C433F5 for ; Sun, 20 Feb 2022 13:49:28 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S243921AbiBTNtp (ORCPT ); Sun, 20 Feb 2022 08:49:45 -0500 Received: from mxb-00190b01.gslb.pphosted.com ([23.128.96.19]:50680 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S243948AbiBTNtl (ORCPT ); Sun, 20 Feb 2022 08:49:41 -0500 Received: from mail-pj1-x1042.google.com (mail-pj1-x1042.google.com [IPv6:2607:f8b0:4864:20::1042]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 9A0D55373E; Sun, 20 Feb 2022 05:49:03 -0800 (PST) Received: by mail-pj1-x1042.google.com with SMTP id gl14-20020a17090b120e00b001bc2182c3d5so868828pjb.1; Sun, 20 Feb 2022 05:49:03 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=Rn1ercMMJUMyOEbe0u0Ak+J0xPsv9d5mep6WrWJN6GQ=; b=nWXVPC+bZ1dRY1l6CPRhHH6Y8GrKl4tzel0VgAmCBCEucF1oADeqWeUZ/eK/im+yLq HA1g+bL3m5IV7oHQ0Rwiecif9NX3b1Q629PM6GerAcQG+h9j/nnAFMhRCeaz8zNaEQPd OStzzleDwtXss6t+NZZB2kftt2bhzhrGf3JQiGkWgX9eFJHgSNOTjderYjlXJQ+JmFSf ihU2ZjKCWPvPpQDYLA3i7e7kEsLLLVmrzyy9PqjZ3s/enkdX9JLzJQyioap7KqxWuX14 /aMSq9AphbTfPfx4+dpHCb33qqbg6NOSvc/vVfaDp2VyuqT9F9QmmFmtqYA2eo+EPu0C vR7g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=Rn1ercMMJUMyOEbe0u0Ak+J0xPsv9d5mep6WrWJN6GQ=; b=R66BPqSUXTO0nEmteraZ5pfEEBarg5Mya6RkAJATJLU562qwthibjfDVEEl3s59vqZ P56nbn9cTZ7TEZCekYfJhb2r8S3eicHjZa1PnqcyRztBKnha6+Dqllow8EsMl45Bpxz1 7H4PvbDwnNvdZG+2s/qNOh9EKvW4EdSqLSgKMHfxTzYarn478FN7ivk9QexWMlqySIxT T9WArNzozqZ2eDrprLAqtmTKEU9hfXr1cifczYQ2SDtRSzQf0SbagLVUMInT+WUoOVtx OGIl6BjVGltSloH7VLlStJWGmmiyc84/ZkWj9kgAJP8ErpUgwyNqk1XKfzFENmusnSpC Amyg== X-Gm-Message-State: AOAM533ftAHifuWuuB7ifHgcJ2vbkmuGdtrhfhzhmmEe47jYj1QMtPex 1kaI67J1fyYyd6eaGSTFE2vOAlQLM/U= X-Google-Smtp-Source: ABdhPJw3d+Y4pWKqabXJSSt2EW67GdBYwUFT8JHXl2qRYTS9L51qRXkU/wcYDBa0xq5cHT38RV7hxA== X-Received: by 2002:a17:90b:4aca:b0:1b9:ed62:b917 with SMTP id mh10-20020a17090b4aca00b001b9ed62b917mr21309060pjb.237.1645364942555; Sun, 20 Feb 2022 05:49:02 -0800 (PST) Received: from localhost ([2405:201:6014:d0c0:6243:316e:a9e1:adda]) by smtp.gmail.com with ESMTPSA id s40sm9914049pfg.145.2022.02.20.05.49.01 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 20 Feb 2022 05:49:02 -0800 (PST) From: Kumar Kartikeya Dwivedi To: bpf@vger.kernel.org Cc: Alexei Starovoitov , Andrii Nakryiko , Daniel Borkmann , =?utf-8?q?Toke_H=C3=B8iland-J=C3=B8?= =?utf-8?q?rgensen?= , Jesper Dangaard Brouer , netfilter-devel@vger.kernel.org, netdev@vger.kernel.org Subject: [PATCH bpf-next v1 15/15] selftests/bpf: Add verifier tests for PTR_TO_BTF_ID in map Date: Sun, 20 Feb 2022 19:18:13 +0530 Message-Id: <20220220134813.3411982-16-memxor@gmail.com> X-Mailer: git-send-email 2.35.1 In-Reply-To: <20220220134813.3411982-1-memxor@gmail.com> References: <20220220134813.3411982-1-memxor@gmail.com> MIME-Version: 1.0 X-Developer-Signature: v=1; a=openpgp-sha256; l=27272; h=from:subject; bh=YKF7x2OTXg6vHQsh24VXYPJ7XmPuz1q4g1niFGl3JyQ=; b=owEBbQKS/ZANAwAIAUzgyIZIvxHKAcsmYgBiEkZYoEI9MXffYxMtqNByf4pbjg2UjrHRRHnC5FMr RRh04NeJAjMEAAEIAB0WIQRLvip+Buz51YI8YRFM4MiGSL8RygUCYhJGWAAKCRBM4MiGSL8RyteGEA CKRX77DFgv6TD15qKvkjb8r5T1arK+cDpgOuFMPI7CcWEcEOubztHl6KI4VjpCVUBHdAtbVaCwDA6e qBDitfV3Z3/mKdIOwi6qposZRksyIcw8hnvu0HlwCAmMiKbwtF4qH39xC0WWGRpLvJfXJQHgEP3xl0 d6zFWXcUyQBmH0UeHmH3uqp+SlXGI5Z0amtEAdL6kuvnJKS+GmoK8/iVdynAYJWx46KQUbAxANiK6/ AxltKMRVn4eX6wFTjMbkfQJcuOID7W/+augiqbM6VA4R0YjsO1ESLFD6yDSQeyvZISt3cmPgbcKByn +naIp+xerOaMoY5olYL6xDUH1Wc/RojS29OgyV09JvzCHONeU3YNr5Oy1tXnM1NvcmjVepLL0Ewa2D wZvpQgn/wZ2i3pq6ahLZ4g4le+ZC4dsfvmoUNWN+IAX36u8S67GU3Im/DVP7GXpB2estnSlzgeP/EG i0D5VHpL/nOQXiKR4pWVfkr4PtqbrODUnd2wSurRYqj5vnqv2qeElhMiuR654/bZNKasDHofFrn9/v ylG6EzNJwEsSu11gNwijPHENMMcxN6Tb8oxlkg5/obETQ6TE8Y1iRer5hjI8L3x10B81xjUaJSuPT4 kpxM71y1tRNW48rMRQah//fx6CX2s7URd0e53lESHIEz/GOJtTJrH7ZiSZuQ== X-Developer-Key: i=memxor@gmail.com; a=openpgp; fpr=4BBE2A7E06ECF9D5823C61114CE0C88648BF11CA Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org X-Patchwork-Delegate: bpf@iogearbox.net Reuse bpf_prog_test functions to test the support for PTR_TO_BTF_ID in BPF map case, including some tests that verify implementation sanity and corner cases. Signed-off-by: Kumar Kartikeya Dwivedi --- net/bpf/test_run.c | 17 +- tools/testing/selftests/bpf/test_verifier.c | 57 +- .../selftests/bpf/verifier/map_btf_ptr.c | 624 ++++++++++++++++++ 3 files changed, 695 insertions(+), 3 deletions(-) create mode 100644 tools/testing/selftests/bpf/verifier/map_btf_ptr.c diff --git a/net/bpf/test_run.c b/net/bpf/test_run.c index f08034500813..caa289f63849 100644 --- a/net/bpf/test_run.c +++ b/net/bpf/test_run.c @@ -1263,8 +1263,23 @@ static const struct btf_kfunc_id_set bpf_prog_test_kfunc_set = { .ret_null_set = &test_sk_ret_null_kfunc_ids, }; +BTF_ID_LIST(bpf_prog_test_dtor_kfunc_ids) +BTF_ID(struct, prog_test_ref_kfunc) +BTF_ID(func, bpf_kfunc_call_test_release) + static int __init bpf_prog_test_run_init(void) { - return register_btf_kfunc_id_set(BPF_PROG_TYPE_SCHED_CLS, &bpf_prog_test_kfunc_set); + const struct btf_id_dtor_kfunc bpf_prog_test_dtor_kfunc[] = { + { + .btf_id = bpf_prog_test_dtor_kfunc_ids[0], + .kfunc_btf_id = bpf_prog_test_dtor_kfunc_ids[1] + }, + }; + int ret; + + ret = register_btf_kfunc_id_set(BPF_PROG_TYPE_SCHED_CLS, &bpf_prog_test_kfunc_set); + return ret ?: register_btf_id_dtor_kfuncs(bpf_prog_test_dtor_kfunc, + ARRAY_SIZE(bpf_prog_test_dtor_kfunc), + THIS_MODULE); } late_initcall(bpf_prog_test_run_init); diff --git a/tools/testing/selftests/bpf/test_verifier.c b/tools/testing/selftests/bpf/test_verifier.c index 92e3465fbae8..9ec0c4457396 100644 --- a/tools/testing/selftests/bpf/test_verifier.c +++ b/tools/testing/selftests/bpf/test_verifier.c @@ -54,7 +54,7 @@ #define MAX_INSNS BPF_MAXINSNS #define MAX_TEST_INSNS 1000000 #define MAX_FIXUPS 8 -#define MAX_NR_MAPS 22 +#define MAX_NR_MAPS 23 #define MAX_TEST_RUNS 8 #define POINTER_VALUE 0xcafe4all #define TEST_DATA_LEN 64 @@ -98,6 +98,7 @@ struct bpf_test { int fixup_map_reuseport_array[MAX_FIXUPS]; int fixup_map_ringbuf[MAX_FIXUPS]; int fixup_map_timer[MAX_FIXUPS]; + int fixup_map_btf_ptr[MAX_FIXUPS]; struct kfunc_btf_id_pair fixup_kfunc_btf_id[MAX_FIXUPS]; /* Expected verifier log output for result REJECT or VERBOSE_ACCEPT. * Can be a tab-separated sequence of expected strings. An empty string @@ -618,8 +619,13 @@ static int create_cgroup_storage(bool percpu) * struct timer { * struct bpf_timer t; * }; + * struct btf_ptr { + * struct prog_test_ref_kfunc __btf_id *ptr; + * } */ -static const char btf_str_sec[] = "\0bpf_spin_lock\0val\0cnt\0l\0bpf_timer\0timer\0t"; +static const char btf_str_sec[] = "\0bpf_spin_lock\0val\0cnt\0l\0bpf_timer\0timer\0t" + "\0btf_ptr\0prog_test_ref_kfunc\0ptr\0kernel.bpf.btf_id" + "\0kernel.bpf.ref\0kernel.bpf.percpu\0kernel.bpf.user"; static __u32 btf_raw_types[] = { /* int */ BTF_TYPE_INT_ENC(0, BTF_INT_SIGNED, 0, 32, 4), /* [1] */ @@ -635,6 +641,26 @@ static __u32 btf_raw_types[] = { /* struct timer */ /* [5] */ BTF_TYPE_ENC(35, BTF_INFO_ENC(BTF_KIND_STRUCT, 0, 1), 16), BTF_MEMBER_ENC(41, 4, 0), /* struct bpf_timer t; */ + /* struct prog_test_ref_kfunc */ /* [6] */ + BTF_STRUCT_ENC(51, 0, 0), + /* type tag "kernel.bpf.btf_id" */ + BTF_TYPE_TAG_ENC(75, 6), /* [7] */ + /* type tag "kernel.bpf.ref" */ + BTF_TYPE_TAG_ENC(93, 7), /* [8] */ + /* type tag "kernel.bpf.percpu" */ + BTF_TYPE_TAG_ENC(108, 7), /* [9] */ + /* type tag "kernel.bpf.user" */ + BTF_TYPE_TAG_ENC(126, 7), /* [10] */ + BTF_PTR_ENC(7), /* [11] */ + BTF_PTR_ENC(8), /* [12] */ + BTF_PTR_ENC(9), /* [13] */ + BTF_PTR_ENC(10), /* [14] */ + /* struct btf_ptr */ /* [15] */ + BTF_STRUCT_ENC(43, 4, 32), + BTF_MEMBER_ENC(71, 11, 0), /* struct prog_test_ref_kfunc __kptr *ptr; */ + BTF_MEMBER_ENC(71, 12, 64), /* struct prog_test_ref_kfunc __kptr_ref *ptr; */ + BTF_MEMBER_ENC(71, 13, 128), /* struct prog_test_ref_kfunc __kptr_percpu *ptr; */ + BTF_MEMBER_ENC(71, 14, 192), /* struct prog_test_ref_kfunc __kptr_user *ptr; */ }; static int load_btf(void) @@ -724,6 +750,25 @@ static int create_map_timer(void) return fd; } +static int create_map_btf_ptr(void) +{ + LIBBPF_OPTS(bpf_map_create_opts, opts, + .btf_key_type_id = 1, + .btf_value_type_id = 15, + ); + int fd, btf_fd; + + btf_fd = load_btf(); + if (btf_fd < 0) + return -1; + + opts.btf_fd = btf_fd; + fd = bpf_map_create(BPF_MAP_TYPE_ARRAY, "test_map", 4, 32, 1, &opts); + if (fd < 0) + printf("Failed to create map with btf_id pointer\n"); + return fd; +} + static char bpf_vlog[UINT_MAX >> 8]; static void do_test_fixup(struct bpf_test *test, enum bpf_prog_type prog_type, @@ -751,6 +796,7 @@ static void do_test_fixup(struct bpf_test *test, enum bpf_prog_type prog_type, int *fixup_map_reuseport_array = test->fixup_map_reuseport_array; int *fixup_map_ringbuf = test->fixup_map_ringbuf; int *fixup_map_timer = test->fixup_map_timer; + int *fixup_map_btf_ptr = test->fixup_map_btf_ptr; struct kfunc_btf_id_pair *fixup_kfunc_btf_id = test->fixup_kfunc_btf_id; if (test->fill_helper) { @@ -944,6 +990,13 @@ static void do_test_fixup(struct bpf_test *test, enum bpf_prog_type prog_type, fixup_map_timer++; } while (*fixup_map_timer); } + if (*fixup_map_btf_ptr) { + map_fds[22] = create_map_btf_ptr(); + do { + prog[*fixup_map_btf_ptr].imm = map_fds[22]; + fixup_map_btf_ptr++; + } while (*fixup_map_btf_ptr); + } /* Patch in kfunc BTF IDs */ if (fixup_kfunc_btf_id->kfunc) { diff --git a/tools/testing/selftests/bpf/verifier/map_btf_ptr.c b/tools/testing/selftests/bpf/verifier/map_btf_ptr.c new file mode 100644 index 000000000000..89d854ce90eb --- /dev/null +++ b/tools/testing/selftests/bpf/verifier/map_btf_ptr.c @@ -0,0 +1,624 @@ +/* Common tests */ +{ + "map_btf_ptr: BPF_ST imm != 0", + .insns = { + BPF_MOV64_REG(BPF_REG_2, BPF_REG_10), + BPF_LD_MAP_FD(BPF_REG_6, 0), + BPF_MOV64_REG(BPF_REG_1, BPF_REG_6), + BPF_ALU64_IMM(BPF_ADD, BPF_REG_2, -4), + BPF_MOV64_IMM(BPF_REG_0, 0), + BPF_ST_MEM(BPF_W, BPF_REG_2, 0, 0), + BPF_RAW_INSN(BPF_JMP | BPF_CALL, 0, 0, 0, BPF_FUNC_map_lookup_elem), + BPF_JMP_IMM(BPF_JNE, BPF_REG_0, 0, 1), + BPF_EXIT_INSN(), + BPF_ST_MEM(BPF_DW, BPF_REG_0, 0, 1), + BPF_EXIT_INSN(), + }, + .fixup_map_btf_ptr = { 1 }, + .result_unpriv = REJECT, + .result = REJECT, + .errstr = "BPF_ST imm must be 0 when writing to btf_id pointer at off=0", +}, +{ + "map_btf_ptr: size != bpf_size_to_bytes(BPF_DW)", + .insns = { + BPF_MOV64_REG(BPF_REG_2, BPF_REG_10), + BPF_LD_MAP_FD(BPF_REG_6, 0), + BPF_MOV64_REG(BPF_REG_1, BPF_REG_6), + BPF_ALU64_IMM(BPF_ADD, BPF_REG_2, -4), + BPF_MOV64_IMM(BPF_REG_0, 0), + BPF_ST_MEM(BPF_W, BPF_REG_2, 0, 0), + BPF_RAW_INSN(BPF_JMP | BPF_CALL, 0, 0, 0, BPF_FUNC_map_lookup_elem), + BPF_JMP_IMM(BPF_JNE, BPF_REG_0, 0, 1), + BPF_EXIT_INSN(), + BPF_ST_MEM(BPF_W, BPF_REG_0, 0, 0), + BPF_EXIT_INSN(), + }, + .fixup_map_btf_ptr = { 1 }, + .result_unpriv = REJECT, + .result = REJECT, + .errstr = "btf_id pointer load/store size must be 8", +}, +{ + "map_btf_ptr: map_value non-const var_off", + .insns = { + BPF_MOV64_REG(BPF_REG_2, BPF_REG_10), + BPF_LD_MAP_FD(BPF_REG_6, 0), + BPF_MOV64_REG(BPF_REG_1, BPF_REG_6), + BPF_ALU64_IMM(BPF_ADD, BPF_REG_2, -4), + BPF_MOV64_IMM(BPF_REG_0, 0), + BPF_ST_MEM(BPF_W, BPF_REG_2, 0, 0), + BPF_RAW_INSN(BPF_JMP | BPF_CALL, 0, 0, 0, BPF_FUNC_map_lookup_elem), + BPF_JMP_IMM(BPF_JNE, BPF_REG_0, 0, 1), + BPF_EXIT_INSN(), + BPF_MOV64_REG(BPF_REG_3, BPF_REG_0), + BPF_LDX_MEM(BPF_DW, BPF_REG_2, BPF_REG_0, 0), + BPF_JMP_IMM(BPF_JNE, BPF_REG_2, 0, 1), + BPF_EXIT_INSN(), + BPF_LDX_MEM(BPF_W, BPF_REG_2, BPF_REG_2, 0), + BPF_JMP_IMM(BPF_JLE, BPF_REG_2, 4, 1), + BPF_EXIT_INSN(), + BPF_JMP_IMM(BPF_JGE, BPF_REG_2, 0, 1), + BPF_EXIT_INSN(), + BPF_ALU64_REG(BPF_ADD, BPF_REG_3, BPF_REG_2), + BPF_LDX_MEM(BPF_DW, BPF_REG_0, BPF_REG_3, 0), + BPF_EXIT_INSN(), + }, + .fixup_map_btf_ptr = { 1 }, + .result_unpriv = REJECT, + .result = REJECT, + .errstr = "btf_id pointer cannot be accessed by variable offset load/store", +}, +{ + "map_btf_ptr: unaligned boundary load/store", + .insns = { + BPF_MOV64_REG(BPF_REG_2, BPF_REG_10), + BPF_LD_MAP_FD(BPF_REG_6, 0), + BPF_MOV64_REG(BPF_REG_1, BPF_REG_6), + BPF_ALU64_IMM(BPF_ADD, BPF_REG_2, -4), + BPF_MOV64_IMM(BPF_REG_0, 0), + BPF_ST_MEM(BPF_W, BPF_REG_2, 0, 0), + BPF_RAW_INSN(BPF_JMP | BPF_CALL, 0, 0, 0, BPF_FUNC_map_lookup_elem), + BPF_JMP_IMM(BPF_JNE, BPF_REG_0, 0, 1), + BPF_EXIT_INSN(), + BPF_ALU64_IMM(BPF_ADD, BPF_REG_0, 7), + BPF_ST_MEM(BPF_DW, BPF_REG_0, 0, 0), + BPF_EXIT_INSN(), + }, + .fixup_map_btf_ptr = { 1 }, + .result_unpriv = REJECT, + .result = REJECT, + .errstr = "btf_id pointer offset incorrect", +}, +{ + "map_btf_ptr: reject var_off != 0", + .insns = { + BPF_MOV64_REG(BPF_REG_2, BPF_REG_10), + BPF_LD_MAP_FD(BPF_REG_6, 0), + BPF_MOV64_REG(BPF_REG_1, BPF_REG_6), + BPF_ALU64_IMM(BPF_ADD, BPF_REG_2, -4), + BPF_MOV64_IMM(BPF_REG_0, 0), + BPF_ST_MEM(BPF_W, BPF_REG_2, 0, 0), + BPF_RAW_INSN(BPF_JMP | BPF_CALL, 0, 0, 0, BPF_FUNC_map_lookup_elem), + BPF_JMP_IMM(BPF_JNE, BPF_REG_0, 0, 1), + BPF_EXIT_INSN(), + BPF_LDX_MEM(BPF_DW, BPF_REG_1, BPF_REG_0, 0), + BPF_JMP_IMM(BPF_JNE, BPF_REG_1, 0, 1), + BPF_EXIT_INSN(), + BPF_LDX_MEM(BPF_W, BPF_REG_2, BPF_REG_1, 0), + BPF_JMP_IMM(BPF_JLE, BPF_REG_2, 4, 1), + BPF_EXIT_INSN(), + BPF_JMP_IMM(BPF_JGE, BPF_REG_2, 0, 1), + BPF_EXIT_INSN(), + BPF_ALU64_REG(BPF_ADD, BPF_REG_1, BPF_REG_2), + BPF_STX_MEM(BPF_DW, BPF_REG_0, BPF_REG_1, 0), + BPF_EXIT_INSN(), + }, + .fixup_map_btf_ptr = { 1 }, + .result_unpriv = REJECT, + .result = REJECT, + .errstr = "R1 is ptr_prog_test_ref_kfunc invalid variable offset: off=0, var_off=(0x0; 0x7)", +}, +/* Tests for unreferened PTR_TO_BTF_ID */ +{ + "map_btf_ptr: unref: reject btf_struct_ids_match == false", + .insns = { + BPF_MOV64_REG(BPF_REG_2, BPF_REG_10), + BPF_LD_MAP_FD(BPF_REG_6, 0), + BPF_MOV64_REG(BPF_REG_1, BPF_REG_6), + BPF_ALU64_IMM(BPF_ADD, BPF_REG_2, -4), + BPF_MOV64_IMM(BPF_REG_0, 0), + BPF_ST_MEM(BPF_W, BPF_REG_2, 0, 0), + BPF_RAW_INSN(BPF_JMP | BPF_CALL, 0, 0, 0, BPF_FUNC_map_lookup_elem), + BPF_JMP_IMM(BPF_JNE, BPF_REG_0, 0, 1), + BPF_EXIT_INSN(), + BPF_LDX_MEM(BPF_DW, BPF_REG_1, BPF_REG_0, 0), + BPF_JMP_IMM(BPF_JNE, BPF_REG_1, 0, 1), + BPF_EXIT_INSN(), + BPF_ALU64_IMM(BPF_ADD, BPF_REG_1, 4), + BPF_STX_MEM(BPF_DW, BPF_REG_0, BPF_REG_1, 0), + BPF_EXIT_INSN(), + }, + .fixup_map_btf_ptr = { 1 }, + .result_unpriv = REJECT, + .result = REJECT, + .errstr = "invalid btf_id pointer access, R1 type=untrusted_ptr_prog_test_ref_kfunc expected=ptr_or_null_prog_test", +}, +{ + "map_btf_ptr: unref: loaded pointer marked as untrusted", + .insns = { + BPF_MOV64_REG(BPF_REG_2, BPF_REG_10), + BPF_LD_MAP_FD(BPF_REG_6, 0), + BPF_MOV64_REG(BPF_REG_1, BPF_REG_6), + BPF_ALU64_IMM(BPF_ADD, BPF_REG_2, -4), + BPF_MOV64_IMM(BPF_REG_0, 0), + BPF_ST_MEM(BPF_W, BPF_REG_2, 0, 0), + BPF_RAW_INSN(BPF_JMP | BPF_CALL, 0, 0, 0, BPF_FUNC_map_lookup_elem), + BPF_JMP_IMM(BPF_JNE, BPF_REG_0, 0, 1), + BPF_EXIT_INSN(), + BPF_LDX_MEM(BPF_DW, BPF_REG_0, BPF_REG_0, 0), + BPF_LDX_MEM(BPF_W, BPF_REG_0, BPF_REG_0, 0), + BPF_EXIT_INSN(), + }, + .fixup_map_btf_ptr = { 1 }, + .result_unpriv = REJECT, + .result = REJECT, + .errstr = "R0 invalid mem access 'untrusted_ptr_or_null_'", +}, +{ + "map_btf_ptr: unref: correct in kernel type size", + .insns = { + BPF_MOV64_REG(BPF_REG_2, BPF_REG_10), + BPF_LD_MAP_FD(BPF_REG_6, 0), + BPF_MOV64_REG(BPF_REG_1, BPF_REG_6), + BPF_ALU64_IMM(BPF_ADD, BPF_REG_2, -4), + BPF_MOV64_IMM(BPF_REG_0, 0), + BPF_ST_MEM(BPF_W, BPF_REG_2, 0, 0), + BPF_RAW_INSN(BPF_JMP | BPF_CALL, 0, 0, 0, BPF_FUNC_map_lookup_elem), + BPF_JMP_IMM(BPF_JNE, BPF_REG_0, 0, 1), + BPF_EXIT_INSN(), + BPF_LDX_MEM(BPF_DW, BPF_REG_0, BPF_REG_0, 0), + BPF_JMP_IMM(BPF_JNE, BPF_REG_0, 0, 1), + BPF_EXIT_INSN(), + BPF_LDX_MEM(BPF_DW, BPF_REG_0, BPF_REG_0, 16), + BPF_EXIT_INSN(), + }, + .fixup_map_btf_ptr = { 1 }, + .result_unpriv = REJECT, + .result = REJECT, + .errstr = "access beyond struct prog_test_ref_kfunc at off 16 size 8", +}, +{ + "map_btf_ptr: unref: inherit PTR_UNTRUSTED on struct walk", + .insns = { + BPF_MOV64_REG(BPF_REG_2, BPF_REG_10), + BPF_LD_MAP_FD(BPF_REG_6, 0), + BPF_MOV64_REG(BPF_REG_1, BPF_REG_6), + BPF_ALU64_IMM(BPF_ADD, BPF_REG_2, -4), + BPF_MOV64_IMM(BPF_REG_0, 0), + BPF_ST_MEM(BPF_W, BPF_REG_2, 0, 0), + BPF_RAW_INSN(BPF_JMP | BPF_CALL, 0, 0, 0, BPF_FUNC_map_lookup_elem), + BPF_JMP_IMM(BPF_JNE, BPF_REG_0, 0, 1), + BPF_EXIT_INSN(), + BPF_LDX_MEM(BPF_DW, BPF_REG_0, BPF_REG_0, 0), + BPF_JMP_IMM(BPF_JNE, BPF_REG_0, 0, 1), + BPF_EXIT_INSN(), + BPF_LDX_MEM(BPF_DW, BPF_REG_1, BPF_REG_0, 8), + BPF_RAW_INSN(BPF_JMP | BPF_CALL, 0, 0, 0, BPF_FUNC_this_cpu_ptr), + BPF_EXIT_INSN(), + }, + .fixup_map_btf_ptr = { 1 }, + .result_unpriv = REJECT, + .result = REJECT, + .errstr = "R1 type=untrusted_ptr_ expected=percpu_ptr_", +}, +{ + "map_btf_ptr: unref: no reference state created", + .insns = { + BPF_MOV64_REG(BPF_REG_2, BPF_REG_10), + BPF_LD_MAP_FD(BPF_REG_6, 0), + BPF_MOV64_REG(BPF_REG_1, BPF_REG_6), + BPF_ALU64_IMM(BPF_ADD, BPF_REG_2, -4), + BPF_MOV64_IMM(BPF_REG_0, 0), + BPF_ST_MEM(BPF_W, BPF_REG_2, 0, 0), + BPF_RAW_INSN(BPF_JMP | BPF_CALL, 0, 0, 0, BPF_FUNC_map_lookup_elem), + BPF_JMP_IMM(BPF_JNE, BPF_REG_0, 0, 1), + BPF_EXIT_INSN(), + BPF_LDX_MEM(BPF_DW, BPF_REG_0, BPF_REG_0, 0), + BPF_JMP_IMM(BPF_JNE, BPF_REG_0, 0, 1), + BPF_EXIT_INSN(), + BPF_EXIT_INSN(), + }, + .fixup_map_btf_ptr = { 1 }, + .result_unpriv = REJECT, + .result = ACCEPT, +}, +{ + "map_btf_ptr: unref: xchg no reference state created", + .insns = { + BPF_MOV64_REG(BPF_REG_2, BPF_REG_10), + BPF_LD_MAP_FD(BPF_REG_6, 0), + BPF_MOV64_REG(BPF_REG_1, BPF_REG_6), + BPF_ALU64_IMM(BPF_ADD, BPF_REG_2, -4), + BPF_MOV64_IMM(BPF_REG_0, 0), + BPF_ST_MEM(BPF_W, BPF_REG_2, 0, 0), + BPF_RAW_INSN(BPF_JMP | BPF_CALL, 0, 0, 0, BPF_FUNC_map_lookup_elem), + BPF_JMP_IMM(BPF_JNE, BPF_REG_0, 0, 1), + BPF_EXIT_INSN(), + BPF_MOV64_IMM(BPF_REG_1, 0), + BPF_ATOMIC_OP(BPF_DW, BPF_XCHG, BPF_REG_0, BPF_REG_1, 0), + BPF_MOV64_IMM(BPF_REG_0, 0), + BPF_JMP_IMM(BPF_JNE, BPF_REG_1, 0, 1), + BPF_EXIT_INSN(), + BPF_EXIT_INSN(), + }, + .fixup_map_btf_ptr = { 1 }, + .result_unpriv = REJECT, + .result = ACCEPT, +}, +/* Tests for referenced PTR_TO_BTF_ID */ +{ + "map_btf_ptr: ref: loaded pointer marked as untrusted", + .insns = { + BPF_MOV64_REG(BPF_REG_2, BPF_REG_10), + BPF_LD_MAP_FD(BPF_REG_6, 0), + BPF_MOV64_REG(BPF_REG_1, BPF_REG_6), + BPF_ALU64_IMM(BPF_ADD, BPF_REG_2, -4), + BPF_MOV64_IMM(BPF_REG_0, 0), + BPF_ST_MEM(BPF_W, BPF_REG_2, 0, 0), + BPF_RAW_INSN(BPF_JMP | BPF_CALL, 0, 0, 0, BPF_FUNC_map_lookup_elem), + BPF_JMP_IMM(BPF_JNE, BPF_REG_0, 0, 1), + BPF_EXIT_INSN(), + BPF_MOV64_IMM(BPF_REG_1, 0), + BPF_LDX_MEM(BPF_DW, BPF_REG_1, BPF_REG_0, 8), + BPF_RAW_INSN(BPF_JMP | BPF_CALL, 0, 0, 0, BPF_FUNC_this_cpu_ptr), + BPF_EXIT_INSN(), + }, + .fixup_map_btf_ptr = { 1 }, + .result_unpriv = REJECT, + .result = REJECT, + .errstr = "R1 type=untrusted_ptr_or_null_ expected=percpu_ptr_", +}, +{ + "map_btf_ptr: ref: reject off != 0", + .insns = { + BPF_MOV64_REG(BPF_REG_2, BPF_REG_10), + BPF_LD_MAP_FD(BPF_REG_6, 0), + BPF_MOV64_REG(BPF_REG_1, BPF_REG_6), + BPF_ALU64_IMM(BPF_ADD, BPF_REG_2, -4), + BPF_MOV64_IMM(BPF_REG_0, 0), + BPF_ST_MEM(BPF_W, BPF_REG_2, 0, 0), + BPF_RAW_INSN(BPF_JMP | BPF_CALL, 0, 0, 0, BPF_FUNC_map_lookup_elem), + BPF_JMP_IMM(BPF_JNE, BPF_REG_0, 0, 1), + BPF_EXIT_INSN(), + BPF_MOV64_IMM(BPF_REG_1, 0), + BPF_ATOMIC_OP(BPF_DW, BPF_XCHG, BPF_REG_0, BPF_REG_1, 8), + BPF_JMP_IMM(BPF_JNE, BPF_REG_1, 0, 1), + BPF_EXIT_INSN(), + BPF_ALU64_IMM(BPF_ADD, BPF_REG_1, 4), + BPF_ATOMIC_OP(BPF_DW, BPF_XCHG, BPF_REG_0, BPF_REG_1, 8), + BPF_EXIT_INSN(), + }, + .fixup_map_btf_ptr = { 1 }, + .result_unpriv = REJECT, + .result = REJECT, + .errstr = "R1 stored to referenced btf_id pointer cannot have non-zero offset", +}, +{ + "map_btf_ptr: ref: reference state created on xchg", + .insns = { + BPF_MOV64_REG(BPF_REG_2, BPF_REG_10), + BPF_LD_MAP_FD(BPF_REG_6, 0), + BPF_MOV64_REG(BPF_REG_1, BPF_REG_6), + BPF_ALU64_IMM(BPF_ADD, BPF_REG_2, -4), + BPF_MOV64_IMM(BPF_REG_0, 0), + BPF_ST_MEM(BPF_W, BPF_REG_2, 0, 0), + BPF_RAW_INSN(BPF_JMP | BPF_CALL, 0, 0, 0, BPF_FUNC_map_lookup_elem), + BPF_JMP_IMM(BPF_JNE, BPF_REG_0, 0, 1), + BPF_EXIT_INSN(), + BPF_MOV64_REG(BPF_REG_7, BPF_REG_0), + BPF_MOV64_REG(BPF_REG_1, BPF_REG_10), + BPF_ALU64_IMM(BPF_ADD, BPF_REG_1, -8), + BPF_ST_MEM(BPF_DW, BPF_REG_1, 0, 0), + BPF_RAW_INSN(BPF_JMP | BPF_CALL, 0, BPF_PSEUDO_KFUNC_CALL, 0, 0), + BPF_JMP_IMM(BPF_JNE, BPF_REG_0, 0, 1), + BPF_EXIT_INSN(), + BPF_ATOMIC_OP(BPF_DW, BPF_XCHG, BPF_REG_7, BPF_REG_0, 8), + BPF_JMP_IMM(BPF_JNE, BPF_REG_0, 0, 1), + BPF_EXIT_INSN(), + BPF_MOV64_IMM(BPF_REG_0, 0), + BPF_EXIT_INSN(), + }, + .prog_type = BPF_PROG_TYPE_SCHED_CLS, + .fixup_map_btf_ptr = { 1 }, + .result_unpriv = REJECT, + .result = REJECT, + .errstr = "Unreleased reference id=4 alloc_insn=17", + .fixup_kfunc_btf_id = { + { "bpf_kfunc_call_test_acquire", 14 }, + } +}, +{ + "map_btf_ptr: ref: reference state cleared for src_reg", + .insns = { + BPF_MOV64_REG(BPF_REG_2, BPF_REG_10), + BPF_LD_MAP_FD(BPF_REG_6, 0), + BPF_MOV64_REG(BPF_REG_1, BPF_REG_6), + BPF_ALU64_IMM(BPF_ADD, BPF_REG_2, -4), + BPF_MOV64_IMM(BPF_REG_0, 0), + BPF_ST_MEM(BPF_W, BPF_REG_2, 0, 0), + BPF_RAW_INSN(BPF_JMP | BPF_CALL, 0, 0, 0, BPF_FUNC_map_lookup_elem), + BPF_JMP_IMM(BPF_JNE, BPF_REG_0, 0, 1), + BPF_EXIT_INSN(), + BPF_MOV64_REG(BPF_REG_7, BPF_REG_0), + BPF_MOV64_REG(BPF_REG_1, BPF_REG_10), + BPF_ALU64_IMM(BPF_ADD, BPF_REG_1, -8), + BPF_ST_MEM(BPF_DW, BPF_REG_1, 0, 0), + BPF_RAW_INSN(BPF_JMP | BPF_CALL, 0, BPF_PSEUDO_KFUNC_CALL, 0, 0), + BPF_JMP_IMM(BPF_JNE, BPF_REG_0, 0, 1), + BPF_EXIT_INSN(), + BPF_ATOMIC_OP(BPF_DW, BPF_XCHG, BPF_REG_7, BPF_REG_0, 8), + BPF_JMP_IMM(BPF_JNE, BPF_REG_0, 0, 1), + BPF_EXIT_INSN(), + BPF_MOV64_REG(BPF_REG_1, BPF_REG_0), + BPF_RAW_INSN(BPF_JMP | BPF_CALL, 0, BPF_PSEUDO_KFUNC_CALL, 0, 0), + BPF_MOV64_IMM(BPF_REG_0, 0), + BPF_EXIT_INSN(), + }, + .prog_type = BPF_PROG_TYPE_SCHED_CLS, + .fixup_map_btf_ptr = { 1 }, + .result_unpriv = REJECT, + .result = ACCEPT, + .fixup_kfunc_btf_id = { + { "bpf_kfunc_call_test_acquire", 14 }, + { "bpf_kfunc_call_test_release", 21 }, + } +}, +{ + "map_btf_ptr: ref: reject STX", + .insns = { + BPF_MOV64_REG(BPF_REG_2, BPF_REG_10), + BPF_LD_MAP_FD(BPF_REG_6, 0), + BPF_MOV64_REG(BPF_REG_1, BPF_REG_6), + BPF_ALU64_IMM(BPF_ADD, BPF_REG_2, -4), + BPF_MOV64_IMM(BPF_REG_0, 0), + BPF_ST_MEM(BPF_W, BPF_REG_2, 0, 0), + BPF_RAW_INSN(BPF_JMP | BPF_CALL, 0, 0, 0, BPF_FUNC_map_lookup_elem), + BPF_JMP_IMM(BPF_JNE, BPF_REG_0, 0, 1), + BPF_EXIT_INSN(), + BPF_MOV64_REG(BPF_REG_1, 0), + BPF_STX_MEM(BPF_DW, BPF_REG_0, BPF_REG_1, 8), + BPF_EXIT_INSN(), + }, + .fixup_map_btf_ptr = { 1 }, + .result_unpriv = REJECT, + .result = REJECT, + .errstr = "referenced btf_id pointer can only be accessed using BPF_XCHG", +}, +{ + "map_btf_ptr: ref: reject ST", + .insns = { + BPF_MOV64_REG(BPF_REG_2, BPF_REG_10), + BPF_LD_MAP_FD(BPF_REG_6, 0), + BPF_MOV64_REG(BPF_REG_1, BPF_REG_6), + BPF_ALU64_IMM(BPF_ADD, BPF_REG_2, -4), + BPF_MOV64_IMM(BPF_REG_0, 0), + BPF_ST_MEM(BPF_W, BPF_REG_2, 0, 0), + BPF_RAW_INSN(BPF_JMP | BPF_CALL, 0, 0, 0, BPF_FUNC_map_lookup_elem), + BPF_JMP_IMM(BPF_JNE, BPF_REG_0, 0, 1), + BPF_EXIT_INSN(), + BPF_ST_MEM(BPF_DW, BPF_REG_0, 8, 0), + BPF_EXIT_INSN(), + }, + .fixup_map_btf_ptr = { 1 }, + .result_unpriv = REJECT, + .result = REJECT, + .errstr = "referenced btf_id pointer can only be accessed using BPF_XCHG", +}, +/* Tests for PTR_TO_PERCPU_BTF_ID */ +{ + "map_btf_ptr: percpu: loaded pointer marked as percpu", + .insns = { + BPF_MOV64_REG(BPF_REG_2, BPF_REG_10), + BPF_LD_MAP_FD(BPF_REG_6, 0), + BPF_MOV64_REG(BPF_REG_1, BPF_REG_6), + BPF_ALU64_IMM(BPF_ADD, BPF_REG_2, -4), + BPF_MOV64_IMM(BPF_REG_0, 0), + BPF_ST_MEM(BPF_W, BPF_REG_2, 0, 0), + BPF_RAW_INSN(BPF_JMP | BPF_CALL, 0, 0, 0, BPF_FUNC_map_lookup_elem), + BPF_JMP_IMM(BPF_JNE, BPF_REG_0, 0, 1), + BPF_EXIT_INSN(), + BPF_LDX_MEM(BPF_DW, BPF_REG_1, BPF_REG_0, 16), + BPF_RAW_INSN(BPF_JMP | BPF_CALL, 0, 0, 0, BPF_FUNC_this_cpu_ptr), + BPF_EXIT_INSN(), + }, + .fixup_map_btf_ptr = { 1 }, + .result_unpriv = REJECT, + .result = REJECT, + .errstr = "R1 type=percpu_ptr_or_null_ expected=percpu_ptr_", +}, +{ + "map_btf_ptr: percpu: reject store of untrusted_ptr_", + .insns = { + BPF_MOV64_REG(BPF_REG_2, BPF_REG_10), + BPF_LD_MAP_FD(BPF_REG_6, 0), + BPF_MOV64_REG(BPF_REG_1, BPF_REG_6), + BPF_ALU64_IMM(BPF_ADD, BPF_REG_2, -4), + BPF_MOV64_IMM(BPF_REG_0, 0), + BPF_ST_MEM(BPF_W, BPF_REG_2, 0, 0), + BPF_RAW_INSN(BPF_JMP | BPF_CALL, 0, 0, 0, BPF_FUNC_map_lookup_elem), + BPF_JMP_IMM(BPF_JNE, BPF_REG_0, 0, 1), + BPF_EXIT_INSN(), + BPF_LDX_MEM(BPF_DW, BPF_REG_1, BPF_REG_0, 8), + BPF_JMP_IMM(BPF_JNE, BPF_REG_1, 0, 1), + BPF_EXIT_INSN(), + BPF_STX_MEM(BPF_DW, BPF_REG_0, BPF_REG_1, 16), + BPF_EXIT_INSN(), + }, + .fixup_map_btf_ptr = { 1 }, + .result_unpriv = REJECT, + .result = REJECT, + .errstr = "invalid btf_id pointer access, R1 type=untrusted_ptr_ expected=percpu_ptr_or_null_", +}, +{ + "map_btf_ptr: percpu: reject store of ptr_", + .insns = { + BPF_MOV64_REG(BPF_REG_2, BPF_REG_10), + BPF_LD_MAP_FD(BPF_REG_6, 0), + BPF_MOV64_REG(BPF_REG_1, BPF_REG_6), + BPF_ALU64_IMM(BPF_ADD, BPF_REG_2, -4), + BPF_MOV64_IMM(BPF_REG_0, 0), + BPF_ST_MEM(BPF_W, BPF_REG_2, 0, 0), + BPF_RAW_INSN(BPF_JMP | BPF_CALL, 0, 0, 0, BPF_FUNC_map_lookup_elem), + BPF_JMP_IMM(BPF_JNE, BPF_REG_0, 0, 1), + BPF_EXIT_INSN(), + BPF_MOV64_IMM(BPF_REG_1, 0), + BPF_ATOMIC_OP(BPF_DW, BPF_XCHG, BPF_REG_0, BPF_REG_1, 8), + BPF_JMP_IMM(BPF_JNE, BPF_REG_1, 0, 1), + BPF_EXIT_INSN(), + BPF_STX_MEM(BPF_DW, BPF_REG_0, BPF_REG_1, 16), + BPF_EXIT_INSN(), + }, + .fixup_map_btf_ptr = { 1 }, + .result_unpriv = REJECT, + .result = REJECT, + .errstr = "invalid btf_id pointer access, R1 type=ptr_ expected=percpu_ptr_or_null_", +}, +{ + "map_btf_ptr: percpu: reject store of user_ptr_", + .insns = { + BPF_MOV64_REG(BPF_REG_2, BPF_REG_10), + BPF_LD_MAP_FD(BPF_REG_6, 0), + BPF_MOV64_REG(BPF_REG_1, BPF_REG_6), + BPF_ALU64_IMM(BPF_ADD, BPF_REG_2, -4), + BPF_MOV64_IMM(BPF_REG_0, 0), + BPF_ST_MEM(BPF_W, BPF_REG_2, 0, 0), + BPF_RAW_INSN(BPF_JMP | BPF_CALL, 0, 0, 0, BPF_FUNC_map_lookup_elem), + BPF_JMP_IMM(BPF_JNE, BPF_REG_0, 0, 1), + BPF_EXIT_INSN(), + BPF_LDX_MEM(BPF_DW, BPF_REG_1, BPF_REG_0, 24), + BPF_JMP_IMM(BPF_JNE, BPF_REG_1, 0, 1), + BPF_EXIT_INSN(), + BPF_STX_MEM(BPF_DW, BPF_REG_0, BPF_REG_1, 16), + BPF_EXIT_INSN(), + }, + .fixup_map_btf_ptr = { 1 }, + .result_unpriv = REJECT, + .result = REJECT, + .errstr = "invalid btf_id pointer access, R1 type=user_ptr_ expected=percpu_ptr_or_null_", +}, +/* Tests for PTR_TO_BTF_ID | MEM_USR */ +{ + "map_btf_ptr: user: loaded pointer marked as user", + .insns = { + BPF_MOV64_REG(BPF_REG_2, BPF_REG_10), + BPF_LD_MAP_FD(BPF_REG_6, 0), + BPF_MOV64_REG(BPF_REG_1, BPF_REG_6), + BPF_ALU64_IMM(BPF_ADD, BPF_REG_2, -4), + BPF_MOV64_IMM(BPF_REG_0, 0), + BPF_ST_MEM(BPF_W, BPF_REG_2, 0, 0), + BPF_RAW_INSN(BPF_JMP | BPF_CALL, 0, 0, 0, BPF_FUNC_map_lookup_elem), + BPF_JMP_IMM(BPF_JNE, BPF_REG_0, 0, 1), + BPF_EXIT_INSN(), + BPF_LDX_MEM(BPF_DW, BPF_REG_1, BPF_REG_0, 24), + BPF_RAW_INSN(BPF_JMP | BPF_CALL, 0, 0, 0, BPF_FUNC_this_cpu_ptr), + BPF_EXIT_INSN(), + }, + .fixup_map_btf_ptr = { 1 }, + .result_unpriv = REJECT, + .result = REJECT, + .errstr = "R1 type=user_ptr_or_null_ expected=percpu_ptr_", +}, +{ + "map_btf_ptr: user: reject user pointer deref", + .insns = { + BPF_MOV64_REG(BPF_REG_2, BPF_REG_10), + BPF_LD_MAP_FD(BPF_REG_6, 0), + BPF_MOV64_REG(BPF_REG_1, BPF_REG_6), + BPF_ALU64_IMM(BPF_ADD, BPF_REG_2, -4), + BPF_MOV64_IMM(BPF_REG_0, 0), + BPF_ST_MEM(BPF_W, BPF_REG_2, 0, 0), + BPF_RAW_INSN(BPF_JMP | BPF_CALL, 0, 0, 0, BPF_FUNC_map_lookup_elem), + BPF_JMP_IMM(BPF_JNE, BPF_REG_0, 0, 1), + BPF_EXIT_INSN(), + BPF_LDX_MEM(BPF_DW, BPF_REG_1, BPF_REG_0, 24), + BPF_JMP_IMM(BPF_JNE, BPF_REG_1, 0, 1), + BPF_EXIT_INSN(), + BPF_LDX_MEM(BPF_W, BPF_REG_1, BPF_REG_1, 8), + BPF_EXIT_INSN(), + }, + .fixup_map_btf_ptr = { 1 }, + .result_unpriv = REJECT, + .result = REJECT, + .errstr = "R1 invalid mem access 'user_ptr_'", +}, +{ + "map_btf_ptr: user: reject store of untrusted_ptr_", + .insns = { + BPF_MOV64_REG(BPF_REG_2, BPF_REG_10), + BPF_LD_MAP_FD(BPF_REG_6, 0), + BPF_MOV64_REG(BPF_REG_1, BPF_REG_6), + BPF_ALU64_IMM(BPF_ADD, BPF_REG_2, -4), + BPF_MOV64_IMM(BPF_REG_0, 0), + BPF_ST_MEM(BPF_W, BPF_REG_2, 0, 0), + BPF_RAW_INSN(BPF_JMP | BPF_CALL, 0, 0, 0, BPF_FUNC_map_lookup_elem), + BPF_JMP_IMM(BPF_JNE, BPF_REG_0, 0, 1), + BPF_EXIT_INSN(), + BPF_LDX_MEM(BPF_DW, BPF_REG_1, BPF_REG_0, 8), + BPF_JMP_IMM(BPF_JNE, BPF_REG_1, 0, 1), + BPF_EXIT_INSN(), + BPF_STX_MEM(BPF_DW, BPF_REG_0, BPF_REG_1, 24), + BPF_EXIT_INSN(), + }, + .fixup_map_btf_ptr = { 1 }, + .result_unpriv = REJECT, + .result = REJECT, + .errstr = "invalid btf_id pointer access, R1 type=untrusted_ptr_ expected=user_ptr_or_null_", +}, +{ + "map_btf_ptr: user: reject store of ptr_", + .insns = { + BPF_MOV64_REG(BPF_REG_2, BPF_REG_10), + BPF_LD_MAP_FD(BPF_REG_6, 0), + BPF_MOV64_REG(BPF_REG_1, BPF_REG_6), + BPF_ALU64_IMM(BPF_ADD, BPF_REG_2, -4), + BPF_MOV64_IMM(BPF_REG_0, 0), + BPF_ST_MEM(BPF_W, BPF_REG_2, 0, 0), + BPF_RAW_INSN(BPF_JMP | BPF_CALL, 0, 0, 0, BPF_FUNC_map_lookup_elem), + BPF_JMP_IMM(BPF_JNE, BPF_REG_0, 0, 1), + BPF_EXIT_INSN(), + BPF_MOV64_IMM(BPF_REG_1, 0), + BPF_ATOMIC_OP(BPF_DW, BPF_XCHG, BPF_REG_0, BPF_REG_1, 8), + BPF_JMP_IMM(BPF_JNE, BPF_REG_1, 0, 1), + BPF_EXIT_INSN(), + BPF_STX_MEM(BPF_DW, BPF_REG_0, BPF_REG_1, 24), + BPF_EXIT_INSN(), + }, + .fixup_map_btf_ptr = { 1 }, + .result_unpriv = REJECT, + .result = REJECT, + .errstr = "invalid btf_id pointer access, R1 type=ptr_ expected=user_ptr_or_null_", +}, +{ + "map_btf_ptr: user: reject store of percpu_ptr_", + .insns = { + BPF_MOV64_REG(BPF_REG_2, BPF_REG_10), + BPF_LD_MAP_FD(BPF_REG_6, 0), + BPF_MOV64_REG(BPF_REG_1, BPF_REG_6), + BPF_ALU64_IMM(BPF_ADD, BPF_REG_2, -4), + BPF_MOV64_IMM(BPF_REG_0, 0), + BPF_ST_MEM(BPF_W, BPF_REG_2, 0, 0), + BPF_RAW_INSN(BPF_JMP | BPF_CALL, 0, 0, 0, BPF_FUNC_map_lookup_elem), + BPF_JMP_IMM(BPF_JNE, BPF_REG_0, 0, 1), + BPF_EXIT_INSN(), + BPF_LDX_MEM(BPF_DW, BPF_REG_1, BPF_REG_0, 16), + BPF_JMP_IMM(BPF_JNE, BPF_REG_1, 0, 1), + BPF_EXIT_INSN(), + BPF_STX_MEM(BPF_DW, BPF_REG_0, BPF_REG_1, 24), + BPF_EXIT_INSN(), + }, + .fixup_map_btf_ptr = { 1 }, + .result_unpriv = REJECT, + .result = REJECT, + .errstr = "invalid btf_id pointer access, R1 type=percpu_ptr_ expected=user_ptr_or_null_", +},