From patchwork Mon Aug 14 17:29:12 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yonghong Song X-Patchwork-Id: 13353140 X-Patchwork-Delegate: bpf@iogearbox.net Received: from lindbergh.monkeyblade.net (lindbergh.monkeyblade.net [23.128.96.19]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 473DAC2FF for ; Mon, 14 Aug 2023 17:29:28 +0000 (UTC) Received: from 66-220-155-178.mail-mxout.facebook.com (66-220-155-178.mail-mxout.facebook.com [66.220.155.178]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 0873810D0 for ; Mon, 14 Aug 2023 10:29:26 -0700 (PDT) Received: by devbig309.ftw3.facebook.com (Postfix, from userid 128203) id 2CCBA24C220BF; Mon, 14 Aug 2023 10:29:12 -0700 (PDT) From: Yonghong Song To: bpf@vger.kernel.org Cc: Alexei Starovoitov , Andrii Nakryiko , Daniel Borkmann , kernel-team@fb.com, Martin KaFai Lau Subject: [PATCH bpf-next 12/15] bpf: Allow bpf_spin_lock and bpf_list_head in allocated percpu data structure Date: Mon, 14 Aug 2023 10:29:12 -0700 Message-Id: <20230814172912.1367692-1-yonghong.song@linux.dev> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230814172809.1361446-1-yonghong.song@linux.dev> References: <20230814172809.1361446-1-yonghong.song@linux.dev> Precedence: bulk X-Mailing-List: bpf@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Spam-Status: No, score=-0.3 required=5.0 tests=BAYES_00, RCVD_IN_DNSWL_BLOCKED,RDNS_DYNAMIC,SPF_HELO_PASS,SPF_SOFTFAIL, TVD_RCVD_IP autolearn=no autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net X-Patchwork-Delegate: bpf@iogearbox.net Currently, bpf_percpu_obj_new() only allows scalar struct. A scalar struct contains scalar member or a struct with scalar member up to 3 nested struct levels. For example, below is a linked list in the per cpu object: struct val_t { long b, c, d; struct bpf_list_head head __contains(foo, node); struct bpf_spin_lock lock; }; struct map_val_t { ... struct val_t __percpu *pc; ... }; This patch added verifier support for the above percpu struct with bpf_spin_lock and bpf_list_head. The change is mostly to add MEM_RCU to various type checking places since the eventual per cpu object is marked as MEM_RCU, which is used as the input to various helpers/kfuncs like bpf_spin_lock(), bpf_spin_unlock() and bpf_list_push_back(), etc. Another possible use case is bpf_rb_root where a rb tree is in a percpu struct. The support can be added similarly in the future if needed. Signed-off-by: Yonghong Song --- kernel/bpf/verifier.c | 12 +++++++++--- 1 file changed, 9 insertions(+), 3 deletions(-) diff --git a/kernel/bpf/verifier.c b/kernel/bpf/verifier.c index 6fa458e13bfc..4c3045c8d25f 100644 --- a/kernel/bpf/verifier.c +++ b/kernel/bpf/verifier.c @@ -7848,6 +7848,8 @@ static int check_reg_type(struct bpf_verifier_env *env, u32 regno, type &= ~MEM_ALLOC; type &= ~MEM_PERCPU; } + if (meta->func_id == BPF_FUNC_spin_lock || meta->func_id == BPF_FUNC_spin_unlock) + type &= ~MEM_RCU; for (i = 0; i < ARRAY_SIZE(compatible->types); i++) { expected = compatible->types[i]; @@ -7929,6 +7931,7 @@ static int check_reg_type(struct bpf_verifier_env *env, u32 regno, } break; } + case PTR_TO_BTF_ID | MEM_ALLOC | MEM_RCU: case PTR_TO_BTF_ID | MEM_ALLOC: if (meta->func_id != BPF_FUNC_spin_lock && meta->func_id != BPF_FUNC_spin_unlock && meta->func_id != BPF_FUNC_kptr_xchg) { @@ -8040,6 +8043,7 @@ int check_func_arg_reg_off(struct bpf_verifier_env *env, case PTR_TO_BTF_ID | MEM_ALLOC: case PTR_TO_BTF_ID | PTR_TRUSTED: case PTR_TO_BTF_ID | MEM_RCU: + case PTR_TO_BTF_ID | MEM_ALLOC | MEM_RCU: case PTR_TO_BTF_ID | MEM_ALLOC | NON_OWN_REF: /* When referenced PTR_TO_BTF_ID is passed to release function, * its fixed offset must be 0. In the other cases, fixed offset @@ -10604,8 +10608,8 @@ static int ref_convert_owning_non_owning(struct bpf_verifier_env *env, u32 ref_o * active_lock.ptr = Register's type specific pointer * active_lock.id = A unique ID for each register pointer value * - * Currently, PTR_TO_MAP_VALUE and PTR_TO_BTF_ID | MEM_ALLOC are the two - * supported register types. + * Currently, PTR_TO_MAP_VALUE, PTR_TO_BTF_ID | MEM_ALLOC and + * PTR_TO_BTF_ID | MEM_ALLOC | MEM_RCU are the three supported register types. * * The active_lock.ptr in case of map values is the reg->map_ptr, and in case of * allocated objects is the reg->btf pointer. @@ -10640,6 +10644,7 @@ static int check_reg_allocation_locked(struct bpf_verifier_env *env, struct bpf_ ptr = reg->map_ptr; break; case PTR_TO_BTF_ID | MEM_ALLOC: + case PTR_TO_BTF_ID | MEM_ALLOC | MEM_RCU: ptr = reg->btf; break; default: @@ -11140,7 +11145,8 @@ static int check_kfunc_args(struct bpf_verifier_env *env, struct bpf_kfunc_call_ break; case KF_ARG_PTR_TO_LIST_HEAD: if (reg->type != PTR_TO_MAP_VALUE && - reg->type != (PTR_TO_BTF_ID | MEM_ALLOC)) { + reg->type != (PTR_TO_BTF_ID | MEM_ALLOC) && + reg->type != (PTR_TO_BTF_ID | MEM_ALLOC | MEM_RCU)) { verbose(env, "arg#%d expected pointer to map value or allocated object\n", i); return -EINVAL; }