From patchwork Thu Nov 10 18:01:30 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yonghong Song X-Patchwork-Id: 13039077 X-Patchwork-Delegate: bpf@iogearbox.net Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 18743C4332F for ; Thu, 10 Nov 2022 18:02:16 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229831AbiKJSCO (ORCPT ); Thu, 10 Nov 2022 13:02:14 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:42270 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231659AbiKJSBj (ORCPT ); Thu, 10 Nov 2022 13:01:39 -0500 Received: from mx0a-00082601.pphosted.com (mx0a-00082601.pphosted.com [67.231.145.42]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 563BD4B9A5 for ; Thu, 10 Nov 2022 10:01:39 -0800 (PST) Received: from pps.filterd (m0109334.ppops.net [127.0.0.1]) by mx0a-00082601.pphosted.com (8.17.1.5/8.17.1.5) with ESMTP id 2AAF8WPw013987 for ; Thu, 10 Nov 2022 10:01:39 -0800 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=fb.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding : content-type; s=facebook; bh=Nbtmoc9lLSKreYlbRoKrmsrKIDLlzBEi1cQlB9kvHQI=; b=anZcP1gfQbwnrnAsxnoGIGNO6cCARxUcId8wfScBsA4oU0n2FNdKEL3eSXPWOa9rBYYT 6+7TP8fU0TTkOLf6lNLLMh9A3VzwgbEOgHjD4CzO4lHXcyJtzSkfnT+kykVYACo2Byub MC2lOZK9n5o128tKWQjUw3seBfaRI5lZ/Sc= Received: from maileast.thefacebook.com ([163.114.130.16]) by mx0a-00082601.pphosted.com (PPS) with ESMTPS id 3krnrhes90-5 (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128 verify=NOT) for ; Thu, 10 Nov 2022 10:01:38 -0800 Received: from twshared14438.02.ash8.facebook.com (2620:10d:c0a8:1b::d) by mail.thefacebook.com (2620:10d:c0a8:82::c) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2375.31; Thu, 10 Nov 2022 10:01:35 -0800 Received: by devbig309.ftw3.facebook.com (Postfix, from userid 128203) id 1C09111F22DBC; Thu, 10 Nov 2022 10:01:30 -0800 (PST) From: Yonghong Song To: CC: Alexei Starovoitov , Andrii Nakryiko , Daniel Borkmann , , Martin KaFai Lau , KP Singh Subject: [PATCH bpf-next v4 1/7] compiler_types: Define __rcu as __attribute__((btf_type_tag("rcu"))) Date: Thu, 10 Nov 2022 10:01:30 -0800 Message-ID: <20221110180130.914410-1-yhs@fb.com> X-Mailer: git-send-email 2.30.2 In-Reply-To: <20221110180124.913882-1-yhs@fb.com> References: <20221110180124.913882-1-yhs@fb.com> MIME-Version: 1.0 X-FB-Internal: Safe X-Proofpoint-GUID: 5BuIceZt14zx3Z1Gz1QQB5lU_creK30H X-Proofpoint-ORIG-GUID: 5BuIceZt14zx3Z1Gz1QQB5lU_creK30H X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.219,Aquarius:18.0.895,Hydra:6.0.545,FMLib:17.11.122.1 definitions=2022-11-10_12,2022-11-09_01,2022-06-22_01 Precedence: bulk List-ID: X-Mailing-List: bpf@vger.kernel.org X-Patchwork-Delegate: bpf@iogearbox.net Currently, without rcu attribute info in BTF, the verifier treats rcu tagged pointer as a normal pointer. This might be a problem for sleepable program where rcu_read_lock()/unlock() is not available. For example, for a sleepable fentry program, if rcu protected memory access is interleaved with a sleepable helper/kfunc, it is possible the memory access after the sleepable helper/kfunc might be invalid since the object might have been freed then. To prevent such cases, introducing rcu tagging for memory accesses in verifier can help to reject such programs. To enable rcu tagging in BTF, during kernel compilation, define __rcu as attribute btf_type_tag("rcu") so __rcu information can be preserved in dwarf and btf, and later can be used for bpf prog verification. Acked-by: KP Singh Signed-off-by: Yonghong Song --- include/linux/compiler_types.h | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/include/linux/compiler_types.h b/include/linux/compiler_types.h index eb0466236661..7c1afe0f4129 100644 --- a/include/linux/compiler_types.h +++ b/include/linux/compiler_types.h @@ -49,7 +49,8 @@ static inline void __chk_io_ptr(const volatile void __iomem *ptr) { } # endif # define __iomem # define __percpu BTF_TYPE_TAG(percpu) -# define __rcu +# define __rcu BTF_TYPE_TAG(rcu) + # define __chk_user_ptr(x) (void)0 # define __chk_io_ptr(x) (void)0 /* context/locking */ From patchwork Thu Nov 10 18:01:35 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yonghong Song X-Patchwork-Id: 13039078 X-Patchwork-Delegate: bpf@iogearbox.net Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 91297C43217 for ; Thu, 10 Nov 2022 18:02:17 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230194AbiKJSCP (ORCPT ); Thu, 10 Nov 2022 13:02:15 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:43032 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230392AbiKJSBj (ORCPT ); Thu, 10 Nov 2022 13:01:39 -0500 Received: from mx0a-00082601.pphosted.com (mx0a-00082601.pphosted.com [67.231.145.42]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 4A19F49B6C for ; Thu, 10 Nov 2022 10:01:38 -0800 (PST) Received: from pps.filterd (m0109334.ppops.net [127.0.0.1]) by mx0a-00082601.pphosted.com (8.17.1.5/8.17.1.5) with ESMTP id 2AAF4239015636 for ; Thu, 10 Nov 2022 10:01:38 -0800 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=fb.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding : content-type; s=facebook; bh=Oa+FvLKeePjCM0Bs5FnU6oFps/r/BqeeRojKd+8rWBU=; b=dJMRZZvqdgC+QNeiiWdaxtfQjHPvmB4lq0nokb9y0VRRo3PAvc+XWnyFjJS9uG+N0ODE QQdtGOHdApM4G7m+g7/QdLZ5BpZpX/Mwnl3kzUJItQSQtjLWPVUBgdbzwTi3A7NrhPXh ZGn3L9rmM0wJEmhhFyCohowm9uVdmJeP8MQ= Received: from mail.thefacebook.com ([163.114.132.120]) by mx0a-00082601.pphosted.com (PPS) with ESMTPS id 3krnrhes9c-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128 verify=NOT) for ; Thu, 10 Nov 2022 10:01:37 -0800 Received: from twshared21592.39.frc1.facebook.com (2620:10d:c085:208::11) by mail.thefacebook.com (2620:10d:c085:21d::5) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2375.31; Thu, 10 Nov 2022 10:01:37 -0800 Received: by devbig309.ftw3.facebook.com (Postfix, from userid 128203) id 59FBF11F22DCE; Thu, 10 Nov 2022 10:01:35 -0800 (PST) From: Yonghong Song To: CC: Alexei Starovoitov , Andrii Nakryiko , Daniel Borkmann , , Martin KaFai Lau Subject: [PATCH bpf-next v4 2/7] bpf: Abstract out functions to check sleepable helpers Date: Thu, 10 Nov 2022 10:01:35 -0800 Message-ID: <20221110180135.914795-1-yhs@fb.com> X-Mailer: git-send-email 2.30.2 In-Reply-To: <20221110180124.913882-1-yhs@fb.com> References: <20221110180124.913882-1-yhs@fb.com> MIME-Version: 1.0 X-FB-Internal: Safe X-Proofpoint-GUID: jgrSBjPBZ4V7m-KP2FCuz-BxOt8HHGRH X-Proofpoint-ORIG-GUID: jgrSBjPBZ4V7m-KP2FCuz-BxOt8HHGRH X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.219,Aquarius:18.0.895,Hydra:6.0.545,FMLib:17.11.122.1 definitions=2022-11-10_12,2022-11-09_01,2022-06-22_01 Precedence: bulk List-ID: X-Mailing-List: bpf@vger.kernel.org X-Patchwork-Delegate: bpf@iogearbox.net Abstract out two functions to check whether a particular helper is sleepable or not for bpf_lsm and bpf_trace. These two functions will be used later to check whether a helper is sleepable or not in verifier. There is no functionality change. Signed-off-by: Yonghong Song --- include/linux/bpf_lsm.h | 6 ++++++ include/linux/trace_events.h | 8 ++++++++ kernel/bpf/bpf_lsm.c | 20 ++++++++++++++++---- kernel/trace/bpf_trace.c | 22 ++++++++++++++++++---- 4 files changed, 48 insertions(+), 8 deletions(-) diff --git a/include/linux/bpf_lsm.h b/include/linux/bpf_lsm.h index 4bcf76a9bb06..d99b1caf118e 100644 --- a/include/linux/bpf_lsm.h +++ b/include/linux/bpf_lsm.h @@ -28,6 +28,7 @@ int bpf_lsm_verify_prog(struct bpf_verifier_log *vlog, const struct bpf_prog *prog); bool bpf_lsm_is_sleepable_hook(u32 btf_id); +const struct bpf_func_proto *bpf_lsm_sleepable_func_proto(enum bpf_func_id func_id); static inline struct bpf_storage_blob *bpf_inode( const struct inode *inode) @@ -50,6 +51,11 @@ static inline bool bpf_lsm_is_sleepable_hook(u32 btf_id) { return false; } +static inline const struct bpf_func_proto * +bpf_lsm_sleepable_func_proto(enum bpf_func_id func_id) +{ + return NULL; +} static inline int bpf_lsm_verify_prog(struct bpf_verifier_log *vlog, const struct bpf_prog *prog) diff --git a/include/linux/trace_events.h b/include/linux/trace_events.h index 20749bd9db71..c3eb4fb78ea5 100644 --- a/include/linux/trace_events.h +++ b/include/linux/trace_events.h @@ -9,6 +9,7 @@ #include #include #include +#include struct trace_array; struct array_buffer; @@ -16,6 +17,7 @@ struct tracer; struct dentry; struct bpf_prog; union bpf_attr; +struct bpf_func_proto; const char *trace_print_flags_seq(struct trace_seq *p, const char *delim, unsigned long flags, @@ -748,6 +750,7 @@ int bpf_get_perf_event_info(const struct perf_event *event, u32 *prog_id, u32 *fd_type, const char **buf, u64 *probe_offset, u64 *probe_addr); int bpf_kprobe_multi_link_attach(const union bpf_attr *attr, struct bpf_prog *prog); +const struct bpf_func_proto *bpf_tracing_sleepable_func_proto(enum bpf_func_id func_id); #else static inline unsigned int trace_call_bpf(struct trace_event_call *call, void *ctx) { @@ -794,6 +797,11 @@ bpf_kprobe_multi_link_attach(const union bpf_attr *attr, struct bpf_prog *prog) { return -EOPNOTSUPP; } +static inline const struct bpf_func_proto * +bpf_tracing_sleepable_func_proto(enum bpf_func_id func_id) +{ + return NULL; +} #endif enum { diff --git a/kernel/bpf/bpf_lsm.c b/kernel/bpf/bpf_lsm.c index d6c9b3705f24..2f993a003389 100644 --- a/kernel/bpf/bpf_lsm.c +++ b/kernel/bpf/bpf_lsm.c @@ -192,6 +192,18 @@ static const struct bpf_func_proto bpf_get_attach_cookie_proto = { .arg1_type = ARG_PTR_TO_CTX, }; +const struct bpf_func_proto *bpf_lsm_sleepable_func_proto(enum bpf_func_id func_id) +{ + switch (func_id) { + case BPF_FUNC_ima_inode_hash: + return &bpf_ima_inode_hash_proto; + case BPF_FUNC_ima_file_hash: + return &bpf_ima_file_hash_proto; + default: + return NULL; + } +} + static const struct bpf_func_proto * bpf_lsm_func_proto(enum bpf_func_id func_id, const struct bpf_prog *prog) { @@ -203,6 +215,10 @@ bpf_lsm_func_proto(enum bpf_func_id func_id, const struct bpf_prog *prog) return func_proto; } + func_proto = bpf_lsm_sleepable_func_proto(func_id); + if (func_proto) + return prog->aux->sleepable ? func_proto : NULL; + switch (func_id) { case BPF_FUNC_inode_storage_get: return &bpf_inode_storage_get_proto; @@ -220,10 +236,6 @@ bpf_lsm_func_proto(enum bpf_func_id func_id, const struct bpf_prog *prog) return &bpf_spin_unlock_proto; case BPF_FUNC_bprm_opts_set: return &bpf_bprm_opts_set_proto; - case BPF_FUNC_ima_inode_hash: - return prog->aux->sleepable ? &bpf_ima_inode_hash_proto : NULL; - case BPF_FUNC_ima_file_hash: - return prog->aux->sleepable ? &bpf_ima_file_hash_proto : NULL; case BPF_FUNC_get_attach_cookie: return bpf_prog_has_trampoline(prog) ? &bpf_get_attach_cookie_proto : NULL; #ifdef CONFIG_NET diff --git a/kernel/trace/bpf_trace.c b/kernel/trace/bpf_trace.c index f2d8d070d024..7804fa71d3f1 100644 --- a/kernel/trace/bpf_trace.c +++ b/kernel/trace/bpf_trace.c @@ -1383,9 +1383,27 @@ static int __init bpf_key_sig_kfuncs_init(void) late_initcall(bpf_key_sig_kfuncs_init); #endif /* CONFIG_KEYS */ +const struct bpf_func_proto *bpf_tracing_sleepable_func_proto(enum bpf_func_id func_id) +{ + switch (func_id) { + case BPF_FUNC_copy_from_user: + return &bpf_copy_from_user_proto; + case BPF_FUNC_copy_from_user_task: + return &bpf_copy_from_user_task_proto; + default: + return NULL; + } +} + static const struct bpf_func_proto * bpf_tracing_func_proto(enum bpf_func_id func_id, const struct bpf_prog *prog) { + const struct bpf_func_proto *func_proto; + + func_proto = bpf_tracing_sleepable_func_proto(func_id); + if (func_proto) + return prog->aux->sleepable ? func_proto : NULL; + switch (func_id) { case BPF_FUNC_map_lookup_elem: return &bpf_map_lookup_elem_proto; @@ -1484,10 +1502,6 @@ bpf_tracing_func_proto(enum bpf_func_id func_id, const struct bpf_prog *prog) return &bpf_jiffies64_proto; case BPF_FUNC_get_task_stack: return &bpf_get_task_stack_proto; - case BPF_FUNC_copy_from_user: - return prog->aux->sleepable ? &bpf_copy_from_user_proto : NULL; - case BPF_FUNC_copy_from_user_task: - return prog->aux->sleepable ? &bpf_copy_from_user_task_proto : NULL; case BPF_FUNC_snprintf_btf: return &bpf_snprintf_btf_proto; case BPF_FUNC_per_cpu_ptr: From patchwork Thu Nov 10 18:01:40 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yonghong Song X-Patchwork-Id: 13039079 X-Patchwork-Delegate: bpf@iogearbox.net Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id A6278C4332F for ; Thu, 10 Nov 2022 18:02:22 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231659AbiKJSCV (ORCPT ); Thu, 10 Nov 2022 13:02:21 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:43110 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231873AbiKJSB4 (ORCPT ); Thu, 10 Nov 2022 13:01:56 -0500 Received: from mx0a-00082601.pphosted.com (mx0a-00082601.pphosted.com [67.231.145.42]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 1F0004FF96 for ; Thu, 10 Nov 2022 10:01:52 -0800 (PST) Received: from pps.filterd (m0044010.ppops.net [127.0.0.1]) by mx0a-00082601.pphosted.com (8.17.1.5/8.17.1.5) with ESMTP id 2AAEQ92o031317 for ; Thu, 10 Nov 2022 10:01:51 -0800 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=fb.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding : content-type; s=facebook; bh=Em/LHiN9t8Yi46+1Thh1R9aX+wsPId5UQwGHRJGjuTE=; b=IwvS2fDeW/4jZtfsoqErVu3hj1RDbkmMAwvofXTt5hYAKLDLwvH9k9rf5pAqFJSdH6UT 9cJ6ybmiol696bGL1Lh00EHwWS8y2wOS2ZPc/XNsMH1jtehCRzyslYM/Ad7CPziaoA+7 2GekaD2+73yGZkwXyACmDxs2Kl8HQWBJElw= Received: from maileast.thefacebook.com ([163.114.130.3]) by mx0a-00082601.pphosted.com (PPS) with ESMTPS id 3kre5uartk-2 (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128 verify=NOT) for ; Thu, 10 Nov 2022 10:01:51 -0800 Received: from twshared14438.02.ash8.facebook.com (2620:10d:c0a8:1b::d) by mail.thefacebook.com (2620:10d:c0a8:82::c) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2375.31; Thu, 10 Nov 2022 10:01:50 -0800 Received: by devbig309.ftw3.facebook.com (Postfix, from userid 128203) id 97E8F11F22DE1; Thu, 10 Nov 2022 10:01:40 -0800 (PST) From: Yonghong Song To: CC: Alexei Starovoitov , Andrii Nakryiko , Daniel Borkmann , , Martin KaFai Lau Subject: [PATCH bpf-next v4 3/7] bpf: Add kfunc bpf_rcu_read_lock/unlock() Date: Thu, 10 Nov 2022 10:01:40 -0800 Message-ID: <20221110180140.915151-1-yhs@fb.com> X-Mailer: git-send-email 2.30.2 In-Reply-To: <20221110180124.913882-1-yhs@fb.com> References: <20221110180124.913882-1-yhs@fb.com> MIME-Version: 1.0 X-FB-Internal: Safe X-Proofpoint-GUID: PHkqLabp_dfdGS4RjZzPSTZqwRTKKhi6 X-Proofpoint-ORIG-GUID: PHkqLabp_dfdGS4RjZzPSTZqwRTKKhi6 X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.219,Aquarius:18.0.895,Hydra:6.0.545,FMLib:17.11.122.1 definitions=2022-11-10_12,2022-11-09_01,2022-06-22_01 Precedence: bulk List-ID: X-Mailing-List: bpf@vger.kernel.org X-Patchwork-Delegate: bpf@iogearbox.net Add two kfunc's bpf_rcu_read_lock() and bpf_rcu_read_unlock(). These two kfunc's can be used for all program types. A new kfunc hook type BTF_KFUNC_HOOK_GENERIC is added which corresponds to prog type BPF_PROG_TYPE_UNSPEC, indicating the kfunc intends to be used for all prog types. Signed-off-by: Yonghong Song --- include/linux/bpf.h | 3 +++ kernel/bpf/btf.c | 8 ++++++++ kernel/bpf/helpers.c | 25 ++++++++++++++++++++++++- 3 files changed, 35 insertions(+), 1 deletion(-) diff --git a/include/linux/bpf.h b/include/linux/bpf.h index 798aec816970..3ed817cf191d 100644 --- a/include/linux/bpf.h +++ b/include/linux/bpf.h @@ -2113,6 +2113,9 @@ bool bpf_prog_has_kfunc_call(const struct bpf_prog *prog); const struct btf_func_model * bpf_jit_find_kfunc_model(const struct bpf_prog *prog, const struct bpf_insn *insn); +void bpf_rcu_read_lock(void); +void bpf_rcu_read_unlock(void); + struct bpf_core_ctx { struct bpf_verifier_log *log; const struct btf *btf; diff --git a/kernel/bpf/btf.c b/kernel/bpf/btf.c index 5579ff3a5b54..84f09235857c 100644 --- a/kernel/bpf/btf.c +++ b/kernel/bpf/btf.c @@ -199,6 +199,7 @@ DEFINE_IDR(btf_idr); DEFINE_SPINLOCK(btf_idr_lock); enum btf_kfunc_hook { + BTF_KFUNC_HOOK_GENERIC, BTF_KFUNC_HOOK_XDP, BTF_KFUNC_HOOK_TC, BTF_KFUNC_HOOK_STRUCT_OPS, @@ -7499,6 +7500,8 @@ static u32 *__btf_kfunc_id_set_contains(const struct btf *btf, static int bpf_prog_type_to_kfunc_hook(enum bpf_prog_type prog_type) { switch (prog_type) { + case BPF_PROG_TYPE_UNSPEC: + return BTF_KFUNC_HOOK_GENERIC; case BPF_PROG_TYPE_XDP: return BTF_KFUNC_HOOK_XDP; case BPF_PROG_TYPE_SCHED_CLS: @@ -7527,6 +7530,11 @@ u32 *btf_kfunc_id_set_contains(const struct btf *btf, u32 kfunc_btf_id) { enum btf_kfunc_hook hook; + u32 *kfunc_flags; + + kfunc_flags = __btf_kfunc_id_set_contains(btf, BTF_KFUNC_HOOK_GENERIC, kfunc_btf_id); + if (kfunc_flags) + return kfunc_flags; hook = bpf_prog_type_to_kfunc_hook(prog_type); return __btf_kfunc_id_set_contains(btf, hook, kfunc_btf_id); diff --git a/kernel/bpf/helpers.c b/kernel/bpf/helpers.c index 283f55bbeb70..0c8382103625 100644 --- a/kernel/bpf/helpers.c +++ b/kernel/bpf/helpers.c @@ -1717,9 +1717,32 @@ static const struct btf_kfunc_id_set tracing_kfunc_set = { .set = &tracing_btf_ids, }; +void bpf_rcu_read_lock(void) +{ + rcu_read_lock(); +} + +void bpf_rcu_read_unlock(void) +{ + rcu_read_unlock(); +} + +BTF_SET8_START(generic_btf_ids) +BTF_ID_FLAGS(func, bpf_rcu_read_lock) +BTF_ID_FLAGS(func, bpf_rcu_read_unlock) +BTF_SET8_END(generic_btf_ids) + +static const struct btf_kfunc_id_set generic_kfunc_set = { + .owner = THIS_MODULE, + .set = &generic_btf_ids, +}; + static int __init kfunc_init(void) { - return register_btf_kfunc_id_set(BPF_PROG_TYPE_TRACING, &tracing_kfunc_set); + int ret; + + ret = register_btf_kfunc_id_set(BPF_PROG_TYPE_TRACING, &tracing_kfunc_set); + return ret ?: register_btf_kfunc_id_set(BPF_PROG_TYPE_UNSPEC, &generic_kfunc_set); } late_initcall(kfunc_init); From patchwork Thu Nov 10 18:01:45 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yonghong Song X-Patchwork-Id: 13039080 X-Patchwork-Delegate: bpf@iogearbox.net Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 17B5AC433FE for ; Thu, 10 Nov 2022 18:02:32 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231809AbiKJSCa (ORCPT ); Thu, 10 Nov 2022 13:02:30 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:44340 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232016AbiKJSCI (ORCPT ); Thu, 10 Nov 2022 13:02:08 -0500 Received: from mx0a-00082601.pphosted.com (mx0a-00082601.pphosted.com [67.231.145.42]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 8757B4E40D for ; Thu, 10 Nov 2022 10:01:56 -0800 (PST) Received: from pps.filterd (m0109334.ppops.net [127.0.0.1]) by mx0a-00082601.pphosted.com (8.17.1.5/8.17.1.5) with ESMTP id 2AAF8WQ6013987 for ; Thu, 10 Nov 2022 10:01:56 -0800 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=fb.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding : content-type; s=facebook; bh=aA1BFRM4JbMhczpDoBeM8bLUK6hZ5CpSkvQuV7BQA14=; b=dAg+2ToCNcW66ze+Ly5ZVBrEHkPILvIGDKlVSzMPVssqVyKyFvwcxyyL4QHNmmJCNX4Z X2swPYs2M+J7zNmYbE/JoOKx4fAkevTYnuRenKBrl3ZK2bNAvDSRri0OzUsqTUtE+AGs SfznmvMphKu/P4uzdnpwrYqwkaXqBNF97GQ= Received: from mail.thefacebook.com ([163.114.132.120]) by mx0a-00082601.pphosted.com (PPS) with ESMTPS id 3krnrhescw-5 (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128 verify=NOT) for ; Thu, 10 Nov 2022 10:01:56 -0800 Received: from twshared5476.02.ash7.facebook.com (2620:10d:c085:208::f) by mail.thefacebook.com (2620:10d:c085:11d::4) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2375.31; Thu, 10 Nov 2022 10:01:55 -0800 Received: by devbig309.ftw3.facebook.com (Postfix, from userid 128203) id D3B5B11F22DED; Thu, 10 Nov 2022 10:01:45 -0800 (PST) From: Yonghong Song To: CC: Alexei Starovoitov , Andrii Nakryiko , Daniel Borkmann , , Martin KaFai Lau Subject: [PATCH bpf-next v4 4/7] bpf: Add bpf_rcu_read_lock() verifier support Date: Thu, 10 Nov 2022 10:01:45 -0800 Message-ID: <20221110180145.915385-1-yhs@fb.com> X-Mailer: git-send-email 2.30.2 In-Reply-To: <20221110180124.913882-1-yhs@fb.com> References: <20221110180124.913882-1-yhs@fb.com> MIME-Version: 1.0 X-FB-Internal: Safe X-Proofpoint-GUID: ZZqnNUpwafpdt52nO43CFKke80zzBNqv X-Proofpoint-ORIG-GUID: ZZqnNUpwafpdt52nO43CFKke80zzBNqv X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.219,Aquarius:18.0.895,Hydra:6.0.545,FMLib:17.11.122.1 definitions=2022-11-10_12,2022-11-09_01,2022-06-22_01 Precedence: bulk List-ID: X-Mailing-List: bpf@vger.kernel.org X-Patchwork-Delegate: bpf@iogearbox.net To simplify the design and support the common practice, no nested bpf_rcu_read_lock() is allowed. A new bpf_type_flag MEM_RCU is added to indicate a PTR_TO_BTF_ID object access needing rcu_read_lock protection. Note that rcu protection is not needed for non-sleepable program, but it is supported to make cross-sleepable/nonsleepable development easier. For sleepable program, the following insns can be inside the rcu lock region: - any non call insns except BPF_ABS/BPF_IND - non sleepable helpers or kfuncs The rcu pointer will be invalidated at bpf_rcu_read_unlock() so it cannot be used outside the current rcu read lock region. Also, bpf_*_storage_get() helper's 5th hidden argument (for memory allocation flag) should be GFP_ATOMIC. If a pointer (PTR_TO_BTF_ID) is marked as rcu, then any use of this pointer and the load which gets this pointer needs to be protected by bpf_rcu_read_lock(). The following shows a couple of examples: struct task_struct { ... struct task_struct __rcu *real_parent; struct css_set __rcu *cgroups; ... }; struct css_set { ... struct cgroup *dfl_cgrp; ... } ... task = bpf_get_current_task_btf(); cgroups = task->cgroups; dfl_cgroup = cgroups->dfl_cgrp; ... using dfl_cgroup ... The bpf_rcu_read_lock/unlock() should be added like below to avoid verification failures. task = bpf_get_current_task_btf(); bpf_rcu_read_lock(); cgroups = task->cgroups; dfl_cgroup = cgroups->dfl_cgrp; bpf_rcu_read_unlock(); ... using dfl_cgroup ... The following is another example for task->real_parent. task = bpf_get_current_task_btf(); bpf_rcu_read_lock(); real_parent = task->real_parent; ... bpf_task_storage_get(&map, real_parent, 0, 0); bpf_rcu_read_unlock(); There is another case observed in selftest bpf_iter_ipv6_route.c: struct fib6_info *rt = ctx->rt; ... fib6_nh = &rt->fib6_nh[0]; // Not rcu protected ... if (rt->nh) fib6_nh = &nh->nh_info->fib6_nh; // rcu protected ... ... using fib6_nh ... Note that the use of fib6_nh is tag with rcu in one path but not in the other path. Current verification will fail since the same insn cannot be used with different pointer types. The above use case is a valid one so the verifier is changed to ignore MEM_RCU type tag in such cases. Signed-off-by: Yonghong Song --- include/linux/bpf.h | 3 + include/linux/bpf_verifier.h | 4 ++ kernel/bpf/btf.c | 31 ++++++++- kernel/bpf/verifier.c | 122 ++++++++++++++++++++++++++++++++--- 4 files changed, 149 insertions(+), 11 deletions(-) diff --git a/include/linux/bpf.h b/include/linux/bpf.h index 3ed817cf191d..f35bba8dc81c 100644 --- a/include/linux/bpf.h +++ b/include/linux/bpf.h @@ -510,6 +510,9 @@ enum bpf_type_flag { /* Size is known at compile time. */ MEM_FIXED_SIZE = BIT(10 + BPF_BASE_TYPE_BITS), + /* MEM is tagged with rcu and memory access needs rcu_read_lock protection. */ + MEM_RCU = BIT(11 + BPF_BASE_TYPE_BITS), + __BPF_TYPE_FLAG_MAX, __BPF_TYPE_LAST_FLAG = __BPF_TYPE_FLAG_MAX - 1, }; diff --git a/include/linux/bpf_verifier.h b/include/linux/bpf_verifier.h index 1a32baa78ce2..484baeffbfb0 100644 --- a/include/linux/bpf_verifier.h +++ b/include/linux/bpf_verifier.h @@ -325,6 +325,7 @@ struct bpf_verifier_state { u32 curframe; u32 active_spin_lock; bool speculative; + bool active_rcu_lock; /* first and last insn idx of this verifier state */ u32 first_insn_idx; @@ -424,6 +425,7 @@ struct bpf_insn_aux_data { u32 seen; /* this insn was processed by the verifier at env->pass_cnt */ bool sanitize_stack_spill; /* subject to Spectre v4 sanitation */ bool zext_dst; /* this insn zero extends dst reg */ + bool storage_get_func_atomic; /* bpf_*_storage_get() with atomic memory alloc */ u8 alu_state; /* used in combination with alu_limit */ /* below fields are initialized once */ @@ -627,6 +629,8 @@ void bpf_free_kfunc_btf_tab(struct bpf_kfunc_btf_tab *tab); int mark_chain_precision(struct bpf_verifier_env *env, int regno); +void clear_all_rcu_pointers(struct bpf_verifier_env *env); + #define BPF_BASE_TYPE_MASK GENMASK(BPF_BASE_TYPE_BITS - 1, 0) /* extract base type from bpf_{arg, return, reg}_type. */ diff --git a/kernel/bpf/btf.c b/kernel/bpf/btf.c index 84f09235857c..17d22abf66e4 100644 --- a/kernel/bpf/btf.c +++ b/kernel/bpf/btf.c @@ -5850,6 +5850,9 @@ static int btf_struct_walk(struct bpf_verifier_log *log, const struct btf *btf, /* check __percpu tag */ if (strcmp(tag_value, "percpu") == 0) tmp_flag = MEM_PERCPU; + /* check __rcu tag */ + if (strcmp(tag_value, "rcu") == 0) + tmp_flag = MEM_RCU; } stype = btf_type_skip_modifiers(btf, mtype->type, &id); @@ -6309,6 +6312,9 @@ static bool btf_is_kfunc_arg_mem_size(const struct btf *btf, return true; } +BTF_ID_LIST_SINGLE(bpf_rcu_read_lock_id, func, bpf_rcu_read_lock) +BTF_ID_LIST_SINGLE(bpf_rcu_read_unlock_id, func, bpf_rcu_read_unlock) + static int btf_check_func_arg_match(struct bpf_verifier_env *env, const struct btf *btf, u32 func_id, struct bpf_reg_state *regs, @@ -6318,7 +6324,7 @@ static int btf_check_func_arg_match(struct bpf_verifier_env *env, { enum bpf_prog_type prog_type = resolve_prog_type(env->prog); bool rel = false, kptr_get = false, trusted_args = false; - bool sleepable = false; + bool sleepable = false, rcu_lock = false, rcu_unlock = false; struct bpf_verifier_log *log = &env->log; u32 i, nargs, ref_id, ref_obj_id = 0; bool is_kfunc = btf_is_kernel(btf); @@ -6327,6 +6333,7 @@ static int btf_check_func_arg_match(struct bpf_verifier_env *env, const struct btf_param *args; int ref_regno = 0, ret; + t = btf_type_by_id(btf, func_id); if (!t || !btf_type_is_func(t)) { /* These checks were already done by the verifier while loading @@ -6357,6 +6364,28 @@ static int btf_check_func_arg_match(struct bpf_verifier_env *env, kptr_get = kfunc_meta->flags & KF_KPTR_GET; trusted_args = kfunc_meta->flags & KF_TRUSTED_ARGS; sleepable = kfunc_meta->flags & KF_SLEEPABLE; + rcu_lock = func_id == *bpf_rcu_read_lock_id; + rcu_unlock = func_id == *bpf_rcu_read_unlock_id; + } + + /* checking rcu read lock/unlock */ + if (env->cur_state->active_rcu_lock) { + if (rcu_lock) { + bpf_log(log, "nested rcu read lock (kernel function %s)\n", func_name); + return -EINVAL; + } else if (rcu_unlock) { + clear_all_rcu_pointers(env); + env->cur_state->active_rcu_lock = false; + } else if (sleepable) { + bpf_log(log, "kernel func %s is sleepable within rcu_read_lock region\n", + func_name); + return -EINVAL; + } + } else if (rcu_lock) { + env->cur_state->active_rcu_lock = true; + } else if (rcu_unlock) { + bpf_log(log, "unmatched rcu read unlock (kernel function %s)\n", func_name); + return -EINVAL; } /* check that BTF function arguments match actual types that the diff --git a/kernel/bpf/verifier.c b/kernel/bpf/verifier.c index d3b75aa0c54d..2aa140dceb9a 100644 --- a/kernel/bpf/verifier.c +++ b/kernel/bpf/verifier.c @@ -23,6 +23,7 @@ #include #include #include +#include #include #include "disasm.h" @@ -513,6 +514,14 @@ static bool is_callback_calling_function(enum bpf_func_id func_id) func_id == BPF_FUNC_user_ringbuf_drain; } +static bool is_storage_get_function(enum bpf_func_id func_id) +{ + return func_id == BPF_FUNC_sk_storage_get || + func_id == BPF_FUNC_inode_storage_get || + func_id == BPF_FUNC_task_storage_get || + func_id == BPF_FUNC_cgrp_storage_get; +} + static bool helper_multiple_ref_obj_use(enum bpf_func_id func_id, const struct bpf_map *map) { @@ -583,6 +592,8 @@ static const char *reg_type_str(struct bpf_verifier_env *env, strncpy(prefix, "user_", 32); if (type & MEM_PERCPU) strncpy(prefix, "percpu_", 32); + if (type & MEM_RCU) + strncpy(prefix, "rcu_", 32); if (type & PTR_UNTRUSTED) strncpy(prefix, "untrusted_", 32); @@ -1203,6 +1214,7 @@ static int copy_verifier_state(struct bpf_verifier_state *dst_state, dst_state->speculative = src->speculative; dst_state->curframe = src->curframe; dst_state->active_spin_lock = src->active_spin_lock; + dst_state->active_rcu_lock = src->active_rcu_lock; dst_state->branches = src->branches; dst_state->parent = src->parent; dst_state->first_insn_idx = src->first_insn_idx; @@ -4682,6 +4694,14 @@ static int check_ptr_to_btf_access(struct bpf_verifier_env *env, return -EACCES; } + if ((reg->type & MEM_RCU) && env->prog->aux->sleepable && + !env->cur_state->active_rcu_lock) { + verbose(env, + "R%d is ptr_%s access rcu-protected memory with off=%d, not rcu protected\n", + regno, tname, off); + return -EACCES; + } + if (env->ops->btf_struct_access) { ret = env->ops->btf_struct_access(&env->log, reg->btf, t, off, size, atype, &btf_id, &flag); @@ -4698,6 +4718,16 @@ static int check_ptr_to_btf_access(struct bpf_verifier_env *env, if (ret < 0) return ret; + /* The value is a rcu pointer. For a sleepable program, the load needs to be + * in a rcu lock region, similar to rcu_dereference(). + */ + if ((flag & MEM_RCU) && env->prog->aux->sleepable && !env->cur_state->active_rcu_lock) { + verbose(env, + "R%d is rcu dereference ptr_%s with off=%d, not in rcu_read_lock region\n", + regno, tname, off); + return -EACCES; + } + /* If this is an untrusted pointer, all pointers formed by walking it * also inherit the untrusted flag. */ @@ -5800,7 +5830,12 @@ static const struct bpf_reg_types scalar_types = { .types = { SCALAR_VALUE } }; static const struct bpf_reg_types context_types = { .types = { PTR_TO_CTX } }; static const struct bpf_reg_types alloc_mem_types = { .types = { PTR_TO_MEM | MEM_ALLOC } }; static const struct bpf_reg_types const_map_ptr_types = { .types = { CONST_PTR_TO_MAP } }; -static const struct bpf_reg_types btf_ptr_types = { .types = { PTR_TO_BTF_ID } }; +static const struct bpf_reg_types btf_ptr_types = { + .types = { + PTR_TO_BTF_ID, + PTR_TO_BTF_ID | MEM_RCU, + } +}; static const struct bpf_reg_types spin_lock_types = { .types = { PTR_TO_MAP_VALUE } }; static const struct bpf_reg_types percpu_btf_ptr_types = { .types = { PTR_TO_BTF_ID | MEM_PERCPU } }; static const struct bpf_reg_types func_ptr_types = { .types = { PTR_TO_FUNC } }; @@ -5874,6 +5909,20 @@ static int check_reg_type(struct bpf_verifier_env *env, u32 regno, if (arg_type & PTR_MAYBE_NULL) type &= ~PTR_MAYBE_NULL; + /* If the reg type is marked as MEM_RCU, ensure the usage is in the rcu_read_lock + * region, and remove MEM_RCU from the type since the arg_type won't encode + * MEM_RCU. + */ + if (type & MEM_RCU) { + if (env->prog->aux->sleepable && !env->cur_state->active_rcu_lock) { + verbose(env, + "R%d is arg type %s needs rcu protection\n", + regno, reg_type_str(env, reg->type)); + return -EACCES; + } + type &= ~MEM_RCU; + } + for (i = 0; i < ARRAY_SIZE(compatible->types); i++) { expected = compatible->types[i]; if (expected == NOT_INIT) @@ -5890,7 +5939,8 @@ static int check_reg_type(struct bpf_verifier_env *env, u32 regno, return -EACCES; found: - if (reg->type == PTR_TO_BTF_ID) { + /* reg is already protected by rcu_read_lock(). Peel off MEM_RCU from reg->type. */ + if ((reg->type & ~MEM_RCU) == PTR_TO_BTF_ID) { /* For bpf_sk_release, it needs to match against first member * 'struct sock_common', hence make an exception for it. This * allows bpf_sk_release to work for multiple socket types. @@ -5966,6 +6016,7 @@ int check_func_arg_reg_off(struct bpf_verifier_env *env, * fixed offset. */ case PTR_TO_BTF_ID: + case PTR_TO_BTF_ID | MEM_RCU: /* When referenced PTR_TO_BTF_ID is passed to release function, * it's fixed offset must be 0. In the other cases, fixed offset * can be non-zero. @@ -6693,6 +6744,17 @@ static void clear_all_pkt_pointers(struct bpf_verifier_env *env) })); } +void clear_all_rcu_pointers(struct bpf_verifier_env *env) +{ + struct bpf_func_state *state; + struct bpf_reg_state *reg; + + bpf_for_each_reg_in_vstate(env->cur_state, state, reg, ({ + if (reg->type & MEM_RCU) + __mark_reg_unknown(env, reg); + })); +} + enum { AT_PKT_END = -1, BEYOND_PKT_END = -2, @@ -7418,6 +7480,18 @@ static int check_helper_call(struct bpf_verifier_env *env, struct bpf_insn *insn return err; } + if (env->cur_state->active_rcu_lock) { + if (bpf_lsm_sleepable_func_proto(func_id) || + bpf_tracing_sleepable_func_proto(func_id)) { + verbose(env, "sleepable helper %s#%din rcu_read_lock region\n", + func_id_name(func_id), func_id); + return -EINVAL; + } + + if (env->prog->aux->sleepable && is_storage_get_function(func_id)) + env->insn_aux_data[insn_idx].storage_get_func_atomic = true; + } + meta.func_id = func_id; /* check args */ for (i = 0; i < MAX_BPF_FUNC_REG_ARGS; i++) { @@ -10605,6 +10679,11 @@ static int check_ld_abs(struct bpf_verifier_env *env, struct bpf_insn *insn) return -EINVAL; } + if (env->prog->aux->sleepable && env->cur_state->active_rcu_lock) { + verbose(env, "BPF_LD_[ABS|IND] cannot be used inside bpf_rcu_read_lock-ed region\n"); + return -EINVAL; + } + if (regs[ctx_reg].type != PTR_TO_CTX) { verbose(env, "at the time of BPF_LD_ABS|IND R6 != pointer to skb\n"); @@ -11869,6 +11948,9 @@ static bool states_equal(struct bpf_verifier_env *env, if (old->active_spin_lock != cur->active_spin_lock) return false; + if (old->active_rcu_lock != cur->active_rcu_lock) + return false; + /* for states to be equal callsites have to be the same * and all frame states need to be equivalent */ @@ -12282,6 +12364,11 @@ static bool reg_type_mismatch(enum bpf_reg_type src, enum bpf_reg_type prev) !reg_type_mismatch_ok(prev)); } +static bool reg_type_mismatch_ignore_rcu(enum bpf_reg_type src, enum bpf_reg_type prev) +{ + return reg_type_mismatch(src & ~MEM_RCU, prev & ~MEM_RCU); +} + static int do_check(struct bpf_verifier_env *env) { bool pop_log = !(env->log.level & BPF_LOG_LEVEL2); @@ -12407,6 +12494,17 @@ static int do_check(struct bpf_verifier_env *env) prev_src_type = &env->insn_aux_data[env->insn_idx].ptr_type; + /* For code like below, + * struct foo *f; + * if (...) + * f = ...; // f with MEM_RCU type tag. + * else + * f = ...; // f without MEM_RCU type tag. + * ... f ... // Here f could be with/without MEM_RCU + * + * It is safe to ignore MEM_RCU type tag here since + * base types are the same. + */ if (*prev_src_type == NOT_INIT) { /* saw a valid insn * dst_reg = *(u32 *)(src_reg + off) @@ -12414,7 +12512,7 @@ static int do_check(struct bpf_verifier_env *env) */ *prev_src_type = src_reg_type; - } else if (reg_type_mismatch(src_reg_type, *prev_src_type)) { + } else if (reg_type_mismatch_ignore_rcu(src_reg_type, *prev_src_type)) { /* ABuser program is trying to use the same insn * dst_reg = *(u32*) (src_reg + off) * with different pointer types: @@ -12553,6 +12651,11 @@ static int do_check(struct bpf_verifier_env *env) return -EINVAL; } + if (env->cur_state->active_rcu_lock) { + verbose(env, "bpf_rcu_read_unlock is missing\n"); + return -EINVAL; + } + /* We must do check_reference_leak here before * prepare_func_exit to handle the case when * state->curframe > 0, it may be a callback @@ -13641,6 +13744,7 @@ static int convert_ctx_accesses(struct bpf_verifier_env *env) break; case PTR_TO_BTF_ID: case PTR_TO_BTF_ID | PTR_UNTRUSTED: + case PTR_TO_BTF_ID | MEM_RCU: if (type == BPF_READ) { insn->code = BPF_LDX | BPF_PROBE_MEM | BPF_SIZE((insn)->code); @@ -14289,14 +14393,12 @@ static int do_misc_fixups(struct bpf_verifier_env *env) goto patch_call_imm; } - if (insn->imm == BPF_FUNC_task_storage_get || - insn->imm == BPF_FUNC_sk_storage_get || - insn->imm == BPF_FUNC_inode_storage_get || - insn->imm == BPF_FUNC_cgrp_storage_get) { - if (env->prog->aux->sleepable) - insn_buf[0] = BPF_MOV64_IMM(BPF_REG_5, (__force __s32)GFP_KERNEL); - else + if (is_storage_get_function(insn->imm)) { + if (!env->prog->aux->sleepable || + env->insn_aux_data[i + delta].storage_get_func_atomic) insn_buf[0] = BPF_MOV64_IMM(BPF_REG_5, (__force __s32)GFP_ATOMIC); + else + insn_buf[0] = BPF_MOV64_IMM(BPF_REG_5, (__force __s32)GFP_KERNEL); insn_buf[1] = *insn; cnt = 2; From patchwork Thu Nov 10 18:01:51 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yonghong Song X-Patchwork-Id: 13039081 X-Patchwork-Delegate: bpf@iogearbox.net Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 6922BC4332F for ; Thu, 10 Nov 2022 18:02:52 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231718AbiKJSCr (ORCPT ); Thu, 10 Nov 2022 13:02:47 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:43356 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231761AbiKJSCW (ORCPT ); Thu, 10 Nov 2022 13:02:22 -0500 Received: from mx0a-00082601.pphosted.com (mx0b-00082601.pphosted.com [67.231.153.30]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 534624D5C9 for ; Thu, 10 Nov 2022 10:02:03 -0800 (PST) Received: from pps.filterd (m0001303.ppops.net [127.0.0.1]) by m0001303.ppops.net (8.17.1.5/8.17.1.5) with ESMTP id 2AAELOsN025954 for ; Thu, 10 Nov 2022 10:02:02 -0800 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=fb.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding : content-type; s=facebook; bh=NKQB4Ev1rEtTcyg271Lvtr5zhSjiWNaEOiL1osiMzvk=; b=FTI9n76qnNKG04ssAYJh9uRAaV4uuev9t53RWuHmHlkLcSpm/Tlsdhk0moQb3zmF3gDW YjktD1+uvUQLWOgRFhj/lthw7PTfjxeJdQQ0Pq+m911pxKNF/p3+pzD6iK1iJN0VtVzQ xwtbvpfUfGBDWSxFPRbQ90ztN10vWSPWeGA= Received: from maileast.thefacebook.com ([163.114.130.3]) by m0001303.ppops.net (PPS) with ESMTPS id 3krca3brq6-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128 verify=NOT) for ; Thu, 10 Nov 2022 10:02:02 -0800 Received: from twshared14438.02.ash8.facebook.com (2620:10d:c0a8:1b::d) by mail.thefacebook.com (2620:10d:c0a8:83::6) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2375.31; Thu, 10 Nov 2022 10:02:01 -0800 Received: by devbig309.ftw3.facebook.com (Postfix, from userid 128203) id 1C6B211F22DFB; Thu, 10 Nov 2022 10:01:51 -0800 (PST) From: Yonghong Song To: CC: Alexei Starovoitov , Andrii Nakryiko , Daniel Borkmann , , Martin KaFai Lau Subject: [PATCH bpf-next v4 5/7] bpf: Enable sleeptable support for cgrp local storage Date: Thu, 10 Nov 2022 10:01:51 -0800 Message-ID: <20221110180151.915563-1-yhs@fb.com> X-Mailer: git-send-email 2.30.2 In-Reply-To: <20221110180124.913882-1-yhs@fb.com> References: <20221110180124.913882-1-yhs@fb.com> MIME-Version: 1.0 X-FB-Internal: Safe X-Proofpoint-GUID: IKmqYS9ra0iULwMVa7jFQuO9c-YYaKtC X-Proofpoint-ORIG-GUID: IKmqYS9ra0iULwMVa7jFQuO9c-YYaKtC X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.219,Aquarius:18.0.895,Hydra:6.0.545,FMLib:17.11.122.1 definitions=2022-11-10_12,2022-11-09_01,2022-06-22_01 Precedence: bulk List-ID: X-Mailing-List: bpf@vger.kernel.org X-Patchwork-Delegate: bpf@iogearbox.net With proper bpf_rcu_read_lock() support, sleepable support for cgrp local storage can be enabled as typical use case task->cgroups->dfl_cgrp can be protected with bpf_rcu_read_lock(). Signed-off-by: Yonghong Song --- kernel/bpf/verifier.c | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/kernel/bpf/verifier.c b/kernel/bpf/verifier.c index 2aa140dceb9a..92cd1cf64026 100644 --- a/kernel/bpf/verifier.c +++ b/kernel/bpf/verifier.c @@ -12958,10 +12958,11 @@ static int check_map_prog_compatibility(struct bpf_verifier_env *env, case BPF_MAP_TYPE_INODE_STORAGE: case BPF_MAP_TYPE_SK_STORAGE: case BPF_MAP_TYPE_TASK_STORAGE: + case BPF_MAP_TYPE_CGRP_STORAGE: break; default: verbose(env, - "Sleepable programs can only use array, hash, and ringbuf maps\n"); + "Sleepable programs can only use array, hash, ringbuf and local storage maps\n"); return -EINVAL; } From patchwork Thu Nov 10 18:01:56 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yonghong Song X-Patchwork-Id: 13039083 X-Patchwork-Delegate: bpf@iogearbox.net Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 44DF4C433FE for ; Thu, 10 Nov 2022 18:03:13 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230032AbiKJSDM (ORCPT ); Thu, 10 Nov 2022 13:03:12 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:44634 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230326AbiKJSCf (ORCPT ); Thu, 10 Nov 2022 13:02:35 -0500 Received: from mx0a-00082601.pphosted.com (mx0b-00082601.pphosted.com [67.231.153.30]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id C55B04E432 for ; Thu, 10 Nov 2022 10:02:10 -0800 (PST) Received: from pps.filterd (m0001303.ppops.net [127.0.0.1]) by m0001303.ppops.net (8.17.1.5/8.17.1.5) with ESMTP id 2AAFIGO0026034 for ; Thu, 10 Nov 2022 10:02:10 -0800 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=fb.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding : content-type; s=facebook; bh=ZtbCoS8TKFqadzRl7cvSQk3O3LvwHowGolPnzm5F0R8=; b=iovJaUbcj+LhsiJa3gbtugXI2O9rmMxRzY98afEGFWR5wkVZ9FyUMfc+1eJSCLyi+tvT nqtjTf2i7Jw4336l8DFsaAr4OfdTnlNrRURzrzBLmT0Uc7ERncMNiH1aGmPlMVy9X19V 21x4CX82nyyBY7WQuKGZHZZzJKRroFsOJC8= Received: from maileast.thefacebook.com ([163.114.130.8]) by m0001303.ppops.net (PPS) with ESMTPS id 3krca3brs2-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128 verify=NOT) for ; Thu, 10 Nov 2022 10:02:09 -0800 Received: from twshared2001.03.ash8.facebook.com (2620:10d:c0a8:1b::d) by mail.thefacebook.com (2620:10d:c0a8:83::7) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2375.31; Thu, 10 Nov 2022 10:02:08 -0800 Received: by devbig309.ftw3.facebook.com (Postfix, from userid 128203) id 5931411F22E0E; Thu, 10 Nov 2022 10:01:56 -0800 (PST) From: Yonghong Song To: CC: Alexei Starovoitov , Andrii Nakryiko , Daniel Borkmann , , Martin KaFai Lau Subject: [PATCH bpf-next v4 6/7] selftests/bpf: Add tests for bpf_rcu_read_lock() Date: Thu, 10 Nov 2022 10:01:56 -0800 Message-ID: <20221110180156.916235-1-yhs@fb.com> X-Mailer: git-send-email 2.30.2 In-Reply-To: <20221110180124.913882-1-yhs@fb.com> References: <20221110180124.913882-1-yhs@fb.com> MIME-Version: 1.0 X-FB-Internal: Safe X-Proofpoint-GUID: HnblUGfeITC6fzUl0nzwgk1nlfNok4i4 X-Proofpoint-ORIG-GUID: HnblUGfeITC6fzUl0nzwgk1nlfNok4i4 X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.219,Aquarius:18.0.895,Hydra:6.0.545,FMLib:17.11.122.1 definitions=2022-11-10_12,2022-11-09_01,2022-06-22_01 Precedence: bulk List-ID: X-Mailing-List: bpf@vger.kernel.org X-Patchwork-Delegate: bpf@iogearbox.net Add a few positive/negative tests to test bpf_rcu_read_lock() and its corresponding verifier support. Signed-off-by: Yonghong Song --- .../selftests/bpf/prog_tests/rcu_read_lock.c | 166 ++++++++ .../selftests/bpf/progs/rcu_read_lock.c | 355 ++++++++++++++++++ 2 files changed, 521 insertions(+) create mode 100644 tools/testing/selftests/bpf/prog_tests/rcu_read_lock.c create mode 100644 tools/testing/selftests/bpf/progs/rcu_read_lock.c diff --git a/tools/testing/selftests/bpf/prog_tests/rcu_read_lock.c b/tools/testing/selftests/bpf/prog_tests/rcu_read_lock.c new file mode 100644 index 000000000000..38ce62cde93b --- /dev/null +++ b/tools/testing/selftests/bpf/prog_tests/rcu_read_lock.c @@ -0,0 +1,166 @@ +// SPDX-License-Identifier: GPL-2.0 +/* Copyright (c) 2022 Meta Platforms, Inc. and affiliates.*/ + +#define _GNU_SOURCE +#include +#include +#include +#include +#include +#include "rcu_read_lock.skel.h" + +static void test_local_storage(void) +{ + struct rcu_read_lock *skel; + int err; + + skel = rcu_read_lock__open(); + if (!ASSERT_OK_PTR(skel, "skel_open")) + return; + + skel->bss->target_pid = syscall(SYS_gettid); + + bpf_program__set_autoload(skel->progs.cgrp_succ, true); + bpf_program__set_autoload(skel->progs.task_succ, true); + bpf_program__set_autoload(skel->progs.two_regions, true); + bpf_program__set_autoload(skel->progs.non_sleepable_1, true); + bpf_program__set_autoload(skel->progs.non_sleepable_2, true); + err = rcu_read_lock__load(skel); + if (!ASSERT_OK(err, "skel_load")) + goto done; + + err = rcu_read_lock__attach(skel); + if (!ASSERT_OK(err, "skel_attach")) + goto done; + + syscall(SYS_getpgid); + + ASSERT_EQ(skel->bss->result, 2, "result"); +done: + rcu_read_lock__destroy(skel); +} + +static void test_runtime_diff_rcu_tag(void) +{ + struct rcu_read_lock *skel; + int err; + + skel = rcu_read_lock__open(); + if (!ASSERT_OK_PTR(skel, "skel_open")) + return; + + bpf_program__set_autoload(skel->progs.dump_ipv6_route, true); + err = rcu_read_lock__load(skel); + ASSERT_OK(err, "skel_load"); + rcu_read_lock__destroy(skel); +} + +static void test_negative_region(void) +{ +#define NUM_REGION_FAILED_PROGS 6 + struct rcu_read_lock *skel; + struct bpf_program *prog; + int i, err; + + for (i = 0; i < NUM_REGION_FAILED_PROGS; i++) { + skel = rcu_read_lock__open(); + if (!ASSERT_OK_PTR(skel, "skel_open")) + return; + + switch (i) { + case 0: + prog = skel->progs.miss_lock; + break; + case 1: + prog = skel->progs.miss_unlock; + break; + case 2: + prog = skel->progs.non_sleepable_rcu_mismatch; + break; + case 3: + prog = skel->progs.inproper_sleepable_helper; + break; + case 4: + prog = skel->progs.inproper_sleepable_kfunc; + break; + default: + prog = skel->progs.nested_rcu_region; + break; + } + + bpf_program__set_autoload(prog, true); + err = rcu_read_lock__load(skel); + if (!ASSERT_ERR(err, "skel_load")) { + rcu_read_lock__destroy(skel); + return; + } + } +} + +static void test_negative_rcuptr_misuse(void) +{ +#define NUM_RCUPTR_FAILED_PROGS 4 + struct rcu_read_lock *skel; + struct bpf_program *prog; + struct btf *vmlinux_btf; + int i, err, type_id; + + vmlinux_btf = btf__load_vmlinux_btf(); + if (!ASSERT_OK_PTR(vmlinux_btf, "could not load vmlinux BTF")) + return; + + /* skip the test if btf_type_tag("rcu") is not present in vmlinux */ + type_id = btf__find_by_name_kind(vmlinux_btf, "rcu", BTF_KIND_TYPE_TAG); + if (type_id < 0) { + test__skip(); + return; + } + + for (i = 0; i < NUM_RCUPTR_FAILED_PROGS; i++) { + skel = rcu_read_lock__open(); + if (!ASSERT_OK_PTR(skel, "skel_open")) + return; + + switch (i) { + case 0: + prog = skel->progs.cgrp_incorrect_rcu_region; + break; + case 1: + prog = skel->progs.task_incorrect_rcu_region1; + break; + case 2: + prog = skel->progs.task_incorrect_rcu_region2; + break; + default: + prog = skel->progs.cross_rcu_region; + break; + } + + bpf_program__set_autoload(prog, true); + err = rcu_read_lock__load(skel); + if (!ASSERT_ERR(err, "skel_load")) { + rcu_read_lock__destroy(skel); + return; + } + } +} + +void test_rcu_read_lock(void) +{ + int cgroup_fd; + + cgroup_fd = test__join_cgroup("/rcu_read_lock"); + if (!ASSERT_GE(cgroup_fd, 0, "join_cgroup /rcu_read_lock")) + return; + + if (test__start_subtest("local_storage")) + test_local_storage(); + if (test__start_subtest("runtime_diff_rcu_tag")) + test_runtime_diff_rcu_tag(); + if (test__start_subtest("negative_tests_region")) + test_negative_region(); + if (test__start_subtest("negative_tests_rcuptr_misuse")) + test_negative_rcuptr_misuse(); + + close(cgroup_fd); +} diff --git a/tools/testing/selftests/bpf/progs/rcu_read_lock.c b/tools/testing/selftests/bpf/progs/rcu_read_lock.c new file mode 100644 index 000000000000..c11b4f8f9a9d --- /dev/null +++ b/tools/testing/selftests/bpf/progs/rcu_read_lock.c @@ -0,0 +1,355 @@ +// SPDX-License-Identifier: GPL-2.0 +/* Copyright (c) 2022 Meta Platforms, Inc. and affiliates. */ + +#include "vmlinux.h" +#include +#include +#include "bpf_tracing_net.h" +#include "bpf_misc.h" + +char _license[] SEC("license") = "GPL"; + +struct { + __uint(type, BPF_MAP_TYPE_CGRP_STORAGE); + __uint(map_flags, BPF_F_NO_PREALLOC); + __type(key, int); + __type(value, long); +} map_a SEC(".maps"); + +struct { + __uint(type, BPF_MAP_TYPE_TASK_STORAGE); + __uint(map_flags, BPF_F_NO_PREALLOC); + __type(key, int); + __type(value, long); +} map_b SEC(".maps"); + +__u32 user_data, key_serial, target_pid = 0; +__u64 flags, result = 0; + +struct bpf_key *bpf_lookup_user_key(__u32 serial, __u64 flags) __ksym; +void bpf_key_put(struct bpf_key *key) __ksym; +void bpf_rcu_read_lock(void) __ksym; +void bpf_rcu_read_unlock(void) __ksym; + +SEC("?fentry.s/" SYS_PREFIX "sys_getpgid") +int cgrp_succ(void *ctx) +{ + struct task_struct *task; + struct css_set *cgroups; + struct cgroup *dfl_cgrp; + long init_val = 2; + long *ptr; + + task = bpf_get_current_task_btf(); + if (task->pid != target_pid) + return 0; + + bpf_rcu_read_lock(); + cgroups = task->cgroups; + dfl_cgrp = cgroups->dfl_cgrp; + bpf_rcu_read_unlock(); + ptr = bpf_cgrp_storage_get(&map_a, dfl_cgrp, &init_val, + BPF_LOCAL_STORAGE_GET_F_CREATE); + if (!ptr) + return 0; + ptr = bpf_cgrp_storage_get(&map_a, dfl_cgrp, 0, 0); + if (!ptr) + return 0; + result = *ptr; + return 0; +} + +SEC("?fentry.s/" SYS_PREFIX "sys_nanosleep") +int task_succ(void *ctx) +{ + struct task_struct *task, *real_parent; + + task = bpf_get_current_task_btf(); + if (task->pid != target_pid) + return 0; + + /* region including helper using rcu ptr */ + bpf_rcu_read_lock(); + real_parent = task->real_parent; + (void)bpf_task_storage_get(&map_b, real_parent, 0, + BPF_LOCAL_STORAGE_GET_F_CREATE); + bpf_rcu_read_unlock(); + return 0; +} + +SEC("?fentry.s/" SYS_PREFIX "sys_nanosleep") +int two_regions(void *ctx) +{ + struct task_struct *task, *real_parent; + + /* two regions */ + task = bpf_get_current_task_btf(); + bpf_rcu_read_lock(); + bpf_rcu_read_unlock(); + bpf_rcu_read_lock(); + real_parent = task->real_parent; + (void)bpf_task_storage_get(&map_b, real_parent, 0, + BPF_LOCAL_STORAGE_GET_F_CREATE); + bpf_rcu_read_unlock(); + return 0; +} + +SEC("?fentry/" SYS_PREFIX "sys_getpgid") +int non_sleepable_1(void *ctx) +{ + struct task_struct *task, *real_parent; + + task = bpf_get_current_task_btf(); + + bpf_rcu_read_lock(); + real_parent = task->real_parent; + (void)bpf_task_storage_get(&map_b, real_parent, 0, + BPF_LOCAL_STORAGE_GET_F_CREATE); + bpf_rcu_read_unlock(); + return 0; +} + +SEC("?fentry/" SYS_PREFIX "sys_getpgid") +int non_sleepable_2(void *ctx) +{ + struct task_struct *task, *real_parent; + + bpf_rcu_read_lock(); + task = bpf_get_current_task_btf(); + bpf_rcu_read_unlock(); + + bpf_rcu_read_lock(); + real_parent = task->real_parent; + (void)bpf_task_storage_get(&map_b, real_parent, 0, + BPF_LOCAL_STORAGE_GET_F_CREATE); + bpf_rcu_read_unlock(); + return 0; +} + +SEC("?iter.s/ipv6_route") +int dump_ipv6_route(struct bpf_iter__ipv6_route *ctx) +{ + struct seq_file *seq = ctx->meta->seq; + struct fib6_info *rt = ctx->rt; + const struct net_device *dev; + struct fib6_nh *fib6_nh; + unsigned int flags; + struct nexthop *nh; + + if (rt == (void *)0) + return 0; + + /* fib6_nh is not a rcu ptr */ + fib6_nh = &rt->fib6_nh[0]; + flags = rt->fib6_flags; + + nh = rt->nh; + bpf_rcu_read_lock(); + if (rt->nh) + /* fib6_nh is a rcu ptr */ + fib6_nh = &nh->nh_info->fib6_nh; + + /* fib6_nh could be a rcu or non-rcu ptr */ + if (fib6_nh->fib_nh_gw_family) { + flags |= RTF_GATEWAY; + BPF_SEQ_PRINTF(seq, "%pi6 ", &fib6_nh->fib_nh_gw6); + } else { + BPF_SEQ_PRINTF(seq, "00000000000000000000000000000000 "); + } + + dev = fib6_nh->fib_nh_dev; + bpf_rcu_read_unlock(); + if (dev) + BPF_SEQ_PRINTF(seq, "%08x %08x %08x %08x %8s\n", rt->fib6_metric, + rt->fib6_ref.refs.counter, 0, flags, dev->name); + else + BPF_SEQ_PRINTF(seq, "%08x %08x %08x %08x\n", rt->fib6_metric, + rt->fib6_ref.refs.counter, 0, flags); + + return 0; +} + +SEC("?fentry.s/" SYS_PREFIX "sys_getpgid") +int miss_lock(void *ctx) +{ + struct task_struct *task; + struct css_set *cgroups; + struct cgroup *dfl_cgrp; + + /* missing bpf_rcu_read_lock() */ + task = bpf_get_current_task_btf(); + bpf_rcu_read_lock(); + cgroups = task->cgroups; + bpf_rcu_read_unlock(); + dfl_cgrp = cgroups->dfl_cgrp; + bpf_rcu_read_unlock(); + (void)bpf_cgrp_storage_get(&map_a, dfl_cgrp, 0, + BPF_LOCAL_STORAGE_GET_F_CREATE); + return 0; +} + +SEC("?fentry.s/" SYS_PREFIX "sys_getpgid") +int miss_unlock(void *ctx) +{ + struct task_struct *task; + struct css_set *cgroups; + struct cgroup *dfl_cgrp; + + /* missing bpf_rcu_read_unlock() */ + bpf_rcu_read_lock(); + task = bpf_get_current_task_btf(); + bpf_rcu_read_lock(); + cgroups = task->cgroups; + bpf_rcu_read_unlock(); + dfl_cgrp = cgroups->dfl_cgrp; + (void)bpf_cgrp_storage_get(&map_a, dfl_cgrp, 0, + BPF_LOCAL_STORAGE_GET_F_CREATE); + return 0; +} + +SEC("?fentry/" SYS_PREFIX "sys_getpgid") +int non_sleepable_rcu_mismatch(void *ctx) +{ + struct task_struct *task, *real_parent; + + task = bpf_get_current_task_btf(); + + /* non-sleepable: missing bpf_rcu_read_unlock() in one path */ + bpf_rcu_read_lock(); + real_parent = task->real_parent; + (void)bpf_task_storage_get(&map_b, real_parent, 0, + BPF_LOCAL_STORAGE_GET_F_CREATE); + if (real_parent) + bpf_rcu_read_unlock(); + return 0; +} + +SEC("?fentry.s/" SYS_PREFIX "sys_getpgid") +int inproper_sleepable_helper(void *ctx) +{ + struct task_struct *task, *real_parent; + struct pt_regs *regs; + __u32 value = 0; + void *ptr; + + task = bpf_get_current_task_btf(); + + /* sleepable helper in rcu read lock region */ + bpf_rcu_read_lock(); + real_parent = task->real_parent; + regs = (struct pt_regs *)bpf_task_pt_regs(real_parent); + if (!regs) { + bpf_rcu_read_unlock(); + return 0; + } + + ptr = (void *)PT_REGS_IP(regs); + (void)bpf_copy_from_user_task(&value, sizeof(uint32_t), ptr, task, 0); + user_data = value; + (void)bpf_task_storage_get(&map_b, real_parent, 0, + BPF_LOCAL_STORAGE_GET_F_CREATE); + bpf_rcu_read_unlock(); + return 0; +} + +SEC("?lsm.s/bpf") +int BPF_PROG(inproper_sleepable_kfunc, int cmd, union bpf_attr *attr, unsigned int size) +{ + struct bpf_key *bkey; + + /* sleepable kfunc in rcu read lock region */ + bpf_rcu_read_lock(); + bkey = bpf_lookup_user_key(key_serial, flags); + bpf_rcu_read_unlock(); + if (!bkey) + return -1; + bpf_key_put(bkey); + + return 0; +} + +SEC("?fentry.s/" SYS_PREFIX "sys_nanosleep") +int nested_rcu_region(void *ctx) +{ + struct task_struct *task, *real_parent; + + /* nested rcu read lock regions */ + task = bpf_get_current_task_btf(); + bpf_rcu_read_lock(); + bpf_rcu_read_lock(); + real_parent = task->real_parent; + (void)bpf_task_storage_get(&map_b, real_parent, 0, + BPF_LOCAL_STORAGE_GET_F_CREATE); + bpf_rcu_read_unlock(); + bpf_rcu_read_unlock(); + return 0; +} + +SEC("?fentry.s/" SYS_PREFIX "sys_getpgid") +int cgrp_incorrect_rcu_region(void *ctx) +{ + struct task_struct *task; + struct css_set *cgroups; + struct cgroup *dfl_cgrp; + + /* load with rcu_ptr outside the rcu read lock region */ + bpf_rcu_read_lock(); + task = bpf_get_current_task_btf(); + cgroups = task->cgroups; + bpf_rcu_read_unlock(); + dfl_cgrp = cgroups->dfl_cgrp; + (void)bpf_cgrp_storage_get(&map_a, dfl_cgrp, 0, + BPF_LOCAL_STORAGE_GET_F_CREATE); + return 0; +} + +SEC("?fentry.s/" SYS_PREFIX "sys_getpgid") +int task_incorrect_rcu_region1(void *ctx) +{ + struct task_struct *task, *real_parent; + + task = bpf_get_current_task_btf(); + + /* helper use of rcu ptr outside the rcu read lock region */ + bpf_rcu_read_lock(); + real_parent = task->real_parent; + bpf_rcu_read_unlock(); + (void)bpf_task_storage_get(&map_b, real_parent, 0, + BPF_LOCAL_STORAGE_GET_F_CREATE); + return 0; +} + +SEC("?fentry.s/" SYS_PREFIX "sys_getpgid") +int task_incorrect_rcu_region2(void *ctx) +{ + struct task_struct *task, *real_parent; + + task = bpf_get_current_task_btf(); + + /* missing bpf_rcu_read_unlock() in one path */ + bpf_rcu_read_lock(); + real_parent = task->real_parent; + (void)bpf_task_storage_get(&map_b, real_parent, 0, + BPF_LOCAL_STORAGE_GET_F_CREATE); + if (real_parent) + bpf_rcu_read_unlock(); + return 0; +} + +SEC("?fentry.s/" SYS_PREFIX "sys_nanosleep") +int cross_rcu_region(void *ctx) +{ + struct task_struct *task, *real_parent; + + /* rcu ptr define/use in different regions */ + task = bpf_get_current_task_btf(); + bpf_rcu_read_lock(); + real_parent = task->real_parent; + bpf_rcu_read_unlock(); + bpf_rcu_read_lock(); + (void)bpf_task_storage_get(&map_b, real_parent, 0, + BPF_LOCAL_STORAGE_GET_F_CREATE); + bpf_rcu_read_unlock(); + return 0; +} + From patchwork Thu Nov 10 18:02:01 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yonghong Song X-Patchwork-Id: 13039082 X-Patchwork-Delegate: bpf@iogearbox.net Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 7E1ADC433FE for ; Thu, 10 Nov 2022 18:02:56 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231251AbiKJSCt (ORCPT ); Thu, 10 Nov 2022 13:02:49 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:42254 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231243AbiKJSC0 (ORCPT ); Thu, 10 Nov 2022 13:02:26 -0500 Received: from mx0a-00082601.pphosted.com (mx0a-00082601.pphosted.com [67.231.145.42]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 8DACC47314 for ; Thu, 10 Nov 2022 10:02:06 -0800 (PST) Received: from pps.filterd (m0109334.ppops.net [127.0.0.1]) by mx0a-00082601.pphosted.com (8.17.1.5/8.17.1.5) with ESMTP id 2AAFQv2G014500 for ; Thu, 10 Nov 2022 10:02:06 -0800 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=fb.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding : content-type; s=facebook; bh=F2A6plehrPUA6Hy5MqYis1yg9Ub+V3rmUI5gFYvjk+w=; b=T2q2X5SCivGeZSPkfd7yG0mlcKkdtbdZ8pUbs2EEXes9PayQ8Llpm9KXr/0ycruA1Lps QAacEFVhd2peNHugcF9miK1CELpC4TQIjGlcdMBMLWKnwEjahrYmYCiOAEGzyeG4KT6S wCLTv5m89wAVPeHE4byTFnWmeonDeMnopQA= Received: from maileast.thefacebook.com ([163.114.130.3]) by mx0a-00082601.pphosted.com (PPS) with ESMTPS id 3krnrhesfd-4 (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128 verify=NOT) for ; Thu, 10 Nov 2022 10:02:06 -0800 Received: from twshared6758.06.ash9.facebook.com (2620:10d:c0a8:1b::d) by mail.thefacebook.com (2620:10d:c0a8:83::4) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2375.31; Thu, 10 Nov 2022 10:02:03 -0800 Received: by devbig309.ftw3.facebook.com (Postfix, from userid 128203) id 94BD411F22E1C; Thu, 10 Nov 2022 10:02:01 -0800 (PST) From: Yonghong Song To: CC: Alexei Starovoitov , Andrii Nakryiko , Daniel Borkmann , , Martin KaFai Lau Subject: [PATCH bpf-next v4 7/7] selftests/bpf: Add rcu_read_lock test to s390x deny list Date: Thu, 10 Nov 2022 10:02:01 -0800 Message-ID: <20221110180201.917531-1-yhs@fb.com> X-Mailer: git-send-email 2.30.2 In-Reply-To: <20221110180124.913882-1-yhs@fb.com> References: <20221110180124.913882-1-yhs@fb.com> MIME-Version: 1.0 X-FB-Internal: Safe X-Proofpoint-GUID: eqXi3Sue_bapVRBkD_F1sAdiE7S1elAN X-Proofpoint-ORIG-GUID: eqXi3Sue_bapVRBkD_F1sAdiE7S1elAN X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.219,Aquarius:18.0.895,Hydra:6.0.545,FMLib:17.11.122.1 definitions=2022-11-10_12,2022-11-09_01,2022-06-22_01 Precedence: bulk List-ID: X-Mailing-List: bpf@vger.kernel.org X-Patchwork-Delegate: bpf@iogearbox.net The new rcu_read_lock test will fail on s390x with the following error message: ... test_rcu_read_lock:PASS:join_cgroup /rcu_read_lock 0 nsec test_local_storage:PASS:skel_open 0 nsec libbpf: prog 'cgrp_succ': failed to find kernel BTF type ID of '__s390x_sys_getpgid': -3 libbpf: prog 'cgrp_succ': failed to prepare load attributes: -3 libbpf: prog 'cgrp_succ': failed to load: -3 libbpf: failed to load object 'rcu_read_lock' libbpf: failed to load BPF skeleton 'rcu_read_lock': -3 test_local_storage:FAIL:skel_load unexpected error: -3 (errno 3) ... So add it to the s390x deny list. Signed-off-by: Yonghong Song --- tools/testing/selftests/bpf/DENYLIST.s390x | 1 + 1 file changed, 1 insertion(+) diff --git a/tools/testing/selftests/bpf/DENYLIST.s390x b/tools/testing/selftests/bpf/DENYLIST.s390x index be4e3d47ea3e..dd5db40b5a09 100644 --- a/tools/testing/selftests/bpf/DENYLIST.s390x +++ b/tools/testing/selftests/bpf/DENYLIST.s390x @@ -41,6 +41,7 @@ module_attach # skel_attach skeleton attach failed: - mptcp netcnt # failed to load BPF skeleton 'netcnt_prog': -7 (?) probe_user # check_kprobe_res wrong kprobe res from probe read (?) +rcu_read_lock # failed to find kernel BTF type ID of '__x64_sys_getpgid': -3 (?) recursion # skel_attach unexpected error: -524 (trampoline) ringbuf # skel_load skeleton load failed (?) select_reuseport # intermittently fails on new s390x setup