From patchwork Fri Feb 14 16:45:16 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Amery Hung X-Patchwork-Id: 13975256 X-Patchwork-Delegate: bpf@iogearbox.net Received: from mail-pl1-f169.google.com (mail-pl1-f169.google.com [209.85.214.169]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 7BBDB264FA9 for ; Fri, 14 Feb 2025 16:45:31 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.214.169 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1739551533; cv=none; b=kdgSLBadNSIWPyzglAImQXz0vgdT/5d8X60/Q+2NCIwepvEAsQpgWtbDAnNokEKAXohQLo6L9MnZbS4IUAFE6/iqyZrT8mDZIDlxpJS8+8k9i6YQThezfvOVh0bxwF+eLPpxlqZHyApSDAE+bDFcM2L4g4nSpq8/hvzsJyf/xKI= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1739551533; c=relaxed/simple; bh=F1LqcDnRgUym3jCFdQfnlgIgsLietRoBiVkYd8ua0Wk=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=oA5hJ5zLCTiTk7808Qbxp5zGssvxXFlgbcQIZC//4yWY3jWwn4dWqrHkGhLST9Jjpv8By2hZnARZ08ufRfi3Ifb6W35+zfofip22fptQawhrTUccdAu0Gnu0oDZdL2Vc/IhNGB82zN3zGVyNs3v2RnjTgctctdwH/cPy/KL/1uk= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=XJ1/1Ekx; arc=none smtp.client-ip=209.85.214.169 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="XJ1/1Ekx" Received: by mail-pl1-f169.google.com with SMTP id d9443c01a7336-21f6d2642faso61428365ad.1 for ; Fri, 14 Feb 2025 08:45:31 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1739551530; x=1740156330; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=2VNl9RoDF4qcMk8lDzfqgVHsgyUHTU4tSw9Z2ZI42WM=; b=XJ1/1EkxLix55Ss6HA0qOu/01dQYuR8wY8ooQuTVtf4+aRvri+WqkmxeoMs2NgULkg /ur7hAQguAkXERc4vOmzpApSeQvrhquTW/WF+YDcad+JZBDU2IMLn+1GKwAYk+LfAeMS PyE/s5W20UxPtN7fjhBQXCxm4uTHt2vsHVDMwiKUHd7EpXWS8sYZf6et11+Af915U+M2 SiMquxkQpzxIM73/skaLQx1RbkFhlggnuoFWBQtQkxvDxZItO0YZOrAP9ySLQrOXxdAK 67cIqriwv3KBHNhqhtm8YWlX6TLFOXNFWV7o+ryrKjk7kMj5j0LZSJB2xGvI5C5V1HlC kO/Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1739551530; x=1740156330; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=2VNl9RoDF4qcMk8lDzfqgVHsgyUHTU4tSw9Z2ZI42WM=; b=VlliSAh/FYSInn10zKaYGGXyXXQLOrgJGJdKb4thuDq7f5JG4nlu6USEO7J2nmear6 eB5WCYWJ0AXPXyD4zTfVmPMBSmLNB6ozMnsPxgYceqdWFbOoVpl/Dpt/6MKwTy/PyTIX P1g6aycB3IvAFgKHg7+Up3T78H1sOrMWszl7ObwzT+IDEHPPiVAasAEEgaQWANwkCO0E djPM5vNEEgXKVl6DJuWZ5QlZGs6whmcOQWS8wAdC2M7WeJPQBtWY5f0u/NPwFeBIZHbN CsuRV3VTiMWn07rcqr45wNlSNbhYXNGvAgvFUQbieThgmTnq7fFuw2Av4kCncoP4McHo Ci4w== X-Gm-Message-State: AOJu0YxuEgW2PkGH1LmDs77BD7bzt8Mngp8SQaiVrSOxBdfNpDR1Yfi0 4wX1IA2WyRPVOyLH1a6tOoBRG/DAIiFYc7e3HEfSx/qoQH1OEM4w4d32sg== X-Gm-Gg: ASbGnct2mxyxX+fqUSg0/eXLm0kIBlWu/43EtBS8nNCij458Hoew9KKpD3syisARG1g g3MsWxoTwg52nnYAilfijA0Fy5lTUcP2K2Mz92CeN1lSjFFJuZPmQhvudu7vWpwBYjdQOfhbWlx CVptpIAv3qSa/WrZdQz1YbetDQLVF1lIk380X1Hu21ECkuiUsmG5tWPPLU+GiJ4/RInWKk2d/hi 1kYvtfa1EeLROvmfWZUEL2y5XCtY4dYx4lqNeEOJ6rgcYkw04eefxJWOVDHNNkUHKhFQnd3o96y p42GiF0Lsgmx8NV4ptTL8POpkKlELf/OLp6wcaVCXRefEQSnLIXcfr4zQl9WVETeJg== X-Google-Smtp-Source: AGHT+IHwEGCTLmHOB8cPiU3P95bHyn26BDUXZU68QjyVWNQAUIuZjvgBv6fREq2rhDG/MIj/HvMKQw== X-Received: by 2002:a05:6a21:6e47:b0:1ee:6fec:3e5c with SMTP id adf61e73a8af0-1ee8caab660mr304502637.7.1739551530378; Fri, 14 Feb 2025 08:45:30 -0800 (PST) Received: from localhost.localdomain (c-76-146-13-146.hsd1.wa.comcast.net. [76.146.13.146]) by smtp.gmail.com with ESMTPSA id 41be03b00d2f7-adbf21517eesm2223346a12.13.2025.02.14.08.45.29 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 14 Feb 2025 08:45:30 -0800 (PST) From: Amery Hung To: bpf@vger.kernel.org Cc: daniel@iogearbox.net, andrii@kernel.org, alexei.starovoitov@gmail.com, martin.lau@kernel.org, eddyz87@gmail.com, ameryhung@gmail.com, kernel-team@meta.com Subject: [PATCH bpf-next v1 1/5] bpf: Make every prog keep a copy of ctx_arg_info Date: Fri, 14 Feb 2025 08:45:16 -0800 Message-ID: <20250214164520.1001211-2-ameryhung@gmail.com> X-Mailer: git-send-email 2.47.1 In-Reply-To: <20250214164520.1001211-1-ameryhung@gmail.com> References: <20250214164520.1001211-1-ameryhung@gmail.com> Precedence: bulk X-Mailing-List: bpf@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Patchwork-Delegate: bpf@iogearbox.net Currently, ctx_arg_info is read-only in the view of the verifier since it is shared among programs of the same attach type. Make each program have their own copy of ctx_arg_info so that we can use it to store program specific information. In the next patch where we support acquiring a referenced kptr through a struct_ops argument tagged with "__ref", ctx_arg_info->ref_obj_id will be used to store the unique reference object id of the argument. This avoids creating a requirement in the verifier that "__ref" tagged arguments must be the first set of references acquired [0]. [0] https://lore.kernel.org/bpf/20241220195619.2022866-2-amery.hung@gmail.com/ Signed-off-by: Amery Hung Acked-by: Eduard Zingerman Acked-by: Martin KaFai Lau --- include/linux/bpf.h | 7 +++++-- kernel/bpf/bpf_iter.c | 13 ++++++------- kernel/bpf/syscall.c | 2 ++ kernel/bpf/verifier.c | 25 +++++++++++++++---------- 4 files changed, 28 insertions(+), 19 deletions(-) diff --git a/include/linux/bpf.h b/include/linux/bpf.h index f3f50e29d639..f4df39e8c735 100644 --- a/include/linux/bpf.h +++ b/include/linux/bpf.h @@ -1507,7 +1507,7 @@ struct bpf_prog_aux { u32 max_rdonly_access; u32 max_rdwr_access; struct btf *attach_btf; - const struct bpf_ctx_arg_aux *ctx_arg_info; + struct bpf_ctx_arg_aux *ctx_arg_info; void __percpu *priv_stack_ptr; struct mutex dst_mutex; /* protects dst_* pointers below, *after* prog becomes visible */ struct bpf_prog *dst_prog; @@ -1945,6 +1945,9 @@ static inline void bpf_struct_ops_desc_release(struct bpf_struct_ops_desc *st_op #endif +int bpf_prog_ctx_arg_info_init(struct bpf_prog *prog, + const struct bpf_ctx_arg_aux *info, u32 cnt); + #if defined(CONFIG_CGROUP_BPF) && defined(CONFIG_BPF_LSM) int bpf_trampoline_link_cgroup_shim(struct bpf_prog *prog, int cgroup_atype); @@ -2546,7 +2549,7 @@ struct bpf_iter__bpf_map_elem { int bpf_iter_reg_target(const struct bpf_iter_reg *reg_info); void bpf_iter_unreg_target(const struct bpf_iter_reg *reg_info); -bool bpf_iter_prog_supported(struct bpf_prog *prog); +int bpf_iter_prog_supported(struct bpf_prog *prog); const struct bpf_func_proto * bpf_iter_get_func_proto(enum bpf_func_id func_id, const struct bpf_prog *prog); int bpf_iter_link_attach(const union bpf_attr *attr, bpfptr_t uattr, struct bpf_prog *prog); diff --git a/kernel/bpf/bpf_iter.c b/kernel/bpf/bpf_iter.c index 106735145948..380e9a7cac75 100644 --- a/kernel/bpf/bpf_iter.c +++ b/kernel/bpf/bpf_iter.c @@ -335,7 +335,7 @@ static void cache_btf_id(struct bpf_iter_target_info *tinfo, tinfo->btf_id = prog->aux->attach_btf_id; } -bool bpf_iter_prog_supported(struct bpf_prog *prog) +int bpf_iter_prog_supported(struct bpf_prog *prog) { const char *attach_fname = prog->aux->attach_func_name; struct bpf_iter_target_info *tinfo = NULL, *iter; @@ -344,7 +344,7 @@ bool bpf_iter_prog_supported(struct bpf_prog *prog) int prefix_len = strlen(prefix); if (strncmp(attach_fname, prefix, prefix_len)) - return false; + return -EINVAL; mutex_lock(&targets_mutex); list_for_each_entry(iter, &targets, list) { @@ -360,12 +360,11 @@ bool bpf_iter_prog_supported(struct bpf_prog *prog) } mutex_unlock(&targets_mutex); - if (tinfo) { - prog->aux->ctx_arg_info_size = tinfo->reg_info->ctx_arg_info_size; - prog->aux->ctx_arg_info = tinfo->reg_info->ctx_arg_info; - } + if (!tinfo) + return -EINVAL; - return tinfo != NULL; + return bpf_prog_ctx_arg_info_init(prog, tinfo->reg_info->ctx_arg_info, + tinfo->reg_info->ctx_arg_info_size); } const struct bpf_func_proto * diff --git a/kernel/bpf/syscall.c b/kernel/bpf/syscall.c index c420edbfb7c8..598f19e6ebd2 100644 --- a/kernel/bpf/syscall.c +++ b/kernel/bpf/syscall.c @@ -2315,6 +2315,8 @@ static void __bpf_prog_put_noref(struct bpf_prog *prog, bool deferred) kfree(prog->aux->kfunc_tab); if (prog->aux->attach_btf) btf_put(prog->aux->attach_btf); + if (prog->aux->ctx_arg_info) + kfree(prog->aux->ctx_arg_info); if (deferred) { if (prog->sleepable) diff --git a/kernel/bpf/verifier.c b/kernel/bpf/verifier.c index 9971c03adfd5..a41ba019780f 100644 --- a/kernel/bpf/verifier.c +++ b/kernel/bpf/verifier.c @@ -22377,6 +22377,18 @@ static void print_verification_stats(struct bpf_verifier_env *env) env->peak_states, env->longest_mark_read_walk); } +int bpf_prog_ctx_arg_info_init(struct bpf_prog *prog, + const struct bpf_ctx_arg_aux *info, u32 cnt) +{ + prog->aux->ctx_arg_info = kcalloc(cnt, sizeof(*info), GFP_KERNEL); + if (!prog->aux->ctx_arg_info) + return -ENOMEM; + + memcpy(prog->aux->ctx_arg_info, info, sizeof(*info) * cnt); + prog->aux->ctx_arg_info_size = cnt; + return 0; +} + static int check_struct_ops_btf_id(struct bpf_verifier_env *env) { const struct btf_type *t, *func_proto; @@ -22457,17 +22469,12 @@ static int check_struct_ops_btf_id(struct bpf_verifier_env *env) return -EACCES; } - /* btf_ctx_access() used this to provide argument type info */ - prog->aux->ctx_arg_info = - st_ops_desc->arg_info[member_idx].info; - prog->aux->ctx_arg_info_size = - st_ops_desc->arg_info[member_idx].cnt; - prog->aux->attach_func_proto = func_proto; prog->aux->attach_func_name = mname; env->ops = st_ops->verifier_ops; - return 0; + return bpf_prog_ctx_arg_info_init(prog, st_ops_desc->arg_info[member_idx].info, + st_ops_desc->arg_info[member_idx].cnt); } #define SECURITY_PREFIX "security_" @@ -22917,9 +22924,7 @@ static int check_attach_btf_id(struct bpf_verifier_env *env) prog->aux->attach_btf_trace = true; return 0; } else if (prog->expected_attach_type == BPF_TRACE_ITER) { - if (!bpf_iter_prog_supported(prog)) - return -EINVAL; - return 0; + return bpf_iter_prog_supported(prog); } if (prog->type == BPF_PROG_TYPE_LSM) { From patchwork Fri Feb 14 16:45:17 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Amery Hung X-Patchwork-Id: 13975257 X-Patchwork-Delegate: bpf@iogearbox.net Received: from mail-pl1-f175.google.com (mail-pl1-f175.google.com [209.85.214.175]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 94F1126772F for ; Fri, 14 Feb 2025 16:45:32 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.214.175 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1739551534; cv=none; b=ckKX713OjUkVvRsvdL2SvCuCbJsQgOWpTvRRAMKHGFLqqCs+IJUcky1PGvKk/KbyNWrmd6fC5N80gIGtVpXw5eBeojQ8/S9unFsf4XPM3uG0BjjbCxkGZrEioIQC9B8hFnfgOOPtzj6bUPJdt7fFqOESElcdVwUadxQDxrVY++0= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1739551534; c=relaxed/simple; bh=yeZJv/5oACo15YEXXmcTwVJDy89oXjHZmA6y15XJmEc=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=h742AU7FQWrAT/biwlWK6JNYYdGjZPaQRwlWolzN62gUv5ZLd4YPG9QENKqBG7IgC2ld7yZWHXgJWtx/py+c1QI41a7LKgCnOyZDj78XG4OgDG3O3SSrrhpzJ7DJpN+E8JZEdeRhygSMBCffNTrMVr7lBfcKEAsWUjlRSewCak4= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=iUXjVLL6; arc=none smtp.client-ip=209.85.214.175 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="iUXjVLL6" Received: by mail-pl1-f175.google.com with SMTP id d9443c01a7336-220c8f38febso42926105ad.2 for ; Fri, 14 Feb 2025 08:45:32 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1739551532; x=1740156332; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=eol6oaMq93kxYk9VK5G7LDWWkiciy0WsGg/oN96JGJ0=; b=iUXjVLL6vgDgGHe4IPgVR/2iypEg3nfKMvqB5g7NrB5ti7w4LzjfpdMymxb/RGQdq8 OkKsvkSdOLj2lCi1vnP6g5N8b7IUpeoz2cZucHwDP6ojCHUB5GDFxKq94DgweYLljuAD e8KD/CuyWMiaLXql4DrpIlhXP+zZp+avBMJE3hkr2bshLYbb1z646i6SYH/I/j8YZGKl C5zu7gGWXZj4UpU9+/zUxa3vWdp0B1scWKJUpFye+TBJfAQFxFW5CHn6qYAaU4AxEbmP R2yxeAWDcN5wgT1k4fAzwQJ76iHPFs8Wl66xBpkDFshoaAckB6w58jQNExOts85j55VR Tr3w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1739551532; x=1740156332; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=eol6oaMq93kxYk9VK5G7LDWWkiciy0WsGg/oN96JGJ0=; b=w5ik6Zens3JqfTTBmXlfCmDStOqYMbpl9zBhl0znNfegYYT7J5W7dwYuoPHzfI8XO0 S77YDPZu6sQtDhH+mx8XH5SGSwIaN6/BvEkcWPhs2Yazhltzg5X37yYgkYj/4XjR170v GVaq2MsdL9hkgAvhwPEzkVNOMV1ceukjE7XWGbAoL36gJBZAdyKnO2o6+d7kgzsQm3EX 3VPNBu0aPpZHHf4Tf5OZQAMleQVRgH0o6sr7rV91XbG3gTWrR63atjtJs+aDykECVi57 n3A0A3m8z29Tu+3AfTIu54ifD/KumVU7VqcTF/I5AKENHybhWqBGe2Jlw/CE0m+satWN qSmA== X-Gm-Message-State: AOJu0YyzVhjtdmm2rbXKW9BZE23jQgeY7GQNDiZA2GdPLXxR0YSD/6hk cIGz8S9ELss0GqW1g5slA23WG+Dq+JwJF31SYSQBkP0tGs8NaAMNuNIPBQ== X-Gm-Gg: ASbGncuvum9G/cSe6tP2fo39Rzvv7PwEk9jil4D4DYEW7jDQzeBdEGafYPRElCNaVwW 3liopNndD5l0M91ItKEGHvqUBHasjztDeX73tnCq47xl4pzIpjrv7oh3TskqE6QpBTc+wNrRHAw WER1f0btteoPNjeXBEJYnSA8mXiQvRc1jTwdGeQwCl7QEI4tR8/AWvuf27+4aYBaIW75cFXl1Wp 4tHm4nAgRw9xwJaADHvSnbFNFR4Vjl0iHP1zVKyXiSWeg963SaucNuUJ7gLAiJfp63WFQaTqlna rsp7RT8UfsYx29+QD0l/Gjh1HbDQM2tF/w95P1XPTUFKhgJKFygTk+pPRW+++t/Rgg== X-Google-Smtp-Source: AGHT+IHx70eukTkuSZewCWhp6flSXMfPbZUKfdgvZ+9watd4UMOcsNIIENE6UlglBoW6M2hlmCC68w== X-Received: by 2002:a05:6a20:43a0:b0:1e6:5323:58cb with SMTP id adf61e73a8af0-1ee8cb8141amr244870637.23.1739551531528; Fri, 14 Feb 2025 08:45:31 -0800 (PST) Received: from localhost.localdomain (c-76-146-13-146.hsd1.wa.comcast.net. [76.146.13.146]) by smtp.gmail.com with ESMTPSA id 41be03b00d2f7-adbf21517eesm2223346a12.13.2025.02.14.08.45.30 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 14 Feb 2025 08:45:30 -0800 (PST) From: Amery Hung To: bpf@vger.kernel.org Cc: daniel@iogearbox.net, andrii@kernel.org, alexei.starovoitov@gmail.com, martin.lau@kernel.org, eddyz87@gmail.com, ameryhung@gmail.com, kernel-team@meta.com Subject: [PATCH bpf-next v1 2/5] bpf: Support getting referenced kptr from struct_ops argument Date: Fri, 14 Feb 2025 08:45:17 -0800 Message-ID: <20250214164520.1001211-3-ameryhung@gmail.com> X-Mailer: git-send-email 2.47.1 In-Reply-To: <20250214164520.1001211-1-ameryhung@gmail.com> References: <20250214164520.1001211-1-ameryhung@gmail.com> Precedence: bulk X-Mailing-List: bpf@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Patchwork-Delegate: bpf@iogearbox.net From: Amery Hung Allows struct_ops programs to acqurie referenced kptrs from arguments by directly reading the argument. The verifier will acquire a reference for struct_ops a argument tagged with "__ref" in the stub function in the beginning of the main program. The user will be able to access the referenced kptr directly by reading the context as long as it has not been released by the program. This new mechanism to acquire referenced kptr (compared to the existing "kfunc with KF_ACQUIRE") is introduced for ergonomic and semantic reasons. In the first use case, Qdisc_ops, an skb is passed to .enqueue in the first argument. This mechanism provides a natural way for users to get a referenced kptr in the .enqueue struct_ops programs and makes sure that a qdisc will always enqueue or drop the skb. Signed-off-by: Amery Hung Acked-by: Eduard Zingerman Acked-by: Martin KaFai Lau --- include/linux/bpf.h | 3 +++ kernel/bpf/bpf_struct_ops.c | 26 ++++++++++++++++++++------ kernel/bpf/btf.c | 1 + kernel/bpf/verifier.c | 35 ++++++++++++++++++++++++++++++++--- 4 files changed, 56 insertions(+), 9 deletions(-) diff --git a/include/linux/bpf.h b/include/linux/bpf.h index f4df39e8c735..15164787ce7f 100644 --- a/include/linux/bpf.h +++ b/include/linux/bpf.h @@ -968,6 +968,7 @@ struct bpf_insn_access_aux { struct { struct btf *btf; u32 btf_id; + u32 ref_obj_id; }; }; struct bpf_verifier_log *log; /* for verbose logs */ @@ -1481,6 +1482,8 @@ struct bpf_ctx_arg_aux { enum bpf_reg_type reg_type; struct btf *btf; u32 btf_id; + u32 ref_obj_id; + bool refcounted; }; struct btf_mod_pair { diff --git a/kernel/bpf/bpf_struct_ops.c b/kernel/bpf/bpf_struct_ops.c index 9b7f3b9c5262..68df8d8b6db3 100644 --- a/kernel/bpf/bpf_struct_ops.c +++ b/kernel/bpf/bpf_struct_ops.c @@ -146,6 +146,7 @@ void bpf_struct_ops_image_free(void *image) } #define MAYBE_NULL_SUFFIX "__nullable" +#define REFCOUNTED_SUFFIX "__ref" /* Prepare argument info for every nullable argument of a member of a * struct_ops type. @@ -174,11 +175,13 @@ static int prepare_arg_info(struct btf *btf, struct bpf_struct_ops_arg_info *arg_info) { const struct btf_type *stub_func_proto, *pointed_type; + bool is_nullable = false, is_refcounted = false; const struct btf_param *stub_args, *args; struct bpf_ctx_arg_aux *info, *info_buf; u32 nargs, arg_no, info_cnt = 0; char ksym[KSYM_SYMBOL_LEN]; const char *stub_fname; + const char *suffix; s32 stub_func_id; u32 arg_btf_id; int offset; @@ -223,12 +226,19 @@ static int prepare_arg_info(struct btf *btf, info = info_buf; for (arg_no = 0; arg_no < nargs; arg_no++) { /* Skip arguments that is not suffixed with - * "__nullable". + * "__nullable or __ref". */ - if (!btf_param_match_suffix(btf, &stub_args[arg_no], - MAYBE_NULL_SUFFIX)) + is_nullable = btf_param_match_suffix(btf, &stub_args[arg_no], + MAYBE_NULL_SUFFIX); + is_refcounted = btf_param_match_suffix(btf, &stub_args[arg_no], + REFCOUNTED_SUFFIX); + if (!is_nullable && !is_refcounted) continue; + if (is_nullable) + suffix = MAYBE_NULL_SUFFIX; + else if (is_refcounted) + suffix = REFCOUNTED_SUFFIX; /* Should be a pointer to struct */ pointed_type = btf_type_resolve_ptr(btf, args[arg_no].type, @@ -236,7 +246,7 @@ static int prepare_arg_info(struct btf *btf, if (!pointed_type || !btf_type_is_struct(pointed_type)) { pr_warn("stub function %s has %s tagging to an unsupported type\n", - stub_fname, MAYBE_NULL_SUFFIX); + stub_fname, suffix); goto err_out; } @@ -254,11 +264,15 @@ static int prepare_arg_info(struct btf *btf, } /* Fill the information of the new argument */ - info->reg_type = - PTR_TRUSTED | PTR_TO_BTF_ID | PTR_MAYBE_NULL; info->btf_id = arg_btf_id; info->btf = btf; info->offset = offset; + if (is_nullable) { + info->reg_type = PTR_TRUSTED | PTR_TO_BTF_ID | PTR_MAYBE_NULL; + } else if (is_refcounted) { + info->reg_type = PTR_TRUSTED | PTR_TO_BTF_ID; + info->refcounted = true; + } info++; info_cnt++; diff --git a/kernel/bpf/btf.c b/kernel/bpf/btf.c index 9de6acddd479..fd3470fbd144 100644 --- a/kernel/bpf/btf.c +++ b/kernel/bpf/btf.c @@ -6677,6 +6677,7 @@ bool btf_ctx_access(int off, int size, enum bpf_access_type type, info->reg_type = ctx_arg_info->reg_type; info->btf = ctx_arg_info->btf ? : btf_vmlinux; info->btf_id = ctx_arg_info->btf_id; + info->ref_obj_id = ctx_arg_info->refcounted ? ctx_arg_info->ref_obj_id : 0; return true; } } diff --git a/kernel/bpf/verifier.c b/kernel/bpf/verifier.c index a41ba019780f..a0f51903e977 100644 --- a/kernel/bpf/verifier.c +++ b/kernel/bpf/verifier.c @@ -1543,6 +1543,17 @@ static void release_reference_state(struct bpf_verifier_state *state, int idx) return; } +static bool find_reference_state(struct bpf_verifier_state *state, int ptr_id) +{ + int i; + + for (i = 0; i < state->acquired_refs; i++) + if (state->refs[i].id == ptr_id) + return true; + + return false; +} + static int release_lock_state(struct bpf_verifier_state *state, int type, int id, void *ptr) { int i; @@ -5981,7 +5992,8 @@ static int check_packet_access(struct bpf_verifier_env *env, u32 regno, int off, /* check access to 'struct bpf_context' fields. Supports fixed offsets only */ static int check_ctx_access(struct bpf_verifier_env *env, int insn_idx, int off, int size, enum bpf_access_type t, enum bpf_reg_type *reg_type, - struct btf **btf, u32 *btf_id, bool *is_retval, bool is_ldsx) + struct btf **btf, u32 *btf_id, bool *is_retval, bool is_ldsx, + u32 *ref_obj_id) { struct bpf_insn_access_aux info = { .reg_type = *reg_type, @@ -6003,8 +6015,16 @@ static int check_ctx_access(struct bpf_verifier_env *env, int insn_idx, int off, *is_retval = info.is_retval; if (base_type(*reg_type) == PTR_TO_BTF_ID) { + if (info.ref_obj_id && + !find_reference_state(env->cur_state, info.ref_obj_id)) { + verbose(env, "invalid bpf_context access off=%d. Reference may already be released\n", + off); + return -EACCES; + } + *btf = info.btf; *btf_id = info.btf_id; + *ref_obj_id = info.ref_obj_id; } else { env->insn_aux_data[insn_idx].ctx_field_size = info.ctx_field_size; } @@ -7367,7 +7387,7 @@ static int check_mem_access(struct bpf_verifier_env *env, int insn_idx, u32 regn struct bpf_retval_range range; enum bpf_reg_type reg_type = SCALAR_VALUE; struct btf *btf = NULL; - u32 btf_id = 0; + u32 btf_id = 0, ref_obj_id = 0; if (t == BPF_WRITE && value_regno >= 0 && is_pointer_value(env, value_regno)) { @@ -7380,7 +7400,7 @@ static int check_mem_access(struct bpf_verifier_env *env, int insn_idx, u32 regn return err; err = check_ctx_access(env, insn_idx, off, size, t, ®_type, &btf, - &btf_id, &is_retval, is_ldsx); + &btf_id, &is_retval, is_ldsx, &ref_obj_id); if (err) verbose_linfo(env, insn_idx, "; "); if (!err && t == BPF_READ && value_regno >= 0) { @@ -7411,6 +7431,7 @@ static int check_mem_access(struct bpf_verifier_env *env, int insn_idx, u32 regn if (base_type(reg_type) == PTR_TO_BTF_ID) { regs[value_regno].btf = btf; regs[value_regno].btf_id = btf_id; + regs[value_regno].ref_obj_id = ref_obj_id; } } regs[value_regno].type = reg_type; @@ -22148,6 +22169,7 @@ static int do_check_common(struct bpf_verifier_env *env, int subprog) { bool pop_log = !(env->log.level & BPF_LOG_LEVEL2); struct bpf_subprog_info *sub = subprog_info(env, subprog); + struct bpf_prog_aux *aux = env->prog->aux; struct bpf_verifier_state *state; struct bpf_reg_state *regs; int ret, i; @@ -22255,6 +22277,13 @@ static int do_check_common(struct bpf_verifier_env *env, int subprog) mark_reg_known_zero(env, regs, BPF_REG_1); } + /* Acquire references for struct_ops program arguments tagged with "__ref" */ + if (!subprog && env->prog->type == BPF_PROG_TYPE_STRUCT_OPS) { + for (i = 0; i < aux->ctx_arg_info_size; i++) + aux->ctx_arg_info[i].ref_obj_id = aux->ctx_arg_info[i].refcounted ? + acquire_reference(env, 0) : 0; + } + ret = do_check(env); out: /* check for NULL is necessary, since cur_state can be freed inside From patchwork Fri Feb 14 16:45:18 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Amery Hung X-Patchwork-Id: 13975258 X-Patchwork-Delegate: bpf@iogearbox.net Received: from mail-pl1-f176.google.com (mail-pl1-f176.google.com [209.85.214.176]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 9E18126773C for ; Fri, 14 Feb 2025 16:45:33 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.214.176 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1739551535; cv=none; b=suauWKIE2Cyv4vtK8E45TBjoHkNfQY/MG+lzU2E4LKLk+rzXk682sjhy7AT9+5HufFwTAQNkDnFNuDNq+AB3HvmTV0DmTY+YjHQeKtIrimEWZTlRyyIHAiUhCBkqK5/9tHWDn86QsXsGCKz4lq90o6Px2GdXBfiqH4OIaX74Z6E= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1739551535; c=relaxed/simple; bh=cu4b3Vbm9gFq/EkCNcqo0m8n8rFskmSadxY4u/enMps=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=Li8nazeTZ33JrhjDY8L9VEJRl+G/BfVqwHl2wwpQJ3pKhSP8M9aDG/FBixMj586phBgRphis77dhQ5A6wqxHRyI9HeKgT3Ck2kBE0fT2kqH9+mulHV5N25HNpxkcb8WttQ98E1wVGMTyXLCvIbBDCWDDjE6EpGu8PWxLVHAwliw= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=KUzyFKGD; arc=none smtp.client-ip=209.85.214.176 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="KUzyFKGD" Received: by mail-pl1-f176.google.com with SMTP id d9443c01a7336-220e6028214so32950015ad.0 for ; Fri, 14 Feb 2025 08:45:33 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1739551533; x=1740156333; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=cB3GDDoQAzmyBzIZYs9Df0cqC8v4dqCoGTbbm/k4J8w=; b=KUzyFKGDp3G1MjoUu2MUXEgS1vizU7w6shP2tBG6fJYK7RfyklcgHWSn65CRIFOqjq nv6+hcfhJjtGEz0hdz12ObeJorDEX7zInFhx86DekzD1atpN34p03dgJAzOVKkMgV1/g GvJmg1AnkzK2YcBKOo9656mWYqG2+Mno7AbmLj/KJ18PZRanKOEFBsKMEWY6irJ3C8OM GvrGB9j+B1EsPGS0JBTLrmjTp78OITF6lSx2rVhikEo+w5xXy7/BrxWRWfHL7Z83EvsB XocSQIYx5NcZEMETuYXkL+r5o3kpve48tXk/rhpuXewGv9iLIBxTTI85DZUUE85lCgt/ 3XUw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1739551533; x=1740156333; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=cB3GDDoQAzmyBzIZYs9Df0cqC8v4dqCoGTbbm/k4J8w=; b=qg9dMLhjvk6MQTxhF7Dfqntu6a+CynIAMNbC9mwANLpahkffd7UHEYE5qEJBELqhv+ A509oTYBecVg2Pr+lMrZsbEo/EZezSkSzn/lW4gIm8Rcc/GF9w5YfheVJ74x9hP1wGr6 fkrN+dEht15DIMMXuBHXYoz7wqSdMaKbtTXTmVajoUBukccqeKHACSUi+lphC4xLmA24 siX2/U6ihaHhECNEY84BYAcYANQqZ5IZYn3L3Fy6OtLYql605pUm/glYSkvYXIfdZwMt 84wqQz+jMOFXuXQm8AVkrLidUDTEv7VdptLaFHvZipKOqR9GejkuuwPwqlCbcco5ayHL TCEA== X-Gm-Message-State: AOJu0YzJ/bMd7QaDRFhoWbNmhbvDISCSwkKBXk1UG25jhgfIGe4wmSuW w+PEfrZ8jibEJp11tbU1vSZggMwPbsyYW2fv911/k/8e/NfnwU/4hwsKwQ== X-Gm-Gg: ASbGncvrJ+K/IbnGgSe7r6mPcBT5PK32t5DoE+xGGCcYfPMo4q4GkcNwvQ1oGlNBJGC K8VyyJe3ntzI3gKmqscm3WSY0m/WPTxj0LEZFKIWC4UGM+sDjbPyuLBLAmTU+Qn7hDsxwKkX+cF UQqWDY5P7qP1GmDpe0Sr6+m0A8z0xCspXQEpzRNaTCvTn0gQIgdsLMmK/xCerDhsGpYn6MaENb1 lYOYd/DoEBwVUYG0qh/Q0/j3GeERifTYmn6BOVXm4cB+rff4gdn2hUaIJSRTxS10P3blCGSUOnl HtP7kYrz7Dv3a96OknbMXbeFtMtwLXwsNSKKZz76zBp/c6zu0WzOAA7poOAD/u8vOQ== X-Google-Smtp-Source: AGHT+IFGwcYZvr5LFXDuZuXUHw4nuzpu6le0QkkFljqtCNpszQ75teFvxgeP7dUDJ3+/ksvr21X79g== X-Received: by 2002:a05:6a21:6e47:b0:1e0:d1c3:97d1 with SMTP id adf61e73a8af0-1ee8cc03463mr222307637.29.1739551532584; Fri, 14 Feb 2025 08:45:32 -0800 (PST) Received: from localhost.localdomain (c-76-146-13-146.hsd1.wa.comcast.net. [76.146.13.146]) by smtp.gmail.com with ESMTPSA id 41be03b00d2f7-adbf21517eesm2223346a12.13.2025.02.14.08.45.31 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 14 Feb 2025 08:45:32 -0800 (PST) From: Amery Hung To: bpf@vger.kernel.org Cc: daniel@iogearbox.net, andrii@kernel.org, alexei.starovoitov@gmail.com, martin.lau@kernel.org, eddyz87@gmail.com, ameryhung@gmail.com, kernel-team@meta.com Subject: [PATCH bpf-next v1 3/5] selftests/bpf: Test referenced kptr arguments of struct_ops programs Date: Fri, 14 Feb 2025 08:45:18 -0800 Message-ID: <20250214164520.1001211-4-ameryhung@gmail.com> X-Mailer: git-send-email 2.47.1 In-Reply-To: <20250214164520.1001211-1-ameryhung@gmail.com> References: <20250214164520.1001211-1-ameryhung@gmail.com> Precedence: bulk X-Mailing-List: bpf@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Patchwork-Delegate: bpf@iogearbox.net From: Amery Hung Test referenced kptr acquired through struct_ops argument tagged with "__ref". The success case checks whether 1) a reference to the correct type is acquired, and 2) the referenced kptr argument can be accessed in multiple paths as long as it hasn't been released. In the fail cases, we first confirm that a referenced kptr acquried through a struct_ops argument is not allowed to be leaked. Then, we make sure this new referenced kptr acquiring mechanism does not accidentally allow referenced kptrs to flow into global subprograms through their arguments. Signed-off-by: Amery Hung Acked-by: Eduard Zingerman Acked-by: Martin KaFai Lau --- .../prog_tests/test_struct_ops_refcounted.c | 12 ++++++ .../bpf/progs/struct_ops_refcounted.c | 31 +++++++++++++++ ...ruct_ops_refcounted_fail__global_subprog.c | 39 +++++++++++++++++++ .../struct_ops_refcounted_fail__ref_leak.c | 22 +++++++++++ .../selftests/bpf/test_kmods/bpf_testmod.c | 7 ++++ .../selftests/bpf/test_kmods/bpf_testmod.h | 2 + 6 files changed, 113 insertions(+) create mode 100644 tools/testing/selftests/bpf/prog_tests/test_struct_ops_refcounted.c create mode 100644 tools/testing/selftests/bpf/progs/struct_ops_refcounted.c create mode 100644 tools/testing/selftests/bpf/progs/struct_ops_refcounted_fail__global_subprog.c create mode 100644 tools/testing/selftests/bpf/progs/struct_ops_refcounted_fail__ref_leak.c diff --git a/tools/testing/selftests/bpf/prog_tests/test_struct_ops_refcounted.c b/tools/testing/selftests/bpf/prog_tests/test_struct_ops_refcounted.c new file mode 100644 index 000000000000..e290a2f6db95 --- /dev/null +++ b/tools/testing/selftests/bpf/prog_tests/test_struct_ops_refcounted.c @@ -0,0 +1,12 @@ +#include + +#include "struct_ops_refcounted.skel.h" +#include "struct_ops_refcounted_fail__ref_leak.skel.h" +#include "struct_ops_refcounted_fail__global_subprog.skel.h" + +void test_struct_ops_refcounted(void) +{ + RUN_TESTS(struct_ops_refcounted); + RUN_TESTS(struct_ops_refcounted_fail__ref_leak); + RUN_TESTS(struct_ops_refcounted_fail__global_subprog); +} diff --git a/tools/testing/selftests/bpf/progs/struct_ops_refcounted.c b/tools/testing/selftests/bpf/progs/struct_ops_refcounted.c new file mode 100644 index 000000000000..76dcb6089d7f --- /dev/null +++ b/tools/testing/selftests/bpf/progs/struct_ops_refcounted.c @@ -0,0 +1,31 @@ +#include +#include +#include "../test_kmods/bpf_testmod.h" +#include "bpf_misc.h" + +char _license[] SEC("license") = "GPL"; + +__attribute__((nomerge)) extern void bpf_task_release(struct task_struct *p) __ksym; + +/* This is a test BPF program that uses struct_ops to access a referenced + * kptr argument. This is a test for the verifier to ensure that it + * 1) recongnizes the task as a referenced object (i.e., ref_obj_id > 0), and + * 2) the same reference can be acquired from multiple paths as long as it + * has not been released. + */ +SEC("struct_ops/test_refcounted") +int BPF_PROG(refcounted, int dummy, struct task_struct *task) +{ + if (dummy == 1) + bpf_task_release(task); + else + bpf_task_release(task); + return 0; +} + +SEC(".struct_ops.link") +struct bpf_testmod_ops testmod_refcounted = { + .test_refcounted = (void *)refcounted, +}; + + diff --git a/tools/testing/selftests/bpf/progs/struct_ops_refcounted_fail__global_subprog.c b/tools/testing/selftests/bpf/progs/struct_ops_refcounted_fail__global_subprog.c new file mode 100644 index 000000000000..ae074aa62852 --- /dev/null +++ b/tools/testing/selftests/bpf/progs/struct_ops_refcounted_fail__global_subprog.c @@ -0,0 +1,39 @@ +#include +#include +#include "../test_kmods/bpf_testmod.h" +#include "bpf_misc.h" + +char _license[] SEC("license") = "GPL"; + +extern void bpf_task_release(struct task_struct *p) __ksym; + +__noinline int subprog_release(__u64 *ctx __arg_ctx) +{ + struct task_struct *task = (struct task_struct *)ctx[1]; + int dummy = (int)ctx[0]; + + bpf_task_release(task); + + return dummy + 1; +} + +/* Test that the verifier rejects a program that contains a global + * subprogram with referenced kptr arguments + */ +SEC("struct_ops/test_refcounted") +__failure __log_level(2) +__msg("Validating subprog_release() func#1...") +__msg("invalid bpf_context access off=8. Reference may already be released") +int refcounted_fail__global_subprog(unsigned long long *ctx) +{ + struct task_struct *task = (struct task_struct *)ctx[1]; + + bpf_task_release(task); + + return subprog_release(ctx); +} + +SEC(".struct_ops.link") +struct bpf_testmod_ops testmod_ref_acquire = { + .test_refcounted = (void *)refcounted_fail__global_subprog, +}; diff --git a/tools/testing/selftests/bpf/progs/struct_ops_refcounted_fail__ref_leak.c b/tools/testing/selftests/bpf/progs/struct_ops_refcounted_fail__ref_leak.c new file mode 100644 index 000000000000..e945b1a04294 --- /dev/null +++ b/tools/testing/selftests/bpf/progs/struct_ops_refcounted_fail__ref_leak.c @@ -0,0 +1,22 @@ +#include +#include +#include "../test_kmods/bpf_testmod.h" +#include "bpf_misc.h" + +char _license[] SEC("license") = "GPL"; + +/* Test that the verifier rejects a program that acquires a referenced + * kptr through context without releasing the reference + */ +SEC("struct_ops/test_refcounted") +__failure __msg("Unreleased reference id=1 alloc_insn=0") +int BPF_PROG(refcounted_fail__ref_leak, int dummy, + struct task_struct *task) +{ + return 0; +} + +SEC(".struct_ops.link") +struct bpf_testmod_ops testmod_ref_acquire = { + .test_refcounted = (void *)refcounted_fail__ref_leak, +}; diff --git a/tools/testing/selftests/bpf/test_kmods/bpf_testmod.c b/tools/testing/selftests/bpf/test_kmods/bpf_testmod.c index cc9dde507aba..802cbd871035 100644 --- a/tools/testing/selftests/bpf/test_kmods/bpf_testmod.c +++ b/tools/testing/selftests/bpf/test_kmods/bpf_testmod.c @@ -1176,10 +1176,17 @@ static int bpf_testmod_ops__test_maybe_null(int dummy, return 0; } +static int bpf_testmod_ops__test_refcounted(int dummy, + struct task_struct *task__ref) +{ + return 0; +} + static struct bpf_testmod_ops __bpf_testmod_ops = { .test_1 = bpf_testmod_test_1, .test_2 = bpf_testmod_test_2, .test_maybe_null = bpf_testmod_ops__test_maybe_null, + .test_refcounted = bpf_testmod_ops__test_refcounted, }; struct bpf_struct_ops bpf_bpf_testmod_ops = { diff --git a/tools/testing/selftests/bpf/test_kmods/bpf_testmod.h b/tools/testing/selftests/bpf/test_kmods/bpf_testmod.h index 356803d1c10e..c57b2f9dab10 100644 --- a/tools/testing/selftests/bpf/test_kmods/bpf_testmod.h +++ b/tools/testing/selftests/bpf/test_kmods/bpf_testmod.h @@ -36,6 +36,8 @@ struct bpf_testmod_ops { /* Used to test nullable arguments. */ int (*test_maybe_null)(int dummy, struct task_struct *task); int (*unsupported_ops)(void); + /* Used to test ref_acquired arguments. */ + int (*test_refcounted)(int dummy, struct task_struct *task); /* The following fields are used to test shadow copies. */ char onebyte; From patchwork Fri Feb 14 16:45:19 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Amery Hung X-Patchwork-Id: 13975259 X-Patchwork-Delegate: bpf@iogearbox.net Received: from mail-pl1-f175.google.com (mail-pl1-f175.google.com [209.85.214.175]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 5AAD4264FA9 for ; Fri, 14 Feb 2025 16:45:34 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.214.175 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1739551536; cv=none; b=kAzLH/FcDZX10LvfCgoiBvAPrAAryJ+zojLvXBw981aRJ3WsLIWk2xdGwZe8N2Eux4svnJqRaFiwlBg1l2AFRQ35F7fFqvBwSAlWd9TVSD5AIdb0ZNh7g+EcCi9oMceTADZ7vbEpMa5v0Pq9proWDsTRyL5Ok5ljk3q6rmCleb4= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1739551536; c=relaxed/simple; bh=jwCq0P0Vuds6Ocs62YCkiXqCDb3BXNojP44mv3vZHMM=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=bnoZOIAEPUCyQH2bFA+bVcme4HWsSRy2agfRkWjnuuOmSZ6MocavwXaUkyGB8gAlVGy2OpXw9f4BJYJmrQR2HyiToeg7GmMgBUdiQynDXFxZOXeEDHZO56vL4jjDfhPKzTKvZZ+4AZlmq5kIvyT0ulmg9eGHjx1JJHicRAVqG+8= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=hI+zsfNy; arc=none smtp.client-ip=209.85.214.175 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="hI+zsfNy" Received: by mail-pl1-f175.google.com with SMTP id d9443c01a7336-21f6a47d617so40449905ad.2 for ; Fri, 14 Feb 2025 08:45:34 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1739551533; x=1740156333; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=j4LOyoAFByV1hms7BAn0ouvPEgqOVY36PJCCwBmaLhY=; b=hI+zsfNy+EQXalZklIlCsOM1Ad/jJPUAem9wQ/u/SGfqwEe2Aobx2i1/FZUXHREZQF OA04poxEOOtuwF4puY1SPbzRc7mHXpiiL0X7kWdhBBPbePxNIRhgbge/RsrkuMaUE/f8 RmwZoIilrxklXgOAzGVyCj9YE8yiRuWZdEXyQbzAQ8amHHahKmtpNiqFvhAIWW4YyF8l 4jMTbLr/qsqqzTSeciTi2tAnepxSGfWc86hHy8evR/5TBX5A/qn+tqS7NeYrqgaKOo5R g0nDLCqhundj6j57nwecD++32kCV/2Q6D8QAw+U2hipea/shKT9dJ/V2OV3R7VOuNEDh qITA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1739551533; x=1740156333; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=j4LOyoAFByV1hms7BAn0ouvPEgqOVY36PJCCwBmaLhY=; b=Q+4BV3TgAHIn8X4L5lnAsyZLm9wWyyjSaCqVPTtG8/pYUEgLmYXq2BBTy3YUYyLnmd FvgiKXHbDh9u+0MbTuvxQwrkXYAvvXrDRxANlNBgfAnYwlVkT0YtFTixLV2G19kV3iOO 7aVvbGEw1shX5ChlfH6lJG8pB/ujC9zYQ4wc2H0vljxqPTMQZHTn3/EYVFnFUtIeZwB+ BH3WViUmE0Kcda1+8VgZj4IH0N9Jqfw3oZPwUvnSWh4Ps8WgEOZvNTHvVpYEo+doO4rN ya5VnccfCqAoCQi7YpdE142hM666VaTR7M8OZ3t93OzFBZWS2Ih9QYk6XQRzJXGvqRo6 wuag== X-Gm-Message-State: AOJu0YxZ171IviBy/iuYUKq5hkEvFSQUQfCCFVw9hWVTqnreutO+IgTm hEaQ4cWT80KKL21wYlLJVuRyaP83K7qjpR/9NF+0w/ud11Yerr2muJjSfA== X-Gm-Gg: ASbGncsSYyZrJHMmaWgGyhcHquoLwcDDpGL8FnUzpbIGq2zeZCgrrpjM7SzumLMWov6 /pilJJ6MACDQnOQaumMRnmhW76b0F2twquG7rsAYinJ7CrxIIKN4r2xgTCEfm+6lXpJGHbhM6rP f4cBGy3AhwQ6HkFwIFIfqJ4WabTMMgUDkAO+Fe5oTiiJwqKQnG/U4BnIW2R6foIfiZxN54njuqo +ENj877DkvQGlFq1fKvmLhYtY+KY7pzdD7RFD4PuBH/g2OF58ot71HRGYYO80+QWU/YOxWvlvGX xAwfuL/wfYB76PGjN1hyDt1pvWgweztyfV+MK7Dj6y19li2Jnp3vVNAJUADWXmSs0w== X-Google-Smtp-Source: AGHT+IG1I0MAJ6fdW3Zvj7g9gZGGyQhTKYUsT3/7NDzxCGmaH7Eu9W1xDX8AtGQ/ahBCWj2z4RegWg== X-Received: by 2002:a05:6a21:b97:b0:1ee:7483:8377 with SMTP id adf61e73a8af0-1ee8cad309cmr297586637.14.1739551533385; Fri, 14 Feb 2025 08:45:33 -0800 (PST) Received: from localhost.localdomain (c-76-146-13-146.hsd1.wa.comcast.net. [76.146.13.146]) by smtp.gmail.com with ESMTPSA id 41be03b00d2f7-adbf21517eesm2223346a12.13.2025.02.14.08.45.32 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 14 Feb 2025 08:45:33 -0800 (PST) From: Amery Hung To: bpf@vger.kernel.org Cc: daniel@iogearbox.net, andrii@kernel.org, alexei.starovoitov@gmail.com, martin.lau@kernel.org, eddyz87@gmail.com, ameryhung@gmail.com, kernel-team@meta.com Subject: [PATCH bpf-next v1 4/5] bpf: Allow struct_ops prog to return referenced kptr Date: Fri, 14 Feb 2025 08:45:19 -0800 Message-ID: <20250214164520.1001211-5-ameryhung@gmail.com> X-Mailer: git-send-email 2.47.1 In-Reply-To: <20250214164520.1001211-1-ameryhung@gmail.com> References: <20250214164520.1001211-1-ameryhung@gmail.com> Precedence: bulk X-Mailing-List: bpf@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Patchwork-Delegate: bpf@iogearbox.net From: Amery Hung Allow a struct_ops program to return a referenced kptr if the struct_ops operator's return type is a struct pointer. To make sure the returned pointer continues to be valid in the kernel, several constraints are required: 1) The type of the pointer must matches the return type 2) The pointer originally comes from the kernel (not locally allocated) 3) The pointer is in its unmodified form Implementation wise, a referenced kptr first needs to be allowed to _leak_ in check_reference_leak() if it is in the return register. Then, in check_return_code(), constraints 1-3 are checked. During struct_ops registration, a check is also added to warn about operators with non-struct pointer return. In addition, since the first user, Qdisc_ops::dequeue, allows a NULL pointer to be returned when there is no skb to be dequeued, we will allow a scalar value with value equals to NULL to be returned. In the future when there is a struct_ops user that always expects a valid pointer to be returned from an operator, we may extend tagging to the return value. We can tell the verifier to only allow NULL pointer return if the return value is tagged with MAY_BE_NULL. Signed-off-by: Amery Hung Acked-by: Eduard Zingerman Acked-by: Martin KaFai Lau --- kernel/bpf/bpf_struct_ops.c | 12 +++++++++++- kernel/bpf/verifier.c | 36 ++++++++++++++++++++++++++++++++---- 2 files changed, 43 insertions(+), 5 deletions(-) diff --git a/kernel/bpf/bpf_struct_ops.c b/kernel/bpf/bpf_struct_ops.c index 68df8d8b6db3..8df5e8045d07 100644 --- a/kernel/bpf/bpf_struct_ops.c +++ b/kernel/bpf/bpf_struct_ops.c @@ -389,7 +389,7 @@ int bpf_struct_ops_desc_init(struct bpf_struct_ops_desc *st_ops_desc, st_ops_desc->value_type = btf_type_by_id(btf, value_id); for_each_member(i, t, member) { - const struct btf_type *func_proto; + const struct btf_type *func_proto, *ret_type; void **stub_func_addr; u32 moff; @@ -426,6 +426,16 @@ int bpf_struct_ops_desc_init(struct bpf_struct_ops_desc *st_ops_desc, if (!func_proto || bpf_struct_ops_supported(st_ops, moff)) continue; + if (func_proto->type) { + ret_type = btf_type_resolve_ptr(btf, func_proto->type, NULL); + if (ret_type && !__btf_type_is_struct(ret_type)) { + pr_warn("func ptr %s in struct %s returns non-struct pointer, which is not supported\n", + mname, st_ops->name); + err = -EOPNOTSUPP; + goto errout; + } + } + if (btf_distill_func_proto(log, btf, func_proto, mname, &st_ops->func_models[i])) { diff --git a/kernel/bpf/verifier.c b/kernel/bpf/verifier.c index a0f51903e977..5bcf095e8d0c 100644 --- a/kernel/bpf/verifier.c +++ b/kernel/bpf/verifier.c @@ -10758,6 +10758,8 @@ record_func_key(struct bpf_verifier_env *env, struct bpf_call_arg_meta *meta, static int check_reference_leak(struct bpf_verifier_env *env, bool exception_exit) { struct bpf_verifier_state *state = env->cur_state; + enum bpf_prog_type type = resolve_prog_type(env->prog); + struct bpf_reg_state *reg = reg_state(env, BPF_REG_0); bool refs_lingering = false; int i; @@ -10767,6 +10769,12 @@ static int check_reference_leak(struct bpf_verifier_env *env, bool exception_exi for (i = 0; i < state->acquired_refs; i++) { if (state->refs[i].type != REF_TYPE_PTR) continue; + /* Allow struct_ops programs to return a referenced kptr back to + * kernel. Type checks are performed later in check_return_code. + */ + if (type == BPF_PROG_TYPE_STRUCT_OPS && !exception_exit && + reg->ref_obj_id == state->refs[i].id) + continue; verbose(env, "Unreleased reference id=%d alloc_insn=%d\n", state->refs[i].id, state->refs[i].insn_idx); refs_lingering = true; @@ -16405,13 +16413,14 @@ static int check_return_code(struct bpf_verifier_env *env, int regno, const char const char *exit_ctx = "At program exit"; struct tnum enforce_attach_type_range = tnum_unknown; const struct bpf_prog *prog = env->prog; - struct bpf_reg_state *reg; + struct bpf_reg_state *reg = reg_state(env, regno); struct bpf_retval_range range = retval_range(0, 1); enum bpf_prog_type prog_type = resolve_prog_type(env->prog); int err; struct bpf_func_state *frame = env->cur_state->frame[0]; const bool is_subprog = frame->subprogno; bool return_32bit = false; + const struct btf_type *reg_type, *ret_type = NULL; /* LSM and struct_ops func-ptr's return type could be "void" */ if (!is_subprog || frame->in_exception_callback_fn) { @@ -16420,10 +16429,26 @@ static int check_return_code(struct bpf_verifier_env *env, int regno, const char if (prog->expected_attach_type == BPF_LSM_CGROUP) /* See below, can be 0 or 0-1 depending on hook. */ break; - fallthrough; + if (!prog->aux->attach_func_proto->type) + return 0; + break; case BPF_PROG_TYPE_STRUCT_OPS: if (!prog->aux->attach_func_proto->type) return 0; + + if (frame->in_exception_callback_fn) + break; + + /* Allow a struct_ops program to return a referenced kptr if it + * matches the operator's return type and is in its unmodified + * form. A scalar zero (i.e., a null pointer) is also allowed. + */ + reg_type = reg->btf ? btf_type_by_id(reg->btf, reg->btf_id) : NULL; + ret_type = btf_type_resolve_ptr(prog->aux->attach_btf, + prog->aux->attach_func_proto->type, + NULL); + if (ret_type && ret_type == reg_type && reg->ref_obj_id) + return __check_ptr_off_reg(env, reg, regno, false); break; default: break; @@ -16445,8 +16470,6 @@ static int check_return_code(struct bpf_verifier_env *env, int regno, const char return -EACCES; } - reg = cur_regs(env) + regno; - if (frame->in_async_callback_fn) { /* enforce return zero from async callbacks like timer */ exit_ctx = "At async callback return"; @@ -16545,6 +16568,11 @@ static int check_return_code(struct bpf_verifier_env *env, int regno, const char case BPF_PROG_TYPE_NETFILTER: range = retval_range(NF_DROP, NF_ACCEPT); break; + case BPF_PROG_TYPE_STRUCT_OPS: + if (!ret_type) + return 0; + range = retval_range(0, 0); + break; case BPF_PROG_TYPE_EXT: /* freplace program can return anything as its return value * depends on the to-be-replaced kernel func or bpf program. From patchwork Fri Feb 14 16:45:20 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Amery Hung X-Patchwork-Id: 13975260 X-Patchwork-Delegate: bpf@iogearbox.net Received: from mail-pl1-f170.google.com (mail-pl1-f170.google.com [209.85.214.170]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id F3045267AEF for ; Fri, 14 Feb 2025 16:45:34 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.214.170 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1739551536; cv=none; b=L4CjQyQs6ogyjLk5AYAhW1B6qsBm7GYFbgf4ZYdjYmZ81Ej/bVdo1YsbmZnpx0T7lQj+VDD7yZoA5uD+Wxb9UEEs0FR0NxQSN4XwpVZHYd1PPKL34FCaBoqrt8oW/bd7hDOOIcj67VfJ2LsltO9iL1XniHUcFyhlQ0y1LYJK6rw= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1739551536; c=relaxed/simple; bh=0CPds7L6bVfgfkSwoGmkmAs/hqh/qG7iMp0sGgwerYY=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=D7GKhvjEuALvp/6hFrWSXgh5/cx8Vs9s3WxR5XL34t+LIPhqhzEff3JBp2MJQQKiIym8gkfjq915to5v0biAPc74PHKrmKmraSIjIBWpCb5bLHBaEKKXZeyuNVGGTuSEKOWYh7dgMY021Z3P2qS3ywsqJe7efmy3rRGJa/pQLDU= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=m3lmmVWr; arc=none smtp.client-ip=209.85.214.170 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="m3lmmVWr" Received: by mail-pl1-f170.google.com with SMTP id d9443c01a7336-220e6028214so32950525ad.0 for ; Fri, 14 Feb 2025 08:45:34 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1739551534; x=1740156334; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=aEMmk+QJ7T9vF+Ze9uAr46CKRIACQEkQZAvqN5YNqmw=; b=m3lmmVWrYeMLiUVWcJOGiccRqe4uyWnJKrjoG5qqFcFAlSLlzYqEWApUtPWB0ad5sB sQkFOP6IZzxm0BIazVZFrSKjxc2+Eib81t/GVDvAo6FvMyKQWtaSVxgeWnsTcKltWQs0 z3/flUuCqDt4oeiCYx/0Bmw03JqP4xjgFJ6TXV7582YGy5Amom/kMmAR/GupkABQ+BYn RpLxoaYzza9lQHaPJUhMbBrsuhmh7TCHmozLBF+nCkyYR/tAIPaCiTFwyQ1HTeJRGag/ ih5gikQunWks0IZYbyDIJTya76v9MV0j9JBP8umK8gI1vKAyhsEreMe8xVkaaeoeRMIE hq9Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1739551534; x=1740156334; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=aEMmk+QJ7T9vF+Ze9uAr46CKRIACQEkQZAvqN5YNqmw=; b=N6zVclOTnW9A7CV9tjMXRRAty05Z5hBq5A5Imuw6i0NELb2vrKqhZK6+9LrJzVeVy+ R1CpXN9sb7Xhakysxxfy/fJ3xjLRrksk+u27nthLN7u3smju41+w1ON1tGtRoWMtbTHR 2qgt4QLji7hHgCWxd37yDDiXGKTKmvE0SmbR/QhO+TKamE2qcnyooimJob8yXBlBjaTC uqwjsz4rJi5Zb/wBXKlFTQgS8UhOKdNHgiSA/Ztzvn2+pdihxyfHh86gKAbXPYzW+aIv weSE+MLPG9uIuSwpoB8v+wqQAEpN3uzYk0wYRIkM8f4R8CohnEo1QBCRPAbmxe/hApC/ ZNCw== X-Gm-Message-State: AOJu0YyrOYVWQ0yXf5C22RRaR/MXFGgYbpRyIvgZP/OCiQlAfuDqhkgK +Oku+Fn1aknjsMo+J+qiQktQcM+6g9vpRqG6G/2MMc6LXK1Vf2VFAte/Ew== X-Gm-Gg: ASbGncvAtOd0xa5Cyt6iR9709n8OuvfSZRdieq9PJPUY4GMH04ew4lFMteUO6RRnypF fNq0c0cASA7TKoYYPMYHxlHKaXMlaNzuEVvON0mG6MRlanpZEjIy85JBUhqqJcnHsvfzjL5UPoo FvSMhOXNOMpp4X4OoV1Is1rL49/bdJcqBPM6NrTLzA7UkdVVGBn8ivSPJzxLcrKsX2IsNg0WT5B 784phU0Reg29jqITQsd3RuBp2mUu4kyKyzd/0wpTmRf+sY+wPw9mHB129Hm6VG8Og0BDXGC4Ssw uLxg/sEoi8g6s7vB0sSWtTmoGJW3mBOrUvxcJPBJAKwqjkfZE9MmkJcr1RfHAiblJA== X-Google-Smtp-Source: AGHT+IFIbc6PUK6Fke3Dxcz+ecX4ln+1dUz1osI2wOwLSybQwzFFFTQdjaQFMdfr/Ix8GS7t7dKztA== X-Received: by 2002:a05:6a21:32a3:b0:1ed:a6d7:3c07 with SMTP id adf61e73a8af0-1ee8cb0e846mr222957637.4.1739551534160; Fri, 14 Feb 2025 08:45:34 -0800 (PST) Received: from localhost.localdomain (c-76-146-13-146.hsd1.wa.comcast.net. [76.146.13.146]) by smtp.gmail.com with ESMTPSA id 41be03b00d2f7-adbf21517eesm2223346a12.13.2025.02.14.08.45.33 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 14 Feb 2025 08:45:33 -0800 (PST) From: Amery Hung To: bpf@vger.kernel.org Cc: daniel@iogearbox.net, andrii@kernel.org, alexei.starovoitov@gmail.com, martin.lau@kernel.org, eddyz87@gmail.com, ameryhung@gmail.com, kernel-team@meta.com Subject: [PATCH bpf-next v1 5/5] selftests/bpf: Test returning referenced kptr from struct_ops programs Date: Fri, 14 Feb 2025 08:45:20 -0800 Message-ID: <20250214164520.1001211-6-ameryhung@gmail.com> X-Mailer: git-send-email 2.47.1 In-Reply-To: <20250214164520.1001211-1-ameryhung@gmail.com> References: <20250214164520.1001211-1-ameryhung@gmail.com> Precedence: bulk X-Mailing-List: bpf@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Patchwork-Delegate: bpf@iogearbox.net From: Amery Hung Test struct_ops programs returning referenced kptr. When the return type of a struct_ops operator is pointer to struct, the verifier should only allow programs that return a scalar NULL or a non-local kptr with the correct type in its unmodified form. Signed-off-by: Amery Hung Acked-by: Eduard Zingerman Acked-by: Martin KaFai Lau --- .../prog_tests/test_struct_ops_kptr_return.c | 16 +++++++++ .../bpf/progs/struct_ops_kptr_return.c | 30 ++++++++++++++++ ...uct_ops_kptr_return_fail__invalid_scalar.c | 26 ++++++++++++++ .../struct_ops_kptr_return_fail__local_kptr.c | 34 +++++++++++++++++++ ...uct_ops_kptr_return_fail__nonzero_offset.c | 25 ++++++++++++++ .../struct_ops_kptr_return_fail__wrong_type.c | 30 ++++++++++++++++ .../selftests/bpf/test_kmods/bpf_testmod.c | 8 +++++ .../selftests/bpf/test_kmods/bpf_testmod.h | 4 +++ 8 files changed, 173 insertions(+) create mode 100644 tools/testing/selftests/bpf/prog_tests/test_struct_ops_kptr_return.c create mode 100644 tools/testing/selftests/bpf/progs/struct_ops_kptr_return.c create mode 100644 tools/testing/selftests/bpf/progs/struct_ops_kptr_return_fail__invalid_scalar.c create mode 100644 tools/testing/selftests/bpf/progs/struct_ops_kptr_return_fail__local_kptr.c create mode 100644 tools/testing/selftests/bpf/progs/struct_ops_kptr_return_fail__nonzero_offset.c create mode 100644 tools/testing/selftests/bpf/progs/struct_ops_kptr_return_fail__wrong_type.c diff --git a/tools/testing/selftests/bpf/prog_tests/test_struct_ops_kptr_return.c b/tools/testing/selftests/bpf/prog_tests/test_struct_ops_kptr_return.c new file mode 100644 index 000000000000..467cc72a3588 --- /dev/null +++ b/tools/testing/selftests/bpf/prog_tests/test_struct_ops_kptr_return.c @@ -0,0 +1,16 @@ +#include + +#include "struct_ops_kptr_return.skel.h" +#include "struct_ops_kptr_return_fail__wrong_type.skel.h" +#include "struct_ops_kptr_return_fail__invalid_scalar.skel.h" +#include "struct_ops_kptr_return_fail__nonzero_offset.skel.h" +#include "struct_ops_kptr_return_fail__local_kptr.skel.h" + +void test_struct_ops_kptr_return(void) +{ + RUN_TESTS(struct_ops_kptr_return); + RUN_TESTS(struct_ops_kptr_return_fail__wrong_type); + RUN_TESTS(struct_ops_kptr_return_fail__invalid_scalar); + RUN_TESTS(struct_ops_kptr_return_fail__nonzero_offset); + RUN_TESTS(struct_ops_kptr_return_fail__local_kptr); +} diff --git a/tools/testing/selftests/bpf/progs/struct_ops_kptr_return.c b/tools/testing/selftests/bpf/progs/struct_ops_kptr_return.c new file mode 100644 index 000000000000..36386b3c23a1 --- /dev/null +++ b/tools/testing/selftests/bpf/progs/struct_ops_kptr_return.c @@ -0,0 +1,30 @@ +#include +#include +#include "../test_kmods/bpf_testmod.h" +#include "bpf_misc.h" + +char _license[] SEC("license") = "GPL"; + +void bpf_task_release(struct task_struct *p) __ksym; + +/* This test struct_ops BPF programs returning referenced kptr. The verifier should + * allow a referenced kptr or a NULL pointer to be returned. A referenced kptr to task + * here is acquried automatically as the task argument is tagged with "__ref". + */ +SEC("struct_ops/test_return_ref_kptr") +struct task_struct *BPF_PROG(kptr_return, int dummy, + struct task_struct *task, struct cgroup *cgrp) +{ + if (dummy % 2) { + bpf_task_release(task); + return NULL; + } + return task; +} + +SEC(".struct_ops.link") +struct bpf_testmod_ops testmod_kptr_return = { + .test_return_ref_kptr = (void *)kptr_return, +}; + + diff --git a/tools/testing/selftests/bpf/progs/struct_ops_kptr_return_fail__invalid_scalar.c b/tools/testing/selftests/bpf/progs/struct_ops_kptr_return_fail__invalid_scalar.c new file mode 100644 index 000000000000..caeea158ef69 --- /dev/null +++ b/tools/testing/selftests/bpf/progs/struct_ops_kptr_return_fail__invalid_scalar.c @@ -0,0 +1,26 @@ +#include +#include +#include "../test_kmods/bpf_testmod.h" +#include "bpf_misc.h" + +char _license[] SEC("license") = "GPL"; + +struct cgroup *bpf_cgroup_acquire(struct cgroup *p) __ksym; +void bpf_task_release(struct task_struct *p) __ksym; + +/* This test struct_ops BPF programs returning referenced kptr. The verifier should + * reject programs returning a non-zero scalar value. + */ +SEC("struct_ops/test_return_ref_kptr") +__failure __msg("At program exit the register R0 has smin=1 smax=1 should have been in [0, 0]") +struct task_struct *BPF_PROG(kptr_return_fail__invalid_scalar, int dummy, + struct task_struct *task, struct cgroup *cgrp) +{ + bpf_task_release(task); + return (struct task_struct *)1; +} + +SEC(".struct_ops.link") +struct bpf_testmod_ops testmod_kptr_return = { + .test_return_ref_kptr = (void *)kptr_return_fail__invalid_scalar, +}; diff --git a/tools/testing/selftests/bpf/progs/struct_ops_kptr_return_fail__local_kptr.c b/tools/testing/selftests/bpf/progs/struct_ops_kptr_return_fail__local_kptr.c new file mode 100644 index 000000000000..b8b4f05c3d7f --- /dev/null +++ b/tools/testing/selftests/bpf/progs/struct_ops_kptr_return_fail__local_kptr.c @@ -0,0 +1,34 @@ +#include +#include +#include "../test_kmods/bpf_testmod.h" +#include "bpf_experimental.h" +#include "bpf_misc.h" + +char _license[] SEC("license") = "GPL"; + +struct cgroup *bpf_cgroup_acquire(struct cgroup *p) __ksym; +void bpf_task_release(struct task_struct *p) __ksym; + +/* This test struct_ops BPF programs returning referenced kptr. The verifier should + * reject programs returning a local kptr. + */ +SEC("struct_ops/test_return_ref_kptr") +__failure __msg("At program exit the register R0 is not a known value (ptr_or_null_)") +struct task_struct *BPF_PROG(kptr_return_fail__local_kptr, int dummy, + struct task_struct *task, struct cgroup *cgrp) +{ + struct task_struct *t; + + bpf_task_release(task); + + t = bpf_obj_new(typeof(*task)); + if (!t) + return NULL; + + return t; +} + +SEC(".struct_ops.link") +struct bpf_testmod_ops testmod_kptr_return = { + .test_return_ref_kptr = (void *)kptr_return_fail__local_kptr, +}; diff --git a/tools/testing/selftests/bpf/progs/struct_ops_kptr_return_fail__nonzero_offset.c b/tools/testing/selftests/bpf/progs/struct_ops_kptr_return_fail__nonzero_offset.c new file mode 100644 index 000000000000..7ddeb28c2329 --- /dev/null +++ b/tools/testing/selftests/bpf/progs/struct_ops_kptr_return_fail__nonzero_offset.c @@ -0,0 +1,25 @@ +#include +#include +#include "../test_kmods/bpf_testmod.h" +#include "bpf_misc.h" + +char _license[] SEC("license") = "GPL"; + +struct cgroup *bpf_cgroup_acquire(struct cgroup *p) __ksym; +void bpf_task_release(struct task_struct *p) __ksym; + +/* This test struct_ops BPF programs returning referenced kptr. The verifier should + * reject programs returning a modified referenced kptr. + */ +SEC("struct_ops/test_return_ref_kptr") +__failure __msg("dereference of modified trusted_ptr_ ptr R0 off={{[0-9]+}} disallowed") +struct task_struct *BPF_PROG(kptr_return_fail__nonzero_offset, int dummy, + struct task_struct *task, struct cgroup *cgrp) +{ + return (struct task_struct *)&task->jobctl; +} + +SEC(".struct_ops.link") +struct bpf_testmod_ops testmod_kptr_return = { + .test_return_ref_kptr = (void *)kptr_return_fail__nonzero_offset, +}; diff --git a/tools/testing/selftests/bpf/progs/struct_ops_kptr_return_fail__wrong_type.c b/tools/testing/selftests/bpf/progs/struct_ops_kptr_return_fail__wrong_type.c new file mode 100644 index 000000000000..6a2dd5367802 --- /dev/null +++ b/tools/testing/selftests/bpf/progs/struct_ops_kptr_return_fail__wrong_type.c @@ -0,0 +1,30 @@ +#include +#include +#include "../test_kmods/bpf_testmod.h" +#include "bpf_misc.h" + +char _license[] SEC("license") = "GPL"; + +struct cgroup *bpf_cgroup_acquire(struct cgroup *p) __ksym; +void bpf_task_release(struct task_struct *p) __ksym; + +/* This test struct_ops BPF programs returning referenced kptr. The verifier should + * reject programs returning a referenced kptr of the wrong type. + */ +SEC("struct_ops/test_return_ref_kptr") +__failure __msg("At program exit the register R0 is not a known value (ptr_or_null_)") +struct task_struct *BPF_PROG(kptr_return_fail__wrong_type, int dummy, + struct task_struct *task, struct cgroup *cgrp) +{ + struct task_struct *ret; + + ret = (struct task_struct *)bpf_cgroup_acquire(cgrp); + bpf_task_release(task); + + return ret; +} + +SEC(".struct_ops.link") +struct bpf_testmod_ops testmod_kptr_return = { + .test_return_ref_kptr = (void *)kptr_return_fail__wrong_type, +}; diff --git a/tools/testing/selftests/bpf/test_kmods/bpf_testmod.c b/tools/testing/selftests/bpf/test_kmods/bpf_testmod.c index 802cbd871035..89dc502de9d4 100644 --- a/tools/testing/selftests/bpf/test_kmods/bpf_testmod.c +++ b/tools/testing/selftests/bpf/test_kmods/bpf_testmod.c @@ -1182,11 +1182,19 @@ static int bpf_testmod_ops__test_refcounted(int dummy, return 0; } +static struct task_struct * +bpf_testmod_ops__test_return_ref_kptr(int dummy, struct task_struct *task__ref, + struct cgroup *cgrp) +{ + return NULL; +} + static struct bpf_testmod_ops __bpf_testmod_ops = { .test_1 = bpf_testmod_test_1, .test_2 = bpf_testmod_test_2, .test_maybe_null = bpf_testmod_ops__test_maybe_null, .test_refcounted = bpf_testmod_ops__test_refcounted, + .test_return_ref_kptr = bpf_testmod_ops__test_return_ref_kptr, }; struct bpf_struct_ops bpf_bpf_testmod_ops = { diff --git a/tools/testing/selftests/bpf/test_kmods/bpf_testmod.h b/tools/testing/selftests/bpf/test_kmods/bpf_testmod.h index c57b2f9dab10..c9fab51f16e2 100644 --- a/tools/testing/selftests/bpf/test_kmods/bpf_testmod.h +++ b/tools/testing/selftests/bpf/test_kmods/bpf_testmod.h @@ -6,6 +6,7 @@ #include struct task_struct; +struct cgroup; struct bpf_testmod_test_read_ctx { char *buf; @@ -38,6 +39,9 @@ struct bpf_testmod_ops { int (*unsupported_ops)(void); /* Used to test ref_acquired arguments. */ int (*test_refcounted)(int dummy, struct task_struct *task); + /* Used to test returning referenced kptr. */ + struct task_struct *(*test_return_ref_kptr)(int dummy, struct task_struct *task, + struct cgroup *cgrp); /* The following fields are used to test shadow copies. */ char onebyte;